Batch and Bulk APIs: Jobs, Partial Failure, and Replay

Batch and Bulk APIs

A batch or bulk API moves several operations through one interface. That can reduce request overhead or support large jobs, but it also concentrates validation, resource use, authorization, and recovery into a larger unit. The core design question is what the caller can safely conclude when only part of the work finishes.

Start by defining the unit of acceptance, execution, and retry. A submitted job, an accepted item, and a completed side effect are different events. Make their identities and outcomes visible before optimizing throughput.

TL;DR

Bound request bytes, item counts, execution, and retained results.
Choose atomic or independent-item behavior deliberately.
Provide stable job and item identities with documented replay semantics.
Separate request acceptance from asynchronous completion.
Design cancellation, export consistency, authorization, and reconciliation from the start.

Quick Example

This complete in-memory Python illustration processes well-formed fictional item dictionaries independently. It records successful item identities, rejects an invalid quantity, recognizes unchanged-payload replay, and rejects reuse of an identity with a different quantity.

The exact assertions passed and the receipt map contained only the accepted item. This is not a general input parser or deployed API: it assumes a list of dictionaries with id and units fields. There is no authentication, durable transaction, concurrency protection, background worker, HTTP response, export, or external side effect. A process restart loses these receipts.

Core Concepts

Batch and bulk contracts

Batch often means grouping several discrete requests; bulk often means applying one operation to many records. Products use these words differently, so document your actual semantics rather than inferring them from the endpoint name.

Acceptance and completion

HTTP 202 means a request was accepted for processing, not that processing completed or will necessarily succeed. RFC 9110 recommends a representation describing current status and pointing to a status monitor. 202 Accepted

Error representation

RFC 9457 offers a standard problem-details format for HTTP errors. Per-item outcome design remains your application contract; do not imply the format defines an entire bulk-job lifecycle. Problem details

Define the failure unit before the transport shape

This matrix describes design choices, not a universal protocol. Choose atomic behavior only when the storage and side-effect boundaries can support the promise. A database transaction cannot automatically roll back an already delivered external message.

For independent items, return stable item identifiers and explicit states. Avoid a response that says success while hiding which records failed. Separate permanent validation or authorization failures from retryable operational conditions, and keep uncertain outcomes visible when reconciliation is required.

Do not rely solely on array positions to identify results across retries. A client may retry only failed items, change ordering, or resume a large input. A stable caller or server operation identity makes those cases easier to reconcile.

Define whether duplicate item IDs within the same request are rejected, combined, or treated as replay. The example uses its receipt map to illustrate repeated successful identities; a public API may prefer to reject duplicates at envelope validation. Make the chosen behavior predictable.

Make asynchronous jobs durable and bounded

Validate the envelope and permissions before consuming expensive resources. Limit bytes and item count separately because a small number of huge records can still exhaust memory. Review decompression, parsing, queued work, active concurrency, and retained-result limits as distinct budgets.

Persist the job identity and acceptance decision before presenting a durable acceptance claim. Workers need a recovery protocol for partial progress and repeated execution. A request timeout can leave the caller uncertain even though work was accepted, so support lookup or replay through the documented identity.

Use an explicit state machine, such as queued, running, succeeded, completed_with_errors, failed, cancellation_requested, and cancelled, with precise meanings. Those names are illustrative. Define which transitions are terminal and how a race between completion and cancellation is resolved.

Bind replay identities to an appropriate caller/tenant, operation, and payload meaning. Reusing an identity for changed input should produce a documented conflict rather than silently returning an unrelated result. Record retention must cover the intended replay period; after expiry, the API cannot promise the same deduplication behavior.

Keep receipts and the authoritative mutation in the same suitable transaction where possible. For external effects, use the downstream system’s supported identity and reconciliation approach. Avoid an unqualified exactly-once promise across unrelated services.

Design imports and exports for resumption

For imports, identify progress at a stable record boundary and retain enough outcome detail to retry only appropriate items. A byte offset into a compressed or transformed stream may not be a meaningful application checkpoint. Preserve the input revision or content identity needed to interpret a resume request.

For exports, decide whether all pages represent one snapshot or a changing view. A cursor alone does not establish snapshot consistency. Define stable ordering and tie behavior, then account for records inserted, updated, or removed while the export runs.

Scope job status, item results, cursors, and download URLs to authorized users and tenants. Permission to submit a job does not automatically authorize reading every resulting object forever. Recheck according to the actual access policy and protect retained export content.

Bound retention and make expiry understandable. Tell callers how long job metadata, replay receipts, input data, and downloadable results remain available. Keep operational logs useful without copying entire sensitive input batches into them.

Test partial worker failure, repeated delivery, changed-payload replay, lost responses, cancellation races, permissions changing during work, and expired cursors/results. Start with a bounded local model, then verify the actual database, worker, and HTTP boundaries before presenting production guarantees.

Best Practices

Publish the outcome contract

Document acceptance, per-item states, terminal job states, retry eligibility, and uncertainty. Give callers a concrete recovery path.

Bound every retained resource

Limit work admission, concurrency, input/result storage, and receipt lifetime. Observe backlog and age alongside throughput.

Test recovery at real boundaries

Exercise duplicate delivery and failures around authoritative writes and external effects. Use actual integration evidence for durable guarantees.

Common Mistakes

Bad: 202 means every item succeeded.

Correct: Treat it as acceptance and retrieve the documented completion and item outcomes.

Bad: Retry the whole batch whenever the response is lost.

Correct: Use scoped stable identities and reconcile accepted or uncertain work under the API’s replay contract.

Bad: Pagination makes an export a snapshot.

Correct: Specify the data-view consistency and cursor behavior independently.

FAQ

Should a batch be all-or-nothing?

Only when that matches the user task and can be implemented across the actual transaction boundary. Independent items often need explicit partial outcomes instead.

Can cancellation undo completed items?

Only if a separately defined compensating or reversal operation does so. Cancellation commonly stops remaining work and must report what already completed.

What did the local example prove?

Per-item fictional outcomes and unchanged-payload replay in one in-memory process. It did not prove durability, concurrency, authorization, or external exactly-once execution.