Error Taxonomy and Failure Recovery
An error taxonomy groups failures so clients and operators can take the right next action. Separate invalid input, missing identity or permission, temporary dependency failure, business conflict, and internal defects. The most useful classification explains what can safely happen next.
HTTP status, stable problem identity, operation semantics, and outcome certainty work together. “Try again” is incomplete advice when the first attempt may already have charged a card or created a record.
TL;DR
- Classify failures by client correction, access decision, transient condition, or internal defect.
- Keep HTTP status and machine-readable problem fields consistent.
- Retry only when repetition is safe, the failure is eligible, and a budget remains.
- Preserve unknown outcomes and use lookup or reconciliation where required.
- Expose useful interface details while protecting internal diagnostics.
Quick Example
This complete Python model uses fictional application rules. It builds problem-shaped responses and separately decides retry eligibility; it is not an HTTP server or a universal status-code mapping.
This conservative model sends unknown outcomes to reconciliation. A real idempotency protocol may safely support retry after an unknown outcome, but its storage, identity, expiry, and duplicate-response rules must be implemented and tested.
Core Concepts
Interface contract
RFC 9457 defines problem details for HTTP APIs, including a problem-type identifier and human-readable fields. Use stable type identifiers for client logic, and document relevant extension fields. Keep the response status aligned with the actual HTTP status. Problem details
Failure cause and recovery action
A malformed field needs correction; an authorization denial needs an allowed permission path; a temporary outage may permit delayed repetition. A domain conflict may require refreshing state or changing the requested operation. These categories should be understandable without revealing database implementation details.
Outcome certainty
A timeout tells the caller it did not receive a response in time. It does not tell the caller whether the server committed the operation. Make that uncertainty explicit in the client state machine and support a stable operation identifier or lookup path where the product needs it.
Define a recovery matrix
Start with concrete operations and assign both the caller action and the operator evidence. The matrix below is editorial guidance, not an automatic mapping for every API.
Place retry policy at a deliberate layer. If browser, gateway, SDK, and worker each retry three times, one user action can produce many attempts. Use an end-to-end deadline, bounded attempts, backoff, and jitter where appropriate. Respect supported server retry guidance, but do not let it overrule whether the operation is safe to repeat.
Choose which errors deserve detailed problem types. Ordinary HTTP semantics may already be sufficient for a generic denial. Avoid proliferating a unique type for every exception class; that exposes internal organization and makes clients depend on implementation churn.
Test recovery without hiding the first failure
For every supported recovery action, test the triggering failure and the resulting state. Include a failure before work starts, a failure after commit but before the response, and a repeated request. Assert that the business effect occurs the intended number of times.
Capture a correlation reference and the original category before a retry succeeds. A green final response should not erase overload or dependency instability from telemetry. Conversely, do not log entire request bodies or tokens just because diagnosis is difficult.
RFC 9457 cautions against exposing implementation internals through error responses. Keep stack traces, queries, credentials, and sensitive record contents in appropriately restricted diagnostics, and return only the interface details the caller needs. Security considerations
Treat a new or unknown problem type conservatively. Preserve the HTTP-level behavior and a safe user message rather than crashing the error handler. Test compatibility with older clients before changing identifiers, required fields, or recovery semantics.
Best Practices
Document actions with the type
Explain whether the caller should correct input, refresh state, authenticate, stop, retry, or reconcile. Include the assumptions that make repetition safe.
Keep diagnostic and public fields separate
Review error serialization as part of the API boundary. A useful correlation reference does not require exposing the underlying exception message.
Make retry budgets observable
Measure initial failures, attempts, exhausted budgets, and final outcomes. This shows whether recovery helps or amplifies a dependency problem.
Common Mistakes
Bad: Retrying every 500 response immediately.
Correct: Classify the failure and operation, then use bounded repetition only when its semantics support it.
Bad: Assuming a timeout means no write occurred.
Correct: Preserve the unknown outcome and use the documented idempotency or reconciliation path.
Bad: Using translated message text as the client error identifier.
Correct: Use a stable machine-readable type and keep display text independent.
FAQ
Is every 4xx error permanent?
No. Status families alone do not define recovery. Follow the specific interface contract and operation semantics.
Should internal exceptions become problem types?
Expose meaningful interface conditions, not a mirror of the implementation’s exception hierarchy.
What is the key retry test?
Demonstrate the intended effect count when the response is lost after a successful write, using the real operation identity mechanism.
Related Topics
- Error Handling Patterns — Design the broader failure contract.
- Backend Development — Apply recovery at service boundaries.
- Background Jobs & Workers — Handle repeated asynchronous attempts.