Idempotency: Surviving the Retry
A response can be lost after the server did the work, so every client will eventually retry a request that already succeeded. Idempotency is the contract property that makes that retry safe — and money paths without it double-charge.
Frame the contract
API design starts with a consumer, a design question and a guarantee — never with a URL.
The failure that creates the requirement
A client calls POST /payments. The server charges the card, writes the row — and the response dies on the way back: the connection resets, a proxy times out, the phone drops off Wi-Fi. From the client's side this is indistinguishable from the request never arriving. It has exactly two options: give up on an operation that may have succeeded, or retry an operation that may have succeeded. Both are wrong unless the contract makes one of them safe.
Notice that no component misbehaved. TCP delivered what it could; the timeout fired as configured; the retry is the *correct* client response to ambiguity — the alternative is dropping a paying customer's order because of a 500ms network blip. The duplicate request is not a rare edge case caused by buggy clients; it is the guaranteed long-run behavior of any client that handles failure at all. Design for it or reconcile refunds by hand.
The window is wider than it looks: retries come from the user (double-click), the client library (automatic retry on timeout), the platform (a mobile OS re-sending on network change), and infrastructure (a gateway retrying an upstream). Several layers retry independently, so one logical request arriving three times is unremarkable.
POST /payments HTTP/1.1
Content-Type: application/json
{ "amount": 4999, "currency": "EUR", "source": "card_abc" }
# ... 10s pass, client times out, retries:
POST /payments HTTP/1.1
Content-Type: application/json
{ "amount": 4999, "currency": "EUR", "source": "card_abc" }# First request: server charged the card, then the
# connection reset before the 201 could be delivered.
# Second request, same contract, no idempotency:
HTTP/1.1 201 Created
{ "id": "pay_8f3…", "status": "processing" }
# The customer has now paid 49.99 twice.Safe, idempotent, neither — the vocabulary
HTTP already carves methods into three retry classes. Safe operations promise no state change at all — a retry is trivially fine, and caches and prefetchers exploit that promise (GET: The Promise of Safety). Idempotent operations may change state, but N identical requests leave the system where 1 would have: PUT replacing a document, DELETE removing a resource. Neither — canonically POST — is where each arrival may create a new effect, and where the duplicate problem lives.
Two precisions keep the vocabulary honest. Idempotent means *same end state*, not *same response*: the second DELETE may return 404 where the first returned 204, and that is still idempotent — the resource is equally gone (DELETE: What Does Gone Mean?). And the classification describes the *contract*, not the route table: a POST /orders/{id}/cancellations that transitions an order to cancelled is naturally idempotent because the state machine has nowhere further to go, while a PUT that does total += item.price inside has broken its method's promise no matter what the verb says.
| Method | Safe (no state change) | Idempotent (N = 1) | What a client may do on timeout |
|---|---|---|---|
| GET, HEAD | yes | yes | Retry freely; intermediaries may even retry for you |
| PUT | no | yes — full replacement | Retry freely, if the server keeps the promise |
| DELETE | no | yes — same end state | Retry; treat a 404 on retry as success |
| PATCH | no | not guaranteed | Depends on the patch semantics — the contract must say (PUT vs PATCH) |
| POST | no | no | Retry only with an idempotency mechanism, or accept duplicates |
Which operations actually need protection
Idempotency machinery is not free (Idempotency Keys: The Mechanism costs a store, expiry policy and concurrency handling), so spend it where duplicates hurt. The test is consequence, not verb: what does the *second* execution cost?
A duplicate read costs nothing. A duplicate full-replacement PUT costs nothing. A duplicate "create comment" costs an embarrassing double post. A duplicate "charge card", "send email", "ship order" or "transfer funds" costs money, trust, or a support ticket — and a duplicate call into a *third-party* API (the payment processor under you) costs a duplicate you cannot roll back locally. Protect in that order.
- Money movement — charges, refunds, payouts, transfers: always protect. This is the canonical Idempotency Keys: The Mechanism use case and the reason
brk-duplicate-paymentexists as a drill. - External side effects — emails, SMS, webhooks you emit, pushes: duplicates are user-visible and unrecallable.
- Resource creation with business identity — orders, subscriptions, signups: a duplicate creates a phantom entity someone must find and clean up.
- State transitions — cancel, approve, ship: often naturally idempotent if the state machine rejects re-transitions (Resources Have State Machines); verify rather than assume.
- Counters and appends — anything with
+=semantics is the opposite of idempotent; either redesign as absolute writes or protect the endpoint.
Idempotency is a clause, not an implementation detail
The client can only retry safely if the contract *says* it may. That means documenting, per operation: whether it is idempotent, by what mechanism (method semantics, idempotency key, natural state-machine convergence), and what the client sees on a duplicate (replayed response? 409? no-op 200?). An API that is accidentally idempotent today, via an implementation detail, will accidentally stop being idempotent in a refactor — and only the incident will announce it.
The clause composes with the rest of the reliability story: Retries and Timeouts as Contract Guidance tells clients *when* to retry; idempotency is *why* the retry is safe; An Error Taxonomy Clients Can Branch On and Retryability: Telling Clients What To Do Next tell them which failures qualify. Ship retry guidance without idempotency and you have documented instructions for duplicating side effects.
Key points
- A lost response makes success and failure indistinguishable to the client; retrying is its only rational move, so duplicates are guaranteed, not exceptional.
- Safe = no state change; idempotent = N requests, one effect. Idempotent means same end state, not same response body.
- Method semantics are promises: GET/PUT/DELETE retryable by contract, POST retryable only with an explicit mechanism.
- Protect by consequence: money, external side effects and entity creation first; reads and full replacements are free.
- Retry guidance without idempotency is documentation for double-charging — the two clauses ship together.
- Accidental idempotency is a refactor away from disappearing; only the documented clause survives.
Progressive depth
Overview
The network loses responses; clients retry; the question "did that happen once or twice?" must have an answer the contract gives, not one the customer discovers on a statement.
Practical
GET, PUT and DELETE are idempotent by HTTP semantics; POST is not — so give retryable POSTs an Idempotency-Key whose first result is stored and replayed to every retry (Idempotency Keys: The Mechanism). Natural unique keys (one membership per user per project) do the same job for creates that have one.
Advanced
Key scope (per client? per endpoint?), expiry (24h is common), parameter consistency (same key, different body → 422), and the concurrency case: two in-flight requests with one key must serialize, so the second waits or gets 409, never a second execution (Idempotency vs Deduplication).
Internals
The key store is a write-once record keyed by (client, key) holding status, response body and a fingerprint of the request; inserting it inside the same transaction as the side effect is what makes the replay exact. Split them and a crash between the two yields an executed-but-unrecorded request — the very duplicate you were preventing (Transactions and ACID, Write-Ahead Logging).
Idempotency Key Flow
Change the contract and observe which guarantee moves.
—
Follow the failure
How the contract fails or gets misused, hop by hop — and what it costs when it completes.
- 1Team → API: ships
POST /paymentswith no idempotency mechanism; staging never loses a response, so nothing looks wrong. - 2Client SDK → API: adds automatic retry-on-timeout, which is best practice on its side of the wire.
- 3Network → client: a gateway timeout fires after the charge succeeded; the SDK retries as designed.
- 4API → processor: charges the card again; both charges settle. Reconciliation finds them days later.
- 5Team → incident review: adds
if (recentDuplicate) skipheuristics per endpoint instead of a contract-level mechanism; the next endpoint repeats the cycle.
- Customers are double-charged or double-shipped; refunds, support load and chargeback fees land immediately.
- Clients that cannot retry safely stop retrying — transient blips become user-facing failures and abandoned checkouts.
- Operators lose trust in their own data: every count of orders, emails and charges carries an unknown duplicate rate.
Design, observe, evolve
A contract decision is incomplete until you know how you would notice it failing and how it changes later.
- • Classify every operation by duplicate consequence at design time and state its idempotency mechanism in the contract — method semantics, [[idempotency-keys]], or state-machine convergence.
- • Keep PUT and DELETE genuinely idempotent in implementation (absolute writes, tolerant re-deletes) so their method promise is real.
- • Design mutations as absolute state ("set quantity to 3") rather than relative deltas ("add 1") wherever the domain allows — absolute writes are idempotent for free.
- • Ship retry guidance ([[retries-and-timeouts]]) only alongside the idempotency clause it depends on.
- • Reconciliation against the downstream (processor settlements vs your payment rows) surfaces duplicates that the API accepted silently.
- • Alert on near-identical requests from one principal within a short window on unprotected endpoints — that is the retry storm arriving.
- • Track timeout rate on mutating endpoints: every timeout is a client left in the ambiguous state, which is your duplicate exposure.
- • Adding an idempotency-key mechanism is additive: old clients keep working, new clients gain safety; requiring the key can be phased per endpoint ([[idempotency-keys]]).
- • An operation can strengthen from "duplicates possible" to "idempotent" without breaking anyone; weakening the promise is a breaking change and needs a version or a migration ([[backward-compatibility]]).
- • Idempotency machinery adds a stateful check to the hot path of your most important writes — a store lookup before every charge.
- • Absolute-write designs push merge logic to clients: "set quantity to 3" requires the client to know the current quantity, which drags in [[optimistic-concurrency]].
- • Natural idempotency via state machines constrains the domain model: re-cancelling must be defined as a no-op, which is a product decision, not just a technical one.