Making Retries Safe: A Practical Guide to Idempotency Keys

TL;DR

TL;DR: Networks drop responses, clients time out, and users double-click "Pay Now." When a client retries a request that already reached your server, you need a way to make that retry harmless. Idempotency keys — a client-generated identifier attached to a request — let a server recognize a retry and return the original result instead of repeating the side effect. This is a protocol-level safety mechanism, not a guarantee of "exactly-once" processing across an entire system, and it's distinct from business-level deduplication (e.g., preventing two different orders for the same cart). Done well, it turns "did that actually go through?" from a support ticket into a solved problem. The article covers why retries are dangerous, what an idempotency key is, a concrete API flow, failure modes, implementation guidance, and the broader reliability strategy.

Why retries are dangerous by default

HTTP clients retry for good reasons: a TCP connection drops mid-response, a load balancer times out, a mobile client loses signal right after sending a request. The client doesn't know whether the server processed the request before the connection failed — only that it didn't get a confirmed response.

For a GET request, retrying is safe because reading data has no side effects. For a POST that creates a charge, submits an order, or sends a notification, a naive retry can duplicate the side effect. That's the gap idempotency keys close.

What an idempotency key actually is

An idempotency key is a unique value the client generates and sends with a request, typically as a header. The server uses that key to recognize "I've seen this exact request before" and to short-circuit reprocessing.

Stripe's implementation is a widely referenced example: the client sends an Idempotency-Key header on POST requests, Stripe stores the key alongside the result of the first successful attempt, and any subsequent request with the same key returns that stored result rather than executing the operation again. Stripe also compares the request parameters against the original call — sending the same key with different parameters is treated as an error rather than silently reused — and keys are not retained forever (Stripe: Idempotent Requests [1]).

This pattern has since been proposed as a general HTTP mechanism, not just a payments-API convention. The IETF's Idempotency-Key HTTP header draft defines the header for making otherwise non-idempotent methods safe to retry, and discusses how keys should be scoped, how long servers should remember them, and the uniqueness properties clients need to provide (IETF: The Idempotency-Key HTTP Header Field, draft-ietf-httpapi-idempotency-key-header [2]).

A concrete API flow

Here's the shape of a typical idempotent POST flow for, say, creating a payment:

1. Client generates a key. Usually a UUID v4, generated once per logical operation — not per HTTP attempt. If the user clicks "Pay" once, one key is generated and reused for every retry of that same click.

The request carries its idempotency key in the HTTP header; the server stores the first outcome and uses the same key to recognize a retry.

1. Server checks for the key. Before executing the operation, the server looks up whether it has already stored a result for that key.

• First time seen: the server processes the request normally, persists the key, the request parameters (or a hash of them), and the outcome (success or error) in durable storage — typically the same transaction that commits the business change, so the key and the side effect succeed or fail together.
• Key already exists, same parameters: the server returns the stored response, without re-running the charge logic.
• Key already exists, different parameters: the server returns an error (Stripe returns a 400-class error here), because the client is misusing the key rather than genuinely retrying.

1. Client receives a response and treats it as final for that logical operation, whether it arrived on the first attempt or the fifth retry.
2. Key expiry. Servers don't keep idempotency records forever. Stripe documents a retention window after which a key can be reused for a new request. Implementers need to pick a window long enough to cover realistic retry storms (minutes to a day, not seconds) but short enough to bound storage growth.

The failure mode people miss

The most common mistake isn't forgetting to check the key — it's a race condition between two concurrent requests carrying the same key. If a client fires a retry before the first attempt has finished (e.g., a client-side timeout that's shorter than the server's actual processing time), two requests with the same key can arrive at the server at nearly the same moment. If the server's "check key, then process, then store result" logic isn't atomic, both requests can pass the "no existing key" check and both execute the side effect — defeating the entire point.

The fix is to make key registration atomic with respect to processing: insert the key into storage with a unique constraint (or an equivalent lock) before starting the side-effecting work, so a concurrent second request fails fast on the uniqueness constraint or blocks until the first request's result is available. This needs to happen at the database or storage layer, not in application logic that merely reads-then-writes.

A related, quieter failure mode: storing the idempotency key in a different data store or transaction than the business record it protects. If the charge commits but the key-write fails (or vice versa), a retry can either duplicate the charge or return a stale "not found" and retry into a duplicate. Keeping the key and the state change in the same transactional boundary avoids this.

What idempotency keys are not

It's worth being precise about scope, because the term gets stretched:

• Not exactly-once delivery. Idempotency keys make retries safe to repeat, but they don't guarantee a message or request is delivered exactly one time end-to-end across distributed systems. A client can still fail to receive the final response even though the server processed the request correctly.
• Not business-level deduplication. Preventing a customer from accidentally placing two separate orders (with two different, legitimately generated keys) is a different problem, usually solved with application logic — like checking for a recent identical order — rather than the idempotency-key mechanism itself.
• Not authentication or authorization. The key identifies a request, not a user; it should be paired with normal auth on every attempt.

Practical implementation guidance

• Scope keys per client-intended operation, not per HTTP call. Generate the key once when the user initiates the action, and reuse it for every retry of that action, including client-side automatic retries.
• Persist keys atomically with the operation they protect, using a unique constraint at the storage layer to close the race condition described above.
• Store enough of the original request to detect mismatched reuse — either the full payload or a hash of it — and return an explicit error when a key is reused with different parameters, rather than silently returning the earlier result.
• Set and document a retention window for idempotency records, balancing realistic retry timeframes against storage cost, and communicate that window to API consumers.
• Apply keys only to non-idempotent methods (POST, sometimes PATCH) — GET, PUT, and DELETE are typically already idempotent by HTTP semantics and don't need this mechanism, though some APIs still support keys on them for consistency.
• Return the same status code and body on a replayed request as on the original successful response, so clients can't distinguish "first success" from "safe replay" and don't need special-case logic.

Where this fits into a broader reliability strategy

Idempotency keys solve one specific problem well: making client-initiated retries of write operations safe. They pair naturally with other reliability patterns — exponential backoff on the client, request timeouts tuned to realistic server processing time, and monitoring for elevated retry rates, which often signal an upstream problem before it becomes a customer-facing incident. None of these individually guarantees end-to-end reliability, but together they remove one of the more common sources of duplicate charges, duplicate emails, and duplicate orders in production systems.

If you're evaluating how your API layer handles retries, duplicate writes, or client timeout behavior, the Netics team can review your current implementation and flag where idempotency handling is missing or incomplete. Visit the [Netics homepage](PLACEHOLDER_NETICS_HOMEPAGE_URL) to learn more, or [book a technical consultation](PLACEHOLDER_NETICS_BOOKING_URL) to walk through your API's reliability posture.

For a practical architecture review, visit the Netics homepage or book a free 30-minute audit.

Sources

1. Stripe: Idempotent Requests
2. IETF: The Idempotency-Key HTTP Header Field, draft-ietf-httpapi-idempotency-key-header