Idempotency Keys for Duplicate API Requests
Duplicate requests are not edge cases reserved for broken clients. They are normal distributed-systems behavior whenever retries, timeouts, network drops, or user double-submits exist.
That means non-idempotent write endpoints eventually face the same risk: duplicate orders, duplicate charges, repeated side effects, and inconsistent state after client retries.
Idempotency keys are one of the most important tools for preventing that class of bug.
What An Idempotency Key Actually Solves
An idempotency key lets the client say:
this request may be a retry of the same logical action
Instead of treating every inbound write as new work, the server can detect that the request represents an already-started or already-completed operation.
For write endpoints, this is the guarantee you want: one logical action produces one effect, and retries replay the original outcome instead of creating a second effect.
That does not mean the endpoint is always safe. It means repeated requests with the same key and the same intent are treated as one operation.
Why Duplicate Requests Happen In Healthy Systems
Duplicates are expected even when everyone behaves reasonably: the client times out after the server already committed, the response is sent but lost in transit, a mobile client retries during connectivity changes, a load balancer or proxy retries, a user double-clicks or resubmits, or a worker retries an upstream call after an ambiguous failure.
If retries exist, duplicate requests exist.
That is why idempotency should be treated as part of the correctness model, not a special feature for rare incidents.
The Main Design Rule
Retries improve availability. Idempotency preserves correctness.
If you add retries without idempotency, partial failures can turn into duplicate writes. If you add idempotency without thinking about retry windows, in-progress behavior, or payload mismatch, the design is still incomplete.
Use them together: client retries with bounded backoff, server-side idempotency on non-idempotent writes, and a replay window aligned to real retry behavior.
If the same action also touches shared rows under contention, idempotency still does not answer how concurrent writers should coordinate. That is where row-level conflict control matters, usually through Optimistic vs Pessimistic Locking in SQL and the broader patterns in How to Prevent Race Conditions in Backend Systems.
The Request Lifecycle You Actually Need
A durable flow for something like POST /payments looks like this:
- client sends an
Idempotency-Key - server validates request shape
- server atomically reserves the key
- if the key is new, process the request
- persist the authoritative response tied to that key
- if the key already completed, replay the stored response
- if the key is already in progress, return a clear retry/conflict response
The critical step is key reservation.
If reservation is not atomic, two duplicate requests can still execute the side effect twice.
A Race Timeline Worth Testing
Many idempotency implementations look fine until two requests arrive close together.
Request A sends Idempotency-Key: abc123 and reaches the server first. Before it finishes charging the payment, request B arrives with the same key because the client retried after a timeout. If both requests perform a read like "does this key already exist?" before either one reserves it durably, both may conclude that they are the first request and both may execute the charge.
That is why the correctness boundary is not the existence check. It is the reservation step. The system only becomes safe when there is a single atomic point where one request wins ownership of the key and the other does not.
A Practical Data Model
A common table shape looks like this:
CREATE TABLE api_idempotency (
id BIGSERIAL PRIMARY KEY,
key TEXT NOT NULL,
scope TEXT NOT NULL,
request_hash TEXT NOT NULL,
status TEXT NOT NULL,
response_code INT,
response_body JSONB,
error_type TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
expires_at TIMESTAMPTZ NOT NULL,
UNIQUE (scope, key)
);
Important design choices are that scope prevents cross-tenant or cross-user key collisions, request_hash lets you reject key reuse with a different payload, response_body lets you replay the original result deterministically, and expires_at lets you bound storage growth.
For many systems, (scope, key) is the true uniqueness boundary, not the key alone.
Request Hashing Is More Subtle Than It Looks
request_hash is often described in one sentence and then implemented too casually.
In practice, the hard part is deciding what counts as "the same request." If one client sends fields in a different order, if optional fields are omitted in one call and defaulted in another, or if harmless metadata headers vary between retries, the server should usually still treat the request as the same logical action.
That means the hash should be based on a canonical representation of the business-relevant payload, not on every byte the client happened to send. If the hash function is too strict, legitimate retries look like conflicts. If it is too loose, genuinely different requests can collide under the same key.
A Concrete Example
Suppose the client calls POST /payments.
Pseudo-code for the server path:
async function handlePayment(req, res) {
const key = req.headers['idempotency-key'];
const scope = req.auth.accountId;
const requestHash = hashCanonical(req.body);
const existing = await repo.findByScopeAndKey(scope, key);
if (existing && existing.requestHash !== requestHash) {
res.statusCode = 409;
res.end('Idempotency key reused with different payload');
return;
}
if (existing?.status === 'completed') {
res.statusCode = existing.responseCode;
res.end(JSON.stringify(existing.responseBody));
return;
}
const reserved = await repo.tryInsertInProgress({
scope,
key,
requestHash,
expiresAt: ttlFromNowHours(24),
});
if (!reserved) {
res.statusCode = 409;
res.setHeader('Retry-After', '1');
res.end('Request with this key is already in progress');
return;
}
try {
const payment = await payments.charge(req.body);
const response = { id: payment.id, status: 'succeeded' };
await repo.markCompleted({
scope,
key,
responseCode: 201,
responseBody: response,
});
res.statusCode = 201;
res.end(JSON.stringify(response));
} catch (error) {
await repo.markFailed({ scope, key, errorType: classify(error) });
throw error;
}
}
The important detail is not the exact framework. It is that tryInsertInProgress must be a single atomic reservation step.
In SQL, that usually means INSERT ... ON CONFLICT DO NOTHING. In Redis, it often means SETNX plus careful state handling.
The Three States You Usually Need
An idempotency record generally needs to model at least in_progress, completed, and failed or an equivalent retry-aware error state.
Why in_progress matters:
If two requests arrive close together, one may still be working while the second checks the key. Without an explicit in-progress state, both can still execute.
Why completed matters:
You want to replay the same result, not recompute business logic and hope it matches.
Why some failure state matters:
You need a policy for ambiguous or partial outcomes rather than silently treating every error the same way.
Ambiguous Failures Need An Explicit Policy
The hardest idempotency cases are not clean successes or clean failures. They are the moments when the downstream side effect may already have happened, but your service cannot prove it yet.
A payment provider timeout is the classic example. If the provider may already have accepted the charge, simply deleting the idempotency record and letting the next retry try again can create the very duplicate charge the key was meant to prevent.
This is where systems often need an intermediate policy such as "request accepted but outcome uncertain," followed by reconciliation or asynchronous confirmation before the key can safely move to a final state.
The exact response contract depends on the product, but the design principle is stable: when the side effect may already exist, prefer preserving uncertainty explicitly over pretending the request cleanly failed.
Response Replay Policy
When the same key returns, replay the original status and body if the original request completed, return a conflict or retry-after response if the request is still in progress, and if the key expired either treat it as new work or reject it based on business risk.
For payment-like systems, replaying the original authoritative result is usually better than recomputing. It gives clients predictable behavior and reduces reconciliation pain.
TTL Strategy
TTL should reflect real retry behavior and business risk.
Typical examples are lower-risk writes with a 1 to 6 hour window, financial writes with a 24 to 72 hour window, and async workflows aligned to maximum processing time plus the retry horizon.
If TTL is too short, duplicates slip through after expiration. If TTL is too long, storage grows and accidental key reuse becomes more likely.
Documenting the replay window in your API contract is worth doing.
Payload Validation Is Not Optional
One easy way to get idempotency wrong is to receive the same key with a different payload and still replay success.
That is dangerous.
The same key must represent the same logical request. That is why request hashing matters.
If the payload changed, return a conflict rather than replaying an unrelated response.
Where Implementations Usually Fail
The most common mistakes are checking for the key before reserving it atomically, not scoping keys by tenant or account, not storing the request hash, storing only "seen = true" without the original response, choosing a TTL shorter than real retry windows, and having no policy for in_progress requests.
The pattern sounds simple. The edge cases are where correctness lives.
A Realistic End-To-End Scenario
Client sends:
POST /payments
Idempotency-Key: 0f95f3cd-5f8f-41f6-80d5-7ab7de5da56a
Content-Type: application/json
{
"orderId": "ord_123",
"amount": 4999,
"currency": "USD",
"methodId": "pm_9x2"
}
Server charges successfully, but the response is lost before the client receives it.
Without idempotency, the client retries and the second request creates a second charge.
With correct idempotency, the retry hits the same key, the server returns the original 201 response, the payment ID stays the same, and customer trust and reconciliation effort are preserved.
That is the business value of the pattern.
How To Test It
Idempotency logic is easy to approve in code review and still break under real timing.
High-value tests include:
- same key, same payload, sequential retry
- same key, same payload, concurrent requests
- same key, different payload
- ambiguous failure followed by retry
- expired key behavior
This is an excellent place for integration tests, because the storage reservation and request boundary matter as much as the pure business logic. See How to Write API Integration Tests.
Where Idempotency Is Not Enough
Idempotency protects the write boundary. It does not automatically solve downstream async consistency.
If a successful request must later publish an event for other systems, you still need a reliable way to persist that publish intent. That is where Transactional Outbox Pattern in Microservices matters.
If duplicate processing continues in async handlers, queues, or webhooks, the same correctness model has to continue there too. See Background Jobs in Production and Webhook Idempotency and Retries in Production.
What To Watch In Production
Once idempotency is live, it should become an observable part of the API rather than invisible glue.
The most useful signals are replay rate, conflict rate from mismatched payloads, in-progress collisions, the age of unresolved keys, and how often requests land in ambiguous states that need later repair.
Those signals tell you whether upstream retry behavior changed, whether a client integration is broken, or whether your own timeout and retry model no longer matches reality.
A Practical Rollout Checklist
Before enabling idempotency in production:
- define the client contract for the key
- enforce scoped uniqueness
- atomically reserve keys
- store request hash
- store authoritative response for replay
- define
in_progressbehavior - choose TTL and cleanup strategy
- instrument metrics for replays, mismatches, and conflicts
- run duplicate-request concurrency tests
If replay and conflict metrics change sharply in production, something upstream changed. Make idempotency observable, not invisible.
Final Thoughts
Retries are necessary in distributed systems. Duplicate side effects are optional.
Idempotency keys are the boundary between the two.
When implemented with atomic reservation, payload validation, scoped uniqueness, and deterministic response replay, they turn a common production failure mode into behavior clients can safely rely on.