API Idempotency: Why It Matters in 2026
API Idempotency: Why It Matters and How to Implement It
A customer clicks "Pay Now." The request times out. Did the payment go through? The customer clicks again. Without idempotency, they get charged twice. With idempotency, the second request returns the same result as the first — no duplicate charge.
Idempotency means: making the same request multiple times produces the same result as making it once.
TL;DR
- POST and PATCH are not naturally idempotent — but they cover the most critical operations (payments, order creation)
- The idempotency key pattern: client generates a UUID, server stores the response keyed by that UUID, retries return the stored response
- Race conditions require a distributed lock (Redis SETNX) before processing — without it, concurrent retries can bypass the idempotency check
- Webhook consumers need idempotency too — the same event may be delivered multiple times
- Database-level upserts (
INSERT ... ON CONFLICT DO NOTHING) complement key-level idempotency for database operations
Which HTTP Methods Are Idempotent?
| Method | Idempotent? | Why |
|---|---|---|
| GET | ✅ Yes | Reading data doesn't change state |
| PUT | ✅ Yes | Replacing a resource with the same data = same result |
| DELETE | ✅ Yes | Deleting an already-deleted resource = same result |
| PATCH | ❌ No | Partial updates may produce different results (e.g., increment) |
| POST | ❌ No | Creating a resource twice = two resources |
The problem: POST and PATCH are not naturally idempotent. But POST is used for the most critical operations — payments, orders, account creation.
The Idempotency Key Pattern
Stripe popularized this pattern. The client generates a unique key per operation and includes it in the request header.
Request:
POST /api/payments
Idempotency-Key: idem_abc123def456
Content-Type: application/json
{
"amount": 4999,
"currency": "usd",
"customer": "cus_123"
}
Server behavior:
- First request: Process the payment, store the result keyed by
idem_abc123def456, return the result - Second request (same key): Look up the stored result, return it without reprocessing
- Different key: Process as a new payment
Implementation Steps
- Client generates a unique key — UUID v4 is the standard choice
- Server receives the request — check if this idempotency key exists in the store
- Key not found — process the request, store
{key, status_code, response_body, created_at} - Key found, processing — return 409 Conflict (request is still being processed)
- Key found, completed — return the stored response (same status code and body)
- Key found, different request body — return 422 (can't reuse a key for a different request)
Storage Requirements
| Field | Purpose |
|---|---|
idempotency_key | The unique key (primary key) |
request_hash | Hash of the request body (detect misuse) |
status_code | Stored HTTP response code |
response_body | Stored response JSON |
status | processing or completed |
created_at | When the key was first used |
expires_at | When to garbage collect (24-48 hours) |
Key Expiration
Idempotency keys should expire after 24-48 hours. After that, the same key can be reused. This prevents unbounded storage growth while covering retry windows.
When to Implement Idempotency
Always implement for:
- Payment processing (charges, refunds, transfers)
- Account creation
- Order placement
- Any operation where duplication has financial or data consequences
Optional for:
- Update operations (PUT is already idempotent)
- Read operations (GET is already idempotent)
- Delete operations (DELETE is already idempotent)
- Operations where duplicates are harmless (sending a notification twice is annoying but not catastrophic)
How Stripe Does It
Stripe accepts Idempotency-Key header on all POST requests:
POST /v1/charges
Idempotency-Key: idem_abc123
Authorization: Bearer sk_live_...
- Keys expire after 24 hours
- Replayed requests return the original response
- Using the same key with different parameters returns an error
- Keys are scoped to the API key (different accounts can use the same key)
How to Generate Keys
Client-Side
UUID v4: "550e8400-e29b-41d4-a716-446655440000"
Use UUID v4. It's universally supported, has negligible collision probability, and requires no coordination.
Alternative: Deterministic keys — hash the operation parameters to generate the key. hash(user_id + amount + currency + timestamp). This way, retrying the same operation automatically uses the same key without the client storing it.
Common Mistakes
| Mistake | Impact | Fix |
|---|---|---|
| No idempotency on payments | Duplicate charges | Idempotency key on all POST endpoints |
| Server-generated keys | Client can't retry safely | Client generates keys |
| No key expiration | Unbounded storage growth | Expire after 24-48 hours |
| Processing not locked | Race condition — two workers process same key | Lock on key before processing |
| Same key, different request allowed | Ambiguous behavior | Return error if request body differs |
| Key scoped globally | Cross-tenant key collision | Scope to API key or tenant |
Beyond Idempotency: At-Least-Once vs Exactly-Once
- At-most-once: Fire and forget. Request may not be processed. (No retries.)
- At-least-once: Retry until success. Request may be processed multiple times. (Retries without idempotency.)
- Exactly-once: Process exactly once. (Retries + idempotency.)
Most APIs should aim for at-least-once delivery with idempotent handlers, achieving effectively exactly-once processing.
Full Implementation in Node.js
The following is a production-ready idempotency middleware for Express or Hono using Redis. The critical detail is the distributed lock: before processing a request, we attempt to acquire a lock using Redis SETNX. If another worker is already processing the same key (a race condition from concurrent retries), the second worker returns 409 Conflict. This prevents the thundering herd problem where multiple retries all reach the "key not found" check simultaneously and all start processing.
import { Request, Response, NextFunction } from 'express';
import Redis from 'ioredis';
import crypto from 'crypto';
const redis = new Redis(process.env.REDIS_URL!);
const IDEMPOTENCY_TTL_SECONDS = 24 * 60 * 60; // 24 hours
const LOCK_TTL_MS = 30_000; // 30 second lock for processing
interface StoredResponse {
statusCode: number;
body: unknown;
requestHash: string;
completedAt: string;
}
function hashRequest(body: unknown): string {
return crypto
.createHash('sha256')
.update(JSON.stringify(body))
.digest('hex');
}
export async function idempotencyMiddleware(
req: Request,
res: Response,
next: NextFunction
): Promise<void> {
const idempotencyKey = req.headers['idempotency-key'] as string | undefined;
// Skip if no idempotency key provided (optional header)
if (!idempotencyKey) {
next();
return;
}
// Scope key to the authenticated user to prevent cross-user collisions
const userId = (req as any).user?.id ?? 'anonymous';
const scopedKey = `idem:${userId}:${idempotencyKey}`;
const lockKey = `idem:lock:${userId}:${idempotencyKey}`;
const requestHash = hashRequest(req.body);
// Check for existing response
const stored = await redis.get(scopedKey);
if (stored) {
const parsed: StoredResponse = JSON.parse(stored);
// Check for request body mismatch
if (parsed.requestHash !== requestHash) {
res.status(422).json({
error: {
code: 'idempotency_key_reuse',
message: 'The idempotency key has already been used with a different request body.',
},
});
return;
}
// Return stored response
res.status(parsed.statusCode).json(parsed.body);
return;
}
// Attempt to acquire a distributed lock before processing
// SETNX: set if not exists — atomic, prevents race conditions
const lockAcquired = await redis.set(
lockKey,
'1',
'PX', LOCK_TTL_MS, // milliseconds
'NX' // only set if not exists
);
if (!lockAcquired) {
// Another worker is processing this key
res.status(409).json({
error: {
code: 'idempotency_key_in_use',
message: 'A request with this idempotency key is currently being processed. Retry after a moment.',
},
});
return;
}
// Intercept the response to store it
const originalJson = res.json.bind(res);
const originalStatus = res.status.bind(res);
let capturedStatusCode = 200;
res.status = function (code: number) {
capturedStatusCode = code;
return originalStatus(code);
};
res.json = function (body: unknown) {
// Store completed response in Redis
const toStore: StoredResponse = {
statusCode: capturedStatusCode,
body,
requestHash,
completedAt: new Date().toISOString(),
};
// Store response and release lock (fire and forget)
redis.set(scopedKey, JSON.stringify(toStore), 'EX', IDEMPOTENCY_TTL_SECONDS)
.then(() => redis.del(lockKey))
.catch(console.error);
return originalJson(body);
};
next();
}
Apply the middleware to your critical endpoints:
// Only apply to state-changing endpoints
app.post('/api/payments', idempotencyMiddleware, createPaymentHandler);
app.post('/api/orders', idempotencyMiddleware, createOrderHandler);
app.post('/api/accounts', idempotencyMiddleware, createAccountHandler);
The key design decisions: scoping the Redis key by user ID prevents a user from using another user's idempotency response. The distributed lock using SETNX (Set if Not Exists) prevents race conditions where two concurrent retries both miss the stored response and both start processing. Releasing the lock after storing the response (not after the database operation) ensures that the lock is held for the minimum necessary time.
Idempotency and Distributed Systems
Idempotency becomes critical precisely because distributed systems are unreliable in specific ways. The Two Generals Problem — a thought experiment from distributed computing — illustrates the fundamental issue: two parties communicating over an unreliable channel can never reach a guaranteed consensus on whether a message was received. HTTP timeouts are a manifestation of this problem. When a client's request times out, the client does not know whether the server processed the request and the response was lost, or whether the request never arrived. Without idempotency, the only safe choice is to not retry — which means accepting data loss.
Network partitions, where two parts of a system lose connectivity, are the extreme case. But everyday API timeouts, server crashes mid-request, and load balancer disconnections create the same problem at smaller scale. Your payment processing server might successfully charge the customer's card and then crash before it can respond to the API. The customer's client sees a timeout. Without idempotency, there is no safe way to retry.
Retries plus idempotency achieve effectively-exactly-once processing. The word "effectively" is important — the operation may actually execute multiple times (at-least-once), but the visible effects (the charge, the order, the account) appear only once (effectively-exactly-once). This is the standard reliability model for distributed financial systems.
The outbox pattern is the complement to idempotency keys. When your API needs to both update a database and send a message (to a queue, webhook, or downstream service), doing both atomically is difficult. The outbox pattern: write the database update and an outbound message record to the same database transaction. A separate process reads pending outbox messages and delivers them, using idempotency keys to prevent double-delivery. This separates the commit of business state from the unreliable delivery of notifications, making both reliable independently.
Understanding these distributed systems principles matters because they shape how you design not just individual endpoints but entire transaction boundaries. For more on designing resilient API architectures, see our guide on API gateway patterns for microservices.
Database-Level Idempotency
Idempotency keys in Redis protect the API layer. But the underlying database operations should also be idempotent where possible — defense in depth against edge cases where the Redis check is bypassed or the distributed lock fails.
Upsert patterns are the primary tool for database-level idempotency. Instead of INSERT (which fails on duplicate) or a separate SELECT then INSERT (which has a race condition), use INSERT ... ON CONFLICT DO NOTHING (PostgreSQL) or equivalent:
-- Prevent duplicate order creation
-- orderId is the idempotency key at the business logic level
INSERT INTO orders (id, user_id, amount, status, created_at)
VALUES ($1, $2, $3, 'pending', NOW())
ON CONFLICT (id) DO NOTHING;
-- For upserts where you want the update to win:
INSERT INTO user_preferences (user_id, key, value, updated_at)
VALUES ($1, $2, $3, NOW())
ON CONFLICT (user_id, key) DO UPDATE SET
value = EXCLUDED.value,
updated_at = EXCLUDED.updated_at;
Deduplication tables are useful when you cannot add a unique constraint to the main table. Create a separate processed_events table with the idempotency key as the primary key:
CREATE TABLE processed_requests (
idempotency_key TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
request_hash TEXT NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW(),
expires_at TIMESTAMPTZ NOT NULL
);
-- In your handler:
INSERT INTO processed_requests (idempotency_key, user_id, request_hash, expires_at)
VALUES ($1, $2, $3, NOW() + INTERVAL '24 hours')
ON CONFLICT (idempotency_key) DO NOTHING;
-- Check if 0 rows were inserted (conflict = already processed)
Natural keys vs surrogate keys for deduplication: natural keys (business identifiers like (user_id, order_number)) are often better for detecting duplicates at the business level than surrogate keys (database-generated UUIDs). An order with the same user_id and order_number should probably be the same order, regardless of what idempotency key the client used. Model both: use idempotency keys for the API layer (short TTL, Redis), and use natural key unique constraints in the database for permanent deduplication.
Testing Idempotency
Idempotency is one of those properties that is easy to claim but important to verify. A comprehensive test suite covers the happy path, the retry path, and the race condition case.
import { describe, it, expect, beforeEach } from 'vitest';
import request from 'supertest';
import app from '../app';
import { redis } from '../lib/redis';
describe('Idempotency middleware', () => {
beforeEach(async () => {
await redis.flushdb(); // Start each test with clean Redis
});
it('first request processes normally', async () => {
const res = await request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-001')
.send({ productId: 'prod_123', quantity: 1 });
expect(res.status).toBe(201);
expect(res.body.id).toBeDefined();
});
it('second request with same key returns stored response', async () => {
const firstRes = await request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-002')
.send({ productId: 'prod_123', quantity: 1 });
const secondRes = await request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-002')
.send({ productId: 'prod_123', quantity: 1 });
// Same response, same order ID — not a new order
expect(secondRes.status).toBe(firstRes.status);
expect(secondRes.body.id).toBe(firstRes.body.id);
});
it('different body with same key returns 422', async () => {
await request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-003')
.send({ productId: 'prod_123', quantity: 1 });
const mismatchRes = await request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-003')
.send({ productId: 'prod_456', quantity: 2 }); // Different body
expect(mismatchRes.status).toBe(422);
expect(mismatchRes.body.error.code).toBe('idempotency_key_reuse');
});
it('concurrent requests with same key: one processes, one gets 409', async () => {
// Send two requests simultaneously
const [res1, res2] = await Promise.all([
request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-concurrent')
.send({ productId: 'prod_123', quantity: 1 }),
request(app)
.post('/api/orders')
.set('Idempotency-Key', 'test-key-concurrent')
.send({ productId: 'prod_123', quantity: 1 }),
]);
const statuses = [res1.status, res2.status].sort();
// One should succeed (201), one should conflict (409)
expect(statuses).toContain(201);
expect(statuses).toContain(409);
});
it('expired key is treated as a new request', async () => {
// Manually set an expired key in Redis
await redis.set('idem:user1:expired-key', JSON.stringify({
statusCode: 201,
body: { id: 'old-order-id' },
requestHash: 'different-hash',
completedAt: new Date(Date.now() - 48 * 3600 * 1000).toISOString(),
}), 'EX', 1); // 1 second TTL
await new Promise(resolve => setTimeout(resolve, 1100)); // Wait for expiry
const res = await request(app)
.post('/api/orders')
.set('Idempotency-Key', 'expired-key')
.send({ productId: 'prod_123', quantity: 1 });
// Should create a new order, not return the expired cached response
expect(res.status).toBe(201);
expect(res.body.id).not.toBe('old-order-id');
});
});
The concurrent request test is the hardest to write correctly. The goal is to verify that when two requests with the same key arrive simultaneously, exactly one is processed and the other gets 409. This test is inherently timing-sensitive — if your processing is too fast, both requests might complete before either checks the lock. In practice, slow down the handler with a small delay to make the race condition reproducible in tests.
Idempotency in Webhooks
Idempotency is typically discussed from the API server's perspective (preventing duplicate operations when clients retry). But the same problem exists on the webhook consumer side, and it is equally important.
Webhook delivery systems guarantee at-least-once delivery. When your API sends a webhook event to a consumer, the consumer's endpoint might be slow, might return a 5xx, or might not respond within the timeout window. The webhook sender retries. The consumer's handler may run twice for the same event.
For a webhook consumer to be truly safe, it must be idempotent. The standard approach is to track processed event IDs:
// Webhook handler for Stripe events
app.post('/webhooks/stripe', express.raw({ type: 'application/json' }), async (req, res) => {
const sig = req.headers['stripe-signature'] as string;
let event: Stripe.Event;
try {
event = stripe.webhooks.constructEvent(req.body, sig, process.env.STRIPE_WEBHOOK_SECRET!);
} catch (err) {
return res.status(400).send(`Webhook Error: ${err}`);
}
// Check if we've already processed this event
const alreadyProcessed = await redis.get(`webhook:processed:${event.id}`);
if (alreadyProcessed) {
// Return 200 to acknowledge — don't reprocess
return res.json({ received: true, duplicate: true });
}
// Process the event
try {
await handleStripeEvent(event);
// Mark as processed (TTL = 7 days, longer than Stripe's retry window)
await redis.set(`webhook:processed:${event.id}`, '1', 'EX', 7 * 24 * 3600);
res.json({ received: true });
} catch (err) {
// Don't mark as processed on error — allow retry
console.error('Webhook processing failed:', err);
res.status(500).json({ error: 'Processing failed' });
}
});
Processing order guarantees: Webhooks are not guaranteed to arrive in order. An order.updated event might arrive before order.created if there is a network anomaly. Design your webhook handlers to be order-independent where possible, or implement a state machine that handles out-of-order events gracefully (e.g., ignore an updated event for an order that doesn't exist yet, and re-process it when the created event arrives).
For a deeper look at webhook design from the API provider's perspective, see our guide on how to design a REST API developers love and the broader API reliability patterns discussed in the blog.
Conclusion
Idempotency is one of the most important reliability properties for any API that handles financial transactions or critical state changes. The implementation is straightforward — a Redis-backed middleware with a distributed lock — but the design decisions (key scoping, lock duration, expiry window, database-level backup) require deliberate thought.
The rule of thumb: any POST endpoint that creates or modifies a resource in a way that would be painful to undo or compensate should have idempotency. Payments and orders always. Account creation usually. Notification sends optionally (duplicate notifications are annoying, not catastrophic).
For building reliable, production-grade APIs, idempotency pairs naturally with API rate limiting best practices and comprehensive API error handling to create an integration that developers trust.