Duplicate requests: retries, double-clicks, and idempotent APIs

After reading, you should be able to:

• Distinguish idempotent methods, idempotency keys, and deduplication stores.
• Design POST handlers that return the same result on replay.
• Store keys in DB or Redis with TTL and unique constraints.
• Align retries, 429 behavior, and consumers with idempotent semantics.

Why — networks guarantee at-least-once

TCP and HTTP do not tell the client whether the server finished work after a timeout. Load balancers, mobile SDKs, and service meshes retry failed requests. POST creates a new resource each time by default—retries create duplicates unless you add an application-level idempotency layer.

Sources of duplicates

• User double-submit (UI no debounce).
• Client retry on timeout or 502/503.
• Gateway retry to another pod (first pod actually succeeded).
• Kafka/SQS redelivery after consumer crash mid-process.
• Outbox/event published twice without dedupe.

HTTP methods (baseline)

Method	Typically idempotent?
GET, HEAD	Yes (no server state change)
PUT, DELETE	Yes for same URL/body semantics
POST	No — needs Idempotency-Key or natural key
PATCH	Depends on implementation

What — detect and scope the problem

1. Business symptoms — duplicate order ids, double charges, two emails, support “charged twice” with same timestamp window.
2. Logs / traces — two POST with same user, same amount, different request_id; or same Idempotency-Key missing.
3. DB evidence — unique constraint violations ignored; duplicate rows with no unique index on business key.
4. Which operations need idempotency — payments, inventory hold, signup, anything irreversible or billed.
5. Retry policy audit — client retries POST? only GET? exponential backoff on 429?

How — implement idempotent operations

1. Idempotency-Key header (standard pattern)

Client sends a UUID (per logical action, not per HTTP attempt):

POST /payments
Idempotency-Key: 7b9e2f1a-4c3d-4e5f-9a8b-1c2d3e4f5a6b
Content-Type: application/json

Server stores (key, request_hash) → response for 24–72 hours.

2. Processing flow

1. Begin transaction (or atomic Redis + DB).
2. Insert idempotency record with unique constraint on idempotency_key.
3. If insert conflicts → load stored response, return same status/body (200/201).
4. If insert succeeds → perform side effect once, save response, commit.

3. Schema sketch

CREATE TABLE idempotency_keys (
  key           VARCHAR(64) PRIMARY KEY,
  user_id       BIGINT NOT NULL,
  request_hash  VARCHAR(64),
  response_code INT,
  response_body JSONB,
  created_at    TIMESTAMPTZ DEFAULT now()
);
-- Optional: UNIQUE (user_id, key) if keys are per-user

4. Redis fast path (with DB authority for money)

SET idem:7b9e2f1a NX EX 86400
# NX fails → return cached response from DB/Redis

For payments, prefer DB unique constraint as source of truth; Redis as cache. On Redis down, fail closed for payment — Redis guide.

5. Natural keys (no header)

• UNIQUE (user_id, external_reference) from partner.
• “Create account” idempotent on email if you return existing user.

6. Message consumers

• Consumer stores processed (topic, partition, offset) or business event_id.
• Side effect in transaction with offset commit (or outbox).
• Handler must tolerate redelivery — same as HTTP retry.

7. Response rules

Case	Response
First success	201 + resource
Replay same key + same body	Same as first (200/201 + same body)
Same key, different body	409 Conflict
Still processing	409 or 202 + Retry-After (optional)

8. Client guidance

• Generate one idempotency key per user action; reuse on retry.
• Do not retry 4xx except 429 with backoff.
• Safe to retry GET/PUT; POST only with key.

Verify

1. Integration test: two parallel POSTs same key → one charge, both get identical response.
2. Chaos: kill pod after DB commit before response → client retry still idempotent.
3. Metrics: idempotency replay rate, 409 conflict rate.

Interview one-liner

Related scenarios

• Inconsistent logs
• Shared data races
• Redis unavailable
• Circuit breaker