INFRASTRUCTURE LEARNINGINFRASTRUCTURE LEARNING
インフラ大全インフラ大全IT INFRASTRUCTURE / LEARNING
Infrastructure learning guide
Find a lesson or lab/

Esc to close · Search terms stay on your device

LEARNING日本語

Topic lessonTopic: Designing event-driven processing

Idempotency: prevent duplicate effects on retry

An operation is idempotent when repeating the same request leaves the intended business result unchanged. Follow a fictional image-processing request that commits successfully but loses its response. Use the request ID and the stored outcome to decide what a retry should return.

What you will learn

Choose the scope of a request ID, distinguish a repeat from a conflict or an unknown outcome, and identify where the operation and its durable record should commit.

Prerequisite

SQS can redeliver messages. This lesson uses a fictional service that creates an intake record. You can work through the examples without connecting to AWS or a database.

A lost response can follow a successful write

A timeout can mean the receiver never started the work, or it can follow a successful commit. If the caller decides whether to redo the operation solely from the presence of a response, it can create the same business result twice.

Fictional intake with a lost first response
Fictional intake with a lost first response1: Caller — Sends img-42 — Image-processing intake. 2: Image-processing intake — Commits one intake record — Intake database. 3: Intake database — Intake returns the stored result — Retrying caller123CallerImage-processingintakeIntake databaseRetrying callerFictional intake with a lost first response1: Caller — Sends img-42 — Image-processing intake. 2: Image-processing intake — Commits one intake record — Intake database. 3: Intake database — Intake returns the stored result — Retrying caller123CallerImage-processingintakeIntake databaseRetrying caller
  1. Caller → Image-processing intakeSends img-42
  2. Image-processing intake → Intake databaseCommits one intake record
  3. Intake database → Retrying callerIntake returns the stored result
Caller
Keeps request ID img-42
Image-processing intake
Compares the ID and parameters
Intake database
Commits the intake record and result ID
Retrying caller
Receives the same result ID J-81

The request ID identifies the operation the caller intended. Reuse the ID for another attempt at that operation. Use a fresh ID for a deliberate new operation, even when the image is the same. Grouping by image contents alone can discard legitimate repeat work.

Define the scope of the key and what a retry returns

Fixed example request. A new resize operation can target a different width.
{
  "requestId": "img-42",
  "image": "sample.png",
  "width": 640
}
Caller A previously submitted img-42 at width 640 and received result ID J-81
Incoming requestCompared with the saved recordIntake decision
A / img-42 / width 640Same caller, ID, and parametersReturn J-81. Keep one intake record.
A / img-42 / width 320Same caller and ID, different parametersRespond with a conflict and keep the original record.
A / img-43 / width 640Same parameters, different request IDEvaluate this as a new request.
B / img-42 / width 640A different callerCheck caller B’s permissions and key scope. Keep caller A’s result private.

While the record is valid, the same ID and parameters can retrieve the original result. Set retention with the retry, redelivery, and manual replay windows in mind. Decide in advance how to handle late requests that arrive after this record expires.

Commit the intake operation and its record together

For an effect within a single intake database, consider a unique constraint on caller plus request ID and one transaction that commits both the intake row and the result record. Two concurrent attempts can both see an absent row, so a read followed by a separate write leaves a race.

A write to another service lies outside the intake database transaction. If an external call has an unknown outcome, the design needs a way to pass the same request key to that service or reconcile the outcome there. The intake record only establishes what the intake service committed.

For SQS batches processed by Lambda, a partial batch response can identify failed messages for retry and reduce reprocessing of successful messages. Consumers still need to handle repeated requests.

Choose what to do when a request repeats

Choose an option to see the result and explanation.

You retry img-42 with identical parameters after an unknown outcome. Which ID do you send?
Read the answer and reason

Answer: Reuse img-42 · Reusing the ID gives the intake service a key to look up the outcome of the original request.

The same caller sends img-42 with a different width. How should intake respond?
Read the answer and reason

Answer: Report a conflict · The key already has another set of parameters, so keep the original record and report the conflict.

Two copies of one request arrive concurrently. What protects against duplicate intake rows?
Read the answer and reason

Answer: Use a unique constraint on caller and ID and commit the result in the same transaction · A database constraint handles the race at the point of writing.

The connection drops after sending an external write. What should you reconcile first?
Read the answer and reason

Answer: Check the request ID and outcome at the external service · Look up the same request ID with the external service and determine what committed there.

Check, then explain in your own words

  1. Identify who assigns the caller identity, request ID, and operation parameters.
  2. Try the same ID with the same parameters, the same ID with different parameters, and a new ID.
  3. Assume the first response was lost after commit and identify which stored outcome the retry should reuse.
Check your understanding: Besides the request ID, what evidence shows whether a redelivered message repeated a business effect?

The committed business outcome associated with that request ID.

The receipt log and the response only show part of the path. Check how many intake records or external writes actually committed.

Sources and conditions checked

The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.