Topic lessonTopic: Designing event-driven processing
Idempotency: prevent duplicate effects on retry
An operation is idempotent when repeating the same request leaves the intended business result unchanged. Follow a fictional image-processing request that commits successfully but loses its response. Use the request ID and the stored outcome to decide what a retry should return.
Choose the scope of a request ID, distinguish a repeat from a conflict or an unknown outcome, and identify where the operation and its durable record should commit.
SQS can redeliver messages. This lesson uses a fictional service that creates an intake record. You can work through the examples without connecting to AWS or a database.
A lost response can follow a successful write
A timeout can mean the receiver never started the work, or it can follow a successful commit. If the caller decides whether to redo the operation solely from the presence of a response, it can create the same business result twice.
- Caller → Image-processing intakeSends img-42
- Image-processing intake → Intake databaseCommits one intake record
- Intake database → Retrying callerIntake returns the stored result
- Caller
- Keeps request ID img-42
- Image-processing intake
- Compares the ID and parameters
- Intake database
- Commits the intake record and result ID
- Retrying caller
- Receives the same result ID J-81
The request ID identifies the operation the caller intended. Reuse the ID for another attempt at that operation. Use a fresh ID for a deliberate new operation, even when the image is the same. Grouping by image contents alone can discard legitimate repeat work.
Define the scope of the key and what a retry returns
{
"requestId": "img-42",
"image": "sample.png",
"width": 640
}| Incoming request | Compared with the saved record | Intake decision |
|---|---|---|
| A / img-42 / width 640 | Same caller, ID, and parameters | Return J-81. Keep one intake record. |
| A / img-42 / width 320 | Same caller and ID, different parameters | Respond with a conflict and keep the original record. |
| A / img-43 / width 640 | Same parameters, different request ID | Evaluate this as a new request. |
| B / img-42 / width 640 | A different caller | Check caller B’s permissions and key scope. Keep caller A’s result private. |
While the record is valid, the same ID and parameters can retrieve the original result. Set retention with the retry, redelivery, and manual replay windows in mind. Decide in advance how to handle late requests that arrive after this record expires.
Commit the intake operation and its record together
For an effect within a single intake database, consider a unique constraint on caller plus request ID and one transaction that commits both the intake row and the result record. Two concurrent attempts can both see an absent row, so a read followed by a separate write leaves a race.
A write to another service lies outside the intake database transaction. If an external call has an unknown outcome, the design needs a way to pass the same request key to that service or reconcile the outcome there. The intake record only establishes what the intake service committed.
For SQS batches processed by Lambda, a partial batch response can identify failed messages for retry and reduce reprocessing of successful messages. Consumers still need to handle repeated requests.
Choose what to do when a request repeats
Choose an option to see the result and explanation.
Check, then explain in your own words
- Identify who assigns the caller identity, request ID, and operation parameters.
- Try the same ID with the same parameters, the same ID with different parameters, and a new ID.
- Assume the first response was lost after commit and identify which stored outcome the retry should reuse.
Check your understanding: Besides the request ID, what evidence shows whether a redelivered message repeated a business effect?
The committed business outcome associated with that request ID.
The receipt log and the response only show part of the path. Check how many intake records or external writes actually committed.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
Scope and conditionsCaller-provided request identities, atomic recording, late arrivals and parameter mismatch.
Used in this lessonA lost response can follow a successful write Define the scope of the key and what a retry returns Commit the intake operation and its record together
- Using Lambda with Amazon SQSChecked:
Scope and conditionsAt-least-once processing, duplicate deliveries and partial batch failure responses.
Used in this lessonCommit the intake operation and its record together