Topic lessonTopic: SQS: redelivery, retention and recovery
SQS visibility timeout: redelivery, deletion and processing time
An SQS visibility timeout temporarily hides a received message from other consumers. It does not stop the worker or report that the work succeeded. Follow receipt, application work and deletion separately to understand how the same job can arrive again.
Read fixed delivery timelines, distinguish the visibility deadline from work completion and deletion, and explain what to check when a deadline expires or a job repeats.
A queue holds requests for work; a consumer receives and processes them. The first section explains these roles. All timings are fixed teaching examples. You do not enter an AWS account, message body or credentials, create a queue, or call an API.
Receipt, application work and deletion are separate events
Amazon SQS (Simple Queue Service) provides message queues. A producer sends a request, and a consumer receives it and does the work. Receiving through ReceiveMessage and removing a message through DeleteMessage are separate actions.
Consider a request to save an image. Distinguish saving the image, recording the result, and deleting the message from the queue. A record that the application finished its work does not prove that deletion also succeeded.
| Event | What it tells you | What it does not establish |
|---|---|---|
| Receipt | The consumer received the message; its visibility interval starts. | Whether the application work has finished. |
| Work completion | Check the intended result in the application and its storage. | Whether the message has been deleted from the queue. |
| Deletion | A deletion action concerns the corresponding receipt. | That a standard queue can never deliver a copy again. |
Compare three examples with a 30-second visibility deadline
In these examples, the first receipt is at 0 seconds and the visibility deadline is at 30 seconds. The calculation excludes network delays, retention expiry and moves to a DLQ. The second receipt times are chosen to explain the sequence; they do not predict actual delivery timing.
Fixed teaching examples / not predictions of delivery timing or processing results
A: delete after completing the work
- 0 sConsumer A receives
- 20 sApplication work finishes
- 25 sDeletion completes
This example deletes before the deadline, so it does not proceed to redelivery caused by the 30-second timeout.
This does not guarantee that a standard queue will never deliver another copy.
B: finish the work but do not delete
- 0 sConsumer A receives
- 20 sApplication work finishes
- 30 sAvailable for receipt again
- 35 sExample second receipt by B
The work may be done, but an undeleted message can become available again.
Handle the repeated request without unintentionally creating the result twice.
C: the deadline expires while work continues
- 0 sConsumer A receives
- 30 sDeadline expires while A works
- 35 sExample second receipt by B
- 45 sA finishes its application work
Expiry does not stop A. Another receipt can overlap the original application work.
A longer visibility timeout alone cannot make an external write or notification happen exactly once.
Both B and C become available again at 30 seconds. The difference is whether the original work has finished. B illustrates repeating work that already succeeded; C illustrates overlapping workers. In both cases, queue state alone cannot tell you the application result.
When changing a duration, check when the clock starts
Allow time for processing and deletion. Too short an interval can expose a message while work continues; too long an interval can delay another attempt at unfinished work. When processing time varies, the consumer can adjust visibility while it works.
| Time | Action in this example | Resulting deadline |
|---|---|---|
| 0 seconds | Receive with a 60-second visibility interval. | Initially, at 60 seconds. |
| 15 seconds | Change the duration for this receipt to 10 seconds. | 10 seconds from the change: at 25 seconds. |
| Next receipt | Do not assume the previous 10-second override persists. | Check the queue setting or override that applies to that receipt. |
ChangeMessageVisibility counts the new duration from the change. A per-message override is not a permanent setting for subsequent receipts. Extending visibility also does not restart the overall 12-hour bound measured from the first receipt.
DeleteMessage uses a ReceiptHandle returned on receipt, rather than the MessageId. A repeated receipt has a different handle, so use the latest corresponding handle. A deletion request using an old handle can succeed without deleting the message. Do not paste a real handle or run an API call for this lesson.
Follow external effects when considering redelivery
Idempotent handling means that repeating the same request does not produce unintended extra effects. A standard queue can deliver a stored copy even after deletion. Adjusting a visibility deadline and handling the application effects of duplicate requests are separate concerns.
| Observed record | What to check | Do not assume |
|---|---|---|
| The worker stopped just after saving the image. | How the saved result relates to the record marking the request complete. | No completion record means no image was saved. |
| The same work request arrives again. | Whether you can identify the same business request and find its existing result. | A different ReceiptHandle means a different job. |
| The visibility interval was increased. | Time required for work and deletion, and the delay before recovery after failure. | A longer interval removes the need to handle duplicate writes. |
With Lambda, identify who receives and retries the work
A Lambda event source mapping polls SQS and invokes the function synchronously with a batch of messages. When the batch succeeds, Lambda deletes its messages. This assigns receipt and deletion differently from an application that calls SQS directly.
By default, an error processing a batch makes its messages visible again after the visibility timeout, including items whose work already succeeded. Partial batch responses let the function report individual failures to avoid retrying successful items. Check both the mapping configuration and the function response.
| Situation | Unit to distinguish |
|---|---|
| SQS redelivery | Receive an undeleted message again. |
| Lambda asynchronous invocation | Retry an asynchronous function invocation; its conditions differ from SQS event source mapping. |
| Step Functions Retry | Retry the relevant workflow state. It does not share a counter with the SQS visibility setting. |
Investigate in order and plan before changing anything
- Identify direct application receipt versus a Lambda event source mapping.
- Correlate receipt, application results and deletion for the same request.
- Check whether the original work was still running when visibility expired.
- Inspect how repeated requests affect external writes, notifications and updates.
- Before changing timing or retry behavior, identify the owner, impact, verification method and recovery conditions.
A request that repeatedly cannot complete may need isolation in another queue so its cause can be investigated. That is the role of a DLQ, covered next. A visibility timeout alone does not establish how many application failures occurred or exactly when a message will move to a DLQ.
Change the time or actor and predict the next step
Choose an option to see the result and explanation.
Check, then explain in your own words
- Separate receipt, application work and deletion.
- At visibility expiry, check whether the original work is still running.
- Distinguish deadline adjustments from handling duplicate application effects.
- Identify the responsible actor and failure unit for direct receipt versus Lambda batches.
Check your understanding: If you extend visibility until the work finishes, can you rule out the same job running twice?
No. Check deadlines and deletion, and handle the application effects of repeated requests.
Visibility is a temporary delivery state. Standard queues do not guarantee exactly-once delivery. Also distinguish how far writes or notifications have progressed.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
- Amazon SQS visibility timeoutChecked:
Scope and conditionsTemporary invisibility, expiry and processing/deletion time; it is not an absolute duplicate-delivery guarantee. No live queue or deadline measurement.
Used in this lessonReceipt, application work and deletion are separate events When changing a duration, check when the clock starts
Scope and conditionsReceipt handles returned on receiving a message; distinguish receiving from processing and deletion. No API call or real payload.
Used in this lessonReceipt, application work and deletion are separate events When changing a duration, check when the clock starts
Scope and conditionsLatest receipt handle rather than MessageId; a stale handle may not delete despite success. Standard delivery can repeat after deletion.
Used in this lessonReceipt, application work and deletion are separate events When changing a duration, check when the clock starts
Scope and conditionsNew duration starts when changed; per-receive override is not permanent. Overall remaining-time bound is not reset. No API invocation.
Used in this lessonWhen changing a duration, check when the clock starts
Scope and conditionsStandard-queue redundant copies can cause repeat delivery; applications must handle duplicate effects. No claim to implement exactly-once business transactions.
Used in this lessonReceipt, application work and deletion are separate events Follow external effects when considering redelivery
- Using Lambda with Amazon SQSChecked:
Scope and conditionsPolling, synchronous batch invocation and deletion on success; default batch failure and partial batch response. No Lambda or event-source mapping created.
Used in this lessonWith Lambda, identify who receives and retries the work