Topic lessonTopic: SQS: redelivery, retention and recovery
SQS dead-letter queues: receive counts, retention and redrive
A dead-letter queue (DLQ) separates messages that cannot be processed successfully so they can be investigated and considered for recovery. Arriving there does not fix a defect or complete the job. Distinguish isolation by receive count, retention, and redriving messages after their cause has been addressed.
Read the relationship between an SQS source queue and its DLQ, then choose an investigation or recovery step by checking receive counts, retention, ordering and the reprocessing destination.
Understand that SQS receipt, visibility timeout and deletion are separate; the visibility-timeout lesson covers these first. All requests and numbers here are fictional. You do not inspect, move or delete real messages or change queue settings.
Isolation is different from repair or success
An SQS DLQ receives messages that the normal source queue could not successfully process. Create a separate queue and configure the relationship. Match a standard queue with a standard DLQ, and a FIFO queue with a FIFO DLQ. FIFO stands for first-in, first-out.
Imagine an image-conversion worker that cannot read an older data format. Returning those requests repeatedly may reproduce the same failure while the defect remains. Separate them first, then establish the cause and the conditions that would make reprocessing appropriate.
- Source queue → Isolate in DLQConfigured receive-count condition
- Isolate in DLQ → Investigate cause and impactInvestigate isolated requests
- Investigate cause and impact → Decide on reprocessingAddress the cause and check conditions
- Source queue
- Process normal requests; check receipt and deletion.
- Isolate in DLQ
- Separate requests that did not complete.
- Investigate cause and impact
- Check fixes, retention, duplicates and order.
- Decide on reprocessing
- Choose the destination, volume and result checks.
maxReceiveCount does not count code exceptions
The source queue's RedrivePolicy identifies a DLQ and maxReceiveCount, a threshold concerning message receipts. Exceeding it makes the message eligible to move to the DLQ. SQS does not directly count how many exceptions your application code throws.
| Observed event | What this record establishes | What else to investigate |
|---|---|---|
| The code tries an operation three times after one receipt. | Three internal attempts, but one recorded queue receipt. | Which results the internal attempts left behind. |
| Work completes, deletion does not, and the request is received again. | A receive count can increase after successful application work. | Completion and deletion records; handling duplicate effects. |
| Someone receives a message for investigation, for example through a console. | Investigative receipt can also affect delivery state. | The operational investigation procedure and permission to receive. |
The DLQ's RedriveAllowPolicy controls which source queues can use it. Distinguish the source's rule for when to move a message from the DLQ's rule for which sources it accepts. Permissions for a person to start a redrive are another concern.
Read the retention clock for the queue type
In these fixed examples, a message moves to the DLQ one day after entering the source queue, and the DLQ retention period is four days. No intervening receipts, deletions or setting changes are included. Reading both examples as four days from DLQ arrival would overstate the remaining time for the standard queue.
Fixed teaching examples / not predictions of delivery timing or processing results
Standard queue → standard DLQ
- Day 0Enters the source queue
- Day 1Moves to DLQ; clock still starts at day 0
- Day 4Retention deadline in this example
Three days remain after the move to the DLQ.
Do not confuse elapsed time reported for the DLQ with time since the original enqueue.
FIFO queue → FIFO DLQ
- Day 0Enters the source queue
- Day 1Moves to DLQ; clock restarts here
- Day 5Retention deadline in this example
Four days remain after the move to the DLQ.
FIFO does not mean that DLQ isolation and return preserve the application's exact original order.
For standard queues, moving to a DLQ preserves the original enqueue timestamp used for expiry. For FIFO, enqueue time resets on the move. The DLQ's ApproximateAgeOfOldestMessage measures time since the move, so for a standard queue it does not by itself reveal the remaining retention.
Check the cause, duplicate effects and ordering before returning messages
| Investigation finding | Conditions to establish first | Do not assume |
|---|---|---|
| The consumer cannot read the older format. | A fix that can read it, with verified results for the affected example. | Moving a message automatically repairs its body. |
| The external write already succeeded. | Business request identity and handling of repeated writes or notifications. | A message in the DLQ has left no application effects. |
| Processing order changes the business result. | Relationships to earlier/later requests and the recovery processing order. | Returning to FIFO necessarily restores the original business sequence. |
| The cause is unknown and retention expiry is near. | The owner, deadline, investigation needed and organizational data handling. | An unknown cause is a reason to immediately return everything. |
Even with FIFO, isolating a message and returning it later can disrupt the original processing sequence. Redriven messages can mix with new messages in the order the destination receives them. If order matters to the application, moving a message back alone is not a complete recovery.
Redrive moves messages; fixing their cause is separate
SQS DLQ redrive moves isolated messages to their original source or another processing queue. In StartMessageMoveTask terminology, the move's source is the DLQ itself. This API supports DLQs whose sources are other SQS queues; do not assume it applies to a DLQ configured for Lambda asynchronous invocation.
Use the same queue type at the destination. SQS redrive does not filter messages or modify their bodies during the move. Redriven messages receive a new MessageId and enqueue time, with retention reset. A new message ID does not make the underlying business request a new job.
- Verify the fix or required prerequisites, and inspect application effects that already occurred.
- Check the destination, data handling, authority to act, retention and ordering.
- Plan a restrained starting rate that the destination can handle alongside normal work.
- Check application results and messages returning to isolation, not just moved-message counts.
- Define stop conditions if things worsen. Stopping a move is not a rollback of completed work.
Canceling a running redrive leaves messages that already moved in the destination. Plan separately for stopping the move and for correcting application writes or notifications. This lesson does not start a move task.
Distinguish an SQS DLQ from a Lambda asynchronous-invocation DLQ
An SQS event source mapping invokes the function synchronously. A receive-count-based DLQ on that SQS source queue differs from a failure destination or DLQ for Lambda asynchronous invocation. Identify which mechanism placed which kind of item in a queue before choosing a recovery method.
Choose the investigation and recovery conditions
Choose an option to see the result and explanation.
Check, then explain in your own words
- Identify the source and mechanism that sent a message to the DLQ.
- Separate message receipts from internal code attempts.
- Check queue type and the starting point of the retention clock.
- Establish the cause, duplicate handling, order, destination and rate before choosing reprocessing.
- Distinguish moved-message counts from application results and define stop conditions.
Check your understanding: Messages moved back to the original queue and the DLQ count is zero. Does that prove recovery is complete?
No. Inspect processing results at the destination and effects that already occurred.
A move is not application success. An unresolved cause can fail again, while duplicate effects and ordering issues may remain.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
Scope and conditionsIsolating unsuccessful messages, FIFO ordering risk and the different Standard/FIFO retention clock origins. No recovery of real business data verified.
Used in this lessonIsolation is different from repair or success maxReceiveCount does not count code exceptions Read the retention clock for the queue type Check the cause, duplicate effects and ordering before returning messages
Scope and conditionsExisting DLQ and matching Standard/FIFO queue type; creation/configuration is not performed by this lesson.
Used in this lessonIsolation is different from repair or success
Scope and conditionsRedrivePolicy/maxReceiveCount, RedriveAllowPolicy and changes to retention affecting existing messages. No policy or retention change applied.
Used in this lessonmaxReceiveCount does not count code exceptions Read the retention clock for the queue type
Scope and conditionsSame-type destination, move-rate planning, new message identity/time, no filtering or modification during redrive, and cancellation without rollback of completed moves.
Used in this lessonCheck the cause, duplicate effects and ordering before returning messages Redrive moves messages; fixing their cause is separate
Scope and conditionsThe move source is the DLQ; supported DLQs have other SQS queues as sources. Do not generalize to Lambda asynchronous-invocation DLQs. No task submitted.
Used in this lessonRedrive moves messages; fixing their cause is separate Distinguish an SQS DLQ from a Lambda asynchronous-invocation DLQ
- Using Lambda with Amazon SQSChecked:
Scope and conditionsPolling, synchronous batch invocation and deletion on success; default batch failure and partial batch response. No Lambda or event-source mapping created.
Used in this lessonDistinguish an SQS DLQ from a Lambda asynchronous-invocation DLQ