INFRASTRUCTURE LEARNINGINFRASTRUCTURE LEARNING
インフラ大全インフラ大全IT INFRASTRUCTURE / LEARNING
Infrastructure learning guide
Find a lesson or lab/

Esc to close · Search terms stay on your device

LEARNING日本語

Topic lessonTopic: Designing event-driven processing

Backpressure: manage SQS backlog and downstream overload

Backpressure uses evidence of overload to slow the upstream flow of work. In a fictional image-processing system, limit concurrent consumers to protect the downstream database, then adjust intake to limit the time work waits in the queue.

What you will learn

Estimate backlog from arrival and completion rates, interpret queue depth and oldest-message age, and explain which upstream or downstream controls to adjust.

Prerequisite

An SQS queue holds work between producers and consumers. The figures below are fixed teaching examples. They require no live AWS metrics or configuration changes.

A queue buys time to catch up

Unfinished work accumulates while arrivals outpace completions. A queue can absorb a short burst. If the gap persists, some requests will miss their required completion time.

Protect the downstream service and use backlog signals to adjust intake
Protect the downstream service and use backlog signals to adjust intake1: Image-processing intake — Enqueues accepted work — SQS queue. 2: SQS queue — Delivers batches — Lambda consumer. 3: Lambda consumer — Writes results — Downstream database. 4: SQS queue — Uses backlog signals to adjust intake — Image-processing intake1234Image-processingintakeSQS queueLambda consumerDownstream databaseProtect the downstream service and use backlog signals to adjust intake1: Image-processing intake — Enqueues accepted work — SQS queue. 2: SQS queue — Delivers batches — Lambda consumer. 3: Lambda consumer — Writes results — Downstream database. 4: SQS queue — Uses backlog signals to adjust intake — Image-processing intake1234Image-processingintakeSQS queueLambda consumerDownstreamdatabase
  1. Image-processing intake → SQS queueEnqueues accepted work
  2. SQS queue → Lambda consumerDelivers batches
  3. Lambda consumer → Downstream databaseWrites results
  4. SQS queue → Image-processing intakeUses backlog signals to adjust intake
Image-processing intake
Adjusts the rate of new work
SQS queue
Holds work that is waiting
Lambda consumer
Limits concurrent work
Downstream database
Accepts result writes

Use arrival and completion rates to estimate recovery

Teaching model: initially empty, steady rates within each interval, no failures or retries
IntervalArrival rateCompletion rateChange in unfinished work
First 120 seconds80/s50/sAdds 30 per second and leaves 3,600 unfinished.
After the peak30/s50/sAfter serving new arrivals, drains 20 per second.
Until the backlog clears30/s50/s3,600 ÷ 20 = 180 seconds.

New work continues after the peak, leaving 20 completions per second to clear the backlog. Dividing by the total completion rate of 50 per second instead models a complete stop in new arrivals. In an actual system, update the estimate as processing time, failures, and retries change.

Read depth, age, and downstream health together

Use ApproximateNumberOfMessagesVisible to follow the trend in messages available to receive. ApproximateAgeOfOldestMessage estimates the age in seconds of the oldest unprocessed message. Compare in-flight counts and end-to-end time from intake to result commit to see whether work is waiting elsewhere.

These metrics have approximation and reporting limits. In standard queues, repeatedly received messages that never complete can be left out of the oldest-age metric. When oldest age improves, also check failures, dead-letter records, and elapsed time for completed requests.

Different causes of backlog call for different controls
EvidenceControl to considerRecovery evidence
Database latency and failures increaseCap consumer concurrency and reduce new intake.Downstream error rate, completion rate, and backlog age.
Database has spare capacity and consumers hit a limitRaise the limit in steps while checking available capacity.Higher completion rate with acceptable downstream latency.
Low-priority batches monopolize intakeSeparate admission limits and defer work that can wait.Age for priority work and the number of deferred requests.

With a standard-mode SQS event source mapping, Lambda maximum concurrency caps how many function instances the mapping can invoke at once. Batch size and processing duration also affect throughput, so translate concurrency into completions per second using observations.

Choose which control can restore throughput

Choose an option to see the result and explanation.

For 120 seconds, 80 requests per second arrive and 50 complete. Starting from zero, what remains?
Read the answer and reason

Answer: 3,600 requests · The queue gains 30 requests per second for 120 seconds.

Arrivals fall to 30 per second and completions stay at 50. How long will it take to clear 3,600 requests?
Read the answer and reason

Answer: 180 seconds · After accounting for new arrivals, 20 completions per second are available to drain the backlog.

Database latency and errors are rising. What response should you try first?
Read the answer and reason

Answer: Review the concurrency cap and reduce new intake · Protect the downstream service, adjust incoming work, and observe errors and backlog.

Visible depth is flat while oldest-message age grows. What do you check next?
Read the answer and reason

Answer: Completion rate, failures, and time from acceptance to result commit · A flat count can coexist with long waits. Compare completion, failure, and end-to-end timing.

Check, then explain in your own words

  1. Compare accepted work and successful completions over the same observation window.
  2. Check oldest-message age, in-flight counts, and downstream latency together.
  3. Check downstream capacity before adjusting consumer concurrency and the intake rate.
Check your understanding: Lower Lambda concurrency protected the database. What should you check to keep waits acceptable too?

Compare the intake rate with the current completion rate and observe the trend in oldest-message age.

A consumer limit leaves the rate of new work unchanged. Adjust intake as needed and observe both completed work and the remaining backlog.

Sources and conditions checked

The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.