Topic lessonTopic: Designing event-driven processing
Trace serverless operating costs across Lambda, SQS, and CloudWatch Logs
Retries and log output can increase usage even when the number of business requests stays constant. Follow an image-processing example and examine Lambda invocations and duration, SQS API requests, and CloudWatch Logs ingestion and storage.
Compare business volume and service usage over matching intervals. Trace whether retries, execution time, or increased log volume contributed to a usage increase.
This scenario uses SQS to buffer work and Lambda to process it. The measurements are fictional. They illustrate usage patterns without estimating a bill or changing any configuration.
Follow work through the services that meter usage
- Image intake → SQS queueSubmits the job
- SQS queue → Lambda workerHands received messages to the worker
- Lambda worker → CloudWatch LogsWrites processing logs
- Image intake
- Counts business requests
- SQS queue
- Send, receive, and delete actions
- Lambda worker
- Invocation count and execution time
- CloudWatch Logs
- Ingestion and stored volume
| Service | Evidence to check | What can increase usage |
|---|---|---|
| Lambda | Invocation count, execution time, and memory setting | Redelivery or dependency delays can add invocations or increase duration. |
| SQS | API requests and message payload sizes | Redelivery, empty receives, and batching affect operations per job. |
| CloudWatch Logs | Ingestion, storage, and bytes scanned by queries | Verbose logging adds input. Retention changes stored volume. |
Compare the same measures across two one-hour windows
| Measure | Before | During |
|---|---|---|
| Accepted requests | 1,000 | 1,000 |
| Lambda invocations | 100 | 170 |
| Total SQS messages received | 1,000 | 1,600 |
| Log ingestion | 20 MB | 85 MB |
Accepted requests stayed constant while received messages and function invocations rose. The receive metric can count redeliveries. The number of received messages is a different measure from billable API requests. Correlate request IDs with processing results to see whether the same work ran again.
Log ingestion rose to 85 MB. Identify the added output, such as repeated error details. Retention changes stored volume while previously ingested bytes remain part of measured ingestion. Inspect ingestion, storage, and query usage separately.
Identify the cause and verify recovery before tuning
Use matching periods and resources to compare accepted and completed work, redeliveries, execution time, and log volume. Fix the cause of continuing errors and confirm that replay can avoid repeating business effects. Preserve the records needed for investigation when you adjust log output, then monitor ingestion.
Choose the next evidence to check
Choose an option to see the result and explanation.
Check, then explain in your own words
- Choose comparable intervals and resources before and during the incident.
- Record intake, redeliveries, execution time, and log ingestion in their own units.
- Connect the changed usage to configuration and the associated billing line items.
Check your understanding: Lambda invocations dropped after an incident. What else would show recovery?
Check successful work completions and confirm that backlog and failure signals have improved.
Invocations can fall when the worker is unable to receive jobs. Compare completions and waits alongside changes in log volume and cost.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
- AWS Lambda PricingChecked:
Scope and conditionsRequest and duration pricing and the role of memory configuration.
Used in this lessonFollow work through the services that meter usage
- Amazon SQS PricingChecked:
Scope and conditionsIndividual API actions, batching and payload size in request pricing.
Used in this lessonFollow work through the services that meter usage Compare the same measures across two one-hour windows
- Amazon CloudWatch PricingChecked:
Scope and conditionsSeparate CloudWatch Logs ingestion, archival storage and query usage.
Used in this lessonFollow work through the services that meter usage Compare the same measures across two one-hour windows Identify the cause and verify recovery before tuning
Scope and conditionsReceived messages can include redeliveries and are distinct from unique work.
Used in this lessonCompare the same measures across two one-hour windows Identify the cause and verify recovery before tuning