Topic lessonTopic: Step Functions: order, branches and retries
AWS Step Functions: workflows, Lambda, Retry and Catch
AWS Step Functions coordinates the order and conditions in which multiple tasks proceed. For example, a workflow can check data, call a task when a condition matches, retry a failure a fixed number of times, and save the outcome. Distinguish coordinating the workflow from the Lambda functions or other components that perform the individual tasks.
Read a small workflow and explain which component performs work, which path a condition selects, when it waits, and how finite retries and error-handling paths work.
Know that Lambda runs code in response to a request and that tasks can fail. This lesson reads a fictional workflow. It does not connect to AWS, create a workflow, or run anything that incurs AWS charges.
Lambda performs work; Step Functions coordinates the workflow
Consider receiving an image, checking its format, resizing it when necessary, and saving the outcome. Lambda or another worker runs the code that resizes the image. A workflow describes when to call the next task and which conditions lead to a different task.
The definition of this flow is a state machine. An execution processes one input according to that definition. Processing image A and processing image B are separate executions even when they use the same definition. Writing the definition does not mean the real data has already been processed.
You do not need to wrap every step in a Lambda function. A Task can also represent an integration with a supported AWS service. Check the work you need and the integration it supports first.
- Coordinate → Run workCall work according to the definition and receive its result
- Coordinate
- Step Functions: order, conditions, and error-handling paths
- Run work
- Lambda in this example: code that checks or resizes an image
Follow one input through the matching path
Assume that the image check returns whether resizing is needed. If it is needed, the workflow takes the resize path. Otherwise, it goes directly to saving the result. Choice selects the matching path; it does not run every path at once.
Also consider input that matches none of the conditions. A Choice Default specifies the path for that case. If no condition matches and there is no Default, the execution cannot transition to the next state and an error occurs. A branching diagram should also account for unexpected input.
- Check input → Choose a pathPass the result of the check
- Choose a path → Resize imageResizing is needed
- Choose a path → Save resultResizing is not needed
- Choose a path → Review inputNo condition matches
- Resize image → Save resultResizing succeeds
- Check input
- Use the result of checking the image
- Choose a path
- Choice: resizing needed, not needed, or unexpected input
- Resize image
- Run this task only when resizing is needed
- Save result
- This state is also used when resizing is not needed
- Review input
- Default: the input matches none of the expected conditions
Distinguish tasks, choices, parallel branches, and waits
| State | Role | Example to read |
|---|---|---|
| Task | A unit of work to call and run | Check or resize an image. The worker implementation or service integration is still required. |
| Choice | Choose the next path using conditions | Decide whether resizing is needed. This does not request parallel execution. |
| Parallel | Run work in separate branches | Use separate branches for independent checks. Normal completion waits for the branches to finish; failure handling must also be defined. |
| Wait | Delay progression for a duration or until a timestamp | Leave an interval before the next task. Distinguish this from keeping Lambda code running just to wait. |
Two retries allow up to three calls including the first
This example retries the resize task only when it fails with an error named TransientFailure. Set MaxAttempts to 2, the initial retry interval to 2 seconds, BackoffRate to 2, and disable jitter. For this one state execution, at most two retries can follow the initial call.
{
"Retry": [
{
"ErrorEquals": [
"TransientFailure"
],
"IntervalSeconds": 2,
"MaxAttempts": 2,
"BackoffRate": 2,
"JitterStrategy": "NONE"
}
],
"Catch": [
{
"ErrorEquals": [
"TransientFailure"
],
"Next": "RecordForReview"
}
]
}ErrorEquals names the matching errors. IntervalSeconds is the interval before the first retry. MaxAttempts limits additional retries. BackoffRate multiplies the interval, and JitterStrategy determines whether the wait is randomized. NONE disables that randomization in this example.
Assume all three calls fail with the same error in the following table. If a call succeeds earlier, the later retries in the table do not happen; the workflow follows the normal next state. The 2-second and 4-second waits precede retries. They are not completion times that include the duration of the work.
| Call | Type of call | Given result | What to read next |
|---|---|---|---|
| 1 | Initial call | TransientFailure | A retry remains: wait 2 seconds |
| 2 | First retry | TransientFailure | A retry remains: wait 4 seconds |
| 3 | Second retry | TransientFailure | This retry allowance is exhausted: check Catch |
Retry repeats work; Catch takes an error-handling path
In this example, a Catch matching TransientFailure leads to another state that records an outcome requiring review. Read the matching Catch and its next state when no Retry matches the error or retrying does not resolve it.
Taking a Catch path does not prove that resizing succeeded. Distinguish recording a failure or taking a fallback path from completing the original task. A differently named error does not match this example's Retry or Catch.
| Given sequence | Calls | Next path in this example |
|---|---|---|
| Success on the initial call | 1 | The normal next state |
| TransientFailure, then success | 2 | The normal next state |
| TransientFailure three times | 3 | The matching Catch leads to RecordForReview |
| A different error name | 1 | This Retry/Catch does not match |
Check the execution model for Standard and Express
Standard is a candidate for long-running work and workflows whose history you need to inspect. Express is a candidate for many short event-driven workflows. Their maximum durations, history mechanisms, and billing models also differ. Do not choose by the name alone.
Standard provides an exactly-once workflow execution model; explicitly configured Retry behavior is an exception to read separately. Asynchronous Express uses an at-least-once model, so duplicate executions can occur. Synchronous Express uses an at-most-once model, which does not guarantee successful completion. Do not extend these terms into a promise that every external write happens exactly once.
| Workflow type | Execution model to check | What to check separately |
|---|---|---|
| Standard | Exactly-once; explicitly configured Retry is an exception | Retries, resumption, and the results of external work |
| Asynchronous Express | At-least-once; duplicate executions can occur | The effects of handling the same request again |
| Synchronous Express | At-most-once; this is not a guarantee of success | The failure outcome and any necessary recovery |
Distinguish retries, redrive, and new executions
Retry repeats work within a state. Starting a new execution begins a separate execution. Standard redrive resumes an eligible unsuccessful execution using its retained results and history. These are different operations even if all are informally called trying again.
Retry counters can reset for states rerun during redrive. The work that reruns also depends on the branch and error. The maximum of three calls in the earlier table is not a lifetime limit of three calls for the business operation.
Idempotency means that handling the same request again does not produce unwanted duplicate effects. For example, consider identifying the item and checking whether its result has already been stored. This lesson's diagram does not verify duplicate prevention in a real data store or payment system.
Explain the path, call count, and failure outcome
Choose an option to see the result and explanation.
Check, then explain in your own words
- Distinguish the workflow definition from an execution for one input.
- Read both the matching path and the path for unexpected input.
- Check the error name, the additional retry allowance, and the path after it is exhausted.
- Separate what the example shows from new executions, redrive, and external task outcomes.
Check your understanding: Does allowing two retries limit the business operation to three calls forever?
A bound for one state execution is different from the cumulative count for the business operation.
This example permits up to three calls: one initial call plus two retries. A new execution and Standard redrive must be considered separately, including the conditions that reset retry counters for rerun states. Check duplicate effects through the design and actual outcomes of the destination, rather than treating the diagram as proof.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
- What is Step Functions?Checked:
Scope and conditionsState machine, state, and execution terminology; coordination with other services and external workers. Regional availability and changing numeric limits are outside this lesson.
Used in this lessonLambda performs work; Step Functions coordinates the workflow
- Task workflow stateChecked:
Scope and conditionsTask as a unit of work; distinguish Lambda from other supported AWS service integrations. The worker implementation, permissions, and successful execution are not verified.
Used in this lessonLambda performs work; Step Functions coordinates the workflow Distinguish tasks, choices, parallel branches, and waits
- Choice workflow stateChecked:
Scope and conditionsConditional next states and Default; transition errors when no condition matches and no Default exists. The example does not evaluate JSONPath or JSONata expressions.
Used in this lessonFollow one input through the matching path Distinguish tasks, choices, parallel branches, and waits
- Parallel workflow stateChecked:
Scope and conditionsNormal completion waits for all branches. Distinguish a Parallel state failing from already invoked Lambda functions or workers stopping immediately.
Used in this lessonDistinguish tasks, choices, parallel branches, and waits
- Wait workflow stateChecked:
Scope and conditionsDelay progression for a relative duration or until a timestamp, subject to workflow-type restrictions. This does not guarantee a completion time including task duration.
Used in this lessonDistinguish tasks, choices, parallel branches, and waits
Scope and conditionsTask ErrorEquals matching, Retry/Catch order, additional retries counted by MaxAttempts, intervals, backoff, and jitter. Only a finite synthetic state example is calculated separately; all error handling and live AWS execution are not verified.
Used in this lessonTwo retries allow up to three calls including the first Retry repeats work; Catch takes an error-handling path
Scope and conditionsSeparate Standard execution semantics and its configured-Retry exception from asynchronous and synchronous Express guarantees. No guarantee is made about duplicate external effects or applicable costs.
Used in this lessonCheck the execution model for Standard and Express
Scope and conditionsEligible Standard redrive, retained results, branch/error-specific rerun scope, and retry-counter resets for rerun states. No redrive operation or procedural test is performed.
Used in this lessonDistinguish retries, redrive, and new executions
Scope and conditionsRetry responsibility depends on the invoker and intervening services; consider side effects when the same event is handled again. Existing lessons cover the details of Lambda asynchronous invocation and queue settings.
Used in this lessonDistinguish retries, redrive, and new executions