Topic lessonTopic: Audit coverage and recovery design
Designing an AWS recovery game day with measurable outcomes
A completed runbook is only one outcome of a drill. Define scope, abort conditions, observers and business criteria, then carry unverified conditions into the next exercise.
Design an isolated recovery exercise and assess elapsed time, data integrity and containment independently.
This is a fictional tabletop drill. No outage is introduced into a live service and no production data is used.
Define what to measure and when to stop
A game day rehearses recovery with representative roles and procedures, measures business outcomes, and produces improvements. The fixture uses synthetic data in an isolated setting.
- Prepare → SimulateDefine scope and roles
- Simulate → EvaluateCollect times and records
- Evaluate → RetestCarry forward gaps
- Prepare
- Agree on scope and abort criteria
- Simulate
- Observers record timestamps
- Evaluate
- Check functions, time and data
- Retest
- Assign owners and measurement conditions
A drill can meet the time objective and still fail
| Measure | Observed | Decision |
|---|---|---|
| Isolation and abort controls | No out-of-scope traffic; abort owner agreed | Proceed |
| Business resumption | 18 minutes | Time passes |
| Committed order reconciliation | 19 of 20 orders match | Data fails |
| Failover under high load | Outside scope | Unverified |
Disaster recovery testing can check workload objectives. The exercise requires all the stated success criteria, so meeting the time objective cannot make up for a mismatched order.
Use measured results to update procedures
Assign authority to stop. Decide who stops the drill if traffic leaves scope, real data appears, or measurement is lost. Record the stop time and safe cleanup.
Keep unverified cases visible. Record topology, workload, input data and measurement window. Retest conclusions when load or dependencies change.
Repeat a controlled measurement. Add a check that detects the missing order, then rerun the reconciliation and timing against the same test data.
Check your decision
Choose an option to see the result and explanation.
Check, then explain in your own words
- Agree on success conditions and abort authority.
- Record observations with timestamps.
- Assign fixes and define the next measurement.
Check your understanding: What does completing every runbook step show?
It shows that the steps could be followed. Business recovery still needs separate time and data evidence.
Execution and outcomes are assessed separately.
Sources and conditions checked
The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.
- Conduct game days regularlyChecked:
Scope and conditionsScope, participants, impact monitoring and improvements after drills.
Used in this lessonDefine what to measure and when to stop Use measured results to update procedures
Scope and conditionsExercise recovery and verify objectives for the workload.
Used in this lessonA drill can meet the time objective and still fail Use measured results to update procedures