INFRASTRUCTURE LEARNINGINFRASTRUCTURE LEARNING
インフラ大全インフラ大全IT INFRASTRUCTURE / LEARNING
Infrastructure learning guide
Find a lesson or lab/

Esc to close · Search terms stay on your device

LEARNING日本語

Topic lessonTopic: Audit coverage and recovery design

Designing an AWS recovery game day with measurable outcomes

A completed runbook is only one outcome of a drill. Define scope, abort conditions, observers and business criteria, then carry unverified conditions into the next exercise.

What you will learn

Design an isolated recovery exercise and assess elapsed time, data integrity and containment independently.

Prerequisite

This is a fictional tabletop drill. No outage is introduced into a live service and no production data is used.

Define what to measure and when to stop

A game day rehearses recovery with representative roles and procedures, measures business outcomes, and produces improvements. The fixture uses synthetic data in an isolated setting.

From preparation to the next exercise
From preparation to the next exercise1: Prepare — Define scope and roles — Simulate. 2: Simulate — Collect times and records — Evaluate. 3: Evaluate — Carry forward gaps — Retest123PrepareSimulateEvaluateRetestFrom preparation to the next exercise1: Prepare — Define scope and roles — Simulate. 2: Simulate — Collect times and records — Evaluate. 3: Evaluate — Carry forward gaps — Retest123PrepareSimulateEvaluateRetest
  1. Prepare → SimulateDefine scope and roles
  2. Simulate → EvaluateCollect times and records
  3. Evaluate → RetestCarry forward gaps
Prepare
Agree on scope and abort criteria
Simulate
Observers record timestamps
Evaluate
Check functions, time and data
Retest
Assign owners and measurement conditions

A drill can meet the time objective and still fail

Resume within 20 minutes and verify 20 committed orders. Any outbound traffic beyond scope stops the drill.
MeasureObservedDecision
Isolation and abort controlsNo out-of-scope traffic; abort owner agreedProceed
Business resumption18 minutesTime passes
Committed order reconciliation19 of 20 orders matchData fails
Failover under high loadOutside scopeUnverified

Disaster recovery testing can check workload objectives. The exercise requires all the stated success criteria, so meeting the time objective cannot make up for a mismatched order.

Use measured results to update procedures

Assign authority to stop. Decide who stops the drill if traffic leaves scope, real data appears, or measurement is lost. Record the stop time and safe cleanup.

Keep unverified cases visible. Record topology, workload, input data and measurement window. Retest conclusions when load or dependencies change.

Repeat a controlled measurement. Add a check that detects the missing order, then rerun the reconciliation and timing against the same test data.

Check your decision

Choose an option to see the result and explanation.

Resumption took 18 minutes and 19 of 20 orders matched. What is the overall result?
Read the answer and reason

Answer: Fail · Both time and data must pass.

An out-of-scope outbound request appears. What next?
Read the answer and reason

Answer: Stop and follow safe cleanup · The request meets the agreed abort condition.

The team skipped a high-load failover test. How should it be reported?
Read the answer and reason

Answer: Mark high-load failover unverified · No evidence establishes the result for that load.

Check, then explain in your own words

  1. Agree on success conditions and abort authority.
  2. Record observations with timestamps.
  3. Assign fixes and define the next measurement.
Check your understanding: What does completing every runbook step show?

It shows that the steps could be followed. Business recovery still needs separate time and data evidence.

Execution and outcomes are assessed separately.

Sources and conditions checked

The official references used in this lesson are collected here with their scope, conditions and corresponding sections. Independent expert review and live AWS comparison have not been performed.