A pilot-light DR setup that tests itself every month and emails you the measured RTO and RPO.
The problem
Most DR plans are a document that has never been tested. When the day comes, the restore takes far longer than anyone promised, or doesn’t work at all. A backup you haven’t restored is only a hope. This project makes restores routine and measures them.
What you’ll build
- An AWS Backup plan backing up everything tagged
dr=truedaily with a cross-region copy - A pilot-light DR region: vault + empty VPC that costs nothing until used
- A Step Functions restore drill: find latest recovery point โ restore โ wait โ health and HTTP checks โ measure โ terminate โ report
- A monthly scheduled drill (EventBridge Scheduler) with CloudWatch metrics for RTO, RPO and pass/fail
- A demo nginx workload to protect, plus vault-lock guidance for ransomware resilience
Architecture
us-east-1 (primary) us-west-2 (DR, "pilot light")
โโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
resources tagged dr=true dr-vault โโโ cross-region copy (daily, 14-day retention)
โโ AWS Backup plan (daily 05:00 UTC) โโโโโโโโถ empty VPC + SG (costs $0 until used)
Step Functions "dr-restore-drill":
find latest point โ restore EC2 โ wait โ
status + HTTP checks โ measure RTO โ terminate โ report
EventBridge Scheduler: first Monday of each month
AWS services used
How it works
- ProtectTag resources
dr=true. Backup does the rest nightly and copies to the DR region. - RehearseOn the first Monday of each month a state machine restores the latest copy into the DR VPC.
- VerifyIt waits for status checks and an HTTP 200, so “restored” really means “working”.
- ReportMeasured RTO/RPO go to CloudWatch and your inbox, and the test instance is cleaned up.
Key design decisions
- Pilot light keeps standby cost near zero while still proving recoverability.
- Drills measure actual RTO, so you replace a guess with a number you can show.
- Clear upgrade paths: Elastic Disaster Recovery for second-level RPO, vault lock and a separate account for ransomware.
Deploy it
npm install
npx cdk bootstrap aws://ACCOUNT/us-east-1 aws://ACCOUNT/us-west-2
npx cdk deploy --all -c alertEmail=you@example.com
# custom regions: -c primaryRegion=us-east-2 -c drRegion=us-west-1
Full prerequisites, parameters and clean-up steps are in the project README on the code page.
See it work
Start the state machine. About 10โ15 minutes later an email arrives reading "result": "PASS", "measured_rto_minutes": โฆ, and the restored instance has already been cleaned up.
Skills this demonstrates
Get the code
Every file needed to build this project, written as a deployable AWS CDK app and validated with the CloudFormation schema.
Leave a Reply