HOW TO VALIDATE OUTAGE RESILIENCE AT SCALE

Modernizing Disaster Recovery Testing

Disaster recovery plans  are essential, and they need to be tested, but only 33% of organizations regularly perform disaster recovery simulations and have an organized response approach, and 71% never test their failovers.

Engineering leaders need a systematic, accurate method that can be done at scale with robust reporting and the capability to provide coverage between annual tests to compensate for changing architectures and systems.

This guide goes over this new approach and how you can use it to safely, easily perform disaster recovery testing across your company.

Download the guide

Thanks for requesting Modernizing Disaster Recovery Testing. Click here to view the guide! (A copy has also been sent to your email.)

‍

About the Authors

Jordan Pritchard

Director of Infrastructure & Site Reliability Engineering

Michael Kehoe

Architect of reliable, scalable infrastructure

Rodney Lester

Technical Lead, Reliability Pillar of Well Architected Program

Tammy Butow

Principal SRE

Jay Holler

Manager, Site Reliability Engineering

Ramin Keene

Founder

In this comprehensive guide, find out:

  • Why only 20% of organizations text failover systems
  • Where the current disaster recovery testing methods aren't enough
  • What a new approach needs to be effective
  • How to systematically roll out that approach across your company

  • Incident classification: SEV descriptions and levels, and SEV and time-to-detection (TTD) timelines

  • Organization-wide critical service monitoring, including key dashboards and KPI metrics emails

  • Service ownership and metrics for organizations maintaining a microservices architecture

  • Effective on-call principles for site reliability engineers, including rotation structure, alert threshold maintenance, and escalation practices

  • Chaos Engineering practices to identify random and unpredictable behavior in your system

  • Monitoring and metrics to detect incidents caused by self-healing systems

  • Creating a high-reliability culture by listening to people in your organization

Over a decade of collective experience unleashing chaos at companies like

Avoid downtime. Use Gremlin to turn failure into resilience.

Gremlin empowers you to proactively root out failure before it causes downtime. See how you can harness chaos to build resilient systems by requesting a demo of Gremlin.

get started
Product Hero ImageShape
© 2026 Gremlin IncAll rights reserved.Privacy Policy