Gremlin and AWS ASG

An Auto Scaling Group is the safety net under everything else you've built on EC2, and it's the component least likely to have been tested. Scaling policies and autohealing only run during the exact conditions you can least afford to have them fail.

Misconfiguration here is quiet. A cooldown period that's too long, a health check that passes on an instance serving errors, or a maximum capacity set below your actual peak will sit dormant for months. Nothing in your dashboards flags them, because from a monitoring perspective the group is doing exactly what it was told.

Bake the Gremlin Agent into your AMI or bootstrap it through user data so every new instance self-registers, then exercise the group deliberately. CPU and memory experiments drive load past your scaling thresholds so you can see where they actually sit relative to the point your application starts degrading. Shutdown experiments confirm that unhealthy instances get replaced, and how quickly traffic stops reaching them.

What you get back is the number you need before your next traffic spike, rather than the assumption you've been running on since the group was created.