Kubernetes memory (OOMKiller) resilience
Validate that Kubernetes' OOMKiller detects excess memory usage and safely restarts the target pod or container.
Memory
Kubernetes
1 minute
How this Scenario works
This Scenario consumes memory inside a targeted Kubernetes pod until it exceeds the pod's memory resource limit. At that point the OOMKiller terminates the process, which also terminates the Gremlin attack. That interruption is the result you want to see, not a failed experiment.
Why run this Scenario?
The Linux operating system has an Out of Memory Manager (often called the OOMKiller) that kills processes when it detects that the system is out of memory. In Kubernetes, the OOMKiller terminates a process in a pod when that process violates its memory resource limit. It is a resilience mechanism, and this Scenario confirms that it works.
- Validate that a runaway process is stopped before it can starve the rest of the node.
- Confirm that pod memory resource limits are set where you think they are.
- Verify that your application returns to a healthy state after the pod restarts.
For more detail on how Gremlin reports this result, see Attack interrupted by OOMKiller.
What to expect when you run it
The OOMKiller terminates the target before it can consume too much memory, and the pod restarts and returns to a healthy state once the Scenario finishes.
More Scenarios to try
Avoid downtime. Use Gremlin to turn failure into resilience.
Gremlin empowers you to proactively root out failure before it causes downtime. See how you can harness chaos to build resilient systems by requesting a demo of Gremlin.
.svg)