Container memory (OOMKiller) resilience
Validate that your container runtime's OOMKiller detects excess memory usage and safely restarts the target pod or container.
Experiment types
Memory
Targets
Containers
Length
1 minute
How this Scenario works
This Scenario forces memory pressure in a pod to verify that Kubernetes terminates it before it consumes too much memory.
Why run this Scenario?
The Linux operating system has an Out of Memory Manager (sometimes referred to as the OOMKiller) that kills processes when it detects that the system is out of memory. Some container runtimes, such as Kubernetes, have an additional OOMKiller that terminates container processes when they violate their memory resource limits. In both cases, the OOMKiller is a resilience mechanism, and this Scenario verifies that it works as intended.
You can use this Scenario to:
- Validate system stability and resiliency in preparation for high traffic events, like Black Friday, Cyber Monday, and other major sales holidays.
- Reduce operating expenses by optimizing infrastructure capacity based on application memory requirements.
- Streamline cloud migrations by simulating cloud conditions on-premises, such as noisy neighbors on shared infrastructure.
In addition to Kubernetes workloads, this Scenario can also help:
- Simulate memory-intensive workloads such as in-memory caches, databases, and machine learning models.
- Test performance when using disk swapping/paging.
What to expect when you run it
The OOMKiller terminates the container before it can consume more memory than its limit allows, and the container restarts and returns to a healthy state.
More Scenarios to try
Kubernetes packet loss
Ensure your Kubernetes systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate liveness/readiness probe and load balancer behavior.
Container packet loss
Ensure your containerized systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate health check and load balancer behavior.
Windows packet loss
Ensure your Windows systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate health check and load balancer behavior.
Avoid downtime. Use Gremlin to turn failure into resilience.
Gremlin empowers you to proactively root out failure before it causes downtime. See how you can harness chaos to build resilient systems by requesting a demo of Gremlin.
.svg)