Recommended Scenario

Kubernetes packet loss

Ensure your Kubernetes systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate liveness/readiness probe and load balancer behavior.

Experiment types

Packet Loss

Targets

Kubernetes

Length

5 minutes

How it works

How this Scenario works

This Scenario gradually increases packet loss on a targeted Kubernetes pod, simulating degraded network conditions. This tests whether your Kubernetes liveness and readiness probes detect the degradation and whether your service and ingress controllers remove the unhealthy pod from rotation.

Use cases

Why run this Scenario?

  • Validate that Kubernetes liveness and readiness probe thresholds are configured to detect packet loss before users are impacted.
  • Verify that Kubernetes services and ingress controllers remove unhealthy pods from endpoints and redistribute traffic.
  • Test that Kubernetes restarts pods that fail liveness probes due to sustained packet loss.
  • Ensure that pods are returned to the service endpoint list correctly once packet loss resolves.
Result

What to expect when you run it

As packet loss degrades network communications to a Kubernetes pod, liveness and readiness probes mark it as unhealthy and traffic is redistributed to healthy pods.

Avoid downtime. Use Gremlin to turn failure into resilience.

Gremlin empowers you to proactively root out failure before it causes downtime. See how you can harness chaos to build resilient systems by requesting a demo of Gremlin.

Product Hero ImageShape