Gremlin Scenario: Kubernetes zone redundancy

How it works

How this Scenario works

This Scenario drops network traffic between your Kubernetes nodes in one availability zone and the rest of your cluster, simulating a complete zone outage. This forces Kubernetes to reschedule pods and for your ingress controllers and service mesh to redirect traffic to pods running in the remaining zones.

Use cases

Why run this Scenario?

Demonstrate that your Kubernetes cluster is truly zone-redundant, with pods distributed using topology spread constraints.
Verify that Kubernetes ingress controllers and service meshes detect the zone outage and redirect traffic appropriately.
Test whether pod anti-affinity rules ensure sufficient replicas in each zone to handle failover traffic.
Validate that pod disruption budgets maintain service availability during the zone failure.

Result

What to expect when you run it

When a zone fails, Kubernetes automatically routes traffic to pods in secondary zones with no interruption in service.

Keep going

More Scenarios to try

Kubernetes packet loss

Ensure your Kubernetes systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate liveness/readiness probe and load balancer behavior.

Container packet loss

Ensure your containerized systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate health check and load balancer behavior.

Windows packet loss

Ensure your Windows systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate health check and load balancer behavior.

Kubernetes zone redundancy