Gremlin Scenario: Kubernetes monitoring and alerting test

How it works

How this Scenario works

This Scenario runs four sequential resource experiments targeting your Kubernetes pods, each generating a short spike in a different resource: CPU, disk I/O, memory, and network latency. This validates that your observability stack has full visibility into Kubernetes-orchestrated workloads.

Use cases

Why run this Scenario?

Verify that your monitoring tools capture pod-level and node-level resource metrics across your Kubernetes cluster.
Detect gaps in alert coverage specific to Kubernetes workloads, including pod resource requests and limits.
Validate that Kubernetes-native monitoring (such as metrics-server or Prometheus) is correctly configured.
Build confidence that resource pressure in your Kubernetes environment will be detected and surfaced before it impacts application health.

Result

What to expect when you run it

As each experiment runs within a Kubernetes pod, your monitoring tools show corresponding spikes in CPU, disk I/O, memory usage, and packet latency with minimal delay.

Keep going

More Scenarios to try

Kubernetes packet loss

Ensure your Kubernetes systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate liveness/readiness probe and load balancer behavior.

Container packet loss

Ensure your containerized systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate health check and load balancer behavior.

Windows packet loss

Ensure your Windows systems can detect and route around resources that fail health checks. This Scenario gradually increases packet loss to validate health check and load balancer behavior.

Kubernetes monitoring and alerting test