Test how your application handles an Azure Event Hubs outage. Run Gremlin network experiments and reliability tests to validate streaming pipeline resilience.
Consumers of Azure Event Hubs track their position through checkpoints, and that mechanism is where disruption does its real damage. A consumer that fails before checkpointing reprocesses events. One that checkpoints before completing work skips them. Neither shows up on a dashboard reporting healthy throughput.
Partition rebalancing makes it more likely:
- A disruption interrupts one consumer, and ownership of its partitions moves to another.
- The handoff happens at exactly the moment checkpoint timing matters most.
- Events either get processed twice or skipped, depending on which side of the checkpoint the interruption landed.
- The discrepancy surfaces weeks later as data that doesn't reconcile, long after anyone would connect it to a brief broker problem.
Gremlin lets you trigger that handoff on purpose. A blackhole experiment against the Event Hubs endpoint interrupts a consumer while events are flowing, and a latency experiment slows processing until a rebalance triggers.
Duplicate processing and dropped events are both survivable if you know which one you have. Both are expensive if a customer finds out first.
Kafka broker load on Linux
Test the impact of I/O pressure on Kafka broker throughput and partition synchronization on Linux hosts. As disk I/O usage increases on the broker's host, pipeline performance will decrease, but should stay within expected bounds.
Dependencies: Failure Test
Simulate a failed dependency by dropping all network traffic to the dependency.
Dependencies: Latency Test
Recreate poor network conditions by delaying all network traffic to a dependency by 100ms.
* Testing a dependency doesn't require installing anything on it. Gremlin runs the experiment from the service that calls it, so compatibility depends on that host rather than on the dependency itself. Check our compatibility documentation for supported operating systems and platforms, or get in touch if you don't see yours.
Intelligent Health Checks - Azure
Automatically monitor your Azure services during testing in one-click with Intelligent Health Checks.
Avoid downtime. Use Gremlin to turn failure into resilience.
Gremlin empowers you to proactively root out failure before it causes downtime. See how you can harness chaos to build resilient systems by requesting a demo of Gremlin.
.svg)