Azure HDInsight clusters are expensive to run and expensive to interrupt. A node lost during a long job invalidates hours of computation, and the cost lands twice: once in the wasted compute, again in every downstream process waiting on output that never arrived.

Teams generally trust the framework to handle this. Spark and Hadoop do have fault tolerance, but how much it protects you depends on how the job was written, how often it checkpoints, and how the data was partitioned. Those choices vary job to job and were usually made for performance rather than resilience.

Install the Gremlin Agent on your cluster nodes and Shutdown experiments separate the jobs that genuinely recover from the ones that restart from the beginning. CPU and memory experiments show whether a resource-starved node degrades job performance quietly before it fails outright, which is the harder problem to spot.

Knowing which jobs are fragile lets you spend engineering effort where an interruption actually costs something, rather than treating every pipeline as equally safe.