Daemon & Client Overview
Client <—> Daemon —> Control Plane
The Gremlin daemon,
gremlind is a binary installed on the operating system or available inside the Gremlin container. It heartbeats with the Gremlin Control Plane to let Gremlin know that the host is active and able to receive attack orders. It only communicates outbound with the Gremlin Control Plane. All traffic is encrypted.
The Gremlin client,
gremlin, refers to the Gremlin command line interface that is responsible for creating the local impact within the host.
The daemon bundled with the command line interface as a unit is referred to as a targetable Client to the platform.
Gremlin clients (infrastructure and application) that have been authenticated to the Gremlin Control Plane appear in the infrastructure clients and application clients lists. You can only run attacks on “active” clients. A client goes into an “idle” state if there is no activity for the past 5 minutes. You cannot run or schedule attacks on idle clients. If Gremlin does not hear from these idle clients for a period of 24 hours, the clients are removed from the list. However, if a client starts communicating with Gremlin again while still within the 24 hour idle window, the client is reactivated and returned to the “active” state.
Logs can be found under the
Daemon log entries can be found in the
daemon.log file. Log entries in this file may indicate events where the daemon is not able to communicate with the Control Plane and eventually trigger the Dead Man Switch.
Each attack on the host is logged under
/var/log/gremlin/executions using its unique attack execution ID.
The daemon uses very little bandwidth in its idle state. In testing over a 15 minute period the daemon used only an average of 1.15 KB/sec over any given 10 seconds.
1Inbound bandwidth average, zeros dropped21.9891882997434Outbound bandwidth average, zeros dropped50.5562149567Aggregate bandwidth average, zeros dropped82.51990083564910Aggregate bandwidth average over testing period111.15347573462
While testing, there is a slight increase in overall bandwidth consumption during attacks. While attacks are being executed, the daemon stays in constant communication with the control plane as it checks for the abort condition to be executed. Regardless of attack being run, the attack state behavior looks the same. Provided are two data sets, one from a CPU Resource attack, and one from a Latency Network attack.
1Inbound bandwidth average, zeros dropped22.7123369798434Outbound bandwidth average, zeros dropped50.8703485871667Aggregate bandwidth average, zeros dropped83.56685480642910Aggregate bandwidth average over testing period113.0556056175
1Inbound bandwidth average, zeros dropped22.6494441615434Outbound bandwidth average, zeros dropped50.91796346940767Aggregate bandwidth average, zeros dropped83.55221005449910Aggregate bandwidth average over testing period113.11712392366
There is no statistically significant difference between the two attacks. Despite a nearly 3X rise in aggregate average traffic, the 3 KB/sec bandwidth utilization is still very low on a per client basis.