Installing Gremlin on Kubernetes with Helm
Overview
You can install the Gremlin Kubernetes client in addition to installing the Gremlin client on a host — both are required to run attacks on Kubernetes. You can install the Kubernetes client using either helm
or kubectl
(manually). The simplest way to install Gremlin on Kubernetes is with Helm. Check out
Gremlin's Helm chart repository for full documentation and usage. To install Gremlin manually, from YAML files and a text editor, see Install Gremlin on Kubernetes manually.
General steps for deploying to Kubernetes with Helm:
Gather your credentials
All Gremlin integration installations require authentication with the Gremlin Control Plane. You can use either secret authentication or signature authentication (certificates). See Authentication to learn more about the two methods.
When using signature-based authentication, you can store your certificates as a Kubernetes secret. For details, see Managed Secrets in the full Helm chart instructions.
For the Gremlin Helm chart, you'll need to download your Team ID with team secret or team certificates from the Gremlin web app.
Deploy the Helm chart
These instructions are for Helm version 3.
1helm repo add gremlin https://helm.gremlin.com/2kubectl create namespace gremlin3helm install gremlin gremlin/gremlin --namespace gremlin \4 --set gremlin.hostPID=true \5 --set gremlin.container.driver=any \6 --set gremlin.secret.managed=true \7 --set gremlin.secret.type=secret \8 --set gremlin.secret.teamID=$GREMLIN_TEAM_ID \9 --set gremlin.secret.clusterID=$GREMLIN_CLUSTER_ID \10 --set gremlin.secret.teamSecret=$GREMLIN_TEAM_SECRET
Setting `gremlin.collect.processes=true` enables Gremlin’s Services Discovery.
Additional configuration
Some environments require additional configuration. Review the following sections to find the best configuration for your environment.
- Declare container driver
- Enable Gremlin on the Kubernetes Master
- Add AppArmor support
- Use a PodSecurityPolicy
- Use a seccomp policy
- Configure a proxy
- Share namespaces with other Gremlin teams
Verify your installation
Now it’s time to check that the Gremlin agent was installed correctly and has successfully connected to the Gremlin Control Plane.
After that, you’re all set to start running attacks on Kubernetes objects!
Check Gremlin agents
Run the following command to check that the Gremlin agents were installed properly:
1kubectl get pods -n gremlin
This should list one Gremlin agent per node (physical/virtual machine in your cluster) plus one for chao
.
Consider the following example, where Gremlin was installed correctly on 3 nodes:
1kubectl get pods -n gremlin23NAME READY STATUS RESTARTS AGE4chao-78bbc7cbf6-9hn7q 1/1 Running 0 5d20h5gremlin-9r4t7 1/1 Running 0 5d20h6gremlin-bwmtz 1/1 Running 1 126d7gremlin-bx6dn 1/1 Running 0 5d20h
The following example shows 2 pending pods, which means the installation is incomplete. Contact your cluster administrator to debug why Gremlin is unable to run on those nodes.
1kubectl get pods -n gremlin23NAME READY STATUS RESTARTS AGE4chao-78bbc7cbf6-9hn7q 1/1 Running 0 5d20h5gremlin-c25ld 0/1 Pending 0 112d6gremlin-n5gt7 0/1 Pending 0 112d7gremlin-zn4kq 1/1 Running 0 126d
Any applications running on nodes Gremlin is not running on cannot successfully execute attacks and those attacks will error out. If Chao is not running you will be unable to target the cluster primitives at all.
Check connection to Gremlin Control Plane
To check the container’s logs and validate the client has successfully connected with the Gremlin Control Plane, run the following command:
kubectl logs gremlin-bwmtz -n gremlin