Gremlin’s “Failure as a Service” makes it easy to find weaknesses in your system before they cause problems for your customers. Gremlin is a simple, safe and secure way to use Chaos Engineering to improve system resilience. This tutorial will provide a walkthrough of the following:
- How to setup an instance using GCE
- How to install Gremlin on the instance
- How to create a CPU Attack using the Gremlin Control Panel
Prerequisites Before you begin this tutorial, you’ll need the following:
- A Google account for Google Cloud access
- A Gremlin Account (sign up here)
In this step, you’ll use GCE to setup a new instance to use with Gremlin. Google Compute Engine (GCE) lets you use virtual machines that run on Google's infrastructure.
First, log in to Google Cloud Console and proceed to Compute Engine, create a project to continue. Now let’s create the instance we need, name it as you please and use the region that is closest to you. For Machine Type, we’ve selected 4 vCPUS / 15 GB Memory and for Boot disk select the ‘Ubuntu 16.04 LTS’ image with standard persistent disk of 10GB.
After this, press the ‘Create’ button to create the instance.
While on the 'VM Instances' tab, find the instance you’ve just created, click the SSH button and a pop-up window will open. We are now connected to the server you’ve created and can now install Gremlin!
In this step, you'll install Gremlin
First, ssh into your host and add the gremlin repo:
ssh username@your_server_ip echo "deb https://deb.gremlin.com/ release non-free" | sudo tee /etc/apt/sources.list.d/gremlin.list
Import the GPG key:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C81FC2F43A48B25808F9583BDFF170F324D41134 9CDB294B29A5B1E2E00C24C022E8EF3461A50EF6
Then install the Gremlin client and daemon:
sudo apt-get update && sudo apt-get install -y gremlin gremlind
After you have created your Gremlin account (sign up here) you will need to find your Gremlin Daemon credentials. Login to the Gremlin App using your Company name and sign-on credentials. These were emailed to you when you signed up to start using Gremlin.
Navigate to Team Settings and click on your Team. Click the blue Download button to save your certificates to your local computer. The downloaded certificate.zip contains both a public-key certificate and a matching private key.
Unzip the downloaded certificate.zip on your laptop and copy the files to the server you will be using with a Linux file transfer tool such as rsync, sftp or scp. Alternatively, you can store these certificates in a storage service such as AWS S3. For example:
rsync -avz /Users/tammybutow/Desktop/tammy-client.pub_cert.pem email@example.com:/var/lib/gremlin
rsync -avz /Users/tammybutow/Desktop/tammy-client.priv_key.pem firstname.lastname@example.org:/var/lib/gremlin
**Creating a gremlind file for your environment variables **
Next create the /etc/default/gremlind file:
sudo vim /etc/default/gremlind
Add your GREMLIN environment variables to the file, for example:
Save the file. Restart the service:
sudo service gremlind restart
Confirming your gremlind configuration
Take a look at /var/log/gremlin/daemon.log to confirm:
You should see an output similar to below if it was successful:
2018-10-31 02:34:20 - Logging successfully initialized2018-10-31 02:34:23 - Using Team ID : 3f242793-018a-5ad5-9211-fb958f8dc0842018-10-31 02:34:23 - Using Identifier : 18.104.22.1682018-10-31 02:34:23 - Found GREMLIN_TEAM_CERTIFICATE_OR_FILE in file:///var/lib/gremlin/tammy-client.pub_cert.pem2018-10-31 02:34:23 - Found GREMLIN_TEAM_PRIVATE_KEY_OR_FILE in file:///var/lib/gremlin/tammy-client.priv_key.pem
We are going to use the Gremlin Control Panel to run a CPU Attack, you can also use the API or the CLI to do so. There are various attacks of attacks you can run using Gremlin, check out the list here. The CPU attack is the ‘Hello World’ of Chaos Engineering, it generates high load for one or more CPU cores. For “Choose the Targets”, we will select ‘Host’ and use 10.142.0.2.
We now have to Choose a Gremlin, the CPU attack is found under the “Resource” Gremlins. I’ve filled out the form for the attack to run on one core for 60 seconds.
Make sure to press the green ‘Unleash Gremlin’ button to unleash your first gremlin! While your attack is running you can run this command to see the gremlin attack in action:
On the top, these processes displayed:
8160 gremlin 20 0 37736 9856 6180 S 99.7 0.1 0:16.30 gremlin 2682 gremlin 20 0 44720 11936 5632 S 0.7 0.1 0:16.68 gremlind
You've now set up an instance using GCE, installed Gremlin and unleashed your first Gremlin. Feel free to explore around using the other gremlins, you can learn about the full range of gremlins available at Gremlin Attacks.