How to Install and Use Gremlin on Google Compute Engine with Ubuntu 16.04

Introduction

Gremlin’s “Failure as a Service” makes it easy to find weaknesses in your system before they cause problems for your customers. Gremlin is a simple, safe and secure way to use Chaos Engineering to improve system resilience. This tutorial will provide a walkthrough of the following:

  • How to setup an instance using GCE
  • How to install Gremlin on the instance
  • How to create a CPU Attack using the Gremlin Control Panel

Prerequisites Before you begin this tutorial, you’ll need the following:

  • A Google account for Google Cloud access
  • A Gremlin Account (sign up here)

Step 1 - Create new Instance using GCE

In this step, you’ll use GCE to setup a new instance to use with Gremlin. Google Compute Engine (GCE) lets you use virtual machines that run on Google's infrastructure.

First, log in to Google Cloud Console and proceed to Compute Engine, create a project to continue. Now let’s create the instance we need, name it as you please and use the region that is closest to you. For Machine Type, we’ve selected 4 vCPUS / 15 GB Memory and for Boot disk select the ‘Ubuntu 16.04 LTS’ image with standard persistent disk of 10GB.

create instance

After this, press the ‘Create’ button to create the instance.

Step 2 - Connect to your instance using GCE tools

While on the 'VM Instances' tab, find the instance you’ve just created, click the SSH button and a pop-up window will open. We are now connected to the server you’ve created and can now install Gremlin!

Step 3 - Installing Gremlin

In this step, you'll install Gremlin

First, ssh into your host and add the gremlin repo:

ssh username@your_server_ip

echo "deb https://deb.gremlin.com/ release non-free" | sudo tee /etc/apt/sources.list.d/gremlin.list

Import the GPG key:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C81FC2F43A48B25808F9583BDFF170F324D41134 9CDB294B29A5B1E2E00C24C022E8EF3461A50EF6

Then install the Gremlin client and daemon:

sudo apt-get update && sudo apt-get install -y gremlin gremlind

Step 4 - Downloading your Gremlin client certificates

After you have created your Gremlin account (sign up here) you will need to find your Gremlin Daemon credentials. Login to the Gremlin App using your Company name and sign-on credentials. These were emailed to you when you signed up to start using Gremlin.

Navigate to Team Settings and click on your Team. Click the blue Download button to save your certificates to your local computer. The downloaded certificate.zip contains both a public-key certificate and a matching private key.

certificates

Unzip the downloaded certificate.zip on your laptop and copy the files to the server you will be using with a Linux file transfer tool such as rsync, sftp or scp. Alternatively, you can store these certificates in a storage service such as AWS S3. For example:

rsync -avz /Users/tammybutow/Desktop/tammy-client.pub_cert.pem tammy@142.93.31.189:/var/lib/gremlin
rsync -avz /Users/tammybutow/Desktop/tammy-client.priv_key.pem tammy@142.93.31.189:/var/lib/gremlin

**Creating a gremlind file for your environment variables **

Next create the /etc/default/gremlind file:

sudo vim /etc/default/gremlind

Add your GREMLIN environment variables to the file, for example:

GREMLIN_TEAM_ID="3f242793-018a-5ad5-9211-fb958f8dc084"GREMLIN_TEAM_CERTIFICATE_OR_FILE="file:///var/lib/gremlin/tammy-client.pub_cert.pem"GREMLIN_TEAM_PRIVATE_KEY_OR_FILE="file:///var/lib/gremlin/tammy-client.priv_key.pem"GREMLIN_CLIENT_TAGS="service=prometheus"

Save the file. Restart the service:

sudo service gremlind restart

Confirming your gremlind configuration

Take a look at /var/log/gremlin/daemon.log to confirm:

tail /var/log/gremlin/daemon.log

You should see an output similar to below if it was successful:

2018-10-31 02:34:20 - Logging successfully initialized2018-10-31 02:34:23 - Using Team ID : 3f242793-018a-5ad5-9211-fb958f8dc0842018-10-31 02:34:23 - Using Identifier : 142.93.31.1892018-10-31 02:34:23 - Found GREMLIN_TEAM_CERTIFICATE_OR_FILE in file:///var/lib/gremlin/tammy-client.pub_cert.pem2018-10-31 02:34:23 - Found GREMLIN_TEAM_PRIVATE_KEY_OR_FILE in file:///var/lib/gremlin/tammy-client.priv_key.pem

Step 5 - Run a CPU Attack

We are going to use the Gremlin Control Panel to run a CPU Attack, you can also use the API or the CLI to do so. There are various attacks of attacks you can run using Gremlin, check out the list here. The CPU attack is the ‘Hello World’ of Chaos Engineering, it generates high load for one or more CPU cores. For “Choose the Targets”, we will select ‘Host’ and use 10.142.0.2. alt text

We now have to Choose a Gremlin, the CPU attack is found under the “Resource” Gremlins. I’ve filled out the form for the attack to run on one core for 60 seconds. alt text

Make sure to press the green ‘Unleash Gremlin’ button to unleash your first gremlin! While your attack is running you can run this command to see the gremlin attack in action:

top

On the top, these processes displayed:

8160 gremlin   20   0   37736   9856   6180 S  99.7  0.1   0:16.30 gremlin                                                        
2682 gremlin   20   0   44720  11936   5632 S   0.7  0.1   0:16.68 gremlind

Conclusion

You've now set up an instance using GCE, installed Gremlin and unleashed your first Gremlin. Feel free to explore around using the other gremlins, you can learn about the full range of gremlins available at Gremlin Attacks.

Avoid downtime. Use Gremlin to turn failure into resilience.

Gremlin empowers you to proactively root out failure before it causes downtime. Try Gremlin for free and see how you can harness chaos to build resilient systems.