Search documentation
Getting Started

Configuring the Gremlin Agent

This documentation page shows you how to configure the Gremlin Agent. You can configure Gremlin using either environment variables or a configuration file, although we recommend using the configuration file in most cases. This guide assumes you've already installed and authenticated the Agent.

Using the configuration file

Gremlin's configuration file is located in a file named config.yaml. On Linux, this file lives at /etc/gremlin/config.yaml, and on Windows, its path is C:\ProgramData\Gremlin\Agent\config.yaml. By default, Gremlin ships with a commented-out version of this file. Use the comments or the following example to set up your configuration.

Accepted variables

identifiera string that uniquely identifies a Gremlin installation (defaults to the hosts IP address)
team_idthe ID of your Gremlin team
tagskey-value pairs that help you target this machine during experiments
team_secretthe secret value (used by secret authentication only)
team_certificatethe public portion of the certificate-pair (required for signature auth only)
team_private_keythe private portion of the certificate-pair (required for signature auth only)
https_proxythe url of a proxy in between Gremlin and
ssl_cert_filethe ssl certificate for your https proxy
push_metricsset to false if you would like to disable system metrics used for experiment charting (defaults to true)
collect_processesenables/disables Process Collection (as of Linux Agent 2.25.0 default is true)

Example: config.yaml

1## Gremlin Identifier; uniquely identifies this machine with Gremlin
2## (can also set with GREMLIN_IDENTIFIER environment variable)
3identifier: gremlin-01
5## Gremlin Team Id; you can find this value at
6## (can also be set with GREMLIN_TEAM_ID environment variable)
7team_id: 11111111-1111-1111-1111-111111111111
9## Gremlin Client Tags; Tag your machine with key-value pairs that help you target this machine during experiments
10## (can also set with GREMLIN_CLIENT_TAGS environment variable)
12 service: pet-store
13 interface: http
15## Gremlin Team Secret, should not be set when using `team_certificate`+`team_private_key`
16## (can also set with GREMLIN_TEAM_SECRET environment variable)
17#team_secret: 11111111-1111-1111-1111-111111111111
19## Gremlin Team Certificate, should not be set when using `team_secret`.
20## Paste certificate content here or a paths to the file (prefixed with `file://`)
21## (can also set with GREMLIN_TEAM_CERTIFICATE_OR_FILE environment variable)
22team_certificate: |
24 1111111111111111111111111111111111111111111111111111111111111111
25 1111111111111111111111111111111111111111111111111111111111111111
26 1111111111111111111111111111111111111111111111111111111111111111
27 1111111111111111111111111111111111111111111111111111111111111111
28 1111111111111111111111111111111111111111111111111111111111111111
29 1111111111111111111111111111111111111111111111111111111111111111
30 1111111111111111111111111111111111111111111111111111111111111111
31 1111111111111111111111111111111111111111111111111111111111111111
32 1111111111111111111111111111111111111111111111111111111111111111
33 11111111111111111111111111111111111111111111111111==
34 -----END CERTIFICATE-----
36## Gremlin Team Certificate, should not be set when using `team_secret`.
37## Paste certificate content here or a paths to the file (prefixed with `file://`)
38## (can also set with GREMLIN_TEAM_PRIVATE_KEY_OR_FILE environment variable)
39team_private_key: file:///var/lib/gremlin/key.pem
41## HTTPS Proxy, set this when routing outbound Gremlin HTTPS traffic through a proxy
42## (can also set with HTTPS_PROXY or https_proxy environment variables)
43https_proxy: https://localhost:3128
45## SSL CERT FILE, set this when using a https proxy with a self-signed certificate
46## Paste certificate content here or a paths to the file (prefixed with `file://`)
47## (can also set with SSL_CERT_FILE environment variable)
48ssl_cert_file: file:///var/lib/gremlin/proxy_cert.pem
50## Push Metrics, tell Gremlin whether to send system metrics to the control plane for charting the impact of experiments in
51## real time. Metrics are only collected during active experiments, and only metrics relevant to the experiment are collected.
52## defaults to `true`
53## (can also set with PUSH_METRICS environment variable)
54push_metrics: true
56## Collect Process Data, data about running processes is sent to Gremlin for service discovery.
57## defaults to `true` as of Gremlin Linux Agent 2.25.0
58collect_processes: true

Restarting the Gremlin Agent

To restart the Gremlin Agent and apply your new changes, run the following commands (depending on your environment):

1sudo systemctl restart gremlind
1Restart-Service -Name gremlind

Using environment variables

The daemon supports the following environment variables:

Environment VariableDescription
GREMLIN_TEAM_IDYour Team ID (required for authentication)
GREMLIN_TEAM_SECRETYour Team Secret (should only require secret or PEM certificates, not both)
GREMLIN_TEAM_PRIVATE_KEY_OR_FILEYour PEM-encoded private key or path to a file containing the private key (required for authentication). Paths should be prefixed with file:// . For example, on Unix: file:///var/lib/gremlin/key.pem , on Windows: file://C:/ProgramData/Gremlin/Agent/key.pem
GREMLIN_TEAM_CERTIFICATE_OR_FILEThe PEM-encoded public-key certificate or path/filename to the file containing your PEM-encoded public-key certificate (required for authentication). Paths should be prefixed with file://. For example, on Unix: file:///var/lib/gremlin/cert.pem , on Windows: file://C:/ProgramData/Gremlin/Agent/cert.pem
GREMLIN_IDENTIFIERCustom name for this client (default as the host's IP address)
GREMLIN_CLIENT_TAGSComma-separated list of custom tags to assign to this client. For example: GREMLIN_CLIENT_TAGS="zone=us-east1,role=mysql,foo=bar"

The following are optional environment variables that may be required in Docker containerized deployments:

Environment VariableDescription
GREMLIN_BYPASS_USERNS_REMAPIndicates to run the Gremlin Daemon on the same Docker namespace as the host. By default, Gremlin does not launch with a namespace defined. To set this variable, change the value to 1
GREMLIN_DOCKER_IMAGEIndicates the location and version of the Gremlin Docker image to use. This variable can be used to indicate an internal repository. For example:
net=hostIndicates to run the Gremlin Daemon to use the host network. By default, Gremlin network experiments will not affect the host network for containerized environments.
pid=hostIndicates to run the Gremlin Daemon to affect host processes. By default, Gremlin process killer will not affect host processes for containerized environments.

In addition, the following standard Linux environment variables allow proxy configuration:

Environment VariableDescription
https_proxyIn the form http[s]://[username:password@]address:port
SSL_CERT_FILEIndicates location of the proxy certificates used to authenticate traffic with the proxy. For Example: SSL_CERT_FILE=/etc/ssl/certs/ca-example.crt

Configuration value priorities

If both the environment variables and the configuration file are used when configuring Gremlin, the environment variable values will take priority. For example if you have the GREMLIN_IDENTIFIER environment variable set to foo and in the config.yaml file the identifier field is set to bar, Gremlin will end up using foo as its identifier when registering with the API.

Passing environment variables to Systemd or SysV

While configuring the daemon with config.yaml is recommended, process managers like systemd and sysvinit can pass environment variables directly to the daemon process from /etc/default/gremlind. Your installation comes with an example of this file at /etc/default/gremlind.example

1# Gremlin Identifier; uniquely identifies this machine with Gremlin
4# Gremlin Team ID; you can find this value at
7# `gremlin init` will run automatically when the gremlind service starts.
8# Supply extra options to `gremlin init` via this variable
9GREMLIN_INIT_OPTS="--tag service=pet-store --tag interface=http"
11# Gremlin Team Secret, should not be set when using `team_certificate`+`team_private_key`

You can set any of the environment variables listed in the previous section in the configuration file.


The Gremlin Agent automatically detects tags from the host system, such as operating system, zone, and local hostname. Gremlin also detects additional tags from certain cloud providers, including Amazon Web Services and Azure. You can also create custom tags and assign them to your Gremlin Agent(s) during installation. See Network Tags for instructions on adding custom tags to Gremlin.

Process Collection

In order to run dependency tests using Gremlin Reliability Management (RM), the Gremlin agent must be configured to collect process data by enabling Process Collection. When Process Collection is enabled, the Gremlin Agent collects process information from hosts, containers, and Kubernetes objects, and uses it to detect dependencies.

To learn more about enabling or disabling process collection, see Process Collection.