Deploying Failure Flags on Pivotal Cloud Foundry (PCF)

Failure Flags lets you run Gremlin experiments on applications running on Cloud Foundry. This includes adding latency to network calls, generating exceptions, and triggering custom behaviors.

Setting environment variables

You will need to provide several secrets to Gremlin, such as your team ID and certificates for authentication. The easiest method is to create a new vars.yml file containing your variables. Configuration comes in via environment variables and or configuration files.

1. GREMLIN_SIDECAR_ENABLED must be set to either true or yes or 1 to enable Failure-Flags-Sidecar. If unset or set to any other value Failure-Flags-Sidecar will operate in NOOP mode.
2. GREMLIN_TEAM_ID must be set to your Gremlin Team ID. This and other credential material is available through the Gremlin UI.
3. GREMLIN_TEAM_CERTIFICATE must be set to your Gremlin Team certificate. Newlines may be preserved using the \n escape characters or omited entirely. This and other credential material is available through the Gremlin UI.
4. GREMLIN_TEAM_PRIVATE_KEY must be set to your Gremlin Team private key. Newlines may be preserved using the \n escape characters or omited entirely. This and other credential material is available through the Gremlin UI.
5. SERVICE_NAME must be set to the name of the service as you want it to appear in Gremlin. Service names can only contain alphanumeric characters, hyphens, and underscores, and must be less than 64 characters long.

The easiest method is to create a new vars.yml file containing these variables and their values.

‍

‍

gremlin_team_id: "YOUR_GREMLIN_TEAM_ID"
gremlin_team_certificate: |-
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----
gremlin_team_private_key: |-
  -----BEGIN EC PRIVATE KEY-----
  ...
  -----END EC PRIVATE KEY-----
...

‍

Configuring your deployment manifest

Failure Flags runs on Cloud Foundry using a sidecar. However, unlike the regular Failure Flags sidecar, PCF sidecars run in the same container as your application. To accomplish this, you must download and extract the Failure Flags sidecar to your project folder.

First, download and extract the archive from one of these links, depending on your architecture: arm64 or x86_64.

Next, add the Gremlin sidecar to your deployment manifest. This example shows how to bundle Gremlin with a Python Flask application, including necessary environment variables. You can adjust the memory a

‍

---
applications:
  - name: python-flask-app                # App name in CF
    memory: 512M                          # Memory for the container
    disk_quota: 1G                        # Disk space for the container
    instances: 1                          # Number of instances

    buildpacks:
      - python_buildpack                  # Use Python buildpack for Flask

    path: .                               # Push contents of current directory

    command: null                         # Let the buildpack detect and use your Procfile

    env:
      DEBUG_MODE: "false"
      FAILURE_FLAGS_ENABLED: "true"
      GREMLIN_SIDECAR_ENABLED: "true"
      GREMLIN_TEAM_ID: ((gremlin_team_id))
      GREMLIN_TEAM_CERTIFICATE: ((gremlin_team_certificate))
      GREMLIN_TEAM_PRIVATE_KEY: ((gremlin_team_private_key))
      SERVICE_NAME: "python-flask-app"

    sidecars:
      - name: gremlin-sidecar
        process_types: ["web"]            # Attach sidecar to 'web' process
        memory: 256M
        disk_quota: 256M
        command: "./bin/failure-flags-sidecar-amd64-linux"  # Run Gremlin sidecar

‍

Finally, push your application to Cloud Foundry:

‍

cf push --vars-file vars.yaml