This guide explains how to deploy the Failure Flags sidecar alongside an application on Azure Container Apps. Each replica runs your application container and the gremlin/failure-flags-sidecar container together.
The configuration below uses Failure Flags by proxy. Inbound requests enter through the sidecar before reaching your application. Outbound HTTP and HTTPS requests pass through the sidecar's dependency proxy, allowing Gremlin to inject faults into calls to downstream services.
Before you begin, you need Azure CLI, access to an Azure subscription, a resource group and Container Apps environment, and your Gremlin team credentials. If you need to create the Azure resources, follow the Azure Container Apps quickstart.
Configuring Failure Flags for Azure Container Apps
Configure the sidecar with the common Failure Flags settings and the proxy settings below. Put sidecar settings on the Gremlin container and application proxy settings on your application container.
Common and credential options (sidecar container)
| Environment variable
|
Description
|
GREMLIN_SIDECAR_ENABLED
|
Use true to activate the sidecar. When disabled or unset, it runs in NOOP mode.
|
GREMLIN_TEAM_ID
|
UUID of the team that will own this application. Get it from Gremlin team settings.
|
GREMLIN_TEAM_CERTIFICATE
|
Your team's complete PEM certificate. The example supplies it through a Container App secret.
|
GREMLIN_TEAM_PRIVATE_KEY
|
The matching PEM private key, also supplied through a Container App secret.
|
SERVICE_NAME
|
Application name to display in Gremlin. Use fewer than 64 characters: letters, numbers, hyphens, or underscores.
|
GREMLIN_DEBUG
|
Optional. Use true to enable debug logging.
|
GREMLIN_TRACE
|
Optional. Use true for additional details about the sidecar's network requests.
|
The certificate and private key are sensitive. The example stores them in Azure Container App secrets and connects them to the sidecar using secretRef. Keep any deployment file containing their values private and out of source control.
Ingress proxy options (sidecar container)
The ingress proxy handles requests arriving at your application. Configure Azure ingress to send traffic to the proxy's listening port.
| Environment variable
|
Description
|
GREMLIN_INGRESS_PROXY_ENABLED
|
Use true to turn on the ingress proxy.
|
GREMLIN_INGRESS_PROXY_PORT
|
Listening address for inbound requests. The example uses :5035 and sets Azure's configuration.ingress.targetPort to 5035.
|
GREMLIN_INGRESS_PROXIED_ENDPOINT
|
Required upstream URL for your application: http://localhost:<APPLICATION_PORT>. Replace the placeholder with its HTTP listening port.
|
Dependency proxy options (sidecar container)
The dependency proxy handles outbound requests from your application to other HTTP or HTTPS services.
| Environment variable
|
Description
|
GREMLIN_DEPENDENCY_PROXY_ENABLED
|
Use true to turn on the dependency proxy.
|
GREMLIN_DEPENDENCY_PROXY_PORT
|
Listening address for outbound proxy connections. The example uses localhost:5034.
|
Application container options
Configure your application's HTTP client to send outbound requests through the dependency proxy. The example includes both uppercase and lowercase environment variable names for clients that recognize different forms.
| Environment variable
|
Description
|
HTTP_PROXY / http_proxy
|
Proxy URL for outbound HTTP requests: http://localhost:5034.
|
HTTPS_PROXY / https_proxy
|
Proxy URL for outbound HTTPS requests: http://localhost:5034.
|
NO_PROXY / no_proxy
|
Addresses that bypass the proxy. Include localhost,127.0.0.1 and any additional exclusions your application needs.
|
Some HTTP clients require explicit proxy configuration and do not read these environment variables automatically. Check your runtime and library settings, and keep dependencies you want to test out of the proxy exclusion list. See Troubleshooting Failure Flags.
Sidecar tuning options
| Environment variable
|
Description
|
GOMEMLIMIT
|
Optional Go runtime soft memory limit for the sidecar. Choose a value below its container memory limit to leave room for other process memory.
|
Size the application and sidecar together. The example allocates 0.5 CPU and 1Gi to the application, plus 0.25 CPU and 0.5Gi to the sidecar. The combined 0.75 CPU and 1.5Gi allocation is supported by the Consumption plan. See Azure's resource requirements before changing these values.
The example keeps one replica running. Adjust its resource and scaling settings for your workload.
Adding the sidecar to your Azure Container App
Add the Gremlin sidecar to the properties.template.containers list alongside your application. Containers in the same replica share networking, so both proxies can reach the application over localhost.
Prepare your deployment values
Replace every <PLACEHOLDER> in the commands and YAML with your own value, including the angle brackets. Keep the surrounding quotes.
| Placeholder
|
Value to provide
|
<AZURE_SUBSCRIPTION_ID>
|
ID of the subscription where you will deploy the Container App.
|
<AZURE_RESOURCE_GROUP>
|
Name of the resource group containing your Container App.
|
<AZURE_REGION>
|
Azure region identifier matching your Container Apps environment.
|
<CONTAINER_APPS_ENVIRONMENT_NAME>
|
Name of the existing Container Apps environment.
|
<CONTAINER_APPS_ENVIRONMENT_RESOURCE_ID>
|
Full Azure resource ID returned by the environment lookup below.
|
<CONTAINER_APP_NAME>
|
Name of the Container App resource to create or update.
|
<APPLICATION_CONTAINER_NAME>
|
Name of your application container within that resource.
|
<APPLICATION_IMAGE>
|
Your application image reference, including registry and tag or digest.
|
<APPLICATION_PORT>
|
Your application's HTTP listening port. Choose a port other than 5034 or 5035.
|
<APPLICATION_TEST_PATH>
|
Safe HTTP endpoint path used to verify your application, including its leading /.
|
<SIDECAR_IMAGE_TAG>
|
Available gremlin/failure-flags-sidecar image tag selected for your deployment.
|
<GREMLIN_TEAM_ID>
|
UUID of your Gremlin team.
|
<GREMLIN_TEAM_CERTIFICATE_PEM>
|
Full PEM certificate, including its BEGIN and END lines.
|
<GREMLIN_TEAM_PRIVATE_KEY_PEM>
|
Full matching PEM private key, including its BEGIN and END lines.
|
<GREMLIN_SERVICE_NAME>
|
Application name to display in Gremlin. It may differ from the Azure Container App name.
|
Sign in to Azure, select your subscription, and install or update the Container Apps CLI extension:
az login
az account set --subscription "<AZURE_SUBSCRIPTION_ID>"
az extension add --name containerapp --upgrade --yes
Get the environment resource ID to use in the YAML. If your environment belongs to a different resource group, use that group's name for this lookup.
az containerapp env show \
--resource-group "<AZURE_RESOURCE_GROUP>" \
--name "<CONTAINER_APPS_ENVIRONMENT_NAME>" \
--query id --output tsv
Define the application and sidecar
Save the following YAML as containerapp.yaml. Replace each PEM placeholder with the complete multiline value, indenting every line beneath value: |.
For an existing app, merge the sidecar and proxy settings into its complete manifest. Preserve its startup command, environment variables, registry access, volumes, probes, and other required configuration. The example uses your image's default startup command.
location: "<AZURE_REGION>"
resourceGroup: "<AZURE_RESOURCE_GROUP>"
type: Microsoft.App/containerApps
name: "<CONTAINER_APP_NAME>"
properties:
environmentId: "<CONTAINER_APPS_ENVIRONMENT_RESOURCE_ID>"
configuration:
activeRevisionsMode: Single
ingress:
external: true
targetPort: 5035
transport: auto
allowInsecure: false
secrets:
- name: gremlin-team-cert
value: |
<GREMLIN_TEAM_CERTIFICATE_PEM>
- name: gremlin-team-key
value: |
<GREMLIN_TEAM_PRIVATE_KEY_PEM>
template:
containers:
- name: "<APPLICATION_CONTAINER_NAME>"
image: "<APPLICATION_IMAGE>"
env:
- name: HTTP_PROXY
value: "http://localhost:5034"
- name: HTTPS_PROXY
value: "http://localhost:5034"
- name: NO_PROXY
value: "localhost,127.0.0.1"
- name: http_proxy
value: "http://localhost:5034"
- name: https_proxy
value: "http://localhost:5034"
- name: no_proxy
value: "localhost,127.0.0.1"
resources:
cpu: 0.5
memory: 1Gi
- name: gremlin-sidecar
image: "gremlin/failure-flags-sidecar:<SIDECAR_IMAGE_TAG>"
env:
- name: GREMLIN_SIDECAR_ENABLED
value: "true"
- name: GREMLIN_TEAM_ID
value: "<GREMLIN_TEAM_ID>"
- name: GREMLIN_TEAM_CERTIFICATE
secretRef: gremlin-team-cert
- name: GREMLIN_TEAM_PRIVATE_KEY
secretRef: gremlin-team-key
- name: SERVICE_NAME
value: "<GREMLIN_SERVICE_NAME>"
- name: GREMLIN_INGRESS_PROXY_ENABLED
value: "true"
- name: GREMLIN_INGRESS_PROXY_PORT
value: ":5035"
- name: GREMLIN_INGRESS_PROXIED_ENDPOINT
value: "http://localhost:<APPLICATION_PORT>"
- name: GREMLIN_DEPENDENCY_PROXY_ENABLED
value: "true"
- name: GREMLIN_DEPENDENCY_PROXY_PORT
value: "localhost:5034"
resources:
cpu: 0.25
memory: 0.5Gi
scale:
minReplicas: 1
maxReplicas: 1
The example uses these routing settings:
- Inbound traffic: Azure ingress targets port
5035. The sidecar forwards requests to http://localhost:<APPLICATION_PORT>. - Outbound traffic: Your application's proxy variables point to
http://localhost:5034. Localhost requests bypass this proxy.
The example enables public HTTPS ingress. Set external: false for internal ingress and verify the app from a client with network access. If your application image is private, add its registry authentication settings before deploying.
The names gremlin-sidecar, gremlin-team-cert, and gremlin-team-key are local container and secret labels. You can rename them; update the corresponding secretRef values and log commands as well.
Deploy the Container App
Use the same app and resource group values in your YAML and CLI commands. For a new Container App, run:
az containerapp create \
--resource-group "<AZURE_RESOURCE_GROUP>" \
--name "<CONTAINER_APP_NAME>" \
--yaml containerapp.yaml
For an existing Container App, apply your merged manifest with:
az containerapp update \
--resource-group "<AZURE_RESOURCE_GROUP>" \
--name "<CONTAINER_APP_NAME>" \
--yaml containerapp.yaml
Verify the deployment
Check that Azure reports successful provisioning, a ready revision, and ingress port 5035:
az containerapp show \
--resource-group "<AZURE_RESOURCE_GROUP>" \
--name "<CONTAINER_APP_NAME>" \
--query '{state:properties.provisioningState,readyRevision:properties.latestReadyRevisionName,targetPort:properties.configuration.ingress.targetPort}' \
--output json
Send a request to your application. Replace <APPLICATION_TEST_PATH> with a safe endpoint path, including its leading /, and add authentication if your app requires it.
ACA_FQDN="$(az containerapp show \
--resource-group "<AZURE_RESOURCE_GROUP>" \
--name "<CONTAINER_APP_NAME>" \
--query properties.configuration.ingress.fqdn --output tsv)"
curl --fail --silent --show-error --max-time 15 \
"https://${ACA_FQDN}<APPLICATION_TEST_PATH>"
Confirm the expected response, then trigger an application operation that calls an HTTP or HTTPS dependency. Open Failure Flags, select your team, and find the name supplied as <GREMLIN_SERVICE_NAME>. After traffic passes through the proxies, check for the http-ingress and egress flags and selectors matching your requests.
Troubleshooting Failure Flags
View the sidecar's logs to check startup, registration, and connection errors:
az containerapp logs show \
--resource-group "<AZURE_RESOURCE_GROUP>" \
--name "<CONTAINER_APP_NAME>" \
--container gremlin-sidecar --tail 100
For additional detail, add GREMLIN_DEBUG with the string value "true" to the sidecar's environment variables and deploy the updated manifest. Enable GREMLIN_TRACE when you also need network request diagnostics. Return these settings to their previous values after troubleshooting.
If the sidecar registers but traffic does not appear in Gremlin, check the Azure ingress target, the upstream application port, and your HTTP client's proxy configuration. If registration fails, check the selected team, certificate and private key, secret references, and connectivity to Gremlin. If a revision fails to start, inspect both containers' logs and verify image access, resource allocation, and startup settings.
For more help, see Troubleshooting Failure Flags.