Eliminate Reliability Blind Spots in AWS, Azure, and GCP

Cloud resilience often feels like an uphill battle. When you’re overseeing hundreds of applications across different providers, identifying potential failure points manually is nearly impossible. You’re left trying to find the needle in a haystack—a needle that could take down your entire application at any moment.

To truly protect your uptime, you have to break the cycle of reactive troubleshooting. This means moving away from post-incident learnings and towards proactive risk detection. And while fault injection is a powerful way of finding hidden failure modes through experimentation, there’s an even faster way: by detecting reliability risks automatically.

We’ve significantly expanded our Detected Risks feature to do just that. Beyond our existing Kubernetes and AWS capabilities, Gremlin now automatically identifies high-priority reliability risks across your Azure and GCP environments. This update lets you eliminate reliability blind spots—like misconfigured deployments or containers stuck in crash loops—before they turn into revenue-impacting outages.

‍

Kubernetes Detected Risks

Gremlin already identifies over a dozen Kubernetes risks, and we’ve added four critical new checks to ensure your orchestration is built for resilience.

‍

No readiness probe defined

Newly provisioned pods need time to “warm up” by downloading dependencies, pre-populating databases, synchronizing files, and opening network connections. Without a readiness probe, Kubernetes can’t distinguish between a pod that’s still warming up and one that’s ready to serve users. This can result in latent or blocked connections when traffic hits a pod that isn’t prepared to handle it. Adding a readiness probe ensures that traffic won’t get sent to the container until it’s ready to process it.

‍

Topology spread constraints absent

During a deployment, Kubernetes uses a scheduling algorithm to find the optimal nodes for your replicas. However, this algorithm doesn’t always prioritize reliability. If a single node has enough capacity for both replicas of a service, Kubernetes might schedule them on that node. This creates a catastrophic single point of failure: if that one node goes down, your entire service vanishes. Topology spread constraints provide the strict rules necessary to distribute pods across failure domains—like nodes, availability zones, and regions—ensuring that a single hardware failure doesn't result in a total service blackout.

‍

PodDisruptionBudget missing

In a production Kubernetes environment, pods and hosts alike frequently start, stop, and migrate. Whether you’re performing cluster upgrades, migrating workloads, or replacing aging nodes, you’ll eventually need to drain pods from a host. Without a PodDisruptionBudget (PDB), Kubernetes treats these operations as individual pod terminations, which can lead to a "thundering herd" effect where too many pods are taken offline simultaneously, dropping your service below its required capacity. Implementing a PDB provides the necessary guardrails, ensuring that your orchestration layer respects your minimum availability requirements during planned maintenance, preventing self-inflicted outages.

‍

minReadySeconds is zero or absent

In the race to lower deployment times, it’s easy to overlook the "settling" period of a container, or the time in between when the container is marked as “ready” and when it’s actually able to receive and process traffic. When minReadySeconds is set to zero or left absent, Kubernetes considers a pod "ready" the very instant it passes its readiness check. However, in complex distributed systems, a container might pass a health check but still be experiencing internal jitter or resource contention as it stabilizes. By defining minReadySeconds, you give your pods time to finish stabilizing before they’re considered eligible to receive traffic. This simple configuration prevents newly provisioned pods from entering the rotation too early and causing intermittent errors for your users.

‍

Azure Detected Risks

Gremlin detects risks in Azure when your service is mapped to an Application Gateway. Application Gateways are the most visible part of your Azure deployment, acting as a “front door” for your services. Reliability gaps here can compromise your entire application’s availability, even if your backend services are perfectly stable.

‍

No availability zone (AZ) redundancy 

Cloud resilience requires failover systems that span not just hosts, but entire zones and regions. Relying on a single availability zone (AZ) creates a glaring reliability blind spot: if a data center experiences a power failure, network outage, or natural disaster, your entire service goes dark. Gremlin identifies environments lacking AZ redundancy to ensure your architecture is geographically distributed. By spreading your workloads across multiple zones, you decouple your service's uptime from the health of any single physical location, providing the high availability that enterprise customers demand. By default, newly created Application Gateways will use all available zones automatically.

‍

Autoscaling missing

A system that can’t grow to meet demand is a system destined to fail under pressure. This is especially true in cloud environments, where traffic spikes are often unpredictable. If your infrastructure lacks configured autoscaling, you either end up over-provisioning (which means wasting budget), or under-provisioning (which means risking a crash). Gremlin flags missing autoscaling rules to help you ensure your Application Gateway can remain performant during peak loads, and cost-effective during lulls. Just remember that Application Gateway v2 can take three to five minutes to scale out, so make sure your minimum instance count is still large enough to handle small traffic jumps.

‍

SSL certificate expiring soon

An expired SSL certificate isn’t just a security vulnerability, but a hard blocker for your users. Most modern browsers and clients refuse to connect to services with invalid certificates, effectively taking your application offline for the public. Despite this, 72% of organizations had at least one certificate-related outage in 2025, and nearly half experience certificate-related outages weekly. These aren’t small organizations, either: Spotify, GitHub, Starlink, Microsoft, and even Google have all suffered high-profile TLS outages.

By proactively flagging expiring certificates, Gremlin helps you shift from responding to expired credentials to a proactive rotation cycle, ensuring that your security posture never becomes a bottleneck for your availability. If your certificates are stored in Key Vault, Application Gateway can automatically rotate certificates by checking for new ones every four hours. Otherwise, you can set up automatic rotations using Azure PowerShell or the Azure CLI.

‍

Google Cloud Platform (GCP) Detected Risks

Gremlin will automatically flag GCP risks if your service is mapped to a Backend Service. Backend services control where and how traffic gets distributed to your instance groups, services, and other network endpoint groups (NEGs).

‍

Single-zone backends

Similar to the risks faced in AWS, locating all of your Backend Service’s NEGs to a single zone introduces unnecessary risk. Gremlin flags single-zone backends in order to encourage a redundant, multi-zone strategy. By distributing your backends across multiple zones, you ensure that your application remains reachable even if a specific zone faces a localized disruption.

‍

Connection draining disabled

In a dynamic environment like the cloud, endpoints are transient—they spin up and down constantly as your system scales. Without connection draining, when a pod or instance is marked for termination (due to a deployment or scale-down event), the system might cut off active user connections immediately. This results in hard failures where users see their connections drop mid-flight. Enabling connection draining allows existing connections to finish gracefully before the endpoint is removed from the rotation, moving from a "break-first" approach to a seamless transition and ensuring infrastructure changes remain invisible to your users.

‍

No circuit breaker configured

Distributed systems are prone to partial failures, where a single downstream dependency—like a database or a third-party API—starts to lag or fail. Without a circuit breaker, your application might keep sending requests to a failing service, causing threads to hang and resources to saturate. This can cause a cascading failure, where one small issue eventually snowballs into a much bigger problem that takes down your entire service. A circuit breaker acts as a safety valve that detects small failures and "trips," immediately failing those requests and allowing the system to recover or provide a graceful fallback. By identifying missing circuit breakers, Gremlin helps you prevent localized issues from becoming enterprise-wide outages.

‍

Outlier detection disabled

Sometimes a single instance in a healthy cluster starts behaving poorly. This could be due to a memory leak, a localized hardware glitch, or other failure mode that doesn't trigger a full crash, but causes high latency. Without outlier detection, unhealthy instances like these remain in the Backend Service’s rotation, periodically serving users and creating an undesirable experience. Outlier detection lets your load balancer automatically detect these deviating instances and eject them from the pool. By flagging this risk, Gremlin helps you build a self-healing infrastructure that automatically isolates bad actors before they can impact your overall reliability score.

GCP lets you set the thresholds it uses to determine endpoint health and spot outliers, including:

• The maximum number of consecutive errors (consecutiveErrors)
• The shortest amount of time a backend can be removed from the pool before being re-evaluated (baseEjectionTime)
• The maximum percentage of backend instances that can be removed at the same time (maxEjectionPercent)

Be careful not to allow the backend service to evict too many endpoints at once. Setting your maxEjectionPercent too high could cause your service to downscale so much that you don’t have enough endpoints to handle your current traffic volume. Conversely, setting consecutiveErrors too low can result in false positives and cause the backend service to evict healthy instances.

‍

Eliminate reliability blind spots with automatic risk detection

Reliability isn't a "set it and forget it" configuration; it's a continuous practice. As your cloud footprint grows across AWS, Azure, or GCP, the complexity of managing every configuration and dependency manually becomes a significant risk to your uptime.

With Gremlin’s Detected Risks, you no longer have to guess where these risks lie. We provide a centralized view of your infrastructure's health, automatically surfacing the critical misconfigurations and architectural gaps that lead to outages.

If you’re ready to start uncovering reliability risks in your AWS, Azure, GCP, or Kubernetes environments, give Gremlin a try, or see how it works with a free product tour.

Take the tour