Gremlin and GCP Cloud Spanner

Test how your application handles a Cloud Spanner outage. Run Gremlin network experiments and reliability tests to validate multi-region failover.

Google Cloud Spanner delivers global consistency, and the price is paid in latency. Writes coordinate across replicas, and where those replicas sit relative to your application determines how long that takes. Teams choose a multi-region configuration for availability without always pricing in what it does to write latency under normal conditions, let alone degraded ones.

When a region becomes unreachable, coordination slows further. Gremlin lets you find out what your application does in that window. With Gremlin, you can:

  • Confirm your write timeouts survive degraded coordination. A latency experiment slows the path to Spanner so you can see whether timeouts tuned against best-case performance start failing writes that would have succeeded.
  • Test regional failover for real. A blackhole experiment removes a regional endpoint and shows whether your application reconnects on its own and how long it takes.
  • Check where stale reads end up. Replication lag is fine for a report and unacceptable for a balance check. Testing reveals which of your read paths can tolerate it.

Both your timeout values and your read routing looked reasonable when someone set them. Testing is what tells you whether the conditions they assumed still hold.

Building resilience on Cloud Spanner with Gremlin

Cloud Spanner is a dependency: a managed service that your service connects to. You can run tests by deploying the Gremlin agent or Failure Flags sidecar to a service that consumes Cloud Spanner*. Using the experiments shown to the right, you can prepare your service for Cloud Spanner failure modes, including:

  • Network outages making Cloud Spanner unavailable
  • Slow performance due to network latency
  • Expiring TLS certificates
You can also run these expert-built workflows designed to replicate real-world failure modes on Cloud Spanner:

* Testing a dependency doesn't require installing anything on it. Gremlin runs the experiment from the service that calls it, so compatibility depends on that host rather than on the dependency itself. Check our compatibility documentation for supported operating systems and platforms, or get in touch if you don't see yours.

Learn more about Gremlin and Cloud Spanner

Avoid downtime. Use Gremlin to turn failure into resilience.

Gremlin empowers you to proactively root out failure before it causes downtime. See how you can harness chaos to build resilient systems by requesting a demo of Gremlin.

Product Hero ImageShape