Reliability & Disaster Recovery

Multi-Zone Deployment

Spreading a workload across the independent facilities inside one region. The reliability is real, the latency is small but not zero, the data placement is subtle, and the cross-zone transfer meter is the line item nobody predicted.

▶ Run the lab

The question this answers

Infrastructure question

What does spreading a workload across zones inside one region actually buy, and what does it charge for?

Application requirement

A facility-level event — power, cooling, a network partition inside one data center — must not take the service down. The business will not fund a second region, and the recovery must be automatic.

What it provides

Survival of the loss of one facility with no manual intervention, at single-region latency and without the data-consistency problems that crossing regions introduces.

Application RequirementInfrastructure RequirementComputeNetworkStorageIdentityDeploymentScalingReliabilityObservabilitySecurityCostTrade-offs

What a zone is, and what spreading across them buys

A zone is an independent failure domain inside a region: its own power, cooling and network, physically separate from its siblings but close enough that the round trip between them is on the order of a millisecond rather than tens. That combination is the entire value proposition — real fault isolation with latency low enough that synchronous replication and cross-zone request routing remain practical.

Spreading across zones is therefore the cheapest large step in reliability available. It survives the failure class that takes out a whole data center, without the consistency problem that a second region creates. Three zones is usually the sweet spot: quorum systems get an odd number of members, and losing one costs 33% of capacity rather than the 50% that two zones imply.

What it does not buy is worth stating plainly. A regional control-plane failure can prevent you from launching instances, reading secrets or resolving internal DNS while every zone is physically fine. A bad deploy reaches all zones. A regional managed service — a queue, an object store, an identity endpoint — is regional, and its outage crosses every zone by definition. Multi-zone protects against a facility, not against a region (Multi-Region Deployment).

Three zones, one region. Every tier spread; every shared service checked.PROVIDER-NEUTRAL
Clientspublic
Region eu-1
Load balancer (all three zones)public
Zone A
app ×Nprivate— each zone runs at <67% so any one can be lost
NATpublic— one per zone, or a zone loss removes egress for the survivors too
DB primaryprivate— synchronous replication is viable at ~1ms between zones
Zone B
app ×Nprivate
NATpublic
DB standbyprivate
Zone C
app ×Nprivate
Object storage (regional)private— regional service — survives a zone, not the region
ClientsLoad balancer (all three zones)crosses boundary
Load balancer (all three zones)app ×N
Load balancer (all three zones)app ×N
Load balancer (all three zones)app ×N
app ×NDB primary
app ×NDB primary· cross-zone ~1ms, metered
app ×NDB primary· cross-zone ~1ms, metered
DB primaryDB standby· sync replication
app ×NNAT
app ×NNAT
app ×NObject storage (regional)

Latency and data placement: small numbers that add up

provider-specific· Whether cross-zone load-balancer traffic and intra-region transfer are billed varies significantly between providers — this is the single most important thing to check locally.

Cross-zone round trips are typically well under two milliseconds — negligible for a single hop and decidedly not negligible for a request that makes forty of them. A chatty service that issues an N+1 query pattern against a primary in another zone pays that cost per query. The fix is the same fix as always (fewer, larger queries), but the failure is easier to overlook because the number looks harmless in isolation.

Data placement is the subtler half. The database primary lives in one zone; two thirds of your application instances are therefore reading and writing across a zone boundary at all times. Some teams pin traffic to the primary's zone for latency and immediately lose the reliability they were buying. The usual honest answer is to accept the millisecond, keep the tiers spread, and reduce chattiness instead.

Zonal storage deserves its own warning. A block volume normally lives in one zone and cannot be attached from another. A workload with a zonal volume is pinned regardless of how cleverly you configured the scheduler — during a zone failure the pod cannot be rescheduled, because its data is in the failed zone. Regional/replicated volumes or shared object storage are what make a stateful workload genuinely movable (Block Storage, Persistent Data and Containers).

TrafficLatency effectCost effectWhat to do about it
App → database primary in another zone~1ms per round trip, multiplied by chattinessMetered per GB in both directions on many providersReduce round trips; do not pin the app tier to fix it
Load balancer → target in another zoneNegligibleMetered on some providers; free on othersCheck whether cross-zone load balancing is billed before disabling it
Synchronous database replicationAdds ~1ms to every commitMetered replication trafficThis is the price of a zero-data-loss standby. Usually worth it
Cache reads across zones~1ms, which can exceed the cache's own service timeMeteredConsider a zone-local read replica of the cache if the hit rate justifies it
Object storage (regional service)No zone crossing from the caller's point of viewUsually not cross-zone meteredPrefer regional services for shared data
Service mesh / sidecar chatterSmall per hop, large in aggregateMetered, and easy to miss because it is not application trafficEnable zone-aware routing if the mesh supports it
What crosses a zone boundary, and what it costs you

The cross-zone transfer line item

Cross-zone data transfer is the classic surprise on a multi-zone bill. It is metered per gigabyte, frequently in both directions, and it applies to traffic that feels entirely internal — an application talking to its own database, a cache read, replication between database nodes, mesh sidecars gossiping. Nobody budgets for it because it does not look like network traffic; it looks like the system working.

The amount is driven by chattiness rather than by user-facing volume. A service that returns a 2 KB JSON response after issuing fifty cross-zone queries moves far more bytes internally than it does externally. This makes cross-zone transfer an unusually good proxy for architectural quality: a bill that climbs faster than traffic is telling you about your query patterns.

The mitigations are real but each has a cost. Zone-aware routing keeps a request within one zone when a healthy local instance exists — and reduces the load spreading that made the design resilient. Read replicas per zone remove cross-zone reads and add replication traffic plus staleness. Caching locally removes repeated reads and adds an invalidation problem. None of these is free, and the right answer is often simply to pay the transfer bill and keep the architecture simple.

The multi-zone bill, by driver. Relative weights, not currency.COST-VARIES
Compute across zones fixed
driven by instances × hours · Roughly unchanged from single-zone for the same total capacity.
N-1 headroom fixed
driven by capacity reserved to survive losing a zone · Idle by design. With three zones this is ~33%, with two it is ~50%.
Cross-zone data transfer · surpriseusage
driven by GB between zones — app↔db, cache, replication, mesh · The line item nobody predicts. Driven by chattiness, not by user traffic.
Per-zone gateways (NAT and similar) fixed
driven by one device per zone × hours · Multiplied by zone count. Skipping it re-creates a single point of failure.
Standby database fixed
driven by a second instance, always on · Usually the largest single fixed increment of going multi-zone.
Replicated / regional storage usage
driven by GB stored, at a higher rate than zonal · Buys the ability to reschedule a stateful workload into another zone.

Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.

Key points

  • A zone is an independent facility inside a region, close enough (~1ms) that synchronous replication and cross-zone routing stay practical.
  • Three zones beats two: an odd number for quorum, and losing one costs 33% of capacity instead of 50%.
  • Zonal storage pins a workload to its zone no matter what the scheduler wants — this is the most common multi-zone reliability bug.
  • Cross-zone data transfer is metered, driven by chattiness rather than user traffic, and is the surprise line item.
  • Per-zone gateways (NAT and friends) must be per-zone, or a zone failure removes egress for the survivors.
  • Multi-zone does not protect against a regional control-plane failure, a regional managed service outage, or a bad deploy.

The loop, answered

Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.

How it works
  • Subnets are zonal: one per zone, each with its own address range and route table.
  • The load balancer registers targets in every zone and health-checks them independently, so a zone failure appears as a set of unhealthy targets.
  • Scaling groups spanning zones rebalance placement as capacity changes, keeping distribution close to even.
  • The database keeps a synchronous standby in a second zone; promotion is automatic and takes tens of seconds, during which connections are reset.
  • Regional services — object storage, managed queues, secret stores — are reachable from every zone and survive the loss of one by design.
What you still own
  • Verify distribution continuously; a fleet that drifted into one zone is the failure mode this design exists to prevent.
  • Keep per-zone gateways, endpoints and route tables in sync. Configuration drift between zones is a real and frequent source of asymmetric failures (Drift: When the File and Reality Disagree).
  • Test by draining a zone deliberately. This is the only way to find the pinned volume, the single-zone NAT and the config service you forgot about.
  • Watch cross-zone transfer as an architectural metric, not only a cost one — it rises when someone adds a chatty call path.
  • Confirm your provider's billing for cross-zone load balancing before turning it off to save money; on some providers you are turning off resilience for nothing.
How it fails
  • One zone holds everything because a placement rule was missing, so the design existed only in the diagram.
  • A stateful workload cannot reschedule after a zone failure because its volume was zonal.
  • Single-zone NAT: the surviving zones are healthy and have no outbound connectivity.
  • Insufficient headroom: the survivors take the failed zone's traffic, saturate, and the partial failure becomes total.
  • Asymmetric configuration: zone C was added later with a subtly different security group, and only fails when it is the one carrying load.
  • Regional service outage: every zone is fine and the service is down anyway, which multi-zone was never going to prevent.
How it scales
  • Adding zones improves resilience and increases cross-zone traffic roughly in proportion to how spread the tiers are.
  • Cross-zone transfer grows with internal chattiness, so it scales with architecture quality rather than with user count.
  • Quorum systems need their members spread across an odd number of zones; adding nodes without adding zones does not improve fault tolerance.
  • Large fleets get cheaper resilience per instance: the N-1 headroom fraction depends on zone count, not on fleet size.
Security
  • Zones share the region's trust boundary. Spreading across zones does not create a security boundary — network controls must be identical in each (Security Groups: The Stateful Firewall).
  • Configuration drift between zones is a security finding as much as a reliability one: an over-permissive rule in one zone is a hole in the whole service.
  • Cross-zone traffic stays inside the provider network and is not automatically encrypted in every case; for regulated data, encrypt in transit explicitly (Key Management and Encryption at Rest).
  • Per-zone gateways multiply the number of egress points to monitor and to allow-list.
Cost shape
  • Compute cost is roughly unchanged for the same total capacity; the increments are headroom, per-zone gateways and the standby database.
  • Cross-zone data transfer is the usage-shaped meter and the one that surprises people.
  • Replicated storage costs more per gigabyte than zonal storage, and is what makes a stateful workload movable.
  • Zone-aware routing reduces the transfer bill and reduces the load spreading you are paying for. Optimize this only with the reliability trade explicit.
What to watch
  • Instance and replica count per zone, alerted when the distribution violates the design.
  • Cross-zone transfer volume by source and destination, which is simultaneously a cost signal and an architecture signal.
  • Per-zone health of gateways, endpoints and NAT — the components most often forgotten in the second and third zone.
  • Latency split by whether the request crossed a zone boundary, which turns "the app feels slow sometimes" into a fact.
  • The signal that lies: regional aggregate metrics. They average away a zone that is entirely gone until it is more than a third of your capacity.
Simpler alternatives
  • Single zone with good backups. If a few hours of downtime after a facility event is acceptable, this is materially simpler and cheaper — and for many internal systems it is the right call.
  • Two zones rather than three, when the workload has no quorum requirement and 50% headroom is affordable.
  • A managed regional service that hides zone placement entirely — regional object storage, a regional managed queue — so multi-zone becomes the provider's problem.
  • Multi-zone for the stateless tier only, with a single-zone database and a documented failover, when the database's recovery time is genuinely acceptable.
What adopting this costs
  • Buys facility-level fault tolerance; costs cross-zone transfer, N-1 headroom and a small latency tax on every cross-zone hop.
  • Three zones cost more per hour than two and less per unit of resilience.
  • Zone-aware routing saves money and weakens the spreading that made the design resilient.
  • Replicated storage makes stateful workloads movable and costs more per gigabyte, with its own consistency behaviour to understand.

Regions, zones and failure domains

Replicas, zones and failure domains
Place N replicas, then remove a zone or a region and read the surviving capacity. Redundancy only counts across a boundary the failure does not cross.
placement
Placement — the boxes are failure domains, not machinesILLUSTRATIVE
Region eu-west
Zone eu-aFAILED— 3 replicas
Replica 1FAILED
Replica 2FAILED
Replica 3FAILED
replicas
3
surviving
0
failure domains
1 zone / 1 region
peak served
no
surviving capacity vs peak need0 replicas · need 2
fails a zone lossdoes not survive a region losssingle-region write path
failure domain = the set of things that fail together
  same process   → a crash takes all of it
  same host      → a kernel panic takes all of it
  same zone      → one power or network event takes all of it
  same region    → a control-plane or provider event can take all of it
copies only count once they stop sharing the domain that failed
3 replicas in one zone is not 3 failure domains — it is one. The replicas protect you from a process crash and a bad host, and from nothing that happens to the building: shared power, shared top-of-rack network, shared cooling. This is the single most common redundancy mistake, and it looks completely healthy on a dashboard right up to the moment it does not.
failure
SIMULATEDcapacity is counted in replicas, not requests

What people believe, and what is true

Claim

Multi-zone means we survive a region failure.

Reality

Zones share a region: its control plane, its regional services and its regional endpoints. A regional failure crosses all of them.

Claim

Cross-zone traffic is internal, so it is free.

Reality

It is metered per gigabyte on most providers, and it is driven by internal chattiness rather than by user-facing traffic.

Claim

The scheduler handles zone spreading for me.

Reality

Only if you asked it to. Without spread constraints and with zonal volumes attached, workloads concentrate in one zone quite naturally.

Apply it