Reliability & Disaster Recovery

Multi-Region Deployment

Running in more than one geographic region. It is the answer to latency, to regional failure and to data-residency law — and "multi-region" on its own says nothing about availability. A multi-region deployment with one primary database in one region is single-region availability with a bigger bill.

The question this answers

Infrastructure question

What does a second region actually buy, and which of those things does it fail to buy unless you also solve the data problem?

Application requirement

European and American users both complain about latency. Compliance requires EU personal data to stay in the EU. And the board has asked what happens if an entire region is unavailable for six hours.

What it provides

Serving capacity near users, a jurisdictional boundary for data, and — only if the data tier is designed for it — the ability to keep serving when an entire region is gone.

Application RequirementInfrastructure RequirementComputeNetworkStorageIdentityDeploymentScalingReliabilityObservabilitySecurityCostTrade-offs

Six reasons, and they are not the same reason

Teams say "multi-region" as if it were one decision. It is at least six, and they pull in different directions. Latency: users far from your region pay 150 ms of round trip before your code runs. Resilience: a region can fail, and single-region means down. Data residency: the law may require certain data to remain in a jurisdiction. Operational complexity: every one of these costs a great deal of it. Cost: near-duplication of the infrastructure plus inter-region transfer. Consistency: the moment data exists in two places, you have a distributed-systems problem you did not have before.

Which reason you have determines the design. Latency alone is often solved by a CDN and edge caching, with no second region at all — see CDN as Infrastructure. Residency is solved by *partitioning* users into regions, not by replicating everything everywhere; EU data in the EU, US data in the US, and no cross-region replication of the regulated tables. Resilience needs a data tier that can be promoted or written to in the second region, which is by far the hardest and most expensive requirement.

The red flag is the assumption that the third comes free with the first two. "Multi-region automatically means high availability" is false. A deployment with application servers in Frankfurt and Virginia, both writing to a primary database in Frankfurt, has doubled its compute bill, added inter-region latency to every US write, and has exactly the availability of Frankfurt. When Frankfurt goes, both regions go. This is a very common architecture and it buys latency for reads and nothing else.

The common shape — and why its availability is the availability of one region.PROVIDER-NEUTRAL
Global DNS / anycast routingpublic— latency- or geo-based routing to the nearest healthy region
Region: EU
Load balancerpublic
app tier (multi-zone)private
Primary databaseprivate
Object storage (EU)private— regional unless explicitly replicated — check before assuming
Region: US
Load balancerpublic
app tier (multi-zone)private
Read replicaprivate— asynchronous — serves reads, cannot accept writes, lags the primary
Global DNS / anycast routingLoad balancer· EU users
Global DNS / anycast routingLoad balancer· US users
Load balancerapp tier (multi-zone)
Load balancerapp tier (multi-zone)
app tier (multi-zone)Primary database· reads + writes
app tier (multi-zone)Read replica· reads (stale by replication lag)
app tier (multi-zone)Primary database· writes — ~90ms each waycrosses boundary
app tier (multi-zone)Object storage (EU)

The six dimensions, scored honestly

Every multi-region decision moves all six dimensions at once, and the movements are not all in your favour. The table is the conversation to have before the design, because afterwards each row becomes an argument with someone.

Consistency is the row that determines everything else. Speed-of-light latency between Frankfurt and Virginia is roughly 40 ms one way, so about 90 ms round trip in practice. Synchronous replication across that distance adds it to every commit — usually unacceptable. Asynchronous replication is therefore the norm, which means the replica is behind, which means a failover loses whatever had not replicated, which is your RPO (RPO & RTO). There is no configuration that removes this trade; it is geometry. The depth of the argument belongs to the Distributed Systems and System Design material — this domain only insists that you decide it explicitly.

Operational complexity is the row teams discount most and pay for most. Two regions means two of every deployment, two sets of secrets, two sets of certificates, two monitoring configurations, two sets of drift, and a failover procedure. The number of things that can be inconsistent between them grows faster than the number of components, and inconsistency between regions is the mechanism behind most multi-region incidents (Scoring Operational Complexity).

DimensionWhat improvesWhat gets worseThe decision it forces
LatencyUsers served from a nearby region; reads are localCross-region writes are ~90ms and feel brokenCan reads be local and stale, or must every read see the latest write?
ResilienceA region failure need not be an outageOnly true if the data tier can be promoted or written to locallyActive-passive or active-active — and who or what triggers failover?
ConsistencyNothing improves hereData now exists in two places with a lag between themWhat RPO is acceptable, and what happens to writes that had not replicated?
Operational complexityNothing improves hereTwo of everything, plus a failover procedure that must be rehearsedDoes the team have capacity to operate two regions correctly?
CostNothing improves hereNear-duplication of infrastructure plus inter-region transferIs a standby that is idle 99.9% of the time worth its price?
ComplianceData can be kept in a required jurisdictionReplication across regions may be exactly what the law forbidsPartition by jurisdiction rather than replicate globally
What a second region does to each dimension

The bill, and the meter people forget

The naive estimate for a second region is "twice as much". The real number is usually more, because duplication is not the only new line. Inter-region data transfer is metered — often at a higher rate than cross-zone — and it carries continuous replication traffic whether or not anyone is using the second region. Every write to the primary is transferred to every replica, forever, at rest-of-the-world prices.

Then there is the cost of the things that do not duplicate cleanly. Some managed services are regional and must be instantiated twice with separate configuration. Reserved capacity or committed-use discounts negotiated for one region may not apply to another. Observability doubles: twice the metrics, twice the logs, twice the retention. And engineering time — the largest cost in this lesson — goes into building and rehearsing the failover that justifies the whole thing.

The honest framing for a business conversation: a second region for resilience is an insurance premium paid continuously against a rare event. Ask what an hour of total outage costs, ask how often a full region has actually been unavailable, and compare that to the annual cost of the standby plus the engineering time to keep it working. For many businesses the answer is that a rehearsed four-hour restore into a second region (Disaster Recovery) is a far better trade than a warm standby nobody has failed over to.

A second region, by driver. Relative weights, not currency.COST-VARIES
Duplicated compute fixed
driven by the same fleet, again · In active-passive most of this is idle capacity you hope never to use.
Duplicated data tier fixed
driven by a second database and its storage · Usually the largest single line, and it grows with your data rather than your traffic.
Inter-region data transfer · surpriseusage
driven by GB replicated + GB of cross-region calls · Continuous, priced above cross-zone transfer, and proportional to write volume.
Duplicated observability · surpriseusage
driven by metrics, logs and traces × 2, plus retention · Doubling the fleet doubles telemetry, and telemetry is already a top-five line item for many teams.
Storage replication usage
driven by GB stored in both regions + replication requests · Cross-region object replication bills for storage twice and for the transfer once.
Engineering and drills fixed
driven by building, rehearsing and maintaining failover · The cost that determines whether any of the above works. Skipping it makes everything else theatre.

Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.

Key points

  • "Multi-region" is at least six separate decisions: latency, resilience, residency, complexity, cost and consistency.
  • Multi-region does not automatically mean high availability. Two regions writing to one primary have the availability of that primary's region.
  • Cross-region round trips are roughly 90 ms in practice; synchronous replication across that distance is usually unacceptable, which makes asynchronous replication and a non-zero RPO the norm.
  • Latency alone is often better solved by a CDN and edge caching than by a second region.
  • Residency is solved by partitioning users into regions, not by replicating everything everywhere — and replication may be exactly what the law forbids.
  • Inter-region transfer and duplicated observability are the two meters nobody budgets for.

The loop, answered

Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.

How it works
  • Global routing — latency-based DNS, geolocation DNS or anycast — sends each client to a region, with health checks that can withdraw a failed one.
  • Each region runs its own full stack: load balancer, application tier, and whatever portion of the data tier the design allows.
  • The data tier replicates across regions, almost always asynchronously; the lag is the data-loss window on failover.
  • Regional services (object storage, queues, secret stores) are regional by default and must be explicitly replicated or duplicated.
  • Failover is a decision plus an action: withdraw a region from routing, promote a data tier, and redirect traffic — the hard part being who decides and how fast (Active-Passive Failover).
What you still own
  • Own two of everything and keep them identical. Region drift is the leading cause of a failover that does not work; manage both regions from the same code (Infrastructure as Code).
  • Rehearse failover on a schedule with the whole path included: routing withdrawal, data promotion, cache warm-up, client reconnection.
  • Measure replication lag continuously and alert on it, because it is your live RPO and it changes with write volume.
  • Decide in advance what happens to writes that were accepted and not replicated. "We will figure it out" is a decision to lose them silently.
  • Keep a written map of which data may cross which border. Compliance failures here are expensive in a different currency (Shared Responsibility).
How it fails
  • Split brain: both regions believe they are primary and accept conflicting writes, and reconciling them afterwards is a manual, lossy project.
  • Failover that has never been rehearsed and fails at the third step, in the middle of the incident it was meant to solve.
  • Region drift: a configuration, certificate or IAM policy exists in one region and not the other, so the standby cannot serve when promoted.
  • DNS-based failover slowed by client-side caching, so a portion of traffic keeps arriving at the dead region for far longer than the TTL suggests.
  • Cross-region latency in the write path making the second region feel broken to its own users while every dashboard says it is healthy.
  • Silent data loss on promotion: the replica was thirty seconds behind, and thirty seconds of orders no longer exist.
How it scales
  • Each region scales independently; global capacity is the sum, but the ability to absorb another region's traffic must be provisioned in advance.
  • Replication traffic and cost scale with write volume, not with read volume or user count.
  • Operational cost scales with region count faster than linearly: every pair of regions is another consistency and routing relationship.
  • More regions means better latency coverage and more places for the data problem to appear; three is materially harder than two.
Security
  • Two regions means two full sets of identities, keys, certificates and network rules. Every drift between them is a potential security gap (Least Privilege in Infrastructure).
  • Cross-region replication traffic leaves one region's network boundary and must be encrypted in transit, and the keys must be available in both regions (Key Management and Encryption at Rest).
  • Data residency is a security and legal control, not only an architectural one: replicating a table to another jurisdiction can be the violation itself.
  • A failover procedure that grants elevated emergency access must still be audited. Break-glass credentials are exactly where audit trails matter most (Audit Trails).
Cost shape
  • Near-duplication of compute and data, plus inter-region transfer at a rate above cross-zone transfer.
  • In active-passive, most of the second region is idle capacity billed continuously.
  • Observability, backups and support all roughly double, which is the part of the estimate that is usually missing.
  • The comparison worth making: annual cost of a standby region versus the cost of a rehearsed multi-hour restore. For many businesses the restore wins.
What to watch
  • Replication lag per region, alerted — it is your live RPO, and it degrades under write bursts exactly when it matters.
  • Per-region request rate, error rate and latency, never blended into a global average that hides one region failing.
  • Global routing health: which region each population is actually being sent to, which is not always what the configuration says.
  • Inter-region transfer volume, as both a cost signal and a detector of new cross-region call paths someone introduced.
  • The signal that lies: a global availability number. It stays comfortable while an entire region is unusable for the users routed there.
Simpler alternatives
  • A CDN plus edge caching. If the problem is read latency for distant users, this solves it for a fraction of the cost and none of the consistency pain. Try it first.
  • Multi-zone in one region plus a tested restore into a second region. You get facility-level HA continuously and regional recovery in hours, without running two live stacks (Disaster Recovery).
  • Regional partitioning with no cross-region replication: EU users live in the EU, US users in the US, and neither region is a failover for the other. Simpler, and it satisfies residency directly.
  • A managed globally-distributed database, if the workload tolerates its consistency model and pricing — buying the hard part rather than building it.
  • Accepting regional risk. Full-region outages are rare; for many businesses the honest answer is a documented RTO of several hours.
What adopting this costs
  • Buys latency, residency and potentially resilience; costs roughly double infrastructure, inter-region transfer and a permanent consistency problem.
  • Asynchronous replication keeps write latency acceptable and guarantees a non-zero data-loss window on failover.
  • Synchronous cross-region replication removes the data-loss window and adds ~90 ms to every commit.
  • Every added region multiplies the operational surface and the number of ways two places can disagree.

What people believe, and what is true

Claim

Multi-region means high availability.

Reality

Only if the data tier can serve or be promoted in the second region. Two regions sharing one primary database have the availability of that one region, plus a larger bill.

Claim

Multi-region is the way to fix latency.

Reality

For read-heavy content a CDN does it for a fraction of the cost. A second region only helps latency for requests that must reach your origin.

Claim

We will replicate everything to both regions for compliance.

Reality

Replication across a border can be the violation. Residency usually means partitioning users by jurisdiction and deliberately *not* replicating.

Apply it