Multi-Region Systems

Active-Active: Every Conflict Scenario Becomes Real

Both regions serve, both accept writes, there is no failover step to get wrong. What you buy is local latency and continuously-proven capacity. What you pay is that every concurrent-write scenario you could previously wave away is now a thing that happens, in production, at a rate you do not control — and that a bad write propagates to both regions at wire speed.

▶ Run the lab

The question this answers

The question

Both regions are serving live traffic. Which problems did that solve, and which did it create?

The guarantee — the property claimed, and its scope

Availability of the write path in each region independently of the others, and local write latency. In exchange, for data written in more than one region the achievable ceiling is causal consistency at best and eventual in practice: no global uniqueness, no cross-region read-modify-write, no invariant that spans regions. Data that is single-owned within an active-active deployment keeps its stronger guarantee — the model applies per dataset, not per system.

Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.

What a node knows — observation versus inference

Each region knows what it has accepted locally and what has arrived from the others. It does not know what the other region accepted in the last replication interval, and cannot: that information is in flight. So every local decision is made against a view that is definitionally out of date by at least one one-way latency, and "has anyone else already taken this username?" is a question no region can answer.

A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.

What guarantee?What does a node know?How does it work?What can fail?How does it fail?Where is coordination?What holds under failure?How does it recover?How would you know?What is the simpler thing?
active-activeconflictsavailabilitymulti-regionblast radius

Reject the framing first

"Active-active is more reliable than active-passive" is stated constantly and is not a claim about reliability at all. It is a claim about *one* failure mode — the failover — and it is true about that one. Active-active removes the promotion step, the DNS repoint, the cold-cache collapse and the entire class of problems in [[region-active-passive]] that come from an unrehearsed procedure. That is a genuine and substantial win, and it is the honest case for the model.

But reliability is not one number. Active-active *adds* failure modes that active-passive does not have, and they are not exotic: concurrent writes to the same record, invariants that cannot be enforced, replication that carries corruption both ways at full speed, and capacity that is only sufficient while both regions are alive. Whether the total is better depends entirely on which failure you are defending against, and how much of the write set is genuinely shared.

The precise version of the claim is worth stating, because it is the one you can defend in a review: active-active converts a rare, high-severity, procedure-dependent failure (a botched failover) into a continuous, low-severity, correctness-dependent one (conflicts and divergence). If your organisation is better at building correct merge logic than at rehearsing procedures, that is a good trade. If it is not, it is a bad one, and it will be discovered slowly.

The word describes traffic, not data — and that is where teams talk past each other

Four quite different architectures are all called active-active in ordinary conversation, and they have almost nothing in common on the axis that matters.

Sorting a design into this table before discussing it saves most of the argument. In particular, rows one and two have *no conflict problem at all*, and they are what a large fraction of "active-active" deployments actually are — which is why some teams report the model as painless and others as a nightmare. They are describing different systems.

VariantWhere writes are orderedConflictsWhat is actually gained
Active reads, single-writer regiontypicalOne regionNoneRead latency and read capacity. This is single-writer with a marketing name.
Active-active with partitioned ownershipprotocolThe owning region, per keyNone within a partitionLocal writes for well-homed data *and* full guarantees. The best variant, and `[[multi-region-write-models]]` is how you get there.
Active-active, mergeable data onlyprotocolEverywhere; a merge function reconcilesResolved by constructionFull local write availability for data that genuinely commutes — logs, counters, sets, CRDT documents.
Active-active, shared mutable recordsprotocolEverywhere; last-write-wins decidesConstant and silentLocal latency, at the cost of an unstated data-loss policy. This is the variant people mean when they say it went badly.
Four things called "active-active"

What "every conflict scenario becomes real" means concretely

In a single-writer system, a large class of bugs is impossible and nobody has to think about it. Turning on a second writer makes each of them a live production concern with a nonzero rate. Here is the list, because it is longer than people expect and each item is a design task, not a configuration flag.

The rate matters as much as the list. The conflict window is the replication lag — typically 100 ms to a few seconds, and minutes under stress — so the probability of a conflict on a given record is roughly the chance that two writes to it fall inside that window. For a per-user record with one active session, that is nearly zero and the model is fine. For a shared record — a team setting, a shared document, an inventory count, an admin-edited config — it is not small, and it rises with exactly the traffic growth that motivated going multi-region in the first place.

  • Lost updates. Two edits to the same record inside the lag window; one is discarded by whatever resolution policy is in effect, usually silently — [[write-conflicts]], [[last-write-wins]].
  • Uniqueness violations. Two regions each accept the same username, email or slug. Neither could have known. The violation appears when the streams merge, not when the write happens.
  • Read-modify-write races. balance = balance - 10 executed in both regions from the same starting value produces one deduction, not two. Every counter, quota and inventory count is this.
  • Non-monotonic reads. A user’s request lands in region A, then region B, and they see their own change appear and disappear — [[monotonic-reads]] is what they have lost and [[session-guarantees]] is what restores it.
  • Ordering-dependent state machines. "Cancelled" and "Shipped" written in different regions; the merged order depends on clock comparison, which is [[clock-skew]] deciding a business outcome.
  • Referential integrity across the boundary. Region A deletes a parent while region B adds a child. Both succeed. The merged state has an orphan and no constraint caught it.
  • Foreign-key-shaped invariants in general. Anything expressed as "must not exist" or "must exist" cannot be checked locally, because absence is exactly what a stale replica cannot tell you.

Replication is a corruption pipeline as well as a durability mechanism

This is the argument that most often changes people’s minds, and it is rarely made. In active-passive with asynchronous replication, the standby is *behind*. That delay is a buffer: a bad migration, a runaway script, a poisoned data feed or a deletion bug takes seconds or minutes to reach the second copy, and a fast operator can cut replication and preserve a clean copy.

Active-active removes the buffer deliberately — the whole point is that both sides are current. So a logical corruption is applied in both regions at wire speed, and the second region is not a recovery option, it is a second victim. The blast radius of a bad deploy or a bad write is not halved by having two regions; for this class of failure it is exactly doubled.

Which means active-active does not replace backups, and specifically does not replace *point-in-time* recovery. The failure it defends against is infrastructure loss. The failure that most often takes systems down is a change someone made, and against that, replication of any kind is an accelerant. Cloud’s backup-strategy and restore-testing remain load-bearing, and [[durability-and-recovery]] is the distinction between a replica and a backup.

The capacity arithmetic nobody does

Two regions, both serving, each at 70% utilisation — a perfectly normal-looking operating point. One region is lost. The survivor now needs 140% of its capacity. It does not have it. The "no failover step" advantage evaporates at the exact moment it was supposed to pay off, and instead of a promotion procedure you get a saturation cascade, which is [[cascading-failure]] and is considerably harder to recover from than a promotion.

True region-loss tolerance in an N-region active-active deployment requires each region to run at no more than (N-1)/N of its capacity — 50% for two regions, 67% for three. Most deployments do not, because paying for 50% idle capacity is a conversation nobody wants, so the actual position is "active-active, and degraded service if we lose a region", which is a legitimate choice but must be a stated one. Three regions is often the cheaper answer than two precisely because the headroom requirement drops from 50% to 33%.

And the load does not arrive gently. All connections from the lost region reconnect at once, caches in the survivor are cold for the new keyspace, and every client retries. The survivor sees a step function, not a ramp — which is where [[load-shedding]] and [[admission-control]] stop being theoretical and become the difference between degraded and dead.

When it is the right answer

None of the above is an argument against active-active. It is an argument against choosing it for the reason people usually give. The genuine cases are specific and identifiable.

Choose it when the write set partitions naturally, so most writes are single-owner anyway and the model is really partitioned ownership with a friendly name. Choose it when the data merges by construction — append-only events, counters, sets, presence, telemetry, CRDT-backed documents — because then conflicts are resolved by definition rather than by policy. Choose it when users genuinely need local write latency and the alternative is a 200 ms floor on every interaction. And choose it when the organisation cannot rehearse failovers, because a model with no failover step is more robust in the hands of a team that will not practise than a model that depends on practice.

The last one is under-appreciated and honest: active-active’s deepest advantage is that both regions are continuously exercised. Certificates, quotas, config, dependency allowlists, capacity and the deploy pipeline are all proven every minute of every day, rather than at 3 a.m. during an incident. That is worth a great deal — and it is available in weaker forms too, which is why a hot standby serving read traffic gets much of the benefit at a fraction of the correctness cost.

Key points

  • "Active-active is more reliable" is a claim about the failover step only. It is true about that step and silent about everything else.
  • The model converts a rare procedure-dependent failure into a continuous correctness-dependent one.
  • Four different architectures are called active-active; two of them have no conflict problem, which is why reports of the model differ so wildly.
  • The conflict window is the replication lag, and it widens under exactly the load that motivated the second region.
  • Uniqueness, counters, state machines and referential integrity cannot be enforced across regions that both accept writes.
  • Replication carries logical corruption to both regions at wire speed — active-active is not a backup and removes the delay buffer that a lagging standby accidentally provides.
  • Region-loss tolerance requires each of N regions to run below (N-1)/N utilisation; most deployments quietly do not.
  • Its best genuine advantage is that both regions are continuously exercised, which is also obtainable from a read-serving hot standby.

The chain, answered

Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.

How it works
  • Each region runs a full stack and accepts reads and writes from local users.
  • Writes are committed locally and replicated asynchronously to the other regions.
  • Arriving writes are applied; where they touch a record modified concurrently elsewhere, a resolution policy runs — timestamp comparison, version vectors, or an application merge.
  • A global routing layer maps users to regions, usually by proximity, with the understanding that it is a performance mechanism and never a correctness one.
  • Data that cannot tolerate conflicts is excluded from the model: given a single owner, escrowed into per-region allocations, or routed to one region on the write path.
  • Each region is sized so that the survivors can absorb the traffic of a lost one — or the degradation is accepted explicitly.
What can fail at the boundary
  • Two regions write the same record inside the replication window, and a resolution policy silently picks one.
  • A uniqueness constraint is satisfied locally in both regions and violated globally.
  • Replication lag grows under load, widening the conflict window precisely when write volume is highest.
  • A logical corruption — a bad migration, a runaway job — replicates to every region within seconds.
  • A region is lost and the survivors do not have the headroom, turning an availability event into a saturation cascade.
  • The routing layer flaps between regions for the same user, producing non-monotonic reads that look like a caching bug.
How it fails — what an operator sees
  • Silent lost update: two agents edit the same customer record from different regions; one edit vanishes with no error, no log line and no metric. The only detection is a human noticing.
  • Duplicate "unique" identity: two accounts exist with the same email. The operator sees a constraint violation in the replication apply stream, hours after the signups, or discovers it when password reset behaves ambiguously.
  • Counter drift: an inventory count in region A and region B disagree by a growing amount. The observable is oversold stock, and the reconciliation cannot say which sale was legitimate.
  • Corruption in stereo: a bad deploy writes malformed records; both regions have them within seconds. The operator’s instinct — "fail over to the other region" — makes nothing better, and the only path is point-in-time restore.
  • Post-loss saturation: one region goes down and the survivor’s latency triples and then errors. The operator sees a second, larger outage minutes after the first, and no failover to perform.
  • Non-monotonic user experience: a user refreshes and their change appears, disappears, and reappears as the routing layer moves them between regions. Every service reports 100% success.
Where coordination is required
  • None on the write path — that is the entire product of the model, and the reason it achieves local latency.
  • Consequently, no invariant spanning regions is enforced. That is not a gap in an implementation; it is the direct consequence of not coordinating.
  • Coordination reappears in three places: resolving conflicts after the fact, any operation that must be globally unique, and any escrow or allocation scheme used to avoid the first two.
  • Introducing "just one" synchronous cross-region check — a uniqueness lookup, a quota decrement — reintroduces the full RTT and the full joint-availability cost on that path, and is how active-active deployments quietly acquire a coordination bottleneck nobody designed.
What still holds under failure
  • Each region continues serving its users through a partition, at full local latency, which is the model working as intended.
  • Divergence accumulates for the duration of the partition at the rate of conflicting writes, and must be merged afterwards.
  • No invariant that spans regions holds during the partition; if one was assumed, the violations are already in the data by the time anyone looks.
  • On losing a region entirely, the survivors keep serving — subject to having the capacity, which is a separate and often unmet condition.
How it recovers
  • Detect: measure conflict rate and resolution outcome per dataset. A conflict rate reported as zero almost always means it is not instrumented.
  • Contain: for the data classes that cannot tolerate divergence, stop accepting writes in all but one region rather than continuing and reconciling — a deliberate, narrow degradation is cheaper than a broad merge.
  • Recover: let replication drain and apply the merge policy, surfacing conflicts to the application or the user wherever the value is high enough to justify it.
  • Reconcile: run a scheduled cross-region comparison on the invariants that matter — uniqueness, balances, counts — because these are the violations that produce no errors at all. [[reconciliation]] is the mechanism.
  • Verify: assert the invariant directly. Equal replication positions prove bytes arrived, not that the two regions agree about anything meaningful.
How you would know
  • Conflict rate and resolution decision per dataset, with a sample of discarded values retained — you cannot reason about lost updates you did not record.
  • Replication lag per direction, which is the conflict window and should be treated as a correctness metric rather than a performance one.
  • Per-region headroom expressed as "could this region absorb the others’ traffic right now", not as CPU percentage.
  • Cross-region invariant checks running on a schedule: duplicate unique keys, balance sums, orphaned rows.
  • Rate of users whose consecutive requests hit different regions, which predicts non-monotonic-read complaints before the tickets arrive.
When it helps
  • The write set partitions naturally by user or tenant, so most writes have a single owner regardless of the model.
  • The data merges by construction — append-only events, counters, sets, presence, collaborative documents.
  • Users on multiple continents need local write latency and the interaction is chatty enough that an RTT per write is unacceptable.
  • The organisation will not rehearse failovers, so a model without a failover step is more robust in practice than one that depends on practice.
  • Read and write capacity genuinely needs to exceed what one region can provide.
When it hurts
  • The dataset has strict invariants — uniqueness, non-negative balances, exactly-once allocation — which no amount of merge logic restores.
  • The team has not yet made one region reliable; active-active doubles the operational surface and halves the observability.
  • Regions run above (N-1)/N utilisation, so losing one causes a cascade rather than a graceful degradation.
  • It is being adopted to satisfy a disaster recovery requirement, which it does not satisfy — replication propagates corruption, and only backups defend against a bad change.
  • The write volume is low and users are concentrated. Then you have taken on the hardest consistency problem in the domain in exchange for a latency improvement nobody asked for.
Simpler alternatives
  • Active-active reads with a single-writer region: most of the latency and capacity benefit, none of the conflict problem. The default that should be beaten before anything harder is chosen.
  • Partitioned ownership, which is active-active from the outside — every region serves writes — while remaining single-writer per key. [[multi-region-write-models]].
  • A hot standby that serves read traffic: continuously exercised, so it captures active-active’s best real advantage without its correctness cost.
  • Restrict multi-writer status to the mergeable subset of the data — events, telemetry, counters — and keep everything else single-owned. Almost every successful active-active system is actually this.
  • Three regions rather than two, if the driver is region-loss tolerance: the headroom requirement drops from 50% to 33% and often costs less overall.

Seven conflicts that become real the day both regions accept writes

Active-active: every conflict scenario becomes real
A support agent in Virginia and a customer in Frankfurt edit the same address. What you buy is local latency and continuously-proven capacity; what you pay is that every scenario you could previously wave away now happens at a rate you do not control.
scenario
resolution policy
Lost updates
Two edits to the same record inside the lag window.
writes concurrent?
no
conflict window
800 ms
outcome
data lost or an invariant broken
error surfaced
none
Broken. One edit is discarded silently, chosen by whichever writer had the faster clock. At 2.00 s apart with 800 ms of lag, the second writer had already seen the first, so this particular pair does not conflict. Widen the lag or narrow the gap and it does — and the lag widens under exactly the load that motivated the second region.
Convergence trace. Not converged within 24 steps: 3 messages are still in flight across 2 disagreeing groups. Run it longer — the topology permits convergence. Right now: fra → "Frankfurt’s value" · iad, sin → "Virginia’s value".
Region-loss tolerance is an arithmetic requirement, not a topology one. With 2 regions each at 70%, losing one leaves the survivors at 140% of their capacity — which they do not have. True tolerance needs each region below 50%: 50% for two regions, 67% for three. Three regions is often cheaper than two precisely because the headroom requirement drops from 50% to 33%.
And replication is not a backup. A migration that writes a malformed value to 4% of rows reaches both regions at wire speed. The blast radius of a bad deploy or a bad write is not halved by having two regions; for this class of failure it is exactly doubled — active-active removes the delay buffer that a lagging standby accidentally provides. The defensible claim about the model is narrower than the usual one: it converts a rare, high-severity, procedure-dependent failure — a botched failover — into a continuous, low-severity, correctness-dependent one.
Where writes are orderedConflictsWhat is actually gained
Active reads, single-writer regiontypicalOne regionNoneRead latency and read capacity. This is single-writer with a marketing name.
Active-active with partitioned ownershipprotocolThe owning region, per keyNone within a partitionLocal writes for well-homed data and full guarantees. The best variant.
Active-active, mergeable data onlyprotocolEverywhere; a merge function reconcilesResolved by constructionFull local write availability for data that genuinely commutes — logs, counters, sets, CRDT documents.
Active-active, shared mutable recordsprotocolEverywhere; last-write-wins decidesConstant and silentLocal latency, at the cost of an unstated data-loss policy. This is the variant people mean when they say it went badly.
Four things called "active-active". Two of them have no conflict problem, which is why reports of the model differ so wildly.
assumptionConflict-window figures assume replication lag of hundreds of milliseconds to seconds, typical for a healthy cross-region stream; under load, bulk imports or a degraded link, minutes are normal. A resolution policy that depends on wall-clock timestamps additionally assumes bounded clock skew — under skew the regions can converge to different values.

What people believe, and what is true

Claim

Active-active is more reliable than active-passive.

Reality

It removes the failover step and adds conflicts, shared-fate replication of corruption, and a capacity requirement. Which is more reliable depends on the failure you are defending against — and this domain treats the blanket claim as a red flag.

Claim

Active-active gives us disaster recovery.

Reality

It gives infrastructure-loss tolerance. Against the more common disaster — a bad change — it replicates the damage to every region within seconds, and only point-in-time backups help.

Claim

Conflicts are handled by the database.

Reality

A resolution policy runs, usually last-write-wins on a timestamp. That is a policy that discards data by design, and it is a business decision that has been made by default.

Claim

We are active-active so we can lose a region with no impact.

Reality

Only if every region runs below (N-1)/N utilisation. Otherwise the survivors saturate and the availability event becomes a cascade.

Claim

We will add a uniqueness check across regions to fix the duplicate signups.

Reality

That check is a synchronous cross-region round trip on the write path. You have reintroduced the coordination you went active-active to avoid, on the path where it hurts most, and made signup unavailable during a partition.

Go deeper

Only the levels this lesson can honestly fill — a missing level is a claim nobody had.

Overview

Both regions serve and both accept writes. No failover to get wrong; every concurrent-write scenario is now real, and a bad write reaches both regions immediately.

Practical

Classify your data before turning it on: mergeable (safe), single-owned (safe, and this is most of it), genuinely shared and mutable (the hard part — and it should be a short list). Instrument conflict rate per dataset and retain the discarded values. Check the capacity arithmetic: each of N regions must sit below (N-1)/N or losing one cascades. And keep backups, because replication is not one.

Advanced

The useful way to evaluate an active-active proposal is to ask what it does to the *availability of each invariant* rather than to the availability of the service. A service that is up in both regions while its uniqueness invariant is unenforceable has not become more available; it has become available for a weaker set of operations, and the difference has been moved into a reconciliation process that may not exist. This is the PACELC "else" clause read in reverse: active-active buys low latency in the common case by pre-committing to a weak consistency answer during partitions, and the honest design records that pre-commitment per dataset. Where an invariant genuinely must hold, the options are the ones from [[coordination-avoidance]] — factor it into per-region shares — or pay the round trip and accept that this one operation is not active-active at all.

Apply it

Build it, then break it
  • 🔧 Take the tables in one service and sort them into mergeable, single-owned and genuinely-shared-mutable. Argue that the third list can be made empty.
  • 🔧 Instrument conflict detection for one active-active dataset and record what the current rate actually is.
Reason about this
  • A support agent in Virginia and a customer in Frankfurt edit the same address within two seconds. Replication lag is 800 ms. Describe every possible final state and which the customer sees first.
  • A migration writes a malformed value to 4% of rows. The team proposes failing over to the other region. Explain why that does not help and what does.
Interview questions
  • 💬 A colleague says active-active is more reliable than active-passive. What do you ask them?
  • 💬 Which of your datasets could safely be written in two regions at once, and how did you decide?
  • 💬 You are active-active across two regions, each at 70% utilisation. One region fails. What happens?
  • 💬 How would active-active have helped, or not helped, with the last incident you were involved in?