The question this answers
What problem am I actually solving by putting this on more than one machine?
Distribution guarantees nothing on its own. Each of the four reasons buys a specific, conditional property: capacity beyond one machine *if* the work partitions; survival of a node failure *if* failures are independent; lower latency *if* the data can be served locally; and fault isolation *if* the components do not share a dependency.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A node knows its own load, its own health, and its own latency to the clients it serves. It does not know whether the system as a whole is meeting its goal — that is a property of the collection, visible only to something aggregating across nodes, and therefore always slightly out of date.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
The four reasons
Capacity. The data does not fit, the throughput does not fit, or the compute does not fit on the largest single machine you can buy. This is the least common reason in practice and the most legitimate when it is true. Note the qualifier: the largest machine you can *buy*, not the one you are currently running. A single modern server has hundreds of gigabytes of RAM and dozens of cores, and a great many systems that "needed to scale out" needed a bigger instance and an index.
Availability. One machine failing must not take the service down. This is the most common honest reason, and it is why the minimal step past a single node is usually a replica rather than a shard. The conditional matters enormously: it buys you availability only against *independent* failures, and Correlated Failure: The Independence Assumption Is Usually False is the lesson about how often that assumption is false.
Latency and geography. Users on another continent are 150ms away at the speed of light before your code runs. No amount of optimisation closes that; only moving data closer does. This reason is unusual in that it is enforced by physics and therefore not arguable — and it is also the reason that most directly forces hard consistency trade-offs, because the same distance that helps reads hurts agreement.
Isolation. A component whose failure or resource consumption must not affect another gets its own machine, its own process, its own failure domain. This is the reason behind bulkheads, behind separate pools for critical and non-critical work, and behind a good deal of the legitimate case for services. It is also the only one of the four where distribution is the *goal* rather than the cost.
| Buys | Only if | Immediate cost | |
|---|---|---|---|
| Capacityassumption | Work beyond one machine | The work partitions with few cross-partition operations | Partition keys, rebalancing, hot shards |
| Availabilityassumption | Survival of a node loss | Failures are genuinely independent | Replication lag, failover logic, split-brain risk |
| Latency / geographyprotocol | Round trips that physics allows | Reads can be served from a local copy | Cross-region consistency choices become mandatory |
| Isolationassumption | One component cannot sink another | They do not share a hidden dependency | Duplicated infrastructure, a network hop, more to operate |
The reasons that are really one of these four
"We need to scale" is capacity, and deserves the follow-up question: scale *what* — storage, read throughput, write throughput, or compute? They have different answers. Read throughput is solved by replicas and is cheap. Write throughput is solved by partitioning and is expensive. Storage is solved by partitioning and is medium. Conflating them is how teams end up with a sharded system that only ever needed read replicas.
"We need to move faster as a team" is isolation, of a specific sort: deployment isolation, so one team can ship without coordinating with another. This is a real and legitimate benefit, and it is worth being honest that it is an *organisational* benefit paid for with a *technical* cost. It is the Conway’s-law argument and it belongs in Four Questions That Test a Proposed Boundary, where the cost is priced.
"We need resilience" is availability, and it is worth asking against what. Against a process crash, a restart supervisor is enough. Against a machine failure, a replica in another rack. Against a datacentre failure, another zone. Against a region failure, another region — and that last step changes your consistency model, so it is not a continuation of the same decision.
"Everyone does it this way" is not a reason. Cloud infrastructure has a good lesson on cargo-culting; the version that matters here is that each of the four reasons is *measurable*, so if you cannot produce the measurement, you have not yet found your reason.
Scaling up before scaling out
The order of operations that produces the least regret is: make it correct, measure it, make the single machine bigger, remove the algorithmic problem, add a read replica, and only then partition. Each step is cheaper and less risky than the next, and each one buys time to learn what the actual constraint is.
The reason this order is under-used is that vertical scaling has a hard ceiling and everyone knows it, so it feels like a delaying tactic. It is — deliberately. A ceiling you will hit in two years is a problem you can solve in two years with far more information than you have now, and with a partition key chosen from real access patterns rather than a guess. The cost of a wrong partition key, discovered late, is one of the more painful migrations in this business.
The exception is availability. If the requirement is "a single machine failure must not lose the service", no amount of vertical scaling helps, and a replica is the first step rather than the last. That asymmetry is worth internalising: capacity problems can wait; availability problems cannot be deferred by buying a bigger machine.
- Storage growth → bigger disk, then partitioning. Rarely urgent.
- Read throughput → cache, then replicas. Cheap and reversible.
- Write throughput → the genuinely hard one; partition, and expect cross-partition operations to be the cost.
- Availability → a replica, immediately; vertical scaling does not help at all.
- Latency to distant users → an edge copy, and a decision about what may be stale.
State the goal as a number
The discipline that separates a designed system from an accreted one is writing the requirement down as something falsifiable *before* choosing the topology. Not "highly available" but "survives the loss of any one availability zone with under 60 seconds of write unavailability and no committed data lost". Not "scalable" but "10,000 writes per second sustained, 40TB of data, growing 3TB a month".
Those numbers are what let you tell whether a design is adequate or excessive, and they are the input to every trade-off in the rest of this domain. They also usually reveal that the four reasons are not equally pressing: most systems have one dominant reason and three that are aspirational, and designing for the dominant one produces something far simpler than designing for all four.
Key points
- Four honest reasons: capacity, availability, geography, isolation.
- Each one buys a conditional property, and the condition is where the design work lives.
- Read scaling, write scaling and storage scaling are different problems with different answers.
- Vertical scaling first is not a delaying tactic to be embarrassed about — it buys information.
- Availability is the one reason that cannot be deferred by buying a bigger machine.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • State the requirement as a number with a time horizon.
- • Identify which of the four reasons that number implies; usually one dominates.
- • Check the condition attached to that reason — does the work partition, are the failures independent, can the data be local?
- • Choose the smallest topology that satisfies the dominant reason.
- • Re-measure after the change, because the constraint moves as soon as you relieve it.
- • The work does not partition as cleanly as assumed, and cross-partition operations dominate.
- • The replicas share a failure domain, so the availability gain is nominal.
- • The geographic copy needs writes as well as reads, which turns a caching problem into a consensus problem.
- • Isolation is defeated by a shared dependency nobody listed — a database, an identity provider, a DNS zone.
- • Sharded but not scaled: writes still bottleneck because one shard holds the hot tenant. The operator sees aggregate CPU at 30% and one node pinned, with latency tracking that node.
- • Replicated but not available: the failover takes eleven minutes because it has never been exercised. The operator sees an outage whose duration is dominated by recovery, not by detection.
- • Distributed for team velocity, coupled in practice: every feature needs changes in three services released together. The operator — here, the team — sees lead time increase after the split rather than decrease.
- • Edge copies that must be fresh: a geo-distributed read cache is correct until someone needs read-your-writes across regions. The operator sees users reporting that their own change did not appear, only when travelling or on mobile networks.
- • Capacity via partitioning needs coordination only for operations that span partitions — which is why the partition key choice is the highest-leverage decision in the design.
- • Availability via replication needs coordination on every write if it is synchronous, and none if it is asynchronous, with correspondingly different guarantees.
- • Geographic distribution makes the coordination cost visible as latency, because a round trip to a quorum now crosses an ocean.
- • Isolation needs no coordination at all, which is why it is the cheapest of the four and the most often overlooked.
- • A well-partitioned system loses only the partitions on the failed node, and only if they were not replicated.
- • A replicated system continues to serve, at reduced redundancy, with a window in which a second failure is far more damaging than the first.
- • A geographically distributed system continues to serve locally, and its regions diverge for the duration of any inter-region failure.
- • Detect: measure against the stated number, not against a generic dashboard — the requirement is the alert.
- • Contain: ensure the loss of one unit degrades a fraction of the service rather than all of it.
- • Recover: rebuild the lost redundancy quickly, because the window of reduced redundancy is when the second failure hurts.
- • Reconcile: after a regional split, merge according to a rule decided in advance, not during the incident.
- • Verify: re-run the capacity or failover exercise after recovery; an untested failover is a hypothesis.
- • Headroom against the stated number — how much of the designed capacity is in use, and the trend that predicts when it runs out.
- • Distribution of load across shards or replicas; the gap between mean and max is what tells you whether partitioning is working.
- • Actual measured failover time from an exercise, not the configured timeout.
- • Fraction of operations that cross a partition or a region boundary, which is the real cost of the chosen split.
- • When one of the four reasons is demonstrable with a number and a horizon.
- • When the condition attached to that reason has been checked rather than assumed.
- • When the reason is anticipated rather than measured, so the design is optimised for a shape of load that never arrives.
- • When several reasons are addressed at once, producing a system that is complex enough to require a platform team the organisation does not have.
- • Buy a bigger machine: the fastest, most reversible capacity intervention available, and it is dull enough that people skip it.
- • Fix the algorithm: an index, a batch, or removing an N+1 routinely beats a topology change by an order of magnitude.
- • Add a cache or a read replica: solves read scaling with a fraction of the complexity of partitioning.
- • Use a managed service that has already solved the distribution: you inherit its guarantees and its failure modes, but not the burden of designing the protocol.
What problem are you actually solving?
What people believe, and what is true
Scaling out is the modern way; scaling up is legacy.
Single machines got very large. Vertical scaling is often the cheapest, fastest and most reversible option, and it buys time to choose a partition key from real data.
More replicas means more availability.
Only against independent failures. Three replicas sharing a power domain give you one failure domain and three times the cost.
Microservices make teams faster.
Deployment isolation can, if the boundaries match the change patterns. If features routinely span services, the split adds coordination rather than removing it.
We will need to scale eventually, so we should build for it now.
Building for a load shape you have not observed usually produces the wrong partition key, which is more expensive to change later than the migration you were avoiding.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Four reasons: it does not fit, it must survive a failure, users are far away, or components must not affect each other. If you cannot name which one, do not distribute yet.
Practical
Write the requirement as a number with a horizon. Identify the dominant reason. Check its condition — does the work partition, are the failures independent, can reads be local, is there a shared dependency. Then pick the smallest topology that meets it, and set an alert against the number rather than against a dashboard.
Advanced
The four reasons pull in different directions, and a design that serves two of them well often serves the others badly. Partitioning for capacity spreads a request across more nodes, which raises the probability that any given request meets a slow one — capacity bought at the cost of tail latency. Replicating for availability adds writes that must be agreed, which costs write throughput. Geographic distribution improves read latency and makes every write-side agreement cross a continent. Recognising these as trades rather than as independent improvements is what separates a designed topology from an accumulated one.
Apply it
- ⚡ A product has 400GB of data, 2,000 reads per second, 30 writes per second, and a requirement to survive a zone failure. Which reason applies, and what is the minimal topology?
- 💬 A team says they need to shard the database. What do you ask before agreeing?
- 💬 Why does adding replicas not necessarily improve availability?
- 💬 Which of the four reasons cannot be deferred by buying a bigger machine, and why?