The question this answers
Should this workload run on one large virtual machine or several smaller ones, and what does each choice cost me?
A service that currently saturates its machine at peak must serve more load, and must keep serving when the machine it runs on disappears — which, on shared infrastructure, is a routine event rather than a hypothetical.
Capacity that is not bounded by the largest instance the provider sells, and a design in which the loss of any single machine removes a fraction of capacity instead of the service.
The two moves, compared honestly
Vertical scaling — making the machine bigger — deserves more respect than it usually gets. It requires no change to the application, introduces no distributed-systems problems, keeps every request in one process where a profiler can see it, and can be done in the time it takes to stop and start an instance. For a workload that is genuinely below the largest available size, it is frequently the correct engineering answer and is dismissed for cultural reasons rather than technical ones.
It has exactly two limits, and both are hard. The first is the ceiling: instance families stop at a size, and once you are on the largest one there is no further move, so the next capacity problem arrives with no cheap answer and no time to build one. The second is the failure domain: one machine is one host, in one zone, with one guest kernel, one root filesystem and one process to be OOM-killed. When it goes, the service goes, and the mean time to recovery is a full provision-boot-configure cycle — the minutes measured in The VM Lifecycle, during which you are entirely down.
Horizontal scaling removes both limits and charges for it. The application must tolerate running as several processes that do not share memory, which means sessions, caches, scheduled jobs, in-process rate limiters and anything that assumed a single writer all need answers — see Stateless vs Stateful Services in the Architecture domain. You acquire a load balancer, health checks that decide correctly, a fleet to keep identical, and a deployment strategy. That is not free, and the honest version of this lesson says so.
The dimension people compare on — cost per unit of compute — is usually the least decisive one, because it is roughly linear in both directions. What actually differs is the ceiling, the failure domain, and the operational surface.
| Dimension | 1 × 16 vCPU / 64 GB | 4 × 4 vCPU / 16 GB |
|---|---|---|
| Application changes needed | None | Statelessness, shared session and cache stores, single-writer jobs moved out |
| Capacity ceiling | The largest size in the family, then nothing | Effectively none; the next constraint is downstream |
| Loss of one machine | 100% of capacity — the service is down | 25% of capacity — the service is degraded |
| Time to recover from that loss | A full provision, boot and configure cycle, fully down throughout | None required immediately; the group replaces the instance while you serve |
| Zone failure | Total, if the machine was in that zone | Partial, if the instances are spread — see Multi-Zone Deployment |
| Deploying a change | Restart the one machine, or accept downtime | Roll through the fleet with no downtime — see Rolling Deployment and the Compatibility It Demands |
| Moving parts to operate | One machine | Load balancer, health checks, scaling group, image discipline |
| Debugging | One process, one log stream, one profiler | Distributed across instances; needs aggregation and request tracing |
| Cost per unit of compute | Roughly linear; very large sizes sometimes carry a premium | Roughly linear, plus the load balancer and the redundancy overhead |
| Right-sizing granularity | Coarse — the next size down may be half | Fine — add or remove one unit at a time |
Redundancy is a count and a placement
Two instances are not automatically redundant. If both are in the same zone, a zone event takes both; if the group is sized so that both are needed to carry peak load, losing one is an overload rather than a degradation; and if the health check cannot tell a broken instance from a slow one, the load balancer will keep sending traffic to the broken one and the redundancy is decorative. Redundancy is a count, a placement, and a working health check — all three, or none.
The useful discipline is N+1 with placement: decide how many instances peak load needs, add at least one, and spread them across zones so that the loss of a zone leaves enough. That means running spare capacity on purpose. It is not waste, it is the thing you are buying, and describing it as waste in a cost review is how availability quietly gets removed from a design — see High Availability and Failure Domains.
The topology below shows the honest version of a horizontally scaled tier, including the part that is usually left off the diagram: the database primary is still one machine in one zone. Scaling the application tier horizontally does not make the system redundant; it makes the application tier redundant and relocates the single point of failure somewhere less visible. That is progress, and it is not the same as being finished.
- Two instances in one zone are two instances, not two failure domains.
- If peak needs N and you run N, you have no redundancy — you have a fleet that fails as a unit under load.
- Spreading across zones adds cross-zone hops: a little latency on every request and, on most providers, a charge per GB.
- A load balancer that only lives in one zone reintroduces the failure domain you were removing.
- Health checks are load-bearing here: redundancy only works if traffic actually stops going to the broken instance.
What each shape does to the bill
The compute cost of one large machine and several small ones is close to linear on most providers, so the interesting differences are elsewhere. Horizontal scaling adds a load balancer, adds a shared store for the state that used to live in one process, adds cross-zone traffic, and — most significantly — adds the redundancy overhead: the capacity you run so that losing a machine is survivable. That overhead is the price of availability, and it should be labelled as such in a cost review rather than being discovered by whoever is asked to reduce spend.
Vertical scaling has a cost profile that is cheaper on paper and lumpier in practice. Instance sizes double, so right-sizing is coarse: a machine at 55% utilisation cannot move down a size without running at 110%. That means a large single machine is usually over-provisioned by construction, and the over-provisioning is invisible because there is only one number to look at. A horizontal fleet right-sizes in units and can shed capacity when demand falls, which is the mechanism Autoscaling depends on and which a single machine simply cannot do.
One asymmetry is worth stating plainly because it decides a lot of real cases. Scaling up is reversible and quick to try; scaling out requires application changes that are slow to make and, once made, are not undone. So the ordering that usually works is: right-size first, scale up while it is still cheap and easy, and start the horizontal work *before* the ceiling — because arriving at the largest instance size during a growth event means doing the hardest engineering under the worst conditions.
Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.
Key points
- Vertical scaling needs no application change and is often the right answer — until the ceiling, which arrives without warning and without a cheap next step.
- One large VM is one failure domain: one host, one zone, one kernel, one process. Losing it is an outage, not a degradation.
- Horizontal scaling removes the ceiling and the single failure domain, and charges statelessness, a load balancer, a shared state store and a fleet to keep identical.
- Redundancy is a count, a placement and a working health check. Two instances in one zone are two instances, not two failure domains.
- Scaling the application tier horizontally relocates the single point of failure to the database; it does not remove it.
- Scaling up is reversible and scaling out is not, so start the horizontal work before the ceiling rather than during the growth event that hit it.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • Vertical: stop the instance, change the virtual hardware profile, start it again. The guest sees a reboot and the service is down for the duration.
- • Horizontal: place instances behind a load balancer, which distributes requests across registered targets and removes any target whose health check fails.
- • A scaling group maintains a desired count, replaces failed instances, and spreads placement across the zones it is configured with.
- • Per-process state — sessions, caches, in-memory rate limits, scheduled jobs — moves to a shared store or is coordinated, because it is no longer in one place.
- • Capacity changes by adjusting the desired count; the constraint on how fast that helps is time-to-ready, not the policy — see Autoscaling.
- • Health checks that reflect the application's real ability to serve, since the whole design rests on traffic actually stopping at a broken instance — see Health Checks.
- • Image and configuration discipline, so that every instance in the fleet is genuinely the same machine — see Mutable Servers and Immutable Images.
- • A placement policy across zones, and a load balancer that is itself not confined to one zone.
- • Log and metric aggregation with a request identifier, because the single log stream you used to grep is now four.
- • A stated N+1 or N+N target reviewed as demand grows, rather than an instance count that was set once and never revisited.
- • The single large VM is lost and the service is down for a full provision-boot-configure cycle, with no partial capacity in between.
- • A workload reaches the largest instance size in its family during a growth event, and the only remaining option is the application work nobody has started.
- • A vertical resize is attempted and the new size is out of capacity in that zone, so the machine is stopped and cannot start again.
- • Four instances are spread across two zones, peak needs three, a zone is lost, and the remaining two are overloaded — redundancy that was never sized for the failure it was bought for.
- • A session that lived in process memory breaks the moment a second instance appears, and it presents as random logouts rather than as a scaling bug.
- • A health check that only proves the process is alive keeps a broken instance in rotation, and a fraction of requests fail steadily while every instance is reported healthy.
- • A scheduled job that ran once per machine now runs four times per interval, sending four emails or writing four rows.
- • Vertical scaling has a hard ceiling and requires a stop and start; plan the horizontal migration before you are within one size of it.
- • Horizontal scaling moves the constraint downstream almost immediately — database connections, a third-party rate limit or a provider quota, not compute.
- • Time-to-ready decides whether adding instances can answer a burst at all; for VMs that number is minutes, which usually means warm headroom rather than reaction.
- • The fleet's effective capacity is what remains after the largest single failure you have designed for, not the sum of the instances.
- • Very small instances bring their own trap: burstable families and per-instance fixed overhead mean sixteen tiny machines are not always equal to one large one.
- • More instances is a larger patch surface and more machines that must actually be identical; drift across a fleet is a security problem, not only an operational one.
- • The load balancer becomes the public edge and the instances move to private subnets — a genuine improvement, and the reason a public load balancer on 443 is the design rather than the finding. See Public Exposure, Read With Context.
- • A shared session or cache store is new sensitive infrastructure with its own access control, and it now holds data that used to be confined to one process.
- • Cross-zone traffic between tiers stays inside your virtual network but crosses a physical boundary; encrypt in transit rather than assuming the network is private.
- • A single large machine concentrates blast radius: one compromise is the whole service and, usually, one broad instance role that could reach everything the service could.
- • Compute cost is roughly linear in both shapes, so the decision is not really a cost decision — it is a ceiling and failure-domain decision with cost consequences.
- • Horizontal adds a load balancer, a shared state store, cross-zone traffic and the redundancy overhead that buys availability.
- • Vertical over-provisions by construction, because instance sizes roughly double and the workload sits somewhere between two of them.
- • Only the horizontal shape can shed capacity when demand falls, which is where autoscaling savings actually come from.
- • Per-instance saturation — CPU, memory, and the connection or thread pool — because a fleet average hides one instance at 100%.
- • Healthy target count against desired count, which is the number that says whether your redundancy currently exists.
- • Request distribution across targets: an uneven distribution usually means a drifted instance, a sticky-session setting, or long-lived connections pinned to one target.
- • Headroom expressed as capacity remaining after the largest designed-for failure, rather than as average utilisation.
- • The signal that lies: fleet-average CPU. At 40% average with one instance at 98%, the graph is calm and a quarter of your users are having a bad time.
- • Right-size before scaling in either direction. A large share of "we need to scale" turns out to be an unindexed query, an N+1 pattern or a misconfigured pool, and adding machines makes it more expensive without making it faster — see Right-Sizing Without Causing an Outage.
- • Scale up and stop there. If the workload fits comfortably within available sizes, has a maintenance window, and losing it for a few minutes is acceptable, one machine is a complete and defensible design.
- • Two instances rather than a scaling group. Most of the availability benefit comes from the first extra machine; autoscaling is a separate decision with its own complexity — see Autoscaling.
- • A managed container service or platform, which provides the load balancer, the replacement and the placement without a VM fleet to keep identical — often less work than doing this yourself.
- • Splitting by workload rather than by instance size: moving batch or report generation off the serving machine can remove the peak entirely, which is cheaper than scaling either way.
- • Vertical buys simplicity — one process, one log, one profiler — and charges a ceiling and a single failure domain.
- • Horizontal buys headroom and survivability and charges statelessness, a load balancer, shared state, cross-zone traffic and fleet discipline.
- • Redundancy buys the ability to lose a machine and charges continuous spare capacity that will look like waste in every cost review.
- • Many small instances buy fine-grained right-sizing and charge more machines to patch, observe and keep identical.
What people believe, and what is true
Horizontal scaling is the professional answer and vertical scaling is a beginner move.
Vertical scaling requires no application change, adds no distributed-systems problems and is frequently correct. Its two limits are real and hard, and neither is that it is unsophisticated.
Two instances means the service is highly available.
Only if they are in different failure domains, either one can carry the load, and the health check actually removes a broken one. Otherwise it is two instances that fail together.
Scaling the app tier makes the system redundant.
It makes the app tier redundant. The database primary is usually still one machine in one zone, and it is now a dependency of every instance you added.
Smaller instances are always more cost-efficient because they right-size better.
Finer granularity is a real benefit, and it is offset by per-instance overhead, burstable credit mechanics on the smallest sizes, and more machines to patch and observe.
Go deeper
Overview
Bigger machine or more machines. Bigger is simpler and has a ceiling and one failure domain; more removes both and requires the application to run as several processes.
Practical
Right-size first, then scale up while it is cheap, and begin the statelessness work before you are within one instance size of the ceiling. When you do go horizontal, size for N+1 across zones and check that the load balancer is not itself single-zone.
Advanced
Express headroom as capacity remaining after the largest failure you have designed for, not as average utilisation — that reframing is what stops a cost review from removing your redundancy. Then find the next constraint immediately, because horizontal scaling moves it downstream: database connections, a third-party limit or a provider quota, usually within one growth step.
Internals
The thing that makes horizontal scaling hard is not distribution, it is shared mutable state. Every piece of state in a single process — a session map, a cache, a counter used for rate limiting, a lock, a scheduler tick — was implicitly protected by there being exactly one of them. Adding a second process removes that guarantee everywhere at once, and the resulting bugs are all the same bug wearing different symptoms: random logouts, a cache hit rate that halved, a job that ran four times, a rate limiter that allows four times the intended rate.