Overprovisioning
The difference between reserve you decided to hold and capacity you bought because nobody knew the right size.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
When is unused capacity a deliberate reserve, and when is it just a number nobody has revisited?
Overprovisioning fails by costing money rather than by causing outages, so nothing pages anybody and it persists indefinitely.
Set generous resource requests and instance sizes. Capacity is cheaper than an incident, and a service with room to breathe will not surprise anyone.
The generous number becomes permanent. It was chosen before the service existed, and there is never a moment that forces a review.
- The generous number becomes permanent. It was chosen before the service existed, and there is never a moment that forces a review.
- Slack hides regressions. A change that doubles memory per request or triples CPU per request produces no symptom at all until the margin is gone, at which point the cause is a year of commits away (Change Correlation).
- It multiplies. Per-instance generosity times fleet size times environments times replicas is a large number nobody ever decided on.
- On a scheduler, oversized requests reduce the density of the cluster: nodes fill with reservations rather than with work, so you pay for nodes that are mostly idle (Requests and Limits).
- It substitutes for understanding. A team that never found its real resource profile cannot answer capacity questions, cannot size a failover, and cannot tell whether a change made things worse (Building a Capacity Model).
- It is not even reliable reserve, because it is unowned: the next person to look at the graphs sees waste and cuts it, without knowing which claims were on it.
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- Headroom and overprovisioning look identical on a utilisation graph. The difference is entirely in whether the unused capacity has a named claim — burst, failure, deployment, degradation — and a policy that sized it (Headroom).
- Right-sizing means setting the allocation from measured usage plus a stated reserve, rather than from a guess plus comfort. The measurement is the part that is usually missing.
- The two allocation numbers behave differently. What is reserved determines cost and scheduling density; what is capped determines failure behaviour. Confusing them produces either expensive idle reservations or unexpected kills (How Resource Settings Go Wrong).
- Overprovisioning is genuinely the safest failure mode available: it fails towards availability and away from outages. That is why it persists, and it is also why the correction has to be careful, because every reduction moves the system towards the dangerous side.
- Right-sizing is a capacity reduction. It deserves the same treatment as any other risky change — measure, canary, soak, watch the constraint you are shrinking, and be able to revert (Canary Analysis: Compared Against What?).
- The interaction with autoscaling is easy to get backwards: oversized units make each scaling step coarser and more expensive, so scaling reacts in bigger, blunter increments (Autoscaling).
The same graph, two different situations
A utilisation graph cannot distinguish these. Only a document can, which is the argument for writing the reserve policy down: it is the only artifact that makes unused capacity legible to the next person.
allocation chosen at project start
from a template, or from comfort
never measured against real usage
no stated claim on the unused part
outcome:
cost multiplied across the fleet
regressions absorbed invisibly
the next reviewer cuts it blindallocation = measured peak usage
+ burst claim
+ failure claim
+ deployment claim
recorded with the measurement date
outcome:
the same graph, but defensible
a regression shows as a shrinking margin
the next reviewer knows what they would removeBoth fleets may run at identical utilisation and identical cost. The difference is that one of them can answer "what happens if we cut this?" — and that answer is the only thing standing between a cost review and an outage.
How right-sizing goes wrong
Every row here is a reduction that looked justified on the graph it was derived from. The pattern is the same throughout: sizing to a central tendency in a system whose failures live in the tail.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Memory sized to observed average | Periodic restarts under load | Peak resident usage is far above the mean, and the runtime is killed at the limit (OOMKilled: Over the Memory Limit) | Size from the peak of the distribution plus reserve, never from the mean |
| CPU sized to observed average | Latency rises with no error rate change | Throttling during bursts, invisible in average CPU (CPU Throttling: The Latency With No Error) | Watch throttling counters, not utilisation, after any CPU reduction |
| Reduction applied fleet-wide at once | Broad latency degradation with no single culprit | Reserve removed everywhere simultaneously, so nothing has slack to absorb anything | Canary and soak per service; never batch a capacity reduction |
| Sized during a quiet period | Failure at the next weekly or seasonal peak | The measurement window did not contain the peak it needed to cover | Require a full traffic cycle in the measurement window |
| Instance class reduced | Throughput plateau below the previous level | Network or IOPS allocation scales with instance class, not only CPU and memory | Check every dimension the class carries, not just the two on the invoice |
| Reduction succeeds, then regresses | The gap reappears months later | A service template or default restored the old value on the next service (Service Templates) | Fix the default, not only the instance |
Deciding how tight to run
How close to measured usage you should allocate is a per-service question, and answering it once per service is much cheaper than answering it repeatedly during incidents.
A service is allocated well above its measured usage. What is the right response?
when The service is on the critical path and the gap is roughly the reserve policy anyway.
cost Nothing changes financially; you have bought legibility, which is worth more than it sounds.
when The gap is large, the workload is well understood and measured over a full cycle.
cost Engineering time, and a period of elevated risk while the new size is proven.
when On a scheduler where the reservation drives cost and the cap only bounds the worst case.
cost Denser packing means a noisy neighbour can now affect this service (Requests and Limits).
when The workload parallelises and scaling steps are too coarse.
cost More units means more connections, more overhead per unit, and more scheduling churn.
when The gap appeared recently, without a deliberate change.
cost Slower, and it is the correct order: a sudden gap is information about the workload (Change Correlation).
How to do it properly
Most important first.
- Measure actual usage under real traffic, including peaks and including the cold period after a restart, before changing any allocation.
- Set allocations from measurement plus the stated reserve, and record which reserve figure you used, so the next person knows what they would be removing.
- Reduce one dimension at a time. Cutting CPU and memory together makes the resulting incident ambiguous.
- Canary the reduction on a subset, soak it through a full traffic cycle including a peak and a deploy, then extend it.
- Keep the distinction between reserved and capped explicit, and know which one your platform uses for scheduling (Requests and Limits).
- Review allocations when the workload changes, not on a calendar — and treat a large gap between allocated and used as a signal to investigate rather than an instruction to cut.
- Watch for the opposite error: a service with no slack at all, where every deploy is a capacity event.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
A fleet-wide allocation change has no natural containment; a canaried change to one deployment, soaked through a peak, has all of it.
What can go wrong
- Right-sizing based on average usage, so the service is correctly sized for the mean and dies at the peak.
- Memory reduced to just above observed usage, and the first garbage collection pause or large request exceeds it, producing kills that look like a code bug (OOMKilled: Over the Memory Limit).
- CPU allocation reduced to observed average, producing throttling that appears as latency with no obvious cause (CPU Throttling: The Latency With No Error).
- A fleet-wide reduction applied by automation in one action, removing the reserve for every service simultaneously.
- Right-sizing that succeeds and is then undone by the next unreviewed default in a service template (Service Templates).
- The mitigation failing: reserve is preserved but never labelled, so the following year someone else removes it as waste.
- "Low utilisation is waste." Some of it is the reserve you decided to hold. The question is not the number, it is whether a claim is attached to it (Headroom).
- "Right-sizing is free savings." It is a capacity reduction, and capacity reductions cause outages when they are wrong. Free savings are idle resources with no users at all (Idle Capacity).
- "Autoscaling makes sizing irrelevant." Autoscaling changes how many units you run; the size of each unit still determines cost, scheduling density and how coarse each scaling step is.
- "Set limits generously; it costs nothing." On a scheduler, what you reserve is what you pay for and what blocks other work, whether or not you use it.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- Measured usage distributions — not averages — for each dimension being sized, covering a peak.
- A recorded allocation decision naming the measurement it came from and the reserve it added.
- After a reduction: unchanged error rate and tail latency through a full traffic cycle, a deploy and a peak.
- Scheduling density, or the ratio of allocated to used capacity across the fleet, tracked over time.
- Restoring an allocation is usually a configuration change and fast — provided the platform can place the larger unit, which is not guaranteed on a cluster that has since filled up with other work.
- Rollback for a downsized stateful resource is often not symmetric: growing a database instance or a volume back may require a maintenance window or a migration (Why Stateful Workloads Are Harder).
- Keep the previous allocation in version control with the reason it was changed, so reverting is a revert rather than a fresh guess (Infrastructure as Code).
- Automate the measurement and the recommendation — usage distributions and a suggested allocation are exactly the kind of arithmetic nobody should do by hand.
- Automate detection of extreme gaps between allocated and used, as a ticket for the owning team.
- Do not automate the application of reductions fleet-wide. A recommender that applies its own advice is a capacity-reduction robot with no canary, and its worst day is everyone's worst day (The Automation Trap).
- Right-sizing converts a cost problem into a reliability risk. That is a real trade and it should be made service by service, with the critical path treated more conservatively than a batch job.
- Tight allocations make the system more sensitive to workload change, so they raise the ongoing cost of vigilance even when they lower the bill.
- Measuring properly costs telemetry and time, and for small services the measurement can cost more than the capacity it saves.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALAllocating more than is needed happens on every platform. What differs is the cost mechanism: on VMs you pay for the instance size, on a scheduler you pay for the nodes your reservations force, and on serverless the per-invocation memory setting sets both cost and available CPU.
- KUBERNETES-SPECIFICThe requests-versus-limits split is Kubernetes vocabulary: requests drive scheduling and effective cost, limits drive throttling and kills. A VM autoscaling group has one number — the instance type — so oversizing costs money but never causes the throttling surprises a low CPU limit produces; a serverless function has a single memory setting that scales CPU with it, so under-allocating shows up as latency rather than as an eviction.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — Testing & Reliability Engineering — treating a capacity reduction as a change that needs evidence rather than as a cleanup.