The question this answers
This instance is far larger than its average usage — how much smaller can it safely be?
The service must serve its daily peak without degrading, must keep serving when one of three zones is lost, and must not be paying for eight times the memory it typically consumes. Someone has to produce a size that satisfies all three, and defend it.
A sizing derived from peak plus failover plus a stated margin, rather than from an average — and an explicit record of which requirement each unit of capacity is buying.
The dashboard that starts the conversation, read properly
The observation is real and worth acting on: instances are routinely provisioned far larger than they need, because somebody picked a size at the start, it worked, and nobody revisited it. Over-provisioning is genuinely one of the largest recoverable costs in most estates. The mistake is in how the resize is derived, and specifically in using the average.
Read the dump below in order. Average memory use is 4.1 GiB on a 32 GiB instance, which is where the "we could go eight times smaller" idea comes from. Then the peak is 11.2 GiB, which already kills the eight-fold claim. Then the daily batch reconciliation, which runs once and touches 19.8 GiB. Then the failover projection: with one of three zones gone, the survivors take 1.5× the load, and the memory that scales with concurrency goes with it.
The defensible size that comes out of the bottom of that reasoning is 16 GiB — a genuine halving, worth doing, and nothing like the eight-fold cut the first line suggested. This is the whole lesson in one worked example: the direction was right and the magnitude was wrong by a factor of four, and acting on the first number would have produced an OOM kill during the next zone failure, which is precisely the moment when you least want to lose a second instance.
instance: 8 vCPU / 32 GiB workload: order-api zones: 3
memory
mean over 30d 4.1 GiB <- "we are using 12% of it"
p95 7.4 GiB
peak (daily 09:00) 11.2 GiB
peak (monthly batch) 19.8 GiB <- runs once a month, on this instance
cpu
mean 0.9 vCPU
p95 3.1 vCPU
peak 5.6 vCPU
failover projection (lose 1 of 3 zones -> survivors take 1.5x)
memory at peak 11.2 x 1.5 = 16.8 GiB
cpu at peak 5.6 x 1.5 = 8.4 vCPU <- exceeds 8 vCPU already
sizing candidates
from the mean -> 8 GiB OOM-kills at every daily peak
from p95 -> 12 GiB survives normal days, dies during failover
from peak -> 16 GiB survives daily peak, tight under failover
peak x failover -> 24 GiB survives failover; monthly batch still risky
decision: 16 GiB + move the monthly batch off this instance,
and raise the zone count so the failover multiplier drops.The three numbers that actually size a workload
Sizing needs three inputs and the average is not one of them. The first is the peak the workload must serve without degrading, measured over a window long enough to include the periodic events — the daily digest, the Monday morning login surge, the month-end batch. A window shorter than the longest cycle in the business will miss the biggest number.
The second is the failover multiplier. If load is shared across N failure domains and any one may be lost, the survivors take N/(N−1) of the peak: 2× with two zones, 1.5× with three, 1.33× with four. This is the input people omit most often, and the omission is specifically dangerous because it fails during an event that is already an incident. A right-sizing exercise that trims every instance to its peak has quietly guaranteed that a zone failure becomes a full outage.
The third is the margin for growth and measurement error. Peaks grow, metrics are sampled and miss short spikes, and a garbage-collected runtime uses more memory under pressure than a steady-state graph suggests. A margin of 20–30% is a common shape; the important thing is that it is stated rather than smuggled in. The compare below shows the two derivations side by side — same workload, same data, and one of them causes an incident.
# "mean memory is 4.1 GiB, mean cpu is 0.9 vCPU,
# so a 2 vCPU / 8 GiB instance is plenty"
resource "instance" "order_api" {
instance_type = "2vcpu-8gb"
count = 6 # unchanged
}
# what happens:
# 09:00 daily peak needs 11.2 GiB -> OOM kill
# platform restarts the process -> peak load lands on 5 instances
# those cross the limit too -> cascade
# the saving was real for eleven hours.# peak 11.2 GiB / 5.6 vCPU (30d, includes daily 09:00 spike)
# failover x1.5 16.8 GiB / 8.4 vCPU (3 zones, lose 1)
# margin +20% 20.2 GiB / 10.1 vCPU
# monthly batch moved to its own scheduled capacity
resource "instance" "order_api" {
instance_type = "4vcpu-16gb" # down from 8vcpu-32gb: a real 50% cut
count = 8 # +2 so the failover multiplier is 1.33, not 1.5
}
# net effect: ~33% cheaper than the original, and it survives
# the daily peak, the monthly batch and the loss of a zone.The first version is derived from a number that describes no moment the system actually experiences. The second is derived from the worst moment the system must survive, states each step, and reaches a smaller instance *and* more of them — which lowers the failover multiplier, so the total capacity needed drops even though the instance count rises. Right-sizing is a shape decision, not only a size decision.
What right-sizing actually saves, and what it risks
Done properly, the savings are substantial and durable: instance families are priced roughly proportionally to their resources, so halving a size halves that line item, and the change persists without ongoing effort. Done from averages, the same exercise produces an outage during the next traffic event and a team that will not approve the next cost initiative.
Two structural moves usually beat pure resizing. Moving periodic heavy work — the monthly batch in the example — off the always-on instance onto scheduled capacity removes the largest peak from the sizing calculation entirely, which lets everything else shrink. And increasing the number of smaller instances lowers the failover multiplier, so a fleet of eight small instances needs proportionally less spare capacity than a fleet of four large ones. Both change the shape rather than the number, and both save more than trimming ever will.
The cost panel separates what the resize saves from what it must not touch. Note the last row: the cost of getting this wrong is not a line item. It is an incident during a zone failure, which is the scenario your entire redundancy budget was purchased to survive.
Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.
Key points
- Average utilization is not a sizing input. Peak, failover multiplier and a stated margin are.
- The failover multiplier is N/(N−1): 2× on two zones, 1.5× on three. Omitting it guarantees a zone failure becomes an outage.
- Measure over a window long enough to contain the longest business cycle, or the monthly batch will be discovered by an OOM kill.
- More, smaller instances lower the failover multiplier, so changing the fleet shape can save more than trimming each instance.
- Moving periodic heavy work onto scheduled capacity removes the peak that was forcing everything else to be large.
- Instance *family* is often a bigger lever than instance *size*: memory-heavy work on a compute-optimized family pays for unused vCPU.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • Collect resource metrics over a window covering the longest periodic cycle — at minimum a month if anything runs monthly.
- • Extract p95 and true peak per resource dimension, separately. CPU and memory peak at different times and size independently.
- • Apply the failover multiplier derived from the failure-domain count and the availability requirement.
- • Add a stated margin for growth and sampling error, and record the number rather than folding it in silently.
- • Choose an instance family whose resource ratio matches the workload's shape, then choose the smallest size in that family that meets the derived requirement.
- • Change one dimension at a time and verify against a real peak before proceeding.
- • Re-run the exercise quarterly and after any significant traffic change; a size that was right last year is now either wasteful or dangerous.
- • Verify at the peak, not at deploy time. A resize that looks fine at 14:00 proves nothing about 09:00.
- • Keep the derivation next to the configuration — peak, multiplier, margin — so the next person can re-derive rather than guess.
- • Watch memory-limit headroom and CPU throttling after every resize; those are the first signals that the cut went too far.
- • Resize one workload at a time in production. Batch resizes remove the ability to attribute the resulting incident.
- • Check attached storage and provisioned IOPS at the same time; they are sized once and forgotten more often than instances are.
- • OOM kill at the daily peak on an instance sized from the mean, followed by a restart cascade as load concentrates on the survivors.
- • CPU throttling that presents as latency with an idle-looking host — the classic misdiagnosis after an aggressive CPU-limit cut.
- • A zone failure that takes down the surviving zones because every instance was trimmed to its own peak with no failover allowance.
- • The monthly batch that was never in the measurement window, discovered on the 31st.
- • Disk or IOPS reduced alongside memory, so the workload becomes I/O-bound in a way that never appears in a CPU or memory graph.
- • A vertical resize requiring a restart, applied to a stateful workload with no drain step — the resize itself causes the outage.
- • Horizontal scaling changes the calculation: with more instances, per-instance peak falls and the failover multiplier improves, so both dimensions get cheaper.
- • Vertical limits arrive eventually — the largest instance in a family is a ceiling, and a workload approaching it needs to be split, not grown.
- • Memory usually runs out before CPU for application workloads, because memory cannot be over-committed the way CPU can.
- • Autoscaling reduces the need for headroom on each instance and moves the risk to scaling latency instead — see Startup Time & Cold Start.
- • A resource-exhaustion attack is easier against a tightly-sized instance; margin is part of what absorbs abuse before rate limiting engages.
- • Instances sized with no headroom cannot run additional workloads during an incident — an agent, a debugger, a forensic collector — exactly when needed.
- • Vertical resizes usually mean replacement, which is an opportunity to redeploy from a current image and pick up patches. Treat it as one.
- • Instance cost is roughly proportional to resources, so a halved size is roughly a halved line item, persisting with no further effort.
- • Family selection can beat size selection: paying for a resource ratio the workload does not have is a permanent overhead.
- • Attached storage and provisioned IOPS bill for what is provisioned, not what is used, and are revisited far less often than compute.
- • The counterfactual dominates the arithmetic: an outage during a zone failure costs more than a year of the saving being pursued.
- • Peak and p95 per resource dimension over a full business cycle, kept as standing panels rather than pulled ad hoc.
- • Headroom to the memory limit at peak, which is the number that predicts the next OOM kill.
- • CPU throttled periods after any CPU reduction — the signal that a cut went too far, and one that host CPU graphs hide.
- • Projected post-failover utilization, recomputed whenever instance count or size changes.
- • The signal that lies: mean utilization. It is the number that starts every right-sizing conversation and the one that must never end it.
- • Autoscaling instead of resizing: let the fleet size follow demand and stop trying to pick one number that fits every hour.
- • Move the periodic peak onto scheduled or interruptible capacity — usually a larger saving than resizing, with less risk.
- • Change the instance family before changing the size when the resource ratio is wrong; it saves money without reducing any dimension the workload uses.
- • Fix the workload: a memory leak or an unbounded cache is a code problem being paid for in infrastructure, indefinitely.
- • For a small fleet, do nothing. Two modest instances are not worth a week of analysis and an OOM risk.
- • Buys a durable, proportional reduction in the largest line item; costs measurement effort and real outage risk if derived from the wrong statistic.
- • Tighter sizing improves utilization and reduces the buffer that absorbs traffic anomalies and failover load.
- • More smaller instances improve the failover multiplier and add per-instance overhead, more network hops and more things to operate.
- • Frequent resizing keeps costs optimal and introduces change into a system where every restart is a small risk.
Where the bill actually comes from
fixed weight is committed at provision time; usage weight follows the workload. idle = 100% − 35% used → headroom 25% (chosen) + waste 40% (not chosen)
What people believe, and what is true
Average utilization is 12%, so we can shrink by eight times.
Averages describe no moment the system experiences. The peak, the periodic batch and the failover multiplier all sit above it, and each one caps how far you can cut.
We are running across three zones, so failover is handled.
Redundancy only helps if the survivors have room for the extra load. Instances trimmed to their own peak turn a zone failure into a full outage.
Right-sizing means smaller instances.
It means the correct shape. Frequently the answer is more, smaller instances — which improves the failover multiplier — or a different family with the right resource ratio.
A month of metrics is enough.
Only if nothing in the business runs quarterly. The measurement window must contain the longest cycle, or the largest peak is not in the data.