Idle Capacity
Resources that are running, billed, and doing nothing — and how to tell them from the reserve that is doing nothing on purpose.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
Which of the things you are paying for have no user, no traffic and no claim on them?
Creating infrastructure is easy, attributed and celebrated. Removing it is risky, unattributed and thankless, so the estate only ever grows.
Run a cleanup: list resources with no recent activity and delete them. The console has the data, and the savings are immediate.
Low activity is not the same as no purpose. A disaster recovery standby, a quarterly batch cluster and a failover reserve all look idle and are all doing exactly their job (Headroom).
- Low activity is not the same as no purpose. A disaster recovery standby, a quarterly batch cluster and a failover reserve all look idle and are all doing exactly their job (Headroom).
- Activity metrics are misleading for storage and network resources. A volume that is attached to nothing still holds the only copy of something, and a reserved address may be in a customer firewall rule.
- Deletion is frequently irreversible in a way that provisioning is not. Getting the resource back does not get the data back (Partial and Logical Data Recovery).
- Ownership is the blocker rather than detection. Finding candidates takes an afternoon; finding out whether anyone needs them takes weeks, and that is where cleanups die.
- A cleanup is a one-off. The estate regrows immediately unless the mechanism that creates unowned resources changes.
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- Idle capacity splits into categories with genuinely different treatment. Orphaned resources have no owner and no consumer. Idle by schedule resources have a purpose only some of the time. Idle by design resources are reserve with a named claim. Overallocated resources are in use but larger than needed (Overprovisioning).
- Only the first two are recoverable savings. The third is a reliability purchase and cutting it is a reliability change; the fourth is a sizing question, not a deletion question.
- Detection needs two signals, not one: no consumption and no owner. Either alone produces false positives, and false positives in this domain delete something load-bearing.
- The safe removal path is not delete — it is deny, wait, then delete. Stopping traffic or detaching a resource is reversible; deletion is not, and the interval between them is where you find out who was using it.
- The cause is almost always a provisioning path with no ownership requirement and no expiry. Fixing the estate without fixing that path means doing this again next year (Platform Engineering).
- Non-production is where most schedule-idle capacity lives, because it is sized like production and used during working hours only (Ephemeral Environments).
Four kinds of idle, four different actions
The classification is the whole method. Almost every cleanup incident is a resource from the third row treated as if it were from the first.
| Category | Looks like | Correct action | What happens if you get it wrong |
|---|---|---|---|
| Orphaned | No owner, no traffic, no references | Deny, soak, snapshot, delete | You delete something with a quarterly consumer |
| Idle by schedule | Busy in working hours, idle otherwise | Schedule it to sleep, with an override | An engineer is blocked at an inconvenient hour |
| Idle by design | Standby, failover reserve, burst headroom | Label it and leave it (Headroom) | You remove the reserve and find out during a failover |
| Overallocated | In use, well below its allocation | Right-size with a canary (Overprovisioning) | You cut into the peak and cause restarts or throttling |
The reclamation path
Every step before deletion exists to make the deletion boring. The order is not negotiable: classification before denial, denial before deletion, snapshot before both.
- 1Inventory
Enumerate resources with owner, creation date, and consumption signals.
fails by Missing resource types entirely — addresses, certificates, snapshots, registry images.
evidence A list a second person cannot add categories to.
- 2Classify
Assign each candidate to one of the four categories.
fails by Treating everything quiet as orphaned.
evidence Each candidate has a category and a reason recorded.
- 3Find the owner
Attempt attribution from labels, provisioning records and change history.
fails by Stopping at "no label" and concluding "no owner".
evidence An owner, or a documented attempt that failed.
- 4Snapshot
Take and verify a copy of anything holding data.
fails by Taking a snapshot nobody has ever restored (Restore Drills).
evidence A snapshot that was restored somewhere, at least once.
- 5Deny
Detach, stop, or block access without deleting anything.
fails by Denying in a way that is hard to reverse quickly.
evidence The resource is unreachable and the reversal is a single documented step.
- 6Soak
Wait long enough to cover the slowest legitimate consumer.
fails by A window shorter than the monthly or quarterly job that needed it.
evidence The soak period elapsed and is recorded, with any complaints logged.
- 7Delete
Remove the resource and record what was removed.
fails by Deleting the snapshot at the same time as the resource.
evidence An inventory entry showing what was removed, when, and by whom (The Audit Trail).
The soak step is the one that gets cut when someone wants the savings this quarter, and it is the only step that catches the mistakes the earlier steps missed.
Why the estate regrows
A cleanup that does not change the provisioning path is a treatment, not a cure. These are the mechanisms that produce orphans, and each one has a fix in the path rather than in the estate.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| A resource created by hand during an incident | An unlabelled instance running for a year | Emergency provisioning outside the normal path (Manual Production Changes) | Require a follow-up to adopt or remove anything created under break-glass |
| A project cancelled | A whole environment with no consumers | Nothing in the process removes infrastructure when work stops | Tie environment lifetime to the project, with an expiry by default |
| A migration completed | The old fleet still running alongside the new one | The decommission step was the last item and was never done | Make decommissioning part of the migration plan, with a date |
| An experiment | A cluster nobody remembers creating | Self-service provisioning with no expiry (Self-Service Infrastructure) | Default expiry on self-service resources, with an explicit extension |
| An owner leaves the company | Resources labelled with a person, not a team | Ownership recorded as an individual | Label with a team and a rotation, never an individual (The Ownership Record) |
| A deployment replaced a resource | Orphaned volumes and load balancers left behind | Resources created outside the declarative path are not cleaned up by it (Drift) | Manage lifecycle declaratively so removal is a consequence of the same change |
How to do it properly
Most important first.
- Require an owner label at creation and enforce it in the provisioning path, so an unowned resource cannot exist (Policy as Code).
- Give non-production resources an expiry by default, so unused ones disappear without anyone having to be brave (Ephemeral Environments).
- Classify before acting: orphaned, schedule-idle, reserve, or overallocated. Each category has a different correct action and mixing them is what makes cleanups dangerous.
- Use deny-then-delete with a soak period long enough to cover the slowest legitimate use — a monthly job needs more than a week.
- Schedule non-production to sleep outside working hours, with an easy override.
- Snapshot before deleting anything that stores data, and keep the snapshot beyond the deletion (Backup Operations).
- Label reserve explicitly so it is never a cleanup candidate. Unlabelled reserve is indistinguishable from waste and will eventually be removed by someone acting reasonably.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
Contained by the deny-then-delete soak and by a verified snapshot. A bulk cleanup script with neither has no containment at all.
What can go wrong
- Deleting a resource whose only consumer runs monthly or quarterly, and finding out at the end of the quarter.
- Removing a volume or snapshot that held the only copy of something not covered by the backup policy.
- Deleting an address, certificate or DNS record referenced by a third party you have no visibility into (DNS in Production).
- Shutting down a non-production environment that a shared platform service quietly lives in (Cost Drivers).
- Reclaiming spare cluster capacity that was, in effect, the failure reserve — so the next node loss has nowhere to place its workloads (The Scheduler, and Why a Pod Is Pending).
- The mitigation failing: a cleanup automation that deletes on an activity threshold with no owner check and no soak.
- "Idle means waste." Idle means not currently working. Reserve, standby and seasonal capacity are all idle and all earning their place (Headroom).
- "Low CPU means unused." Plenty of legitimate resources sit at low CPU: a standby database replicating, a queue consumer waiting, a load balancer with a nightly job behind it.
- "We can delete it and recreate it if needed." True for the resource, false for its data, its identity, its address and everything referencing it.
- "Automated cleanup solves this." Automated detection solves detection. Automated deletion without ownership and a soak is the mechanism that turns a cost project into an incident.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- A resource inventory with owner coverage tracked as a percentage, moving in the right direction.
- A recorded classification for each removal candidate, and a soak period that elapsed without complaint.
- Non-production spend that varies with the working week rather than being flat.
- A count of resources removed and, just as importantly, a count of removals reversed during soak — that number being non-zero is the process working.
- Deny-then-delete exists precisely to have a rollback: during the soak period, reversing is re-enabling. After deletion, there may be nothing to reverse.
- For anything holding data, the rollback path is a restore, which means the snapshot must exist and must have been verified before the deletion, not after (Restore Drills).
- Keep an inventory record of what was removed and when. During a later incident, "was there something here?" is a real question and the answer should not be archaeology.
- Automate detection and classification: unowned, unused, unattached and expired are all mechanically determinable.
- Automate expiry for anything ephemeral by construction — preview environments, test clusters, CI resources (Preview Environments).
- Automate the deny step, with an obvious reversal.
- Keep the delete step human for anything holding data or serving an external name. The saving is not worth an unrecoverable mistake, and this is the clearest case in the module of automation that should stop one step short (The Automation Trap).
- A soak period delays savings by weeks and is the entire reason the process is safe. Shortening it is where every bad cleanup story begins.
- Enforcing owner labels adds friction to provisioning, which is the cost of not having this problem later — and friction in the provisioning path is exactly what platform work is supposed to remove.
- Non-production schedules save real money and cost engineer time when someone needs an environment outside the window, so the override has to be genuinely easy.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- CLOUD-SPECIFICAssumes resources are created on demand and billed while they exist, which is what makes orphans expensive. On owned hardware an idle machine costs power and opportunity rather than an invoice line, so the equivalent problem is capacity allocated to a team that no longer needs it.
- ORG-SPECIFICWhether a platform team may delete another team's resources, and after how much notice, is a policy question that has to be agreed before the first cleanup rather than argued during it.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — System Design — decommissioning as an explicit phase of a migration rather than an assumed one.