What an Environment Is For
Each environment is an instrument that measures some production properties and is blind to others — which makes an extra one a cost to justify, not a safety improvement to assume.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
How many environments should we have, and what is each one actually evidence for?
Teams accumulate environments one at a time, each added to solve a specific problem, and end up with a chain nobody can explain — where a change waits in four queues and the failures still land in production.
Have dev, test, staging, pre-prod and production. Each stage catches what the previous one missed, so more stages means fewer escapes.
Stages only catch what they can model. Four environments that all have small data and one test user catch exactly the same class of bug four times, and none of them catch the migration that locks a 200-million-row table.
- Stages only catch what they can model. Four environments that all have small data and one test user catch exactly the same class of bug four times, and none of them catch the migration that locks a 200-million-row table.
- Every environment is a system that must be built, configured, credentialled, monitored and paid for. An environment nobody maintains drifts, and a drifted environment produces confident, wrong answers (Environment Drift).
- Each stage adds latency between merge and production. Longer latency means larger batches, and larger batches mean each release carries more suspects (Change Size: Why Small Changes Are Safer, and When They Are Not).
- A long chain trains people to treat the final green tick as proof. The more stages a change passed, the less anyone questions the evidence any single one produced.
- Environments multiply the places a credential lives. Five environments means five copies of every integration credential, usually with the weakest access controls on the ones nobody thinks about (What Counts as a Secret, and Where It Must Not Be).
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- An environment is a complete running instance of the system — artifact, configuration, data, dependencies and infrastructure — that is not the one serving users.
- Its value is not "closer to production". Its value is the specific set of production properties it preserves: code paths, integration wiring, data volume, concurrency, dependency behaviour, cost behaviour, real user behaviour. Every environment preserves some and fakes the rest.
- So the useful design question is not "how many stages" but "which property does each stage measure that no cheaper stage already measures?" An environment that measures nothing new is pure cost.
- What genuinely cannot be modelled outside production — real traffic, real data shape, real dependency failure — has to be handled by controlling blast radius inside production instead: canaries, flags, progressive rollout (Progressive Delivery: Exposure as a Dial).
Environments as instruments, not as a ladder
A ladder implies each rung is closer to the truth. Instruments imply each one measures something and is blind to the rest, which is the accurate model and produces better decisions.
Read the matrix column-wise for what a given environment buys you, and row-wise for the uncomfortable answer: several rows are only measurable in one place.
| Property | Local | CI | Preview per PR | Staging | Production |
|---|---|---|---|---|---|
| Compiles, unit behaviour | Good | Good | Good | Good | Good |
| Integration wiring, config plumbing | Weak | Partial | Good | Good | Good |
| UI and product review by a human | Partial | None | Best here | Good | Late |
| Data volume and shape | None | None | None | Weak | Only here |
| Concurrency, locks, races | None | Weak | None | Weak | Only here |
| Dependency failure behaviour | Stubbed | Stubbed | Stubbed | Partial | Only here |
| Cost per request | None | None | None | Weak | Only here |
| Real user behaviour | None | None | None | None | Only here |
What an extra environment actually costs
The §28 warning is worth stating plainly: more environments do not automatically mean more safety. Each one is a running system with its own maintenance, credentials, drift and bill, and each one adds delay between merge and production.
The costs below are the ones teams consistently forget when adding an environment, because none of them appear on the day it is created.
- Maintenance: it needs the same upgrades, certificate renewals and dependency bumps as production, or it stops resembling it (Renewal: Automating the Thing That Expires).
- Credentials: another full set of secrets, usually with weaker controls than production because "it is only staging" (Secrets in CI).
- Drift: an environment nobody watches diverges silently, and then answers questions confidently and wrongly (Environment Drift).
- Latency: one more queue between a merge and a user, which grows batch size and therefore risk (Change Size: Why Small Changes Are Safer, and When They Are Not).
- Cost: compute, storage, managed services and data transfer, running whether or not anyone is testing (Idle Capacity).
- False confidence: the most expensive item, and the only one with no line on the invoice.
A team wants a new pre-production environment. What should you weigh?
when It measures a property no existing environment measures — real data volume, a real third-party integration, a real HA topology — and someone owns it.
cost A permanent maintenance and credential surface, plus delay on every change that must pass through it.
when The complaint is that staging is unreliable or contended rather than that it is missing a property.
cost Investment in something unglamorous; contention stays until the underlying cause is fixed.
when The demand is really for isolation — several people needing their own instance at once.
cost Everything must be codified first: infrastructure, config, seed data, secrets (Ephemeral Environments).
when The property in question is traffic, data or dependency behaviour, which no environment reproduces.
cost Real users are exposed to unverified code inside a bounded blast radius (Canary: One Percent, Then Five, Then Watch).
The shape of a defensible chain
A defensible chain is one where each hop adds evidence and the artifact does not change. Note that the artifact enters once and is promoted; only configuration differs per environment (Artifact Plus Configuration).
The diagram is deliberately short. Chains longer than this exist, and each additional hop should be able to name the property it adds.
How to do it properly
Most important first.
- Write down, per environment, the one sentence: "this exists to give us evidence about X". If you cannot finish the sentence, the environment is a candidate for deletion.
- Prefer few, well-maintained environments over many neglected ones. Two environments that are genuinely current beat five that are approximations.
- Promote one artifact through the chain rather than rebuilding at each hop, so the thing tested is the thing shipped (Build Once, Deploy Many, Promotion Between Environments).
- Cover the properties no environment models with in-production techniques and a bounded blast radius (Reducing Blast Radius).
- Give every environment an owner and a lifecycle. Unowned environments are how drift and stale credentials arrive.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
Nothing. A misunderstood environment model is what produces a full-traffic release backed by evidence that never covered the failing property.
What can go wrong
- Environment sprawl: instances created for a project that ended, still running, still holding credentials, still costing money (Idle Capacity).
- The "extra safety" environment that is never actually used as a gate, so it drifts until its results are noise.
- Environments that share a dependency — one database, one message broker, one third-party sandbox — so a load test in one degrades another.
- The pre-prod environment that becomes a second production because a team quietly started depending on it.
- "More environments means more safety." It means more surface. Safety comes from what an environment can actually measure, and adding a fourth copy of an environment that cannot model your data volume adds nothing but delay and cost.
- "Staging is a smaller production." It is a different instrument, not a scaled model. Some properties do not scale down at all — races and lock contention simply are not present (Why Local Success Predicts So Little).
- "We need an environment per team." Sometimes true, usually a symptom of environments that take too long to create. Fix the creation cost first.
- "Production is just the last environment." Production is the only one with users, irreplaceable data and cost. That difference in kind is why it gets different controls (Production Access).
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- For each environment you can name its owner, the property it is evidence for, its monthly cost, and the last time its results changed a decision.
- The count of environments has gone down at least once, deliberately.
- Incidents are not routinely followed by "staging did not have that" as a novel discovery — the gaps are already known and written down.
- Removing an environment is reversible if it is defined as code: delete the stack, keep the definition in the repository, recreate if the gap turns out to be real (Infrastructure as Code).
- Removing one that was configured by hand is not reversible. That is itself an argument for codifying before deleting.
- Automate environment creation and teardown so the count is a decision rather than an accident of history (Ephemeral Environments).
- Do not automate the promotion decision for changes whose risk the environment chain cannot measure — a migration or a config change needs a human reading the right evidence, not a green tick from an environment that never had the data.
- Fewer environments means less isolation between teams testing at the same time, which costs coordination.
- Investing in in-production safety instead of another environment means real users are briefly exposed to unverified code, bounded rather than avoided.
- A single shared staging environment is cheap and becomes contended; per-team environments remove the contention and multiply the maintenance (Cost Drivers).
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALThe instrument framing holds on any stack. What varies is which properties are cheap to reproduce: a stateless API is easy to stand up faithfully, a system built on a managed data warehouse or a third-party payment sandbox has properties no lower environment can reproduce at all.
- ORG-SPECIFICRegulated environments often mandate a separated pre-production stage with formal sign-off. That is a compliance requirement, not a reliability argument, and it should be labelled as such internally so nobody mistakes the sign-off for evidence.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — Testing & Reliability Engineering — which test tier belongs in which environment, and what each tier can establish.