Promotion Between Environments
Promotion moves one immutable artifact forward and changes only its configuration — which is what makes the evidence from earlier environments mean anything at all.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
What exactly moves between environments, and what does each promotion gate actually prove?
If each environment builds its own version of the software, then every test result applies to an artifact that no longer exists, and the thing that reaches production has been tested nowhere.
Each environment has a pipeline that builds from its branch and deploys. Staging builds from develop, production builds from main. Same code, so same result.
It is not the same artifact. Dependency resolution, base images, build tool versions and timestamps differ between two builds of the same source, so the tested bytes and the shipped bytes are different objects (Reproducible Builds).
- It is not the same artifact. Dependency resolution, base images, build tool versions and timestamps differ between two builds of the same source, so the tested bytes and the shipped bytes are different objects (Reproducible Builds).
- The merge itself changes the code. What was tested on
developis not what exists onmainafter other changes land, and nobody re-tests the result. - When production breaks you cannot answer "was this exact thing tested" — which is the first question, and the answer is "something like it was".
- Rollback becomes a rebuild of an older commit, which takes minutes you do not have and can produce a different artifact than the one that was previously healthy (Rollback: Only Useful If It Is Actually Safe).
- A supply-chain compromise between builds is invisible: the artifact that passed scanning is not the artifact that shipped (The Delivery Chain as Attack Surface).
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- Promotion is a pointer move, not a build. One artifact is produced once, identified by content digest, and each environment is told to run that digest (Tags Versus Digests).
- Everything that differs per environment is therefore configuration, which is why the config lessons sit adjacent to this one: promotion only works if the artifact is environment-agnostic (Build-Time and Runtime Configuration).
- A gate between environments is an assertion that a specific class of evidence exists. The gate is only meaningful if it names the evidence — "integration tests passed against digest X" — rather than recording that time passed and someone clicked.
- The release record is what makes the chain auditable afterwards: which digest, from which commit, with which configuration, approved by whom, at what time (The Release Manifest).
Rebuild per environment versus promote one artifact
This is the single highest-leverage structural decision in the module. Everything the environment chain is supposed to establish depends on the thing that was tested being the thing that ships.
develop -> build #811 -> staging tests green against #811 merge to main (other changes land too) main -> build #814 -> production different deps resolved different base image never tested anywhere
commit a1b2c3d
-> build #811
-> app@sha256:9f3e...
staging runs sha256:9f3e...
canary runs sha256:9f3e...
prod runs sha256:9f3e...
rollback = promote the
previous digestIn the left column every test result describes an artifact that no longer exists, and the artifact serving users has been tested nowhere. The right column is what makes "staging passed" a statement about the running system rather than about a sibling of it (Promotion).
What each gate is asserting
A gate is only a control if it names its evidence. Written out this way it also becomes obvious which gates are ceremony — and which risks nothing in the chain is asserting at all.
- 1Build and identify
Produce one artifact from one commit and address it by content digest.
fails by Publishing under a mutable tag, so the identity can silently change.
evidence The digest, recorded against the commit, in the build record.
- 2Automated checks
Unit, integration and security checks run against that digest.
fails by Checks run against source in CI and never against the built artifact (Securing the Pipeline Itself).
evidence Check results keyed to the digest, not to the branch.
- 3Promote to staging
Point staging at the digest with staging configuration.
fails by Staging builds its own copy, breaking the chain at the first hop.
evidence Staging reports the same digest the build produced.
- 4Integration evidence
Exercise wiring, config plumbing and cross-service contracts.
fails by Passing while blind to data volume and concurrency — a real pass about a narrow property (Parity That Is Worth Paying For).
evidence Named checks against the digest, plus an explicit statement of what was not covered.
- 5Promote to canary
Run the same digest with production configuration on a small share of real traffic.
fails by Promoting to full traffic because staging was green (Why Local Success Predicts So Little).
evidence Canary metrics compared against a baseline, not just "it started" (Canary Analysis: Compared Against What?).
- 6Promote to production
Move the remaining traffic to the same digest.
fails by A gate with no attached evidence — a click, recorded as approval.
evidence A release record naming digest, commit, configuration version and approver (The Audit Trail).
Read the failsBy column as the audit. If more than one of them describes your pipeline, the chain is asserting less than the number of gates suggests.
Configuration promotes too — and that is where it goes wrong
Promotion is usually described as moving an artifact, which quietly leaves the other half of a running service unmanaged. The combination that runs in production is artifact plus configuration, and if only one of them was promoted through the chain, the combination has been tested nowhere (Artifact Plus Configuration).
1release: r-2026-08-26.32commit: a1b2c3d3artifact: app@sha256:9f3e7c21b4a8d5f60e19c3aa7b2d84f15c0e6a93d7b18f424 5promoted_from: staging6evidence:7 - integration-suite: pass (against sha256:9f3e...)8 - canary: 40m at 5%, error rate within baseline9 - migration: expand phase applied, contract deferred10 - not_covered: peak concurrency, real payment provider11 12config:13 version: cfg-2026-08-26.114 changed_keys: [PAYMENTS_TIMEOUT_MS]15 previous: cfg-2026-08-19.216 17rollback:18 artifact: app@sha256:5b1d0e88ff3a2c4790d6e15b8c73a2f04e91dbb6a3f27c5819 config: cfg-2026-08-19.220 blocked_by: noneThree fields do the work. not_covered states what the chain did not measure, so nobody reads six green gates as full coverage. changed_keys makes a config-only release visible as a release. rollback.blocked_by is the field that says, before you need it, whether going back is actually possible — an applied contract-phase migration would name itself there (Expand, Migrate, Contract).
How to do it properly
Most important first.
- Build once in CI, push by digest, and make every deployment reference the digest rather than a mutable tag (Build Once, Deploy Many).
- Make each gate name its evidence. "Promoted" should expand to a specific set of checks that ran against this exact digest.
- Promote configuration through the same review path as code, because a config-only promotion is a production change with its own blast radius (A Config Change Is a Production Change).
- Keep the promotion path identical for a rollback: rolling back is promoting the previously healthy digest, using the same mechanism, not a special emergency procedure.
- Record the environment diff at the moment of promotion, so the reviewer sees what the earlier environment did *not* cover (Environment Drift).
- Skip environments deliberately when the change class does not need them, and say so in the record. A documented skip is better than a ritual pass.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
Contained by whatever the final promotion into production uses — a canary or a progressive rollout bounds it, a straight promotion to all instances does not (Progressive Delivery: Exposure as a Dial).
What can go wrong
- Deploying by mutable tag, so
:latestor:stagingpoints at different bytes at different times and two environments silently run different code. - A promotion gate that is a human clicking a button with no evidence attached, which is ceremony rather than a control.
- Configuration promoted separately from the artifact, so the version running in production has never seen the configuration it is running with anywhere.
- A gate that blocks rollback as well as rollout, because the emergency path was not exempted — the control designed for safety becomes the reason the outage lasted longer (Break-Glass Access).
- Promotion that succeeds while nothing changes: the digest was already deployed, the pipeline reports success, and the fix everybody is waiting for never shipped (A Successful Deploy Is Not Evidence of a Healthy System).
- "Same commit means same artifact." Only if the build is reproducible, which by default it is not — dependency resolution and base images move underneath you (Dependency Pinning).
- "It passed every gate, so it is safe." It is as safe as the evidence those gates collected. A chain of environments that all lack production data volume produces four passes and no coverage of the risk (Parity That Is Worth Paying For).
- "Promotion is a deployment." Promotion makes an artifact eligible; deploying it and releasing it to users can be separate steps, and separating them is what makes progressive rollout possible (Deployment Is Not Release).
- "We cannot promote the same artifact because environments need different builds." That is usually build-time configuration that could be runtime configuration. It is worth the refactor, because it is what buys the entire evidence chain.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- The digest running in production appears in the staging deployment record and in the CI build record, unchanged.
- A rollback executed recently used the ordinary promotion mechanism, not a special path.
- Each promotion record names the checks that ran against that digest and the configuration it was promoted with.
- Asking "what is running in production and what exactly was tested" takes one query, not an investigation.
- Promote the previous digest. This is the cleanest rollback story in the domain, and it is the direct payoff of building once.
- It stops being clean when state has moved forward: a schema migration, a message format change or a consumed queue message can make the previous artifact unable to run (Version Coexistence: N and N+1, in Both Directions).
- It also stops being clean when configuration was promoted separately, because reverting the artifact leaves the new configuration in place — a combination that has run nowhere (A Config Change Is a Production Change).
- Automate the mechanics fully: build, digest, push, promote, deploy, verify, record.
- Automate promotion to lower environments on a green build; keep production promotion gated on a signal, a human, or both, depending on the change class (Continuous Deployment).
- Do not automate promotion of a change whose risk class the chain never measured. A migration or a config change should require someone to look at evidence the pipeline cannot produce.
- One artifact for all environments means no environment-specific build-time behaviour, so debug tooling has to be runtime-switchable rather than compiled in (Build-Time and Runtime Configuration).
- Strict gates slow delivery, and slower delivery means larger batches, which is its own risk (Change Size: Why Small Changes Are Safer, and When They Are Not).
- Digest-based deployment is less readable than tags. People want to see
v2.3.1, notsha256:9f3e..., so the tooling has to show both or nobody adopts it.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- GENERALBuild once and promote by identity applies to containers, binaries, packages and bundles alike. What changes is the identity primitive — a content digest, a checksum, or a signed package version — not the principle.
- PLATFORM-SPECIFICSome platforms build from source at deploy time, so each environment builds separately whether you want it to or not. There the equivalent discipline is pinning the source revision and the lockfile, verifying the resulting artifact hash between environments, and accepting that a same-hash guarantee is weaker than promoting one object (Build Environments).
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.
- — Testing & Reliability Engineering — which checks belong at which gate, and what a gate can honestly assert about the artifact behind it.