Guardrails, Not Gates
A gate is a human approving everything; a guardrail is automation that makes the invalid action impossible. A gate scales as a queue, a guardrail scales as code.
The question, the obvious approach, and why it breaks
Every lesson starts where the work starts: an operational problem, a first attempt that is entirely reasonable, and the way production disagrees with it.
Something dangerous is possible. Do you require someone to approve it, or make it impossible to do by accident?
Every organisation eventually discovers a way to break production, and the reflex response is to require approval for the category of action that caused it — which is why mature systems accumulate approvals faster than they accumulate safety.
Add a review step. If a human has to look at every change of this kind before it goes out, the mistake cannot happen again.
The approver frequently cannot evaluate the change. Approving a deployment manifest you did not write, for a service you do not own, is a signature rather than a check (Review as a Gate).
- The approver frequently cannot evaluate the change. Approving a deployment manifest you did not write, for a service you do not own, is a signature rather than a check (Review as a Gate).
- The gate becomes a queue with a person's working hours as its throughput. Changes batch behind it, and batched changes are larger and riskier (Change Size: Why Small Changes Are Safer, and When They Are Not).
- Because the gate is expensive, people bundle more into each passage through it, which is the opposite of what safety wanted.
- Urgency defeats it. At 3am with an outage running, the gate is bypassed, and the bypass path is the one that was never designed or tested (Break-Glass Access).
- The gate does not prevent the invalid action. It records a human who did not stop it, which is useful for attribution and not for reliability.
What is actually happening
Underneath the tooling, which is the part that survives a change of tool.
- A gate asks a human to decide whether an action is acceptable. A guardrail encodes what "acceptable" means and refuses everything else, immediately, on every attempt.
- The economics diverge completely. A gate costs approver-time per change, so its cost grows linearly with change volume — it scales as a queue. A guardrail costs engineering time once, so its cost is amortised over every future evaluation — it scales as code.
- The safety diverges too. A gate has a variable hit rate depending on who is approving, how tired they are and how many they have seen today. A guardrail evaluates the same rule the same way at the tenth attempt and the ten-thousandth.
- The critical asymmetry: a guardrail catches only what it was written to catch, and is silent about everything else. A human might notice something unmodelled. That is the entire remaining case for human review, and it is a real one.
- Most controls are not either/or. The strong pattern is a guardrail that makes the common invalid action impossible, plus a narrow, deliberately human decision at the point where judgement genuinely applies.
The same risk, two controls
Take one concrete risk — a service deployed with no resource limits, which can starve everything sharing its node. Both controls below address it. They differ in when they act, who acts, and what happens on the four-hundredth deployment.
Engineer opens a deployment change
-> platform team review required
-> approver checks the manifest by eye
-> approves (it is 4pm and there are nine more)
-> limits were missing; nobody spotted it
-> incident three weeks later
-> response: add a second approverEngineer opens a deployment change
-> policy evaluates the manifest in CI
-> refused: "no memory limit set; see <rule>"
-> engineer fixes it in ninety seconds
-> admission control enforces the same rule
at the cluster, so the API cannot be
used to bypass CI
-> the failure mode is gone, not delegatedThe gate's hit rate depends on the attention of whoever is approving, and degrades exactly when volume is high — which is when mistakes are most likely. The guardrail evaluates identically every time and costs nothing per change. Note also what the gate produced after failing: another gate. That is how organisations end up with four approvals and the same incident rate.
Which risks convert, and what is left over
Most controls people ask for are mechanical questions in disguise, and mechanical questions convert cleanly. The residue — the rows where a human remains — is small, and worth protecting precisely because it is small.
The right reading of the last column is not "a human also approves". It is "this is the part of the decision a rule cannot make".
| Risk | As a gate | As a guardrail | What a human still decides |
|---|---|---|---|
| Public storage bucket | Security reviews every bucket change | Policy refuses public access at plan time and at the API (Policy as Code) | The rare, documented case that must be public — approved once, with an expiry |
| Missing resource limits | Platform reviews every manifest | Admission refuses workloads without limits (Requests and Limits) | Nothing. This is fully mechanical |
| Untagged resources | Finance chases owners quarterly | Provisioning refuses claims without owner and cost centre | Nothing |
| Unreviewed code reaching production | Release manager approves each release | Branch protection plus required checks (Protected Branches) | Whether this change is wise — which is code review, not release approval |
| Risky deploy at a bad time | Change advisory board approves the window | Automated freeze windows; canary with automatic rollback (Canary Analysis: Compared Against What?) | Whether to ship during a known-critical period at all |
| Schema change that could lock a table | DBA reviews every migration | Linter refuses known-dangerous statements; migrations run with a lock timeout (Zero-Downtime Migrations) | Whether this particular contraction is safe given what is still deployed |
| Deleting a production database | Manager approval | Deletion protection; two-step with a retention window | The deletion itself. Irreversible and rare: keep the human |
| Access to production data | Ticket per request | Time-boxed, scoped, audited elevation (Break-Glass Access) | Whether this access is warranted — often reviewed after the fact rather than before |
When a gate is the right answer
Gates are not a failure of imagination. There is a narrow band where a human decision is genuinely the correct control, and it is worth being able to name it, because that is what keeps the argument about a specific control rather than about a philosophy.
For this specific action, which control actually reduces the chance of a bad outcome?
when The rule is expressible, the action is frequent, and the invalid case is well understood.
cost Engineering effort up front; the rule covers only what it was written to cover.
when The rule cannot capture everything, but the consequence can be bounded — canary, flag, one tenant (Progressive Delivery: Exposure as a Dial).
cost Rollout complexity and the discipline to actually watch the canary.
when The action destroys data or cannot be undone, and it is rare enough that the latency does not matter.
cost A queue exists, but only on a path that should be walked infrequently.
when An external obligation requires that the person who authored a change is not the person who released it.
cost Real delivery latency. Mitigate by shrinking scope and by generating the evidence automatically (Change Management).
when Legitimately, as an interim control immediately after an incident.
cost Becomes permanent unless the rule is actually written. Give it an owner and a date.
when The action is reversible, low blast radius and frequent — most deployments.
cost Some bad changes reach production, which is acceptable exactly when rollback is fast and the radius is small.
How to do it properly
Most important first.
- Whenever an incident produces "we should require approval for X", first ask what rule would have made X impossible, and whether that rule can be evaluated automatically.
- Put the guardrail as early as the earliest place it can be evaluated, and enforce it at the last place it can be bypassed. Early is fast feedback; late is enforcement (Policy as Code).
- Reserve human decisions for the irreversible and the unmodelled: destroying data, a regulatory sign-off, a change nobody has a rule for.
- When a gate is genuinely required, give the approver the evidence needed to actually evaluate it — the plan, the diff, the canary result — or you have built ceremony (The Plan: Desired vs Current).
- Prefer limiting blast radius to limiting permission. A change that can only reach one percent of traffic needs far less approval than one that reaches everyone (Reducing Blast Radius).
- Count your gates periodically and ask, for each, when it last stopped something.
How much can this affect
Every production change has a blast radius. Stated as a scale so it is comparable between changes rather than adjectival — and paired with what actually contains it, because a wide scope with a real containment mechanism is a different situation from a wide scope with none.
This is a control over other changes, so getting it wrong is felt through them: too loose and invalid actions reach production, too tight and everything queues. Contained by shipping guardrails warn-only first and by keeping a recorded, time-boxed override path.
What can go wrong
- Approval theatre: a gate everybody passes, which adds latency and creates a false sense that the change was checked.
- Guardrails written so tightly that legitimate work is impossible, which produces bypass mechanisms nobody reviews.
- A guardrail whose bypass is a single flag, used routinely, so the rule is advisory in practice.
- Guardrails that only run in one place — for example only in the pipeline — while the underlying API remains reachable directly.
- Confusing "we have a policy" with "the policy is enforced". A document is not a guardrail.
- Removing all human judgement, so an unmodelled catastrophic action proceeds at machine speed because no rule forbade it (The Automation Trap).
- "Guardrails mean no human review." It means no human review of the *mechanical* question. A guardrail can confirm a deployment manifest is valid; it cannot tell you whether shipping this feature today is wise.
- "Gates are always bureaucracy." Some are load-bearing. Separation of duties in a regulated environment, and a final confirmation before destroying data, are gates worth their cost.
- "We have policies, so we have guardrails." A policy that is not evaluated by a machine at the point of action is documentation. Its compliance rate is unknown by construction.
- "The approver is responsible, so the risk is covered." Responsibility was reassigned; the failure mode was not removed. Those are different things and only one of them is reliability.
Operating it
Evidence is the signal, not the intention. Rollback is sometimes 'you cannot, and that is the point'.
- A deliberately invalid action is refused automatically, in seconds, with a message that says what rule and how to comply.
- For each remaining approval gate, someone can name a recent occasion when it changed an outcome.
- The bypass path exists, is used rarely, and every use is recorded and reviewed afterwards (Break-Glass Access).
- Median time from change to production is not dominated by waiting for people.
- A guardrail that turns out to be wrong blocks legitimate work immediately and visibly, which is the good failure mode — it is reported within minutes rather than discovered later.
- Ship new guardrails in warn-only mode first, measure what they would have blocked, then enforce. Rolling back to warn-only is a one-line change.
- Removing a gate is harder to reverse socially than technically: reinstating it after an incident is read as an admission. Decide with the failure data, not the last incident.
- Automate the rule, the evaluation, the message and the audit record.
- Do not automate the irreversible without a human at the last step: dropping a table, deleting a bucket with data, destroying a stateful resource (Destructive Migrations).
- Do not automate away the ability to override in an emergency. Make the override loud, time-boxed and reviewed rather than absent (Break-Glass Access).
- A guardrail is code: it must be written, tested, maintained and versioned, and it is a system that can itself be wrong.
- Guardrails encode today's understanding of danger. They do not generalise to the failure nobody has had yet.
- Enforcing early and everywhere means the rule exists in multiple places, and those places can disagree.
Where this applies
This domain is unusually tool- and organisation-dependent. These labels say what each claim is specific to, and what a different platform, provider or organisation does instead.
- ORG-SPECIFICWhere separation of duties is mandated, some gates are not optional and no amount of automation removes them. The useful move there is to shrink what the gate covers — approve the release, not every configuration value — and to give the approver machine-generated evidence so the approval is a decision rather than a signature.
- GENERALThe queue-versus-code scaling property holds anywhere: an approval's cost is paid per change by a person, an automated rule's cost is paid once by an engineer. That is arithmetic, not a preference.
Where the depth lives
This domain teaches delivery and operation, and hands the mechanism off to the domain that owns it.