The question this answers
What exactly does a plan tell me, and which line in it means production is about to be destroyed and rebuilt?
Every infrastructure change must be previewable and approvable before it touches production, because the resources involved — a database, a load balancer, a network — cannot be rolled back by reverting a commit.
A per-resource statement of intent, computed from the current world, that can be attached to a pull request, read by a reviewer, and applied unchanged.
Current 2, desired 3: the simplest possible plan
Start with the trivial case, because everything else is a variation on it. Reality has two application servers. The configuration says count = 3. The plan says: one resource will be created, nothing will change, nothing will be destroyed. Apply, and there are three. Run plan again and it is empty, because the desired state is now true. That empty second plan is the whole point of the model.
Now vary it. Change count to 1 and the plan says two will be destroyed — and which two is determined by index, which is why counted stateful resources are a trap. Change the instance size and, depending on the provider, you get either an update in place or a replacement. The plan is the only place that difference is visible before it happens.
The discipline that follows: a plan is generated in CI on the pull request, posted where the reviewer reads it, and the apply uses *that saved plan file* rather than recomputing. Recomputing at apply time means the thing applied is not the thing approved — the world may have moved in between.
- 1Configuration change
A pull request changes
count = 2tocount = 3, or an instance type, or a security rule.A change whose real effect is not obvious from the diff — most force-replacement changes look like one-word edits.
- 2Plan in CI
The pipeline refreshes state, computes the delta, prints it and saves it as a file.
Refresh fails on expired credentials, or the plan is 900 lines and nobody reads past the first screen.
- 3Human review
A reviewer reads the plan alongside the diff, and specifically looks for destroy and replace lines.
This is the step that gets skipped under time pressure, and it is the only step that catches a replacement.
- 4Apply the saved plan
The pipeline applies the exact plan that was approved, in dependency order.
Applying a freshly computed plan instead: the approved change and the executed change are no longer the same.
- 5Verify the workload
Health checks and application metrics confirm the thing on top of the infrastructure is actually serving.
Treating
apply completeas verification. It means the API calls returned, nothing more.
The symbols, and the one that ends careers
There are four actions and they are not equally dangerous. + create is usually safe and always billable. ~ update in place changes an attribute on the live resource; disruptive or not depending on what it is. - destroy removes it. And -/+ — replace, sometimes printed as "must be replaced" — destroys the existing resource and creates a new one. On a stateless instance behind a load balancer that is routine. On a database, a persistent volume or an object store bucket, it is data loss with an audit trail.
What makes replacement dangerous is that it is rarely what the diff looks like. Changing an availability zone, a name, a subnet, an encryption setting, a disk type — any argument the provider marks force-new — turns a one-line edit into a destroy. The plan tells you, in a line that says # forces replacement, and it is frequently in the middle of two hundred lines of unchanged attributes.
The practical defences, in order of how much they help: read the plan, every time, searching for destroy and replace before anything else; put a lifecycle rule preventing destroy on the handful of resources whose loss would be unrecoverable; require a second approver on pull requests that touch stateful resource paths; and make the pipeline fail the build when a plan contains a destroy unless a label explicitly authorises it. The last one converts a reading discipline into a mechanism, which is the only version that survives a busy quarter.
Terraform will perform the following actions:
# cloud_instance.api[2] will be created
+ resource "cloud_instance" "api" {
+ id = (known after apply)
+ instance_type = "std-2"
+ subnet_id = "subnet-0a91f3c7"
+ tags = { "role" = "api" }
}
# cloud_lb_target.api[2] will be created
+ resource "cloud_lb_target" "api" {
+ instance_id = (known after apply)
+ port = 8080
}
# cloud_db_instance.orders must be replaced
-/+ resource "cloud_db_instance" "orders" {
~ id = "db-orders-prod" -> (known after apply)
~ availability_zone = "eu-central-1a" -> "eu-central-1b" # forces replacement
engine = "postgres"
allocated_storage = 512
}
Plan: 3 to add, 0 to change, 1 to destroy.
# Read that last line again. "1 to destroy" is the production orders database.
# The pull request that produced this plan was titled "spread instances across zones".Apply is not atomic, and that is the part people are unprepared for
An apply is a sequence of API calls, not a transaction. If the eleventh of twenty calls fails — a quota limit, a permissions gap, a provider timeout — the first ten stand. State records them. There is no rollback, because "roll back" would mean destroying resources that may already have data or traffic on them, and the tool will not guess.
What you do next is run plan again. The second plan is computed from the new reality and tells you exactly what is left to do. That is the recovery procedure, and it works because the model is convergent: the tool does not care how the world got into this shape, only what the difference is now. The teams that handle partial applies badly are the ones that treat the failure as something to undo rather than something to continue from.
Two operational habits follow. Set a timeout and a lock on the pipeline apply job so an abandoned run does not hold state forever. And do not run apply from a laptop for anything shared, because a closed laptop mid-apply is a held lock plus a partial graph plus no log.
| Symbol | Action | Typical risk | What to check before approving |
|---|---|---|---|
+ | Create | Low, but it starts a meter | Is the size right? Defaults in modules are how bills grow. |
~ | Update in place | Usually low; occasionally a restart | Does this attribute cause a restart or a brief interruption? |
- | Destroy | High if it holds data or serves traffic | Was removing this from the configuration intentional, or a merge artifact? |
-/+ | Destroy then create (replace) | The highest — data loss on stateful resources | Find the # forces replacement line and decide whether that resource can be rebuilt. |
| (no symbol) | Unchanged | None | Nothing. This is the noise the dangerous lines hide in. |
Key points
- A plan is a per-resource statement of intent computed from current reality, and it is the only preview a declarative tool can give you.
-/+means destroy and recreate. On a stateful resource that is data loss, and it is usually triggered by a one-word change to a force-new attribute.- Apply the saved plan file, not a freshly computed one, or the change applied is not the change approved.
- Apply is not atomic: a mid-run failure leaves what succeeded in place, and the recovery is another plan, not an undo.
- Convert plan-reading from a discipline into a mechanism — fail the pipeline on unapproved destroys — because disciplines lapse under deadline.
The loop, answered
Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.
- • Refresh: the provider reads every managed resource so state reflects current attributes.
- • Diff: each resource's desired attributes are compared to refreshed ones, consulting the provider schema for which differences force replacement.
- • Graph: actions are ordered by dependency; a replaced resource forces re-evaluation of everything that references it.
- • Serialization: the result is written as a plan file that can be reviewed and applied later without recomputation.
- • Apply: actions execute in order, parallelised where the graph allows, with state written back as each completes.
- • You own the review gate: who must approve a plan containing a destroy, and how the pipeline enforces it.
- • You own lifecycle protections —
prevent_destroyon the resources whose loss is unrecoverable, accepting that removing them later becomes a two-step change. - • You own the apply job's timeout, lock behaviour and logs, because these are what you have during a partial-apply recovery.
- • You own the decision that some resources are not managed by the tool at all, when their replacement risk outweighs the reproducibility benefit.
- • A force-new attribute change that reads as a trivial diff and replaces a production database.
- • A plan recomputed at apply time that differs from the approved one because someone else changed the world in between.
- • A partial apply after a quota error, leaving a half-built graph and a state that describes it accurately but unhelpfully.
- • A plan too long to read, so the destroy line scrolls past — the most common way the review gate fails without anyone skipping it.
- • A stale plan file applied hours later, whose resource ids no longer exist, failing mid-apply on a resource that was already deleted.
- • Plan time grows with managed resources because refresh is per resource; this is what pushes teams to split state.
- • Plan *length* is the human-scaling limit and it binds sooner: a plan nobody finishes reading provides no safety at all.
- • Apply parallelism is capped by the graph shape and provider rate limits; raising it past the throttle converts speed into flaky failures.
- • The apply identity can destroy the estate. It should be assumable only by the pipeline, only from the protected branch, and only with a short-lived credential.
- • Plan output can contain sensitive attribute values. Posting a raw plan into a public pull request comment is a genuine leak path.
- • A required second approver on stateful paths is a security control, not just a safety one — it is the same separation-of-duties argument as a code review on authentication logic.
- • Every
+starts a meter. A plan is also a cost preview, and reading it as one catches the accidental large instance type before the invoice does. - • Replacements can cost double briefly if the create precedes the destroy, and can cost an outage if the destroy precedes the create.
- • Long plans have a productivity cost that pushes teams toward planning less often, which is the expensive failure mode.
- • Plan diff counts per pull request, tracked over time; a sudden jump in destroys is worth a look regardless of the reason given.
- • Apply duration and failure rate, with the failing resource type recorded — permission and quota failures cluster.
- • Whether the applied plan file matches the approved one, which the pipeline should assert rather than assume.
- • The signal that lies:
Apply complete! Resources: 3 added, 0 changed, 1 destroyed.It is a report about API calls. The workload health check is a separate question and it belongs to Health Checks.
- • For a change you can make safely by hand and reconcile immediately, the console plus a follow-up commit is sometimes the right call during an incident — as long as the reconciliation actually happens.
- • Provider-native change sets, which give the same preview with the state managed for you.
- • Policy-as-code gates that reject a plan automatically on rules you can state — no public database, no untagged resource, no destroy of a data store — for the classes of mistake a human reviewer misses in a long plan.
- • For a resource whose replacement risk is unacceptable, not managing it declaratively at all is a defensible choice. A hand-created production database with an import comment beats a plan that can destroy it.
- • Buys a preview of production change; costs a review step that is only as good as the attention paid to it.
- • Buys reproducible application from a saved plan; costs staleness — a plan is a statement about a world that keeps moving.
- • Buys destroy protection through lifecycle rules; costs a two-step dance whenever you genuinely do want to remove the protected resource.
Plan and apply: config, state, reality
Plan: 1 to add, 0 to change, 0 to destroy.
| Attribute | Changing it means |
|---|---|
| count | adds or removes whole resources — and with a list instead of a count, removing the middle element renumbers every one after it |
| instance_type | update in place: stop, resize, start. A restart, and anything on instance storage is gone |
| root_disk_gb (grow) | update in place, online |
| root_disk_gb (shrink) | ForceNew — no API shrinks a volume, so the tool destroys and recreates |
| image_id | ForceNew — the boot image is fixed when the machine is created |
| availability_zone / subnet_id | ForceNew — a machine cannot move networks |
What people believe, and what is true
A successful plan means the apply will succeed.
A plan is computed from the current world with the credentials it had at the time. Quotas, permissions, provider errors and concurrent changes all surface only at apply.
If the apply fails I can roll back.
There is no rollback. Resources created before the failure exist. The recovery is to plan again against the new reality and continue or converge back.
Small diff, small change.
A one-word change to a force-new attribute replaces the resource. The diff size and the blast radius are unrelated.