Containers & Images

Configuration Belongs Outside the Image

Application Image + Environment Configuration = Running Workload. One artifact promoted through every environment, with configuration injected at start — and never a secret in a layer, because a layer is forever and docker history reads it back.

▶ Run the lab

The question this answers

Infrastructure question

What belongs inside the image, what is injected at start, and where do secrets actually live?

Application requirement

The same checkout service must run against three databases, two payment providers and four log levels across development, staging and production — and the artifact that production runs must be the identical artifact staging tested.

What it provides

A single promotable artifact whose behaviour is a function of injected configuration, so an environment change is a restart rather than a rebuild, and a compromised image does not hand over a credential.

Application RequirementInfrastructure RequirementComputeNetworkStorageIdentityDeploymentScalingReliabilityObservabilitySecurityCostTrade-offs

One artifact, many environments

The rule is a small equation with large consequences: Application Image + Environment Configuration = Running Workload. The image contains everything that is the same everywhere — code, runtime, dependencies, sensible defaults. Configuration contains everything that differs — endpoints, credentials, feature flags, concurrency limits, log level. If an environment value is baked into a layer, you no longer have one artifact; you have three artifacts that happen to share a Dockerfile, and Build Once, Promote the Same Bytes stops meaning anything.

Configuration arrives in three shapes and they are not interchangeable. Environment variables suit small scalar values and are read once at start; they are visible in process listings and in orchestrator manifests, which makes them fine for a database *host* and wrong for a database *password*. Configuration files mounted at a path suit structured or large configuration and can be swapped without rebuilding. Secret references are the important one: the workload receives a pointer plus an identity, and fetches the value at start from a secret manager — so the value exists only in memory, can be rotated without redeploying, and every access is logged. See Secrets in Infrastructure and ConfigMap vs Secret — and the Honest Limit of a Secret.

A useful test: if changing this value requires a rebuild, it is in the wrong place. Environment configuration should be a restart, at most.

Environment baked in, credential in a layer, build argument recorded in metadata
FROM python:3.11-slim
ARG STRIPE_KEY                       # recorded in the image config — readable by anyone
ENV DATABASE_URL=postgres://app:hunter2@prod-db.internal:5432/checkout
ENV STRIPE_KEY=${STRIPE_KEY}
ENV LOG_LEVEL=debug
COPY .env.production /app/.env       # a credential file, now a permanent layer
COPY . /app
CMD ["python", "app.py"]

# consequences:
#  - a separate image per environment; staging never tested these bytes
#  - rotating the Stripe key requires a rebuild and a redeploy
#  - the credential is in the registry, in every node's layer cache, forever
Defaults only in the image; endpoints injected, secrets fetched by reference at start
FROM python:3.11-slim
ENV LOG_LEVEL=info \
    PORT=8080 \
    HTTP_TIMEOUT_MS=3000            # safe defaults, identical everywhere
COPY . /app
USER 10001
ENTRYPOINT ["python", "app.py"]

# at start, injected by the platform — not present in any layer:
#   DATABASE_HOST=prod-db.internal          (env var: not sensitive)
#   DATABASE_PASSWORD -> secret://prod/checkout/db   (reference, resolved via workload identity)
#   STRIPE_KEY        -> secret://prod/checkout/stripe
#   /etc/checkout/features.yaml             (mounted file, swappable without rebuild)

# consequences: one artifact for every environment; rotation is a restart;
# an attacker who pulls the image gets code, not credentials.

The image on the right is environment-agnostic, so the digest staging tested is the digest production runs. Rotating a credential no longer touches the build pipeline, and a registry compromise no longer implies a credential compromise.

A layer is forever, and it is trivially readable

containers· Docker CLI shown for concreteness; any OCI tooling reads image config and layer contents the same way.

The most important sentence in this lesson: an image layer cannot be un-published. Layers are additive and content-addressed, so a RUN rm in a later instruction records a whiteout and ships the bytes anyway (What Is Inside a Container Image). Build arguments are worse than files, because they are stored in the image *config document* — no filesystem archaeology required, just one command.

The demonstration below takes about ten seconds and needs nothing more than pull access. Every node in the fleet has that. Every CI job that ever built on top of this base has that. If the registry was ever public, or a token ever leaked, everyone has it.

The response to a baked secret is therefore not "rebuild without it". It is incident response: rotate the credential first, then rebuild, then work out how far the image travelled — mirrors, laptop caches, node layer caches, backups. Prevention is a build-time secret mount that never becomes a layer, plus a CI scan that fails the build when a credential appears in a layer or in the config.

$ docker history --no-trunc registry.example.com/checkout:1.14.2
CREATED BY                                                          SIZE
ARG STRIPE_KEY=sk_live_51H9x2eKq7Vd8mNpR4bTc                        0B     <-- build arg, in the config
ENV DATABASE_URL=postgres://app:hunter2@prod-db.internal:5432/…     0B     <-- ENV, in the config
COPY .env.production /app/.env                                      412B   <-- a layer
RUN rm /app/.env                                                    0B     <-- whiteout only

$ docker save registry.example.com/checkout:1.14.2 | tar -x -O --wildcards "*/layer.tar" \
    | tar -x -O app/.env
STRIPE_SECRET_KEY=sk_live_51H9x2eKq7Vd8mNpR4bTc
DB_PASSWORD=hunter2

# The rm did not remove anything. The lower layer still ships the file.
# Everyone with pull access has had these credentials since the day it was pushed.
ILLUSTRATIVE. Reading a credential back out of a published image with pull access and nothing else.

Deciding where each value lives

Most configuration mistakes are placement mistakes rather than mechanism mistakes. The decision has three inputs: does the value differ between environments, is it sensitive, and does it need to change without a restart? Those three answers determine the home, and the table below is the whole decision.

One nuance is worth stating because teams get it backwards. A value that must change *without a restart* — a feature flag, a rate limit, a circuit-breaker threshold — does not belong in an environment variable at all, because environment variables are read once at process start. It belongs behind a configuration service or a flag system the application polls. Restarting the fleet to flip a boolean is a deployment, with a deployment's risk, for a change that was supposed to be cheap (Deployment Is Not Release).

ValueDiffers per env?Sensitive?Correct homeCost of baking it into the image
Default log level, port, timeoutsNoNoImage ENV defaultsNone — this is the right place.
Database host, queue URL, regionYesNoEnvironment variable at startOne image per environment; promotion becomes meaningless.
Database password, API key, signing keyYesYesSecret manager reference resolved via workload identityPermanent disclosure to everyone with pull access; rotation needs a rebuild.
TLS certificate and private keyYesYesMounted secret volume, or terminated at the load balancerA leaked key that outlives every rotation you will ever do.
Structured config: routing rules, tenant mapYesNoMounted configuration fileA rebuild for every routing change.
Feature flags, rate limits, thresholdsYesNoFlag or config service the app polls at runtimeA full deployment to flip a boolean.
Build metadata: commit SHA, build timeNoNoImage labelsNone — labels are exactly for this.
Where each kind of value belongs, and what happens when it is put in the image instead.

Key points

  • Application Image + Environment Configuration = Running Workload. If a value differs per environment, it is not in the image.
  • An image layer is permanent. A secret written into one has been published to everyone with pull access, and rm in a later layer changes nothing.
  • Build arguments are recorded in the image config document — docker history reads them back with no filesystem work at all.
  • Environment variables suit non-sensitive scalars; sensitive values belong behind a secret-manager reference resolved by workload identity at start.
  • A value that must change without a restart belongs in a flag or config service, not in an environment variable read once at boot.
  • The response to a leaked secret in an image is rotate first, rebuild second, trace distribution third — in that order.

The loop, answered

Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.

How it works
  • The image ships code plus non-sensitive defaults in its config document: env defaults, entrypoint, user, workdir.
  • At start, the platform merges deployment-supplied environment variables over those defaults and mounts any configuration files.
  • For secret references, the workload presents its identity to the secret manager and receives the value into memory — no credential is stored on disk or in the manifest.
  • The process reads configuration once at start; values that must change later are fetched at runtime from a configuration or flag service.
  • Rotation replaces the value in the secret manager; the workload picks it up on its next start or on its next fetch, with no image change.
  • Build-time secret mounts expose a credential to one instruction without recording it in any layer or in the config document.
What you still own
  • You own configuration schema and validation at start. A workload that boots with a missing variable and fails on the first request has deferred a startup error into an outage.
  • You own the environment-to-value mapping and its review. Configuration changes reach production faster than code changes and get less scrutiny — that asymmetry is where outages come from.
  • You own secret rotation, including whether the workload can pick up a new value without a full redeploy.
  • You own CI scanning for credentials in layers and config, because prevention is the only cheap control here.
  • You own the audit trail of who read which secret, which the secret manager gives you and an environment variable does not.
How it fails
  • A missing environment variable starts the container successfully and fails on the first request that needs it — a green deployment followed by a 100% error rate.
  • A staging endpoint is left in a production manifest, and production writes to the staging database for hours before anyone notices.
  • A rotated credential breaks every workload that cached it at start, because nothing was designed to re-read it.
  • A secret is baked into a layer and discovered months later; rotation is urgent, and the distribution trace covers mirrors, node caches and backups.
  • A configuration file mount shadows a directory that also contains application files, and the workload starts with parts of itself missing.
  • Environment variables leak into a crash log or an error-reporting payload, distributing the credential to a third party.
How it scales
  • Environment variables scale badly past a few dozen values: no structure, no types, no validation, and a manifest nobody can read.
  • Secret fetches at start scale with replica count, so a large rollout produces a burst of secret-manager requests — usually rate-limited.
  • A configuration service adds a runtime dependency on the start path; if it is down, new replicas cannot start even though existing ones are fine.
  • What runs out first is human: an environment matrix wide enough that no one can state the difference between staging and production is already broken.
Security
  • Never pass a secret as a build argument or an ENV instruction; both are recorded in the image config and readable with pull access.
  • Prefer a reference resolved by workload identity over a value injected as an environment variable — it gives rotation, revocation and an audit trail.
  • Environment variables are visible in process listings, in orchestrator manifests, and often in crash dumps and error reporting.
  • Scope each secret to one workload and one environment; a shared credential makes revocation an outage decision instead of a routine one.
  • Scan images for credentials at build and on a schedule — see The Infrastructure Supply Chain and the Security domain's secret-lifecycle material.
Cost shape
  • Secret-manager requests are metered per operation on most platforms; caching the value for the process lifetime keeps that small.
  • Per-environment images multiply build minutes and registry storage by the number of environments — a direct cost of getting this wrong.
  • The real cost of a baked secret is the incident: rotation across every consumer, plus the audit of where the image travelled.
  • A configuration service is a fixed cost that replaces a variable one — rebuilds and redeploys for values that should never have needed them.
What to watch
  • Startup configuration validation results, so a missing or malformed value fails loudly at start rather than quietly at request time.
  • Secret fetch success rate and latency, which is a start-path dependency and therefore a scale-out dependency.
  • A record of which configuration version each replica started with — the fastest way to explain why some replicas behave differently.
  • Secret-manager access logs: which identity read which secret, when. That is the audit trail an environment variable cannot give you.
  • The signal that lies: a successful rollout. It proves the containers started, not that they were given the right configuration.
Simpler alternatives
  • A plain mounted file with restrictive permissions, injected by the platform, when a full secret manager is more machinery than the team needs. Far better than baking it in.
  • Terminating TLS at the load balancer so the certificate and key never reach the workload at all — the simplest way to remove an entire class of secret.
  • For a single-environment internal tool, a .env file that is deliberately never committed remains a defensible answer; the failure mode is committing it, not using it.
  • Provider-native workload identity for provider services, which removes the credential entirely: no key exists to bake, leak or rotate. Prefer this wherever it is available — see Roles vs Static Keys.
What adopting this costs
  • External configuration buys one promotable artifact and costs a new start-path dependency that must be available for new replicas to boot.
  • Secret references buy rotation, revocation and audit; they cost a secret manager, a workload identity setup, and latency on every cold start.
  • Environment variables buy simplicity and cost structure, validation and any protection for sensitive values.
  • A runtime configuration service buys change without redeployment and costs a component whose outage prevents scale-out.

The lifecycle of one database credential

One database password, from creation to audit
The same credential, walked through seven stages twice: managed properly, and stored the way it usually ends up being stored.
Secret manager + workload identity Created
The database generates a credential for one role with the grants that role needs. A human never sees it — it is written straight into the secret manager by the provisioning step.
baked into the container image Created
A human creates it, copies it out of a terminal, and pastes it into a Dockerfile or a build arg. The value now exists in shell history and a text editor.
Where the value actually lives (anti-pattern)
a layer of every build of the image, in every registry and on every node that pulled it
Who can read it
anyone who can pull the image, and anyone who can read a cached layer on a node
If it leaksExposure windowHow you find out
static credential, no expiry
full database access from anywhere on the internet that can reach the endpoint
until someone notices and dares to revoke it — typically monthsa scanner, a bill, or a customer
short-lived token, 1 hour
the same access, for one hour, from one identity that the log names
at most 60 minutes, then it is refused with no action from youthe issuing log shows the unexpected request
Stage 1 of 7 — created. Track one property across the stages: how many copies of the value exist, and whether any system can name who read it. The managed path keeps exactly one authoritative copy and a log line per read; the baked into the container image path makes a copy at every step and produces no log at all. The stage that decides this is Requested by the workload — either the workload proves who it is and receives the secret at runtime, or the secret travels inside the artifact and there is nothing left to check. Step to Rotated to see the bill for that choice. The last row of the table is the real lesson: a leaked 1-hour token is an hour of exposure that the issuing log has already recorded, while a leaked static key is an open door with no clock and no record. Prefer an identity that gets credentials over a workload that holds one.
1/7 · Created
PROVIDER-NEUTRAL

What people believe, and what is true

Claim

Deleting the file in a later instruction removes the secret.

Reality

It records a whiteout. The bytes are in a lower layer and still ship on every pull. Rotate the credential; the image cannot be fixed retroactively.

Claim

Build arguments are safe because they are not in the filesystem.

Reality

They are stored in the image config document, which is exactly what docker history prints. They are the easiest secret to extract from an image.

Claim

A private registry makes baked secrets acceptable.

Reality

Every node, every CI job and every mirror has pull access. Private means access-controlled, not secret.

Claim

Environment variables are a secure way to pass credentials.

Reality

They are better than a layer and worse than a reference: visible in process listings, in manifests, and often in crash dumps and error reports.

Apply it