Failure & Recovery in Production

Distributed Debugging: The Question Ladder

Which request? Which service? Which region? Which version? Which dependency? Which state transition? Six questions, in order. If your system cannot answer them for a single failing request, every incident is solved by guessing, and the guesses are shaped by whoever spoke first.

▶ Run the lab

The question this answers

The question

One request in ten thousand fails. How do I find out why, instead of guessing?

The guarantee — the property claimed, and its scope

The ladder guarantees a bounded search, not an answer. Each rung either eliminates a dimension or fails to, and a rung you cannot answer is not a dead end — it is the finding, because it names exactly which instrumentation is missing.

Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.

What a node knows — observation versus inference

Each service knows what it did with the requests it saw. No service knows what the request looked like before it arrived or after it left, and none knows which of its peers handled the same logical request. The whole difficulty of distributed debugging is that the object you want to reason about — one user’s request — exists nowhere as a whole. It has to be reassembled from fragments, and the only thing that makes reassembly possible is an identifier every fragment carried.

A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.

What guarantee?What does a node know?How does it work?What can fail?How does it fail?Where is coordination?What holds under failure?How does it recover?How would you know?What is the simpler thing?
debuggingobservabilityheisenbugsincident analysis

Six questions, in order

The ladder narrows a failure from "the system is broken" to a specific line of code, one dimension at a time. It is ordered so that each rung reduces the search space for the next.

Which request? Get one concrete failing request id. Not a rate, not a graph — one identifier you can follow. Almost all failed debugging sessions failed here: the team reasoned about aggregates for an hour, and aggregates cannot tell you *why*. Which service? In the trace for that request, which span carries the error, and is it the first error or an error propagated from below? Look for the deepest failing span, not the top one. Which region or zone? If failures cluster in one, the cause is environmental, not logical, and half the hypotheses die immediately. Which version? If failures appear only on instances running a particular build, you are done — this rung ends more incidents than any other, and it is answerable in seconds if version is a dimension on your telemetry.

Which dependency? Within the failing service, which downstream call failed, timed out, or returned something unexpected? Which state transition? The last and hardest: what state was the entity in when the failure occurred, and what transition was attempted? Distributed bugs concentrate here — a workflow that received a step-3 event while in step 1, a record updated by two writers, a retry that arrived after a compensation.

The rungs are cheap when the data exists and impossible when it does not. Which is the real content of this lesson: the ladder is a specification for what your telemetry must carry. Request id propagated end to end. Service and span identity. Region, zone and version as dimensions on every signal. Dependency calls as child spans. State transitions logged with before and after.

Each rung either eliminates a dimension or names the missing instrumentation
no id propagatedversion not a dimensionno transition log1. Which request? → trace / request id2. Which service? → deepest failing span3. Which region? → environmental vs logical4. Which version? → ends most incidents5. Which dependency? → child span, not a log line6. Which state transition? → before/after, and who else wroteCannot answer a rung? That is the finding.
ClientGateway / LBServiceWorkerDatabaseCacheQueue / LogObject storageCDNExternal system

Why aggregates cannot answer "why"

A 0.1% error rate is a fact about ten thousand requests and tells you nothing about any of them. The distribution hides the mechanism: those hundred failures may be one bad instance, one poisoned key, one tenant with an unusual payload, one region, one version, or a hundred unrelated causes. Every one of those produces the same number on the dashboard.

This is why the first rung is "get one request". A single complete trace of a single failing request contains more causal information than a month of aggregate metrics, because it preserves the ordering and the identity that aggregation destroys. Metrics tell you *that* and *how much*; only a trace or a correlated log set tells you *why*.

The corollary is a sampling problem. Traces are usually sampled, and uniform sampling keeps mostly successful requests — the ones you do not need. Tail-based or error-biased sampling, which decides after the request completes, is what makes rung one answerable for rare failures. If your sampler is uniform at 1%, a 0.1% failure mode is invisible one time in a thousand, and rung one fails for the exact failures you care about.

# aggregate view — true, and useless
checkout_error_rate  0.0011   (rising since 14:20)

# ladder view — 4 minutes of work, if the dimensions exist
1. which request?    req_8f3a21 (from an error-biased trace sample)
2. which service?    deepest failing span = inventory-svc, not checkout
3. which region?     97% of failures in eu-west-1; other regions clean
4. which version?    100% on build 4471; build 4470 instances clean
5. which dependency? inventory-svc -> stock-cache, connection refused
6. which transition? reserve() attempted on an entity already in RESERVED

finding: build 4471 changed the cache client's default port; only
eu-west-1 had finished rolling. Rung 4 alone would have ended it.
The same incident, seen as an aggregate and as a ladder

Heisenbugs: failures that vanish under observation

Some distributed bugs are timing-sensitive, and the act of observing them changes the timing enough to hide them. Attach a debugger and the race resolves the other way. Add a log line in the hot path and the window closes. Enable full tracing and the added latency reorders two messages that used to overlap. Reproduce it in staging with one instance and the interleaving that caused it cannot occur at all. These are heisenbugs, and distributed systems produce them abundantly because the timing is set by a network nobody controls.

The distributed variety has a particular shape: the failure needs two events to land in a specific order at a specific separation — a message overtaking another, a retry arriving during a compensation, a lease expiring between check and use, a leader change landing mid-write. The window may be milliseconds wide, and anything that widens or narrows it makes the bug appear or disappear. Concurrency owns this phenomenon as heisenbugs; what is different here is that you have no shared memory to inspect and no single scheduler to control, so the techniques that work inside one process do not transfer.

What does work: stop trying to reproduce and start recording. Always-on, low-overhead, structured records that carry the identifiers and the ordering information — request id, causal parent, sequence numbers, state before and after — so that the *next* occurrence is fully described without you being present. Sample by error rather than uniformly. Keep exemplars. Record the state transition, not just the error, because for this class of bug the error is far downstream of the cause.

And the second technique: reason about ordering with causality rather than time. If two events are related, record the relationship explicitly — a parent span id, a causal token, a version vector — because [[causal-ordering]] survives observation while wall-clock timestamps do not.

  • Symptom: fails reliably in production, never under a debugger, never in staging.
  • Cause shape: two events must land in a specific order within a narrow window.
  • Trap: adding instrumentation to catch it changes the timing that produces it.
  • Technique: record continuously with causal identifiers rather than trying to reproduce.
  • Technique: bias sampling toward errors, and retain exemplars of the failing case.
  • Technique: log state transitions with before and after, not just the resulting error.

The rung you cannot answer is the deliverable

When a rung is unanswerable, the incident produces a better artefact than the fix would have. "We could not determine which version" is a concrete, cheap, permanent piece of work: add version as a dimension to error metrics and spans. "We could not follow the request past the queue" means the request id is not propagated through the message envelope. "We could not tell what state the order was in" means transitions are not logged with before and after.

Each of those turns "improve observability" — an action item that never gets done because nobody knows when it is finished — into a specific change with a clear completion test: run the ladder again on the next incident and see whether the rung answers.

A useful discipline for postmortems: record which rungs answered and which did not. Over a handful of incidents the same rung fails repeatedly, and that rung is where the observability investment belongs. This is more reliable than a wish list, because it is driven by what actually blocked real diagnosis.

Key points

  • Six questions in order: request, service, region, version, dependency, state transition.
  • Rung one requires one concrete failing request id — aggregates cannot tell you why.
  • "Which version" ends more incidents than any other rung, and costs one telemetry dimension.
  • A rung you cannot answer names the missing instrumentation, which is a better output than a guess.
  • Heisenbugs vanish under observation: stop reproducing, start recording with causal identifiers.
  • Uniform trace sampling keeps the requests you do not need; bias toward errors.

The chain, answered

Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.

How it works
  • Obtain one failing request identifier, from an error-biased trace sample or an error log carrying the id.
  • Retrieve the full trace and find the deepest span carrying an error, not the topmost.
  • Filter the failure population by region and zone to separate environmental causes from logical ones.
  • Filter by build version to test whether the failure is a deploy.
  • Within the failing service, identify which downstream call failed and how.
  • Retrieve the entity’s state and the attempted transition at the moment of failure, plus any concurrent writers.
  • Record which rungs could not be answered, and treat each as a specific instrumentation task.
What can fail at the boundary
  • The request id is not propagated across an async boundary, so the trace ends at the queue.
  • Sampling dropped every example of the failing request class.
  • Clocks differ between services, so log ordering is misleading — see [[correlating-distributed-logs]].
  • Version, region and zone are not dimensions on error signals, so rungs three and four cannot be answered at all.
  • The failing state transition is not logged, only its consequence, so rung six can only be inferred.
  • Adding the instrumentation needed to see the bug changes the timing and the bug disappears.
How it fails — what an operator sees
  • Debugging by aggregate: the operator spends an hour comparing dashboards and forms three hypotheses, because no single failing request id was ever obtained.
  • Trace ends at the boundary: the operator follows a request through four services and loses it at a queue, because the id was not carried in the message envelope.
  • Wrong service blamed: the operator sees the topmost span in error and pages that team, while the actual first failure is three spans deeper and belongs elsewhere.
  • Sampled away: the operator has an exact request id from a customer report and finds no trace for it, because sampling is uniform at 1%.
  • Heisenbug disappears: the operator adds debug logging to the suspected path, the failure rate drops to zero, and it returns the moment the logging is removed.
  • Version invisible: the operator cannot tell whether failures are confined to the new build, and the deploy is rolled back on suspicion — sometimes correctly, always without evidence.
Where coordination is required
  • The ladder itself requires no coordination between services at runtime, but it requires prior agreement on identifier propagation — every service must forward the id it received, including across queues.
  • That agreement is the coordination cost of debuggability, and it is paid once, at design time, in the request contract.
  • Rung two crosses team boundaries, so span ownership metadata matters: a trace that names services but not owners still leaves you asking in a chat channel.
  • Nothing about the ladder should require the failing service to be healthy — telemetry that must be scraped from a saturated process is unavailable exactly when needed.
What still holds under failure
  • Trace and log collection is usually best-effort: under load the telemetry pipeline sheds first, so the data is thinnest during the worst incidents.
  • Sampling decisions made at ingress cannot be revised later, so a request not sampled is permanently unavailable.
  • Identifiers survive failure well — an id in a message envelope outlives the process that created it — which is why they are the most robust debugging asset you have.
  • State-transition logs written to the same store that is failing will be missing exactly the transitions you need.
How it recovers
  • Detect: an error-biased sampler guarantees an exemplar exists for every failure class, which is what makes rung one answerable.
  • Contain: when rung four implicates a version, roll it back — that is containment, and it is available before you understand the bug.
  • Recover: fix the identified transition or dependency, not the symptom the top span reported.
  • Reconcile: entities that took the failing transition may be in an invalid state and need explicit repair; the error rate returning to zero does not fix them.
  • Verify: re-run the ladder on a request of the same class and confirm each rung now answers.
How you would know
  • Request id propagated end to end, including through queues, retries and scheduled work.
  • Region, zone, instance and build version as dimensions on error signals and spans.
  • Error-biased or tail-based trace sampling, with retained exemplars per failure class.
  • Dependency calls as child spans, with status and duration — not as free-text log lines.
  • State transitions logged with entity id, before state, after state, actor and causal parent.
  • Per-incident record of which rungs answered — the empirical guide to where observability spend belongs.
When it helps
  • Rare failures in a call graph deeper than two hops, where no single service’s logs contain the answer.
  • Incidents where several plausible causes exist and the team is about to choose one by seniority rather than evidence.
  • Any system where deploys are frequent, since rung four resolves a large fraction of incidents almost immediately.
When it hurts
  • A single-service failure with a clear stack trace — walking six rungs is slower than reading the exception.
  • When applied as ceremony during an active outage where containment, not diagnosis, is the priority. Roll back first; run the ladder afterwards.
  • Systems too small to have the dimensions the ladder queries, where building them costs more than the incidents do.
Simpler alternatives
  • Roll back first and diagnose later: when a deploy correlates with the failure, containment beats understanding, and the ladder runs afterwards on retained telemetry.
  • Bisect by deployment or by feature flag: coarser than the ladder but often faster, and it needs no per-request instrumentation.
  • Reproduce in a controlled environment when the bug is deterministic — far cheaper than production forensics, and useless for heisenbugs.
  • Deterministic simulation replay for protocol-level bugs, where the whole execution can be replayed with the same interleaving.

One request in ten thousand fails. Six questions, in order.

One request in ten thousand fails. Six questions, in order.
Each rung either eliminates a dimension or it does not. A rung you cannot answer is not a dead end — it is the finding.
simplifiedThe candidate counts are an illustrative product of dimension sizes for one 14-service system. The point is the shape — each answerable rung divides the space, and the first unanswerable one is your observability roadmap — not the arithmetic.
instrumentation we actually have
#QuestionNeedsDivides byCandidates left
1✕ Which request?a request id generated at the edge and propagated across every boundary — including queue messages, retries and background jobs907.2M
2✓ Which service?that id on every service's structured log lines÷ 1464.8M
3✕ Which region?region and zone stamped on every error signal64.8M
4✕ Which version?build version stamped on every error signal64.8M
5✕ Which dependency?a span per outbound call, carrying status and duration64.8M
6✕ Which state transition?state transitions logged with the value before and the value after64.8M
candidates at the start
907.2M
after the ladder
64.8M
rungs answerable
1/6
first gap
Which request?
The ladder stops at Which request? — the id exists, but traces are sampled uniformly, so a 1-in-10,000 failure is almost never in the sample. Sampling by error rather than by rate is the fix, and it costs almost nothing. Without it the object you want to reason about — one user's request — exists nowhere as a whole, and every later question is asked about a population instead of an instance. Record, per incident, which rung failed: that list is your observability roadmap, and it is a much better one than any vendor's feature matrix. Meanwhile 64.8M candidates remain, which is the size of the guess you are being asked to make at 03:00.

What people believe, and what is true

Claim

The dashboard shows the error rate, so we know what is happening.

Reality

You know how much. Aggregates discard the identity and the ordering that carry the cause; one complete trace beats a month of them for answering why.

Claim

The service reporting the error is the one that is broken.

Reality

It is usually the one propagating an error from below. Look for the deepest failing span, not the topmost.

Claim

The bug is not reproducible, so we cannot debug it.

Reality

You do not need to reproduce it. You need the next occurrence to be fully recorded — identifiers, causal parents, state before and after.

Claim

Adding more logging will find the race.

Reality

For a timing-sensitive bug, added logging often closes the window and hides it. Record continuously and cheaply instead, and reason about causal order rather than log order.

Claim

We sample traces at 1%, which is plenty.

Reality

Uniform sampling keeps mostly successes. For a 0.1% failure mode it retains one failure in a thousand, so the first rung fails exactly for the requests you needed.

Go deeper

Only the levels this lesson can honestly fill — a missing level is a claim nobody had.

Overview

Which request, which service, which region, which version, which dependency, which state transition. Without those you are guessing.

Practical

Propagate a request id across every boundary including queues. Put region, zone and version on error signals. Sample traces by error, not uniformly. Log state transitions with before and after. Then record, per incident, which rung failed — that is your observability roadmap.

Advanced

The ladder is a search over a product space of dimensions, ordered by how much each one divides it. Version is early because it is usually binary and almost free to check; state transition is last because it is the largest space and needs the others to have narrowed it first. Reordering is legitimate when your priors differ — but the discipline of picking a dimension and eliminating it, rather than accumulating hypotheses, is what makes the search terminate.

Apply it

Build it, then break it
  • 🔧 Take your last incident and mark each rung of the ladder as answered or not. The first unanswered rung is your next observability task.
  • 🔧 Pick a request that crosses an async boundary and verify by inspection that the request id survives the queue, the retry and the dead-letter path.
Reason about this
  • Failures appear only on one instance of twelve, only for one tenant, and only after 14:00. Which rungs narrow this fastest, and in what order?
Interview questions
  • 💬 One request in ten thousand fails. Walk me from that fact to a line of code.
  • 💬 Which single telemetry dimension resolves the most incidents, and why that one?
  • 💬 A bug fails in production and never under a debugger. What do you do instead of reproducing it?
  • 💬 Your trace ends at a queue. What is missing, and where does it have to be added?