The Critical Path Is the Only Path That Pays
In a fan-out, only the slowest branch controls when the request finishes. Optimizing any other branch produces a beautiful graph in your dependency dashboard and zero improvement for users — until the critical path moves, and then a different branch matters.
Frame the diagnosis
Performance work starts from a symptom and a signal — never from a resource dashboard.
Only one branch is holding the request
When a request fans out to three services concurrently and waits for all three, its completion time is set by max(branch), not by their sum and not by their average. The two faster branches have slack: they finish and then wait. Slack is free time, and time spent optimizing something with slack is time spent producing no user-visible change.
In the waterfall below, shipping returns at 112 ms and inventory at 192 ms. Rendering cannot start until 192 ms regardless of what shipping does. Cut shipping to zero and the request still takes 314 ms. Cut inventory by 50 ms and the request gets 50 ms faster, exactly. That asymmetry is invisible on a per-service dashboard, where both services look equally like "a dependency of checkout".
This is why "we optimized the pricing service" is not by itself a performance result. The result is what happened to the endpoint's p99, and the honest version of the claim names which path the optimization was on.
The path moves, so the answer expires
Fix inventory and the critical path immediately relocates. Take inventory from 120 ms to 30 ms and the slowest branch becomes pricing at 64 ms: the request improves by 56 ms, not 90, because pricing was hiding behind inventory the whole time. The remaining 34 ms of your "90 ms improvement" was never available. Predicting this before shipping is the difference between a credible estimate and an embarrassed retrospective.
This is The Bottleneck Moves After Every Fix in miniature, and it has a practical consequence: performance work should be planned one hop at a time, with the trace re-read after each change. A roadmap that promises "optimize inventory, pricing and shipping for a 224 ms saving" is adding numbers that cannot be added.
It also means slack is a property of *today's* numbers, not a permanent label. The shipping service with 80 ms of slack is one dependency regression away from becoming the critical path, and nothing about it will look different until it does. Slack is a reason not to optimize now, not a reason to stop measuring.
| Change | Branch time after | New slowest branch | End-to-end saving |
|---|---|---|---|
| Shipping 40 ms → 5 ms | 5 ms | inventory 120 ms | 0 ms — pure slack |
| Pricing 64 ms → 10 ms | 10 ms | inventory 120 ms | 0 ms — pure slack |
| Inventory 120 ms → 30 ms | 30 ms | pricing 64 ms | 56 ms — path moves to pricing |
| Inventory 120 → 30 *and* pricing 64 → 10 | 30 / 10 ms | shipping 40 ms | 80 ms — not 146 ms |
| Render 120 ms → 60 ms | n/a, serial | unchanged | 60 ms — serial work always pays |
Serial work always pays; parallel work pays only at the top
The general rule that falls out: serial segments pay one-for-one, parallel segments pay only if they are the maximum. In the example, auth (22 ms) and render (120 ms) are serial — every millisecond removed from them is a millisecond off the request, guaranteed. The three-way fan-out is parallel, and only its maximum matters. Ranking optimization opportunities by "serial first, then the max of each parallel group" is a better prioritization than ranking by span duration.
The corollary is that converting serial work to parallel work is often worth more than optimizing anything. If auth, inventory and pricing were stacked serially at 22 + 120 + 64 = 206 ms, running them concurrently drops that to 120 ms without making a single service faster — 86 ms for a scheduling change (Sequential or Parallel: Same Work, Different Latency). Where a dependency exists (nothing can run before auth), it stays serial and stays worth optimizing.
One caution: at the tail, parallel fan-out gets worse rather than better, because a request waiting for all branches inherits the tail of every branch. At p50 the maximum of three branches is close to the maximum of their medians; at p99 you are exposed to whichever branch is having a bad moment, and more branches means more chances. That is Fan-Out: Waiting for the Slowest of Seven, and it is the reason "just parallelize everything" has a cost.
Key points
- A concurrent fan-out completes at
max(branch), so every branch except the slowest has slack and returns nothing for optimization. - Serial segments pay one-for-one; parallel segments pay only while they are the maximum — prioritize serial work and the top of each parallel group.
- Fixing the critical path relocates it, so savings do not add: cutting two branches yields the new maximum, not the sum of both cuts.
- Converting serial calls to concurrent ones is often worth more than making any individual call faster, and costs no service-level work.
- Slack is a property of today's numbers; a branch with 80 ms of slack becomes critical the moment its dependency regresses.
Follow the diagnosis
The causal chain, hop by hop — and the readings that invite the wrong conclusion.
- 1Team → pricing service: three weeks of optimization takes it from 64 ms to 10 ms, confirmed by its own dashboard.
- 2Pricing → request: pricing was never the maximum of its fan-out group, so the request still waits for inventory at 120 ms.
- 3Endpoint p99 → team: unchanged, because no serial segment and no group maximum was touched.
- 4Trace → team: the critical path runs auth → inventory → render and always did; the work was aimed at a branch with 56 ms of slack.
- • "Our dependency latency dashboard improved 6×, so the endpoint got faster." Per-dependency dashboards cannot show slack; only the trace can.
- • "We can add up the savings from optimizing each dependency." You cannot. After the first fix, the path moves and the rest of the arithmetic is wrong.
- • "The branch with slack does not need monitoring." It needs monitoring precisely because slack disappears silently.
- • "The longest span is the critical path." The longest span may be a parallel branch with slack; the critical path is defined by dependency, not duration.
Measure, fix, validate
An optimization is not finished until the metric that motivated it has moved.
- • Mark the critical path on a representative slow trace: the chain where each span's start is gated by the previous span's end.
- • For each parallel branch, record finish time and compute slack against the slowest branch; anything with slack is deprioritized.
- • Model the change before making it — recompute `max(branch)` with the proposed number and predict the end-to-end saving.
- • Do the same on a p99 trace, not just a median one: the critical path at the tail is frequently a different branch ([[tail-latency]]).
- • Optimize the maximum branch or a serial segment — nothing else changes end-to-end latency today.
- • Convert genuinely independent serial calls into concurrent ones; it is usually the largest single win available and needs no downstream changes.
- • Remove work from the critical path entirely where the result is not needed for the response: defer it to a job ([[distributed-tracing]] shows the payment version of this).
- • Where a branch is critical only because it is required, question the requirement — a stale-but-instant cached value often removes the branch from the path.
- • End-to-end p99 must move; a dependency dashboard improving is not the result and should not be reported as one.
- • Re-read the trace and confirm the critical path is now a *different* chain — that is the signature of a real fix.
- • Compare predicted saving against actual: a large mismatch means the model of the path was wrong and the next estimate needs revisiting.
- • Check the tail separately; a fix that helps p50 and not p99 usually means the tail path differs from the median path.
- • Parallelizing increases concurrent load on downstreams and raises tail exposure — more branches means more chances to hit someone's bad moment ([[fan-out-latency]]).
- • Deferring work off the critical path buys latency and costs consistency: the response no longer reflects completed work.
- • Optimizing only the critical path leaves branches unoptimized, which is correct today and technical debt when the path moves.
- • Modelling paths adds analysis time to every performance task, which is only worth it when the fan-out is genuinely concurrent.
- • Alert on the endpoint SLI rather than per-dependency latency, so the thing that matters is the thing that pages.
- • Track slack per branch as a derived metric; a branch whose slack trends toward zero is the next critical path.
- • Keep the critical-path chain in the endpoint's runbook, with a date — it expires and the date says how much to trust it.
- • Include a critical-path check in performance review of new features: adding a serial call to a hot endpoint is a latency change even if the call is fast.
Accuracy
Performance numbers are conditional. These are the conditions.
- ILLUSTRATIVEAll durations and the savings table are constructed arithmetic from the example waterfall, not measurements.
- WORKLOAD-SPECIFICWhich branch is critical depends on the request shape — a different tenant, cache state or payload size can change the path within the same endpoint.