Distributedfan-outtail amplificationparallel callshedgingp99

Fan-Out: Waiting for the Slowest of Seven

Call seven services in parallel and wait for all of them, and your latency is not the average — it is the maximum. A dependency that is slow one time in a hundred becomes a request that is slow seven times in a hundred, which is how p99 problems become p93 problems.

▶ Run the labFollow the diagnosis

Frame the diagnosis

Performance work starts from a symptom and a signal — never from a resource dashboard.

Diagnostic question
Every dependency reports a healthy p99, so why is my request latency so much worse than any of them?
Symptom
Aggregating service latency dashboards suggests the page should render in 120ms. Real request p99 is over a second, and no single dependency looks responsible.
Signal
The distribution of per-request maximum dependency latency, not the distribution of each dependency separately. The misleading signal is per-service p99, which is individually reassuring and collectively meaningless.
SymptomSignalMeasurementHypothesisEvidenceRoot CauseChangeValidationRegression Check

The maximum, not the average

When a request fans out to N dependencies and needs all of their answers, its latency is the maximum of N samples, plus merge time. That is a fundamentally different quantity from the average, and it behaves worse the more dependencies you add — which is the opposite of the intuition that parallelising work makes it fast.

In the waterfall below, six dependencies return between 85ms and 130ms and one takes 580ms. The request takes 660ms. Optimising any of the six fast ones changes nothing at all; they are not on the critical path, in exactly the sense The Critical Path Is the Only Path That Pays means. Only the slow one matters, and only until it stops being the slowest.

This is also why fan-out latency is so resistant to dashboard-driven optimisation. Each dependency team looks at their own p99, sees a healthy number, and correctly concludes they are not the problem. The problem is a property of the composition, and it is nobody's dashboard.

A page assembling seven dependencies in parallel
critical pathILLUSTRATIVE
0175350525700
GET /dashboard (total)660 ms
profile-service95 ms
notifications-service110 ms
billing-service85 ms
catalog-service130 ms
usage-service105 ms
team-service90 ms
recommendations-service580 ms
merge + render40 ms
GET /dashboard (total)Bounded by the slowest dependency, not the average
profile-serviceFast. Optimising it buys nothing
recommendations-serviceThe only span on the critical path. Everything else is noise

Tail amplification: how p99 becomes p93

Suppose each dependency independently exceeds 500ms one time in a hundred. A request that waits on one of them is slow 1% of the time. A request that waits on seven is slow whenever *any* of them is slow: 1 − 0.99⁷, which is about 6.8%. At twenty dependencies it is 18%; at a hundred, 63%.

Those figures are an estimate under an explicit assumption — that the dependencies fail slowly *independently*. That assumption is usually wrong in both directions. Shared infrastructure (the same database, the same network, the same noisy neighbour) correlates slowness, which makes the real number better than the model in quiet periods and dramatically worse during an incident, because the correlated case is precisely when everything is slow at once.

The practical consequence is that the per-service latency target must be *stricter* than the request target, not equal to it. If a page composed of ten dependencies needs a 300ms p99, each dependency needs a p99 well below 300ms and, more importantly, a bounded tail — because it is the tail, not the median, that composes.

Probability that at least one dependency is slow, as fan-out width grows (assumes independence)ESTIMATED
11
22
55
77
1010
1820
3950
63100
1 dep 1 dependencies — 1% of requests slow — matches the dependency's own p997 deps 7 dependencies — ≈6.8% slow: the dependency p99 has become roughly the request p9320 deps 18 dependencies — ≈18% slow — nearly one request in five100 deps 63 dependencies — ≈63% slow: the median request is now a tail request

What you can actually do about it

The strongest fix is to need fewer answers. Dependencies that are not required for the first meaningful render can be fetched after it, or rendered optimistically, or dropped when slow — a page that degrades to "recommendations unavailable" in 120ms is usually a better product than one that is perfect in 660ms.

The second fix is a per-dependency timeout with a defined fallback, which converts an unbounded tail into a bounded one. This is a real trade: you are choosing to sometimes return incomplete data rather than sometimes return late data, and which is correct is a product decision rather than an engineering one.

Hedging — issuing a duplicate request after a short delay and taking whichever answers first — genuinely cuts the tail, and it costs extra load on the very dependency that is already struggling. It is a reasonable tool with a strict precondition: the operation must be safe to duplicate (see Idempotency), and the hedge must be capped so it cannot amplify an incident into a Retry Storms: The Load You Generated Yourself pattern.

Fan-out tail mitigations
ApproachEffect on tailCostWhen it is wrong
Need fewer answers — defer or drop non-essential dependenciesRemoves them from the maximum entirelyProduct work: what does degraded look like?When every dependency is genuinely required for correctness
Per-dependency timeout + fallbackBounds the tail at the timeout valueIncomplete responses some of the timeWhen partial data is misleading rather than merely incomplete
Hedged requestsCuts the tail substantially in the uncorrelated caseExtra load on an already-slow dependencyNon-idempotent operations; or during correlated slowness, where it adds fuel
Reduce fan-out width — batch or aggregate upstreamFewer samples to take the maximum ofCoupling, a new aggregation service to ownWhen the aggregator becomes a bottleneck of its own
Tighten per-dependency p99Attacks the actual causeSlow, distributed across many teamsNever wrong; just rarely fast enough to fix today's incident

Key points

  • Fan-out latency is the maximum of the dependency latencies, not the average — adding parallel dependencies makes the tail worse, not better.
  • Under independence, N dependencies each slow 1% of the time produce a request slow 1 − 0.99^N of the time: ~6.8% at seven, ~18% at twenty.
  • Independence is an assumption, and shared infrastructure violates it exactly when it matters most — during an incident.
  • Per-dependency latency targets must be stricter than the composed request target, because tails compose.
  • Optimising a dependency that is not the per-request maximum changes nothing the user can perceive.

Fan-Out Tail Amplification

Change an input and watch which number moves — and which one does not.

One request, N dependencies, all required
ESTIMATED
request hits a slow one
4.9%
svc1svc2svc3svc4svc5

Each dependency is slow only 1% of the time — a number any team would call healthy. But the request waits for all 5, so it is slow whenever any of them is: 4.9% of the time.

Assumes the dependencies are slow independently, which real systems violate — they share hosts, networks and databases, so correlated slowness makes this worse, not better.

Follow the diagnosis

The causal chain, hop by hop — and the readings that invite the wrong conclusion.

  1. 1
    Client → API: requests the dashboard, which requires data from seven services.
  2. 2
    API → dependencies: seven calls issued in parallel at 40ms; six answer between 85ms and 130ms.
  3. 3
    recommendations-service → API: takes 580ms because it is doing its own fan-out internally, invisible from here.
  4. 4
    API → merge: cannot begin until the last answer arrives at 620ms; the six fast responses have been sitting in memory doing nothing.
  5. 5
    API → client: responds at 660ms, and every dependency dashboard shows a healthy p99 for the interval.
What this evidence makes people conclude — wrongly
  • "Every dependency p99 is under 130ms, so the request should be under 130ms." The request takes the maximum, and the maximum has a much worse tail than any individual.
  • "We parallelised the calls, so latency is solved." Parallelising converts a sum into a maximum, which is a large win once and then stops helping as width grows.
  • "The slow service is only slow 1% of the time." Multiplied across seven dependencies that is nearly 7% of requests.
  • "Hedging is free tail reduction." It doubles load on the dependency you are hedging against, which is counterproductive precisely when it is overloaded.
  • "No single dependency is responsible." Correct — and the composition still is. Look at blame share, not per-service dashboards.

Measure, fix, validate

An optimization is not finished until the metric that motivated it has moved.

How to measure it
  • • The distribution of per-request *maximum* dependency latency, which is the quantity that actually determines request latency.
  • • For each dependency, how often it is the slowest one in a request — the "blame share" that identifies where optimisation would pay.
  • • Fan-out width per endpoint from traces, so growth in the number of dependencies is visible over time.
  • • Correlation between dependency slowness events, to test whether the independence assumption holds in your system.
  • • Per-dependency timeout and fallback rates, so bounded tails are distinguishable from missing data.
What actually fixes it
  • • Remove non-essential dependencies from the blocking path: render without them, fetch them after first paint, or drop them under load.
  • • Give every dependency call a timeout and a defined fallback so the tail is bounded by design rather than by luck.
  • • Reduce fan-out width by aggregating related calls, accepting the coupling that introduces.
  • • Hedge only idempotent, capped, and uncorrelated calls — and disable hedging automatically when the dependency is already degraded.
  • • Set per-dependency latency objectives derived from the composed target, and hold them as tail objectives rather than median ones.
How you know it worked
  • • Request p99 for the affected endpoint, compared against the same window before — the per-dependency dashboards will not show the improvement.
  • • The distribution of per-request maximum dependency latency, which should tighten if the fix worked.
  • • Fallback and timeout rates, to confirm a bounded tail was achieved by degrading gracefully rather than by silently dropping data.
  • • Downstream load on any hedged dependency, confirming the hedge did not add meaningful pressure.
What it costs
  • • Timeouts with fallbacks trade completeness for predictability — some users see partial data that was previously merely late.
  • • Hedging costs extra load and requires idempotency; it makes correlated incidents worse.
  • • Aggregating dependencies to reduce width introduces coupling and a new component that can itself become the bottleneck.
  • • Stricter per-dependency SLOs push work onto many teams and are slow to land, however correct they are.
Stop it coming back
  • An alert on fan-out width per endpoint, so a new blocking dependency added to a hot path is visible at the time it is added.
  • A per-dependency timeout requirement enforced in review — an un-timed-out call on a critical path fails the check.
  • An SLO on request p99 for composed endpoints, distinct from the per-service SLOs, so the composition has an owner.
  • A dashboard of blame share by dependency, reviewed periodically rather than only during incidents.

Accuracy

Performance numbers are conditional. These are the conditions.

What these numbers depend on
  • ESTIMATEDThe 1 − (1−p)^N amplification figures assume dependency slowness events are statistically independent. Shared infrastructure correlates them, which makes the model optimistic during quiet periods and pessimistic about how independent your incidents will be.
  • ILLUSTRATIVEThe seven-dependency waterfall timings are invented to show the shape where one span dominates. Real fan-outs vary in width, timing and correlation.

Misconceptions

Claim
“Parallel calls make a request as fast as its fastest dependency.”
Reality
If all answers are required, the request is as slow as its slowest. Parallelism converts a sum into a maximum, which helps once and then degrades as width grows.
Claim
“If every dependency meets its p99 target, the request meets it too.”
Reality
Tails compose. Seven dependencies each at a 1% slow rate produce a request slow nearly 7% of the time, so the composed endpoint needs stricter per-dependency targets.
Claim
“Hedged requests are a free way to cut the tail.”
Reality
They add load to the dependency being hedged against, require idempotency, and actively worsen correlated slowness — which is when the tail hurts most.

Apply it