Sequential or Parallel: Same Work, Different Latency
Four dependency calls take 740ms in a chain and 300ms fanned out. The parallel version is not simply better: it triples the instantaneous load on everything downstream and turns one failure into four things to reason about at once.
Frame the diagnosis
Performance work starts from a symptom and a signal — never from a resource dashboard.
The same four calls, stacked
Sequential dependency calls produce a latency that is the sum of every hop, and a resource profile where the process is idle almost the whole time. This is the default shape that falls out of ordinary imperative code: each await completes before the next line runs, whether or not the calls have any relationship to each other.
The first question is always whether the sequence is *required*. Call B genuinely needs A's output when it uses A's id. It does not need A's output when the two are simply written in that order, which is the far more common case and is usually an accident of how the function grew rather than a decision anyone made.
In the sequential waterfall, the process is waiting for 740ms and computing for perhaps 20ms of it. That idleness is the opportunity — and it is invisible on a CPU dashboard, which is exactly the Computing or Waiting? distinction that decides which fixes can possibly work.
The same four calls, overlapped
Issued together, the four calls cost the maximum rather than the sum: 240ms instead of 740ms, for identical work and identical downstream cost in aggregate. When the calls are genuinely independent, this is one of the largest latency wins available anywhere, and it requires no downstream team to change anything.
What changes is the shape of the load. Instantaneous concurrency against the downstream services triples, connection pool demand triples, and the request now holds four in-flight operations rather than one. A pool sized for the sequential pattern will saturate under the parallel one, and the resulting queueing can eat the entire gain — a bottleneck that has simply moved, in the sense of The Bottleneck Moves After Every Fix.
Error handling also changes character. Sequentially, the first failure stops the chain and the rest never happen. In parallel, all four are in flight when one fails, so you must decide: cancel the others, wait for them and discard, or return partial results. That is a real design decision, and it is easy to get wrong in a way that leaks resources or produces inconsistent partial state.
Choosing deliberately
Parallel is the right default for independent calls, and it is not universally correct. It is wrong when the downstream cannot absorb the concurrency, when the calls contend for the same scarce resource so overlapping them only creates queueing, and when a failure in one makes the others pointless and expensive — issuing four calls to discard three of them wastes real capacity during exactly the incidents where capacity matters.
It is also worth being honest that parallelism moves you onto the fan-out tail curve. Four parallel calls means the request is slow whenever any of the four is slow, which is a worse tail than any individual dependency — the amplification described in Fan-Out: Waiting for the Slowest of Seven. The latency win is real and it comes with a tail cost that grows with width.
The decision procedure that holds up: parallelise independent calls; bound the concurrency explicitly rather than issuing everything at once; size pools for the new pattern before shipping it; decide the cancellation policy for partial failure; and verify the downstream services can take the concurrency you are about to send them.
| Consideration | Sequential | Parallel |
|---|---|---|
| Latency | Sum of all hops | Maximum of the hops |
| Instantaneous downstream load | One call in flight | N calls in flight — pools and downstream capacity must absorb it |
| Tail behaviour | Each dependency's tail added | Slow whenever *any* dependency is slow — worse tail, better median |
| Failure handling | First failure stops the rest naturally | Must decide: cancel, drain, or return partial |
| Wasted work on failure | None — later calls never happen | Up to N−1 calls completed and discarded |
| Required when | A call genuinely needs a previous response | Calls are independent and downstream can absorb the concurrency |
Key points
- Sequential costs the sum; parallel costs the maximum. For independent calls this is often the single largest latency win available.
- Most sequential dependency chains are sequential by accident — written in an order nobody chose deliberately.
- Parallelism multiplies instantaneous concurrency: connection pools and downstream capacity must be sized for the new shape or the gain is eaten by queueing.
- Failure handling changes: in-flight calls must be cancelled, drained or returned partially, and work is wasted where sequential would have skipped it.
- Parallelising moves the request onto the fan-out tail curve — better median, worse tail, and the tail grows with width.
Follow the diagnosis
The causal chain, hop by hop — and the readings that invite the wrong conclusion.
- 1Handler → auth-service: 120ms, and its token is genuinely needed by the calls that follow.
- 2Handler → profile-service: 200ms, issued after auth returns even though it only needs the user id already in the request.
- 3Handler → billing-service: 240ms, issued after profile returns for no reason other than statement order.
- 4Handler → usage-service: 180ms, same pattern; the process has now been idle for over half a second.
- 5Handler → client: responds at 800ms, having spent roughly 20ms computing and the rest waiting on calls that could have overlapped.
- • "CPU is low, so the service is healthy." Low CPU with high latency is the signature of waiting, and waiting is exactly what parallelism removes.
- • "The calls must be sequential, they are written that way." Statement order is not a dependency; check whether any call uses a previous response.
- • "Parallel is always better." It is better for independent calls with capacity to absorb it; it wastes work on failure and worsens the tail.
- • "We parallelised and latency did not improve." Check pool saturation — the wait moved from the network to the connection pool.
- • "p50 improved, so we are done." Check p99: fan-out width is now on the tail curve.
Measure, fix, validate
An optimization is not finished until the metric that motivated it has moved.
- • A trace waterfall for the endpoint, checking whether dependency spans overlap or stack — the picture answers the question immediately.
- • Whether each call's arguments actually reference a previous call's response, which decides if the sequence is required or accidental.
- • Connection pool utilisation and wait time under the parallel pattern, before shipping it to full traffic.
- • Downstream concurrency and queue depth for each dependency, so the added instantaneous load is visible on their side too.
- • Request p50 and p99 together, since parallelising typically improves the first and can worsen the second.
- • Identify calls whose arguments do not reference any previous response, and issue them concurrently.
- • Bound the concurrency explicitly rather than firing everything at once, so the pattern degrades predictably under load.
- • Resize client connection pools for the new in-flight count before shipping, and confirm downstream services can absorb the concurrency.
- • Define the partial-failure policy: cancel in-flight calls, or drain them, or return partial results — and implement cancellation so failures do not leak work.
- • Keep genuinely dependent calls sequential, and record why in a comment so the next person does not "optimise" a real dependency away.
- • Request p50 and p99 before and after, on the same endpoint and traffic mix — expect p50 to fall and watch p99 carefully.
- • A trace waterfall confirming the spans now overlap rather than stack.
- • Connection pool wait time, which must not have absorbed the latency saving.
- • Downstream service concurrency and queue depth, confirming the added instantaneous load was absorbed rather than queued.
- • Parallel execution wastes downstream work when one call fails and the others are discarded — costly during incidents.
- • Larger pools consume memory and file descriptors, and can move queueing onto the downstream service.
- • Concurrent code is harder to read and to reason about under partial failure; cancellation is easy to implement incorrectly.
- • Better median latency is bought with a worse tail as fan-out width grows.
- • A trace-based check on critical endpoints that flags newly-stacked dependency spans, which is how an accidental sequence creeps back in.
- • A load test at peak concurrency after any change to the parallelism of a hot path, since pool sizing is the usual failure.
- • An alert on connection pool wait time, independent of request latency.
- • A review rule requiring new dependency calls on a hot path to state whether they are independent and why.
Accuracy
Performance numbers are conditional. These are the conditions.
- ILLUSTRATIVEBoth waterfalls are invented to show the shape difference. Real gains depend on how independent the calls truly are and whether pools and downstream services absorb the concurrency.
- WORKLOAD-SPECIFICThe benefit assumes the calls are I/O-bound and the process is idle while waiting. For CPU-bound work, issuing calls concurrently on one thread changes nothing.