Coordination & Limits

Single-Flight Coalescing

One hundred callers need the same value. Issue one fetch and give everyone the same result. The implementation is fifteen lines and two of them are the ones that go wrong: what the dedup key includes, and what happens to the shared in-flight entry when the fetch fails.

▶ Run the lab

The question this answers

The question

A hundred concurrent callers want the same value — how do I make that one fetch, and what do they all get when it fails?

The work

100 concurrent requests for the same user profile arriving within 40 ms of a cache miss, against an origin that takes 200 ms to answer.

What is shared

A Map<key, Promise<T>> of in-flight fetches. Every caller reads it and the first caller writes it; the entry is the coordination point, and its lifecycle — created on the first miss, removed on settlement — is where all the bugs live.

The invariant — what must stay true under every interleaving

At most one fetch is in flight per dedup key at any moment, every caller receives the result of a fetch that started at or after its own arrival, and the map holds no entry for a key whose fetch has already settled.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

WorkCan it overlap?Can it parallelise?What is shared?What ordering?What synchronization?Where is contention?What can deadlock?What can race?What is gained?What complexity?

Fifteen lines, and the two that matter

The mechanism is a map from key to in-flight promise. The first caller finds nothing, starts the fetch, stores the promise, and awaits it. Every later caller finds the promise and awaits the same one. When it settles, the entry is removed so the next miss starts a fresh fetch. That is the whole thing, and in the good case it turns 100 origin requests into 1.

The two lines that decide whether it works in production are the key and the cleanup. The key must include everything that changes the result — the user id, the locale, the API version, the tenant, the authorisation scope. A key that omits the tenant will serve one tenant's data to another, which is a security incident rather than a caching bug. The cleanup must run on rejection as well as fulfilment, and it must be conditional on identity, or a slow caller's cleanup deletes a newer caller's in-flight entry (Futures & Promises).

The failure policy is a genuine design decision with no default answer. Sharing the error means one transient blip fails a hundred callers at once. Not sharing it means the hundred all retry independently, which is the herd you were preventing. The usual compromise is to share the failure but keep the window short — remove the entry immediately on rejection, so the very next caller starts a new attempt rather than joining a doomed one.

1type Entry<T> = { promise: Promise<T>; subscribers: number; startedAt: number }
2
3class SingleFlight<T> {
4 private inFlight = new Map<string, Entry<T>>()
5
6 // key MUST include everything that changes the result: tenant, locale,
7 // api version, auth scope. Omitting the tenant is a data-leak bug, not a
8 // cache-hit-rate bug.
9 async do(key: string, fn: () => Promise<T>, signal?: AbortSignal): Promise<T> {
10 const existing = this.inFlight.get(key)
11 if (existing) {
12 existing.subscribers += 1
13 // Each caller may give up independently WITHOUT cancelling the shared fetch.
14 return this.withSignal(existing.promise, signal)
15 }
16
17 const entry: Entry<T> = { promise: null as never, subscribers: 1, startedAt: Date.now() }
18
19 entry.promise = (async () => {
20 try {
21 return await fn()
22 } finally {
23 // Conditional on identity: if a newer entry replaced ours, leave it alone.
24 // Deleting unconditionally evicts someone else's in-flight fetch and the
25 // coalescing silently stops working under exactly the load it exists for.
26 if (this.inFlight.get(key) === entry) this.inFlight.delete(key)
27 }
28 })()
29
30 // Set BEFORE any await, so concurrent callers in the same tick find it.
31 this.inFlight.set(key, entry)
32 return this.withSignal(entry.promise, signal)
33 }
34
35 // A caller that aborts stops waiting; the shared fetch continues for everyone else.
36 private withSignal(p: Promise<T>, signal?: AbortSignal): Promise<T> {
37 if (!signal) return p
38 return Promise.race([
39 p,
40 new Promise<never>((_, reject) => {
41 if (signal.aborted) return reject(signal.reason)
42 signal.addEventListener('abort', () => reject(signal.reason), { once: true })
43 }),
44 ])
45 }
46
47 get inFlightCount() { return this.inFlight.size }
48}
49
50// Usage in front of a cache. Note the double-check INSIDE the flight:
51// between the miss and the fetch starting, someone may have filled the cache.
52const sf = new SingleFlight<Profile>()
53
54async function getProfile(tenant: string, userId: string, locale: string, signal?: AbortSignal) {
55 const key = tenant + '|' + userId + '|' + locale + '|v3'
56 const hit = await cache.get(key)
57 if (hit) return hit
58 return sf.do(key, async () => {
59 const second = await cache.get(key) // filled while we were waiting?
60 if (second) return second
61 const fresh = await origin.fetchProfile(tenant, userId, locale)
62 await cache.set(key, fresh, jitteredTtl(300)) // jitter, or you rebuilt the herd
63 return fresh
64 }, signal)
65}
Single-flight with an identity-checked cleanup, a shared-failure policy, and per-caller cancellation.

The two schedules that break it

Both failures below come from the entry's lifecycle, and both are invisible in tests: the first needs a rejection under concurrency, the second needs a caller to arrive in a specific window. The first is the more dangerous, because its symptom is not an error — it is that the coalescing quietly stops working, which looks exactly like ordinary load.

The second schedule is the staleness question, and it is the one that has no purely technical answer. A caller that arrives 190 ms into a 200 ms fetch joins it and receives data that was read from the origin before that caller existed. For a profile that is fine. For a value the caller just wrote, it is a lost update from the reader's point of view. If read-your-writes matters, the caller must bypass single-flight after a write, or the key must incorporate a version that the write bumps (Optimistic Concurrency Control).

Unconditional cleanup on rejection, plus the join-window staleness question.ILLUSTRATIVE
Invariant · At most one fetch in flight per key, and no caller's cleanup removes another caller's entry
#Caller A (first miss)Caller B (joins)Caller C (arrives after)In-flight mapState
1miss; creates entry E1; starts origin fetch···entry=E1 originCalls=1 subs=1
2·miss; finds E1; subscribes··entry=E1 originCalls=1 subs=2
3E1 rejects (origin 503); A's finally deletes the map entry···entry=none originCalls=1 subs=2
4··miss; finds nothing; creates entry E2; starts a fresh fetch·entry=E2 originCalls=2 subs=1
5·B's continuation also runs its cleanup: `map.delete(key)` — unconditional··entry=none originCalls=2 subs=1
✕ B deleted E2, which belongs to C. The map is now empty while a fetch is in flight, so the next caller starts a third one. Under sustained failure this degenerates to one origin call per caller — the coalescing stops precisely when the origin is already struggling.
6···with the identity check `if (map.get(key) === entry)`, B's delete is a no-op and E2 survivesentry=E2 originCalls=2 subs=1
7··later scenario: a caller arriving 190 ms into E2's 200 ms fetch joins it·entry=E2 joinedAt=190ms dataAsOf=0ms
✕ The caller receives a value read from the origin before it arrived. Harmless for a profile; a read-your-writes violation if that caller just performed a write.
Make the cleanup conditional on entry identity and run it in a finally so rejections clean up too. For staleness, decide the join policy deliberately: unconditional joining is right for read-mostly data, and a caller that just wrote must bypass single-flight or use a key that its write bumped.

Designing the key and the failure policy

These two decisions carry the whole design, and both have a wrong answer that looks reasonable. Under-specifying the key is a correctness and sometimes a security bug; over-specifying it means every caller has a unique key and coalesces with nobody, so the machinery does nothing while looking like it works. The measurement that tells you which you have is the coalescing ratio: origin calls divided by callers. It should be far below 1, and if it is 1 you have either an over-specified key or the cleanup race above.

The scope decision matters at least as much. This is a per-process map, so twenty instances means twenty flights — a 20× reduction, not a 100× one. Making it cross-process needs a distributed lock or a shared coordination service, with all the correctness warnings in A Mutex on Server A Does Nothing About Server B; the honest position is that per-process single-flight plus jittered TTLs is usually enough, and reaching for distributed coordination for a cache fill is rarely worth what it costs.

DecisionOptionConsequenceFails as
Dedup keyUnder-specified (omits tenant / locale / scope)Callers coalesce who should notOne tenant receives another's data — a security incident, not a cache bug
Dedup keyOver-specified (includes request id, timestamp)Nobody ever coalescesMachinery in place, coalescing ratio stays at 1, no benefit and extra code
Dedup keyCorrect: every input that changes the result, nothing that does notOne flight per distinct resultCorrect — verify with the coalescing ratio
Failure policyShare the rejection with all subscribersOne blip fails 100 callers at onceA correlated error spike; usually acceptable if the entry is removed immediately
Failure policyRetry inside the flight before rejectingSubscribers wait longer but usually succeedThe shared fetch now holds a resource much longer; needs a hard deadline
Failure policyDo not share failures — each caller retries aloneThe herd you were preventing, at the worst momentOrigin overload during exactly the incident that caused the failure
Cleanup timingDelete on settle, unconditionallyA late continuation evicts a newer entryCoalescing silently degrades to 1:1 under sustained failure
Cleanup timingDelete on settle, only if identity matchesNewer flights survive older cleanupsCorrect
ScopePer processN instances means N flightsFine at 20 instances; a 20x reduction, not 100x
ScopeCross-process via a distributed lockExactly one flight in the fleetA lock that can be lost or expire mid-fetch — see A Mutex on Server A Does Nothing About Server B
The two decisions that carry the design, and how each one fails.

Key points

  • A map from dedup key to in-flight promise: the first caller fetches, everyone else subscribes to the same handle.
  • The key must include every input that changes the result — omitting a tenant or a scope is a data-leak bug, not a cache-hit-rate bug.
  • Over-specifying the key is the silent failure: the machinery runs and nobody ever coalesces.
  • Cleanup must run in a finally so rejections clean up, and must be conditional on entry identity so a late continuation cannot evict a newer flight.
  • The failure policy is a real decision: share the rejection and remove the entry immediately, so the next caller starts fresh rather than joining a doomed flight.
  • Set the map entry before any await, or two callers in the same tick both start a fetch.
  • It is per-process. Twenty instances means twenty flights, and making it fleet-wide costs far more than it usually saves.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • A caller computes the dedup key from every input that affects the result.
  • It looks the key up in the in-flight map. A hit means an identical fetch is already running; it subscribes to that promise and returns.
  • A miss means it is first: it constructs the fetch promise, stores it in the map *before* awaiting anything, and awaits it.
  • Storing before the first suspension is what makes the check-and-set atomic on an event loop — there is no yield point between the lookup and the write (Event Loops as a Concurrency Model).
  • When the fetch settles, a finally removes the entry — but only if the map still holds *this* entry, so a newer flight is never evicted.
  • Every subscriber's continuation runs with the same value or the same error; the promise's broadcast semantics do the fan-out for free.
  • A caller may abandon its wait (an abort signal) without cancelling the shared fetch, because the other subscribers still need it.
Interleavings that matter
  • A misses and creates E1; B and C arrive 5 ms later and find E1; the origin sees one request and three callers are served — the intended schedule.
  • A misses, awaits the cache a second time *before* setting the map entry; B misses in the same window and also finds nothing; two flights start for one key. Set the entry before any await.
  • E1 rejects; A's cleanup deletes the entry; C creates E2; B's cleanup runs unconditionally and deletes E2; the next caller starts a third flight — coalescing degrades to 1:1 exactly when the origin is failing.
  • The same schedule with an identity-checked cleanup: B's delete is a no-op, E2 survives, and the origin still sees one flight at a time.
  • A caller arrives 190 ms into a 200 ms fetch and joins it, receiving data read before it arrived — fine for a profile, a read-your-writes violation if it just wrote.
  • The origin hangs; the shared promise never settles; 100 subscribers wait forever and the map entry is permanent. A deadline inside the flight is not optional.
  • Twenty instances each run their own single-flight; the origin sees twenty concurrent fetches, not one. Correct, and a 20× reduction rather than the 400× the local metric suggests.
What it guarantees — and does not
  • Guaranteed: at most one fetch in flight per key per process, for as long as the entry exists.
  • Guaranteed: every subscriber receives the same value or the same error — the promise settles once and broadcasts.
  • Guaranteed: a subscriber that gives up does not affect the shared fetch or the other subscribers.
  • NOT guaranteed: freshness. A late joiner receives data read before it arrived, which breaks read-your-writes if that caller just wrote.
  • NOT guaranteed: fleet-wide deduplication. This is a per-process map and nothing more.
  • NOT guaranteed: liveness. If the fetch never settles, every subscriber waits forever and the entry is permanent — put a deadline inside the flight.
  • NOT guaranteed: isolation of failures. One rejection fails every subscriber at once, by design.
  • NOT guaranteed: correctness with a bad key. Under-specify it and you serve one caller's data to another with complete confidence.
Where contention appears
  • The map is the coordination point and is uncontended on one event loop; across threads it needs a lock, and that lock is held only for the lookup and insert.
  • The real contention it *removes* is at the origin: N concurrent identical fetches become one.
  • It concentrates risk instead: one slow fetch now holds N callers, so the shared fetch's p99 becomes every subscriber's p99.
  • Settlement wakes every subscriber in the same tick — a small, bounded wake burst that is fine at 100 and worth thinking about at 100,000 (Thundering Herd).
  • Under sustained origin failure with a broken cleanup, contention returns to the un-coalesced level at the worst possible moment.
How it fails
  • Data leak from an under-specified key: callers coalesce across a boundary they should not cross.
  • No coalescing at all from an over-specified key, with the ratio stuck at 1 and nobody noticing.
  • Cleanup race: an unconditional delete evicts a newer in-flight entry and coalescing degrades to 1:1.
  • Cached rejection: an entry not removed on failure means every later caller joins a promise that is already rejected — a poisoned key until restart.
  • Permanent entry from a fetch that never settles, holding every subscriber indefinitely.
  • Stale read for a caller that joined a flight started before its own write (Optimistic Concurrency Control).
  • Correlated failure: one origin blip fails 100 callers simultaneously, which shows up as a spike rather than a trickle.
  • False confidence at fleet scale: per-process metrics show 100:1 coalescing while the origin sees one flight per instance.
When it helps
  • Read-heavy caches with hot keys, where a miss is expensive and many callers want the same value at once.
  • Cache-fill paths, as the direct answer to a stampede (Thundering Herd, cache-stampede).
  • Expensive idempotent computations — a rendered page, an aggregate, a model inference — where duplicating the work is pure waste.
  • Rate-limited third-party APIs, where N identical calls consume N units of a budget that one call would have used.
  • Any fan-in where the callers genuinely want the same answer and none of them needs a private one.
When it hurts
  • When the callers do not actually want the same value, and the key was written to make them look like they do.
  • When read-your-writes matters, because a joiner can receive data read before its own write.
  • When failures should be independent — a single rejection failing 100 callers may be worse than 100 independent retries.
  • When keys are naturally unique, in which case the map is overhead that never pays for itself.
  • When the value is cheap to compute, and the coordination costs more than the duplication it removes.
How you would know
  • Coalescing ratio: origin calls divided by callers. Far below 1 is working; stuck at 1 means an over-specified key or the cleanup race.
  • In-flight map size over time. A monotonic climb is a leak — entries that never settle or never get removed.
  • Subscriber count per flight as a distribution; a long tail is exactly the hot-key concentration this exists to serve (Hot Keys: When Aggregate Metrics Hide a Saturated Node).
  • Shared-fetch latency at p99, because it is now every subscriber's latency, not just the first caller's.
  • Rejection fan-out: errors returned to callers divided by origin errors. It should equal the average subscriber count, and that is your correlated-failure exposure.
  • Origin request rate divided by *instance count* to check the fleet-wide effect, rather than trusting a per-process ratio (A Mutex on Server A Does Nothing About Server B).
Complexity it introduces
  • A shared mutable map with a lifecycle you own: created on miss, removed on settle, and never removed by the wrong owner.
  • The dedup key becomes a contract that must be updated whenever a new input starts affecting the result — and forgetting is a data-correctness bug.
  • The failure policy and the deadline are additional decisions with no defaults, and both are invisible until an incident.
  • It interacts with cancellation: an aborting caller must not cancel the shared work, which is a special case in every cancellation design.
  • Testing requires forcing concurrency and forcing failures together, because the cleanup race needs both.
Simpler alternatives
  • A cache with a short TTL and jitter. If the value can be slightly stale, caching removes most duplicate fetches without any coordination at all (Thundering Herd).
  • Serve stale while revalidating in the background, so there is no simultaneous miss to coalesce.
  • Batching: collect requests for a few milliseconds and issue one combined call, which also deduplicates and additionally amortises across *different* keys.
  • A concurrency limit on the origin, which caps the damage without deduplicating anything — weaker, but it needs no key design (Bounding Concurrency, Semaphores: Counting Permits as a Resource Limit).
  • Precompute and push, so the value is always present and the miss path never runs.
  • Nothing at all, when the fetch is cheap. Single-flight is coordination, and coordination is only worth it when the duplicated work costs more than the map (Concurrency Is Always Bought With Complexity).

Single-flight: N callers, one call

Single-flight — N callers want the same key
64 requests for the same cache key arrive while it is being recomputed. Coalescing lets the first one do the work and parks the rest on its result.
Callers
1 leader issues the call · 63 followers park on its promise
Downstream
1 call against a service sized for 20 concurrent
downstream calls
1
work saved
63 calls (98.4%)
caller latency
60 ms
callers that see an error
0
without    64 callers →  64 downstream calls   latency 192 ms  (queued behind each other)
with       64 callers →   1 downstream call    latency 60 ms  (everyone waits for the leader)

failure   one attempt, 64 disappointed callers — the blast radius of a single bad call is now N
retry     the followers cannot retry independently; they only ever saw the leader's outcome
64 callers, 1 downstream call, 98.4% of the load gone. The mechanism is a map from key to in-flight promise: the first caller creates the entry and does the work, everyone else finds it and awaits it, and the entry is removed when it settles. What you buy is load reduction; what you pay is coupling — every caller now has the latency and the fate of the leader, and a slow leader makes all 64 slow. Flip the failure toggle to see the sharp edge.
attempts against the origin this window: 1
SIMULATED

Thundering herd: 10,000 waiters

Thundering herd — one event, ten thousand waiters
A cache entry expires (or a leader is elected, or a socket becomes readable) and every one of 10K waiters wakes at once and hits the same resource, which serves 900/s.
t = 0— — capacity per 100 ms buckett = 3000 ms
requests issued
10K
peak arrivals in one bucket
10K
shed / rejected
9,640
queue drained by
300 ms
no mitigation   10K requests in one 100 ms bucket vs a capacity of 90/bucket → 9,640 shed
this setting    10K requests spread over one instant → peak 10K/bucket, 9,640 shed

jitter    sleep(base + random() * window)  — decorrelates wakeups; costs a little latency
batching  one call serves N waiters        — cuts the request count, not the wakeup count
both      are cheaper than the capacity you would otherwise have to buy for one instant per hour
9,640 of 10K waiters get nothing. Peak demand is 10K in a 100 ms bucket against a capacity of 90. The resource is not undersized for the average load — it is undersized for one instant, and that instant is created by the fact that every waiter was released by the same event. Note what does not fix this: retries. A client that retries immediately after being shed lands inside the same spike and makes the second peak worse than the first. Spread the wakeups (jitter) or reduce them (batch, or coalesce with single-flight).
SIMULATEDqueue bounded at 4× capacity; overflow is shed

Optimistic concurrency lab

Optimistic vs pessimistic under contention
A version-checked update: read the row with its version, compute, write back only if the version is unchanged, retry if it moved.
SIMULATEDThe shape of the curve is the lesson; the axis numbers are not.

Conflict probability grows with the number of writers touching the same item, each conflict costs a full retry, and retries consume the same cores the successful work needs. Real systems add their own effects — backoff, hot keys, transaction size — but the turnover is real and it is why an optimistic scheme that benchmarked beautifully at low load can collapse at high load rather than merely slow down.

optimistic (version check + retry)pessimistic (lock, then work)x: 1–48 · y max 730 commits/s
optimistic
646/s
pessimistic
196/s
attempts per commit
1.24
work thrown away
19%
retry rate (attempts that fail the version check)19.2%
commits landing646.4/s
At 8 writers and 3% contention the optimistic scheme commits 646/s against the lock’s 196/s: 19% of attempts retry, which is cheap enough that never waiting wins. Note where it peaks — 730/s at 4 writers — and that past that point adding writers makes the system slower, not faster.
Crossover at 48 writers: below it, waiting is the waste; above it, retrying is. Neither scheme is “the fast one” — the contention rate decides, and the contention rate is a property of your data, not of your code.
Both schemes preserve the same invariant — no lost update — and both are correct. What differs is where the cost lands: predictable waiting versus unpredictable wasted work, and a retry loop that must be bounded or it becomes a livelock. The version check itself is the contract; API Design and Database Engineering cover how it is exposed and how a store implements it.
optimistic leads at 8 writersSIMULATED

What people believe, and what is true

Claim

Single-flight means the origin sees one request.

Reality

One per process. Twenty instances means twenty flights, and per-process metrics will happily report a 100:1 ratio while the origin sees twenty concurrent fetches.

Claim

Sharing the promise means everything happens once.

Reality

The fetch happens once; every subscriber still runs its own continuation. Any cleanup or side effect in that continuation runs once per subscriber — which is exactly the cleanup race.

Claim

If the fetch fails, the next caller will just retry.

Reality

Only if the entry was removed. A rejected promise left in the map is served to every later caller as an instant failure — the key is poisoned until something clears it.

Go deeper

Overview

A map from key to in-flight promise: the first caller fetches, everyone else awaits the same handle, and the entry is removed when it settles.

Practical

Put every result-affecting input in the key, set the entry before any await, clean up in a finally conditional on entry identity, and put a deadline inside the flight.

Advanced

The join window is a freshness policy, not an implementation detail: a caller joining a flight started before its own write will not see that write. Decide whether read-your-writes matters and bypass or version the key accordingly.

Internals

It works because a promise is a settle-once broadcast handle — safe publication with an intrusive continuation list (Futures & Promises). The map is the only mutable state, and on one event loop the lookup-and-insert pair is atomic because nothing can run between them.

Apply it