The question this answers
What shape does this coordination problem already have a known answer for?
An ingestion service that reads a file, parses records, enriches each one from three APIs, and writes batches to a database — every pattern in this catalogue appears somewhere in that sentence.
Varies by pattern, and that is the useful axis: producer/consumer and message passing share a queue, pools share a work queue and a worker set, fork/join shares a result array with disjoint slots, and the actor model shares nothing at all.
Every submitted unit of work is executed exactly once, its result is observed by whoever is waiting for it, and the system's resource usage stays bounded regardless of arrival rate.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
The catalogue
These are not thirteen independent ideas. They cluster: three are about handing work to a set of workers (thread pool, worker pool, producer/consumer), three about splitting one computation (fork/join, fan-out/fan-in, pipeline), three about avoiding shared state (actor, message passing, immutable snapshot), and three about limiting or gating access (read/write lock, semaphore, barrier). Single-flight is the odd one out, and it is the one most often missing from a system that needs it.
For each: the problem is the situation you are in, the structure is the arrangement of actors and state, and the failure modes are what you will actually be paged for. Deep-linked lessons carry the reasoning; this table is for recognition.
| Pattern | Problem it solves | Structure | Use when | Cost | Fails as |
|---|---|---|---|---|---|
| Producer/consumer | A fast producer and a slow consumer with no natural rendezvous | Producers enqueue, consumers dequeue, a bounded queue between them | Rates differ, or the two stages should scale independently | A queue to size, and a backpressure policy nobody wants to choose | Unbounded queue growing until memory dies; queue age rising while every stage looks healthy — Producer / Consumer |
| Thread pool | Thread creation costs more than the work, and unbounded threads kill the machine | A fixed worker set pulling from a shared queue | Many short tasks, and a ceiling on concurrency is required | Queue latency; a blocked worker is capacity removed | Saturation with all workers blocked on I/O and a growing queue — Thread Pools |
| Worker pool | The same, where the workers are processes or machines rather than threads | N workers consuming from a shared durable queue | Work must survive a worker dying, or exceed one process | Serialization, distribution and at-least-once delivery semantics | Poison message retried forever; duplicate execution on redelivery — Worker Pools Beyond Threads |
| Fork/join | One computation that splits into independent parts and must be recombined | Split, run children in parallel, wait for all, combine | Work is divisible and the parts are genuinely independent | Split and join overhead; the slowest child sets the time | Overhead exceeding the work on small inputs; one slow child pinning the join — Fork/Join |
| Fan-out/fan-in | One request needing results from N independent services | Issue N calls concurrently, await all, merge | The calls do not depend on each other | N times the downstream load, per request | Tail latency equal to the slowest branch; multiplied load overwhelming a downstream — Fan-Out / Fan-In: One Request Becomes N |
| Pipeline | A multi-stage transform where every item passes through every stage | Stages connected by bounded queues, each stage concurrent with the others | Stages have different costs and can overlap in time | Buffering between stages; end-to-end latency exceeds any single stage | The slowest stage setting throughput while faster ones idle — Pipeline Parallelism: Different Items, Different Stages |
| Actor | Shared mutable state with many concurrent writers | One actor owns the state; all access is a message to its mailbox | The state has a natural owner and per-entity ordering matters | Everything becomes asynchronous; request/response needs correlation | Mailbox growth under load; one slow actor blocking its whole entity — The Actor Model |
| Message passing | Two tasks needing to coordinate without sharing memory | Send owned or immutable values over a channel | Ownership can be transferred rather than shared | Copy or transfer cost; no shared view of state | Deadlock on unbuffered channels when both sides send first — Message Passing |
| Read/write lock | Many readers, occasional writer, on one structure | Shared read mode, exclusive write mode | Reads dominate and are long enough to justify the overhead | More expensive than a mutex when reads are short | Writer starvation under a steady reader stream — Read/Write Locks, Honestly |
| Semaphore / permits | A resource that supports N simultaneous users, not one | Acquire a permit, use the resource, release it | Bounding concurrency against a limited downstream | A permit leaked on an error path is capacity gone forever | Permit leak under an exception; deadlock when a holder waits for another permit — Semaphores: Counting Permits as a Resource Limit |
| Barrier | N tasks that must all reach a point before any proceeds | Each arrival blocks until the count is met, then all release | Phased computation where phase k+1 depends on all of phase k | Every phase costs the slowest participant | One participant never arriving and hanging all the others — Barriers |
| Single-flight | N concurrent callers requesting the identical missing thing | The first caller does the work; the rest await its in-flight result | A cache miss or lazy initialization under concurrent demand | Shared fate — everybody gets the one caller's error | Without it: a thundering herd of identical work on every miss — Single-Flight Coalescing |
| Immutable snapshot | Readers needing a stable view while a writer updates | Publish a new complete version; readers hold the one they got | Reads vastly outnumber writes | A full copy per write; old versions retained while referenced | Version retention growing with the slowest reader — Copy-on-Write as a Concurrency Strategy |
They compose, and the composition is where systems are designed
The ingestion service in the opening line is not one pattern; it is five, arranged. A pipeline of stages, each stage a thread pool consuming from a bounded queue, the enrichment stage doing a fan-out to three APIs behind a semaphore that bounds total in-flight calls, single-flight collapsing duplicate lookups for the same key, and the batch writer using an immutable snapshot of the mapping rules.
What composition buys is that each pattern handles one concern with a known failure mode. What it costs is that the failure modes interact: the semaphore limits enrichment concurrency, which slows that stage, which fills the queue behind it, which triggers backpressure on the parser, which is exactly the behaviour you wanted — but only if every queue in the chain is bounded. One unbounded queue anywhere converts the whole chain's backpressure into memory growth at that point.
The design rule that falls out: every boundary between stages needs an explicit answer to "what happens when the downstream is slower than the upstream". Block, drop, or reject. Those are the three, there is no fourth, and "grow" is not one of them because it is only "die later". See Backpressure.
Choosing: three questions in order
Most pattern selection collapses to three questions asked in this order. First: is there shared mutable state, and can it be removed? If it can — by immutability, by ownership transfer, or by giving it a single owner — the pattern you want is in the "avoid sharing" cluster and you can stop reading. This is the same move as Immutability as a Concurrency Strategy, applied at the architectural level.
Second: is the work divisible, or is it a stream? Divisible work with a join point is fork/join or fan-out/fan-in. A continuous stream is producer/consumer or a pipeline. Getting this wrong produces the two most common structural mistakes in the catalogue: a pipeline built where a fork/join was needed, so results arrive out of order and nobody notices; and a fork/join built where a stream was needed, so the system batches when it should flow.
Third: what is the ceiling, and who enforces it? Every pattern here needs a bound — pool size, queue capacity, permit count, fan-out width. A pattern chosen without its bound is an anti-pattern with a nice name, which is precisely the argument of Unbounded Concurrency and Bounding Concurrency.
1// "Pipeline" with unbounded stages and unbounded fan-out.2const parsed = records.map(parse) // all in memory3const enriched = await Promise.all( // 50,000 concurrent4 parsed.map(async (r) => ({5 ...r,6 ...(await Promise.all([apiA(r), apiB(r), apiC(r)])), // x3 = 150,0007 })),8)9await db.insertAll(enriched)1const sem = new Semaphore(64) // total in-flight API calls2const inflight = new Map<string, Promise<Enrich>>() // single-flight by key3 4async function enrich(r: Record): Promise<Enriched> {5 const key = r.customerId6 let p = inflight.get(key)7 if (!p) {8 p = sem.run(() => Promise.all([apiA(r), apiB(r), apiC(r)]))9 .finally(() => inflight.delete(key))10 inflight.set(key, p)11 }12 return { ...r, ...(await p) }13}14 15// Bounded stage: at most 32 records in flight, queue applies backpressure.16for await (const batch of chunked(pool(parseStream, 32), 500)) {17 await db.insertAll(await Promise.all(batch.map(enrich)))18}Both versions use fan-out/fan-in, a pipeline and a pool. Only the second has a number attached to each one. The bound is not an optimization added later — it is the part of the pattern that makes it a pattern rather than a hope.
Key points
- Thirteen recurring shapes, in four clusters: hand work to workers, split one computation, avoid sharing, and gate access.
- Recognising the shape is most of the work; the implementation is nearly always already in the standard library.
- Real systems compose several patterns, and the composition works only if every boundary between them is bounded.
- Choose in order: can the shared state be removed, is the work divisible or streaming, and what enforces the ceiling.
- A pattern named without its bound — pool size, queue capacity, permit count, fan-out width — is an anti-pattern with better branding.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Identify the shared mutable state, if any, and check whether ownership transfer or immutability removes it.
- • Classify the work: one divisible computation with a join, or a continuous stream of independent items.
- • Pick the cluster the classification implies, then the specific pattern by whether the stages have different costs and whether ordering matters.
- • Attach a bound to every dimension the pattern exposes, and decide the policy when that bound is reached: block, drop or reject.
- • Compose by connecting patterns through bounded boundaries, so that pressure propagates upstream instead of accumulating in memory.
- • Producer/consumer, bounded: P enqueues until the queue is full; P blocks; C dequeues one; P proceeds — the block is the backpressure signal, and it is the design working.
- • Producer/consumer, unbounded: P enqueues 4 million items while C processes 900 per second; nothing blocks, nothing errors, and the process dies of memory forty minutes later.
- • Fork/join with one slow child: children 1-7 finish in 10 ms, child 8 takes 900 ms, the join returns at 900 ms — parallelism gained nothing because the span, not the work, set the time.
- • Fan-out under a semaphore: 200 records arrive, each wanting 3 calls; 64 permits are held, record 22 blocks on acquire until a permit is released — downstream never sees more than 64 concurrent calls regardless of arrival rate.
- • Single-flight: 300 requests miss the cache for key K within 5 ms; the first issues the load, 299 attach to the in-flight promise; one backend call is made and 300 responses are served — and if that call fails, all 300 fail together.
- • Actor: two writers send "increment" to the same actor; the mailbox orders them; both are applied — no lock anywhere, and per-entity ordering is structural rather than enforced.
- • Barrier with a lost participant: 7 of 8 workers arrive at the barrier and block; the eighth threw an exception and exited without arriving; all 7 wait forever.
- • A bounded queue guarantees memory is bounded and that a full queue is observable; it does NOT guarantee the producer handles the block sensibly.
- • A thread pool guarantees at most N tasks run simultaneously; it does NOT guarantee any task starts within a bounded time, or that a blocked worker will ever come back.
- • Fork/join guarantees all children complete before the join returns; it does NOT guarantee any of them ran in parallel, or that the split was worth it.
- • A semaphore guarantees at most N permits are outstanding; it does NOT guarantee a leaked permit is ever returned, and a leak is permanent capacity loss.
- • An actor guarantees serial processing of its own mailbox; it does NOT guarantee ordering between different actors, or that the mailbox is bounded.
- • Single-flight guarantees one execution per key per window; it does NOT isolate callers from each other's failures — shared work means shared fate.
- • A barrier guarantees all participants have arrived before any proceeds; it does NOT notice a participant that died before arriving.
- • Queue-based patterns contend on the queue head and tail; at very high rates that becomes the bottleneck and argues for per-worker queues with stealing. See Work Stealing.
- • Pool patterns contend for workers: the queue wait time is the visible cost, and a blocked worker is capacity that has silently left the pool.
- • Gate patterns (semaphore, barrier, read/write lock) contend by design — that is their function — and the cost is measured as wait time on acquire.
- • Sharing-avoidance patterns move contention to the mailbox, the channel or the allocator rather than eliminating it.
- • Composition concentrates contention at the slowest stage, which is where every queue in the chain will be full and every earlier stage idle.
- • Unbounded queue growth, presenting as memory exhaustion or as queue age rising while throughput looks normal.
- • Pool saturation: every worker blocked on I/O, the queue growing, and CPU near zero — the signature that says the pool is not the bottleneck the pool metrics suggest.
- • Tail-latency amplification in fan-out: the request takes as long as the slowest of N branches, so p99 per branch becomes near-certain per request.
- • Downstream overload from multiplied load, where N-way fan-out per request turns a 2x traffic increase into a 6x increase somewhere else.
- • Permit or worker leak on an exception path, removing capacity permanently and silently.
- • Deadlock between patterns: a pool worker submitting to the same pool and waiting for the result, with no worker left to run it.
- • Shared-fate failure under single-flight, where one bad load fails every attached caller at once.
- • Silent reordering when a pipeline is used where ordering mattered and nothing enforces it.
- • When the problem is recognisably one of these shapes, which it usually is — reaching for the named pattern gets you its known failure modes and its known bounds for free.
- • When a team needs shared vocabulary: "this is a fan-out under a semaphore with single-flight" communicates a design in one sentence.
- • When composing stages with different costs, because bounded boundaries make the pressure visible and localise the bottleneck.
- • When reviewing: the catalogue turns "does this look right" into "which bound is missing".
- • When the work is not actually concurrent — a pattern applied to a sequential problem adds queues, workers and failure modes to buy nothing. See Concurrency Is Always Bought With Complexity.
- • When patterns are stacked without measurement, producing a pipeline of six stages where one stage was 95% of the time.
- • When the pattern name is adopted without its bound, which is the most common way a catalogue does damage.
- • When ordering requirements are real and the chosen pattern does not preserve order, which is discovered downstream and much later.
- • Queue depth and queue age at every boundary; age is the one that reveals a stalled consumer while depth still looks acceptable.
- • Pool utilization split into running versus blocked workers — the split is the diagnosis, the total is not.
- • Per-branch latency distribution in any fan-out, so the branch setting your p99 is identifiable rather than inferred.
- • Permit acquisition wait time and outstanding permit count, which together detect leaks and mis-sized limits.
- • Single-flight coalescing ratio: callers served per underlying execution. A ratio near 1 means the pattern is doing nothing.
- • End-to-end latency against the sum of per-stage latencies in a pipeline; a large gap is queueing, not processing.
- • Every pattern adds a bound to choose, a policy for exceeding it, and a metric to watch — three decisions each, multiplied by composition.
- • Composed patterns interact, so debugging requires understanding the whole chain rather than one stage.
- • Asynchrony spreads: introducing a queue converts a call stack into a correlation problem, and stack traces stop telling the whole story.
- • Error propagation must be designed per boundary — a failure in stage three has to reach whoever cares, and queues break the default path.
- • Cancellation must be designed per boundary too, or a cancelled request leaves work queued in three places. See Cancellation Propagation.
- • Do it sequentially and measure. A single-threaded loop is the correct answer far more often than a catalogue tempts you to believe. See Concurrency Is Always Bought With Complexity.
- • Use the runtime's built-in structure — a stream with backpressure, a bounded channel, a task group — instead of assembling one from primitives.
- • Push the coordination into infrastructure: a real queue broker gives you durability, retries and dead-lettering that an in-process queue does not.
- • Remove the shared state so that most of the catalogue becomes unnecessary. See Immutability as a Concurrency Strategy and Message Passing.
Concurrency lab
They come from a queueing and contention model inside Engineer Atlas. What is faithful is the behaviour: work that waits benefits from more workers, work that computes does not, a wide critical section pins parallelism near 1 no matter how many cores you buy, and arrivals past capacity produce an unbounded queue rather than a large latency. Real arrivals are burstier than this model assumes, so real systems reach every one of these walls earlier than the sliders suggest. Do not quote a millisecond from this page.
Producers, a bounded queue, consumers
One request, N downstream calls
Pipeline visualizer
What people believe, and what is true
Patterns are interchangeable — pick whichever is familiar.
They differ in what they guarantee about ordering, bounding and failure propagation. A pipeline where a fork/join was needed silently reorders results; a fork/join where a stream was needed batches work that should have flowed.
Using a well-known pattern makes the code correct.
It makes the failure modes known, which is not the same thing. Every pattern here fails, and most of them fail because a bound was omitted or a permit leaked on an error path.
More patterns means a better design.
Each one adds a queue, a bound, a metric and a failure mode. A pipeline of six stages where one stage is 95% of the cost is strictly worse than one stage and a measurement.
Go deeper
Overview
Most concurrency problems are one of about thirteen recognisable shapes. Identify the shape, use the library implementation, and attach its bound.
Practical
Ask in order: can the shared state be removed, is the work divisible or streaming, and what enforces the ceiling. Then choose, and write the bound down as a configured number.
Advanced
Design compositions so pressure propagates upstream: every boundary bounded, every bound with a stated policy on block/drop/reject, and cancellation and error propagation designed per boundary rather than inherited.
Internals
The clusters map onto what they do with memory. Queue patterns share a data structure and contend on its endpoints. Split patterns share a result region with disjoint writes and contend only at the join. Sharing-avoidance patterns move the cost to copying and allocation. Gate patterns contend on a single counter, which is one cache line and therefore a scaling limit of its own.