Async & Event Loops

Async Is Not Parallelism

Marking a function async does not make it run on another core. Async lets one execution context switch between tasks *while they are suspended* — so it buys you nothing at all when the tasks never suspend. A CPU-bound loop inside an async function is a CPU-bound loop that also allocates a state machine.

▶ Run the lab

The question this answers

The question

Why did wrapping this CPU-heavy function in async and awaiting them all together change nothing?

The work

Ten images to resize, roughly 300 ms of pure CPU each, in a Node handler that already had async on every function in the call chain.

What is shared

Nothing between the resize tasks — each owns its own pixel buffer. Which is exactly the point: this workload has no synchronization problem at all, only an execution problem, and async solves the wrong one.

The invariant — what must stay true under every interleaving

Total wall-clock for the batch is at least the total CPU time divided by the number of execution contexts actually available. On one loop that number is one, and no amount of async changes it.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

Work→Can it overlap?→Can it parallelise?→What is shared?→What ordering?→What synchronization?→Where is contention?→What can deadlock?→What can race?→What is gained?→What complexity?

Suspension is the mechanism, and CPU work never suspends

The reason async helps I/O is precise: an awaited I/O operation *gives back the executor* while it waits. Ten concurrent HTTP calls overlap because during the 200 ms each spends on the network, the executor is free to start the next one. Ten concurrent resizes do not overlap, because during the 300 ms each spends in a pixel loop the executor is not free — it is running that loop.

Promise.all([r(a), r(b), r(c)]) on CPU-bound functions does not run them together. It starts the first one, which runs to completion because it never yields, then the second, then the third. The total is the sum, plus the cost of the promises. The word "concurrent" is doing no work here at all — see Which One Does This Workload Need? for the distinction being applied.

The honest framing: async is a scheduling tool for tasks that wait; parallelism is an execution tool for tasks that compute. Reaching for the first when you needed the second is one of the most common performance mistakes in JavaScript and Python codebases, and it is expensive because the code looks like it should have worked.

Ten CPU-bound tasks under Promise.all on one loop. The lane is never free; nothing overlaps.ILLUSTRATIVE
Event loop — 10 CPU tasks via Promise.all
resize 1
resize 2
resize 3
resize 4 … 10 (continues)
Other pending requests on the same loop
ready at t=1, cannot run
For contrast — 10 I/O tasks via Promise.all
all 10 suspended; executor free
↑ I/O batch done↑ CPU batch done — 10x longer
runningreadywaitingblockedidle1 tick ≈ 100 ms

The code that looks parallel and is not

Three versions below. The first is what people write; the second is the yielding version that fixes *responsiveness* and not *duration*; the third is the one that actually uses more cores. The middle one is worth dwelling on, because it is the fix people reach for and it does something different from what they expect.

Yielding to the loop between chunks lets other pending callbacks run, so the health check answers and frames get painted. It does not make the batch finish sooner — in fact it makes it slightly slower, because you have added scheduling to the same total CPU. If your problem is "the server stopped responding", yielding is the fix (Blocking the Event Loop). If your problem is "the batch takes three seconds", only more execution contexts will help (Worker Threads).

Note the third version does not claim a speedup number. Actual scaling depends on cores available to the process, whether the resize library releases anything while it works, memory bandwidth for the pixel data, and the transfer cost of the buffers. Measure it; do not assume it.

1// 1. Looks parallel. Is strictly sequential, plus promise overhead.
2async function resizeAll(images: ArrayBuffer[]) {
3  return Promise.all(images.map((img) => resizeCpuBound(img)))
4  // resizeCpuBound never awaits anything, so it runs to completion
5  // the moment it is called. Ten of them = ten in a row.
6  // Total: ~3000 ms. Loop lag during it: ~3000 ms.
7}
8 
9// 2. Yields between chunks. Fixes responsiveness, NOT duration.
10async function resizeAllYielding(images: ArrayBuffer[]) {
11  const out: ArrayBuffer[] = []
12  for (const img of images) {
13    out.push(resizeCpuBound(img))
14    await new Promise((r) => setImmediate(r))   // let the loop breathe
15  }
16  return out
17  // Total: ~3000 ms or slightly more. Loop lag during it: ~300 ms per chunk.
18  // Other requests now get served. The batch is not one millisecond faster.
19}
20 
21// 3. Adds execution contexts. This is the one that reduces wall-clock.
22async function resizeAllParallel(images: ArrayBuffer[], pool: RenderPool) {
23  return Promise.all(images.map((img) => pool.run(img)))
24  // Now Promise.all is doing what it looked like it was doing all along:
25  // the tasks suspend (waiting on a worker) so the executor IS free.
26  // Wall-clock depends on cores available, transfer cost and memory
27  // bandwidth. Measure it. Do not assume images.length / workers.
28}
Same work, three execution models. Only the third adds execution contexts.

What the speedup curve actually looks like

Adding "async" to CPU-bound work produces a speedup of exactly 1.0 no matter how much you add — the curve is a flat line, and it is flat for a structural reason rather than an overhead reason. Adding real execution contexts produces a curve that rises and then bends, and the bend is the interesting part: transfer costs, memory bandwidth for large pixel buffers, and the serial fraction of the handler all pull it away from linear (Amdahl's Law, Why Eight Cores Give You Four and a Half).

The curve below is modelled, not measured, and that matters: the position of the bend is a property of your data size, your library and your machine. What is *not* machine-dependent is the flat line. That one is arithmetic.

Modelled: async-only versus real worker parallelism on a 300 ms-per-image CPU batch.SIMULATED
1 workerdashed = linear speedup16 workers · max 16.0×
The async-only variant is not plotted as a curve because it has none: ten async functions, a hundred, or a thousand all give a speedup of 1.0 on one loop, because none of them suspends. The plotted curve is the worker-pool variant, and it bends for the ordinary reasons — serial fraction, transfer cost and memory bandwidth — before turning down at oversubscription.

Key points

  • Async exploits *suspension*. Work that never suspends gets no benefit, and Promise.all over CPU-bound functions runs them one after another.
  • The word "concurrent" describes overlapping progress, not simultaneous execution — see Which One Does This Workload Need?.
  • Yielding to the loop between chunks fixes responsiveness for other tasks and does not shorten the batch by a single millisecond.
  • Only additional execution contexts — worker threads, processes, native threads — reduce wall-clock for CPU-bound work.
  • The speedup from adding async to CPU work is 1.0 regardless of how much you add. That is arithmetic, not a benchmark result.
  • Real parallel speedup bends away from linear because of serial fractions, transfer cost and memory bandwidth, and turns down at oversubscription.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • • An async function returns a promise and installs a state machine; that is the entire runtime effect of the keyword.
  • • The state machine only splits execution at await points, and only if the awaited thing is not already settled.
  • • A function body with no await inside a hot loop has no split points, so it runs to completion the first time it is called.
  • • Promise.all starts each function in order; a function that never yields finishes before the next one starts.
  • • The executor — one event loop — can only run one of those bodies at a time, so total wall-clock is the sum of the bodies.
  • • Handing the body to a worker changes this because the calling task now *does* suspend: it waits on a message, freeing the executor, while an OS thread runs the work on another core.
Interleavings that matter
  • • Promise.all over ten CPU tasks: task 1 runs 0–300 ms, task 2 runs 300–600 ms, and so on. There is no interleaving at all — that is the finding.
  • • Promise.all over ten I/O tasks: all ten start within a millisecond, all ten suspend, the executor idles, and all ten resume as responses arrive. Wall-clock is roughly the slowest one.
  • • Mixed: one CPU task and nine I/O tasks; the nine I/O responses arrive during the CPU task's 300 ms and every one of their continuations queues behind it, so all nine appear to take 300 ms longer than they did.
  • • Chunked with a yield: task 1 runs 0–300 ms, the loop serves a health check at 300 ms, task 2 runs 301–601 ms. Other work is served; the batch still ends at ~3 s.
  • • Worker pool of four: four tasks run simultaneously on four cores, the loop is free the whole time, and the batch ends in roughly a quarter of the time minus transfer overhead.
What it guarantees — and does not
  • • Guaranteed: async gives you a promise and the ability to suspend at await. Nothing else.
  • • Guaranteed: on one loop, exactly one task body executes at a time, so CPU time is strictly additive across tasks.
  • • Guaranteed: yielding between chunks lets other pending callbacks run.
  • • NOT guaranteed: any overlap. Overlap requires suspension, and suspension requires something to wait for.
  • • NOT guaranteed: that Promise.all does anything concurrently. It waits on a collection; whether the members overlap is a property of the members.
  • • NOT guaranteed: that adding workers gives linear speedup — that depends on cores, transfer cost, memory bandwidth and the serial fraction.
  • • NOT guaranteed (CPython): that threads give you CPU parallelism for pure-Python bytecode; that is what multiprocessing and native extensions are for. See Python: Threads, Processes and the GIL.
Where contention appears
  • • The executor is the contended resource, and a CPU-bound task holds it for its entire duration with no preemption.
  • • Every other pending task on the loop is queued behind it, so the contention shows up as latency on endpoints that share nothing with the slow one.
  • • With a worker pool the contention moves to the pool queue and, past the core count, to the CPU scheduler itself.
  • • Large pixel buffers add memory-bandwidth contention between cores, which is why eight workers rarely give eight times the throughput on memory-heavy work (Memory Bandwidth: More Cores, Same Bus).
How it fails
  • • False parallelism: a batch that was "made concurrent" and takes exactly as long as before, with nobody able to explain why.
  • • Event-loop starvation: the CPU batch blocks health checks and unrelated endpoints, and the instance is pulled from rotation.
  • • The wrong fix applied: chunked yielding shipped to solve a duration problem, restoring responsiveness while the batch stays slow and everyone believes it is fixed.
  • • Oversubscription after over-correcting: a worker per image, more runnable threads than cores, and per-job latency doubles while throughput stays flat.
  • • Benchmark illusion: measuring the batch on an idle laptop with 10 cores and shipping it to a container limited to 0.5 CPU.
When it helps
  • • Async helps when tasks genuinely wait: network calls, disk, database queries, sleeping, waiting on a worker or another process.
  • • Async helps when per-task memory matters and the tasks are numerous and mostly idle.
  • • Parallelism helps when the work is CPU-bound, divisible, and large enough that the handoff cost is small relative to the compute.
  • • Both help together in the hybrid shape: async at the edge, real threads or processes for the compute (Hybrid Runtimes: It Was Never Threads Versus Async).
When it hurts
  • • Async on CPU-bound work adds a state machine, promise allocations and microtask scheduling for zero overlap — a small, measurable regression.
  • • Parallelising work smaller than the handoff cost is slower than doing it inline (Parallel Overhead).
  • • Parallelising memory-bound work scales far worse than core count suggests, because the bottleneck is bandwidth, not compute.
  • • Adding workers past the cores the process can actually use degrades latency for everyone.
How you would know
  • • Compare wall-clock against total CPU time for the batch. If wall-clock ≈ sum of the parts, nothing overlapped, whatever the code says.
  • • Event-loop lag during the batch: high lag means the work is on the loop; near-zero lag with a long batch means it is genuinely elsewhere.
  • • Per-task start and end timestamps. Printing them is the fastest way to prove that "concurrent" tasks ran back-to-back.
  • • CPU utilisation across cores: one core at 100% and seven idle is the signature of async applied to a parallelism problem.
  • • For the worker version, measure speedup at 1, 2, 4 and 8 workers on the target hardware — including inside the container CPU limit, which is where the surprise is.
Complexity it introduces
  • • Marking things async propagates through the call graph and is hard to undo, so a change that bought nothing still costs you readability forever.
  • • Chunked yielding introduces a chunk size that must be tuned, and interleaving points where state can change — reintroducing the await-point reasoning of Await Is a Yield Point for no throughput gain.
  • • Real parallelism adds a pool, a serialisation boundary, a queue bound and a failure path for dead workers.
  • • The diagnosis itself is the hidden cost: teams often spend weeks on the wrong axis because the code reads as though it should be parallel.
Simpler alternatives
  • • Make the work smaller: resize on upload rather than on request, cache the result, or use a smaller source image. The fastest parallel batch is the one you do not run.
  • • Move it off the request path entirely: enqueue and return 202, then process on a dedicated worker fleet. See async-job-pattern.
  • • Use a native library that releases the interpreter lock or runs its own threads, so one call uses several cores with no pool of your own.
  • • Scale out processes instead of threads — one process per core — when shared memory is unnecessary and per-process overhead is acceptable.
  • • CPython specifically: multiprocessing or a native extension for CPU work; threads there overlap I/O well and do not run pure-Python bytecode in parallel (Python: Threads, Processes and the GIL).

CPU parallelism simulator

Scaling 100 CPU tasks
100 independent tasks of 20 ms each. The tasks do not share anything — the job around them does.
SIMULATEDA composed model, not a benchmark.

Amdahl’s term, a synchronisation term, an oversubscription term and a bandwidth ceiling, each one a knob you can switch off. Real curves have more causes than four and are rarely this smooth. There is no ideal core count to read off this chart.

Cores
The serial part is the split and the merge, not the tasks. The sync term is what each worker pays to coordinate with the others. The ceiling is where the memory system stops feeding cores, whatever the core count says.
1 workerdashed = linear speedup16 workers · max 16.0×
ideal
4.0× · 500 ms
Amdahl only
3.48×
modelled
3.28× · 610 ms
efficiency
82%
Where the 4× went
delivered3.3×
lost to the serial part0.5×
lost to sync, switching and bandwidth0.2×
At 4 cores the model delivers 3.28× of a possible 4×, so 109 ms of the run is overhead rather than work. The serial part dominates. Splitting the input, merging the results and the one section that cannot overlap now cost more than the cores save — and no core count fixes that term.
One hundred tasks that share nothing still do not scale linearly, because the job that owns them is not the tasks. Read the gap between the dashed line and the curve as the price of coordination — and note it is charged even when every task is independent.
limited by: serialSIMULATED

Why is 8 cores only 4.5×?

Why is 8 cores only 4.5×?
Amdahl is one term of four. Turn each effect off and watch which part of the curve straightens.
1 workerdashed = linear speedup16 workers · max 16.0×
workersidealAmdahl onlyrealisticlimited by
11.0×1.00×1.00×none
22.0×1.90×1.85×serial
44.0×3.48×3.19×serial
66.0×4.80×4.17×serial
88.0×5.93×4.17×bandwidth
1010.0×6.90×4.17×bandwidth
1212.0×7.74×4.17×bandwidth
1414.0×8.48×4.17×bandwidth
1616.0×9.14×4.17×bandwidth
speedup at 8 workers
4.17×
best point on the curve
4.17× @ 6
past best, adding workers
costs
distinct causes on curve
serial, bandwidth
At 8 workers this configuration reaches 4.17× and the dominant cause is "bandwidth". Past 6 workers the cores are fed by a memory system that is already saturated — they are stalled, not computing. More threads make the stall queue longer. The fix is fewer bytes per unit of work (better locality, smaller types), not more parallelism. The reason to name the cause is that each one has a different fix, and three of the four get worse if you respond by adding threads.
SIMULATEDcomposed from named effects, not fitted to a measurement

Amdahl's law: the serial ceiling

Amdahl's law — the serial fraction sets a ceiling
Speedup = 1 / (s + (1 − s)/n). The serial part does not get faster, so it decides the answer long before the core count does.
1 workerdashed = linear speedup32 workers · max 32.0×
speedup at 32
7.80×
ceiling at ∞ workers
10.0×
efficiency
24.4%
workers doing nothing
24.2 of 32
s = 0.10   n = 32
Amdahl    S(n) = 1 / (s + (1 − s)/n) = 7.805×        ← fixed problem, more machine
                 S(1 000 000)        = 10.000×     ← a million cores, and still under 10×
Gustafson S(n) = s + n(1 − s)        = 28.900×        ← fixed time, bigger problem
10.0% serial caps you at 10.0×, forever. At 32 workers you get 7.80× — 24.4% efficiency, with 24.2 workers' worth of capacity paid for and idle. A million cores would only reach 10.00×. The lever is not the core count; it is the 10.0%. Shrink the serial region (a smaller critical section, a lock-free counter, a per-worker accumulator merged once) and the whole curve moves. Buy hardware and nothing moves.
fixed problem, growing machineSIMULATED

What people believe, and what is true

Claim

I used Promise.all, so the work runs in parallel.

Reality

Promise.all waits on a collection. Whether its members overlap depends entirely on whether they suspend. Ten CPU-bound functions run strictly one after another.

Claim

Making the function async lets the runtime move it to another thread.

Reality

No runtime does that. async installs a state machine on the same execution context; moving work to another thread is an explicit act.

Claim

Chunking with setImmediate made it faster.

Reality

It made the *rest of the system* responsive. The batch takes the same time or marginally longer, because you added scheduling to the same CPU work.

Go deeper

Overview

Async overlaps waiting. If the task never waits, there is nothing to overlap and nothing changes.

Practical

Compare wall-clock with total CPU. If they match, the work is serial. Yield to fix responsiveness; add workers or processes to fix duration; do not confuse the two.

Advanced

Real speedup is bounded by the serial fraction, the handoff cost and memory bandwidth. Measure at 1, 2, 4, 8 contexts on the target hardware — including the container CPU limit, which is where the model usually breaks.

Internals

An async function compiles to a resumable state machine with suspension points only at await. With no suspension point the compiled body is an ordinary function that happens to return a promise, and the scheduler never gets a chance to run anything else.

Apply it

🔍 Every Endpoint Got Slower At Once🔍 We Made the Endpoint 30x Faster and Broke the Database💬 Concurrency or Parallelism?💬 Async or Threads?💬 What Does the GIL Actually Prevent?💬 Why Is Eight Cores Not Eight Times Faster?
← Event Loop or Threads?Blocking the Event Loop →