The question this answers
How do I get CPU work off the event loop without giving up the safety the single-loop model was buying me?
A report endpoint that renders a 6 MB spreadsheet — roughly 900 ms of pure CPU — on a Node server that also answers ordinary API requests.
By default, nothing. Each worker has its own isolate, its own heap and its own event loop; a posted message is a *copy*. The exceptions are explicit and dangerous: SharedArrayBuffer (genuinely shared memory across isolates) and transferred buffers (moved, not shared — the sender loses access).
The main loop's p99 stays bounded by the cost of its own handlers, not by the cost of the heaviest report; and every byte a worker mutates is either owned exclusively by that worker or accessed through Atomics.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Separate isolates, separate loops, messages in between
A Worker is an OS thread running a fresh V8 isolate with its own heap and its own event loop. It is a real thread — the OS schedules it on any core, and a while (true) {} in a worker occupies a core without touching the main loop at all. That is the whole point, and it is the thing async alone cannot give you (Async Is Not Parallelism).
Because the isolates are separate, objects are not shared. postMessage(obj) runs the structured clone algorithm: a deep copy that handles Maps, Sets, TypedArrays, Dates and cyclic references, but throws on functions, class prototypes, DOM-like host objects and anything holding a closure. The copy costs time proportional to the payload, and that cost is paid *on the sending loop* — which means a badly-sized message can block the very loop you were trying to protect.
Transfer is the escape hatch: postMessage(buf, [buf]) moves an ArrayBuffer's backing store instead of copying it. Constant time, no copy — and the sender's buffer is detached, byteLength becomes 0, and any later read throws. Ownership moved; treat it exactly like a move in C++.
A pool, not a worker per request
Spawning a Worker costs a thread plus a fresh isolate plus module evaluation — order tens of milliseconds, and tens of megabytes of RSS. Doing it per request converts a CPU problem into a memory-and-startup problem, and under load into Oversubscription: more runnable threads than cores, so everything context-switches and nothing finishes sooner.
The shape that works is a bounded pool sized against cores, with a queue in front of it and a limit on that queue. Note what the code below refuses to do: it does not promise a thread count formula. The right size depends on how much of the job is genuinely CPU, whether the workers also do I/O, what else shares the machine, and what your latency target is. Size it, measure the pool's queue depth and the job's wall-clock, and adjust — see Sizing a Thread Pool.
The queue bound is not optional. Without it, a burst of report requests grows an unbounded array of pending jobs, memory climbs, and every queued job eventually times out having consumed a worker slot for nothing. Bound it and reject early: Bounding Concurrency.
1import { Worker } from 'node:worker_threads'2import os from 'node:os'3 4type Job = { payload: ArrayBuffer; resolve: (r: ArrayBuffer) => void; reject: (e: Error) => void }5 6class RenderPool {7 private idle: Worker[] = []8 private queue: Job[] = []9 10 // Starting point only. os.availableParallelism() reports what the process is11 // ALLOWED to use (cgroup-aware); it is not an answer, it is a first guess.12 constructor(private size = Number(process.env.RENDER_WORKERS ?? os.availableParallelism()),13 private maxQueue = 100) {14 for (let i = 0; i < this.size; i++) this.idle.push(this.spawn())15 }16 17 private spawn(): Worker {18 const w = new Worker(new URL('./render.worker.js', import.meta.url))19 w.on('error', (err) => { this.replace(w, err) }) // an uncaught throw kills the thread20 w.on('exit', (code) => { if (code !== 0) this.replace(w, new Error('worker exit ' + code)) })21 return w22 }23 24 run(payload: ArrayBuffer, timeoutMs = 30_000): Promise<ArrayBuffer> {25 return new Promise((resolve, reject) => {26 if (this.queue.length >= this.maxQueue) {27 return reject(new Error('render queue full')) // shed load HERE, not in the OOM killer28 }29 const job: Job = { payload, resolve, reject }30 const w = this.idle.pop()31 if (w) this.dispatch(w, job, timeoutMs)32 else this.queue.push(job)33 })34 }35 36 private dispatch(w: Worker, job: Job, timeoutMs: number) {37 const timer = setTimeout(() => { w.terminate(); job.reject(new Error('render timeout')) }, timeoutMs)38 w.once('message', (result: ArrayBuffer) => {39 clearTimeout(timer)40 job.resolve(result)41 const next = this.queue.shift()42 if (next) this.dispatch(w, next, timeoutMs)43 else this.idle.push(w)44 })45 // Transfer, not clone: O(1) handoff, and job.payload is detached afterwards.46 w.postMessage(job.payload, [job.payload])47 }48 49 private replace(dead: Worker, err: Error) { /* drop, respawn, fail the in-flight job */ }50}SharedArrayBuffer: the moment data races come back
Everything above is message passing, and message passing is safe because nothing is shared. SharedArrayBuffer deliberately breaks that: the same bytes are visible to several isolates on several OS threads. Plain reads and writes to a Uint8Array over a SharedArrayBuffer from two threads, where at least one writes, are an actual data race under the ECMAScript memory model — not merely a race condition. JS defines the outcome (unlike C++, where it is undefined behaviour), but what it defines is that you may observe values from any interleaving, with no ordering guarantee between separate locations.
Atomics.add, Atomics.compareExchange, Atomics.load/store are the sanctioned accesses: indivisible, sequentially consistent with respect to each other, and the only way to build a correct counter or lock over shared bytes. Atomics.wait and Atomics.notify give you blocking and wakeups — wait is forbidden on the main thread in the browser and allowed but almost always wrong on Node's main thread, because it blocks the loop by design.
The honest recommendation is to reach for SharedArrayBuffer only when the copy cost is genuinely the bottleneck and the access pattern is a numeric buffer, not an object graph. See Atomics: What Is Actually Indivisible and What a Memory Model Defines for what Atomics actually orders, and False Sharing: Different Variables, Same Cache Line for why two workers hammering adjacent counters can be slower than one.
| # | Worker 1 (core 0) | Worker 2 (core 1) | State |
|---|---|---|---|
| 1 | load view[0] → 7 | · | memory=7 w1=7 w2=- |
| 2 | · | load view[0] → 7 | memory=7 w1=7 w2=7 |
| 3 | compute 7 + 1 = 8 | · | memory=7 w1=8 w2=7 |
| 4 | · | compute 7 + 1 = 8 | memory=7 w1=8 w2=8 |
| 5 | store view[0] = 8 | · | memory=8 w1=8 w2=8 |
| 6 | · | store view[0] = 8 | memory=8 w1=8 w2=8 ✕ Two increments produced one. The counter says 8 and both workers returned success; nothing logged an error and no exception was thrown. |
view[0] += 1 is three machine steps, and on separate cores they interleave. Atomics.add(view, 0, 1) makes the read-modify-write one indivisible operation and the counter is correct. The same schedule on one event loop would be impossible — which is exactly the safety you gave up when you shared the buffer.Key points
- A worker thread is a real OS thread with its own V8 isolate, heap and event loop — the only way to run JavaScript on two cores in one Node process.
- Nothing is shared by default:
postMessagestructured-clones, and the clone cost is paid on the *sending* loop, so oversized messages block the loop you were protecting. - Transfer moves an
ArrayBuffer's backing store in O(1) and detaches the sender's view — ownership transfer, not sharing. - Use a bounded pool with a bounded queue; a worker per request trades a CPU problem for a memory, startup and oversubscription problem.
SharedArrayBufferreintroduces genuine data races. Plain+= 1from two workers loses updates;Atomics.adddoes not.- There is no correct universal worker count. Size it from cores, job mix and latency target, then measure queue depth and wall-clock.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- •
new Worker(path)creates an OS thread, a fresh isolate and a fresh event loop, and evaluates the module in it — tens of milliseconds and tens of megabytes. - • The two sides communicate over a
MessagePortpair; eachpostMessageenqueues onto the receiver's loop as an ordinary task. - • Structured clone serialises the payload on the sender and deserialises on the receiver; both halves cost CPU on their respective loops.
- • Listing an
ArrayBufferin the transfer list moves its backing store instead: the receiver gets the same memory, the sender's view is detached. - • A
SharedArrayBufferis mapped into both isolates; no copy, no transfer, and no protection — onlyAtomicsprovides indivisibility and ordering. - • An uncaught exception in a worker fires
erroron the parent and terminates that thread, so a pool must detect the exit and respawn or the pool shrinks to nothing.
- • Main loop posts a 6 MB object; the structured clone takes 40 ms on the main loop; three queued HTTP handlers each see 40 ms of added latency — the offload made the loop worse, not better.
- • Worker 1 loads counter (7); Worker 2 loads counter (7); both store 8 — two increments become one, on two cores, with no error anywhere.
- • Main transfers a buffer to a worker and then reads
buf.byteLength; it is 0 and the subsequent read throws — the buffer was moved, not copied. - • A worker throws; the parent's
errorhandler fires but the in-flight job's promise was never rejected; the caller waits until its timeout with a worker slot held — the pool leaks capacity one job at a time. - • Sixteen workers on eight cores, each pegged: every job's wall-clock roughly doubles while total throughput stays flat. Oversubscription in its purest form.
- • The schedule that works: main transfers the input, the worker computes with an isolate nobody else touches, and transfers the result back. No shared state, no interleaving to reason about.
- • Guaranteed: worker code cannot observe or mutate main-thread objects. Isolation is enforced by the isolate boundary, not by convention.
- • Guaranteed: messages between a given port pair are delivered in order.
- • Guaranteed: a worker runs on a real OS thread, so CPU work in it genuinely proceeds in parallel with the main loop.
- • Guaranteed:
Atomicsoperations on aSharedArrayBufferare indivisible and sequentially consistent with respect to one another. - • NOT guaranteed: that
postMessageis cheap. Clone cost is proportional to the payload and is charged to the sender. - • NOT guaranteed: that plain reads and writes over a
SharedArrayBufferare safe. They are a data race with defined-but-useless outcomes. - • NOT guaranteed: that a worker is preemptible from outside. Only
terminate()stops a spinning worker, and it stops it without unwinding. - • NOT guaranteed: any speedup. If the job is I/O-bound, moving it to a worker adds copies and scheduling and gains nothing.
- • Worker startup contends for CPU with the main loop; spawning a pool during a traffic spike makes the spike worse.
- • Structured clone is a serial CPU cost on the sender's loop — the contention is with every other handler on that loop.
- • Runnable threads beyond available cores produce context-switch churn and cache pollution; wall-clock per job rises while throughput does not.
- • The pool queue is the real backpressure point; its depth is the number you alert on, and its bound is what keeps the process alive.
- •
SharedArrayBuffercounters on adjacent cache lines produce coherence traffic between cores — False Sharing: Different Variables, Same Cache Line, which turns a "shared, therefore fast" design into the slow one.
- • Data race on a
SharedArrayBufferaccessed withoutAtomics: lost updates, torn logical values across multiple fields. - • Detached-buffer error: reading a transferred
ArrayBufferon the sender after handing it over. - • Serialisation failure:
postMessagethrowsDataCloneErroron functions, class instances with prototypes, and host objects. - • Worker death without job rejection: the pool loses a slot and the caller hangs until timeout.
- • Unbounded job queue under burst: memory growth, then every queued job times out having achieved nothing.
- • Oversubscription: more busy workers than cores, so latency degrades for everyone with no throughput gain.
- • Deadlock via
Atomics.waiton the main thread — the loop stops, and the worker that would notify it is queued behind the loop.
- • Genuine CPU work on the request path: rendering, image and video processing, compression, cryptographic hashing, large parses and transforms.
- • Work whose input and output are buffers, so transfer makes the handoff free.
- • Isolating a job that may crash or spin, since
terminate()kills a worker without taking the process down. - • Multi-core throughput in a single process, when running multiple processes is undesirable because of memory or connection-pool count.
- • I/O-bound work — the loop already handled that, and you have added copies and thread scheduling for nothing.
- • Small jobs, where clone plus dispatch plus response exceeds the work; the crossover is real and you find it by measuring.
- • Large object graphs, where structured clone dominates and there is no buffer to transfer.
- • Designs that reach for
SharedArrayBufferto avoid copies and thereby import the entire threaded-memory-model problem into a codebase built on the assumption it did not have one.
- • Main-loop lag before and after the offload — if it did not fall, the clone cost ate the win. See Event-Loop Lag: One Callback, Everybody Waits.
- • Job wall-clock split into queue time, clone time, compute time and response time. The first two are the ones that surprise people.
- • Pool queue depth and rejection count; depth trending up means the pool is undersized or the jobs got heavier.
- • Process RSS per worker, since each isolate carries its own heap and a pool of 32 is a memory decision as much as a CPU one.
- • For the shared-memory case: an assertion that the counter equals the expected total after a stress run. A data race will not show up in latency or in errors — only in accounting.
- • You now maintain two runtimes in one process, with separate module graphs, separate error handling and separate lifecycle.
- • Every message payload becomes a serialisation contract; adding a class instance to a job object breaks it at runtime, not at compile time.
- • Pool lifecycle — spawn, respawn on death, drain on shutdown, timeout and terminate — is a few hundred lines you must get right or the pool degrades silently.
- • Debugging spans threads: a stack trace from a worker has no main-thread frames, and profilers must be pointed at each isolate.
- • If
SharedArrayBufferenters the design, the team now needs the memory-model reasoning the single-loop model had spared them.
- • Split the work into chunks and yield to the loop between them (
setImmediate), when the job is a loop over items and latency for others matters more than the job's own duration. - • A separate process —
child_processor a queue-backed worker service. Higher memory, slower handoff, far better isolation, and it scales past one machine. Usually the right answer for reports. - • A native addon that releases the isolate lock and does the work in C++ on its own threads — no clone cost at all, at the price of a build toolchain.
- • Do it out of band: enqueue the job, return
202, and let the client poll. If the work is 900 ms, the honest answer is often that it should not be on the request path at all — seeasync-job-pattern.
CPU parallelism simulator
Amdahl’s term, a synchronisation term, an oversubscription term and a bandwidth ceiling, each one a knob you can switch off. Real curves have more causes than four and are rarely this smooth. There is no ideal core count to read off this chart.
Thread pool: utilization and queue
capacity = workers / service = 8 / 50 ms = 160.0 req/s ρ = arrivals / capacity = 120 / 160.0 = 0.750 Little L = λ × W → 0.120/ms × 59.8 ms = 7.2 in flight engine status = healthy
More workers than cores
What people believe, and what is true
Worker threads let me share objects between threads.
They let you *copy* or *move* them. The only genuinely shared thing is a SharedArrayBuffer, and it comes with a memory model attached.
postMessage is basically free.
Structured clone is proportional to the payload and is charged to the sending loop. A 6 MB message can block the main loop longer than the work you offloaded.
More workers means more throughput.
Past the number of cores the process can actually use, extra runnable workers add context switches and cache pressure. Throughput flattens and per-job latency rises.
Go deeper
Overview
A real thread with its own isolate and loop; you send it a copy of the work and it sends back a copy of the result.
Practical
Bounded pool, bounded queue, transfer buffers instead of cloning them, reject when the queue is full, and respawn workers that die.
Advanced
SharedArrayBuffer plus Atomics avoids the copy entirely and reintroduces data races, memory ordering and false sharing. Worth it for numeric buffers under measured copy pressure; rarely worth it otherwise.
Internals
Each worker is a V8 isolate on its own libuv loop inside the same process, so they share the address space but not the object graph. SharedArrayBuffer works precisely because the address space is shared — the isolate boundary is a language-level construct, not a memory-protection one.