The question this answers
Why did one slow synchronous function make every unrelated endpoint slow at the same time?
A /settings handler that parses a 12 MB JSON body and hashes a password with bcrypt.hashSync — about 200 ms of synchronous CPU — on a service also serving /health and /orders.
The event loop itself. It is not data, but it is the shared resource under contention, and every pending callback in the process is queued on it.
No task occupies the loop long enough to push another task past its deadline: practically, the longest synchronous span stays below the tightest deadline any pending task has.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
The cost is multiplied, not added
A 200 ms synchronous handler is easy to dismiss because 200 ms sounds tolerable. The arithmetic that matters is different: if forty callbacks are ready or become ready during that window, the loop has added up to 200 ms to each of them. Forty requests, each with an unrelated deadline, each pushed by work they never touched. This is head-of-line blocking with the whole process as the queue (Head-of-Line Blocking).
The diagnostic signature is distinctive once you know it. Latency rises on endpoints that share no code, no database and no dependency with the slow one. /health — which does nothing — degrades in lockstep. The p99 rises while the p50 barely moves, because only requests unlucky enough to overlap a block are affected. And the traces blame whichever dependency happened to be in flight, because its span includes the queue wait.
The most expensive consequence is usually not the latency itself. It is that the health check times out, the load balancer removes the instance, and the traffic it was carrying moves to instances that are about to do the same thing. A CPU-bound handler is how a single slow endpoint becomes a cascading outage.
What the evidence looks like
Event-loop lag is the direct measurement: schedule a timer for 10 ms, measure how late it actually fired, and publish the distribution. A healthy loop reads sub-millisecond; a blocked one reads whatever the block was. The performance domain treats this as a first-class signal — see event-loop-lag.
The read-out below is what a real incident looks like: lag p99 spikes to the length of the block, /health p99 crosses its timeout, and /orders p99 rises by roughly the block length while the database's own latency does not move at all. That last row is the one that ends the argument about whose fault it is.
Two more sources worth wiring up. In the browser, the Long Tasks API reports every main-thread block over 50 ms with attribution. In Node, a CPU profile taken during the spike shows the synchronous frame directly, and --prof or a continuous profiler will name the function without any guessing.
# event loop lag (ms) — self-scheduling 10ms timer, per 30s window window p50 p95 p99 max 14:00–14:30 0.4 1.1 2.3 6 14:30–15:00 0.5 1.2 2.6 8 15:00–15:30 0.6 48.0 198.4 241 <-- deploy at 15:02 added bcrypt.hashSync 15:30–16:00 0.6 51.2 204.1 256 # per-route p99 latency (ms), same windows route 14:30–15:00 15:30–16:00 delta POST /settings 240 445 +205 <-- the culprit; its own work got no slower GET /health 1 203 +202 <-- does nothing. shares nothing. timeout is 100ms. GET /orders 14 219 +205 <-- unrelated route, unrelated DB GET /static/app.js 2 198 +196 <-- served from memory # dependency latency as reported BY the dependency, same windows postgres p99 8.1 8.3 +0.2 <-- did not move redis p99 0.9 1.0 +0.1 <-- did not move # what the distributed trace says (and why it is misleading) GET /orders total 219ms |- span: db.query 211ms <-- 8ms of query, 203ms waiting for the loop to run the continuation |- span: serialize 6ms # The span starts when the request is issued and ends when the continuation runs. # Loop wait is inside the child span, so the trace accuses the database.
The fix depends on which problem you have
There are three fixes and they solve different problems, so pick by symptom. If the work is unavoidable and must stay on the request path, move it to a worker: the loop is free and the batch is also faster (Worker Threads). If the work is a loop over items and you mostly need other requests served, chunk it and yield. If the work is a single opaque synchronous call — hashSync, a synchronous compression call, a native module — neither chunking nor yielding is available, and the only options are the async variant of the same API or a worker.
The comparison below shows the most common real instance: a synchronous crypto call plus a large parse. The async variant of bcrypt.hash is not merely a wrapper — it dispatches to libuv's thread pool, so the hashing genuinely happens on another thread. That is also why a burst of concurrent hashes serialises behind a pool whose size defaults to a small number: fixed, not free (Thread Pools).
And the fix nobody writes down: stop accepting 12 MB bodies. A size limit on the route removes the parse cost entirely and is one line. Bounding the input is almost always cheaper than parallelising the work done to it.
1app.post('/settings', (req, res) => {2 // 12 MB body: JSON.parse is synchronous and unbounded. ~60 ms, all on the loop.3 const body = JSON.parse(req.rawBody)4 5 // bcrypt cost factor 12: ~150 ms of pure CPU, synchronous, no yield point.6 const hash = bcrypt.hashSync(body.password, 12)7 8 db.saveSettings(body.userId, { ...body, hash }).then(() => res.json({ ok: true }))9 // The await is at the END. Everything expensive already happened inline.10})1// 1. Bound the input. The cheapest fix, and the one that is skipped.2app.use(express.json({ limit: '256kb' }))3 4app.post('/settings', async (req, res) => {5 const body = req.body // already parsed, and bounded6 7 // 2. The async variant dispatches to libuv's thread pool: this task SUSPENDS,8 // the loop is free, and the hashing runs on a real thread.9 // Note the pool is small and shared with fs/dns — concurrent hashes queue.10 const hash = await bcrypt.hash(body.password, 12)11 12 await db.saveSettings(body.userId, { ...body, hash })13 res.json({ ok: true })14})15 16// 3. For work with no async variant, chunk and yield — fixes responsiveness,17// not duration. Use a worker instead when duration matters.18async function summarise(rows) {19 let acc = emptyAccumulator()20 for (let i = 0; i < rows.length; i += 500) {21 acc = reduceChunk(acc, rows.slice(i, i + 500))22 await new Promise((r) => setImmediate(r)) // macrotask yield, so I/O gets a turn23 }24 return acc25}The size limit removes 60 ms of parse outright. bcrypt.hash turns 150 ms of loop occupancy into a suspension, so every other pending callback runs during it. The chunked reducer keeps the loop responsive for work that has no async variant — but note the yield must be a macrotask (setImmediate / setTimeout(0)), because await Promise.resolve() is a microtask and lets no I/O in at all.
Key points
- A 200 ms synchronous handler costs 200 ms times the number of tasks waiting, not 200 ms once.
- The signature is latency rising on endpoints that share nothing with the slow one —
/healthdegrading is the giveaway. - Distributed traces blame the dependency, because loop wait is counted inside the dependency's span while the dependency's own latency does not move.
- Event-loop lag — a self-scheduling timer's scheduled-versus-actual delta, published as a distribution — is the direct measurement.
- A blown health check turns a slow endpoint into a cascading outage as the load balancer removes instances that are all doing the same thing.
- Three fixes for three problems: bound the input, suspend instead of running inline (async API or worker), or chunk-and-yield for responsiveness only.
- Yield with
setImmediateorsetTimeout(0);await Promise.resolve()is a microtask and lets no I/O through.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • The loop runs one callback at a time and cannot preempt it — there is no quantum and no scheduler interrupt for JavaScript tasks.
- • While that callback runs, socket readiness, timer expiry and completed pool operations continue to arrive and are enqueued.
- • None of the enqueued work executes until the running callback returns or suspends.
- • When it finally returns, the entire backlog drains at once, producing a burst of near-simultaneous responses whose measured latency includes the wait.
- • Timers that were due during the block fire late by the length of the block; that lateness is exactly what an event-loop-lag probe measures.
- • If a health check was among the delayed tasks and its deadline is shorter than the block, the instance is marked unhealthy and its traffic is redistributed.
- • Loop enters
bcrypt.hashSyncat t=25 ms;/healthbecomes ready at t=50 ms; the health check's 100 ms deadline expires at t=150 ms while its callback has still not been called; the loop returns at t=225 ms and the health check answers in 1 ms — correct, and 175 ms too late. - • A database response arrives at t=30 ms; its continuation is queued; it runs at t=226 ms; the trace records a 196 ms query against a database that answered in 8 ms.
- • A metrics-flush timer due at t=100 ms fires at t=226 ms, so the metric batch is attributed to the wrong window and the graph shows a gap followed by a spike.
- • Ten concurrent requests each call the *async*
bcrypt.hash; the loop is free, but libuv's pool has few threads, so hashes serialise there instead — the loop is healthy and the endpoint is still slow. The bottleneck moved, which is progress, and it moved somewhere you must now size. - • The schedule with no problem: the same handler with a 256 KB body limit and
await bcrypt.hash. Longest synchronous span is under a millisecond; every other task runs on time.
- • Guaranteed: a running callback completes before any other task runs. That is the model, and it is why the block is total rather than partial.
- • Guaranteed: no task on the loop is preempted, so no amount of priority configuration helps.
- • Guaranteed: work enqueued during a block is not lost — it is delayed, and it all becomes runnable at once.
- • NOT guaranteed: that a "fast" endpoint stays fast. Its latency is a property of the whole process, not of its own code.
- • NOT guaranteed: that moving to the async API removes the bottleneck — it moves it to libuv's thread pool, which is small and shared.
- • NOT guaranteed: that chunking shortens anything. It restores responsiveness and leaves the duration alone (Async Is Not Parallelism).
- • NOT guaranteed: that a CPU profile shows it. If sampling happens between blocks you see nothing; profile *during* the spike.
- • The loop is the single contended resource, and contention is measured as queue delay rather than as lock waiting.
- • The backlog burst after a block is itself a contention event: forty callbacks all become runnable at once and drain serially.
- • libuv's thread pool is the second contention point once you move CPU work to async APIs; a burst of
bcrypt.hash,fsorzlibcalls queues there while sockets stay fast. - • Under sustained load, blocks compound: the backlog from one block delays the next request enough that its own block starts later, and lag ratchets upward instead of recovering.
- • Head-of-line blocking: unrelated endpoints degrade together, and the correlation is the diagnosis.
- • Health-check timeout leading to instance removal, traffic redistribution, and a cascading outage across a fleet running identical code.
- • Trace misattribution: the dependency is blamed because loop wait sits inside its span.
- • Timer drift: scheduled work fires late, so rate limiters, cache sweeps and metric flushes all operate on the wrong windows.
- • Watchdog kills: some supervisors kill a process whose loop is unresponsive, converting slow into crashed.
- • Thread-pool saturation after the "fix", where the loop is healthy and the endpoint is still slow.
- • In the browser: dropped frames, unresponsive clicks, and a Long Task that shows up as a failed interaction metric (Web Workers).
- • Nothing about blocking helps — but *knowing* the model helps: for a single-user CLI or a batch script with one task, synchronous code is simpler and the block costs nobody anything.
- • A short synchronous span is genuinely cheaper than a suspension: a sub-millisecond parse should stay inline rather than paying for a worker handoff.
- • During startup, before the server accepts connections, synchronous work is free — that is the right place for config parsing and schema compilation.
- • Any multi-tenant server process, because the cost falls on requests that had nothing to do with the work.
- • Any process with a health check or heartbeat whose deadline is shorter than the block.
- • Browsers always: the same thread renders, so a block is visible jank rather than a latency number.
- • Systems with tight tail-latency targets, where a rare 200 ms block dominates p99 even at a low rate of occurrence.
- • Event-loop lag as a distribution (p50/p95/p99/max) from a self-scheduling timer; alert on p99, never on the mean.
- • Count of synchronous tasks exceeding a threshold, with attribution — Node's async hooks or a continuous profiler; the browser's
longtaskPerformanceObserver. - • Correlate p99 across unrelated routes. If they move together and dependency latency does not, it is the loop.
- • Compare dependency-reported latency with client-observed latency for the same call; the gap is loop wait.
- • A CPU profile captured *during* the spike, not after — the frame is obvious once you have the right window.
- • Health-check success rate and instance-removal events, which is where this stops being a latency issue and becomes an availability one.
- • Fixing it usually means introducing a worker pool or an async API path, with the queueing, sizing and failure handling that come with them.
- • Chunk-and-yield introduces interleaving points where state can change mid-computation, so the reasoning from Await Is a Yield Point now applies to a function that used to be atomic.
- • Once CPU work moves to libuv's pool, you own a second capacity dimension that has no obvious dashboard.
- • Guarding against regression needs a test or a lint rule — a
hashSyncreintroduced in a later PR is invisible in code review and obvious only in production.
- • Bound the input first: body size limits, row limits, pagination. Removing the work beats moving it, and it is one line.
- • Use the async variant of the same API where one exists — most crypto, compression and filesystem calls have one that uses the runtime's thread pool.
- • Move the work off the request path entirely: enqueue it, return
202, and let a worker fleet do it (Bounding Concurrency andasync-job-pattern). - • Run one process per core and accept that a block affects one process's share of connections rather than all of them — mitigation, not a fix, but it caps the blast radius.
- • Choose a threaded model for this service, if blocking work is intrinsic to it and the loop was the wrong model from the start (Event Loop or Threads?).
Three I/O calls, one thread
sequential const a = await getProfile(); const b = await getFlags(); const c = await getOrders() concurrent const [a, b, c] = await Promise.all([getProfile(), getFlags(), getOrders()]) wall clock 261 ms → 127 ms (2.06× less waiting) CPU used 21 ms → 21 ms (identical — no extra core was touched)
Scheduler timeline
Concurrency lab
They come from a queueing and contention model inside Engineer Atlas. What is faithful is the behaviour: work that waits benefits from more workers, work that computes does not, a wide critical section pins parallelism near 1 no matter how many cores you buy, and arrivals past capacity produce an unbounded queue rather than a large latency. Real arrivals are burstier than this model assumes, so real systems reach every one of these walls earlier than the sliders suggest. Do not quote a millisecond from this page.
What people believe, and what is true
200 ms is fine, users will not notice.
The user of *that* endpoint might not. Every other in-flight request pays it too, and the health check with a 100 ms timeout definitely notices.
The trace says the database is slow, so the database is slow.
The span includes the time the continuation waited for the loop. Compare against latency reported by the database itself; if that did not move, the loop is the problem.
We switched to the async API, so the loop problem is solved.
The loop problem is solved and a thread-pool problem may have replaced it. libuv's pool is small and shared; a burst of concurrent hashes now queues there.
Go deeper
Overview
One callback runs at a time and cannot be interrupted. A long synchronous callback delays everything that was waiting, by its full length.
Practical
Measure loop lag as a distribution. Bound inputs, use async variants, move real CPU work to workers, and yield with setImmediate — never with Promise.resolve().
Advanced
Under sustained load the backlog from one block delays the next arrival enough that lag ratchets rather than recovering; the system has a load level above which it does not return to baseline without shedding.
Internals
The loop is blocked inside your stack frame, so it never reaches epoll_wait to collect readiness. Events are not lost — the kernel buffers them — but nothing is dequeued until the frame returns, which is why the recovery is a burst rather than a ramp.