RuntimesRUNTIME-SPECIFICGENERAL

Blocking the Event Loop

One function that does not yield holds the only thread that makes progress, so every concurrent request pays for it.

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

Why did latency rise on every endpoint at once when only one of them changed?

The requirement

The service must keep serving all its callers even when one endpoint does something expensive. Today it does not, and nobody can see why.

The obvious build

The expensive endpoint is slow. Optimise that handler, or give it a longer timeout, and the rest of the service is unaffected.

Why it breaks

The rest of the service is affected. On a single-loop runtime, every in-flight request, every timer and every pending connection waits for the blocking function to finish (The Node Event Loop).

How it breaks in production

The rest of the service is affected. On a single-loop runtime, every in-flight request, every timer and every pending connection waits for the blocking function to finish (The Node Event Loop).
Health checks time out during the block, so the orchestrator restarts a process that was working correctly — turning a latency problem into a capacity problem (Health Checks: Startup, Readiness, Liveness).
Adding instances helps only in proportion: each still blocks completely whenever it runs the expensive path, so p99 stays bad while average load per instance falls.
The profile of the slow endpoint looks unremarkable — the function really does take the time it takes. The damage is entirely in what else was waiting.
The bug is intermittent by traffic shape: it only hurts when something else is in flight, so it never reproduces in a single-user test.

Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

A single-loop runtime runs one callback to completion before starting the next. There is no preemption, so a function that does not return does not give the thread back.
While blocked, three things stop: pending I/O completions are not delivered, timers do not fire, and new connections are not accepted (Accepting Connections).
The visible effect is correlated latency: unrelated endpoints get slower at the same moments, in the same amounts. That correlation is the fingerprint.
The usual sources are ordinary code: a large JSON.parse or JSON.stringify, synchronous file or crypto calls, template rendering of a big result set, a regular expression with catastrophic backtracking, an in-memory sort or filter over a very large array, and tight loops over a big collection.
await does not help unless what you awaited is genuinely asynchronous — a promise that resolves after CPU work still holds the thread for that work (The Node Event Loop).
Garbage collection is a second, less obvious blocker: a major collection pauses the same thread, so heap pressure shows up in the same lag metric (Memory Leaks in Backend Services).
The same shape exists in Python asyncio — a synchronous call inside an async handler blocks that loop identically (Python Runtime Models). The failure is a property of single-loop concurrency, not of a language.

The fingerprint: correlated latency

The reason this is hard to diagnose is that the evidence appears everywhere except the cause. A health endpoint that does nothing gets slow. A cached lookup gets slow. They get slow at exactly the same moments, by exactly the same amount, and none of their code changed.

That correlation is the diagnosis. Application-level problems are endpoint-shaped; runtime-level problems are process-shaped. Once you can tell those apart in a dashboard, this class of incident goes from days to minutes.

UserLLMAgentToolDataDecisionHumanGuardrail

Where the blocks actually come from

Almost none of these look dangerous in review. They are ordinary operations whose cost is proportional to an input that used to be small — which is why this bug is usually introduced long before it is noticed.

Common blockers and what to do instead

Trigger	Symptom	Cause	Response
Serializing a large result set to JSON	Lag spikes proportional to response size	`JSON.stringify` is synchronous and CPU-bound	Paginate; project fewer columns; stream the response (Pagination That Survives a Large Table)
Sorting or aggregating a large array in memory	Lag scales with row count; database looks idle	Work the database could do was pulled into the process	Push the aggregate into SQL; bound the row count
Regular expression over user input	Rare, enormous spikes from tiny requests	Catastrophic backtracking (ReDoS)	Cap input length; avoid nested quantifiers; use a non-backtracking engine
`readFileSync` / `pbkdf2Sync` / `gzipSync` in a handler	Lag proportional to file size or cost factor	The synchronous API runs on the loop thread	Use the async variant, which dispatches to the libuv pool (The Node Event Loop)
Template rendering of a large page	Lag correlates with item count	String building is CPU work	Stream the render; cache fragments; reduce the item count
Heap pressure	Regular lag spikes unrelated to any endpoint	Major garbage collections pause the same thread	Reduce allocation and retention; investigate as a leak if it grows (Memory Leaks in Backend Services)

Three ways out, in order of preference

The first question is always whether the work needs to be in the request at all. Most of the time it does not, and moving it to a job is both the simplest and the most durable fix (Request or Background?).

When it must be synchronous with the request, the choice is between chunking — cheap to implement, keeps the loop responsive, slower overall — and a worker thread, which restores real parallelism at the cost of a boundary you must pass data across.

Chunking, and offloading

1// (1) Chunk: yield to the loop between slices.
2// Total time goes UP slightly; the loop keeps serving.
3async function transformInChunks<T, R>(items: T[], fn: (t: T) => R, size = 500) {
4  const out: R[] = []
5  for (let i = 0; i < items.length; i += size) {
6    for (const item of items.slice(i, i + size)) out.push(fn(item))
7    // setImmediate targets the check phase: the loop gets to run
8    // pending I/O callbacks before we take the thread back.
9    await new Promise((resolve) => setImmediate(resolve))
10  }
11  return out
12}
13 
14// (2) Offload to a worker thread: real parallelism, at the cost
15// of a serialization boundary. Keep the messages SMALL -- passing
16// a huge object costs structured-clone time on the sending side,
17// which is back on the loop you were protecting.
18import { Worker } from 'node:worker_threads'
19 
20function renderReport(rowIds: string[]): Promise<Buffer> {
21  return new Promise((resolve, reject) => {
22    const w = new Worker('./report-worker.js', { workerData: { rowIds } })
23    w.once('message', resolve)
24    w.once('error', reject)
25    w.once('exit', (code) => {
26      if (code !== 0) reject(new Error(`worker exited with ${code}`))
27    })
28  })
29}
30 
31// (3) Best of all, usually: do not do it in the request.
32//   const job = await queue.add('render-report', { rowIds })
33//   res.status(202).json({ jobId: job.id })

Option 3 changes the API contract and is still normally the right answer: the work is unbounded, the client rarely needs it synchronously, and a queue gives you retries and a place to see failures (Background Jobs).

How to build it

Most important first.

Measure loop lag first and continuously. Correlated latency plus a lag spike is a diagnosis; either one alone is a guess (The Metrics a Backend Must Emit).
Move work by category: latency-sensitive CPU to a worker thread, everything deferrable to a job queue, and anything the database can do to the database (Background Jobs).
Bound inputs so that per-request work has a ceiling — page sizes, array lengths, nesting depth, upload size. Unbounded input turns a linear function into an unbounded block (Pagination That Survives a Large Table).
Chunk what must stay in-process: process a slice, yield with setImmediate, continue. Total time rises slightly; the loop keeps serving.
Push serialization and aggregation down where possible — computing an aggregate in SQL rather than pulling a million rows and reducing them in application memory (What Serialization Costs).
Ban the synchronous APIs on request paths with a lint rule rather than a code review habit. readFileSync at startup is fine; in a handler it is a time bomb.
Treat regular expressions over user input as a hazard: prefer non-backtracking constructs, cap input length, and test with adversarial strings.

What can go wrong

Failure modes

Cascading restarts: health checks fail during blocks, the orchestrator kills instances, the remaining instances take more traffic and block more often (Cascading Failure).
Timeouts firing on requests that were never actually slow — they were waiting for the thread, and the timeout is measured from when they arrived.
Client and load balancer retries arriving during a block, multiplying the queue that already exists (Retry Storms).
A worker thread introduced as the fix, then given large objects to serialize across the boundary, moving the cost rather than removing it.
Chunking done with setTimeout(fn, 0) inside a hot loop, adding timer overhead per chunk; setImmediate belongs to the phase designed for this.
The mitigation failing: work moved to a queue whose consumers run in the same process on the same loop.

What can race

A block delays timers, so timeout logic fires late and can cancel work that had in fact completed — the cancellation and the completion race (Timeouts).
During a block, retries queue behind the same thread; when it frees, they all execute nearly simultaneously and can produce duplicate effects if the endpoint is not idempotent (Idempotency in Backends).
Chunked work yields between slices, so another request can observe and mutate intermediate state that a synchronous version would never have exposed (Backend Races).

Security

This is a denial-of-service surface reachable by a single unauthenticated request if the expensive path is public. Complexity limits are a security control (Transport Validation).
Catastrophic regular-expression backtracking turns a short attacker-supplied string into seconds of CPU — the classic ReDoS, and it blocks the entire process, not one request.
Deeply nested or very large JSON is a parser-cost attack within a modest byte limit; limit depth and element counts, not only size (Deserialization: Bytes to Objects).
Because blocking degrades every concurrent user, its impact is service-wide, which makes it a higher-severity availability issue than the same bug on a multi-worker runtime (The Backend Security Checklist).

Misreads

"Only that endpoint is slow." Every endpoint is slow while the block runs. The affected requests are the ones with no relationship to the cause.
"Wrapping it in an async function fixes it." Async-ness is about yield points, not about where code runs. No yield point, no relief.
"Promise.all runs things in parallel." It runs them concurrently on one thread; if the work is CPU-bound it is strictly sequential with extra bookkeeping (The Node Event Loop).
"Scale out and it goes away." Blast radius shrinks; each instance still stops completely during each block.
"This is a Node problem." Any single-loop runtime has it, including Python asyncio and single-threaded C++ event loops. The mitigations are the same shape everywhere.

Operating it

How you see it in production

Event-loop lag p99, graphed against request latency. When they move together, the runtime is the cause and no handler is.
Latency correlation across unrelated endpoints — including a trivial health endpoint — is the cheapest confirmation available.
A CPU profile captured during a spike shows the blocking frame at the top of the stack directly (Why Is My API Slow?).
Garbage-collection pause duration and frequency, so heap pressure can be separated from application CPU.
Per-endpoint request-size and result-size histograms: blocks usually scale with an input someone stopped bounding (Pagination That Survives a Large Table).

What changes at 10x and 100x

At 10x traffic, the probability that something else is in flight during a block approaches one, so an occasional annoyance becomes a permanent p99 problem.
At 100x, blocking is the difference between a service that scales horizontally and one that does not: each instance loses full availability during each block, so capacity planning must count blocked time as downtime.
More processes reduce blast radius proportionally — with 8 processes a block costs one-eighth of capacity rather than all of it — which is a real mitigation and not a fix (Worker Processes).

What this costs

Chunking keeps the loop responsive and makes the chunked operation slower overall and the code harder to read.
Worker threads restore parallelism and add message-passing cost, a second heap, and a lifecycle to manage.
Moving work to a queue removes it from the request entirely and changes the contract: the caller now gets a 202 and needs somewhere to find the result (Status Codes From the Server's Side).
Input limits protect the loop and reject legitimate large requests, which becomes a product conversation rather than an engineering one.

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

RUNTIME-SPECIFICSevere on single-loop runtimes — Node, Python asyncio, single-threaded C++ event loops — where one thread makes all the progress. On a thread-per-request JVM server or a Go service the same code consumes one thread or one core and degrades capacity proportionally instead of stopping everything; on a pre-fork Python setup it occupies exactly one worker (Worker Processes).
GENERALThe general rule survives every runtime: work that cannot be preempted holds whatever unit of concurrency it is running on. Only the size of that unit relative to your total capacity changes.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

Securityinput-validation

Performanceevent-loop-lag cpu-profiling flame-graphs tail-latency garbage-collection

Concurrencyblocking-the-event-loop event-loops async-is-not-parallel bounding-concurrency

OS & Networkingevent-loop io-models