The Event-Driven Server

The kernel already knows when a socket becomes readable — it is the same wake-up that unblocks recv() in Follow send() Through the OS to recv(). I/O Multiplexing: select, poll, epoll, kqueue, IOCP exposes that knowledge: register a set of descriptors with an epoll instance (Linux), a kqueue (BSD, macOS) or an I/O completion port (Windows — a completion model, where you learn an operation *finished* rather than that it *could start*), then make one blocking call that returns when any of them is ready. The thread sleeps in that call; when it returns it holds a list of exactly the sockets with work.

Every socket is set non-blocking, so handling a ready socket means calling recv() until it returns EAGAIN, then moving on — never sleeping inside a handler. The cost per idle connection is a kernel entry in the interest set (a few hundred bytes) plus whatever the application keeps; the cost per event is one loop iteration. Ten thousand idle connections cost one sleeping thread. This is the C10K answer, and the reason C10K: Ten Thousand Connections, Then a Million stopped being a problem around the time epoll and kqueue shipped.

epoll is O(1) per ready event because the kernel maintains the ready list as sockets change state; select/poll are O(n) per call because the caller passes the whole set every time and the kernel scans it. At 10,000 sockets that difference is the difference between a server and a space heater. Level-triggered vs edge-triggered mode changes when a socket is reported (while ready vs on becoming ready) and is the source of most epoll bugs.

With threads, a connection’s state lived on the thread’s stack: "I have read the headers, I am waiting for the body". In an event loop the stack unwinds after every event, so the state must live in an explicit per-connection object: the bytes read so far, the parser’s position, the response not yet written. Every handler is "given the event and the connection state, advance the state machine one step, return". This inversion of control is the real cost of the model — the code is a state machine, and stack traces no longer tell a story.

Coroutines put the story back. async/await in JavaScript and Python, Rust futures, Kotlin coroutines, let the compiler or runtime turn the state machine back into sequential-looking code: await is the point where the handler returns to the loop and its locals are saved for later. Underneath it is still an event loop; the The Event Loop lesson covers how JavaScript’s loop, microtasks and the libuv thread pool fit together, and Async I/O: What `await readFile()` Actually Does covers the general form.

Writes need the same care as reads. A handler that calls send() with more than the send buffer can take gets a short write or EAGAIN; it must keep the remainder in the connection state and ask the loop for a writable event. Forgetting this is the most common bug in hand-written event servers and the reason frameworks own the write path — see What Happens When the Receiver Is Slow.

One thread serves every connection, so a handler that takes 200 ms takes 200 ms from every connection. A synchronous file read (fs.readFileSync, a plain open().read() in an asyncio handler), a large JSON.parse, a regular expression with catastrophic backtracking, a tight loop over a big array, a synchronous DNS lookup, a blocking call into a native library — each stalls the loop for its duration, and during that time no new connections are accepted, no responses are written, and every timer is late. The kernel keeps filling receive buffers; the application is not reading; senders see rwnd shrink. See the challenge Timers fire late, health checks fail, one core at 100%.

The rule is absolute because the failure is total and invisible from inside the handler. Diagnose it from outside: event-loop lag (the delay between a timer’s due time and its firing) is the metric; Node exposes it via perf_hooks.monitorEventLoopDelay, asyncio via a debug mode that logs slow callbacks. A loop with 50 ms lag is a server whose every request is 50 ms slower than it needs to be, regardless of the network.

Disk I/O deserves its own warning: on Linux a regular file is always reported ready by epoll, because the page cache will satisfy the read — eventually, after a disk seek that blocks the caller. Readiness does not apply to files. Runtimes hide this by doing file reads on a thread pool (libuv’s default of 4 threads) or, on recent Linux, with io_uring — see Memory Mapping, the Page Cache and Network I/O.

nginx runs one event loop per worker process, one worker per core, and handles hundreds of thousands of connections per box; it does almost no CPU work per request — routing, buffering, sendfile — which is exactly the profile the model rewards. Redis runs a single event loop for command processing and gets its throughput from the fact that every command is microseconds; a slow command (KEYS * on a large keyspace) stalls every client, which is why the documentation is so insistent about it. Node.js, HAProxy, Envoy, Netty, Tokio, asyncio, Go’s netpoller — all the same shape.

None of them are purely event-driven. CPU-heavy work — TLS handshakes in some designs, compression, image processing, request parsing for big bodies, any blocking library — goes to a worker pool, and the pool posts a completion event back to the loop. The The Thread Pool Server returns as a component: the loop does I/O and dispatch; the pool does work; a queue sits between them with the sizing and overflow questions from that lesson. Node’s libuv pool, nginx’s thread pools for file I/O, Envoy’s per-worker threads with a shared dispatcher, Go’s runtime with its M:N scheduler over a netpoller are all versions of this hybrid.

The honest comparison: an event loop wins when connections are many and mostly idle and per-event work is small; a thread pool wins when work per request is large and CPU-parallel and connections are few; a runtime with cheap threads over a netpoller lets you write the second and run the first. Choose by the shape of the workload, not by the fashion of the decade.