I/O Multiplexing: select, poll, epoll, kqueue, IOCP
A thread per socket is too expensive and a polling loop burns CPU, so the kernel provides one call that sleeps on many descriptors and returns the ready ones — select and poll scan the whole set per call, epoll and kqueue keep an interest list and return only what changed, and Windows IOCP reports completions instead of readiness.
The problem
EAGAIN. You need the kernel to wake you only when something actually happened — and to tell you what.Progressive depth
The same mechanism at different altitudes — start where you are.
Instead of one thread per connection or checking each connection in a loop, a server hands the kernel its whole list and goes to sleep. The kernel wakes it with the short list of connections that have data. The server handles those, then sleeps again. Node, Go and nginx all work this way.
Deriving the primitive
Start from what does not work. A thread per socket (Thread per Connection) parks each thread in a blocking read; 10,000 threads means 10,000 stacks, 10,000 scheduler entries, and a context switch every time any of them wakes — memory in the gigabytes and CPU time spent mostly switching. Polling in a loop with non-blocking sockets (Blocking, Non-blocking, Multiplexed, Asynchronous) uses one thread but issues 10,000 read calls per pass; at ~200 ns each that is 2 ms per lap to discover that nothing happened, and the core is pegged at 100% while idle.
What you actually want is to sleep until any of these is ready, then learn which. The kernel already knows: every socket has a wait queue that the network stack signals when data lands in its receive buffer (The Buffer Chain). Multiplexing is a syscall that puts one thread on all of those wait queues at once and, when woken, reports the subset that fired. The application then does real work — non-blocking reads — only on that subset.
The four designs below differ in how the set is described (per call or persistently), how the result is reported (the whole set with flags, or just the ready ones), and whether the kernel reports *readiness* (you may now read without blocking) or *completion* (the read you asked for has finished).
select and poll: describe the set every time
select (BSD, 1983) takes three fd_set bitmaps — readable, writable, exceptional — plus the highest descriptor number plus one and a timeout. The bitmaps are fixed at FD_SETSIZE bits, 1024 on Linux glibc, so a descriptor numbered 1024 or above cannot be watched at all and silently corrupts memory if you try. The kernel walks every bit up to nfds, registers on each descriptor’s wait queue, sleeps, and on wake walks them all again to fill in the result — O(n) per call in the kernel. Because the sets are overwritten with results, the caller must rebuild them from scratch before every call, another O(n) in user space.
poll (System V) replaces the bitmaps with an array of struct pollfd { fd, events, revents }. No 1024 ceiling and no rebuilding — the kernel writes results into revents and leaves events alone — but it is still O(n): the whole array is copied into the kernel and scanned on every call, whether one descriptor is active or all of them. With 10,000 descriptors and a few active, each poll costs tens of microseconds of pure scanning, and the cost grows with idle connections, which is exactly the wrong direction. Both are portable and fine for small sets; neither scales.
1std::vector<pollfd> fds; // one entry per socket, rebuilt as sockets come and go2for (auto s : sockets) fds.push_back({ s, POLLIN, 0 });3int n = poll(fds.data(), fds.size(), -1); // kernel copies + scans all of them4for (auto& p : fds)5 if (p.revents & POLLIN) handle_readable(p.fd); // and so do weepoll: a persistent interest list and a ready list
Linux’s epoll (2.5.44, 2002) splits registration from waiting. epoll_create1 returns a descriptor for an epoll instance — itself a kernel object with two structures: an interest list (a red-black tree of watched descriptors and the events each cares about) and a ready list. epoll_ctl(EPOLL_CTL_ADD) inserts a descriptor once; from then on the kernel hooks that descriptor’s wait queue so that when data arrives, the callback appends it to the ready list *at the moment of the event*, with no scanning. epoll_wait then sleeps until the ready list is non-empty and copies out only those entries: O(ready), independent of how many idle descriptors are registered.
The trigger mode is the part that bites. Level-triggered (default) reports a descriptor on every epoll_wait as long as it is ready — a socket with unread bytes keeps appearing until you drain it. Safe, forgiving, slightly wasteful. Edge-triggered (EPOLLET) reports it once, when its state *changes* from not-ready to ready; if you read some bytes and leave the rest, you will not be told again until *new* data arrives. Edge mode therefore requires non-blocking descriptors and a drain-until-EAGAIN loop, but it avoids repeated wakeups and makes multi-thread designs cleaner. EPOLLONESHOT disables a descriptor after one report so one thread at a time handles it; EPOLLEXCLUSIVE stops the thundering herd when several threads wait on one listening socket.
Two properties complete the picture. An epoll instance is a descriptor, so it can be registered in another epoll instance (nesting) and closed like anything else (Everything Is I/O). And because the hook is on the underlying open file description, a descriptor closed in one process while a dup survives elsewhere stays registered — the classic source of "epoll reports events on an fd I closed".
1int ep = epoll_create1(EPOLL_CLOEXEC);2epoll_event ev{ .events = EPOLLIN | EPOLLET, .data = { .fd = sock } };3epoll_ctl(ep, EPOLL_CTL_ADD, sock, &ev); // once per socket, O(log n) into the interest tree4 5epoll_event out[256];6for (;;) {7 int n = epoll_wait(ep, out, 256, -1); // sleeps; returns only ready entries8 for (int i = 0; i < n; ++i)9 drain_until_eagain(out[i].data.fd); // edge-triggered: must read until EAGAIN10}kqueue and IOCP: the other operating systems
kqueue (FreeBSD 2000; macOS, other BSDs) is the same idea as epoll with a broader vocabulary. One kevent call both modifies the interest list (a *changelist*) and waits (an *eventlist*), saving a syscall per iteration. Its filters cover more than sockets: EVFILT_READ/WRITE, EVFILT_VNODE (file changes — what fswatch and editors use), EVFILT_PROC (child exit), EVFILT_SIGNAL, EVFILT_TIMER. Linux reaches the same coverage only by wrapping each source in its own descriptor (inotify, pidfd, signalfd, timerfd) and adding those to epoll. Edge (EV_CLEAR) and one-shot (EV_ONESHOT) flags mirror epoll’s.
Windows does not multiplex readiness at scale; it multiplexes completions. An I/O completion port (IOCP, since NT 3.5) is a kernel queue you associate handles with. You issue *overlapped* operations — WSARecv with a buffer you own — and they return immediately; when the kernel has finished the receive, including the copy into your buffer, it posts a completion packet to the port. A small pool of threads calls GetQueuedCompletionStatus to dequeue packets, and the port itself limits how many of them run concurrently (typically the core count), waking a waiting one only when a running one blocks. Windows select exists but only for sockets and is O(n); WaitForMultipleObjects is capped at 64 handles. IOCP is the primitive, and it is completion-based — the same contract Linux later adopted in io_uring (Blocking, Non-blocking, Multiplexed, Asynchronous).
| Primitive | OS | Set described | Cost per wait | Reports | Limit | Notes |
|---|---|---|---|---|---|---|
select | POSIX, Windows (sockets only) | bitmaps, rebuilt every call | O(n) kernel + O(n) user | readiness (flags in the sets) | fd < 1024 (FD_SETSIZE) | portable; memory corruption above the limit |
poll | POSIX | array of pollfd, every call | O(n) | readiness (revents) | none | portable; scales badly with idle fds |
epoll | Linux | persistent interest list | O(ready) | readiness; level or edge | none | regular files not allowed |
kqueue | BSD, macOS | persistent; change + wait in one call | O(ready) | readiness; many filters (vnode, proc, timer, signal) | none | closest to a universal event source |
| IOCP | Windows | handles associated with a port | O(completions) | completion (the op is done, buffer filled) | none | kernel-managed thread concurrency |
Who uses which: libuv, Go, nginx
Node.js / libuv wraps all of them: epoll on Linux, kqueue on macOS and BSDs, event ports on Solaris, IOCP on Windows. On the readiness platforms libuv performs the non-blocking reads itself after epoll_wait; on Windows it issues overlapped reads and dequeues completions. Regular-file operations go to a thread pool (default size 4, UV_THREADPOOL_SIZE) on every platform, because readiness has nothing to say about them. The The Event Loop you see in JavaScript is the loop around one of these waits.
Go runs a netpoller — epoll, kqueue or IOCP — owned by the runtime, not by your code. A goroutine calling conn.Read on a socket with no data is parked (its tiny stack saved) and the descriptor registered with the netpoller in edge-triggered mode; when the kernel reports readiness the scheduler makes the goroutine runnable again. You write blocking-looking code and get multiplexing underneath; blocking *file* syscalls, by contrast, occupy an OS thread and the runtime spawns more as needed.
nginx is the reference design for the C10k era: one worker process per core, each running an epoll (or kqueue) loop over thousands of client and upstream connections, with EPOLLEXCLUSIVE or accept_mutex so that a new connection wakes one worker rather than all of them. No thread per connection, no blocking calls on the loop; disk-heavy work (sendfile of large static files, AIO) is offloaded to a thread pool. Redis, HAProxy, Envoy and every serious proxy are variations on the same shape (The Event-Driven Server).
Key points
- A thread per socket costs memory and switches; polling costs CPU. Multiplexing sleeps one thread on all wait queues and returns the ready subset.
select: bitmaps, 1024-descriptor ceiling, O(n) and rebuilt every call.poll: arrays, no ceiling, still O(n) per call.epoll: register once (interest list), wait many times (ready list), O(ready). Level-triggered repeats until drained; edge-triggered fires on change and demands drain-until-EAGAIN.kqueueis epoll’s BSD/macOS counterpart with filters for files, processes, signals and timers; Linux adds those via extra descriptors.- Windows IOCP reports *completions* of overlapped operations, not readiness — a different contract, later mirrored on Linux by
io_uring. - libuv, Go’s netpoller and nginx are all loops around one of these primitives, with thread pools for the file I/O the primitives cannot cover.
Why does this exist?
Mechanisms are answers to constraints. Open each question before reading the answer.
▸Why did epoll separate registration from waiting?
Because the set of interesting descriptors changes slowly while waits happen constantly. Registering once lets the kernel attach a callback to each descriptor’s wait queue, so events append themselves to a ready list as they occur instead of being discovered by scanning.
▸Why does edge-triggered mode exist if level-triggered is safer?
Level mode re-reports a socket on every wait until it is drained, which wastes wakeups under load and makes it awkward for several threads to share one epoll instance. Edge mode reports each transition once, so a busy loop does less redundant work — at the price of a strict drain discipline.
▸Why does readiness not work for regular files?
A file is always "readable" — the question is whether the bytes are in the page cache, and the kernel only finds out inside the read. There is no state change to report. Completion-based I/O sidesteps this by owning the whole read.
▸Why do runtimes hide all this behind `async`/goroutines?
Because a hand-written epoll loop forces every handler into a resumable state machine. Coroutines and green threads let the runtime park and resume code at the wait points, so the developer writes sequential logic while the kernel sees one multiplexed thread.
I/O multiplexing
t=0 select(nfds, &readfds, …) → returns 5 in a bitmap you must scan app work this tick: 12 fds inspected
How it fails
What the failure looks like from inside real software.
- A server compiled with
selectcrashes or corrupts memory once it has more than 1024 descriptors open; the accept loop starts returning garbage under load. poll-based loop spends most of its CPU insidepollas idle connections accumulate — throughput drops as the *idle* population grows.- Edge-triggered handler reads one chunk and returns; remaining bytes never trigger another event; the client waits forever on a half-processed request.
- Multiple workers sharing a listening socket all wake on every connection (thundering herd); CPU spikes with connection rate although each worker does almost nothing.
- A descriptor is closed while a
dupelsewhere keeps its open file description alive; epoll keeps reporting events for a number the application no longer recognises. - A blocking call on the loop thread — synchronous DNS, a page-cache miss on a big file, JSON parsing of a 200 MB body — stalls every connection the loop owns.