Async I/O vs threads

Both are ways to have many tasks in progress. Threads give each task its own stack and let the kernel schedule it: blocking calls are fine, CPU-bound work runs in parallel (outside the GIL), and a slow task is preempted so it cannot hurt the others’ latency. The cost is per-thread state — an 8 MB virtual stack of which tens of KB are resident, a kernel task — plus context switches and the synchronization needed for shared data. Ten thousand threads is possible; a hundred thousand is where scheduler cost and memory become the ceiling (Threads versus Async versus Processes).

Async multiplexes many tasks on one thread by never blocking it: every wait becomes a registration with epoll/kqueue/IOCP and a continuation. The per-task cost is a few hundred bytes to a few KB, so a million idle connections is feasible, and there are no data races between tasks on one loop. The price is cooperation — a CPU-heavy or accidentally blocking step stalls every task — and function colouring: everything in the call path must be async-aware, so one synchronous driver or library poisons the design (Async I/O: What `await readFile()` Actually Does).

Choose async when tasks are numerous and mostly waiting: websocket fan-out, proxies, chat, long-polling, an API gateway. Choose threads (or processes) when tasks are CPU-bound, when you depend on blocking libraries, or when you need the kernel’s fairness so one heavy request cannot degrade all the others. The practical answer in most runtimes is the hybrid: one event loop per core for I/O, a bounded thread pool for CPU and blocking calls — the shape of Node.js, Tokio, .NET and nginx.