Contention & Oversubscription

Busy Waiting

while (!ready) {} occupies a core at 100% doing nothing, and on an oversubscribed machine it actively prevents the thread that would set ready from running. This is not an argument against all spinning — bounded spinning is a real technique — it is an argument against unbounded spinning where a blocking wait belongs.

The question this answers

The question

What does spinning on a flag cost, and when is waiting without blocking ever the right choice?

The work

A consumer thread waiting for a producer to publish a result, expressed as a loop that re-reads a ready flag until it becomes true.

What is shared

A ready flag and the result it guards. The publication ordering between them is a memory-model question (Safe Publication: Handing Over a Finished Object); this lesson is about the *waiting*, and about what the waiting costs everyone else.

The invariant — what must stay true under every interleaving

When the consumer observes ready == true, the result is fully written and visible to it. That is the correctness invariant, and both spinning and blocking can satisfy it given correct publication. The invariant busy waiting breaks is a resource one: a thread that cannot proceed should not consume a core. Violating it does not corrupt anything — it denies CPU to the very thread whose progress you are waiting for.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

Work→Can it overlap?→Can it parallelise?→What is shared?→What ordering?→What synchronization?→Where is contention?→What can deadlock?→What can race?→What is gained?→What complexity?

The same loop in four languages, four different disasters

The naive spin loop is a good test of whether you understand a runtime, because it fails differently in each one — and in two of the four it does not merely waste CPU, it never terminates at all. Read each entry for the mechanism rather than the syntax.

The C++ case is the subtlest and the most important: reading a non-atomic variable that another thread writes is a data race, which is undefined behaviour under the C++ memory model. The practical consequence is not a crash but an optimisation — the compiler is entitled to hoist the load out of the loop, because within a race-free program nothing could change it, producing if (!ready) for(;;);. The loop then spins forever regardless of what the producer does. Note the precision here: this is a data race, not merely a race condition — the two are different things, and Data Race Is Not Race Condition draws the line.

The JavaScript case fails for an entirely different reason: there is one thread and one event loop, so the callback that would set ready cannot run while the loop is running. Spinning does not merely delay it — it makes it *impossible*. CPython occupies the middle ground: the loop holds the interpreter lock and releases it every switch interval, so the producer does eventually run, extremely slowly, while a core burns.

The same intent — "wait until another task publishes a result" — and four different failure mechanisms. — Wait until another thread or task sets `ready`, then read the result it published.
C++LANGUAGE-SPECIFIC
1bool ready = false; Result r;
2 
3// consumer -- UNDEFINED BEHAVIOUR: a data race on `ready`
4while (!ready) { }        // compiler may hoist the load: if(!ready) for(;;);
5use(r);                   // and `r` has no ordering guarantee either
6 
7// correct, blocking:
8std::mutex m; std::condition_variable cv; bool ready = false;
9std::unique_lock lk(m);
10cv.wait(lk, []{ return ready; });   // predicate loop; thread parks, 0% CPU
11use(r);

Reading a non-atomic bool that another thread writes is a data race and therefore UB — the compiler may cache the load in a register and spin forever. std::atomic<bool> with acquire/release makes it defined and also fixes the visibility of r; a condition variable additionally stops burning the core.

JavaScriptBROWSER
1let ready = false
2fetchResult().then(r => { result = r; ready = true })
3 
4// consumer -- HANGS FOREVER, and freezes the tab
5while (!ready) { }   // the .then callback can never be dequeued
6                     // because this loop never yields to the event loop
7 
8// correct: there is nothing to wait for -- await it
9const result = await fetchResult()

One thread, one event loop. The continuation that sets ready is a queued task, and tasks run only when the call stack empties. Spinning guarantees it never does: this is not slow, it is a permanent hang plus an unresponsive page. See Blocking the Event Loop.

TypeScriptNODE.JS
1// Across worker threads there IS shared memory, and a real blocking wait.
2const sab = new SharedArrayBuffer(4)
3const flag = new Int32Array(sab)
4 
5// consumer, in a worker -- spins, burning a core:
6while (Atomics.load(flag, 0) === 0) { }
7 
8// consumer, blocking -- parks the thread, 0% CPU:
9Atomics.wait(flag, 0, 0)          // returns when the producer notifies
10// producer:
11Atomics.store(flag, 0, 1); Atomics.notify(flag, 0)

Atomics.wait is a genuine blocking wait on a shared buffer and is the correct primitive between worker threads. It is deliberately forbidden on the main browser thread, precisely because blocking there would freeze the UI. See Worker Threads and Web Workers.

PythonCPYTHON
1ready = False
2 
3# consumer -- terminates, but burns a core and slows the producer
4while not ready:
5    pass          # holds the interpreter lock between switch intervals
6                  # (sys.getswitchinterval() default 5 ms in CPython 3.12)
7 
8# correct:
9evt = threading.Event()
10evt.wait()        # thread parks in the OS; ~0% CPU; woken by evt.set()

In CPython 3.12 the spin loop does terminate, because the interpreter releases its lock every switch interval — but it burns a core and gives the producer only a fraction of the interpreter's time, so the wait can be orders of magnitude longer than necessary. threading.Event parks the thread properly.

What actually differs
  • C++: unsynchronised reads of a written variable are a data race and undefined behaviour — the loop may be optimised into an infinite one. Correctness requires std::atomic or a lock, before any performance discussion.
  • Browser JavaScript: single-threaded, so a spin loop makes the awaited callback unreachable. The failure is a permanent hang, not wasted CPU.
  • Node worker threads: real shared memory and a real blocking wait exist via SharedArrayBuffer and Atomics.wait — the only place in JavaScript where "block this thread until notified" is available, and it is banned on the main thread.
  • CPython: the spin terminates but competes with the producer for the interpreter lock, so it converts a microsecond wait into a multi-millisecond one while occupying a core.
  • Across all four the correct primitive parks the waiter and is woken by the producer: condition variable, await, Atomics.wait, threading.Event. See Condition Variables: Waiting Until a Predicate Is True.

Why it is worse than merely wasteful

The intuitive cost of busy waiting is a wasted core, and on a machine with spare cores that is roughly the whole story — annoying, expensive, not fatal. The real damage appears when there is no spare core, which is exactly the situation in which you are most likely to be waiting.

The timeline shows the case. One core, two threads: a spinner waiting for a result and the producer that must compute it. The spinner is runnable, so the scheduler gives it half the CPU, and the producer takes twice as long as it would have if the consumer had simply parked. The consumer's waiting *caused* the delay it was waiting through. On an oversubscribed machine with eight spinners this becomes the dominant effect, and it is the mechanism behind "the system got slower when we added a polling health check".

Two mitigations sit between spinning and blocking and are worth knowing. A yield in the loop body (std::this_thread::yield, sched_yield) tells the scheduler to run someone else, which removes the starvation without removing the CPU burn or the wakeup latency. A spin-then-block hybrid spins for a bounded number of iterations and then parks, which captures the low latency of spinning for short waits and the zero cost of blocking for long ones. That hybrid is what production mutexes actually do, and it is the subject of Spin Locks — this lesson is not an argument against it.

One core, two threads. The spinner is not merely idle — it delays the producer it is waiting for.SIMULATED
Spinning consumer — while (!ready) {}
spinning at 100% CPU
preempted
spinning
preempted
spinning
preempted
spinning
sees ready — proceeds
Producer, sharing that core with the spinner
ready (preempted)
computing
ready
computing
ready
computing
sets ready — done at t=7
Blocking consumer — cv.wait / Event.wait
parked — off the run queue, 0% CPU
woken + scheduled (~3 µs)
proceeds
Producer with a blocking consumer
computing at full speed
notify; done
↑ blocking: result ready↑ spinning: result ready — 75% later
runningreadywaitingblockedidle1 unit ≈ 1 ms

When spinning is right — and it sometimes is

Do not read this lesson as "never spin". Spinning wins in a specific and well-defined regime: when the expected wait is *shorter than the cost of parking and waking a thread*. A park/unpark round trip is a few microseconds; a critical section that lasts 200 nanoseconds is over before the wakeup would have completed. Spinning through it is not merely acceptable, it is the fastest correct thing to do.

The conditions are strict and all of them must hold. The wait must be short and bounded. There must be a free core, or you starve the thread you are waiting for. The spin must be bounded, so a mis-prediction degrades to a park rather than to an infinite loop. And the flag must be an atomic with correct ordering, or the C++ case above applies and the whole discussion is moot.

The compare below is the shape to write when you decide spinning is justified: a bounded spin with a CPU relaxation hint, followed by a fall-back to a real blocking wait. That is what an adaptive mutex does internally, and it is the only spin loop that belongs in application code. Unbounded spinning is what this lesson rejects; bounded spin-then-block is a legitimate technique with its own lesson in Spin Locks.

Unbounded spin: a data race, a burned core, and a starved producer.
1bool ready = false;
2 
3while (!ready) { }          // 1. data race -> UB; load may be hoisted
4                            // 2. 100% of a core while making no progress
5                            // 3. on a busy machine, starves the producer
6                            // 4. unbounded: a 2-second wait spins 2 seconds
7use(result);
Bounded spin, then park. Fast for short waits, free for long ones.
1std::atomic<bool> ready{false};   // no data race; release/acquire orders `result`
2 
3for (int i = 0; i < kSpinLimit; ++i) {        // ~100s of ns of spinning
4  if (ready.load(std::memory_order_acquire)) { use(result); return; }
5  _mm_pause();                  // tell the CPU this is a spin-wait loop:
6}                               // saves power, avoids memory-order violations
7 
8std::unique_lock lk(m);         // long wait: pay the wakeup, free the core
9cv.wait(lk, [&]{ return ready.load(std::memory_order_acquire); });
10use(result);

The bounded version wins the short case (no park/unpark round trip) and the long case (the core is released) — the only cost is a constant to tune and slightly more code. It also fixes the correctness bug the naive version had: std::atomic removes the data race and the acquire load gives you the ordering that makes result safe to read. Note the order of concerns: correctness first, then the waiting strategy.

Key points

  • An unbounded spin loop occupies a core at 100% while making no progress, and on a busy machine it delays the thread it is waiting for.
  • In C++ spinning on a non-atomic flag is a data race and undefined behaviour — the compiler may hoist the load and the loop may never terminate.
  • In browser JavaScript a spin loop is a permanent hang, because the callback that would end it can never be dequeued.
  • Blocking waits park the thread off the run queue at ~0% CPU, at the cost of a few microseconds of wakeup latency.
  • Spinning is correct when the expected wait is shorter than a park/unpark round trip — bounded, with a free core, on an atomic. That is Spin Locks, and it is a different lesson.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • • The waiting thread executes a loop that repeatedly loads a flag; it is runnable throughout, so the scheduler keeps giving it CPU.
  • • Because no blocking syscall is made, the OS has no idea the thread is waiting and cannot give its slice to anyone who could make progress.
  • • On a saturated machine the spinner and the producer share cores, so the producer runs at a fraction of full speed and the wait lengthens.
  • • A blocking wait instead calls into the OS, which removes the thread from the run queue entirely until a notification arrives.
  • • A hybrid spins for a bounded count — with a CPU pause hint to reduce power and pipeline penalties — and then falls back to blocking.
Interleavings that matter
  • • Free core, 200 ns wait: the spinner observes the flag almost immediately and beats a blocking wait, which would have spent microseconds parking and waking.
  • • Saturated core, 4 ms wait: the spinner takes half the core, the producer takes 7 ms instead of 4, and the consumer waits 75% longer than if it had parked.
  • • C++ with a non-atomic flag: the compiler hoists the load, the loop becomes infinite, and the producer's write is never observed — a hang whose cause is in the optimiser, not the schedule.
  • • Browser JavaScript: the spin loop holds the call stack, so the microtask that sets the flag is never dequeued. Not slow — impossible.
  • • CPython 3.12: the spin holds the interpreter lock between switch intervals, so the producer advances in 5 ms increments while a core is fully consumed.
  • • Bounded spin-then-block: the spinner tries for a few hundred nanoseconds, gives up, parks, and is woken 4 ms later. Best case of both, worst case of neither.
What it guarantees — and does not
  • • A blocking wait guarantees the thread consumes no CPU while waiting and is woken when notified. It does not guarantee it is woken *promptly* — see Lost Wakeups: The Notify That Arrived Before the Wait and Spurious Wakeups: Why It Is `while`, Not `if`, and always use a predicate loop.
  • • A spin loop guarantees the lowest possible latency to observe a change, if and only if a core is available for it.
  • • An atomic load guarantees the value is read without a data race and, with acquire ordering, that prior writes by the publisher are visible. A plain load guarantees neither.
  • • Atomics.wait guarantees a real blocking wait between JavaScript worker threads, and is guaranteed *unavailable* on the main browser thread.
  • • Nothing about spinning guarantees termination if the thing you are waiting for cannot run — which is exactly the browser case and the oversubscribed case.
Where contention appears
  • • A spinner contends for a core with every runnable thread, including the one whose progress ends the wait — the most direct form of self-inflicted contention there is.
  • • Multiple spinners on the same flag contend for its cache line: every load pulls it, and the publishing write must invalidate all of them. This is why a naive spin lock degrades sharply with waiter count. See What a Shared Write Costs.
  • • On hyperthreaded cores a spinner steals execution resources from its sibling thread, which is why the pause instruction exists — it yields pipeline resources to the sibling.
  • • Polling a remote resource is the distributed version and contends downstream instead: N clients polling every 100 ms is a load floor that exists whether anything is happening or not. See Thundering Herd.
How it fails
  • • CPU pegged at 100% with no work being done — the signature, and it looks identical to a hot loop in a profiler until you read the frames.
  • • Infinite loop from a data race in C++, where the compiler caches the flag in a register.
  • • Permanent hang in single-threaded event-loop runtimes, where the loop prevents the setter from ever running.
  • • Producer starvation on a saturated machine, where the spinning lengthens the wait it is spinning through.
  • • Battery and thermal cost on client devices, plus a cloud bill for cores doing nothing — the least dramatic and most common consequence.
  • • A "temporary" poll loop with a sleep(100ms) inside that becomes permanent, adding a fixed 50 ms average latency to an operation that could have been event-driven.
When it helps
  • • When the expected wait is shorter than a park/unpark round trip — sub-microsecond critical sections, where spinning is genuinely the fastest correct option.
  • • When a core is dedicated to the waiter, as in a pinned real-time or high-frequency-trading thread where latency dominates every other consideration.
  • • As the first phase of an adaptive lock, where a bounded spin captures the common short case and a park handles the rest. This is what production mutexes do.
  • • In kernel or interrupt contexts where blocking is not permitted at all and a spinlock is the only available primitive.
When it hurts
  • • Whenever the wait may be long or unbounded — waiting for I/O, for a network response, for a user, or for a job whose duration you do not control.
  • • On any machine where runnable threads already meet or exceed core count, because the spinner is taking CPU from the thread it is waiting for.
  • • In single-threaded event-loop runtimes, where it is not a performance problem but a correctness one.
  • • On non-atomic variables in languages with a memory model that makes that undefined, which is a correctness bug before it is a performance one.
How you would know
  • • A CPU profile whose hottest frames are a loop with no side effects — flame-graphs will show a wide, flat plateau on the waiting function.
  • • 100% CPU on a thread with a flat or zero business-work counter. Utilisation without completions is the same signature as Livelock, and the two are close relatives.
  • • Voluntary context switches near zero for a thread that is logically waiting: a properly blocked waiter shows voluntary switches, a spinner shows none. pidstat -w splits them.
  • • For polling variants, request rate to the polled resource that is independent of actual event rate — a flat floor of traffic when nothing is happening.
  • • Latency of the operation being waited on, measured with and without the spinner running. If it is faster when the waiter blocks, the spinner was starving the producer.
Complexity it introduces
  • • A correct blocking wait requires a predicate loop and correct handling of spurious wakeups and lost wakeups — genuinely more code than while (!ready) {}, and that is why the bad version keeps getting written.
  • • A bounded spin adds a spin-count constant that is hardware- and workload-dependent and cannot be derived, only measured.
  • • Correctness comes first: making the flag atomic with the right ordering is a memory-model question that must be settled before any spinning decision. See What a Memory Model Defines and Happens-Before: The Edge That Makes a Write Visible.
  • • Adaptive strategies are best left to the runtime. Hand-written spin-then-block in application code is a maintenance burden that a standard mutex already carries for you.
Simpler alternatives
  • • A condition variable with a predicate loop — the standard answer for "wait until a condition holds" in threaded code. See Condition Variables: Waiting Until a Predicate Is True.
  • • An event, latch or barrier for one-shot signals, which is simpler than a condition variable when the condition never becomes false again. See Latches & Countdowns and Barriers.
  • • await on a promise or future in async runtimes, which suspends the task and frees the thread for other work. See Await Is a Yield Point and Futures & Promises.
  • • A blocking queue receive, which combines the wait and the data transfer and eliminates the flag entirely. See Producer / Consumer and Channels.
  • • For remote resources, replace polling with a push: webhooks, server-sent events or websockets remove the traffic floor entirely. See polling-vs-sse-vs-websockets in the systems domain.

What people believe, and what is true

Claim

Spinning is faster because it avoids a context switch.

Reality

Only when the wait is shorter than the round trip it avoids — a few microseconds. For anything longer, spinning costs a full core for the whole wait and, on a busy machine, lengthens the wait itself.

Claim

Busy waiting is just wasteful, not wrong.

Reality

In C++ on a non-atomic flag it is undefined behaviour and may never terminate. In browser JavaScript it makes the awaited event unreachable. In both cases it is a correctness bug.

Claim

Adding a small sleep to the loop fixes it.

Reality

It converts a CPU problem into a latency problem: a 100 ms poll adds 50 ms average delay to an event that had already happened. It is better than spinning and much worse than being notified.

Go deeper

Overview

while (!ready) {} uses a whole core to check a variable. If the machine is busy, that core is one the producer needed, so you have made your own wait longer. Park the thread and let something wake you.

Practical

Use the blocking primitive your runtime provides: condition variable with a predicate loop, Event, await, or a blocking queue receive. If you have measured that the wait is sub-microsecond and a core is free, a *bounded* spin with an atomic and a pause hint, falling back to a park, is legitimate.

Advanced

The decision is a straight comparison: expected wait time against park/unpark round-trip cost, with a hard side condition that a core must be available. Both terms are measurable, which makes this one of the few concurrency trade-offs with a clean decision rule — and the reason adaptive mutexes implement exactly that rule so you do not have to.

Internals

Modern mutexes are adaptive: they spin briefly (sometimes only if the current holder is observed to be running on another core, since spinning for a descheduled holder is pure waste) and then fall back to a futex-style park. That single heuristic explains most of the performance difference between a naive spinlock and a production mutex, and it is why hand-rolling either one is rarely worth it. See Spin Locks and Mutexes: What They Protect and What They Do Not.

Apply it

🔍 Every Endpoint Got Slower At Once🔍 A Worker Asleep on a Full Queue⚡ Lose the Wakeup💬 When Do You Reach for a Mutex?💬 What Stalls an Event Loop?💬 Race Condition or Data Race?
← More Threads Is Not More Speed