The question this answers
When we say several tasks are making progress at once, what is each of them actually doing at a given instant?
Four tasks on one core: two API handlers awaiting upstream calls, one JSON serialisation, and one background compaction that periodically yields.
The run queue itself — every task competes for a place in it — plus an in-memory session cache that two of the handlers read and write across their suspension points.
A task that has been made ready eventually runs (no starvation), and a task that reads shared state before a suspension point and writes it afterwards must still be writing a value derived from state nobody changed in between.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Four states, one of which is progress
Running means a core is executing this task's instructions right now. On an eight-core box, at most eight tasks in the entire system are in this state. Everything else is in one of the other three, and confusing "in progress" with "running" is the source of most bad reasoning about concurrent systems.
Ready means the task could run immediately and no core has picked it up. This is the invisible state: no syscall is outstanding, no lock is held against it, nothing is wrong — it is simply not its turn. Time spent ready is pure queueing delay, it appears in no application timer, and on a saturated box it is the majority of a request's latency.
Waiting means the task has asked for something and will not be made ready until it arrives — an upstream response, a timer, a disk read. Blocked, as this lesson uses it, means the task is waiting specifically on another task to release something: a mutex, a semaphore permit, a pool slot. The distinction is practical rather than formal: waiting is usually somebody else's latency, blocked is usually your own contention, and only one of those you can fix. Operating Systems has the full state machine in [[process-states]].
The timeline below is one core and four tasks. Look at the ready segments in particular: task B is ready for four ticks while nothing is wrong with task B at all.
The transitions, and who causes each one
Every transition has an owner, and knowing the owner tells you who can fix a problem. Ready → running is the scheduler's decision, and if a task sits ready too long the answer is fewer runnable tasks or more cores, never anything in your code. Running → waiting is your code's decision — you called something that waits. Waiting → ready is an external event you do not control. Running → ready is preemption, which is the scheduler taking the core back, and in a cooperative runtime it does not exist at all.
That last point is the one with teeth. In a cooperative runtime — coroutines, an event loop, a generator-based scheduler — there is no running → ready edge except the one you write. A task that never awaits and never yields holds the execution context until it returns, and every other task stays ready indefinitely. That is not a bug in the runtime; it is the contract. See [[coroutines]] and [[blocking-the-event-loop]].
The other consequence is where interleaving can happen. Under preemption, the transition can occur between any two instructions, so any non-atomic operation on shared state is exposed. Under cooperation, it can only occur at a yield point you wrote — which sounds safer and is, right up until someone adds an await inside a region that was implicitly atomic because it had none.
Every suspension point is an interleaving point
The reason these four states are worth this much attention is that the boundary between running and not-running is exactly where other tasks get to act. Code between two suspension points is atomic with respect to other tasks on the same execution context. Code that spans a suspension point is not — and it looks identical.
The schedule below is on a single-threaded event loop, where there are no data races at all and where a great many engineers therefore believe there are no races. Two handlers read a session's cached permission set, await a database round-trip, and write back a modified copy. Both reads happen before either write. One update is lost, and the lost one granted an admin role.
Nothing here is a data race: there is one thread, and no unsynchronised conflicting memory access under the language's memory model. It is a race condition — logical correctness that depends on timing — and single-threadedness does not touch it. Keeping those two ideas separate is the difference between "we cannot have this bug" and finding it. See [[data-races]], [[reasoning-about-races]] and [[atomicity-illusion]].
| # | Handler 1 — grant "billing" | Handler 2 — grant "admin" | State |
|---|---|---|---|
| 1 | read cache["s7"] → ["read"] | · | cache.s7=read h1=read h2=- |
| 2 | await db.checkQuota() — suspends, loop free | · | cache.s7=read h1=read h2=- |
| 3 | · | read cache["s7"] → ["read"] | cache.s7=read h1=read h2=read |
| 4 | · | await db.checkQuota() — suspends | cache.s7=read h1=read h2=read |
| 5 | quota OK; resume; write cache["s7"] = ["read","billing"] | · | cache.s7=read,billing h1=read,billing h2=read |
| 6 | return 200 — "billing granted" | · | cache.s7=read,billing |
| 7 | · | quota OK; resume; write cache["s7"] = ["read","admin"] | cache.s7=read,admin ✕ The "billing" role that handler 1 successfully granted and reported is gone. H2 wrote a set derived from a snapshot taken before H1's write. |
| 8 | · | return 200 — "admin granted" | cache.s7=read,admin |
[[finding-the-critical-section]] and [[optimistic-concurrency-control]].Key points
- A task is running, ready, waiting or blocked. Only running consumes a core; only running is execution.
- "Concurrent" means several tasks are unfinished, not that several are running.
- Time spent ready is queueing delay and appears in no application timer — it is the hidden half of tail latency.
- Waiting is usually someone else's latency; blocked is usually your own contention. Only the second is yours to fix.
- In cooperative runtimes there is no preemption edge: a task that never yields holds the execution context until it returns.
- Every suspension point is an interleaving point. Code between two of them is atomic; code spanning one is not, and looks the same.
- Single-threaded does not mean race-free. It means data-race-free. Race conditions survive perfectly well on one thread.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • A spawned task enters the run queue as ready.
- • The scheduler selects a ready task and makes it running on a core; on N cores, at most N tasks are running system-wide.
- • A running task leaves the core by returning, by waiting on something external, by blocking on a held resource, or by being preempted.
- • A waiting task is made ready by the external event — a completed I/O, an expired timer, a delivered message. Readiness is not resumption.
- • A blocked task is made ready when the holder releases the lock, permit or slot; whether it is next depends on the fairness policy. See
[[fairness]]. - • The task resumes with its own saved state, at the instruction after the suspension point, having missed everything that happened in between.
- • A awaits at t=1 and its response arrives at t=6, but C is running until t=7 — A is ready for a full tick, and that tick is latency attributed to the upstream call in every trace.
- • C runs for 20 ms without yielding; B, which became ready at t=0, does not run until t=5. Nothing is wrong with B, and B's p99 is now C's duration.
- • H1 reads the cache, suspends, H2 reads the same value, both write — the lost update above, on one thread, with no lock and no parallelism.
- • B is blocked on the cache mutex from t=6 to t=9 while the holder is itself waiting on I/O: a lock held across a suspension point converts one task's latency into every waiter's latency. See
[[lock-scope]]. - • D yields voluntarily between chunks, so it is ready rather than running at t=7 — cooperative multitasking working correctly, and it works only because someone wrote the yield.
- • The scheduler guarantees that a ready task eventually runs, under most policies. It does not guarantee when, or that the wait is bounded, unless the scheduler is a real-time one.
- • A suspension point guarantees the execution context is released. It guarantees nothing about what the shared state looks like when you resume.
- • Running-to-completion between suspension points guarantees atomicity with respect to other tasks *on the same execution context* — and nothing at all with respect to other threads or processes.
- • Waiting → ready guarantees the event arrived. It does not guarantee your task ran, and the gap between them is invisible to your code.
- • Cooperative scheduling guarantees no preemption, which guarantees implicit atomicity that a future refactor can silently remove.
- • The run queue is the first contention point: many ready tasks and few cores means every task's ready time grows, and it grows for reasons entirely outside that task.
- • The cache mutex is the second: B is blocked while the holder waits on I/O, which multiplies one task's latency across every waiter.
- • A long-running non-yielding task is contention without a lock — it holds the scarcest resource in the system and no wait-for graph shows it.
- • On a preemptive scheduler, contention also costs context switches, and at high switch rates the cost is cache pollution rather than the switch itself. See
[[context-switching-cost]].
- • Starvation: a task stays ready indefinitely because higher-priority or longer-running tasks keep taking the core. See
[[starvation]]. - • Race condition across a suspension point: read-modify-write spanning an
await, producing a lost update on a single thread. - • Convoy: one long-running task makes every other task ready simultaneously when it finishes, producing a burst that looks like a traffic spike. See
[[lock-convoy]]. - • Invisible latency: time spent ready is attributed to whatever span happened to be open, usually the upstream call, sending the investigation to the wrong team.
- • Cooperative stall: a task with no yield point monopolises the runtime, and the symptom appears everywhere except in that task's own metrics.
- • When reading a trace where the sum of the child spans is much less than the parent span — that gap is ready time, and this model is what names it.
- • When deciding where a lock may be held: the rule "never hold a lock across a suspension point" comes directly from these states.
- • When reasoning about a single-threaded runtime, where the four states explain why races exist despite the absence of parallelism.
- • As a substitute for measurement. The state model tells you what to look for; only loop lag, run-queue delay or a scheduler trace tells you what happened.
- • When "ready" and "waiting" are conflated in dashboards, which makes contention indistinguishable from upstream latency and misdirects every investigation.
- • When applied too literally to runtimes that subdivide these states further — many have distinct states for uninterruptible I/O, parked, and pinned to a carrier thread.
- • Event-loop lag: schedule a zero-delay timer and measure how late it fires. That number is the maximum ready-time any task on the loop is currently experiencing.
- • Scheduler run-queue delay — available from OS scheduler statistics — which is ready time for OS threads directly.
- • Span gaps in a distributed trace: parent duration minus the sum of child durations approximates time spent ready or blocked with nothing recorded.
- • Lock wait time as a metric distinct from lock hold time, which separates blocked from running. See
[[lock-wait-metrics]]. - • Thread or task dumps taken during a stall: the ratio of RUNNABLE to WAITING to BLOCKED frames tells you which state dominates. See
[[thread-dumps]].
- • You now need four categories in your head instead of "fast" and "slow", and dashboards that do not distinguish them will actively mislead.
- • Reasoning about correctness requires identifying every suspension point in a function, including ones inside helpers you did not write.
- • Cooperative runtimes make atomicity implicit, which means it is invisible in code review: adding an
awaitto an existing function can break a caller three layers up. - • Latency attribution needs instrumentation designed for it — a span that only covers "waiting" hides the ready time on either side.
- • Fewer concurrent tasks. If ready time dominates, reducing admitted concurrency reduces latency, which is counter-intuitive and frequently correct. See
[[bounding-concurrency]]. - • Dedicated execution contexts: putting latency-sensitive work on its own loop or pool means it never queues behind an unrelated long task.
- • Chunk long-running work so it yields, converting one 20 ms running segment into many short ones and shrinking everyone else's ready time.
- • Move the long task off the shared context entirely — a worker thread or a separate process — which is the only fix that fully removes the interference.
Scheduler timeline
Two increments, twenty schedules: find the one that loses an update
| # | Task A — counter++ | Task B — counter++ | State |
|---|---|---|---|
| 1 | rA ← counter | · | counter=0 rA=0 rB=0 |
| 2 | rA ← rA + 1 | · | counter=0 rA=1 rB=0 |
| 3 | counter ← rA | · | counter=1 rA=1 rB=0 |
| 4 | · | rB ← counter | counter=1 rA=1 rB=1 |
| 5 | · | rB ← rB + 1 | counter=1 rA=1 rB=2 |
| 6 | · | counter ← rB | counter=2 rA=1 rB=2 |
The lost update, step by step
| # | Task A — counter++ | Task B — counter++ | State |
|---|---|---|---|
| 1 | rA ← counter | · | counter=0 rA=0 rB=— |
| 2 | · | rB ← counter | counter=0 rA=0 rB=0 |
| 3 | rA ← rA + 1 | · | counter=0 rA=1 rB=0 |
| 4 | counter ← rA | · | counter=1 rA=1 rB=0 |
| 5 | · | rB ← rB + 1 | counter=1 rA=1 rB=1 |
| 6 | · | counter ← rB | counter=1 rA=1 rB=1 ✕ 2 increments completed, counter = 1 |
What people believe, and what is true
If a task is in progress it is running.
On an eight-core box at most eight tasks in the entire system are running. Ten thousand can be in progress, and most of them are ready or waiting.
The response arrived, so my handler resumed.
The response made the task ready. Running requires a free core and the scheduler's agreement, and the interval between them is real latency that no application timer sees.
Single-threaded code cannot have race conditions.
It cannot have data races. Any read-modify-write spanning an await is a race condition, and the lost-update schedule above needs exactly one thread.
Go deeper
Overview
Running, ready, waiting, blocked. Only running uses a core. Concurrency means many tasks unfinished, not many tasks executing.
Practical
When latency is unexplained, ask which state the time was spent in. Ready means queueing — reduce concurrency or add cores. Blocked means contention — shrink the critical section. Waiting means somebody else.
Advanced
The running-to-not-running boundary is where other tasks act, which makes suspension points the unit of atomicity in a cooperative runtime. That atomicity is implicit and unenforced, so adding an await inside a previously-atomic region is a correctness change that no type system and no review checklist will flag.