What a Mutex Actually Does
A mutex is not an operating-system object you call into. In the uncontended case it is one atomic instruction and no system call at all — which is why an uncontended lock is nearly free and a contended one costs thousands of times more.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
Three layers, and most calls stop at the first
A modern mutex is layered. The top layer is a word in ordinary user-space memory holding the lock state. Acquiring attempts an atomic compare-and-swap or exchange from *unlocked* to *locked*. If that succeeds, the acquisition is complete: no system call, no scheduler involvement, nothing but one atomic instruction and whatever ordering it carries.
If the CAS fails, the lock is held. Most implementations then spin briefly — a bounded number of attempts, sometimes adaptive based on whether the holder appears to be running — because a lock held for a few hundred cycles is cheaper to wait out than to sleep on. Only if spinning fails does the third layer engage: a system call that parks the thread on a wait queue until the lock is released. On Linux this is futex, on Windows WaitOnAddress, on macOS the ulock family.
Unlocking mirrors it. The fast path is a store releasing the lock word, with release ordering so the critical section's writes are visible before the lock appears free. Only if the state records waiting threads does the unlock make a system call to wake one. This is the design that makes the common case cheap: the kernel learns about the lock only when there is genuine contention to arbitrate.
The cliff between uncontended and contended
The cost difference is not a gradient but a cliff, and it falls in two places. The first is contention on the cache line: even when the CAS succeeds, if another core recently held that line the atomic must take ownership back, which is a coherence transaction rather than a local operation. The second and much larger step is parking: a system call, a context switch out, and later a wake and a context switch back in — plus the cache the thread lost while it was off the core, which Cache Warmth and the Real Cost of Migration covers.
This shape has a clear design implication. Optimising an uncontended lock is almost never worth it, because it is already a single atomic operation, and replacing it with a hand-rolled atomic protocol trades a well-tested primitive for one atomic operation plus your own bugs. Optimising a *contended* lock is frequently worth a great deal, because you are paying for context switches and lost cache, not for the lock instruction.
The fixes that work therefore all reduce contention rather than the cost of locking. Shorten the critical section so the window for collision narrows. Shard the lock so different threads touch different locks and different cache lines. Use per-thread state and aggregate. Replace the shared counter with fetch_add if that is all it was. Each removes contention; none makes the lock instruction faster, because the lock instruction was never the problem.
What this means for choosing a primitive
The first consequence is that "avoid locks in hot paths" is bad advice as usually stated. If the lock is uncontended, it costs one atomic operation, and replacing it with a hand-rolled atomic protocol buys you approximately nothing while costing correctness risk — the alternatives in Atomic Instructions: What the Hardware Actually Guarantees and Compare-and-Swap: The Primitive Everything Is Built On have their own contention behaviour and their own hazards.
The second is that lock granularity is the lever that actually matters. A coarse lock held across a long critical section produces the parking behaviour that costs thousands of times the fast path. The same work under several finer locks, or under a lock held for a much shorter window, may never leave the fast path at all. That is a structural change to the code rather than a change of primitive.
The third is that this connects directly to the coherence material. A mutex is a cache line that every contending thread writes to. Threads spinning on it are generating exactly the ownership ping-pong described in Cache Coherence: Why Shared Memory Works At All, and two unrelated mutexes sharing a cache line produce False Sharing: Independent Data, Shared Line between critical sections that have nothing to do with each other. Padding locks onto separate lines is a routine and effective fix that has nothing to do with locking semantics at all.
| Situation | Reasonable choice | Why |
|---|---|---|
| Uncontended, short critical section | Mutex | One atomic operation; hand-rolling saves nothing and risks correctness |
| Unconditional counter update | fetch_add | One transaction, no loop, no critical section at all |
| Conditional update, tiny critical section, measured contention | CAS loop | Avoids parking, but pays retries and needs an ABA answer for pointers |
| High contention, long critical section | Restructure: shard, shorten, or use per-thread state | The cost is parking and lost cache, so reduce contention rather than change primitive |
| Read-heavy shared state | Reader-writer lock or a copy-based scheme | Lets readers proceed together instead of serialising on one line |
| Two unrelated locks in one struct | Pad onto separate cache lines | Otherwise their critical sections contend through False Sharing: Independent Data, Shared Line |
Key points
- An uncontended mutex acquisition is a single atomic operation in user space, with no system call and no scheduler involvement.
- The kernel is engaged only when the lock is held and a thread must be parked — the expensive path is contention, not locking.
- The cost gap between the fast path and a park-and-wake is roughly three orders of magnitude, plus the cache the thread loses.
- Fixes that work reduce contention: shorter critical sections, sharding, per-thread state — not a different locking primitive.
- A mutex is a shared cache line, so lock contention is coherence traffic and unrelated locks sharing a line contend through false sharing.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1lock() → atomic CAS: attempts to swap the lock word from unlocked to locked entirely in user space.
- 2CAS success → caller: the lock is held; no system call, no scheduler, just the atomic and its ordering.
- 3CAS failure → bounded spin: the thread retries briefly, betting the holder will release sooner than a context switch costs.
- 4Spin exhausted → kernel: a wait syscall parks the thread on a queue keyed by the lock address and the scheduler runs something else.
- 5unlock() → release store, and a wake syscall only if the lock state records waiters; the woken thread retries the CAS.
- • "Locking is a system call" — the uncontended path never enters the kernel; only parking does.
- • "This profile shows little time in
lock(), so the lock is not the problem" — the cost appears as context switches and cache misses elsewhere. - • "Lock-free will be faster" — an uncontended mutex is already one atomic operation; under contention a retry loop can generate more coherence traffic than parking does.
- • "Making the lock finer-grained always helps" — more locks mean more lock words, more cache lines and more chance of deadlock; past a point it costs more than it saves.
Consequences, controls and cost
- • Profiles show lock cost as cache-coherence traffic and context switches, not as time inside a locking function.
- • A lock that is uncontended in testing and contended in production changes cost by orders of magnitude with no code change.
- • Threads that park lose their cache working set, so the cost of contention exceeds the context switch itself.
- • Two unrelated locks placed adjacently in a struct can serialise unrelated critical sections through false sharing.
- • Shorten critical sections so collisions become unlikely and the fast path stays the common path.
- • Shard the lock — per-bucket, per-partition or per-core — so contending threads touch different locks on different lines.
- • Replace a lock with `fetch_add` or another single atomic where the critical section was only an arithmetic update.
- • Pad independently-used locks onto separate cache lines so their critical sections do not contend accidentally.
- • Measure before replacing a mutex with a lock-free protocol; if the lock is uncontended, there is nothing to win.
- • Count context switches (`perf stat`, `vmstat`) during the workload — a contended lock shows as involuntary switches rather than as time in a lock symbol.
- • Sample cache-coherence events for the lock's cache line to confirm ownership ping-pong.
- • Instrument acquisition to record the fraction taking the fast path versus spinning versus parking; that ratio is the diagnosis.
- • Compare throughput as thread count rises — a contended lock flattens or regresses while an uncontended one scales.
- • Spinning avoids a context switch but burns cycles that another runnable thread could use, and the right spin budget is workload-dependent.
- • Finer-grained locking reduces contention while increasing memory, complexity and deadlock risk.
- • Sharding removes contention at the cost of making any operation that must see all shards more expensive.
Scope
§224 — what these claims are specific to.
- PLATFORM-SPECIFICSpin policy, wait primitive and wake strategy differ per platform and library: Linux futex, Windows WaitOnAddress, macOS ulock. The three-layer structure is common; none of the parameters are.
- MICROARCH-SPECIFICRelative costs are illustrative ratios. The parking cost in particular depends on scheduler behaviour, core count and how much cache the descheduled thread loses.
- SIMPLIFIEDOmits reader-writer locks, priority inheritance, adaptive spin heuristics and fairness policies, all of which real implementations include and all of which change the cost profile.
Misconceptions
Apply it
Where the rest of this lives
This lesson covers what a mutex costs and why. Choosing a locking discipline, avoiding deadlock and reasoning about critical-section design belong to concurrency.
Managed runtimes add their own layers — biased locking, thin-to-fat inflation, goroutine parking — between the language primitive and the atomic instruction described here.