Atomics & Lock-Free

Atomics Are Not Magic

Two atomic variables give you two indivisible operations, not one. Every invariant that relates them lives in the gap between the two operations, and no amount of making each one more atomic closes that gap.

▶ Run the lab

The question this answers

The question

Why do two individually correct atomic operations still break the invariant that spans them?

The work

Transferring 50 from account A to account B, and a reporting thread that reads both balances to check the books still balance.

What is shared

Two atomic balances, a and b — and, implicitly, the relationship between them that no variable holds.

The invariant — what must stay true under every interleaving

a + b is constant across every transfer, and neither balance ever goes negative.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

WorkCan it overlap?Can it parallelise?What is shared?What ordering?What synchronization?Where is contention?What can deadlock?What can race?What is gained?What complexity?

Two atomics, one dead invariant

Make both balances atomic. Every read is indivisible, every write is indivisible, no value is ever torn, and a race detector will report nothing. The books still fail to balance, because the invariant was never about a balance — it was about the *sum*, and no location holds the sum.

A transfer is two operations. Between them, the system is in a state that violates the invariant by construction: the money has left A and has not arrived at B. That window is not a bug in the atomics; it is inherent to representing one logical change as two physical ones. Making each half indivisible narrows the window to nanoseconds and leaves it wide open.

The second failure in the same code is check-then-act. if (a >= 50) a -= 50 is an atomic read followed by an atomic write, with a fully preemptible gap between them where another thread can also read, also decide yes, and also subtract. Both threads observed a legal state and both transitions were individually legal; the composition is not. This is the shape Reasoning About Races: A Method, Not an Instinct teaches you to enumerate.

Both balances atomic. Both operations correct. The invariant dies in the gap between them.SIMULATED
Invariant · a + b == 300 at every observable instant, and neither balance is negative
#Transfer threadReporting threadState
1atomic fetch_sub(a, 50) -> 200·a=150 b=150 a+b=300
2·atomic load a -> 150a=150 b=150 T2.a=150
3·atomic load b -> 150a=150 b=150 T2.a=150 T2.b=150
4·report total = 150 + 150 = 300T2.total=300
5atomic fetch_add(b, 50) -> 150·a=150 b=200 a+b=350
✕ The sum is now 350. T1's two atomic operations are correct individually; the pair briefly created — and now permanently reports — money that does not exist relative to the reader's snapshot.
6·[alternative schedule] load a -> 150, load b -> 150 after fetch_sub but before fetch_addT2.total=250
✕ The other ordering: the reporting thread sees 250 and the books appear to have lost 50. No thread did anything wrong.
The invariant a + b == 300 is a property of a *pair* of locations. Atomics protect locations. Nothing about making each location indivisible gives you an indivisible pair, which is why this needs a lock, a queue to a single owner, or a transaction.

Drawing the atomicity boundary where the invariant is

The fix is not a better atomic. It is to make the *whole transition* the unit that other threads cannot observe half-done. Concretely: take one lock covering both balances, do both updates, release. The critical section is chosen by the invariant, not by the variables — which is the argument in Finding the Critical Section.

The reporting thread has to participate too. A reader that does not take the lock still sees the intermediate state, so a "safe" writer plus an unsynchronized reader is still a broken system. If the reader only needs an approximate total, say so explicitly and document the tolerance; if it needs a consistent snapshot, it needs the lock or a snapshot mechanism.

There is a narrow case where one atomic genuinely does cover a two-field invariant: pack both fields into one word and CAS the word. A 32-bit count plus a 32-bit version in one 64-bit atomic is the standard trick, and it works exactly as far as the width the platform can CAS. Past that width you are back to a lock, and the packing itself is a maintenance cost — see The ABA Problem: The Value Came Back for what versions are usually for.

Two atomics. Race-detector clean, invariant broken.
1std::atomic<int64_t> a{200}, b{100};
2
3bool transfer(int64_t amount) {
4 if (a.load() < amount) return false; // check
5 a.fetch_sub(amount); // ... and act, in a different instant
6 b.fetch_add(amount); // ... and the pair is not atomic either
7 return true;
8}
9
10int64_t total() { return a.load() + b.load(); } // may observe the gap
One lock chosen by the invariant, taken by writers and by readers.
1std::mutex m;
2int64_t a = 200, b = 100; // plain ints; the mutex provides both
3 // mutual exclusion and the happens-before edge
4
5bool transfer(int64_t amount) {
6 std::lock_guard<std::mutex> g(m);
7 if (a < amount) return false; // check and act are now one indivisible step
8 a -= amount;
9 b += amount;
10 return true;
11}
12
13int64_t total() {
14 std::lock_guard<std::mutex> g(m); // readers must participate
15 return a + b;
16}

The mutex is not protecting a and it is not protecting b. It is protecting the statement "a + b is constant and neither is negative", which is why both the check and both writes are inside it and why the reader takes it too. Note also that the plain int64_t fields are correct here precisely because the mutex supplies the happens-before edge — see Happens-Before: The Edge That Makes a Write Visible.

The patterns that look atomic and are not

Almost every real instance of this bug is one of a handful of shapes. Recognising them by sight is more useful than any general rule, because the general rule — "the atomic unit must be the invariant, not the variable" — is easy to agree with and hard to apply under deadline pressure.

The tell is grammatical: if the specification of the operation contains the words "and", "if", "then", or "unless", it spans more than one instant, and a single atomic cannot express it. "Increment the counter" is one instant. "If the counter is below the limit, increment it" is two.

Note where this reasoning already exists elsewhere in your stack. A database gives you a transaction precisely so a multi-row invariant has an atomic unit — see Isolation Levels and The Database Solves Concurrency For Its Data, Not For Your Memory. An HTTP API gives you If-Match for the same reason. Application memory is the one layer where nobody hands you transactions, so you draw the boundary yourself.

What the code doesWhat the atomic givesWhat is still brokenWhat it actually needs
counter.fetch_add(1)The whole increment, indivisiblyNothing — this is the case atomics are forNothing
if (n < limit) n.fetch_add(1)An indivisible read and an indivisible writeBoth threads read n == limit-1 and both increment; the limit is exceededCAS loop on n, or a semaphore — see Semaphores: Counting Permits as a Resource Limit
a.fetch_sub(x); b.fetch_add(x)Two correct single-account updatesAny reader between them sees a sum that never legally existedOne lock over both, or a single owning thread
if (!p.load()) p.store(new T())An indivisible pointer read and writeTwo threads both see null and both construct; one object leaks or is used by nobodycall_once / a function-local static — see Double-Checked Locking: The Canonical Cautionary Tale
ratio = hits.load() / total.load()Two indivisible readsThe two reads are from different instants; the ratio can exceed 1A lock, or accept and document an approximate value
if (q.size() > 0) q.pop()A consistent size and a correct popAnother consumer pops in between; this pop hits an empty queueA try_pop that fuses the test and the removal
Common shapes, what the atomic actually gives you, and what is still broken.

Key points

  • Atomics protect locations. Invariants relate locations. When the invariant spans two, the atomics are individually correct and the system is wrong.
  • Check-then-act is two instants no matter how atomic each instant is: both threads can observe a legal state and both transitions can be legal while the composition is not.
  • The atomicity boundary is chosen by the invariant, not by the variables — which usually means a lock over the whole transition.
  • Readers must participate in whatever the writers use, or they observe the intermediate state that the writers were careful to make brief.
  • Grammatical tell: if the operation's specification contains "and", "if" or "then", it needs more than one atomic.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • Write down the invariant as a sentence about state — "a + b is constant", "in-flight never exceeds the limit".
  • List every location the sentence mentions. If it is more than one, no single atomic can enforce it.
  • Identify the transition that takes the system from one state satisfying the invariant to another, and make that whole transition the unit no other thread can observe half-done.
  • Give every reader of the invariant the same mechanism, because an unsynchronized reader observes exactly the window the writers eliminated for each other.
  • Only if the invariant fits in one CAS-able word — count plus version, head plus tag — can a single atomic replace the lock.
Interleavings that matter
  • T1 fetch_sub(a, 50); T2 loads a and b; T1 fetch_add(b, 50) — T2 reports 250 for a system whose total is 300. Every operation was atomic.
  • T1 checks a >= 50 (true); T2 checks a >= 50 (true); T1 fetch_sub(a, 50); T2 fetch_sub(a, 50) — a is now negative from a balance of 50. Check-then-act with atomic reads and atomic writes.
  • T1 loads hits (=90); T2 records a hit and a request (hits=91, total=100); T1 loads total (=100) — T1 computes 0.90 from state that never coexisted. Harmless here; the same shape decides whether to shed load elsewhere.
  • With one mutex over both: T1 acquires, subtracts, adds, releases; T2 acquires, reads 150 and 150, releases. Every observable state satisfies the invariant. No schedule breaks it.
What it guarantees — and does not
  • Promises: each atomic operation is indivisible, and no individual value is ever torn or lost.
  • Promises: with a lock covering the whole transition, no thread that also takes the lock can observe an intermediate state.
  • Does NOT promise: that a sequence of atomic operations is itself atomic. There is no composition rule that makes it so.
  • Does NOT promise: that "no data race" means "no race condition". A race detector checks unsynchronized memory access; it cannot know your invariant. See Data Race Is Not Race Condition.
  • Does NOT promise: that narrowing the window helps. A window of ten nanoseconds hit a thousand times a second is hit every day in production.
Where contention appears
  • The correct fix — one lock over the transition — serialises the whole transition rather than each variable, so the contended region is larger than the two atomics were. That is the price, and it is the right price.
  • Readers taking the lock turn a read-mostly workload into a contended one; a read-write lock or a snapshot pointer is the usual next step. See Read/Write Locks, Honestly and Copy-on-Write as a Concurrency Strategy.
  • Packing two fields into one CAS-able word keeps the contention at a single line but concentrates it there — every reader and every writer now fight for one line. See What a Shared Write Costs.
How it fails
  • Lost update — check-then-act where both threads pass the check.
  • Race condition without a data race — every access synchronized, invariant still broken. The most under-diagnosed failure in this module, because every tool reports clean.
  • Torn logical read — a reader assembling a snapshot from several individually atomic reads, producing a state that never existed.
  • Limit overrun — a counter checked against a bound and then incremented, letting N threads past a limit of one.
  • Double initialization — two threads both observing "not yet initialized". See Initialization Races.
When it helps
  • Recognising the pattern early: the cheapest moment to notice a two-location invariant is before the atomics are written, not after the incident.
  • Code review — "which invariant does this atomic enforce?" is a question that finds this bug in seconds and has no polite way to be answered wrongly.
  • Deciding deliberately that an approximate value is acceptable, and writing that down, so the next reader does not "fix" it into a lock.
When it hurts
  • When the reasoning is applied so broadly that every atomic gets a lock "just in case" — a genuine single-location counter does not need one.
  • When the response is a single global lock over unrelated invariants, converting a correctness bug into a contention bug. See Concurrency Anti-Patterns.
  • When the packed-word trick is used past the point of readability to avoid a mutex nobody measured.
How you would know
  • Add an assertion that checks the cross-location invariant from a thread that takes the same lock, and run it under stress. An invariant checked only in tests that never interleave proves nothing. See Stress Testing: A Test That Passed Once Proves Nothing.
  • Reconcile against an independent source: sum the accounts against the ledger, compare in-flight counts against the load balancer. Drift that scales with concurrency is this bug.
  • Note explicitly that a thread sanitizer will not find it. TSan reports unsynchronized access; here every access is synchronized. This is the case where tooling gives a false all-clear.
  • Log the pair of values together under the lock rather than separately, so the log itself cannot show a state that never existed.
Complexity it introduces
  • You accept a real critical section with a real contention cost, in exchange for an invariant you can state and check.
  • Every reader of the invariant now has an obligation, which means the lock becomes part of the type's public contract and cannot be an implementation detail.
  • The alternative — accepting approximation — has its own complexity: someone must own the tolerance, document it, and defend it the next time a number looks wrong.
Simpler alternatives
  • One mutex over the whole transition. The default answer, and correct far more often than lock-free reasoning is. See Mutexes: What They Protect and What They Do Not.
  • A single owning thread reached through a queue: the invariant is enforced by there being exactly one writer. See The Actor Model and Message Passing.
  • An immutable snapshot swapped in with one atomic pointer store — the multi-field state becomes one location by construction. See Immutability as a Concurrency Strategy and Copy-on-Write as a Concurrency Strategy.
  • Move the invariant to a layer that already has transactions: a database row with a CHECK constraint, or a conditional write. See Transactions and ACID.
  • Redefine the requirement. "Approximately balanced, reconciled hourly" is a legitimate specification and is much cheaper than the alternatives — but it must be a decision, not an accident.

Two increments, twenty schedules: find the one that loses an update

Two increments, twenty schedules
Both tasks run counter++ on the same variable. Drive the schedule yourself: read, add, write are three separate steps, and the scheduler may cut between any two of them.
6/6 steps
counter
2
increments completed
2
rA / rB
1 / 2
invariant
holds
Invariant · after k completed increments, counter === k. No update is lost.
#Task A — counter++Task B — counter++State
1rA ← counter·counter=0 rA=0 rB=0
2rA ← rA + 1·counter=0 rA=1 rB=0
3counter ← rA·counter=1 rA=1 rB=0
4·rB ← countercounter=1 rA=1 rB=1
5·rB ← rB + 1counter=1 rA=1 rB=2
6·counter ← rBcounter=2 rA=1 rB=2
counter = 2, and both callers are right. This schedule happens to be safe because one task finished entirely before the other started. Safe once is not safe: press "Enumerate all" to see how many of the possible schedules do not. Testing samples this space; it does not cover it.
SIMPLIFIEDcounter++ modelled as three indivisible steps. Real compilers and CPUs can split it further, or fuse it into one atomic instruction.

if (balance >= 100) withdraw(100) — drive it until it overdraws

if (balance >= 100) withdraw(100)
Two withdrawals of 100 from an account holding 100. The check and the debit are separate operations; you decide who runs when.
balance = 100

withdraw(amount):        # both tasks run this concurrently
    b = read(balance)    # 1
    if b >= amount:      # 2  <- decided on a value that may already be stale
        debit(amount)    # 3
0 schedules tried
balance
0
paid out
100
A decided
withdraw
B decided
Invariant · balance >= 0 — the account is never overdrawn.
#Withdrawal A (100)Withdrawal B (100)State
1rA ← read balance·balance=100 paidOut=0
2if rA >= 100·balance=100 paidOut=0
3debit 100·balance=0 paidOut=100
Balance is 0 and nothing has broken yet. Watch for the shape: both tasks passing step 2 before either reaches step 3. That is check-then-act, and the check is only as good as the instant it was made.
SIMPLIFIEDThe debit itself is modelled as atomic. The bug is the gap between the check and the act — not the arithmetic.

The lost update, step by step

The lost update, step by step
One fixed schedule of two concurrent increments. Nothing to choose — watch where the invariant dies, and where the cause actually was.
1/6 · A · rA ← counter
Invariant · after k completed increments, counter === k. No update is lost.
#Task A — counter++Task B — counter++State
1rA ← counter·counter=0 rA=0 rB=—
2·rB ← countercounter=0 rA=0 rB=0
3rA ← rA + 1·counter=0 rA=1 rB=0
4counter ← rA·counter=1 rA=1 rB=0
5·rB ← rB + 1counter=1 rA=1 rB=1
6·counter ← rBcounter=1 rA=1 rB=1
✕ 2 increments completed, counter = 1
step
1 of 6
counter
0
increments completed
0
invariant
holds
A reads 0. Correct at this instant, and about to stop being correct. A read-modify-write is a window, not an instant. It stays open from the read to the write.
SIMPLIFIEDOne of twenty possible interleavings of this program, chosen because it fails.

What people believe, and what is true

Claim

I made every shared variable atomic, so the code is thread-safe.

Reality

Thread-safety is a property of operations against an invariant, not of variables. Every variable atomic and every invariant broken is a completely ordinary state for a program to be in.

Claim

The thread sanitizer found no races, so there is no race.

Reality

It found no *data* races. A race condition over correctly synchronized accesses is invisible to it, because it does not know what must stay true.

Claim

The window is only a few nanoseconds, so it will not happen.

Reality

At ten thousand transfers a second, a few nanoseconds is hit thousands of times a day. Narrow windows produce rare bugs, and rare bugs at scale are daily bugs.

Go deeper

Overview

An atomic makes one variable safe. If what must stay true involves two variables, you need something bigger — usually a lock.

Practical

Say the invariant out loud before choosing a primitive. Count the locations in the sentence. One location means an atomic may be enough; more than one means it is not, and no combination of atomics will change that.

Advanced

The only escape is to make the multi-field state a single location: pack it into one CAS-able word, or make it an immutable object published by one atomic pointer store. Both convert a composition problem into a publication problem, which is Safe Publication: Handing Over a Finished Object.

Internals

There is no hardware primitive that atomically updates two arbitrary addresses. Double-width CAS updates two *adjacent* words, which is why version-tagged pointers work and why "atomically update these two unrelated counters" does not. Hardware transactional memory attempted the general case and is not something to build on today.

Apply it