Coordination

The Stale Lock Holder: A Paused Process Does Not Know It Was Paused

The canonical distributed-systems accident: A takes a lock, A pauses, the lease expires, B takes the lock, A resumes believing nothing happened, and both act. Nobody is at fault, no error is logged, and the resolution is not a longer timeout.

▶ Run the lab

The question this answers

The question

A process holding a lock stalls for a minute and then wakes up. What does it believe, and what does it do?

The guarantee — the property claimed, and its scope

Without fencing: none. The lock service correctly records a single holder while two processes act on the resource. With fencing at the resource: at most one actor can successfully affect the resource, regardless of pause duration, clock behaviour or partition — the pause becomes a rejected write rather than a corruption.

Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.

What a node knows — observation versus inference

This lesson is the sharpest case in the domain. A paused process observes nothing at all — no gap, no signal, no elapsed time. On resumption its next instruction executes as though the previous one had just completed. It does not know its lease expired, that another holder exists, or that any time passed, unless it explicitly re-reads a monotonic clock and even then only if it thinks to. Every local variable, including "I hold the lock", is still there and still says what it said.

A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.

What guarantee?What does a node know?How does it work?What can fail?How does it fail?Where is coordination?What holds under failure?How does it recover?How would you know?What is the simpler thing?
stale lockGC pauselease expirydouble writer

The sequence, step by step

Worker A acquires a 30-second lease on shard-7 and begins writing. Two seconds in, the runtime begins a stop-the-world collection — or the hypervisor suspends the VM, or the container is throttled to near-zero CPU. A stops executing. It does not log anything, because logging requires executing.

Forty-five seconds later the lock service, which has received no renewal, expires A’s lease. Worker B acquires shard-7 and begins writing. B is entirely correct: the lock service told it the lock was free, and it was.

A resumes. Its program counter is exactly where it was; its stack says it holds the lock; its next statement is the write it was about to perform. It performs it. Two workers are now writing to shard-7, both believing they hold exclusive access, and the lock service’s records are perfectly accurate throughout.

The crucial observation is that no component behaved incorrectly. A followed its code, B followed its code, the lock service followed its protocol. The defect is architectural: the design assumed that holding a lock at time T implies holding it at time T + ε, and across a machine boundary that implication does not hold.

The canonical accident, with every component behaving correctlyprotocol
Worker A is down over this spanWorker ALock serviceWorker BResourcewrite(token=34): deliveredwrite(token=34)write(token=33): deliveredwrite(token=33)acquire shard-7 (30s lease, token 33) (decide) at t=0acquire shard-7 (30s lease, token 33)STOP-THE-WORLD pause begins (crash) at t=2STOP-THE-WORLD pause beginsno renewal — lease expires (decide) at t=8no renewal — lease expiresacquire shard-7 (token 34) (decide) at t=9acquire shard-7 (token 34)B writes; accepted (write) at t=11B writes; acceptedpause ends — no observation of any gap (recover) at t=13pause ends — no observation of any gapwrites, still believing it holds the lock (write) at t=14writes, still believing it holds the lockt=0time →t=15
delivereddelayed (dashed, long)duplicated (×2)dropped — stops short, never arriveswritecrashrecoverdecide
A’s write arrives after B’s. Without a token check the resource applies both. With one, A’s is rejected — and A learns from the rejection what it could not learn from its own clock.

Why the obvious fixes do not work

Every team meets this problem and proposes the same four remedies. Each of them narrows the window and none of them closes it, which is exactly the distinction that matters when the consequence is data corruption.

The pattern across all four is identical: they are attempts to make the holder’s belief reliable. The belief cannot be made reliable, because the holder is the one component that is guaranteed to be uninformed during precisely the interval in question.

  • "Make the lease longer." Now it takes a longer pause. Multi-minute GC pauses and VM suspensions both exist, and every crash now stalls the system for the full lease. You have traded a rare corruption for a routine outage and kept the corruption.
  • "Check the lock before writing." The pause can occur between the check and the write. Verify-then-act is not atomic across a network — this is the same impossibility as A Timeout Tells You Nothing About Whether It Happened, and no amount of re-checking removes it.
  • "Tune the GC / add more memory." Reduces frequency, not possibility. And it addresses only one cause: VM suspension, CPU throttling, disk stalls and network partitions produce identical behaviour.
  • "Detect the pause and abort." A useful defence — compare a monotonic clock before each effect — but it runs *after* the pause, and the effect may already be in flight. It reduces exposure; it does not create a guarantee.

The resolution: move the check to where the effect lands

Since the holder cannot know whether it still holds authority, stop asking it to. The resource knows which token it last accepted, and the resource is where the effect actually takes place. Give every grant a monotonically increasing token, attach it to every write, and have the resource reject anything below its high-water mark, atomically with the write.

Now A’s resumption is harmless. Its write carries token 33, the resource has seen 34, and the write is refused. A learns it has been superseded — from the rejection, which is the only channel that could have told it. This is what "safe rather than merely unlikely" means: the correctness argument no longer contains a sentence about how long a pause might last.

The full mechanism, its granularity choices and the resources that cannot support it are in Fencing Tokens: Making the Stale Actor Safe, Not Just Unlikely. What belongs here is the reasoning that forces it: the holder is structurally incapable of knowing, so the check must live elsewhere, and the only "elsewhere" that is guaranteed to be involved in every effect is the resource itself.

ApproachCloses the window?What it costs
Longer leaseprotocolNo — needs a longer pauseLonger stall on every genuine crash
Re-check before writeprotocolNo — pause can land betweenAn extra round trip per write
GC tuning / bigger heaptypicalNo — reduces frequency onlyAddresses one cause of several
Monotonic pause detectiontypicalNo — acts after the factCheap; a good defence in depth
Fencing token at the resourceprotocol**Yes** — timing-independentThe resource must check; some cannot
Idempotent effecttypicalNot applicable — makes it harmlessOnly possible for some effects
What each mitigation actually achieves

How it looks in production, and why nobody notices

This class of incident is unusually hard to attribute, and the reason is worth stating: every participant logs a successful, ordinary run. A logs "acquired lock, did work, finished". B logs the same. The lock service logs a normal expiry and a normal grant. There is no exception, no timeout, no error rate to alert on.

What you see instead is downstream and delayed: a file with interleaved content, a counter that is wrong by a small amount, two invoices, a segment overwritten. The gap between the event and the symptom is often days, by which time the logs have rotated.

The signals that actually catch it are proactive: lease expiries that occur while the holder is still working (the precursor), pause durations measured by the holder itself with a monotonic clock, and rejected-token counts at the resource (evidence the fencing is doing its job). None of these are default metrics; all three are cheap.

worker-a  10:14:02  acquired lock shard-7 (lease 30s, token 33)
worker-a  10:14:04  writing segment 0004
lockd     10:14:34  lease shard-7 expired (no renewal since 10:14:02)
worker-b  10:14:35  acquired lock shard-7 (lease 30s, token 34)
worker-b  10:14:37  writing segment 0004
worker-a  10:14:47  writing segment 0004        <-- resumed after a 43s pause
worker-a  10:14:48  released lock shard-7        <-- released B's lock
worker-a  10:14:48  job complete
worker-b  10:15:05  job complete

# both jobs "succeeded"; segment 0004 contains bytes from both
The incident as it appears in the logs — no error anywhere

Key points

  • A paused process observes no elapsed time and resumes with every belief intact, including "I hold the lock".
  • The lock service can be perfectly correct while two processes act on the resource.
  • Longer leases, pre-write checks and GC tuning narrow the window; none of them close it.
  • The resolution is to move the authority check to the resource, via a fencing token.
  • The correctness argument must not contain a sentence about how long a pause might last.
  • The incident produces no errors — detect it with expiry-during-work, self-measured pause duration, and rejected-token counts.

The chain, answered

Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.

How it works
  • A acquires the lock and receives a lease and a token.
  • A is descheduled — GC, hypervisor suspension, CPU throttling, or a network partition from the issuer.
  • The lease expires at the issuer because no renewal arrives; no detection of A is required.
  • B acquires the lock with a strictly higher token and begins acting.
  • A resumes with unchanged local state and continues from its next instruction.
  • Both act. Whether that is harmful depends entirely on whether the resource checks tokens.
What can fail at the boundary
  • A stop-the-world garbage collection lasting longer than the lease.
  • A virtual machine suspended and later resumed, or live-migrated with a stall.
  • A container throttled to near-zero CPU by a cgroup quota.
  • A disk stall or swap storm making the process unschedulable in practice.
  • A network partition between holder and issuer, producing the same outcome with no pause at all.
  • A slow release arriving after expiry, deleting the new holder’s lock.
How it fails — what an operator sees
  • Interleaved writes with two clean logs: the operator sees a corrupted file or record and two workers each reporting a successful run, minutes apart, with an expiry between them.
  • Released someone else’s lock: the resumed holder calls release and removes the current holder’s lock, so a third worker starts while the second is mid-flight. The operator sees a lock changing hands with no expiry event.
  • Duplicate external effects: two payouts, two exports, two emails. The operator sees duplicates with distinct request ids and no errors — usually reported by a customer rather than by monitoring.
  • Silent counter drift: both holders increment a shared counter, producing a value that is wrong by a small amount and never reconciles. The operator sees a slow divergence between two systems’ totals with no event to attach it to.
  • Rejection storm after fencing is added: the resumed holder retries its stale-token write in a loop. The operator sees a burst of rejections from one client — which is the mechanism working, and the holder failing to stop on rejection.
Where coordination is required
  • The lock provided coordination at acquire time; the failure is that the design assumed that coordination persisted through the pause.
  • Fencing does not add coordination — it adds a local check at the resource, which costs nothing and works during partitions.
  • This is the clearest instance of the module’s theme: coordinate once to obtain authority, verify locally where the effect lands.
What still holds under failure
  • With fencing, the resource remains correct through any pause of any length; the stale actor simply fails.
  • Without fencing, the resource is corrupted and the lock service is no help — it was never wrong.
  • Effects on unfenceable systems remain exposed regardless, which is why idempotence is the fallback.
How it recovers
  • Detect: alert on lease expiry while the holder still reports work in progress, and on self-measured pauses exceeding a fraction of the lease.
  • Contain: fence every effect; make the holder check a monotonic clock before each effect and abort if the deadline passed; use owner-checked release.
  • Recover: the stale holder must stop on the first rejection, re-acquire, and obtain a higher token before resuming.
  • Reconcile: audit for duplicate or interleaved effects across every expiry boundary — that is the entire search space.
  • Verify: deliberately pause a holder past its lease in a test environment and confirm the resource rejects its subsequent writes.
How you would know
  • Lease expiries that occur while the holder believes it is working — the leading indicator.
  • Holder-measured pause durations from a monotonic clock, as a distribution.
  • Rejected-token counts at the resource, by client — proof the fencing path is exercised.
  • Runtime pause metrics (GC pause time, CPU throttling, steal time) alongside lease TTL on the same graph.
  • Duplicate-effect counters keyed by job or lock name.
When it helps
  • Understanding this is what makes the case for fencing tokens concrete rather than theoretical.
  • It is the argument that ends the "just increase the timeout" conversation, permanently.
  • It generalises to every timeout-based authority: sessions, leader leases, ownership assignment.
When it hurts
  • When it produces paralysis — for efficiency-grade locks, an occasional double run is exactly the acceptable outcome the lock was chosen for.
  • When teams add fencing to the primary write path only, leaving caches, webhooks and exports unprotected while believing the problem is solved.
Simpler alternatives

A paused process does not know it was paused

A paused process does not know it was paused
The canonical accident: A takes a lock, A pauses, the lease expires, B takes the lock, A resumes believing nothing happened, and both act. Nobody is at fault, no error is logged, and the resolution is not a longer timeout.
overlap window
30s
A observed the pause
no
errors logged anywhere
0
exposed effects
4 of 5
Time runs left to right. The one thing to notice is that A’s lane contains no event at all between t=5 and t=45.simplified
lock serviceholder A is down over this spanholder Aholder Bresourcegranted, token 41: deliveredgranted, token 41write #1 (token 41): deliveredwrite #1 (token 41)no such message exists: sent, never arrives — dropped in flightno such message existsdropped — never arrivesgranted, token 42: deliveredgranted, token 42write #2 (token 42): deliveredwrite #2 (token 42)write #3 (token 41): deliveredwrite #3 (token 41)acquire (decide) at t=0acquirewrite #1 (write) at t=5write #1paused — observes nothing (crash) at t=5.5paused — observes nothinglease expires (t=15) (decide) at t=15lease expires (t=15)acquire (decide) at t=16acquirewrite #2 (write) at t=19write #2resumes — still believes it holds (recover) at t=45resumes — still believes it holdswrite #3 (write) at t=46write #3t=0time →t=46.4
delivereddelayed (dashed, long)duplicated (×2)dropped — stops short, never arriveswritecrashrecoverdecide
A’s lease ended at t=15 and A resumed at t=45 — an overlap of 30s during which two processes both believed they held the lock. Neither logged anything unusual. The lock service was correct throughout: it recorded exactly one holder at every instant.
what A holds in memory when it resumes
holdsLock      = true          // never changed; nothing changed it
leaseToken     = 41            // the only token A has ever had
acquiredAtWall = 12:04:01      // still says what it said

// A's next instruction executes as though the previous one
// had just completed. There is no gap to observe, no signal,
// and no elapsed time — unless A explicitly re-reads a
// monotonic clock, and even then only if it thinks to.
effect inventory — tick what actually carries and checks a token
4 effects would be performed twice: object-store segment write, cache invalidation, outbound webhook, customer email. This is the classic incomplete adoption — consistent primary data with a stale cache and duplicate webhooks, and a team that believes the problem is solved. Audit every write path including admin tools and migration scripts, which are the usual bypass.
cache invalidation, outbound webhook, customer email cannot check a token, because there is no atomic conditional write to hang the comparison on. The only remaining defence is idempotence: make a second application converge to the same state. Where even that is impossible — a customer email — the honest answer is that it is exposure you have chosen to accept, and it should be written down as such.
It is rare per operation and certain at scale: a one-in-a-million window at ten thousand operations per second occurs several times a day. And the general principle outlives locks entirely — authority granted at time T cannot be assumed to hold at T + ε when ε is unbounded and unobservable to the holder. Any correctness argument containing “the pause will be shorter than the lease” is an assumption about the world rather than a property of the system.
protocolThe unsafe window cannot be eliminated by any timing parameter: for any lease duration there exists a longer pause, and verify-then-act is not atomic across a network.
protocolWith a monotonic token checked atomically at the resource, at most one actor succeeds regardless of pause duration. That argument uses no timing assumption at all.
typicalMulti-second GC pauses are routine in large heaps and multi-minute pauses are documented. VM suspension, CPU throttling and disk stalls produce identical behaviour with no runtime involvement — non-GC languages are not exempt.
simplifiedOne resource, one key, no network delay on the grant. Real exposure is spread across every effect the job performs.

What people believe, and what is true

Claim

This is rare enough to ignore.

Reality

It is rare per operation and certain at scale. A one-in-a-million window at ten thousand operations per second occurs several times a day.

Claim

The lock service had a bug.

Reality

It behaved correctly throughout. The defect is the assumption that a grant remains true while the holder is not executing.

Claim

The process will notice it was paused.

Reality

Only if it explicitly re-reads a monotonic clock and checks. Nothing in the runtime tells it, and its own state is unchanged.

Claim

It only happens with garbage collection.

Reality

VM suspension, CPU throttling, disk stalls and network partitions all produce identical behaviour. Non-GC languages are not exempt.

Go deeper

Only the levels this lesson can honestly fill — a missing level is a claim nobody had.

Overview

A process holding a lock can pause long enough for its lease to expire, then resume with no idea anything happened. Two processes then act. The fix is to have the resource reject the old holder, not to make the lease longer.

Practical

Attach a token to every effect and check it at the resource. Monitor expiries that occur during work, and have the holder measure its own pauses with a monotonic clock and abort. Use owner-checked release. Then test it by actually suspending a holder past its lease.

Advanced

The general principle is that authority granted at time T cannot be assumed to hold at T + ε when ε is unbounded and unobservable to the holder. Any correctness argument containing "the pause will be shorter than the lease" is an assumption about the world rather than a property of the system. Relocating the check to the point of effect replaces that assumption with an integer comparison, which is the only move in this space that changes the class of guarantee rather than its probability.

Apply it

Build it, then break it
  • 🔧 Reproduce the scenario deliberately: pause a holder past its lease and observe the resource with and without a token check.
  • 🔧 List every effect a locked job in your system performs and mark which are fenced, which are idempotent, and which are exposed.
Reason about this
  • A compaction job holds a lock and writes segments to object storage. It is throttled for 90 seconds. Describe what the storage bucket contains afterwards, with and without conditional writes.
Interview questions
  • 💬 Walk me through what happens when a lock holder pauses for longer than its lease.
  • 💬 Why does increasing the lease duration not fix it?
  • 💬 Why can the client not just check that it still holds the lock before writing?
  • 💬 What would you monitor to know this is happening in your system today?