Time & Ordering

Happens-Before: The Only Ordering You Actually Have

If clocks cannot order events across machines, something must. Lamport's happens-before relation orders exactly the pairs of events that could have influenced each other — and deliberately leaves everything else unordered. It is a partial order, and that is the point, not a limitation.

▶ Run the lab

The question this answers

The question

If I cannot use clocks, what does it even mean for one event to come before another?

The guarantee — the property claimed, and its scope

The happens-before relation a → b is a partial order over events. It guarantees that if a → b then a could have influenced b, and if neither a → b nor b → a then neither could have influenced the other — they are *concurrent*. It guarantees nothing about physical time: a → b does not mean a occurred earlier by any clock, and two concurrent events may be hours apart.

Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.

What a node knows — observation versus inference

A node knows the order of its own events (it executed them in that order) and knows that every message it received was sent before it was received. From those two local facts alone it can derive a large part of the global causal order — without any clock, and without any coordination. What it cannot derive locally is whether an event it has never heard about is concurrent with one of its own; that requires the metadata of Vector Clocks: Buying Concurrency Detection at O(N).

A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.

What guarantee?What does a node know?How does it work?What can fail?How does it fail?Where is coordination?What holds under failure?How does it recover?How would you know?What is the simpler thing?
causalityhappens-beforepartial orderlamportconcurrency

Three rules, and everything follows

Lamport's definition is famously small. a → b ("a happens before b") is the smallest relation satisfying three rules:

That is the whole definition. Everything in this module — Lamport clocks, vector clocks, causal consistency, version vectors — is machinery for tracking or approximating this relation, so it is worth being exact about what it says.

Crucially, is not a total order. Take two events on different nodes with no chain of messages connecting them: neither happens before the other. They are concurrent, written a ∥ b. Concurrency here does not mean "at the same time" — two concurrent events can be a week apart. It means *neither could have known about the other*, which is the property that actually matters when you are deciding whether they conflict.

  • Program order. If a and b happen on the same node and a comes first, then a → b.
  • Message order. If a is the sending of a message and b is its receipt, then a → b.
  • Transitivity. If a → b and b → c, then a → c.
  • Concurrency is the default. Anything not related by those rules is concurrent — the relation is a partial order, not a total one.
Which pairs are ordered, and which are genuinely concurrentprotocol
Node PNode QNode Rupdate(x): deliveredupdate(x)p1: write x (write) at t=1p1: write xp2: send to Q (write) at t=5p2: send to Qq1: write y (write) at t=2q1: write yq2: receive from P (read) at t=7q2: receive from Pq3: write z (write) at t=9q3: write zr1: write w (write) at t=4r1: write wt=1time →t=9
delivereddelayed (dashed, long)duplicated (×2)dropped — stops short, never arriveswriteread
p1 → p2 (program order), p2 → q2 (message), so p1 → q3 by transitivity. But q1 ∥ p1: no chain of messages connects them, so neither could have influenced the other, and r1 is concurrent with every event shown. Note that q1 is drawn *later* than p1 in real time and is still not ordered after it — physical position on the page is irrelevant to the relation.

The same relation you already met in Concurrency — at a different scale

If you have read the Concurrency domain, this will feel familiar, and it should: it is literally the same relation. Concurrency's happens-before is the edge that makes one thread's write visible to another — established by a lock release paired with an acquire, or by a release-store paired with an acquire-load. Ours is established by a message send paired with a receive. Same relation, same partial order, same consequence: anything not ordered is concurrent, and concurrent access to the same datum is where the bugs live.

The two differ only in what plays the role of the synchronising edge, and in what happens when you get it wrong. Inside a machine, the edge is provided by hardware and compiler mechanisms — the memory model defines which pairs are ordered, and violating it gives you a torn read or a stale value. Across machines, the edge is provided by an actual message, the ordering is not enforced by anything, and violating it gives you a conflicting write that nobody detects until a merge.

The naming here is deliberate: this domain calls it causal-ordering so that it can never be confused with the memory-model edge, which the Concurrency domain owns. Read them together. The transferable insight is that "concurrent" is a *structural* property — the absence of a synchronising edge — and not a statement about time. Once that clicks in one domain it is free in the other.

Inside a machine (Concurrency)Across machines (here)
What creates the edgeLock release/acquire, release-store/acquire-load, thread start/joinMessage send/receive, and program order on a node
Who enforces itprotocolCompiler and CPU, per the memory modelNobody. The application must track it if it wants it.
Cost of the edgetypicalA barrier: tens to hundreds of cyclesA network round trip: microseconds to hundreds of milliseconds
Symptom when missingStale read, torn value, a race that reproduces once a monthConflicting writes, lost update, divergent replicas
DetectionassumptionRace detectors; the schedule is enumerable in principleVersion metadata; concurrency is detectable only if you carried it
One relation, two scales

Why a partial order is the right answer, not a weaker one

The instinct on first meeting is that it is deficient: it fails to order some pairs, so surely a better mechanism would order all of them. That instinct is exactly backwards, and getting past it is the point of this lesson.

A total order over all events is *available* — you can always impose one (Total Order Broadcast Is Consensus Wearing a Different Hat) — but it is expensive, because it requires the nodes to agree, which requires consensus, which requires round trips and a majority to be reachable. The partial order is free: every node can derive it from information it already has, with no messages beyond the ones the application was already sending.

More importantly, a total order manufactures information that does not exist. If two users edit different fields of a document with no knowledge of each other, there is no fact of the matter about which came first. Imposing an order does not discover the truth; it invents one, and inventing one is how Last Write Wins Is Data Loss You Chose by Default silently discards a real edit. The partial order preserves the useful distinction: *ordered* pairs have a right answer, *concurrent* pairs need a merge decision from the application (Only the Application Knows What the Merge Means).

So the shape of a good design is: track causality, let the partial order do the work it can do for free, and reserve coordination for the specific places where you genuinely need concurrent events to be ordered — usually because an invariant spans them (Start From the Invariant, Not From the Architecture).

What causality buys you, concretely

Causal ordering is the strongest consistency model that can be provided without giving up availability during a partition — a result worth remembering, because it puts a precise ceiling on what "available" systems can offer (Causal Consistency: Never Show an Effect Before Its Cause, CAP: What the Theorem Actually Says).

Concretely, it is what makes these scenarios behave sanely: a reply never appears before the comment it replies to; a photo never shows up in a feed before the album that contains it; after you remove someone from an access list and then post, they do not see the post. Each of those is a causal dependency created by one user's actions passing through the system, and each is a real bug when the dependency is dropped.

What it does *not* buy: any statement about events that are genuinely concurrent. If two people simultaneously edit the same field, causality tells you truthfully that they are concurrent and hands the problem back to you. That is not a failure of the model. It is the model correctly reporting that your application, not the ordering mechanism, has to decide.

Alice:  removes Bob from "close friends"        (event a)
Alice:  posts "close friends only" photo         (event b)     a -> b
Bob:    sees the photo

Cause: the two writes went to different replicas, and b propagated
to Bob's replica before a did. Without causal delivery there is nothing
in the system that knows b depended on a.

The fix is not "make replication faster". It is to carry the dependency
with the write, and hold b until a is applied.
The classic causality violation, in three lines of user-visible behaviour

Key points

  • Happens-before is defined by three rules: program order, message send-before-receive, and transitivity.
  • It is a partial order. Events not related by it are *concurrent* — meaning neither could have influenced the other, not that they occurred simultaneously.
  • It is the same relation the Concurrency domain calls happens-before; only the synchronising edge differs (a message rather than a lock or a barrier).
  • The partial order is free — derivable locally. A total order costs consensus.
  • A total order over concurrent events invents information that does not exist, which is how silent data loss enters a system.
  • Causal consistency is the strongest model available without sacrificing availability under partition.

The chain, answered

Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.

How it works
  • Each node records its own events in the order it executed them; this establishes program order locally with no coordination.
  • When a node sends a message it attaches whatever causal metadata the scheme requires (a counter, a vector, a set of dependencies).
  • On receipt, the receiver merges that metadata into its own state, establishing the send → receive edge.
  • Transitivity then propagates: the receiver's subsequent events inherit the whole causal past of the message it received.
  • Two events are declared concurrent exactly when neither node's metadata dominates the other's.
What can fail at the boundary
  • A message is lost, so an edge that should exist never forms and two causally related events look concurrent.
  • Causal metadata is stripped at a boundary — a queue, a proxy, a serialisation format that drops unknown fields — silently converting a causal chain into unrelated events.
  • Causality flows *outside* the system: two services communicate through a user's browser, a phone call, or a shared spreadsheet, and no message inside the system records the dependency.
  • A node restarts and loses its causal state, so its subsequent events appear unrelated to its own past.
  • Metadata grows without bound and someone truncates it as an optimisation, breaking the relation it encoded.
How it fails — what an operator sees
  • The reply-before-comment bug: a user sees a response to something that is not there yet. The operator sees a support ticket describing an impossible UI state, and both writes look correct in every log.
  • Privacy violation via reordering: an access-control change and a subsequent post are applied in the wrong order on a replica, and someone sees content they were just removed from. Nothing errors; the audit log shows both operations succeeding.
  • Hidden causality through a side channel: two services are actually dependent because a human copied a value between them, so the system sees concurrent writes and merges them, dropping one. The operator cannot reproduce it because the dependency is not in any trace.
  • Metadata stripped by an intermediary: after introducing a message broker, previously-ordered writes start being reported as concurrent. The operator observes a step change in conflict rate at exactly the deploy time of the broker.
  • Causal delivery stalls: a node holds a message waiting for a dependency that was lost, and that key silently stops updating on that replica while everything else works.
Where coordination is required
  • Deriving the partial order requires no coordination at all — that is its defining economic advantage, and it is why causal consistency is available under partition.
  • The cost is metadata carried on messages, which is bandwidth and storage rather than round trips and availability.
  • Turning the partial order into a total order requires agreement among nodes: see Four Orderings, Four Prices and Total Order Broadcast Is Consensus Wearing a Different Hat for what that costs.
What still holds under failure
  • The relation itself is unaffected by node failure or partition — it is defined over the messages that did happen, not over the ones that should have.
  • During a partition, each side continues to extend the order locally and correctly; events across the partition are concurrent, which is a true statement.
  • After healing, the two histories merge into a single partial order with a large set of concurrent pairs — exactly the set the conflict module has to resolve.
How it recovers
  • Detect: measure the rate of concurrent-pair detections. A sudden change usually means metadata is being lost, not that user behaviour changed.
  • Contain: buffer messages whose causal dependencies have not arrived, rather than applying them out of order — with a bounded buffer and an explicit policy when it fills.
  • Recover: request missing dependencies explicitly from a peer, or fall back to a full state sync when the gap is too large (Anti-Entropy: Repairing Divergence Nobody Reported).
  • Reconcile: resolve the accumulated concurrent pairs by application rule, not by timestamp (Only the Application Knows What the Merge Means).
  • Verify: replay a known causal chain end to end and assert the effects appear in dependency order at every replica.
How you would know
  • Rate of pairs classified as concurrent versus causally ordered — a direct measure of how much real conflict your workload generates.
  • Size of the pending-dependency buffer per replica; growth means an edge is missing and something is stuck.
  • Time from an event's creation to its causal dependencies being satisfied at each replica — the practical latency of causal delivery.
  • Count of messages arriving with missing or unparseable causal metadata, broken down by ingress path, which is how you find the intermediary that strips it.
When it helps
  • Anywhere a user's actions create dependencies that must be visible in order: social feeds, comment threads, access-control changes followed by content, multi-step workflows.
  • As the correctness frame for any system that must stay available under partition, since causal is the ceiling there.
  • As a design tool: asking "is this pair genuinely ordered, or am I inventing an order?" is the fastest way to find a lurking lost-update bug.
When it hurts
  • Where a single node already sequences everything, tracking causality adds metadata and buys nothing — the node's program order is already a total order.
  • For workloads where every write is to a distinct key with no cross-key dependency, the machinery is pure overhead.
  • Where the business genuinely requires a global total order (a ledger, a sequence of trades), causality alone is insufficient and you must pay for consensus.
Simpler alternatives

Happens-before: three rules, and everything follows

Happens-before: three rules, and everything follows
Program order, send-before-receive, transitivity. Pick any two events and read what the system can honestly say about their order.
Pick two events below: the first click sets a, the second sets b.
#NodeEventPick
1Pwrite x = 1a
2Psend → Q: x = 1
3Qunrelated work
4Qreceive
5Qsend → R: derived y
6Rwrite z (nobody told it anything)b
7Rreceive
8Pwrite x = 2
a
P · write x = 1
b
R · write z (nobody told it
relation
concurrent
ordering exists?
no — and that is the answer
a ∥ b — concurrent: neither could have influenced the other. No chain of program-order and message edges connects them in either direction. That is not "we could not tell" — it is a fact about the system: neither event is in the other’s causal past.
Three rules, drawn.protocol
PQRx = 1: deliveredx = 1derived y: deliveredderived y write x = 1 (decide) at t=0 write x = 1 send → Q: x = 1 (write) at t=1 send → Q: x = 1 unrelated work (write) at t=2 unrelated work receive (read) at t=3 receive send → R: derived y (write) at t=4 send → R: derived y write z (nobody told it anything) (decide) at t=5 write z (nobody told it anything) receive (read) at t=6 receive write x = 2 (write) at t=7 write x = 2t=0time →t=7
delivereddelayed (dashed, long)duplicated (×2)dropped — stops short, never arriveswritereaddecide
Every edge in this picture is either a lane running left to right (program order) or an arrow between lanes (a message). Those two edge types, closed under transitivity, are the entire relation.
Concurrent does not mean "at the same time" — two concurrent events can be a week apart, and two events a nanosecond apart can be causally ordered. A partial order is not a weaker answer than a total one; it is the honest one. It orders every pair for which an ordering exists and refuses to invent one for the rest, and inventing is exactly what loses data. The partial order is free — each node derives it from local facts. A total order costs consensus.
protocolThe relation is defined by those three rules; nothing here is estimated. The relation only sees dependencies carried by messages the system observed — a user who reads a value on a phone and types it into a laptop creates a real dependency that no clock and no vector can see.

What people believe, and what is true

Claim

Concurrent means "at the same time".

Reality

It means neither event could have influenced the other. Two concurrent events can be a week apart, and two events a nanosecond apart can be causally ordered.

Claim

happens-before is about time.

Reality

It is about *potential influence*. It deliberately says nothing about physical time, which is exactly what makes it usable when clocks are untrustworthy.

Claim

A partial order is a weaker version of a total order.

Reality

It is a more honest one. It orders every pair for which an ordering exists, and refuses to invent one for the rest. Inventing is what loses data.

Claim

This is different from the happens-before in the Concurrency domain.

Reality

It is the same relation. The synchronising edge is a message instead of a lock or barrier, and nothing enforces it for you.

Go deeper

Only the levels this lesson can honestly fill — a missing level is a claim nobody had.

Overview

One event happens before another if it could have influenced it — through program order on a node, or through a message. Everything else is concurrent, meaning neither knew about the other.

Practical

Design by asking which pairs of operations are genuinely ordered. Carry causal metadata on writes so the system can tell ordered from concurrent, and route concurrent pairs to a real merge rule instead of a timestamp comparison. Watch for intermediaries that strip metadata.

Advanced

The relation is the reflexive-transitive closure of program order and message order, and the resulting structure is a directed acyclic graph of events. Causal consistency is the strongest model implementable in an always-available system, which is why it is the practical ceiling for AP designs. Beyond it you are paying for agreement.

Apply it

Build it, then break it
  • 🔧 Draw the spacetime diagram for a three-node interaction in your own system and list every concurrent pair.
  • 🔧 Find a place where your code compares timestamps to decide precedence, and determine whether the two events are ever genuinely concurrent. If they are, that comparison is losing data.
Reason about this
  • A user removes a follower and immediately posts. The removed follower sees the post. Both operations succeeded. Explain the mechanism and the fix.
  • Two microservices exchange no messages, but a human operator copies an id from one UI to the other. Does the system see a causal dependency? What are the consequences?
Interview questions
  • 💬 Define happens-before. Then give two events that are concurrent despite occurring an hour apart.
  • 💬 Why is causal consistency the strongest model available to a system that must stay up during a partition?
  • 💬 Your system reports a sudden increase in concurrent writes after a deploy that added a message broker. What is your first hypothesis?