The question this answers
How do I guarantee a username is unique when no single node holds all the usernames?
Depends entirely on the design chosen. Routing by key gives uniqueness per name, enforced by the single node that owns that name — the strongest practical guarantee and the cheapest. A central authority gives global uniqueness at the cost of its availability. Reservation schemes guarantee uniqueness of the *reservation* and require a repair path for abandoned ones. Optimistic insert-and-detect guarantees nothing up front and relies on the store to reject the second write.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
To check "no other user has this name", a node must know about every user — which is precisely what sharding prevents. This invariant contains a negation over global state, so no node can check it locally under any partitioning of the *data*. The way out is to partition the decision instead: make one node the sole decider for each name, so the negation shrinks from "no user anywhere" to "no user among the ones I own", which is locally checkable.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
Why this invariant is genuinely hard
Most invariants in this module can be dissolved by Coordination Avoidance: Restructuring the Problem Instead of Paying for It. This one resists all three of its questions, and it is worth seeing why, because the pattern generalises to every uniqueness-shaped requirement.
It is not repairable in any satisfying way — renaming one of two users who both got alice breaks their links, their identity and their trust. It is not commutative: two claim("alice") operations produce different outcomes depending on order, and that is the entire point of the constraint. And a naive partitioning of *users* does not help, because two users on different shards can both claim the same name.
What does work is the third question applied to the right thing. Partition by the name, not by the user. Every name maps to exactly one owner, and that owner sees every claim for it. The global negation becomes a local one, and the invariant becomes checkable without any coordination at claim time.
Four designs, four failure modes
These are the options in practice. They are ordered roughly by how much coordination they require, and the differences between them are almost entirely about what happens when something fails mid-claim.
| Design | How it works | Fails by |
|---|---|---|
| Route by keyprotocol | `hash(name)` selects the one node that owns that name; it decides alone | That shard being unavailable — no claims for names it owns |
| Central authorityprotocol | One service or table owns all names | Its availability and throughput become yours |
| Consensus per claimprotocol | Each claim is a consensus decision | Cost and latency; needs a majority for every signup |
| Reservation / two-phaseassumption | Reserve the name with a TTL, complete signup, confirm | Abandoned reservations; needs expiry and a repair path |
| Optimistic insert-and-detecttypical | Insert and let a unique index reject the loser | Only works if one store holds all names — i.e. it is really "central authority" |
Route by key: the answer that is usually right
Take the name, hash it, and let that select the shard responsible. Every claim for alice — from any client, at any time — lands on the same node, which holds every existing claim for the names it owns and can therefore decide with a purely local check. No lock, no consensus round, no coordination at claim time.
The coordination has moved to the ownership map: which shard owns which range of hashes. That map changes rarely, is small, and is exactly the kind of fact Do You Actually Need Consensus? endorses paying for. You buy agreement once per rebalance instead of once per signup.
Two details make or break it. During rebalancing, ownership of a hash range moves, and the old and new owners must not both accept claims for it — so the handover needs a fencing token or a brief pause for the affected range, exactly as Rebalancing: A Load Spike You Schedule for Yourself describes. And the shard that owns a name is a single point of availability for that name: if it is down, alice cannot be claimed, while every other name is unaffected. That is a far better failure profile than a global outage, and it must be a deliberate choice rather than a surprise.
- Shard 1 — owns hashes 0x00–0x55: holds "alice", "dave"
- Shard 2 — owns 0x56–0xAA: holds "bob"
- Shard 3 — owns 0xAB–0xFF: names here cannot be claimed
- Client A — claim("alice") → hash → Shard 1
- Client B — claim("alice") → hash → Shard 1
- s1believes “I can decide "alice" alone, correctly”✓ and it is true
- c2believes “"alice" is available because my shard has no record”✕ and it is false
- s3believes “I still own 0xAB–0xFF”✕ and it is false
Every node above is acting on what it believes. Nothing in the cluster tells the mistaken one that it is mistaken.
Reservation schemes, and the reason they exist
Uniqueness rarely stands alone. A signup flow claims a name, then does several other things — creates an account, charges a card, sends a verification email — any of which can fail. If the name is claimed permanently at step one, failed signups accumulate names nobody owns. If it is claimed at the end, two users can pass the check and collide.
A reservation resolves this: claim the name with a TTL, complete the rest of the flow, then confirm to make the claim permanent. If the flow fails or is abandoned, the reservation expires and the name returns to circulation. This is Leases: Authority With an Expiry Date applied to a namespace, and it inherits the same properties — including that the expiry is a guess about how long the flow takes.
Two failure modes come with it. The TTL is too short and a slow-but-legitimate signup loses its name mid-flow, which is a bad experience and a support ticket. The TTL is too long and a name is unavailable for hours after an abandoned attempt, which is an attack surface: an adversary can hold a large portion of a namespace by starting and abandoning signups. Rate-limit reservations per client, and monitor the ratio of confirmed to expired.
1shard = owner_of(hash(name)) # one decider per name2 3# 1. reserve — atomic conditional insert at the owning shard4ok = shard.insert_if_absent(name, {state: "reserved",5 owner: signup_id,6 expires_at: now() + 15m})7if not ok: return NAME_TAKEN # includes names merely reserved8 9# 2. the rest of the flow: account, payment, verification — any may fail10 11# 3. confirm — only the reservation holder may promote it12shard.update_if(name, where={state: "reserved", owner: signup_id},13 set={state: "confirmed", expires_at: null})14 15# repair path (required, not optional):16# sweep expired reservations and delete them17# monitor confirmed:expired ratio, and reservations per clientThe uncomfortable questions worth asking first
Before building any of this, two questions dissolve the problem more often than any design solves it.
Does it need to be globally unique? Unique per tenant, per region or per organisation is a completely different and much cheaper invariant, because the tenant is a natural partition key and the negation shrinks to a set one node can hold. A great many "global" uniqueness requirements are global only by accident.
Does the user-visible identifier need to be the unique one? Systems that separate a stable internal id from a display name avoid this problem for everything except the name lookup itself — and a display name that is unique only within a discriminator (as Discord and others do) or not unique at all removes the constraint entirely. The cheapest uniqueness constraint is the one the product does not require.
When the answer to both is that yes, you really do need it: route by key, use a reservation if the flow has multiple steps, and monitor the repair path. That combination is correct, cheap, and degrades per name rather than globally.
Key points
- Uniqueness is a negation over global state, so no node can check it locally under a partitioning of the data.
- Partition the *decision* instead: route by hash of the name so one node owns every claim for it.
- The coordination moves to the ownership map, which changes rarely — the good kind of coordination.
- Route-by-key degrades per name, not globally: an unavailable shard blocks only the names it owns.
- Multi-step flows need a reservation with a TTL, plus an expiry sweep and monitoring of the confirmed:expired ratio.
- Ask first whether uniqueness must be global, and whether the user-visible name must be the unique one.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • Derive the owner from the name itself —
hash(name) → shard— so routing is deterministic and client-independent. - • The owning shard performs an atomic conditional insert: succeed only if no record for that name exists.
- • For single-step flows, the successful insert is the claim and the work is done.
- • For multi-step flows, the insert records a reservation with an expiry and an owning signup id.
- • On completion, the reservation is promoted to confirmed, conditional on it still being held by the same signup.
- • A background sweep deletes expired reservations, returning names to circulation.
- • Ownership changes go through the consensus-backed map, with fencing so the old owner cannot accept claims for a transferred range.
- • The owning shard is unavailable, so names in its range cannot be claimed.
- • The claim succeeds but the response is lost, so the client retries and must find its own reservation rather than a conflict.
- • A reservation expires mid-flow and another user takes the name before the first completes.
- • The expiry sweep stops running and abandoned reservations accumulate.
- • Ownership moves during rebalancing and two shards briefly accept claims for the same range.
- • The hash function or shard count changes without a migration, so a name maps to a different shard than the one holding it.
- • Duplicate names after a rebalance: two shards accepted claims for the same range during handover. The operator sees two records for one name and no error in either shard’s log — this is why the handover needs fencing.
- • Names exhausted by abandoned reservations: the sweep has been failing silently. The operator sees rising NAME_TAKEN rates and a reservations table growing monotonically, while the confirmed count is flat.
- • Namespace squatting: a client starts thousands of signups and abandons them, holding desirable names. The operator sees an extreme confirmed:expired ratio concentrated on one client or IP range.
- • Lost name on retry: a client whose success response was lost retries and receives NAME_TAKEN for its own reservation. The operator sees signup abandonment at the confirm step and support tickets about a name being "already taken by me".
- • Hot shard: names are not uniformly distributed (a promotion drives many claims to a pattern) and one shard saturates. The operator sees latency and rejections concentrated on one owner while the rest of the fleet is idle.
- • Route-by-key requires no coordination at claim time — the owning shard decides alone.
- • Coordination is confined to the ownership map and to rebalancing, both of which are rare.
- • A central authority or per-claim consensus moves coordination onto the signup path, which is the cost route-by-key exists to avoid.
- • The invariant is never violated by unavailability: a shard that is down rejects claims rather than accepting conflicting ones.
- • Availability degrades per name range rather than globally — the key property that makes this design good.
- • Reservations left behind by a failure are recovered by expiry, provided the sweep is running.
- • Detect: a continuous violation query — group by name, alert on any count greater than one. Cheap, and the only thing that catches a rebalancing bug.
- • Contain: fence ownership handovers so the old owner cannot accept claims for a transferred range; rate-limit reservations per client.
- • Recover: restore the unavailable shard; claims for its names resume with no data repair needed.
- • Reconcile: sweep expired reservations continuously and monitor the sweep’s output, not merely its liveness.
- • Verify: run the duplicate-name query after every rebalance, and check the confirmed:expired ratio for abuse.
- • Duplicate-name count from the violation query — expected to be exactly zero.
- • Reservation confirmed:expired ratio, overall and per client.
- • Reservations table size and age of the oldest unswept entry.
- • Claim rejection rate by shard, which reveals both hot shards and unavailable ranges.
- • Claims accepted during a rebalancing window, per range — the direct check for double-ownership.
- • Usernames, handles, email addresses, slugs, tenant subdomains, short links — any user-chosen identifier in a shared namespace.
- • Any allocation of a scarce named resource where two owners is unacceptable.
- • Wherever a single database can no longer hold the whole namespace.
- • When the namespace fits comfortably in one database — a unique index is simpler, transactional and correct, and you should use it.
- • When uniqueness is really per tenant, in which case the tenant is the partition key and this whole design is unnecessary.
- • When the product does not actually require unique display names and the constraint is inherited rather than required.
- • A unique index in a single database. The correct answer whenever the namespace fits, and it fits far more often than sharding enthusiasm suggests.
- • Server-generated identifiers (UUIDs, snowflake ids) which are unique by construction with no coordination at all — available whenever the user does not choose the value.
- • Uniqueness scoped to a tenant or region, turning a global invariant into a local one.
- • A discriminator suffix (
alice#4821) so the user-chosen part need not be unique — the product-level dissolution of the constraint. - • Optimistic insert with a store-enforced constraint, where one store holds the whole namespace.
One name, many shards
hash32("name:alice") % 4 = shard 1
shard 1 holds every claim for "alice", so it can answer the question alone.
An atomic conditional insert there is the entire mechanism.SELECT name, count(*) FROM users GROUP BY name HAVING count(*) > 1
What people believe, and what is true
Sharding the users table shards the uniqueness check.
Two users on different shards can claim the same name. You must shard by the *name*, not by the user.
A distributed lock on the name solves it.
It works and it is more expensive and more fragile than routing by key, which achieves the same exclusivity with no lock service and no lease to expire.
Check-then-insert is fine because collisions are rare.
Two concurrent checks both pass and both insert. Rare per request means routine at scale, and the outcome is unrepairable.
Reservations can be added later.
The reservation TTL and the expiry sweep are the design. Adding a TTL to a system with permanent claims means deciding what to do with years of abandoned names.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Uniqueness cannot be checked locally because it is a statement about everything. Route each name to exactly one owning node so that node can check it locally, and the problem becomes ordinary.
Practical
Shard by hash of the name. Use an atomic conditional insert at the owning shard. For multi-step signup flows, reserve with a TTL, confirm at the end, and run an expiry sweep you actually monitor. Fence ownership handovers. Run a duplicate-name query continuously — it is the only thing that catches a rebalancing bug.
Advanced
Uniqueness is the canonical non-monotonic invariant: adding information can retract a previous conclusion, so by CALM it has no coordination-free implementation. Route-by-key does not evade that — it minimises the *scope* of the coordination to a single node per key, which is the best available outcome. The residual coordination shows up exactly where scope changes hands, which is why rebalancing is the one operation that can violate the invariant, and why it needs the same fencing discipline as any other authority transfer.
Apply it
- 🔧 Implement uniqueness with route-by-key and write the violation query that would catch a rebalancing bug.
- 🔧 Design the abuse defence for a reservation scheme and state which metric detects squatting.
- ⚡ A team shards users by user id and adds a unique index on username in each shard. Explain exactly how two users end up with the name "alice", and what you would change.
- 💬 How would you guarantee usernames are unique across a sharded system?
- 💬 Why does sharding by user id not help, and what should you shard by instead?
- 💬 Design the signup flow when claiming a name and completing signup are separate steps.
- 💬 What breaks during a rebalance, and how do you prevent it?