Sticky Sessions
Pinning a client to one instance buys cache locality and hides instance-local state — and it costs you failover, rebalancing and clean scale-down.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
When is it right to send a user's requests to the same instance every time, and what does that cost?
Users are being logged out at random since we scaled to three instances, and someone has suggested affinity as the fix.
Turn on session affinity at the load balancer. Each user keeps hitting the same instance, so the in-memory session works again and nothing else has to change.
The instance restarts — deploy, scale-down, crash, node drain — and every user pinned to it loses their session at once. The bug is now rarer and much larger (Graceful Shutdown).
- The instance restarts — deploy, scale-down, crash, node drain — and every user pinned to it loses their session at once. The bug is now rarer and much larger (Graceful Shutdown).
- Affinity is decided when the user first arrives, so scaling out does not move anyone: new instances receive only new users, and the existing instances stay hot (Autoscaling a Backend).
- Load becomes uneven in a way no algorithm corrects, because the balancer is no longer allowed to choose.
- A rolling deploy replaces every instance, so every pinned user is redistributed at once — a session-loss event on every release.
- The underlying instance-local state is now hidden rather than fixed, so the next scaling problem is harder to diagnose than the first (Making an Existing Service Stateless).
What is actually happening
- Affinity is implemented either by a cookie the balancer sets (it remembers which instance served the first request) or by hashing a request property — client IP, or a header — to an instance.
- Cookie-based affinity is precise and survives client IP changes; hash-based affinity requires no state at the balancer and breaks when the input changes (a mobile client moving between networks) or when the instance set changes.
- Consistent hashing limits the disruption of an instance set change to roughly one instance's share of keys instead of remapping everyone — which is why it is the standard choice when affinity is for cache locality (Consistent Hashing in Software Architecture).
- Affinity is a best-effort routing preference, not a guarantee. The instance can disappear at any time, and then the request goes somewhere else. Any design that requires the pin to hold is incorrect.
- It changes the failure model: instead of a small failure per request spread across users, you get a total failure for the subset of users pinned to a lost instance.
- Some protocols are inherently sticky. A WebSocket or SSE stream is a connection to one instance for its lifetime, so state associated with it is instance-local whether or not you enabled affinity (Polling vs Long Polling vs SSE vs WebSockets in Networking).
What it buys and what it costs
Affinity is worth having when there is a real per-user working set on the instance — a warm cache, an expensive-to-rebuild context, an open connection. It is not worth having as a way to keep sessions in memory, because the failure it creates is strictly worse than the one it hides: instead of an occasional inconsistency, a cohort of users fails together.
The honest way to hold it is: affinity may make the common case faster; it must never make the uncommon case incorrect.
| Dimension | Without affinity | With affinity | What decides it |
|---|---|---|---|
| Instance failure | A few in-flight requests fail | Every pinned user loses their context at once | Whether per-user state is recoverable. |
| Scale-out | New instances immediately take a share | New instances only receive new users | How quickly load must be relieved. |
| Rolling deploy | Requests move seamlessly | Every pin is broken during the rollout | Whether losing the pin is visible to the user. |
| Load distribution | The balancer can correct skew | Skew persists; the balancer cannot choose | How uniform user activity is. |
| Per-user cache | Hit rate divided across the fleet | Warm on one instance | Whether the cached data is expensive to rebuild. |
| Long-lived connections | Not applicable — pinned regardless | Pinned regardless | Reconnection and resume, not routing. |
The failover cost, concretely
The scenario worth picturing: six instances, affinity on, one instance terminated by a node drain. Without affinity, that event costs a handful of in-flight requests. With affinity plus in-memory sessions, it costs one sixth of all logged-in users their session, simultaneously, with no error that says why.
This is why "affinity fixed our session bug" is a dangerous sentence. The bug was not fixed; its frequency was reduced and its blast radius was concentrated.
The right way to use it
Affinity used correctly sits on top of a service that does not need it. Sessions live in a shared store; per-user caches live in instance memory as an optimisation; the pin improves hit rate and its loss costs a cache miss.
The comparison below is the difference between affinity as an optimisation and affinity as load-bearing structure. The code differs by one fallback path, and that path is the whole design.
// Session lives only in this process.
const sessions = new Map<string, Session>()
app.use((req, res, next) => {
const s = sessions.get(req.cookies.sid)
if (!s) return res.status(401).send() // <-- wrong instance
req.session = s // = logged out
next()
})
// Requires the pin to hold. It will not:
// deploys, scale-down, node drains, crashes.
// When instance C dies, 1/6 of users are
// logged out simultaneously.// Shared store is the source of truth.
// Local map is a per-instance cache.
const local = new LRU<string, Session>({ max: 10_000, ttl: 60_000 })
app.use(async (req, res, next) => {
const sid = req.cookies.sid
let s = local.get(sid) // fast path: pinned
if (!s) {
s = await store.get(sid) // slow path: any instance
if (!s) return res.status(401).send()
local.set(sid, s)
}
req.session = s
next()
})
// Losing the pin costs one lookup, not a session.The second version is correct with affinity disabled, with an instance killed, and mid-deploy — affinity only decides whether the lookup is a memory read or a network call. That is what makes it an optimisation: removing it degrades latency and cannot change behaviour. The first version encodes a routing preference as a correctness requirement, and routing preferences are not honoured during the exact events where correctness matters most.
How to build it
Most important first.
- Use affinity as a performance optimisation over correct stateless behaviour, never as the mechanism that makes behaviour correct. The test: if affinity were disabled right now, would anything be wrong — or merely slower?
- Fix the state first. Sessions to a shared store or tokens, then affinity on top if there is a measured locality benefit (Where Sessions Live).
- Prefer consistent hashing where affinity exists for cache locality, so that adding or removing an instance remaps a fraction of keys rather than all of them (Local vs Distributed Cache).
- Bound the affinity lifetime. A cookie that pins for a session is reasonable; one that pins for weeks guarantees the fleet can never rebalance.
- For long-lived connections, add a maximum lifetime and make the client reconnect. Reconnection is the rebalancing mechanism, and it needs to be able to resume (Idempotency in Backends).
- Design the client for re-establishment: a lost pin should mean a reconnect and a resume, not an error the user sees.
- Measure per-instance load distribution continuously. Affinity plus autoscaling produces skew that no algorithm will correct on its own.
What can go wrong
- Instance loss taking a whole cohort of users with it — the concentrated version of the failure affinity was hiding.
- Uneven load that persists indefinitely, because pinned users never move and only new users can be routed to new instances.
- Deploys becoming session-loss events, since a rolling deploy replaces every instance in turn.
- IP-hash affinity broken by clients behind a large NAT (everyone hashes to one instance) or by mobile clients changing network (the pin moves constantly).
- Affinity silently masking a statelessness defect until an unrelated incident makes it visible under the worst conditions.
- Autoscaling that scales up while the hot instances stay hot, so the metric improves in aggregate and nothing improves for pinned users.
- A pinned instance can be terminated while a request is in flight, so the client's retry lands on an instance with none of the accumulated context.
- During a rolling deploy, a user's consecutive requests can be served by two different versions after their pin is broken and re-established (Rolling Deployments).
- A reconnecting real-time client can briefly hold two connections on two instances — the old one not yet torn down and the new one established.
- The affinity cookie identifies an instance and should carry no other meaning; if it is guessable or manipulable, it lets a caller select which instance serves them, which is a targeting primitive (The Trust Boundary).
- Session data held in instance memory is not covered by a central revocation. A logout must invalidate in the shared store, or a pinned instance can keep honouring the session (Where Sessions Live).
- IP-based affinity leaks a little about network topology and behaves badly for privacy-preserving clients; it is the weakest of the mechanisms in every respect.
- "Sticky sessions make in-memory state safe." They make it survive routine load balancing. They do not make it survive instance death, deploys, or scale-down — which happen constantly.
- "Affinity is a guarantee." It is a preference. The instance can vanish mid-session and the request will be served elsewhere.
- "We need affinity because we have WebSockets." The connection is inherently pinned; that is a reason to make the connection's state recoverable, not a reason to enable HTTP affinity.
- "Affinity gives better performance." It gives better *locality*, which usually helps and can hurt: a skewed fleet has worse tail latency than a balanced one (Tail Latency: Why p50 Being Fine Does Not Help in Observability & Performance).
Operating it
- Plot request rate per instance. With affinity enabled, sustained skew is expected and its magnitude is the thing to watch.
- Alert on session-loss or re-authentication rate; a spike aligned with a deploy or scale-down means state is still instance-local.
- Track per-instance cache hit rate — the benefit affinity is supposed to buy. If it has not improved, affinity is pure cost.
- Record which instance served each request so a user-specific problem can be tied to an instance (Structured Logging).
- Skew grows with fleet size: the larger the fleet, the more visible the difference between instances that accumulated pinned users and those that did not.
- With autoscaling, affinity and scale-up fight each other — new capacity only helps new arrivals, so a load spike from existing users is not relieved by adding instances.
- For very large connection counts (real-time services), affinity is unavoidable and the design shifts to making reconnection cheap and state recoverable rather than to avoiding the pin.
- Cache locality and warm per-user state are genuine wins, and they cost failover behaviour, rebalancing and clean scale-down.
- Affinity as a stopgap is a legitimate emergency measure and becomes a liability the moment it is treated as the solution.
- Consistent hashing reduces remapping on fleet changes and adds a concept the whole team has to understand when debugging routing.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALThe trade — locality against failover and rebalancing — holds for every affinity mechanism.
- CLOUD-SPECIFICManaged balancers differ in what they support: some offer cookie-based affinity with a configurable duration, others only client-IP hashing, and Kubernetes Services offer
sessionAffinity: ClientIPwith a timeout but no cookie option — which is the weakest form and the one most affected by NAT. - PROTOCOL-SPECIFICWebSocket and SSE connections are pinned by construction for the connection's lifetime, so the design question is reconnection and resume rather than routing policy.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.