WebSockets

A WebSocket (RFC 6455) begins as an ordinary HTTP/1.1 GET to make it pass through everything that already understands HTTP — the same port 80/443, the same TLS, the same cookies. The request carries Upgrade: websocket, Connection: Upgrade, a random Sec-WebSocket-Key, and optionally the subprotocols and extensions it wants. A server that agrees answers `101 Switching Protocols` with Sec-WebSocket-Accept = base64(SHA-1(key + a fixed GUID)) — not a security measure, but proof that the responder actually understood the WebSocket handshake rather than being an HTTP cache echoing headers. From the end of that response, the TCP connection no longer carries HTTP: it carries WebSocket frames until one side sends a close frame.

The upgrade is HTTP/1.1-specific because Upgrade is a hop-by-hop header forbidden in HTTP/2; RFC 8441 defines an extended CONNECT method that tunnels a WebSocket inside one HTTP/2 stream, supported by some browsers and proxies but not universally, so most WebSocket deployments still run on HTTP/1.1 connections. Browser WebSocket objects cannot set custom headers, so authentication is a cookie, a token in the URL query, or a first message after connect — Authorization headers do not exist here.

TCP gives a byte stream; the first thing every protocol on top of it does is invent message boundaries, and WebSocket does it once for everyone. Each frame has a 2–14-byte header: a FIN bit (last fragment of a message), an opcode (text 0x1, binary 0x2, close 0x8, ping 0x9, pong 0xA, continuation 0x0), a mask bit, and a payload length encoded in 7, 16 or 64 bits. Large messages may be fragmented across frames; the receiver reassembles by FIN. The application sees onmessage with a complete text or binary message — no length prefixes, no delimiters, no partial reads.

Masking is the odd rule: every frame from client to server must be XORed with a random 4-byte key carried in the frame header; server-to-client frames must not be. It protects not the data but the internet: without it, a malicious page could craft a WebSocket payload that looks like an HTTP request and poison a badly written proxy’s cache for other users. Masking makes the bytes on the wire unpredictable to the page author. Servers reject unmasked client frames; libraries handle it invisibly.

Ping/pong control frames give liveness: either side sends a ping, the other must answer with a pong carrying the same payload. Browsers do not expose pings to JavaScript, so applications add their own heartbeat message. What WebSocket does *not* give is request/response: messages have no ids and no correlation, so an application that needs "send a query, get its answer" must build ids, timeouts and matching itself — or use a protocol layered on top (STOMP, GraphQL over WebSocket, Socket.IO, JSON-RPC).

A WebSocket is a long-lived connection that carries no HTTP after the first request, and every piece of HTTP infrastructure between the browser and the server has to be told. Reverse proxies must forward the upgrade: nginx needs proxy_http_version 1.1, proxy_set_header Upgrade $http_upgrade and Connection "upgrade", or the client gets a 200 with an HTML page instead of a 101 (Forward and Reverse Proxies). Load balancers must either work at L4 or support upgrade at L7; either way the connection is pinned to one backend for its lifetime, so balancing happens at connect time only — a backend added during scale-out receives no existing connections, and a backend removed during a deploy drops all of its connections at once, producing a reconnect storm against the survivors (Load Balancers: L4 vs L7).

Idle timeouts apply throughout: an ALB closes connections idle for 60 s, nginx after proxy_read_timeout (60 s), NATs after minutes. A quiet WebSocket is closed by whichever timer fires first, and the browser sees onclose with code 1006. The fix is a heartbeat below the shortest timeout in the path, and a client reconnect loop with exponential backoff and jitter — every serious WebSocket client is really a reconnect state machine.

Fan-out and state are the scaling problem. The server that holds a user’s connection is the only one that can send to it. When a message must reach 10,000 users spread across 20 servers, the servers need a shared bus — Redis pub/sub, NATS, Kafka — so that a publish on one node reaches the node holding each subscriber. Presence ("who is online") and per-user routing therefore live in shared state, not in process memory, and a server holding 50,000 connections is a C10K: Ten Thousand Connections, Then a Million problem: one event loop, non-blocking I/O, and a bufferedAmount check before sending so a slow client cannot exhaust memory (What Happens When the Receiver Is Slow).