Bandwidth vs Latency

Bandwidth is capacity: bits per second the link can carry. Latency is delay: the time for a bit to travel from one end to the other (one-way) or there and back (RTT). They are set by different things — bandwidth by the link technology and the narrowest hop on the path, latency by distance and queueing — and neither implies the other. A satellite link has high bandwidth and 600 ms RTT; a loopback interface has modest bandwidth and 20 µs RTT. A truck full of hard drives has enormous bandwidth and a latency of days.

The practical consequence: a workload of small request/response exchanges is bounded by latency and barely notices bandwidth, because each exchange waits a full RTT and sends a few hundred bytes. A bulk transfer is bounded by bandwidth — once it gets going. Upgrading a link from 100 Mbit/s to 10 Gbit/s makes the API feel exactly as slow, because a 1 kB request never used more than a sliver of either.

The first-order model: the first byte arrives after one latency, and the rest stream in at the bandwidth. Time to move S bytes ≈ RTT + S / B (plus, in reality, the slow-start ramp described below). The two terms cross over at S = RTT × B — below that size latency dominates, above it bandwidth does.

Worked examples make the crossover visible. A 2 kB API response at 100 ms RTT on 1 Gbit/s: 100 ms + 0.016 ms — latency is 99.98 % of the time; the link speed is irrelevant. A 1 GB backup on the same link: 100 ms + 8 s — bandwidth is everything, and moving the peers closer saves nothing. A 1 MB web page sits near the crossover and is sensitive to both, which is why web performance work argues about both RTT reduction and compression.

TCP only sends as much data as the receiver has advertised it can hold (Flow Control: The Receive Window) and as much as its congestion window allows (Congestion Control: Protecting the Network), then waits for acknowledgements — which take one RTT to come back. To keep a link full, the sender must have a full RTT’s worth of bytes in flight: that quantity is the bandwidth-delay product, BDP = bandwidth × RTT. A 10 Gbit/s link at 100 ms RTT holds 1 Gbit = 125 MB in flight. A 1 Gbit/s link at 100 ms holds 12.5 MB; at 1 ms, 125 kB.

If the window is smaller than the BDP, the sender fills it, stops, and idles until the first ACK returns; throughput is capped at window / RTT no matter what the link can do. The original TCP window field is 16 bits — 64 kB — which at 100 ms RTT caps a connection at 64 kB / 0.1 s = 5.2 Mbit/s. Window scaling (RFC 7323) raises the limit to 1 GB, and modern kernels autotune buffers, but only up to a configured maximum: Linux’s default net.ipv4.tcp_rmem maximum is 6 MB, which at 100 ms gives 6 MB / 0.1 s = 60 MB/s ≈ 480 Mbit/s — the number from the problem statement, on a link fifty times wider. Raising tcp_rmem/tcp_wmem maxima to the BDP, or using several connections in parallel (what download managers, aria2 and browsers do), is the fix.

Slow start compounds this on short transfers: cwnd begins around 10 segments (~14.6 kB) and doubles per RTT, so reaching a 12.5 MB window takes ~10 RTTs — a full second at 100 ms — during which the link is mostly idle. A transfer that finishes inside slow start never sees the bandwidth at all.

Latency-bound workloads: cut round trips. Reuse connections (Keep-Alive and Connection Reuse, Connection Pooling), batch and pipeline requests, parallelise independent calls, move the peer closer (CDNs: The Networking View, regional deployment), use TLS 1.3 and session resumption to shave handshake trips. Compression barely helps: 2 kB or 1 kB is the same one RTT.

Bandwidth-bound workloads: size the TCP window to the BDP (buffer maxima on both ends), use parallel connections when a single one cannot be tuned, compress if the CPU is cheaper than the link, and remember that the bottleneck is the narrowest hop on the path, not the fastest NIC in the rack. And measure which one you are: a transfer that plateaus at a round number well below link speed on a long-RTT path is a window problem; one that plateaus at link speed is done.