The Storage Path: Why One Small Read Is the Worst Case
An SSD is a parallel device pretending to be a disk. Give it one request at a time and you measure its latency; give it many and you measure its throughput — and those two numbers are not related the way rotational intuition expects.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The path, and which parts are hardware
A read passes through several layers before any hardware is involved: the application, the system call boundary, the filesystem translating an offset into block addresses, a block layer that may merge and schedule requests, and a driver that builds a command. Only then does the request reach the controller, and only then does anything physical happen.
This domain's concern is the last part, but the layers above matter because they decide *how many* requests are in flight when it gets there. The OS layers own caching, merging and scheduling, and the operating systems material covers them properly — what belongs here is what the device does with what it is given.
What it does is exploit parallelism. An SSD contains many independent channels, each with multiple dies, each able to service an operation concurrently. The controller's job is to spread outstanding requests across them. With one request outstanding, one path is busy and the rest are idle. This is the single most important fact about modern storage performance and it is invisible from any read call.
Queue depth is the whole story
Queue depth is how many requests are outstanding at the device simultaneously. At depth one, latency is what you measure and it is the device's service time for a single operation — a number that has improved far less across SSD generations than throughput has. At higher depths, the controller overlaps operations across its channels and aggregate throughput rises, often close to linearly, until the channels are saturated.
Which means the same device produces wildly different numbers depending only on how you ask. A benchmark at depth 1 and a benchmark at depth 32 on identical hardware can differ by an order of magnitude in throughput while individual latency barely moves. Neither number is wrong; they answer different questions, and quoting one as "the device's speed" is how storage benchmarks mislead.
The corollary for application design is direct: a loop that reads, processes, reads, processes runs the device at depth one no matter how fast the storage is. To use the hardware you must have multiple requests in flight — through asynchronous I/O, threads, readahead, or batching — and none of those change how much data you read, only how many requests are outstanding while you wait.
| Access pattern | Queue depth | What you measure | Device parallelism used |
|---|---|---|---|
| Synchronous small reads in a loop | 1 | Per-operation latency | A small fraction |
| Same reads, issued asynchronously | Many | Aggregate throughput | Most or all |
| One large sequential read | Effectively many | Near peak bandwidth | Most — the layers split it |
| Random reads, many threads | Many | High IOPS | Most — flash has no seek penalty |
| Random reads, one thread | 1 | Latency, repeatedly | A small fraction |
What flash inverted
On rotational media the dominant cost was mechanical: moving a head and waiting for the platter. Sequential access was enormously cheaper than random, and concurrency created contention for a single physical mechanism. Every piece of storage folklore — sort your accesses, read sequentially, avoid concurrent readers — follows from that.
Flash removed the mechanism. There is no seek, so random access is far closer to sequential than it used to be. And there is real internal parallelism, so concurrent requests help rather than hurt. Two of the three folk rules inverted, while the third — larger requests amortise fixed overhead better — survived, because per-request overhead exists at every layer regardless of media.
What did *not* invert is that a single small synchronous read is the worst case. It pays every layer's fixed cost, uses a fraction of the device, and gives the system no opportunity to overlap anything. The relative scale below makes the ordering concrete: the gap between a page-cache hit and a device read is what makes caching decisive, and the gap between depth-1 and deep-queue operation is what makes asynchrony decisive.
Key points
- An SSD is internally parallel; a single outstanding request uses a fraction of it.
- Queue depth, not request size alone, determines whether you measure latency or throughput.
- A read-process-read loop runs the device at depth one regardless of how fast the storage is.
- Flash inverted two storage folk rules: random is no longer catastrophic, and concurrency now helps.
- One small synchronous read remains the worst case — it pays every fixed cost and overlaps nothing.
Where the Data Is
Change an input and watch which number moves — and which one refuses to.
The exact ratios vary by machine and the absolute times vary far more, which is why none are shown. What is stable enough to build intuition on is the shape: each level is several times the one above, and the gap between the last cache level and memory is the one that decides most program performance.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Application → system call: the request crosses the privilege boundary and enters the kernel's I/O path.
- 2Filesystem → block layer: the file offset becomes block addresses, which may be merged or reordered with other pending requests.
- 3Driver → controller: a command is placed on a submission queue and a doorbell write tells the device to look.
- 4Controller → channels: the device spreads outstanding commands across independent channels and dies, which is where parallelism is won or lost.
- 5Device → RAM → completion: data is DMAed into memory and completion is signalled, at which point a core touches the data for the first time.
- • "Storage is slow, so we should read less." Often the problem is not volume but depth — the same bytes in fewer, deeper-queued requests can be dramatically faster.
- • "Concurrent readers will thrash the disk." True of rotational media; on flash concurrency is how the device reaches its capability.
- • "The device is rated at N IOPS, so we should see N." That rating assumes a queue depth your synchronous code never reaches.
- • "Random access is catastrophic." It was, mechanically. On flash the gap to sequential is much smaller, though not zero.
Consequences, controls and cost
- • Synchronous per-record reads achieve a small fraction of a device's rated throughput no matter how fast it is.
- • Benchmarks at different queue depths on identical hardware differ by an order of magnitude, and both are honest.
- • Storage advice inherited from rotational media leads to designs that under-use flash, particularly the avoidance of concurrent readers.
- • Adding threads to an I/O-bound workload can improve throughput substantially, which contradicts the intuition that they will contend.
- • Keep multiple requests in flight — asynchronous I/O, a thread pool, readahead or explicit batching all achieve it.
- • Prefer fewer larger requests to many small ones, so per-request overhead at every layer is amortised.
- • Let the page cache do its job for repeated reads rather than bypassing it, unless you genuinely have a better caching policy.
- • Measure at the queue depth your application actually generates, not at whatever depth the benchmark defaults to.
- • Average queue depth at the device during the workload — the single most diagnostic storage number, and usually the surprising one.
- • Achieved IOPS and bandwidth against the device rating *at the depth you actually generate*.
- • Average request size, which reveals whether per-request overhead is dominating.
- • Page cache hit rate, to separate "the device is slow" from "we are reaching the device at all when we need not".
- • Deeper queues raise throughput and raise individual request latency, because requests wait behind others.
- • Asynchronous I/O uses the hardware properly and makes application control flow substantially more complex.
- • Large requests amortise overhead and increase the latency of the first byte, which matters for interactive paths.
Scope
§224 — what these claims are specific to.
- PLATFORM-SPECIFICChannel counts, internal parallelism and controller behaviour differ per device and per class — consumer SSD, enterprise NVMe and cloud block storage behave differently at the same queue depth.
- SIMPLIFIEDOmits the flash translation layer, garbage collection, write amplification and the write cliff, all of which make write behaviour considerably more complicated than the read path described here.