DMA: Moving Bytes Without the CPU
A disk read does not consume a core, because the CPU never touches the bytes. It writes a descriptor, the device masters the bus and writes straight into RAM, and the CPU finds out afterwards.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
Programmed I/O and its replacement
The naive mechanism is programmed I/O: the CPU issues a load from a device register, gets a word, stores it to memory, and repeats. It works, it is simple, and it makes data transfer cost CPU time proportional to data volume. For a keyboard that is fine. For a network card at ten gigabits per second it is absurd — the core would do nothing else.
DMA inverts the arrangement. The CPU describes the transfer once — source, destination, length — and the device performs it, arbitrating for the interconnect and writing into memory on its own. The core is free the entire time. This is why copying a large file consumes almost no CPU while saturating a disk, and why "the CPU is idle but the transfer is slow" is a coherent and common state.
The cost moved rather than vanished. Descriptor setup is per-transfer overhead, which is why one large transfer beats many small ones by a wide margin: the same fixed cost is amortised over more bytes. That is the same shape as the interrupt argument, and it is why batching appears everywhere in I/O paths.
The NIC receive path, concretely
The clearest instance is a packet arriving. The driver has, in advance, filled a ring of descriptors each pointing at an empty buffer in RAM — the card must never have to ask permission, because packets arrive whether or not anyone is ready. When a packet lands, the card takes the next free descriptor, DMAs the bytes into the buffer it names, marks the descriptor used, and then signals: an interrupt if armed, or a status flag if the driver is polling.
Only then does a core get involved, and what it processes is a packet that is *already in memory*. The core's work is protocol handling, not byte movement. This is the mechanism that makes line-rate networking possible at all, and it is why the receive path is described in terms of descriptor rings and buffer refill rather than reads.
It also explains a failure mode that looks mysterious from above: if the driver does not refill the ring fast enough, the card runs out of descriptors and drops packets — on a machine whose CPU is not saturated and whose network link is not saturated. The bottleneck is the ring, and nothing in application code refers to it.
descriptor ring (in RAM, shared with the card)
+--------+--------+--------+--------+--------+--------+
idx | 0 | 1 | 2 | 3 | 4 | 5 |
owner | DRIVER | DRIVER | CARD | CARD | CARD | CARD |
state | filled | filled | empty | empty | empty | empty |
+--------+--------+--------+--------+--------+--------+
^ ^
| |
driver processes card DMAs the next
these, then arriving packet here,
refills them then advances
packet arrives -> card takes idx 2, DMAs bytes into the buffer it points at
-> marks it filled, hands ownership back to the driver
-> raises an interrupt, or sets a flag a polling driver reads
ring exhausted (driver too slow to refill) -> card has nowhere to put packets
-> drops, while CPU and link both
look far from saturatedTwo things DMA breaks, and how the machine patches them
A device writing into RAM is writing into memory that cores may have cached. If a core holds a stale copy of a line the device just overwrote, it will read the stale value — the device did not participate in the cache coherence protocol the cores use among themselves. On coherent interconnects the hardware handles this: DMA writes snoop the caches and invalidate. On non-coherent platforms, common in embedded systems, the driver must explicitly invalidate before reading and flush before writing, and getting it wrong produces data corruption that is maddening to debug.
The second problem is addressing. Your program has virtual addresses; the device needs physical ones, and the buffer must stay resident and physically contiguous — or be described in scatter-gather form — for the duration. This is why DMA-capable buffers are allocated through special interfaces and pinned, and why "just pass a pointer" is not available at this layer. On systems with an IOMMU the device gets its own translation layer, which restores both isolation and the ability to use non-contiguous memory.
Both patches are invisible from application code and both are places where the platform genuinely differs, which makes this a §224 hot spot: nothing in this section is safe to assume without knowing the machine.
| Requirement | Coherent platform with IOMMU | Non-coherent platform |
|---|---|---|
| Cache consistency | Hardware snoops and invalidates automatically | Driver must invalidate before read, flush before write |
| Address translation | IOMMU translates device addresses | Driver supplies physical addresses directly |
| Buffer residency | Pinned for the transfer; IOMMU allows scattered pages | Pinned and often physically contiguous |
| Isolation from a faulty device | IOMMU confines it to mapped regions | A device can write anywhere in memory |
Key points
- The CPU describes a transfer; the device performs it. No core touches the bytes, which is why large I/O consumes almost no CPU.
- Per-transfer descriptor overhead is fixed, so one large transfer beats many small ones for the same total bytes.
- The NIC receive path is a descriptor ring the driver refills in advance; exhausting it drops packets with CPU and link both idle.
- Devices do not participate in cache coherence the way cores do — coherent platforms snoop, non-coherent ones need explicit maintenance.
- Devices address physical memory, so DMA buffers are pinned and translated, with an IOMMU providing isolation where present.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1CPU → descriptor ring: the driver writes source, destination and length into a descriptor in memory the device can read.
- 2CPU → device register: a doorbell write tells the device work is available, and the core moves on to something else.
- 3Device → interconnect → RAM: the device arbitrates for the bus and writes the payload directly into the named buffer.
- 4Device → coherence fabric: on a coherent platform the writes invalidate any cached copies the cores hold; otherwise the driver must do it.
- 5Device → CPU: completion is signalled by interrupt or by a status flag the driver polls, and only now does a core touch the data.
- • "The CPU is idle so the transfer must be finished." DMA runs with the core idle by design; idleness says nothing about transfer state.
- • "Throughput is low and the CPU is free, so the device is slow." The descriptor ring, the interconnect or the transfer size may be the limit instead.
- • "DMA means zero copy." DMA removes the CPU from the *device-to-memory* move. Whether the data is then copied again is a separate question about the software path.
- • "Cache coherence handles everything." It does on many platforms and does not on many others, and the failure mode when it does not is silent corruption.
Consequences, controls and cost
- • Bulk transfers saturate a link or a disk while CPU utilisation stays near zero — an entirely normal state that looks like a stall.
- • Many small transfers perform far worse than their byte count suggests, because descriptor overhead dominates.
- • Packet drops can occur with spare CPU and spare bandwidth, when the descriptor ring is the thing that ran out.
- • On non-coherent platforms, missing cache maintenance produces intermittent corruption rather than a clean failure.
- • Batch: fewer, larger transfers amortise the fixed descriptor and completion cost over more bytes.
- • Size receive rings and refill promptly so the device never runs out of buffers under burst.
- • Use the platform's DMA-buffer allocation interfaces rather than ordinary allocations, so pinning and translation are handled correctly.
- • Where the API supports it, prefer paths that avoid an extra CPU copy after DMA — the point of DMA is undone by copying the buffer again.
- • Bytes transferred per transfer, not just total throughput — a small average transfer size points at descriptor overhead.
- • Device-reported drop or overrun counters, which distinguish "ring exhausted" from "link saturated".
- • CPU cycles attributed to the I/O path per megabyte moved; a high figure suggests a copy that DMA was supposed to eliminate.
- • Interrupt or completion rate against transfer count, to see how much batching the path is actually achieving.
- • Large transfers amortise overhead but add latency for the first byte and need larger pinned buffers.
- • Pinned memory cannot be paged out, so a generous ring reserves physical memory that nothing else may use.
- • An IOMMU buys isolation and addressing flexibility at the cost of a translation step on the device's accesses.
Scope
§224 — what these claims are specific to.
- GENERALDescriptor-based transfer with the device as bus master, and completion by interrupt or polled flag, is the common design across storage, network and accelerator devices.
- PLATFORM-SPECIFICCache coherence for device writes, IOMMU presence, and whether physically contiguous buffers are required all differ by platform — server x86-64/AArch64 versus embedded is the sharpest divide.