Every Address Your Program Uses Is Fake
A pointer is not a memory location. It is an index the CPU must translate before anything can be read, and that translation happens on every load and every store, in hardware, before the access can even begin.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
mmap is not a copy — is a consequence of this translation layer. It is invisible in source code and unavoidable in execution.Two address spaces, one translation
The address in your pointer is meaningful only inside your process. The CPU cannot put it on the memory bus. Between the execution unit and physical memory sits the memory management unit, which converts the virtual address to a physical one on every access — not once at load time, not at allocation, but every time.
That translation is what makes two things possible at once. Isolation: process A's address 0x7fff_0000 and process B's 0x7fff_0000 map to different physical frames, so neither can name the other's memory even by accident. And indirection: physical memory can be reorganised, shared, paged out or copied lazily underneath a process without a single pointer in that process changing value.
The picture below is the whole module in one diagram. Everything else here is a detail of one arrow: how the MMU finds the mapping (The Page-Table Walk: Dependent Loads All the Way Down), how it avoids doing that work repeatedly (The TLB: A Cache for Addresses, Not Data), and what it checks while it is there (What Actually Stops One Process Reading Another's Memory).
The OS builds the map; the hardware reads it
This is the division that the rest of the module depends on, and it is the one most often blurred. The operating system decides *what* is mapped where: it allocates frames, constructs the page tables, chooses what to evict, and handles a fault when one is raised. The hardware performs the translation on every access, caches recent translations, enforces the permission bits, and raises the fault — but it decides nothing.
The consequence is that neither side can be reasoned about alone. "Why was my first write to this buffer slow?" is an OS question (the mapping did not exist yet, so a fault was raised and the OS supplied a frame) with a hardware mechanism (the MMU found no valid entry and trapped). "Why is my sparse traversal stalling?" is a hardware question (When Translation Itself Is the Bottleneck) about a structure the OS populated.
The Operating Systems domain owns the policy half in depth — Why Virtual Memory?, The Virtual Address Space, Page Faults and Page Tables there are about what the OS does and why. This module never repeats them; it explains what the silicon does with what the OS built.
| Concern | Operating system | Hardware (MMU) |
|---|---|---|
| Deciding what is mapped | Allocates frames, builds and edits page tables | Nothing — it only reads them |
| Performing translation | Nothing on the fast path | Every load and store, before the access |
| Caching translations | May hint (huge pages, prefault) | Owns the TLB entirely |
| Checking permissions | Sets the read/write/execute bits | Enforces them on every access |
| On a missing mapping | Handles the fault: allocate, page in, or kill | Raises the fault and stops the instruction |
| Address-space switch | Schedules it, points the CPU at new tables | Invalidates or tags stale translations |
It is on the critical path, which is why it is cached
Translation is not free and it is not off to the side. It sits between the address computation and the cache lookup, on the critical path of every single memory operation. If a translation required reading tables from DRAM each time, every load would cost several DRAM accesses before the data access even started, and the machine would be unusable.
So the hardware caches translations aggressively in the TLB, and the cost of a memory access splits into two very different cases: a translation hit, which is close to free and overlaps with the cache lookup, and a translation miss, which triggers a walk that is itself several dependent memory accesses.
The scale below is deliberately unitless. What transfers between machines is the ratio — that a TLB hit is cheap enough to ignore and a walk that misses in cache is not — never a nanosecond figure, which would be wrong on any machine other than the one it was measured on.
Key points
- A pointer holds a virtual address; the MMU translates it to a physical one on every load and store, in hardware.
- The OS decides what is mapped and handles faults; the hardware walks, caches, enforces and traps — it decides nothing.
- Translation sits on the critical path of every memory access, which is the entire reason a TLB exists.
- The same pointer value in two processes reaches different physical memory; that is what isolation actually is.
- A translation hit is close to free; a translation miss is a walk of several dependent memory accesses.
Progressive depth
Overview
Your pointers hold virtual addresses. Hardware converts each one to a physical address on every access, using tables the operating system built. This is what keeps processes from seeing each other's memory.
Practical
The conversion is cached in the TLB. A hit costs almost nothing; a miss costs a walk through several levels of tables, each of which is itself a memory access that may miss in cache. So the practical question is not "is translation slow" but "how many distinct pages does my hot loop touch".
Advanced
Translation and cache lookup are overlapped on most designs: the L1 lookup can begin using the page-offset bits, which do not change during translation, while the TLB resolves the frame number in parallel. This is why an L1 hit with a TLB hit costs roughly what an L1 hit costs, and it constrains L1 size and associativity in ways that are otherwise hard to explain.
Internals
The walk itself is performed by hardware page-walker units, and its intermediate results are cached in dedicated structures separate from the TLB, so a miss on one level need not re-read every level above it. Multiple walks may be in flight concurrently. Under virtualization, guest translation is composed with host translation, so a fully uncached walk multiplies out to substantially more dependent accesses — see What a vCPU Actually Is.
TLB Reach
Change an input and watch which number moves — and which one refuses to.
Only 3% of the working set can be mapped at once, so accesses outside that window trigger page-table walks — several dependent memory accesses each. Note this can happen while the data itself sits comfortably in cache, which is why the symptom is so often misread as a cache problem. Raise the page size and watch reach grow without adding a single entry.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Execution unit → address generation: the effective address is computed, e.g. base plus scaled index.
- 2Address generation → MMU: the virtual address is presented for translation before the access can proceed.
- 3MMU → TLB: the translation cache is consulted for a matching entry with adequate permissions.
- 4TLB → physical address: on a hit, the frame number is combined with the page offset and the cache access proceeds.
- 5TLB → walk → fault: on a miss the tables are walked; if no valid entry exists, or permissions forbid the access, the instruction is aborted and the OS is trapped into.
- • Concluding a pointer value identifies a physical location, and reasoning about aliasing or sharing on that basis.
- • Assuming translation is a startup cost rather than a per-access cost, and therefore ignorable in a hot loop.
- • Blaming the data cache for stalls that are actually translation stalls, because both look like "memory" in a coarse profile.
- • Reading "virtual memory" as "swapping to disk". Swapping is one policy the OS can implement on top of this hardware; the translation happens whether or not anything is ever swapped.
Consequences, controls and cost
- • Two processes holding identical pointer values access completely different memory, with no cooperation required.
- • The first touch of freshly allocated memory is often far slower than later touches, because the mapping is created on demand.
- • A program whose *data* fits comfortably in cache can still stall badly if its *pages* are scattered ([[tlb-misses]]).
- • Physical memory can be shared, deduplicated, moved or paged out without any pointer in the process changing.
- • Memory-access cost is bimodal in a way source code cannot express: the same line is cheap or expensive depending on translation state.
- • Keep the working set dense in *pages*, not just in bytes — sequential and blocked access patterns reuse translations as well as cache lines.
- • Prefer larger contiguous allocations over many scattered small ones, so fewer distinct pages are in play.
- • Consider [[huge-pages]] when a large working set is genuinely translation-bound, accepting the fragmentation cost.
- • Touch or pre-fault large buffers before a latency-sensitive phase if the first-touch cost would land in the wrong place.
- • Otherwise: almost nothing directly. Translation is not under program control — but it is measurable, and measuring it is what tells you whether any of the above is worth doing.
- • Read the TLB miss counters — `dTLB-load-misses` and `iTLB-load-misses` under `perf stat` on Linux — alongside cache misses, so the two are not conflated.
- • Compare data-cache miss rate against TLB miss rate: low cache misses with high TLB misses is the signature of a page-sparse access pattern.
- • Count page faults (`perf stat` reports minor and major separately); major faults mean I/O and belong to a different conversation than translation cost.
- • Run the same workload with and without huge pages enabled and diff the TLB counters — the delta attributes the cost directly.
- • Deep dives in [[counters]] and [[cpu-bound-vs-io-bound]] cover interpreting these readings against a broader picture.
- • Translation buys isolation, relocation and lazy allocation at the price of hardware on the critical path of every access.
- • The TLB makes it cheap on average and expensive in the tail, which converts a uniform cost into a bimodal one that is harder to reason about.
- • Reducing translation pressure usually means larger or more contiguous allocations, which costs memory footprint and allocator flexibility.
Scope
§224 — what these claims are specific to.
- GENERALPaged virtual memory with hardware translation is near-universal on application-class CPUs. Small embedded cores and some real-time targets have no MMU at all and run with physical addressing.
- MICROARCH-SPECIFICTLB sizes, walk-cache structures and the relative costs above vary substantially between vendors and even between generations of one vendor. The ordering transfers; the ratios do not.
Misconceptions
Where the rest of this lives
Whether an allocator returns contiguous memory or scattered blocks decides how many distinct pages your working set spans, and therefore how much translation pressure a program generates before any of its own code runs.