The TLB and application performance

With 4-level page tables, translating one virtual address by walking the table costs four dependent memory reads. If every load and store did that, memory traffic would multiply by five. The Translation Lookaside Buffer caches recent translations so that the common case is a hit in a few cycles; the page walk happens only on a miss (The TLB).

The numbers set the scale. A core has on the order of 64 L1 dTLB entries and ~1,500–2,000 second-level entries. With 4 KB pages that is a reach of roughly 6–8 MB: touch more than that with poor locality — a large hash table, a graph traversal, a JVM heap of many gigabytes, a column store scanning randomly — and translations miss constantly. A miss served by the hardware walker costs tens of cycles if the page-table lines are in cache, and memory latency (~100 ns) per level if they are not. Programs can spend 10–30% of their time in page walks without any of it showing in a profiler as a function (Page Tables).

The lever is page size: a 2 MB huge page covers 512× the memory per entry, so the same TLB reaches gigabytes. Linux offers transparent huge pages and explicit hugetlbfs/madvise(MADV_HUGEPAGE); databases, JVMs and hypervisors use them deliberately (label: Linux). The trade-offs are internal fragmentation, latency spikes from background compaction and promotion, and RSS growth for sparse heaps — which is why THP is on for some workloads and explicitly off for others.

Context switches interact: switching address spaces invalidates the TLB unless entries are tagged with a process id (PCID on x86, ASID on ARM). Even with tagging, the new process’s working set competes for the same entries, which is part of the indirect cost of a switch (Context Switching).