I/Ocpu cachepage cachedistinctionkernelmemory

CPU Cache Is Not the Page Cache

Both are called cache, both make things faster, both live in the machine. One is hardware holding lines of physical memory and you cannot address it; the other is ordinary RAM the kernel fills with file data and you can control it precisely.

Follow the mechanism

Software view, hardware view

The gap between what you wrote and what the machine does is where this whole domain lives.

The question
When someone says "it is in cache", which cache do they mean — and does the answer change what I should do about it?
What you wrote
There is a vague sense that recently used things are faster to reach, and that "cache" explains it. The word covers CPU caches, the page cache, the CDN and the application's own memoisation, all at once.
What the hardware does
Two mechanisms with nothing in common but the name. CPU caches are small hardware structures holding lines of physical memory, managed entirely by hardware, invisible to software. The page cache is ordinary DRAM that the kernel has chosen to fill with file contents, managed by software, and very much addressable.
Conflating them produces confident wrong diagnoses. "The data is cached so the read is fast" might mean an L1 hit costing a handful of cycles, or a page cache hit costing a memory copy — different by orders of magnitude and fixed by completely different actions. §224 makes this distinction mandatory precisely because the shared word hides how little they share.
Source→Compiler→Instructions→Front End→Execution→Registers→Caches→Memory→I/O→Behavior

Two things, one word

The CPU cache is hardware. It holds cache lines — fixed-size blocks of physical memory — in structures physically close to the execution units. No instruction addresses it; you cannot allocate in it, pin to it, or ask what is in it. Its capacity is measured in kilobytes and megabytes. Software influences it only indirectly, by choosing access patterns that happen to suit it, which is what Memory Moves in Lines, Not Variables and Spatial Locality are about.

The page cache is software. It is ordinary DRAM, allocated by the kernel, holding copies of file contents so that repeated reads need not reach storage. Its capacity is measured in gigabytes — often most of the machine's free memory. It has an API: you can advise it, bypass it, flush it, map into it, and observe its size in ordinary system tools.

The similarity is that both exploit reuse. Everything else differs: what they hold, who manages them, how big they are, whether software can address them, and what to do when they are not helping.

The two caches, compared on everything except the name
CPU cacheOS page cache
What it isDedicated hardware on the CPUOrdinary DRAM the kernel is using for file data
What it holdsCache lines of physical memoryPages of file contents
Managed byHardware, autonomouslyThe kernel, in software
Typical sizeKilobytes to a few megabytesGigabytes — often most of free RAM
Addressable by software?No — no instruction names itYes — advise, bypass, map, flush
Miss costs youA trip further down the memory hierarchyA trip to storage
How you influence itAccess patterns and data layoutExplicit APIs and read patterns
Visible in tooling asHardware miss countersMemory used for file caching

They stack, and both can miss

These are layers, not alternatives. A read of file data that the page cache holds is a copy from DRAM into your buffer — and that copy moves through the CPU cache like any other memory access. So a page cache hit still involves the CPU cache, and can itself hit or miss there. Four outcomes are possible, and they differ by orders of magnitude.

This is why "is it cached?" is an underspecified question. The useful version names the layer: is this data in the CPU cache, meaning the access costs a few cycles? In the page cache, meaning it costs a memory copy but no device I/O? Or on the device, meaning it costs a storage access? Only the third is I/O at all, and only the first two are what most people mean by fast.

It also explains an observation that confuses people: re-reading a file is enormously faster the second time, but re-reading a *small* structure you just built in memory is faster again by a wide margin. The first is the page cache eliminating device I/O. The second is the CPU cache eliminating a memory access. Different mechanism, different magnitude, same word.

One block of file data, by which layer actually satisfies it. Ratios only; magnitudes vary per machine. — 1 unit ≈ an L1 cache hitSIMPLIFIED
CPU cache hit×1
CPU cache miss, page cache hit×60
Page cache miss, local SSD×20000
Page cache miss, network storage×100000
Ratios, not times. Absolute latencies depend on the processor, its clock, the memory it is attached to and what else is running — publishing them would be wrong everywhere except one machine. The bars are log-scaled, so each step is larger than it looks.
CPU cache hit — the data is already in a cache line near the core
CPU cache miss, page cache hit — a copy from DRAM — no device involved
Page cache miss, local SSD — the first access that is genuinely I/O
Page cache miss, network storage — device latency plus a network round trip

Different problems, different fixes

Because the mechanisms differ, so do the remedies, and applying one to the other achieves nothing. If CPU cache misses are the problem, the fixes are about layout and access order: contiguous data, sequential traversal, smaller working sets, blocking. Nothing about file I/O is involved and no API exists to call.

If page cache misses are the problem, the fixes are about I/O behaviour: read patterns that let readahead work, keeping the working set small enough to stay resident, using hints where the access pattern is known, and *not* bypassing the cache unless you genuinely have a better policy — which databases often do, and applications usually do not.

The diagnostic question that separates them is simply whether the device was touched. If storage shows no activity and the code is still slow, the page cache is doing its job and the problem is elsewhere — possibly in the CPU cache, possibly not in memory at all. If storage is busy, the page cache is missing and no amount of data-layout work will help.

Diagnosing the wrong cache
1symptom: "processing this file is slow"
2 
3assumption: "cache misses"
4action:     restructure the in-memory data layout
5            for better locality
6 
7result:     no change.
8 
9why: storage was busy the whole time. every pass was
10     re-reading from the device because the file does
11     not fit in the page cache. the CPU cache was never
12     the constraint, so improving it changed nothing.
Establishing which layer first
1symptom: "processing this file is slow"
2 
3check:   is the device doing I/O during the slow part?
4 
5  yes -> page cache is missing.
6         fix: read patterns, working set size,
7              readahead hints, more RAM.
8 
9  no  -> data is already in RAM.
10         now ask the CPU cache question:
11         check miss counters, then fix layout
12         and access order.

One check — whether the device is active — separates two problems that share a word and share no solution. Doing it first costs seconds; skipping it costs however long the wrong optimisation takes.

Key points

  • The CPU cache is hardware holding lines of physical memory; the page cache is ordinary RAM holding file data.
  • One is unaddressable and influenced only through access patterns; the other has an explicit API.
  • They differ in size by roughly three orders of magnitude, and in miss cost by a similar factor.
  • They stack: a page cache hit is a memory copy, which itself hits or misses in the CPU cache.
  • Whether the device was touched is the one check that tells you which cache you are actually dealing with.

Follow the mechanism

The path through the machine, hop by hop — and the conclusions it invites that are wrong.

  1. 1
    Core → CPU cache: the access looks for the line in L1, then L2, then the last level, entirely in hardware.
  2. 2
    CPU cache miss → DRAM: the line is fetched from main memory; if that memory holds file data, it is a page cache hit.
  3. 3
    Page cache miss → filesystem → device: only now does storage I/O occur, costing orders of magnitude more.
  4. 4
    Device → DMA → DRAM: the data lands in a page the kernel adds to the page cache, so the next read need not repeat this.
  5. 5
    DRAM → user buffer: the copy to the application's buffer traverses the CPU cache like any other memory access.
What people conclude from this — wrongly
  • • "The file is cached, so reads are basically free." A page cache hit is a memory copy — cheap relative to storage, expensive relative to an L1 hit.
  • • "Most of my RAM is used by cache, so I am short of memory." Page cache is reclaimable on demand; it is memory being useful rather than memory being consumed.
  • • "We improved locality and it did not help, so the cache theory was wrong." It may have been the right theory about the wrong cache.
  • • "Bypassing the page cache avoids a copy, so it is faster." Only if you replace it with something better; otherwise it converts hits into device reads.

Consequences, controls and cost

What it causes
  • • "It is cached" is ambiguous by a factor of thousands, and the ambiguity routinely misdirects optimisation work.
  • • Layout optimisations aimed at CPU caches produce no improvement when the real constraint is page cache misses, and the reverse.
  • • Memory that appears "used" for file caching is reclaimable, which makes free-memory readings look alarming without cause.
  • • Bypassing the page cache helps only where the application genuinely has a better caching policy — which databases do and most services do not.
What you can do
  • • Establish which layer is missing before optimising: check whether the device is active during the slow phase.
  • • For CPU cache misses, change data layout and access order — see [[cache-lines]] and [[spatial-locality]].
  • • For page cache misses, change I/O patterns, reduce the working set, or give the kernel hints about the access pattern.
  • • Leave the page cache enabled unless you are implementing your own buffer pool with a policy you can defend.
How to see it
  • • Device activity during the slow phase — the single check that separates the two cases.
  • • Hardware cache miss counters for the CPU-cache question, which say nothing at all about file caching.
  • • Page cache hit behaviour: repeated reads of the same file should not produce device I/O.
  • • Memory used for file caching, and whether it is being reclaimed under pressure.
What it costs
  • • Relying on the page cache means the kernel's eviction policy governs your data, and it does not know your access pattern.
  • • Managing your own buffer pool gives control and duplicates work the kernel already does, wasting memory if done carelessly.
  • • Optimising for CPU cache locality often means restructuring data, which costs readability and flexibility for a benefit that only appears when that layer is the constraint.

Scope

§224 — what these claims are specific to.

What these claims are specific to
  • GENERALThe distinction holds on every general-purpose system: a hardware cache the software cannot address, and a software cache in ordinary memory that it can.
  • PLATFORM-SPECIFICPage cache APIs, eviction policy, hint mechanisms and how the memory is reported differ substantially between operating systems.

Misconceptions

Claim
“The page cache is a hardware cache like L1 and L2, just bigger.”
Reality
It is ordinary DRAM the kernel decided to fill with file data. There is no dedicated hardware, and software allocates and evicts it explicitly.
Claim
“A page cache hit means no memory access is needed.”
Reality
It means no *device* access is needed. The data still has to be copied from DRAM, which is a memory access like any other.
Claim
“High page cache usage means the machine is low on memory.”
Reality
It is reclaimable on demand. A machine with most of its RAM holding file data is a machine using its memory well.
← The Storage Path: Why One Small Read Is the Worst Case