What Is Actually Inside a CPU

A useful first model of a core has four regions. The front end turns bytes in memory into operations the machine can schedule: it fetches from the instruction cache, decodes, and predicts where control flow goes next. The back end executes: a scheduler picks operations whose inputs are ready and issues them to execution units — integer ALUs, load/store units, vector units, and usually more than one of each. The register file is the working storage those units read and write. The cache hierarchy sits behind everything, because main memory is far enough away that a core would otherwise spend most of its life waiting.

What is worth noticing is the proportion. The arithmetic — the part that corresponds to a + b in source — is a small fraction of the machine. Most of the transistor budget goes to *keeping the arithmetic fed*: caches, prediction, buffering, scheduling. That proportion is the entire reason this domain exists. If arithmetic were the bottleneck, performance work would be about counting operations. It usually is not, so it usually is not.

The split between front end and back end is the first diagnostic distinction to learn. A core can be starved at the front (it cannot supply operations fast enough — instruction cache misses, mispredictions, decode limits) or stalled at the back (it has operations but cannot complete them — waiting for data, waiting for a busy unit, waiting on a dependency). These are different problems with different fixes, and Busy Is Not the Same as Working is where that split gets turned into a measurement.

Because the two halves fail differently, the same symptom — "the CPU is busy but slow" — has two completely different explanations, and the counters distinguish them cleanly. A front-end problem means the machine cannot get operations to the scheduler: the code footprint is too large for the instruction cache, or branches are unpredictable enough that the fetcher keeps being redirected. A back-end problem means operations are queued and cannot retire: they are waiting for a load, for a divider, or for each other.

This is not an academic split. The fixes are disjoint. Front-end pressure is addressed by making code smaller and more predictable — less inlining, fewer branches on unpredictable data, hot paths kept together (Your Code Is Data Too, Branch Prediction: Guessing Well Enough to Matter). Back-end pressure is addressed by making data closer or dependency chains shorter (Memory Moves in Lines, Not Variables, Dependency Graphs: The Real Shape of Your Code). Applying the wrong one is how a week disappears with nothing to show.

A learner should leave this lesson able to ask, of any slow loop: is the machine failing to *supply* work, or failing to *finish* it? That question is answerable with counters and unanswerable by reading source code, which is the recurring shape of this whole domain.

Every block diagram of a CPU is a lie of omission, and it is worth naming the omissions rather than discovering them later as contradictions. The model above shows one of each unit; real cores have several integer ALUs and can issue multiple operations per cycle (Superscalar Execution). It shows instructions flowing in order; real cores execute them as inputs become available and only *retire* them in order (Out-of-Order Execution, The Reorder Buffer and Precise State). It shows a register file with the registers the ISA defines; real cores have many more physical registers and rename onto them (Register Renaming).

It also omits everything about how bytes become operations on a variable-length ISA, where a single instruction may decode into several internal operations, and where a cache of already-decoded operations can bypass the decoders entirely. That machinery is real and it is microarchitectural — meaning it differs between vendors, between generations from the same vendor, and sometimes between cores on the same die.

The reason to state this explicitly is that these diagrams are how misconceptions get installed. A learner who believes the picture is the machine will later conclude that a modern CPU executes one instruction at a time, in order, and that its speed is its clock rate. All three are wrong, and ISA vs Microarchitecture: The Distinction Everything Depends On and The Clock: Why GHz Is Not Performance exist to unpick exactly those.