Fetch, Decode, Execute — and Why That Story Is Incomplete
Every introduction to CPUs teaches a tidy four-step cycle. It is genuinely useful and it has not described a shipping processor since the early 1990s. Both halves of that sentence matter: learn the model, then learn precisely which parts of it modern hardware abandoned.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The four phases
Every instruction, on every processor, must logically do four things. Fetch: read the instruction bytes from memory at the address the program counter holds. Decode: work out what those bytes mean — which operation, which registers, which immediate values. Execute: perform the operation in a functional unit. Write back: make the result visible, in a register or in memory.
The phases are ordered by dependency, not by convention. You cannot decode bytes you have not fetched, and you cannot execute an operation you have not identified. That dependency ordering is the part that survives into modern hardware unchanged — everything else about the model is negotiable.
Notice what the model already tells you that source code does not. Fetching an instruction is a *memory access*, which means instructions compete for cache with data (Your Code Is Data Too). Decoding is real work that costs transistors and time. And the program counter is state the CPU must maintain and predict (The Program Counter: Deciding What Happens Next) — which is exactly the hook that Control Hazards: The CPU Does Not Know Where You Are Going pulls on.
What a modern CPU actually does instead
The textbook model says one instruction is in the machine at a time. A modern out-of-order core has on the order of hundreds of instructions in flight at once, spread across fetch, decode, rename, scheduling, execution and retirement — the exact number is a design parameter that differs between every microarchitecture and generation.
Three departures matter most. First, the phases overlap: while instruction 3 executes, instruction 4 decodes and instruction 5 is fetched. That is Pipelining: Throughput Without Making Anything Faster, and it is why throughput is not one instruction per cycle-count-of-the-phases. Second, instructions are not executed in program order — they execute when their inputs are ready (Out-of-Order Execution), and are only *retired* in order (The Reorder Buffer and Precise State). Third, many instructions are decoded into several smaller internal operations, so the count of machine instructions and the count of things the execution units actually do are different numbers.
The consequence for reasoning about code: instruction count is a weak predictor of time. A loop body with more instructions can run faster than one with fewer, if the longer one has more independent work the machine can overlap. This is the gap IPC: Instructions Per Cycle exists to expose and the reason Why Reading the Source Cannot Tell You the Cost is its own lesson.
| Aspect | Four-step model | Modern out-of-order core |
|---|---|---|
| Instructions in flight | One | Many — often hundreds |
| Execution order | Program order | Whenever operands are ready |
| Result visibility | Immediately at write back | At retirement, in program order |
| Unit of work | One machine instruction | One or more internal micro-operations |
| Time per instruction | Sum of the phases | Amortised — depends on what else is in flight |
| Prediction | None; the next address is known | Branch direction and target predicted ahead of resolution |
Why keep the model
Because the phases are still the right decomposition for reasoning about *why something is slow*. When a program stalls, it stalls in a phase: the front end cannot fetch fast enough, the decoder is the bottleneck, an execution unit is saturated, or a result is not ready when a dependent instruction needs it. Those are the four phases, and the performance counters on real hardware are organised roughly along the same lines (The CPU Counts Itself).
The pipeline below shows the model doing the one job it is genuinely good at: making it obvious that overlapping the phases gives you one completed instruction per cycle in steady state, even though each individual instruction still takes five cycles start to finish. That distinction — per-instruction latency versus completion throughput — is the foundation of Throughput Improved, Latency Did Not, and it is much harder to see without the diagram.
So the honest framing is: this is a model of the *logical* work an instruction requires, not a description of the machine's structure. Use it to know what work must happen. Do not use it to predict how long that work takes, or in what order it happens relative to its neighbours.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | |
|---|---|---|---|---|---|---|---|
| ADD r1, r2, r3 | I | I | E | M | W | ||
| SUB r4, r5, r6 | I | I | E | M | W | ||
| AND r7, r8, r9 | I | I | E | M | W |
Key points
- Fetch, decode, execute and write back are the logical phases every instruction requires — that part is universal and does not change.
- The claim the model gets wrong is *sequencing*: modern cores overlap the phases, execute out of program order, and retire in order.
- Instruction count is a weak predictor of runtime, because independent instructions overlap and dependent ones do not.
- Fetching an instruction is a memory access, so code competes with data for cache capacity.
- Keep the model for reasoning about which phase is the bottleneck; discard it for predicting time or order.
Progressive depth
Overview
An instruction must be fetched from memory, decoded into an operation and operands, executed by a functional unit, and its result written back. Four phases, in that order, because each depends on the previous one.
Practical
The phases overlap. While one instruction executes, the next decodes and a third is fetched, so a filled pipeline completes roughly one instruction per cycle even though each takes several cycles individually. This is why instruction count alone does not predict runtime.
Advanced
Modern cores go further: instructions are decoded into micro-operations, register names are remapped to break false dependencies (Register Renaming), and operations execute as soon as their inputs are ready rather than in program order (Out-of-Order Execution). Program order is restored only at retirement.
Internals
The practical consequence is that the machine is a throughput engine with a limited window. Performance is governed by whether the window can be kept full: front-end fetch bandwidth, branch prediction accuracy, dependency chain length, and above all whether operands are arriving from cache or from DRAM. A dependent chain of loads defeats every one of those mechanisms simultaneously, which is why Pointer Chasing: The Address You Do Not Have Yet is the canonical worst case.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1PC → fetch unit: the program counter supplies an address, and the fetch unit requests bytes from the instruction cache.
- 2Fetch → decode: raw bytes arrive and the decoder identifies the operation, source registers, destination and any immediate value.
- 3Decode → execution unit: the decoded operation is dispatched to a functional unit capable of performing it.
- 4Execution unit → write back: the result is produced and written to the destination register or forwarded to a waiting instruction.
- 5Write back → PC: the program counter advances to the next instruction, or is redirected by a branch.
- • "Fewer instructions means faster code" — true only when the removed instructions were on the dependency critical path.
- • "These two lines run one after the other" — they may execute in either order internally, as long as the visible result is as if they had not.
- • "The CPU is idle because IPC is low" — low IPC usually means the CPU is *waiting*, most often on memory, not that it has nothing to do.
- • "This model describes my CPU" — it describes the logical work, not the structure of any processor built in the last thirty years.
Consequences, controls and cost
- • Programs with the same instruction count can differ several-fold in runtime, depending on how much of the work is independent.
- • A large or scattered code footprint slows a program through instruction-cache pressure, with no change to the algorithm.
- • Reasoning that assumes line-by-line execution produces wrong conclusions about both performance and multithreaded visibility.
- • Reduce the *dependent* work on the critical path rather than the total instruction count — independent instructions are close to free on a wide machine.
- • Keep hot code compact so the front end can keep the back end fed; this is what makes inlining a trade-off rather than a free win ([[instruction-cache]]).
- • Measure with counters rather than reasoning from source order, because the source order is not the execution order.
- • Accept that for most application code the answer is "almost nothing directly" — the leverage is in data layout and memory access, not in instruction selection.
- • Read instructions retired and cycles together; their ratio is [[ipc]] and it separates "doing a lot of work" from "doing work quickly".
- • Compare instruction count between two builds against their actual runtime — the two frequently move in opposite directions.
- • Use a top-down analysis if your CPU vendor supplies one: it attributes stalls to the front end, back end, bad speculation or retiring, which maps onto the phases.
- • Check front-end stall counters before assuming a slow loop is an arithmetic problem.
- • Teaching the simplified model first costs accuracy: learners must later unlearn the sequencing claim, which is the single most common source of wrong hardware intuition.
- • Optimising for the real machine (independent work, compact code) makes code less obvious to read than optimising for the model would.
- • The counters that reveal actual behaviour are microarchitecture-specific, so the diagnostic skill transfers less cleanly between vendors than the concepts do.
Scope
§224 — what these claims are specific to.
- SIMPLIFIEDThe four-phase decomposition and five-stage pipeline are teaching models. Real cores add rename, schedule, issue and retire stages and use micro-operations; the phases remain a valid *logical* decomposition on every architecture.
- MICROARCH-SPECIFICThe number of in-flight instructions, decode width and pipeline depth differ between every microarchitecture and generation, including between performance and efficiency cores in the same package.
Misconceptions
Where the rest of this lives
The instructions the CPU fetches are chosen by a compiler or JIT that has already reordered and eliminated work relative to your source. The machine never sees the code you wrote.