What Actually Happens When You Add Two Numbers
One line of source becomes a handful of instructions, and the addition itself is the cheapest thing in it. The expensive question — the one this entire domain exists to answer — is where a and b were when the CPU went looking for them.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
One line, and the machine's answer to it
Source code is a description of intent. The machine does not execute it; it executes instructions the compiler produced from it, and those instructions operate on registers — a few dozen named slots physically inside the CPU. Nothing arithmetic happens to memory directly on most architectures: values must be loaded into registers, operated on, and stored back.
So int x = a + b; becomes roughly: load a, load b, add them, store the result. The exact instructions depend on where the compiler decided those variables live, which depends on register pressure, optimisation level and the calling convention. If a and b were already in registers — because they were just computed, or the loop keeps them there — the loads vanish entirely and only the add remains.
The path below is the loop this whole domain follows. Each arrow is a lesson: Instruction Fetch: Code Is Data Too gets the bytes, Decode: Turning Bytes Into Intent works out what they mean, Registers: The Fastest Storage, and There Is Almost None of It supplies the operands, The ALU: Where Arithmetic Actually Happens does the arithmetic, and the result becomes architecturally real at The Reorder Buffer and Precise State. What the source line shows you is the middle of that chain and none of its cost.
The second question: where were `a` and `b`?
Now ask the question the source line cannot answer. A load instruction says "fetch the value at this address". It does not say how long that takes, because that depends entirely on where the value currently sits — and the same instruction, in the same program, can resolve in a single-digit number of cycles or in several hundred depending on nothing more than what ran before it.
The scale below is deliberately unitless. Published nanosecond figures are wrong on every machine except the one they were measured on, and they age badly; what survives the move from one CPU to another is the *ratio*. The shape — registers are essentially free, each cache level costs several times the last, main memory costs a couple of orders of magnitude more than L1 — is stable across decades of hardware even as every absolute number changes.
Read the ratio, not the row. A cache hit and a DRAM access differ by roughly two orders of magnitude, which means a loop that misses every iteration can be a hundred times slower than the identical loop over data that fits in cache. That single fact explains more real performance mysteries than any other in this domain, and The Memory Hierarchy is where it gets developed properly.
What this domain is actually about
The table below is the set of beliefs this domain exists to correct. None of them are stupid — each is a reasonable extrapolation from how source code reads. They are simply not how the machine behaves, and every one of them leads to a specific class of wrong prediction about performance.
Notice the shape of the corrections. Source code is sequential and the machine is not (Out-of-Order Execution). Source code treats memory as flat and the machine does not (The Memory Hierarchy). Source code makes arithmetic and memory access look equally cheap, and they differ by two orders of magnitude. Each gap is a place where reasoning from the source alone produces confident, wrong answers — which is exactly what Why Reading the Source Cannot Tell You the Cost is about.
The goal is not to memorise CPU terminology. It is to be able to look at a line of code and ask a better set of questions: where is this data, has anything touched it recently, can these operations overlap, and what will the machine do when it cannot tell what comes next.
| Reasonable belief | What the machine actually does | Where it is developed |
|---|---|---|
| Instructions execute one at a time, in order | Many are in flight at once, executed as their inputs become ready and retired in program order | Out-of-Order Execution |
| Memory access takes a fixed amount of time | Cost spans about two orders of magnitude depending on which level holds the data | The Memory Hierarchy |
| Adding is work; reading a variable is free | The add is nearly free; the read is what you pay for when it misses | What a Cache Actually Is |
| The CPU knows which branch it will take | It guesses, runs ahead speculatively, and discards the work when wrong | Branch Prediction: Guessing Well Enough to Matter |
| Source order is execution order | The compiler reorders, then the hardware reorders again within permitted limits | The Compiler Reordered It Before the CPU Did, Why Your Loads and Stores Happen Out of Order |
| More clock speed means proportionally more work done | Work per cycle varies enormously with stalls, and clock is only one factor | The Clock: Why GHz Is Not Performance, IPC: Instructions Per Cycle |
Key points
- The arithmetic in
a + bis the cheapest part; obtaining the operands is where the cost lives. - Arithmetic happens between registers — values must be loaded in and stored back on most architectures.
- The cost of a load spans roughly two orders of magnitude depending on which level of the hierarchy holds the data.
- Relative cost transfers between machines; absolute nanosecond figures do not, and go stale quickly.
- The purpose of this domain is better questions about a line of code, not more terminology.
Progressive depth
Overview
A line of arithmetic becomes a few instructions. The arithmetic is cheap. Fetching the operands is what varies, and it varies by roughly a hundredfold depending on where they are.
Practical
Because loads dominate, performance follows data layout and access order rather than operation counts. Sequential access over compact data is fast; scattered access over pointer-linked data is slow, even at identical asymptotic complexity. This is why Both Are O(n). One Is Far Slower. is a real performance question rather than a style preference.
Advanced
The CPU works hard to hide these costs: it prefetches lines it expects to need (Prefetching: The Hardware Guesses What You Will Read Next), keeps many loads in flight simultaneously (Misses That Overlap Are Nearly Free), and executes independent instructions while a load is outstanding (Out-of-Order Execution). These mechanisms succeed on predictable access patterns and fail on unpredictable ones, which is why pointer chasing is slow in a way that raw latency alone does not explain.
Internals
A load that misses everywhere becomes a physical event: address translation through the The MMU: Translation and Protection in One Check and The TLB: A Cache for Addresses, Not Data, a request to the memory controller, DRAM row activation, and a burst transfer of an entire Memory Moves in Lines, Not Variables worth of data — never just the bytes you asked for. The instruction depending on that value cannot retire until it returns, so the reorder buffer fills behind it and issue eventually stalls. That is what a "slow line of code" physically is.
Where the Data Is
Change an input and watch which number moves — and which one refuses to.
The exact ratios vary by machine and the absolute times vary far more, which is why none are shown. What is stable enough to build intuition on is the shape: each level is several times the one above, and the gap between the last cache level and memory is the one that decides most program performance.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Source → compiler:
int x = a + b;becomes instructions, with the decision about whereaandblive made at compile time. - 2Program counter → instruction cache: the fetch unit requests the bytes of the next instruction.
- 3Decode → register file: the instruction names two source registers and one destination.
- 4Load unit → cache hierarchy: if an operand is not already in a register, a load searches L1, then L2, then L3, then main memory, at sharply rising cost.
- 5ALU → register file: the addition executes in a single cycle and the result is written back, becoming architecturally visible when the instruction retires.
- • Concluding that because the algorithm is O(n) either way, the implementations will perform similarly — complexity says nothing about the constant that memory imposes.
- • Assuming the compiler's output resembles the source statement by statement; it frequently does not, and reading the disassembly is the only way to know.
- • Treating one published latency table as universal truth rather than as one measurement of one machine.
- • Concluding that since the addition is fast, the line is fast — the load in front of it is the part that varies.
Consequences, controls and cost
- • Two programs performing identical arithmetic can differ by a large factor purely because of where their data sits.
- • Optimising arithmetic in a memory-bound loop produces no measurable improvement, which is the most common wasted optimisation in practice.
- • Performance becomes sensitive to data layout and access order — properties invisible in the algorithm itself.
- • Reasoning that counts operations will systematically mispredict which of two implementations is faster.
- • Arrange for data to be reused while it is still close: access it in the order it is stored, and finish with it before moving on ([[spatial-locality]], [[temporal-locality]]).
- • Choose layouts that put the bytes you need together, rather than scattering them across the address space ([[data-oriented-design]]).
- • Let the compiler keep hot values in registers by keeping loops tight and avoiding unnecessary indirection.
- • Measure before assuming which half of the problem you have — arithmetic and memory need different fixes ([[cpu-bound-vs-memory-bound]]).
- • Compile with optimisation and read the generated assembly; count the loads, not just the arithmetic.
- • Compare cycles against instructions retired to get **IPC** — a low value with high instruction counts usually means stalls, not expensive work ([[ipc]]).
- • Run the same loop over a dataset that fits in cache and one that does not; the ratio is the memory effect isolated from everything else.
- • Sample cache-miss counters during the hot loop rather than reasoning about them ([[performance-counters]]).
- • Layouts optimised for one access pattern are usually worse for another; there is no universally cache-friendly arrangement.
- • Restructuring data for locality often costs readability and can hurt maintainability more than it helps performance.
- • Reasoning at this level is only worth it for code that actually runs hot; applied everywhere it is a large cost for no measurable return.
Scope
§224 — what these claims are specific to.
- SIMPLIFIEDThe fetch → decode → execute → writeback chain shown here is a teaching model. Real cores split these into many more stages, run several instructions per cycle, and reorder aggressively — see Out-of-Order Execution.
- GENERALThe register-to-register arithmetic model holds for common load-store architectures such as AArch64 and RISC-V. x86-64 instructions may name a memory operand directly, but internally the value is still loaded before the ALU sees it.
- PLATFORM-SPECIFICEvery relative figure in the cost scale varies by CPU, generation and memory configuration. The ordering and rough magnitudes transfer; the numbers do not.
Misconceptions
Apply it
Where the rest of this lives
Which instructions the compiler emits — and whether a and b ever reach memory at all — is a code-generation decision made before the CPU is involved. That domain does not exist yet; for now, read the disassembly.