The ALU: Where Arithmetic Actually Happens
The arithmetic logic unit performs the operations that source-level arithmetic compiles into, sets the flags that comparisons and branches depend on, and — crucially — does not treat all arithmetic as equal. Add and XOR are nearly free; divide is not.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
What the unit actually does
The ALU performs the integer operations an ISA defines: addition and subtraction, bitwise AND, OR, XOR and NOT, shifts, and comparison. Comparison is worth pausing on, because it is usually implemented as a subtraction whose result is discarded and whose flags are kept — zero, negative, carry, overflow. Those flags are what a subsequent conditional branch reads.
That indirection explains a pattern you will see constantly in disassembly: a cmp immediately followed by a conditional jump. The two are a pair. The compare does the arithmetic and sets flags; the jump consumes them. It also explains why comparison is cheap — it is an addition the machine was already good at, with the result thrown away.
Floating-point arithmetic is generally handled by separate units with their own latencies and their own rules, and vector operations by yet another set (SIMD: One Instruction, Many Elements). The word "ALU" in a block diagram usually means the integer unit specifically, and a real core has more than one of them so that independent integer operations can proceed in parallel (Superscalar Execution).
| Class | Examples | Notes |
|---|---|---|
| Arithmetic | ADD, SUB, NEG, INC | The cheapest operations the machine has; typically single-cycle latency |
| Bitwise | AND, OR, XOR, NOT | As cheap as addition; the basis of Bit Masks and flag manipulation |
| Shift | SHL, SHR, arithmetic shift right | Cheap; a shift by a constant is how compilers implement division by powers of two |
| Comparison | CMP, TEST | A subtraction whose result is discarded and whose flags are kept |
| Multiply | MUL, IMUL | Noticeably more expensive than add, but pipelined — throughput is much better than latency suggests |
| Divide | DIV, IDIV | The outlier: high latency and often only partially pipelined, so back-to-back divides serialise |
Flags, and how a comparison becomes a branch
The flag mechanism is worth seeing concretely, because it is the seam between arithmetic and control flow — and therefore the point at which Branch Prediction: Guessing Well Enough to Matter enters the picture. A high-level if (x > y) becomes a compare that sets flags, followed by a conditional jump that reads them and redirects the program counter if the condition holds.
This is also where a subtle hazard lives: the flags are a shared, implicitly-written resource. Almost every arithmetic instruction updates them, which means the compare must be adjacent to the branch that consumes it, and which historically made flags a source of false dependencies that renaming had to handle carefully.
The annotated fragment below is deliberately labelled as belonging to one instruction set. The *shape* — compare, then conditional branch on flags — generalises across ISAs. The mnemonics, the flag names and even whether flags exist as an architectural concept at all do not.
cmp %rbx, %rax ; compute rax - rbx, discard result, set flags
jle .Lskip ; if flags say rax <= rbx, jump over the body
add %rax, %rcx ; total += x (also sets flags, which is why
; cmp must sit next to its branch)
.Lskip:
... ; execution continues here either way
; note: there is no "greater than" arithmetic. There is a subtraction,
; four flag bits, and a jump that interprets them.Divide is the one that is genuinely different
The practical payload of this lesson: integer and floating-point division cost dramatically more than the other operations, and unlike multiply, division units are frequently not fully pipelined — meaning a second divide may not be able to start until the first is well advanced. A loop with a division in the inner body can be limited entirely by the divider while every other unit sits idle.
The good news is that this is one of the few places where source-level intent survives to the machine. Compilers aggressively convert division by a *compile-time constant* into a multiply-and-shift sequence, which is why x / 8 is not a division at all. Division by a *runtime* value cannot be transformed that way, and it is that case — dividing by a loop-invariant variable, over and over — that is worth hoisting: compute the reciprocal once outside the loop and multiply inside it.
The ratios below are deliberately relative and deliberately labelled. The absolute cycle counts differ between vendors, between generations, and between operand widths — and divider performance in particular has improved substantially on recent designs. What has remained stable across all of that is the *ordering*: add and XOR at the bottom, multiply somewhat above, divide far above everything.
x / 8 is actually implementedKey points
- The ALU handles arithmetic, bitwise operations, shifts and comparison; comparison is a subtraction that keeps only its flags.
- Flags are the seam between arithmetic and control flow, which is why
cmpand a conditional jump always appear as a pair. - Operations do not cost the same: add and XOR are the cheapest, multiply is moderate and pipelined, divide is an outlier.
- Division by a compile-time constant is not a division — the compiler turns it into a multiply and shift.
- Division by a loop-invariant runtime value is worth hoisting: compute a reciprocal once and multiply inside the loop.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Decode → scheduler: the operation is tagged with which execution unit class it needs.
- 2Scheduler → ALU: an integer operation is issued to a free ALU once its register operands are available.
- 3ALU → flags: arithmetic and comparison operations write condition flags as a side effect.
- 4Flags → branch unit: a conditional branch reads those flags to decide whether to redirect the program counter.
- 5Divider → scheduler: a division occupies its unit for many cycles and, if not fully pipelined, blocks subsequent divisions from starting.
- • "Multiplication is expensive, so I should replace it with additions." Multiply is pipelined and cheap in throughput terms; a chain of dependent adds is often worse.
- • "My code has no divisions." Modulo is division. So is any
%in a hot loop, including hash-table index computation (Hash Table). - • "The compiler will optimise my division away." Only when the divisor is a compile-time constant. A runtime divisor stays a division.
Consequences, controls and cost
- • A loop containing a runtime division can be limited by the divider while the rest of the core is idle.
- • Replacing a division by a loop-invariant value with a hoisted reciprocal multiply is one of the few reliable source-level micro-optimisations.
- • Bitwise-heavy code is usually far cheaper than its instruction count suggests, because those operations are the cheapest available and several issue per cycle.
- • Hoist division by a loop-invariant value out of the loop and multiply by the precomputed reciprocal inside it.
- • Prefer compile-time-constant divisors where the algorithm allows, so the compiler can transform them away entirely.
- • For floating point, be aware that reciprocal-multiply is not bit-identical to division — check whether your correctness requirements permit it.
- • For everything else, leave arithmetic selection to the compiler; it knows the target's costs better than a general rule does.
- • Disassemble the hot loop and look for divide instructions; their presence in an inner loop is usually worth investigating directly.
- • Compare against a variant with a hoisted reciprocal multiply and measure end to end — the change is small enough to A/B cheaply.
- • Watch execution-unit-specific counters where the platform exposes them; a divider-bound loop shows low IPC with low memory activity.
- • Reciprocal multiplication changes floating-point results in the last bits; for some numerical code that is unacceptable.
- • Hand-replacing arithmetic makes code less readable in exchange for a win that only matters if the loop is genuinely hot.
- • Divider performance varies enough between generations that a change tuned for one machine may be irrelevant on the next.
Scope
§224 — what these claims are specific to.
- MICROARCH-SPECIFICOperation latencies, throughputs and how fully the divider is pipelined vary between vendors and generations; recent designs have improved division substantially.
- ISA-SPECIFICWhether condition flags exist at all is an ISA choice — x86-64 and AArch64 have them, RISC-V deliberately does not and uses compare-and-branch instructions.
Misconceptions
% can dominate.