IPC: Instructions Per Cycle
The ratio that connects "how much work" to "how long it took". It is the most useful single number for diagnosing a CPU-bound loop — and one of the easiest to misuse, because a change that raises IPC can leave the program slower.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The three-term model
CPU time for a piece of work is instruction count × cycles per instruction × cycle time. Equivalently, and more usefully for diagnosis: time = instructions ÷ (IPC × frequency). Three independent variables, and an optimisation can move any of them — sometimes in opposite directions.
This immediately disposes of the gigahertz question that the interview guide poses. A core at a lower clock with a higher sustained IPC can complete the same work sooner, and frequently does, because a wider machine with better prediction and bigger caches extracts more per cycle. Frequency is one term of three.
CPI is simply the reciprocal, and both are in use. IPC reads more naturally when you are talking about how much the machine is getting done; CPI reads more naturally when you are decomposing where cycles went, because contributions from cache misses, mispredictions and stalls add up in CPI terms. The domain's treatment of CPI as a decomposition tool is in CPI and IPC: The Number Everyone Misreads.
| Change | Instructions | IPC | Frequency | Net effect |
|---|---|---|---|---|
| Better algorithm, fewer operations | Down sharply | Unchanged or lower | Unchanged | Faster — the usual biggest win |
| Vectorising a loop | Down | Often down | May drop under load | Usually faster despite both terms falling |
| Improving cache locality | Unchanged | Up sharply | Unchanged | Faster |
| Unrolling with more instructions | Up | Up | Unchanged | Ambiguous — must measure time, not IPC |
| Sustained heavy vector load | Unchanged | Unchanged | May drop | Slower than the counters suggest |
Why IPC is a diagnostic, not a goal
Optimising IPC directly is a trap, and vectorisation is the clearest example. Replacing eight scalar adds with one vector add cuts the instruction count by roughly eight. If cycles fall by less than that, IPC goes *down* — while the loop got substantially faster. A team tracking IPC as a target would read that as a regression.
The reverse trap also exists. Padding a loop with cheap independent instructions raises IPC because the machine issues them into otherwise idle slots. The program does strictly more work and takes at least as long, and the metric improves. IPC rewards keeping the machine busy, which is not the same as finishing sooner.
The correct use is comparative and diagnostic: for a fixed amount of work, a low IPC says the machine is stalling and the counters say on what. That is genuinely valuable, and it is why the number appears in every top-down methodology. But the thing you optimise is wall-clock time for the workload; IPC is how you find out why it is what it is.
SCALAR instructions 8.00e9 cycles 2.20e9 IPC 3.64 elapsed 0.73 s VECTORISED (8 elements per instruction) instructions 1.15e9 <- 7x fewer cycles 0.46e9 <- 4.8x fewer IPC 2.50 <- LOWER elapsed 0.15 s <- 4.8x faster Tracking IPC as a target would call this a regression. Tracking elapsed time calls it a 4.8x win.
Reading a low IPC
A low IPC on a wide core means the machine is not issuing much, and there are only so many reasons for that. Each has a distinct counter signature and a distinct fix, which is what makes this a productive starting point rather than a dead end.
Memory stalls show up as high cache-miss counts alongside the low IPC, and send you to the memory hierarchy (Hits, Misses and What a Miss Actually Costs, Three Kinds of Miss, Three Different Fixes). Bad speculation shows up as high misprediction counts (Misprediction: What a Wrong Guess Costs). A dependency chain shows up as low IPC with *low* counts on both, since nothing is missing and nothing is mispredicting — the machine simply has nothing ready to issue (Dependency Graphs: The Real Shape of Your Code). Front-end starvation shows up in frontend-bound categories where available (Your Code Is Data Too).
One caveat that catches people: IPC is meaningless across different workloads. Comparing the IPC of a database scan against that of a physics kernel says nothing about either. The comparison must be the same work, on the same machine, before and after one change — which is the same discipline the Observability & Performance domain applies to every metric.
- Low IPC + high cache misses → memory. Fix locality.
- Low IPC + high branch misses → speculation. Make the branch predictable or remove it.
- Low IPC + both counters low → dependency chain. Break it up.
- Low IPC + frontend-bound → code footprint. Shrink the hot loop.
- High IPC + slow program → you are executing too many instructions. Look at the algorithm.
Key points
- Time = instructions ÷ (IPC × frequency); every optimisation moves at least one of the three terms.
- Higher IPC does not mean faster — vectorising typically lowers IPC while cutting time substantially.
- IPC is a diagnostic for a fixed workload, never a target to maximise.
- A low IPC plus counter signatures identifies which of four stall causes is binding.
- IPC comparisons across different workloads are meaningless; compare the same work before and after one change.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Retirement → instruction counter: the counter increments as instructions retire, not as they execute, so discarded speculative work does not count.
- 2Clock → cycle counter: cycles accumulate whether or not anything retired, so stalls show up as cycles without instructions.
- 3Stall source → ratio: whichever of memory, speculation, dependencies or front end binds determines how many cycles pass per retired instruction.
- 4Counters → IPC: the ratio is computed after the fact from two counters; it is not something the core tracks or targets.
- 5Frequency scaling → wall clock: the same IPC at a lower clock is less work per second, which is why elapsed time remains the real measure.
- • "IPC went up, so the change was good." Not if instruction count went up more.
- • "IPC went down, so we regressed." Vectorising lowers IPC by design while cutting time.
- • "IPC of 1.2 is bad." Bad relative to what? It depends entirely on the workload and the core.
- • "CPU is at 100%, so we are compute-bound." Utilisation says occupied; IPC says productive.
- • "We should target IPC in CI." A benchmark can be gamed into higher IPC and longer runtime simultaneously.
Consequences, controls and cost
- • Vectorised code often shows lower IPC than the scalar version it replaced, while running several times faster.
- • Adding pointless independent instructions raises IPC and does not help.
- • A stalled core reports as fully utilised to the OS while achieving a small fraction of its issue width.
- • The same binary shows very different IPC on different microarchitectures, so targets do not transfer.
- • Where counters are unavailable — many containers and VMs — this diagnosis has to be replaced by controlled experiment.
- • Measure elapsed time as the goal; use IPC to explain the time, never to replace it.
- • Pair IPC with cache-miss, branch-miss and front-end counters so the cause is identified rather than guessed.
- • Compare before and after a single change on identical work and the same machine.
- • When counters are unavailable, substitute controlled A/B experiments on the real workload.
- • Treat any absolute IPC target inherited from another team or machine as meaningless until re-derived locally.
- • Collect retired-instruction and cycle counters over the same fixed workload, then compute the ratio.
- • Always collect elapsed time alongside, and treat time as the decision variable.
- • Collect cache-miss, branch-miss and stall-category counters in the same run so the cause is attributable.
- • Pin frequency where the platform allows it, so that comparisons are not confounded by boost behaviour ([[frequency-scaling]]).
- • Repeat on each target microarchitecture rather than assuming a result transfers.
- • Counter collection has overhead and may perturb the very behaviour being measured on short workloads.
- • Pinning frequency for reproducibility gives up the performance behaviour production actually sees.
- • Time spent on counter-level diagnosis competes with algorithmic work that usually has a higher ceiling.
- • Counters expose microarchitectural detail that encourages tuning to one machine at the cost of portability.
Scope
§224 — what these claims are specific to.
- MICROARCH-SPECIFICAchievable IPC depends on issue width and port mix. A given IPC value is meaningful only relative to the core it was measured on; the same code on a narrower core yields a different number with no change in quality.
- PLATFORM-SPECIFICCounter names, availability and multiplexing behaviour differ by vendor, kernel version and hypervisor. Counters are commonly unavailable inside containers and restricted on shared cloud instances.