The CPU Counts Itself
Every modern CPU carries a small unit whose only job is to tally what the rest of the chip did: cycles, instructions retired, misses at each cache level, mispredicted branches, stalled cycles. It is the only direct evidence you will ever get about the hardware — and it is sampled, approximate, and named differently on every chip.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
A small tally unit bolted onto a large machine
The Performance Monitoring Unit is a handful of hardware registers plus the logic to increment them when a chosen event occurs. A few are fixed-function — cycles and instructions retired are usually always available — and a few more are programmable, meaning you select which event each one watches. That number is small, typically a handful per core, and that scarcity drives most of the practical awkwardness in using them.
The events themselves fall into a small number of families that map directly onto the rest of this domain. Cycle and instruction counts tell you the rate of progress. Cache events at each level tell you where data was found. Branch events tell you how well the predictor is doing. Stall or "cycles with no instruction issued" events tell you the front end or back end is starved. TLB events tell you translation is the constraint rather than the data.
What makes this qualitatively different from every other performance tool is that it is not a model. A profiler that samples stacks is inferring where time went; a counter that says forty million last-level misses occurred is reporting a fact about the silicon. That is worth a great deal — and it is why the discipline of Busy Is Not the Same as Working is possible at all.
| Family | Typical events | The question it answers | Mechanism |
|---|---|---|---|
| Progress | cycles, instructions retired | How fast is work completing, and how much work is there? | CPI and IPC: The Number Everyone Misreads |
| Data supply | L1/L2/LLC load misses, fill buffer occupancy | Where is the data being found, and is the core waiting for it? | Hits, Misses and What a Miss Actually Costs |
| Control flow | branches, branch mispredictions | Is the predictor keeping the front end fed? | Misprediction: What a Wrong Guess Costs |
| Translation | dTLB/iTLB misses, page walk cycles | Is address translation, rather than data, the bottleneck? | When Translation Itself Is the Bottleneck |
| Stalls | cycles with no issue, port saturation | Which side of the machine is starved — front end or back end? | Superscalar Execution |
| Front end | instruction cache misses, decode stalls | Is the code itself failing to arrive fast enough? | Your Code Is Data Too |
Counting is sampling, and sampling skids
There are two modes and they answer different questions. Counting runs a workload and reports totals: fifty billion cycles, thirty billion instructions, four hundred million last-level misses. It is cheap, it is accurate in aggregate, and it tells you nothing about *where*. Sampling configures a counter to overflow every N events and raise an interrupt, recording the instruction pointer at that moment. That gives you attribution — this function, this line — at the price of statistics.
Sampled attribution is approximate in a specific and important way. By the time the interrupt is delivered, the machine has moved on: out-of-order execution and pipeline depth mean the recorded instruction pointer is generally *near* the instruction responsible, not exactly it. This is universally called skid, and it is why a sampled profile will sometimes blame the instruction after a load rather than the load. Modern chips offer precise-event mechanisms that reduce skid substantially, and those are worth using when attribution matters — but they are their own vendor-specific feature set.
The second practical constraint is multiplexing. Ask for more events than there are programmable counters and the kernel will time-slice them, running each for a fraction of the interval and scaling the result up. The totals then carry sampling error, and for a short or bursty workload that error can be large. If two counters disagree with each other in a way that seems impossible, multiplexing is the first thing to check.
Performance counter stats for './workload':
52,431,908,221 cycles
31,220,455,109 instructions # 0.60 insn per cycle
1,904,551,882 branches
12,880,417 branch-misses # 0.68% of all branches
4,551,203,884 L1-dcache-loads
988,204,551 L1-dcache-load-misses # 21.71% of all L1-dcache accesses
402,118,904 LLC-load-misses
8,220,551 dTLB-load-misses
14.882301019 seconds time elapsedWhat the counter is not telling you
A counter reports an event, not a cost. Four hundred million last-level misses is a fact; whether those misses cost you anything depends on whether the core had other work to do while they were outstanding, which is the subject of Misses That Overlap Are Nearly Free. A workload with many overlapping misses and a workload with the same number of serialized dependent misses produce the same miss count and wildly different runtimes.
Counters also sit in an uneasy relationship with speculation. Work that was executed and then discarded still consumed resources and, depending on the event and the chip, may or may not be counted. This is why instructions-retired and instructions-issued are different events, and why a branchy workload can show a large gap between them. Retired counts are the ones to reason about for work completed; issued counts tell you about effort spent, including effort wasted.
Finally, measurement is not free and is not neutral. Enabling counters costs a little; sampling at a high rate costs more and perturbs the very caches you are measuring. In a virtualized environment, counter access may be restricted, virtualized, or simply unavailable — which is a recurring frustration when the machine you need to diagnose is a cloud instance rather than a box you own.
- Event ≠ cost. A miss that overlapped with other work is nearly free; the same miss on a dependency chain is not.
- Retired ≠ issued. The gap between them is speculation that did not survive.
- Rates beat totals. Misses per instruction or per operation compares across runs; raw totals do not.
- Two counters beat one. A miss count without a cycle count cannot tell you whether the misses mattered.
- The observer perturbs. High-rate sampling pollutes the cache you are trying to characterise.
Key points
- The PMU is a small set of hardware registers counting selected events; the number of programmable counters is small, and that scarcity shapes how you use them.
- Counting gives accurate totals with no attribution; sampling gives attribution with statistical error and skid.
- Event names, availability and semantics are among the most microarchitecture-specific things in the entire domain — never assume an event ports across chips.
- A counter reports that an event happened, not that it cost anything; interpret every count alongside cycles.
- Counters are the only direct evidence about the hardware you will ever get, which is why they settle arguments a wall-clock profile cannot.
Progressive depth
Overview
The CPU can count its own events — cycles, instructions, cache misses, branch mispredictions — and report the totals. Those counts are the only direct evidence about what the hardware did, as opposed to what you think it did.
Practical
Run a counting pass first to learn *what* resource is implicated, then a sampling pass to learn *where*. Normalize to rates per instruction so runs compare. Ask for few events at a time to avoid multiplexing error.
Advanced
Sampled attribution skids because the interrupt arrives well after the triggering event in a deep out-of-order pipeline. Precise-event facilities tag the sample at the instruction responsible, at extra cost. Retired and issued counts diverge by exactly the work speculation threw away.
Internals
Counters are physical registers with event-select logic distributed across the pipeline, and the number of them is a silicon area decision. Multiplexing is a kernel-level time-slice over that scarce resource, with the reported totals scaled by the observed fraction — which is why a short workload multiplexed across many events produces numbers that can be internally inconsistent.
Performance Counters
Change an input and watch which number moves — and which one refuses to.
Low IPC with most last-level accesses missing: the core is waiting on DRAM. Adding cores will not help if bandwidth is already saturated; moving fewer bytes will.
Open the lesson →Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Software → kernel: a profiling tool asks for a set of events and a sampling period.
- 2Kernel → PMU: event selectors are programmed into the available counter registers, multiplexing if there are more events than counters.
- 3Execution → counter: the chosen event fires somewhere in the pipeline and increments its register.
- 4Counter → interrupt: on overflow the PMU raises an interrupt, and the handler records the current instruction pointer and call stack.
- 5Interrupt → report: samples are aggregated into per-function or per-line attribution, with the skid between event and interrupt baked in.
- • Treating a high miss count as proof of a problem, without checking whether cycles were actually lost to it.
- • Reading a sampled line-level profile as exact, and optimising the instruction after the one that really stalled.
- • Comparing counter totals between two runs of different lengths instead of comparing rates.
- • Assuming an event means the same thing on a different vendor's chip because the name is spelled the same.
- • Concluding a workload is fine because instructions-retired is high, when much of that work was speculative and discarded.
Consequences, controls and cost
- • Two workloads with identical wall-clock time and identical instruction counts can be separated instantly by their miss and stall profiles.
- • Sampled profiles occasionally attribute cost to the instruction after the real culprit, which misleads anyone reading a line-level profile literally.
- • Asking for many events at once quietly degrades accuracy through multiplexing rather than failing loudly.
- • Counter availability in cloud and virtualized environments is inconsistent, so the machine you most want to diagnose is often the one that will not tell you.
- • Start with a counting run, not a sampling run: get cycles, instructions, and misses at each level before asking where.
- • Ask for few events per run and repeat the run, rather than asking for many and accepting multiplexing.
- • Use precise-event sampling where the chip provides it whenever line-level attribution actually matters.
- • Normalize to a rate — per instruction, per element, per request — so numbers compare across runs and machines.
- • Cross-check any surprising counter against a second, independent one before building a theory on it.
- • A counting run over the whole workload: cycles, instructions, branch misses, L1/LLC misses, dTLB misses.
- • Derived rates: misses per thousand instructions, mispredictions per thousand branches, [[cpi]].
- • A precise-event sampling run scoped to the hot region once the counting run says which resource is implicated.
- • A repeat run with a different event set to confirm rather than trusting a single multiplexed measurement.
- • Sampling frequently enough for good attribution perturbs the caches and branch predictors being measured.
- • Precise-event mechanisms cost extra overhead and are not uniformly available.
- • Counter-based investigation is powerful but chip-specific; the knowledge transfers less well between machines than higher-level profiling.
- • Time spent learning one vendor's event set is partially wasted when the fleet changes vendor.
Scope
§224 — what these claims are specific to.
- MICROARCH-SPECIFICEvent names, counts of available counters, precise-event support and skid behaviour vary by vendor and by generation; treat any specific event name as local to the chip it was read on.
- PLATFORM-SPECIFICAccess to counters depends on the OS, on permissions, and on the hypervisor; many cloud instances expose a reduced set or none at all.
Misconceptions
Where the rest of this lives
A JVM or .NET profiler reports at the level of methods and allocations, layered on top of these same hardware counters; the mapping from a hardware event back to a source method passes through the JIT's code map.