advanced

A Hundred Per Cent Busy and Getting Nothing Done

Read the counters before the options. Nothing here is labelled with the answer.

The report

Our aggregation job pins every core at 100% CPU. We moved it to instances with twice the cores and it got about 8% faster, not twice. Finance is asking why we doubled the bill for 8%. Someone wants to try the highest-clocked instance type next.

The aggregation kernel — one pass over a large array of records

// records: ~40 GB, far larger than any cache
for r in records:
    bucket[r.key & MASK] += r.amount

CountersSIMULATED

CPU utilisation	99–100% on every core	No core is idle at any point during the run.
IPC	0.19	Fewer than one instruction retires every five cycles.
L1-dcache-load-misses	4.2% of loads	Most loads are satisfied by the first-level cache.
LLC-load-misses	61.3% of last-level accesses	Most accesses that reach the last level do not find their data there.
memory bandwidth	≈ 94% of the platform maximum	The memory system is delivering close to as much as it is rated to deliver.
stalled-cycles-backend	78% of cycles	For most cycles the back end cannot make progress.

What is the hardware doing?

Busy Is Not the Same as Working When the Memory Bus Is the Bottleneck Latency and Bandwidth Are Different Resources CPI and IPC: The Number Everyone Misreads The CPU Counts Itself Array of Structs, or Struct of Arrays?