advanced

One Field of Sixteen, and All of the Bandwidth

Read the counters before the options. Nothing here is labelled with the answer.

The report

Our physics step reads one field from each particle to build a bounding box. It is 20 million particles and it takes far longer than it should for what is essentially a min/max scan. Profiling says memory-bound, but we are only reading 4 bytes per particle — that is 80 MB, which should stream in no time.

The bounding-box pass over an array of structs

struct Particle {         // 64 bytes total
    float px, py, pz      // position    12
    float vx, vy, vz      // velocity    12
    float mass            //              4
    float radius          //              4
    uint32 id, flags      //              8
    float pad[6]          //             24
}
Particle particles[20_000_000]

// the pass — reads px only
for p in particles:
    minx = min(minx, p.px)
    maxx = max(maxx, p.px)

CountersSIMULATED

logical bytes read	80 MB (4 bytes × 20M)	The algorithm needs one 4-byte field per particle.
bytes fetched from memory	≈ 1.28 GB	The memory system delivered roughly sixteen times the logically required volume.
L1-dcache-load-misses	99.1% of loads	Nearly every load misses the first-level cache.
LLC-load-misses	96.4% of last-level accesses	Nearly every access reaches main memory.
memory bandwidth	≈ 88% of the platform maximum	The memory system is close to fully occupied.
IPC	0.14	Very little retires per cycle.

What is the hardware doing?

Array of Structs, or Struct of Arrays?Memory Moves in Lines, Not Variables Data-Oriented Design, Without the Dogma When the Memory Bus Is the Bottleneck Padding: Why Your Struct Is Bigger Than Its Fields Spatial Locality