Auto-Vectorization: Verify, Do Not Assume
Compilers vectorise loops automatically, sometimes. It is a best-effort optimisation with no guarantee, it fails silently, and it can stop working after an unrelated edit — so the only responsible position is to check rather than believe.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
Why it fails silently
Vectorisation is an optimisation, not a semantic feature. Declining to apply it is always correct behaviour, so there is nothing to warn about — a compiler that emitted a diagnostic every time it chose not to vectorise a loop would produce an unusable amount of noise. The result is that a loop which vectorised last month and does not today produces exactly the same output, exactly the same tests passing, and a performance difference nobody attributes to anything.
The triggers for losing it are mundane. Adding a function call the compiler cannot inline. Introducing a pointer parameter that might alias. Adding a bounds check with a data-dependent branch. Changing a type so the element count no longer divides evenly. Upgrading the compiler. None of these look like performance changes in review.
The estimate side matters too. Even where the transformation is legal, a compiler applies a cost model and may decide the trip count is too small or the access pattern too expensive to be worth it. That model differs between compilers and between versions, which is why the same source vectorises under one toolchain and not another.
| Blocker | Looks like | Report says | Fix |
|---|---|---|---|
| Possible aliasing | Ordinary pointer parameters | Cannot prove ranges are disjoint | restrict-style annotation or local copy |
| Loop-carried dependency | A running total or previous-element read | Dependence between iterations | Per-lane partials, if reassociation is allowed |
| Float reassociation | A plain floating-point sum | Reduction requires reassociation | Explicit per-loop pragma or flag |
| Non-inlined call | Any function call in the body | Call prevents vectorisation | Inline it, or hoist it out |
| Data-dependent branch | An if inside the loop | Control flow could not be flattened | Mask, or split into uniform passes |
| Unknown trip count | A while loop on a condition | Trip count not computable | Restructure as a counted loop |
Three ways to check, in increasing order of trust
The vectorisation report is the cheapest. Every major compiler can be asked to state which loops it vectorised and, more usefully, why it declined the others — often naming the exact blocking condition. Read the report first; it usually turns a mystery into a one-line fix.
The disassembly is the definitive check. The report describes intent; the generated code is what runs. Look for vector instructions and vector-width registers in the hot loop. This also catches the case where the loop vectorised but the compiler also emitted a runtime overlap check, so the vector path only executes when the ranges happen to be disjoint.
The measurement is what actually matters, and it is the only one of the three that tells you whether vectorising helped. A loop can vectorise and run no faster because it was bandwidth-bound all along. Time it on realistic data, with realistic trip counts, on the target machine — the discipline that Every Way a CPU Microbenchmark Lies exists to protect.
hot.c:14:5: remark: loop not vectorized: cannot identify array bounds
hot.c:14:5: remark: loop not vectorized: value that could not be identified
as reduction is used outside the loop
hot.c:22:5: remark: loop not vectorized: unsafe dependent memory operations
in loop. Use #pragma loop vectorize(assume_safety)
hot.c:31:5: remark: vectorized loop (vectorization width: 8,
interleaved count: 2)
Line 22 is the interesting one: the compiler is telling you
exactly which promise it is missing.Keeping it once you have it
Because the failure is silent and the triggers are ordinary edits, verification has to be repeatable rather than a one-off investigation. The practical options are a benchmark in continuous integration that would catch the regression in time, or an assertion on the generated code for the few loops where it genuinely matters.
The honest scoping matters here. This is worth doing for a small number of demonstrably hot numeric loops, and not worth doing anywhere else. Most code is not performance-critical, most loops are not vectorisable, and treating auto-vectorisation as something to defend everywhere produces a large amount of fragile tooling around code that does not need it.
Where it does matter, writing the loop in the canonical vectorisable shape — counted loop, contiguous unit-stride access, no calls, no branches, aliasing settled — makes it far more likely to survive compiler upgrades and refactors, because you are not depending on the cost model making a marginal call in your favour.
- Check the report — cheapest, and usually names the exact blocker.
- Check the disassembly — definitive; the report states intent, not outcome.
- Measure on realistic data — the only check that says whether it helped.
- Guard the few loops that matter — a CI benchmark or a codegen assertion, not blanket tooling.
Key points
- Auto-vectorisation is best-effort with no guarantee, and declining is always correct behaviour.
- It fails silently: the code still works, tests still pass, only the time changes.
- Ordinary edits — a call, a branch, a pointer parameter — routinely disable it.
- Report, then disassembly, then measurement: three checks in increasing order of trust.
- Guard only the loops that demonstrably matter; defending it everywhere is not worth the tooling.
Progressive depth
Overview
Compilers can turn simple numeric loops into vector instructions automatically, but they are not obliged to and they do not tell you when they decline. If a loop matters, check rather than assume.
Practical
Three checks in order: read the vectorisation report, which usually names the blocker outright; confirm in the disassembly that vector instructions exist; then measure, because a vectorised loop that was bandwidth-bound is no faster. Most failures come down to aliasing or to a float reduction needing permission to reassociate.
Advanced
Where disjointness cannot be proved statically, compilers often emit both paths with a runtime overlap check. That keeps the vector path available but adds a branch and code size, and the fast path only runs when the data happens to cooperate. Interleaving — processing several vectors per iteration — interacts with this, and the reported vectorisation width alone does not tell you how much work the loop does per iteration.
Internals
The decision is made by a cost model comparing estimated scalar and vector costs at an assumed trip count, using per-target instruction cost tables. Those tables are MICROARCH-SPECIFIC and change between compiler releases, which is why a loop can lose vectorisation on a compiler upgrade with no source change at all. Where the loop genuinely must vectorise, explicit vector code removes the dependence on that model at the cost of portability and a scalar fallback path to maintain.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Compiler → legality analysis: dependence and aliasing are checked; anything unprovable blocks the transformation.
- 2Legality → cost model: even when legal, the compiler estimates whether vectorising is profitable at the expected trip count.
- 3Decision → code generation: either vector instructions with a scalar remainder, or plain scalar code, with no diagnostic either way.
- 4Optional → runtime versioning: where overlap is possible, both paths may be emitted with a dispatch check at loop entry.
- 5Source edit → silent regression: an added call or branch changes the analysis, and the loop quietly reverts to scalar.
- • "It compiles at high optimisation, so it is vectorised." Optimisation level enables the attempt, not the outcome.
- • "The report says vectorized, so the loop is fast." It may be bandwidth-bound and unchanged in time.
- • "No warning means it worked." There is no warning either way; silence carries no information.
- • "It vectorised on my machine, so it will in production." Different compiler, flags or target can all change the decision.
- • "This loop is simple, so it must vectorise." Simplicity does not supply the aliasing proof.
Consequences, controls and cost
- • A performance regression can ship with no failing test and no visible code change of consequence.
- • The same source performs differently across compilers and compiler versions.
- • Reading a vectorisation report is usually faster than guessing at why a loop is slow.
- • Runtime-versioned loops only take the fast path when the data happens to be disjoint.
- • A vectorised loop that is bandwidth-bound shows no improvement, so vectorisation status alone is not a success criterion.
- • Read the vectorisation report for the hot loop before changing anything.
- • Confirm in the disassembly that vector instructions were actually emitted.
- • Write hot numeric loops in the canonical shape — counted, contiguous, call-free, branch-free — so the decision is not marginal.
- • Supply the missing fact the report names, whether that is non-aliasing or permission to reassociate.
- • Add a CI benchmark for the small number of loops where the regression would matter.
- • Enable the toolchain's vectorisation or optimisation remarks and read them for the specific loop.
- • Disassemble the hot function and look for vector instructions and vector-width registers.
- • Time the loop against a deliberately scalar build on realistic data and trip counts.
- • Check whether a runtime overlap branch was emitted, which indicates static disjointness could not be proved.
- • Re-verify after compiler upgrades, since cost models and analyses change between versions.
- • Verification costs build configuration and developer attention on every hot loop you choose to defend.
- • Writing in the canonical vectorisable shape constrains how the code can be expressed and can hurt readability.
- • Codegen assertions in CI are brittle across compiler upgrades and can produce false failures.
- • Relying on auto-vectorisation rather than explicit vector code trades control for portability, and the trade is not always right.
Scope
§224 — what these claims are specific to.
- PLATFORM-SPECIFICWhich loops vectorise depends on the compiler, its version, the optimisation level, the target flags and the cost model. Two toolchains given identical source routinely make different decisions, so results must be verified against the shipping configuration.
- ISA-SPECIFICWhether masked execution and gather are available determines whether branchy and scattered loops are candidates at all. Building for a baseline target commonly disables vector extensions the deployment machine actually has.
Misconceptions
Apply it
Where the rest of this lives
Auto-vectorisation is one instance of a general property: optimisations are permitted transformations, not guarantees. Reasoning about which are applied, and when a language semantics change enables or blocks them, belongs with compiler internals.