advancedLegality
Why does undefined behavior matter to compiler optimization?
Whether the candidate understands undefined behavior as a licence the optimizer holds rather than as an outcome the program suffers. The discriminator is whether they can trace a transformation from the assumption to the deleted code.
What a strong answer covers
- Because the optimizer is allowed to assume the program never does it. Undefined behavior is not a promise that something bad will happen — it is the language declining to constrain the implementation, which the implementation reads as: no valid execution reaches this state, so I may transform on that basis.
- That assumption is load-bearing for ordinary optimizations. If signed overflow is undefined, then
i + 1 > iis unconditionally true, and a loop bound can be simplified. If dereferencing null is undefined, then a pointer that has already been dereferenced is known non-null, and a later null check is dead code the compiler may delete — which is how a defensive check disappears from a function whose earlier line dereferenced the same pointer. If reading an uninitialized value is undefined, the compiler may pick whichever value makes the surrounding code cheapest, and pick differently in two places. - The consequence engineers actually meet is nonlocality. The transformation is legal in isolation, the code that disappears is often the check that would have caught the problem, and the symptom appears far from the cause — commonly only at higher optimization levels, or only after an unrelated change let the inliner see both sides at once.
- The defence is not "avoid the optimizer". It is to make the constraint explicit: sanitizers to catch the execution, well-defined constructs where you need the behaviour, and — where a value must not be optimized away — a mechanism the language actually defines, such as an explicit secure-zero function rather than a memset the compiler is entitled to delete.
✓ Green flags
- States the direction correctly: UB licenses assumptions, it does not produce diagnostics.
- Traces one concrete chain — assumption, deduction, deleted code — end to end.
- Names that the same source can behave differently at -O0 and -O2 without either compiler being wrong.
- Distinguishes undefined from unspecified and implementation-defined behaviour.
- Names sanitizers or UBSan as the practical way to find it, not code review.
✗ Red flags
- "UB means the program crashes." It means nothing is guaranteed. Crashing is the lucky case; silently working during testing and failing in production is the normal case.
- "It only matters in C and C++." Every language with an unsafe subset has it, and any language with a foreign function interface inherits it at the boundary.
- "The compiler is being malicious when it exploits UB." It is applying transformations that are correct for all defined programs; the program left the defined set.
- "If it works at -O0 the code is fine." -O0 not exploiting an assumption is not the same as the assumption not existing.
- "Just add a volatile and it goes away." Volatile constrains one narrow thing about accesses and does not make an out-of-bounds write or a signed overflow defined.
Follow-up
You have a null check that the compiler removed. Show me the earlier line that made the removal legal, and give me two fixes — one that changes the code and one that changes what the compiler is allowed to assume.
Implementation challenge
What to ask them to write or trace on a whiteboard.
Write a function where reordering two statements is legal only because of an undefined-behavior assumption, and then write the version where the same reordering would be illegal. Name the assumption in one sentence.