Debugging Optimized Code
Why a variable reads "optimized out", why the instruction pointer jumps backwards between two functions, and why a breakpoint on a line you can see never fires. Three symptoms, three specific transformations, and none of them is a bug.
The debugger says my variable is optimized out and the cursor is jumping around. What is actually happening?
Two descriptions of the same execution that no longer line up one-to-one. The instructions are the optimized program: reordered, merged, duplicated, deleted, and with values living wherever the allocator put them. The debug metadata is a best-effort record of how those instructions relate to source positions and variables — a many-to-many relation, not a function. Debugging optimized code is the practice of reading that relation honestly rather than pretending it is still one line, one variable, one place.
Every transformation causing these symptoms is legal precisely because debug information is not observable behavior. The [[as-if-rule]] requires that the program's defined output be preserved; it says nothing about whether a variable remains inspectable, whether a source line retains an address, or whether instructions stay in source order. A compiler is permitted to destroy all three. The obligation it *does* have is to update the metadata it keeps — so a variable that has genuinely disappeared should report as unavailable, and a pass that leaves a stale location instead has produced a wrong description of correct code.
Key points
- "Optimized out" means the location list has no entry for the current address — an honest report, not a failure.
- Three causes: the register was reused after the last use, the value was never materialised, or it was rematerialised and has no single identity.
- Scope and liveness are different: a variable can be lexically in scope and have no value anywhere.
- The instruction pointer jumps because scheduling interleaves independent source lines and inlining brings another function's lines into this one.
- Inlining and tail calls remove stack frames; recovering the inlined ones requires inline records and a tool that asks for them (
addr2line -i). - A breakpoint fails because the line has no address, no
is_stmtaddress, was inlined elsewhere, or the function was cloned or removed. - Diagnose breakpoint failures by decoding the line table and checking whether the line appears at all.
- Prefer
-Og, per-functionoptnone/inline(never), or reading the metadata directly over rebuilding everything at-O0. - Rebuilding at
-O0changes timing and frame layout, so it frequently makes the bug you are chasing disappear.
Symptom one: "optimized out"
You print a variable and the debugger says <optimized out>. This is not an error and it is not the debugger giving up — it is the location list for that variable having no entry covering the current program counter, which is the honest report of a genuine fact. There are three distinct underlying causes and they call for different responses.
The value is in a register that has been reused. [[register-allocation]] assigns a variable to a physical register for the range where it is live and hands the register to something else afterwards. Past the variable's last use, its value is simply gone: nothing wrote it to memory, because nothing needed to. This is the most common case, and the giveaway is that the variable is readable earlier in the function and unavailable later.
The value was never materialised. Constant folding computed it at compile time; copy propagation replaced every use with the thing it was copied from; the expression was rewritten into a form where no operation corresponds to the variable at all. There is no register and no stack slot because there is no value — DWARF can sometimes still describe it as DW_OP_stack_value, which is why you occasionally get a readable value you cannot assign to.
The variable was rematerialised. The compiler decided recomputing it is cheaper than keeping it live, so it exists at three different points as three different computations and has no single identity. This is the case debug formats represent worst.
The practical responses, in order of cost: read the value at a different point where it is live; look at the register the compiler actually used (info registers, or read the location list directly); mark the one function __attribute__((optnone)) or #[inline(never)] and rebuild; or fall back to -Og. Reaching straight for -O0 should be last, because it changes timing, changes frame layout, and frequently makes the bug disappear.
n is unavailable after instruction 5 — the register was needed by something elseRead it asAt program point 6 the debugger is asked for n. The location list has no range covering point 6, because n stopped being live at point 4 and rbx now holds x. Reporting <optimized out> is correct; reporting the contents of rbx would be a confident lie. Note that the source still contains the declaration of n and it is still in lexical scope — scope and liveness are different things, and the debugger reports liveness.
Symptom two: the instruction pointer jumps around
You press "step" and the cursor goes to line 40, then line 12, then back to line 41, then into a function you did not call. Two transformations produce this, and both are doing exactly what they are supposed to.
Instruction scheduling reorders instructions to keep the pipeline busy — see [[instruction-scheduling]]. Two adjacent source lines with no dependency between them get interleaved instruction by instruction, so the line table alternates between them as you step. There is no single "current line"; there are several lines partially executed at once, and the debugger picks the one attached to the instruction at the program counter.
Inlining copies a callee's body into the caller — see [[inlining]]. Those instructions belong to the callee's source lines, so stepping through a caller walks into the callee's file and back out, repeatedly, with no call instruction anywhere. If the callee was itself inlined into three places, the same source line has three separate address ranges in three different contexts.
The inlining case has a second consequence that matters more than the stepping annoyance: stack frames disappear. An inlined function has no frame, so a naive backtrace shows the caller and not the callee, and a profiler attributes the callee's cost to the caller. The fix is that the debug information records inlined instances explicitly — DW_TAG_inlined_subroutine in DWARF — and the tool must ask for them. This is why addr2line -i exists and why omitting -i silently loses frames, and why a profiler that does not understand inline records produces flame graphs missing whole layers.
Tail calls are the third frame-eating transformation and the sneakiest: a call in tail position replaces the current frame rather than adding one, so the caller is genuinely gone from the stack. The trace is not wrong; the frame does not exist. -fno-optimize-sibling-calls disables it when you need the frame back.
static int scale(int v) { return v * 2; } /* line 3 */
int total(int *xs, int n) {
int s = 0;
for (int i = 0; i < n; i++)
s += scale(xs[i]); /* line 9 */
return s;
}int total(int *xs, int n) {
int s = 0;
for (int i = 0; i < n; i++)
s += xs[i] * 2; /* instructions attributed to BOTH line 3 and line 9 */
return s; /* no call, no frame for `scale` */
}Inlining preserves observable behavior whenever the callee's body can be substituted at the call site with the arguments bound — no recursion that would fail to terminate, no reliance on the callee having its own stack frame, and the same evaluation order for any side effects. It is a size-and-compile-time decision, not a correctness one, in every case where those hold.
It is not behavior-preserving where the callee's identity is observable: if its address is taken and compared, if the language guarantees a distinct stack frame (a coroutine boundary, a setjmp target), if it is marked noinline for a reason such as a stack-probing or security check that must remain a real call, or where recursion means substitution does not terminate. And even where it is legal, it destroys the callee's frame — which is why the debug format must record the inlined instance and why addr2line without -i silently drops it.
Symptom three: the breakpoint never fires
You set a breakpoint on a line, run, and it never hits — even though the surrounding code obviously executes. Every cause is a transformation that removed the correspondence between that line and any address.
The line has no code. It was folded to a constant, or eliminated as dead, or its work was [[loop-invariant-code-motion]]-hoisted out of the loop so the line executes once at the preheader rather than every iteration. The debugger silently moves the breakpoint to the next address with a line record, which is why it appears on a different line than the one you clicked.
The line has code but no `is_stmt` address. DWARF marks which addresses are reasonable breakpoint locations, and after scheduling most addresses for a line are mid-expression. If none of the line's addresses is marked, some debuggers will not place a breakpoint there.
The code was inlined elsewhere. A breakpoint on a line inside a small function needs to be placed at *every* inlined instance of it, which requires the debugger to enumerate inline records. Good debuggers do; a breakpoint set by address rather than by line does not.
The function was cloned or specialised. Constant-propagation cloning (-fipa-cp-clone) and monomorphization produce several copies of one source function, and a breakpoint on one copy leaves the others unhit — see [[monomorphization]].
The whole function was optimized away — devirtualized to something else, or proved unreachable.
The diagnostic is always the same and it takes one command: ask the line table whether the line has any addresses. objdump --dwarf=decodedline or llvm-dwarfdump --debug-line, then look for your line number. If it is absent, the breakpoint was never placeable and no amount of retrying will help.
- Breakpoint moved to a different line: your line had no address; the debugger picked the next one that did.
- Breakpoint hit far fewer times than expected: the work was hoisted out of the loop.
- Breakpoint hit in an unexpected function: the code was inlined and the address belongs to two source positions.
- Breakpoint never hit at all: folded, eliminated, cloned into a copy you did not break on, or the function was removed.
- Diagnose all of these with a decoded line table. The question "does this line have an address" is directly answerable.
Working with it rather than against it
llvm-dwarfdump --statistics exists precisely so that "debug info quality" can be measured per build rather than argued about. GCC and Clang both invest in it continuously and are not equally good at the same things; -Og is a GCC and Clang flag with no exact MSVC equivalent. Measure your own build rather than assuming a level of fidelity from experience with a different toolchain.The instinct on hitting these symptoms is to rebuild at -O0, and it is usually the wrong first move. It changes timing, which makes races and heisenbugs vanish; it changes frame layout, which makes stack-overrun bugs vanish; it can be too slow to reproduce the workload at all; and if the bug is a consequence of a legal optimization exploiting undefined behavior, it will simply not appear. Reach for narrower tools first.
`-Og` is the designed answer: optimizations chosen not to wreck debuggability. It is a different build, not a different debugger, and it is the right default for a development configuration.
Disable one thing. __attribute__((optnone)) or #pragma GCC optimize("O0") on one function, #[inline(never)] on one Rust function, -fno-inline or -fno-optimize-sibling-calls or -fno-omit-frame-pointer on one translation unit. Each keeps the rest of the program at production settings, which is what you want when reproduction depends on it.
Read the metadata directly instead of trusting the debugger's summary. llvm-dwarfdump --statistics reports what fraction of each variable's scope actually has a location, which tells you immediately whether "optimized out" is expected here or whether your build has poor debug-info coverage. info frame and info registers in gdb let you read the machine state when the source-level view has nothing to say.
Change technique, not build. Logging, [[reading-compiler-output]] to see what the compiler actually did, a sanitizer to find the undefined behavior, or rr-style record-and-replay to run the failure backwards. When the source-level view is degraded, the machine-level view is still complete, and reading the disassembly for one function is frequently faster than fighting the abstraction.
How it works
The steps, in the order the compiler takes them.
- Register allocation assigns each value a physical register only for its live range and reuses the register immediately afterwards, so the value is unrecoverable past its last use.
- Constant folding and copy propagation remove the computation entirely, leaving a variable with no storage anywhere in the program.
- Rematerialization recomputes a value at each use instead of keeping it live, so it has several definitions and no single location.
- Instruction scheduling reorders independent instructions, interleaving the address ranges of adjacent source lines in the line table.
- Inlining copies the callee's instructions into the caller, attributing them to the callee's source positions and recording the inlined instance in the debug information.
- Tail-call optimization reuses the current frame for the call, so the calling frame genuinely ceases to exist.
- Loop-invariant code motion moves a computation to the loop preheader, so the source line inside the loop executes once rather than per iteration.
- Function cloning and monomorphization produce several machine functions from one source function, each with its own address range for the same lines.
How it breaks
What the engineer observes when it goes wrong — not what goes wrong internally.
- A variable that is clearly in scope prints as
<optimized out>, and the engineer concludes the debug build is broken rather than that the value is genuinely gone. - A stack trace is missing three frames, and the missing functions are exactly the small ones that were inlined; the bug is attributed to the wrong function.
- A profiler shows all the time in one large function because every callee was inlined and the tool did not read inline records, so the flame graph has no useful layers.
- Stepping alternates between two source lines that look unrelated, and the engineer suspects a threading bug that does not exist.
- A breakpoint inside a loop fires once, because the statement was hoisted to the preheader, and the engineer concludes the loop is not running.
- Rebuilding at
-O0to investigate makes the failure disappear, and days are spent on a heisenbug that was a timing-sensitive race or a frame-layout-sensitive overrun. - A debugger prints a plausible wrong value because a pass left a stale location entry, and the investigation proceeds from a false premise.
When it helps
- Debugging production or production-like binaries, which are optimized, and where reproducing at
-O0is either impossible or changes the failure. - Performance investigations, where you need optimized code by definition and the source correspondence is the only way to attribute cost.
- Reading crash reports from the field, where inline records are the difference between blaming a helper function and blaming its caller.
- Understanding what the compiler actually did, since every symptom in this lesson is a transformation announcing itself.
When it hurts
- Early exploratory debugging of unfamiliar logic, where faithful stepping is worth far more than production fidelity — use
-Ogor a debug build. - Teaching and onboarding, where the degraded view actively misleads someone who does not yet know these transformations exist.
- When the debug information itself is poor: an old toolchain, an unusual target, or a language whose optimizer does not maintain metadata carefully, where the reported values may simply be wrong.
- When the temptation to disable optimization spreads from one function to the whole build, and the production configuration quietly stops being what you test.
What it costs
Every one of these is paid by something.
- Debugging the optimized build buys fidelity to what actually ships and pays a degraded source view: missing variables, jumping lines and unplaceable breakpoints.
-Ogbuys a usable stepping experience and pays a performance level nobody ships, so it is a third configuration rather than a substitute for either default.- Disabling optimization for one function buys a readable view of that function and pays a program that is no longer quite the one that failed — different inlining, different timing, different frame layout.
-fno-omit-frame-pointerbuys reliable stack walking for profilers and crash handlers and pays one general-purpose register plus a small, measurable performance cost.- Preserving inline records buys recoverable stack frames and pays debug-information size, which is already the largest part of a
-gbuild. - Investing compiler effort in metadata accuracy buys trustworthy optimized-build debugging and pays engineering in every pass, which is why coverage varies so much between toolchains.
What else you could do
What a different compiler or language does instead, and when that is better.
- Record-and-replay debugging (
rr, time-travel debuggers) lets you run the failure backwards from the crash, which sidesteps the "I cannot see the value now" problem by letting you go back to when it existed. - Logging and tracing: instrument the values you need, accepting that the instrumentation is itself an optimization barrier and changes what the compiler may do around it.
- Read the disassembly. When the source view is degraded, the machine view is complete, and
[[reading-compiler-output]]is frequently the faster path for a single function. - Sanitizers and dynamic analysis, which report at the point of the fault with a stack trace rather than requiring you to inspect state interactively.
- A managed runtime, where deoptimization can restore an interpreter frame with all values present on demand — see
[[deoptimization]]and[[on-stack-replacement]]. That is the design that solves this properly, and it requires the runtime to be able to reconstruct the unoptimized state, which an ahead-of-time compiler cannot.
See it for yourself
The flag, dump or tool that shows you this directly.
- gdb:
info locals,info args,info registers,info frame,info line *0xADDR, anddisassemble /sto interleave source with instructions.set print inline-frames on(usually the default) makes inlined frames visible in a backtrace. - lldb:
frame variable,register read,image lookup -va $pc(which shows the inlined function chain at an address), anddisassemble -mfor mixed source. - Check whether a line is breakpointable at all:
objdump --dwarf=decodedline ./bin | grep -w 47orllvm-dwarfdump --debug-line ./bin. If the line has no address, no breakpoint can be placed on it. - Measure the damage:
llvm-dwarfdump --statistics ./binreports per-variable location coverage — the fraction of each variable's scope that actually has a location. This is the objective answer to "is my debug info any good". - Recover inlined frames:
addr2line -e ./bin -f -C -i 0xADDR, orllvm-symbolizer --inlines. Omitting the inline flag is the single most common reason a stack trace is missing frames. - Narrow the optimization instead of disabling it:
__attribute__((optnone))or__attribute__((noinline))on one function in C/C++,#[inline(never)]in Rust,-fno-inline,-fno-optimize-sibling-calls,-fno-omit-frame-pointerper translation unit. - See what the compiler decided:
-Rpass=inlineand-Rpass-missed=inlinein Clang,-fopt-info-inlinein GCC. If the debugger is behaving strangely around a call, these tell you whether it is still a call.
Plausible wrong readings
Stated the way a confident engineer states them.
- "Optimized out means the debugger could not find the value." It means the value does not exist at this address. Nothing wrote it anywhere, because nothing needed it after its last use.
- "The variable is in scope, so it must have a value." Scope is a source-level property and liveness is a machine-level one. They stopped agreeing at register allocation.
- "The stack trace is wrong — that function is definitely on the stack." If it was inlined it has no frame, and if the call was in tail position the caller's frame is gone. Ask for inline frames explicitly before concluding the trace is broken.
- "I will just rebuild at -O0 to look at it." That changes timing, frame layout and inlining, and if the bug depends on any of those it will disappear. Try
-Ogor a single-functionoptnonefirst. - "The compiler is jumping around because the debug info is broken." Interleaved line records are the correct description of scheduled code. Two source lines really are executing at once.
Misconceptions
The claim, and what is actually true.
Go deeper
The same idea at increasing depth. Stop wherever it stops being useful.
overview
In optimized code the neat correspondence between source and machine breaks in three visible ways. A variable reads as "optimized out" because its value only ever lived in a register that has since been reused, or was folded away entirely — there is nothing to show you. The cursor jumps between lines because the compiler interleaved independent instructions and pasted other functions in via inlining, so several source lines are executing at once. And a breakpoint may never fire because that line has no machine code left. None of these is a bug; each is a specific transformation showing through.
practical
Do not immediately rebuild at -O0 — it changes timing and stack layout, and a race or an overrun will vanish. Instead: use -Og for a development build; put optnone or #[inline(never)] on the one function you need to see; add -fno-omit-frame-pointer if stacks are unreliable; and use addr2line -i or llvm-symbolizer --inlines so you stop losing inlined frames. When a breakpoint will not fire, decode the line table and check whether your line has an address at all — that takes one command and answers the question definitively.
advanced
The structural problem is that debug information describes a *relation* between source and machine that optimization makes many-to-many, while every debugger interface presents it as a function. One address maps to several source lines through inlining; one source line maps to several address ranges through cloning and scheduling; one variable maps to several locations over its lifetime, and to none over part of it. Every symptom in this lesson is that mismatch surfacing. The interesting comparison is with a managed runtime, which solves the problem by construction: a JIT keeps a deoptimization state map for every compiled frame, so it can reconstruct an interpreter frame with all values present on demand — see [[deoptimization]]. That works because the runtime kept the unoptimized representation available, which is exactly what an ahead-of-time compiler discards. The cost of the ahead-of-time model is paid here, once per debugging session, forever.
How much this depends on
Nothing in this domain is true of every compiler. These say how much.
-O2 inline aggressively, schedule across statements and reuse registers immediately, which produces all three symptoms described here; the same source at -O1 may produce none of them, and -Og is explicitly designed to avoid the transformations that hurt most. None of this is required behaviour — it is a description of GCC and Clang as they currently are.If you were asked this in an interview
- A debugger says a variable is optimized out. Give me three different reasons that can be true.
- A stack trace is missing frames you know are on the call path. What happened and how do you get them back?
- Why is rebuilding at
-O0often the wrong first move when a production bug is hard to inspect?
Connections
- Programming Languages & Runtime Internals — Deoptimization state maps that let a JIT reconstruct an unoptimized frame on demandA managed runtime solves the inspectability problem structurally by keeping enough state to rebuild an interpreter frame from an optimized one. How that mechanism works at runtime is owned there; why an ahead-of-time compiler cannot do the same, and what that costs every debugging session, is ours.
- Testing & Reliability Engineering — Reproducing a failure that only appears in the optimized configurationWhen a bug exists at
-O2and not at-O0, the investigation is a reproduction-strategy problem — narrowing the configuration, capturing a deterministic replay, deciding what evidence to collect from a build you cannot step through. That process is owned there; which transformations caused the observation is ours.