Deoptimization
A guard fails, and execution must continue correctly in code that assumed nothing — which means reconstructing an interpreter frame from an optimized one. Keeping that reconstruction possible is a standing obligation, and the obligation, not the mechanism, is what this lesson is about.
When an optimized function's assumption turns out to be wrong, how does the program carry on correctly — and what did the compiler have to give up to make that possible?
Two descriptions of one activation, plus a map between them. The optimized frame is concrete: values in registers, in compiler-chosen stack slots, inlined callees with no frames of their own, and some values that do not exist at all because they were folded away. The interpreter state is abstract: an instruction pointer at a bytecode offset, an operand stack, and named slots, exactly as [[vm-state-model]] describes. The *state map* is the third thing, and it is the real representation this lesson is about — a compiler-emitted description, per deoptimization point, of how to compute the abstract state from the concrete one.
The optimizer may perform a transformation only if, at every deoptimization point that the transformation is visible to, the abstract machine state remains derivable from the optimized frame. This is a genuinely stronger condition than preserving observable behavior: a value that no longer affects any output may still be named by a state map, and deleting it is then illegal even though nothing observes it. Resumption itself is legal only if the reconstructed state is exactly the state the interpreter would have had at that bytecode offset had it executed the prefix — no effect performed twice, none skipped, and the operand stack at the depth the offset expects.
Key points
- Deoptimization reconstructs interpreter state from an optimized frame and resumes at a recorded bytecode offset; the compiled code is then usually discarded and the profile widened.
- One optimized frame can expand into several interpreter frames, because inlined callees never had frames of their own.
- The state map is compiler-emitted metadata, and producing it is what makes this a compiler concern rather than a runtime one.
- Each state map is an extra observer of intermediate state, so the optimizer must preserve values that nothing in the program observes.
- That constraint is genuinely stronger than the as-if rule, and it is why a JIT declines transformations an ahead-of-time compiler performs routinely.
- Rematerialization is the standard escape: describe how to recompute a value on the cold path instead of keeping it on the hot one.
- Assumptions about global state are invalidated by dependency rather than by a guard, and may need to unwind activations that are currently running.
- The loud failure is a deoptimization storm; the quiet one is a wrong state map, which produces a wrong answer with nothing to point at.
What has to happen when a guard fails
The mechanism first, because it is short. A guard fails and control leaves for an out-of-line stub. The stub reads the state map for that deoptimization point, which names every value the interpreter would need — locals, operand-stack entries, and, if callees were inlined, the same for each of the frames that were never physically created. It reads those values out of registers and stack slots, or recomputes them where the map says they were rematerializable, and writes them into a fresh interpreter frame — or several, one per inlined callee, because a single optimized frame can correspond to a whole stack of virtual ones.
It sets the interpreter's instruction pointer to the bytecode offset recorded with the guard, replaces the optimized activation on the stack with the reconstructed frames, and resumes interpretation. The compiled code is typically marked not-entrant so no further calls reach it, the site's profile is widened so the next compilation makes a different bet, and a deoptimization count is recorded against the function.
None of that is conceptually difficult, and all of it is expensive relative to the guard: a frame reconstruction is on the order of thousands of times the cost of the comparison that triggered it. But the expense is not the lesson, because deoptimization is rare by design. The lesson is what had to be true, continuously, everywhere in the optimized function, for that reconstruction to be possible at all.
| In the optimized frame | In the reconstructed interpreter state | How the map bridges it |
|---|---|---|
| A local in a machine register | A named slot in a frame | Register number, plus which slot of which virtual frame |
| A value on the operand stack conceptually, nowhere physically | An entry at a specific stack depth | A description of where to read it, or a recipe to recompute it |
| An inlined callee | Its own complete frame, with its own instruction pointer | One virtual frame descriptor per inlined call, chained |
| An object that escape analysis never allocatedimplementation | A live heap object the program can reach | A rematerialization recipe: allocate it now, from these field values |
| A constant folded from a speculated invariant | Whatever the value actually is | A constant in the map, valid because the invariant held up to this point |
| A value the optimizer wanted to delete | Still needed at this offset | It may not be deleted — this row is the constraint |
The obligation, which is the actual subject
Read the last row of that table again, because it is the lesson. An ordinary optimizer's rule is [[as-if-rule]]: any transformation is legal that preserves the program's defined observable behavior. Under that rule, a value nothing observes may be deleted, a computation may be sunk past code that does not depend on it, and an allocation whose object never escapes need not happen.
A speculative compiler cannot use that rule unmodified, because each state map is effectively an additional observer of the intermediate state. If a map at some guard names local t, then t must be recoverable at that guard — even if no output of the program depends on t, and even if the guard fails once in a billion executions. The optimizer is therefore answerable to two things: the program's semantics, and a set of metadata descriptions that must remain satisfiable at every point where a speculation might unwind.
This is why deoptimization belongs to a compilers domain rather than to a runtime one. The stack-rewriting machinery is runtime work. The state maps are compiler output, produced by the same passes that do the optimizing, and the requirement to produce them changes what those passes are allowed to do. It is one of the clearest cases in the whole domain of a runtime mechanism reaching backwards into the middle-end and taking freedom away.
%t = mul %a, %a ; t is never used again in optimized code guard is_int(%b) else deopt@offset_22 ; state map at offset_22 names local t %r = add %b, 1 return %r
guard is_int(%b) else deopt@offset_22 %r = add %b, 1 return %r ; the multiply removed — nothing observed t
This removal is legal only if no state map at any deoptimization point reachable from the definition names t, or if the map instead carries a *rematerialization recipe* — a description sufficient to recompute t from values that are still live at that point, here %a * %a. Rematerialization is the standard escape: it lets the optimizer delete the computation from the fast path while keeping the value derivable on the cold path, at the cost of a more complex map and of %a itself remaining live.
The state map at offset_22 names t and %a has also been overwritten or was itself folded away. Then failing the guard resumes the interpreter at bytecode offset 22 with a garbage or stale value in the slot that t should occupy, and the program continues from a state it was never in. The symptom is not a crash at the guard — it is a wrong value appearing later, in interpreted code, in a function that looks entirely innocent.
Two ways to be wrong, and two ways to find out
A speculation can be invalidated locally or globally, and the two need different machinery. A local violation is a value that failed its check — this operand is not an integer, this object is not that shape. It is detected by a guard, on the executing thread, at the moment it matters, and it affects one activation.
A global violation is a change to something the compiled code assumed about the world: a method was redefined, a class was loaded that provides a second implementation of an interface the compiler had devirtualized, a field the compiler treated as constant was written. There is no per-execution check for these — the whole point of assuming them was to avoid one — so the runtime maintains a dependency from the compiled code to the fact, and the mutation triggers invalidation directly.
Global invalidation is the harder case, because the code being invalidated may be *currently executing*, possibly on several threads, possibly deep inside a loop. Marking it not-entrant stops future calls; the running activations must be dealt with too, which is why engines have the notion of forcing a deoptimization at the next safepoint rather than at a guard. HotSpot's class-hierarchy dependencies are the canonical example: a single class load can invalidate a large number of compiled methods across the process.
- Guard failure — local, detected per execution, affects the current activation. Cheap to detect, and the common case.
- Dependency invalidation — global, triggered by a mutation elsewhere, may affect many compiled methods at once and code that is currently running.
- Eager deoptimization — invalidate and unwind running activations at the next safepoint, needed when the assumption is already false and cannot be allowed to persist.
- Lazy deoptimization — mark not-entrant so no new calls arrive, and let running activations finish, which is only sound when the stale assumption cannot cause incorrect behavior for the remainder of those activations.
- Debugger-triggered — attaching a debugger, or setting a breakpoint in an inlined callee, forces deoptimization because the inlined frames must be made real before they can be inspected.
When it goes wrong: storms and silence
There are two bad outcomes and they look nothing alike. The loud one is a deoptimization storm: a function is compiled, deoptimizes, is recompiled from a profile that has not learned enough, and deoptimizes again. Throughput collapses below the interpreted baseline because every cycle costs a compile plus a reconstruction, and the application's own CPU profile shows nothing wrong because the time is in the compiler. Engines defend with per-function deoptimization counts, profile widening on failure and eventual permanent refusal to optimize — and when the defence does not engage, this is what a mysteriously slow production process often turns out to be.
The quiet one is worse. If a state map is *incomplete or wrong* — it omits a live value, names the wrong location, or describes a frame the optimizer subsequently changed — the reconstruction succeeds and produces a state the program was never in. Execution resumes in the interpreter with a stale local, and the wrong answer surfaces somewhere else entirely. There is no exception, no trace entry, and nothing pointing at the guard. This is the failure that makes differential testing of optimized against interpreted execution non-negotiable in any engine that ships this machinery — see [[differential-testing]].
The asymmetry is worth naming: the mechanism failing loudly is an operational problem you can find in a profile, and the metadata being subtly wrong is a [[miscompilation]] you may never attribute. Which is exactly why the constraints on the optimizer are enforced structurally — a guard is modelled as a use of every value in its state map, so ordinary liveness analysis keeps them alive automatically rather than by anyone remembering to.
How it works
The steps, in the order the compiler takes them.
- At each guard, record the bytecode offset to resume at and a state map naming every value the interpreter would need there, including one virtual frame descriptor per inlined callee.
- Model the guard as a use of every value the state map names, so ordinary liveness analysis keeps them recoverable without any pass needing special knowledge.
- Where keeping a value live is too expensive, record a rematerialization recipe instead, and keep live only the inputs that recipe needs.
- Emit the guard as a branch to an out-of-line stub, so the reconstruction sequence is outside the hot path's instruction footprint.
- On failure, read the state map, gather the values from registers, stack slots and recipes, and reallocate any objects that escape analysis eliminated but the reconstructed state can reach.
- Build one interpreter frame per virtual frame descriptor, set each frame's instruction pointer to its recorded offset, and replace the optimized activation on the stack with the chain.
- Mark the compiled code not-entrant so new calls do not reach it, widen or invalidate the feedback that motivated the failed speculation, and increment a deoptimization count for the function.
- For global invalidation, maintain a dependency from compiled code to the fact assumed, and on violation either mark not-entrant, or force running activations to deoptimize at the next safepoint when the stale assumption cannot be tolerated.
How it breaks
What the engineer observes when it goes wrong — not what goes wrong internally.
- A state map omits a live value and reconstruction resumes the interpreter with a stale local. The wrong answer appears in unrelated interpreted code with no error, no trace entry and no connection to the guard.
- A deoptimization storm: compile, deopt, recompile, deopt. Throughput falls below the interpreted baseline, a compiler thread saturates a core, and the application profile shows nothing because the time is not in application code.
- An optimization deletes a value a state map still names, and the resulting reconstruction reads whatever occupies that register now — presenting as memory corruption in a memory-safe language.
- A guard fails inside a deeply inlined region and the reconstruction produces frames in the wrong order or with the wrong offsets, so a stack trace shows a call chain the program never made.
- A dependency is not registered for an assumption about global state, and redefining a method leaves compiled code running with an assumption that is already false — a correctness bug reachable only by programs that mutate at run time.
- Eager invalidation unwinds a long-running activation at a safepoint deep inside a loop, and the loop restarts from a much slower tier with no on-stack replacement available to rescue it.
- Deoptimization is triggered by attaching a debugger, so the performance being investigated cannot be reproduced while the investigation is under way.
When it helps
- It is the precondition for speculation. Without a correct fallback, none of
[[speculative-optimization]]is legal, so the mechanism is what buys every gain in the module. - Handling genuinely rare cases without paying for them: an exception path, an overflow, an unusual type can be excluded from the compiled code entirely and reached by deoptimizing.
- Runtime mutability. Languages that permit redefining methods, adding fields or loading classes late can still be optimized aggressively, because a violating change can invalidate the affected code.
- Debugging support: forcing a deoptimization is how an engine makes inlined frames real so a debugger can inspect them.
- Adapting to phase changes — code compiled for one workload can be discarded and recompiled when the workload moves, rather than being wrong or permanently suboptimal.
When it hurts
- Latency-sensitive paths, where a reconstruction plus a drop to a lower tier landing inside a request is a large, rare and effectively unpredictable spike.
- Workloads that trigger it repeatedly, where the cost of the mechanism exceeds every benefit speculation provided.
- Optimizer effectiveness generally: the obligation to keep state maps satisfiable forbids transformations that would otherwise be legal, everywhere, whether or not any guard ever fires.
- Memory footprint, since state maps are emitted for every deoptimization point in every compiled method and can rival the code itself in size.
- Debugging and profiling, where inlined frames must be materialized to be inspected and the act of inspecting changes what is executing.
What it costs
Every one of these is paid by something.
- Deoptimization buys the legality of every speculation in the module and pays by making each state map an observer of intermediate state, which removes optimizer freedom uniformly across the compiled function.
- Detailed state maps buy the ability to deoptimize at many points — and therefore to speculate densely — and pay in metadata size, which for some engines rivals the generated code.
- Rematerialization buys back the deleted computation on the fast path and pays with more complex maps, longer live ranges for the recipe's inputs, and a slower reconstruction.
- Eager global invalidation buys immediate correctness after a mutation and pays by unwinding activations that were running perfectly well, including long-running loops that will now restart slowly.
- Discarding compiled code on deoptimization buys a fresh start with a corrected profile and pays the entire compile cost again, which is exactly the loop that becomes a storm when the correction does not help.
What else you could do
What a different compiler or language does instead, and when that is better.
- Do not speculate, and need no fallback. The generic code is always correct and always slower —
[[interpreter-performance]]and the general paths it describes. - Speculate only where a cheap general path can be branched to inline, without unwinding the frame — a two-sided
ifrather than a guard with an exit. Much simpler, and it forecloses inlining and cross-region optimization because both paths must remain in the function. - Keep the general case as a slow path within the compiled code and never leave the compiled frame at all. This is what a polymorphic inline cache effectively does, which is why
[[inline-caches]]is a smaller bet with a much cheaper failure. - Prove instead of assuming: whole-program analysis, sealed hierarchies and static types remove the need for both guard and fallback —
[[whole-program-optimization]],[[monomorphization]]. - Recompile rather than deoptimize when the violation is detected between activations rather than during one — sound only when no running activation depends on the invalidated assumption.
See it for yourself
The flag, dump or tool that shows you this directly.
- V8:
--trace-deoptprints the deoptimization reason, the function and the bytecode offset; the reason strings are a direct catalogue of what the engine speculated on and how it was wrong. - V8:
--print-opt-codeincludes the deoptimization exits and their associated data, which makes the "many exits off one linear fast path" shape visible. - HotSpot:
-XX:+PrintCompilationshowsmade not entrantandmade zombietransitions, which are the code being invalidated and then reclaimed. - HotSpot debug builds:
-XX:+TraceDeoptimizationnames the reason and action per event, and-XX:+PrintCompilation2adds timing so a storm is visible as a repeating pattern. - JITWatch reconstructs which inlining decisions produced the frames a deoptimization has to rebuild, which is the clearest way to see one optimized frame corresponding to several virtual ones.
- .NET: the
Microsoft-Windows-DotNETRuntimeETW provider emits method rejit and tiering events;dotnet-tracecollects them and a repeating pattern for one method is the storm signature.
Plausible wrong readings
Stated the way a confident engineer states them.
- "Deoptimization means the JIT made a mistake." It means an assumption stopped holding, which was always a possible outcome. The mechanism working is the system behaving as designed; the mechanism working *repeatedly for the same function* is the problem.
- "Deoptimization is expensive, so engines avoid it." It is expensive and rare, and its expense is not the interesting cost. The interesting cost is paid continuously by the optimizer, in transformations it may not perform because a state map must stay satisfiable.
- "The interpreter just picks up where the compiled code left off." It picks up at a bytecode offset in a frame that had to be constructed, possibly several frames, possibly including objects that had to be allocated because escape analysis had removed them.
- "If a value is dead, the compiler can remove it." Only if no state map names it and no rematerialization recipe is needed. Deadness in the ordinary sense is not sufficient in a speculative compiler, which is the single most surprising consequence of this machinery.
- "A deoptimization is like an exception." An exception is defined program behavior with defined semantics. A deoptimization is invisible to the program: the same computation continues, in different code, with the same result.
Misconceptions
The claim, and what is actually true.
Go deeper
The same idea at increasing depth. Stop wherever it stops being useful.
overview
Fast compiled code assumes things and checks them. When a check fails, the program cannot keep running that code — so the system rebuilds the situation the slow interpreter would have been in at that exact point, drops the fast version, and carries on interpreting. The program never notices; only the speed changes.
practical
Two things to do with this. First, when performance falls off a cliff and stays there, check for deoptimization before anything else: --trace-deopt or -XX:+PrintCompilation will show a function that went not entrant and never came back, and that single line usually explains the whole regression. Second, the fix is almost always to stop violating the assumption rather than to fight the engine — make types at a site consistent, stop adding fields to objects after construction, avoid funnelling every call through one generic helper. Repeated deoptimization of the same function is a signal about your code, not about the JIT.
advanced
The structural claim worth carrying away is that deoptimization inverts the usual direction of influence in a compiler. Normally the front end constrains the middle end, which constrains the back end, and the runtime consumes whatever comes out. Here a runtime mechanism reaches all the way back into the optimizer and takes freedom away from it, uniformly, in code that will almost never exercise the mechanism. The optimizer must serve two observers — the program's defined behavior, and a set of metadata descriptions that must remain satisfiable — and only the first is what "optimization is legal if it preserves observable behavior" was ever about. Everything engines do around this is an attempt to buy the freedom back: rematerialization instead of liveness, compact map encodings, restricting deoptimization points to a chosen set rather than every guard, and object reallocation so that escape analysis remains usable. It is the clearest example in the domain of a correctness obligation determining a performance ceiling, and it is why the interesting question about a speculative compiler is not what it can assume but what it can afford to describe.
How much this depends on
Nothing in this domain is true of every compiler. These say how much.
src/compilers/sim/vm.ts — function, instruction pointer, slots, stack base — is exactly the shape a state map would have to be able to produce.If you were asked this in an interview
- Walk me through what happens between a guard failing and the program continuing correctly.
- Why can a JIT not delete a value that nothing observes?
- One optimized frame, three interpreter frames. How did that happen and what does the compiler have to have recorded?
- How would an assumption about a redefinable global be invalidated, given that there is no guard for it?
- A production process is using a whole core in the compiler and making no progress. What is your first hypothesis?
Connections
- Programming Languages & Runtime Internals — Safepoints, stack rewriting and code invalidation: performing the swap on a live stack while collectors and other threads are runningThe compiler produces the state map; actually replacing an activation — atomically with respect to garbage collection, stack walking and other threads, and at a point where every thread can be brought to a known state — is runtime machinery whose safety argument is about runtime invariants entirely.
- Testing & Reliability Engineering — Differential testing of optimized against interpreted execution, and fuzzing the deoptimization paths specificallyAn incorrect state map produces a wrong answer with no diagnostic, which no ordinary test suite is likely to reach because it only manifests when a rare guard fails. Systematically forcing deoptimization at every point and comparing results is the only practical defence, and the technique belongs to testing rather than to compilers.