Why Runtime Information Helps
A static compiler must be correct for every type that could occur, every branch that could be taken and every target a call could reach. A JIT sees which ones actually occur, and specializing to the actual case is worth far more than any amount of extra analysis on the general one.
What can a compiler know at run time that it genuinely cannot know at build time, and why is that worth so much?
The program is bytecode annotated with a *profile*: per site, the set of operand and receiver types observed, the taken and not-taken counts of each branch, the call targets actually reached, and the number of times the site executed. That annotation is a new representation, not a decoration — it turns a set of possibilities into a distribution, and it exists to answer the one question static analysis cannot: not "what could happen here" but "what has happened here".
A profile licenses nothing on its own. Specializing to an observed fact preserves observable behavior only if the specialized code is guarded by a runtime check of that exact fact, and control transfers to a general path when the check fails. The precondition is therefore: for every fact taken from the profile, there must exist a cheap check that is *sufficient* — it must not merely correlate with the assumption but establish it — and a correct fallback. A fact that cannot be cheaply and soundly checked cannot be speculated on, no matter how consistent the profile is.
Key points
- Three facts vary between executions and are exactly the ones a code generator wants: the actual types, the actual branch bias, and the actual call targets.
- A profile turns a set of possibilities into a distribution, which is a different representation with different uses — not just more detail on the same one.
- Observation is evidence, never proof; every use of it for semantics requires a runtime check and a fallback.
- A fact is only speculatable if it can be checked cheaply and soundly, which rules out most interesting properties and admits type and shape tests.
- PGO can use a profile for layout and heuristics; a JIT can use it for semantics, because only a JIT can check the assumption at the moment it matters.
- Skew is what makes a profile valuable. A site that has seen one thing is informative; a site that has seen forty is informative that specialization will not pay.
- Profiles describe the past and programs change phase, so the information decays and the system must be able to notice.
Three things a static compiler cannot know
The gap is specific and it is worth naming precisely, because "the JIT knows more" is not an argument. There are three facts that vary between executions of the same program text, and all three are exactly the facts a code generator most wants.
First, the actual types. In a dynamically typed language, a + b must handle every pair of types the language allows — small integers, large integers, floats, strings, objects with user-defined addition. A static compiler emits the general operation. A JIT that has seen ten thousand small-integer additions at that site emits an integer add and a check.
Second, the actual branch bias. A static compiler can guess, and guessing is what [[profile-guided-optimization]] exists to replace, but without a profile it is guessing. A JIT has the counts. Knowing that a branch is taken 99.99% of the time changes code layout, which paths get inlined into, and where the cold code goes — and being wrong about it costs a branch mispredict on the hot path.
Third, the actual call targets. A virtual call, an interface call, a function-valued variable and a dynamically dispatched method all look unresolvable at build time. In practice the overwhelming majority of such sites see exactly one target. A JIT can see that, inline through it, and put a guard where the dispatch used to be — which is what [[devirtualization]] becomes when the evidence is a profile rather than a proof.
| Question at a site | Static compiler | PGO build | JIT |
|---|---|---|---|
| What types occur? | Every type the language permits; emit the general operation | Sometimes recorded, if the instrumentation captured types at all | Exactly what this execution has seen, per site, updated live |
| Which way does this branch go? | A heuristic — loop back edges taken, null checks not — and otherwise a guess | Measured counts from a training run | Measured counts from this run, including a shift mid-execution |
| Which function does this call reach? | Unknown unless the whole program proves it; often devirtualizable only for final or sealed types | Recorded target distribution from training | The observed targets, with the option to inline the dominant one behind a guard |
| Is this value ever null?implementation | Cannot know; emit the check | Rarely captured | If it never has been, hoist the check and guard |
| How hot is this code? | Unknown; optimize everything to the same level | Known for the training workload | Known for the workload actually being served |
Evidence is not proof, and the difference is a guard
The temptation, reading the table above, is to think a JIT *knows* the type. It does not. It knows the type of every value that has flowed through this site so far, which is a statement about the past. The eleventh call may pass a string. Specialization on that basis is a bet, and a bet needs a settlement mechanism.
That mechanism is the whole reason the rest of the module exists. [[speculative-optimization]] is the bet. [[guards]] is the cheap check that settles it. [[deoptimization]] is what happens when it settles badly. And the requirement that a fact be *checkable cheaply* is a real constraint on what can be speculated: "this argument is a small integer" is one comparison, "this list is sorted" is not, so one is speculated on routinely and the other never.
This is also what separates a profile-guided ahead-of-time build from a JIT, and the difference is not degree. A PGO build bakes the training profile into code that must be correct without any runtime check, so it can only use the profile for *layout and heuristics* — which path is likely, what to inline, where to place cold code. A JIT can use the profile for *semantics*, because it can check. That is why PGO gains are typically in the tens of percent and JIT specialization gains on dynamic code can be multiples — see [[pgo-tradeoffs]] for the other side of that comparison.
result = generic_add(a, b) // dispatches on the runtime types of a and b: // int+int, int+float, string+string, object with user-defined add, ...
guard is_small_int(a) else deoptimize@bytecode_offset_14 guard is_small_int(b) else deoptimize@bytecode_offset_14 result = machine_add(unbox(a), unbox(b)) // overflow of the machine add also exits to the general path
Only if both guards are checked before any observable effect of the fast path, the checks are sufficient rather than merely indicative of the operand representation, the overflow case exits to a path that produces the language-defined result, and the interpreter state at bytecode offset 14 is fully reconstructible from the optimized frame at each guard. The profile that motivated the specialization is not part of the legality argument at all — it justifies the choice, never the correctness.
The site is polymorphic and the guard is placed after work that is already observable — say, after a user-visible property read on a that could run a getter. Then failing the guard cannot cleanly resume at bytecode offset 14, because the getter has already run and the general path would run it a second time. The same specialization is also illegal if the language defines integer addition to promote on overflow and the fast path silently wraps: the guard checked the operands and not the result.
Where the information runs out
The argument has an edge, and it is important to know where it is. Runtime information helps in proportion to how *skewed* the actual behaviour is. A call site that has seen one target is enormously informative; a site that has seen forty is informative in the opposite direction — it tells the compiler not to bother, which is why megamorphic sites in [[inline-caches]] fall back to a general lookup rather than accumulating cases forever.
The information also decays. A profile describes the past, and programs change phase: a server that spent its first minute parsing configuration and the next hour serving requests has a profile whose early entries describe a workload that will never return. Engines handle this by recompiling, by ageing counters and by discarding profiles on deoptimization, and none of those is free.
And the information can be actively misleading. A benchmark that warms up on one input shape produces beautiful specialized code for a workload nobody runs — which is not a JIT problem but a measurement problem, and one that [[jit-costs]] treats as a first-class hazard rather than an inconvenience.
- Monomorphic — one observed type or target. Maximum information; specialize, inline, guard, done.
- Polymorphic — a handful. Still useful: a short guarded chain of cases, or inlining the dominant one and dispatching for the rest.
- Megamorphic — many. The profile now says "do not specialize", and honouring that is as valuable as specializing when it says the opposite.
- Phase change — the distribution shifts mid-run. The old profile is not just useless, it is wrong, and code compiled from it will deoptimize repeatedly until something re-profiles.
- Unrepresentative warmup — the distribution is an artifact of the benchmark harness rather than the workload. The engine behaves correctly and the number is meaningless.
How it works
The steps, in the order the compiler takes them.
- Attach a small feedback slot to each site the compiler might want to specialize: arithmetic operations, property accesses, calls, and comparisons.
- On each execution of the site, record what was seen — the operand representation, the receiver shape, the resolved target — and either confirm the existing entry or widen the record.
- Count executions per function and per loop back edge, so hotness and type feedback are collected by the same instrumentation pass.
- When a function is queued for compilation, read the feedback slots and classify each site as monomorphic, polymorphic or megamorphic.
- For monomorphic and small polymorphic sites, emit the specialized operation preceded by a guard on the recorded fact; for megamorphic sites, emit the generic operation and do not guard.
- Record, alongside each guard, the bytecode offset and the state map needed to resume interpretation there.
- Age or reset the feedback when code deoptimizes, so that a recompile does not immediately reinstate the assumption that just failed.
How it breaks
What the engineer observes when it goes wrong — not what goes wrong internally.
- A site that is monomorphic during warmup becomes polymorphic under real traffic, and the function deoptimizes on the first request of the new shape — a latency spike that appears minutes after deployment and cannot be reproduced locally.
- A guard checks something correlated with the assumption rather than the assumption itself, and a value that passes the check takes a fast path that is wrong for it. The result is a wrong answer with no diagnostic anywhere.
- Type feedback is collected but the counter and the feedback age at different rates, so a function is compiled hot with a profile that reflects an execution phase that has ended.
- A microbenchmark drives one shape through a site ten million times, and the measured "language performance" is the performance of code specialized to a workload that does not exist.
- Feedback slots are attached per site and the site is inside an inlined callee, so two callers with different shapes pollute one another's profile and neither gets specialized.
- A profile is reset on every deoptimization, and a site that legitimately sees two shapes cycles between specialize, deoptimize and reset forever.
When it helps
- Dynamically typed languages, where the distance between "types the language permits here" and "types that occur here" is largest.
- Object-oriented code with deep interface hierarchies, where nearly every call is virtual in the source and nearly every call site is monomorphic in practice.
- Long-running services whose traffic is stable enough for a profile to describe it, and long-lived enough to amortize the compilation.
- Any place where the general operation is dramatically more expensive than the specialized one — boxed arithmetic, dictionary-based property lookup, reflective dispatch.
- Deciding *not* to optimize. Knowing a site is megamorphic saves the compiler from a speculative inline that would have deoptimized immediately.
When it hurts
- Genuinely polymorphic code, where the profile is flat, no bet is worth making, and the instrumentation cost was paid for nothing.
- Programs with distinct execution phases, where a profile collected in one phase actively misleads compilation for the next.
- Short runs, where the profile is thin: a handful of observations is not a distribution, and specializing on it is closer to guessing than to measuring.
- Measurement and benchmarking, where profile-dependent behaviour makes results depend on execution order, input order and iteration count — the harness itself becomes a variable.
- Security-sensitive reasoning, where "this has always been true" is exactly the shape of assumption an adversary constructs inputs to violate.
What it costs
Every one of these is paid by something.
- Collecting type feedback buys the specialization that makes a JIT worth having and pays a store and a compare inside the interpreter on every execution of every instrumented site, which is a direct tax on the tier that has not been optimized yet.
- Feedback slots buy per-site precision and pay in memory that scales with the number of sites in the loaded program, not with the number that are hot.
- Specializing on a skewed profile buys large speedups on the common case and pays with deoptimization storms when the distribution moves, plus the compile time spent producing code that was thrown away.
- Widening a site's record to polymorphic buys correctness under varied input and pays by making the guard chain longer and the inlining decision worse, until the site is written off as megamorphic entirely.
- Using the profile for semantics rather than layout buys multiples instead of percentages, and pays with the entire guard-and-deoptimize apparatus, which a PGO build does not need at all.
What else you could do
What a different compiler or language does instead, and when that is better.
- Collect the profile in a training run and bake it into an ahead-of-time build: no runtime instrumentation, no guards, and the profile is from a different execution than the one being served —
[[profile-guided-optimization]]. - Prove instead of observe. Whole-program analysis can sometimes establish that a call site has exactly one possible target, which needs no guard at all —
[[whole-program-optimization]]and[[link-time-optimization]], at the price of needing the whole program. - Remove the uncertainty at the language level. Static types, sealed classes, final methods and monomorphization make the fact a compile-time property rather than a runtime observation —
[[monomorphization]]. - Let the programmer assert it. Type annotations that the runtime checks once at a boundary, or explicit specialization directives, move the decision from inference to declaration and make the failure mode a clear error instead of a silent slowdown.
- Do not specialize at all and spend the effort on the general path — better data structures for property lookup, a faster generic arithmetic routine. Less spectacular, and it never deoptimizes.
See it for yourself
The flag, dump or tool that shows you this directly.
- V8:
--trace-icprints inline-cache state transitions per site — monomorphic to polymorphic to megamorphic — which is the type-feedback story in one stream. - V8:
--trace-deoptnames the guard that failed and the bytecode offset it exited at, which tells you which observed fact stopped being true. - HotSpot:
-XX:+PrintInliningreports refusals with reasons, and "not inlineable" versus "too big" versus "not reached" distinguishes a profile problem from a budget problem. - .NET: set
DOTNET_TieredPGO=1and compare against0on the same workload; the difference is exactly what dynamic profile information is worth for that program. - For the static side of the same question, build with and without
-fprofile-usein GCC or Clang and diff the generated code — the delta is what a profile buys a compiler that cannot check anything at run time.
Plausible wrong readings
Stated the way a confident engineer states them.
- "The JIT knows the types, so a dynamic language can be as fast as a static one." It knows what the types have been. Every use of that knowledge carries a check, the checks cost something, and code that is not type-stable collects none of the benefit.
- "If a profile says a branch is always taken, the compiler can delete the other side." It can make the other side cold, out of line, and expensive to reach. It cannot delete it, because "always so far" is not "always" — unless the runtime can guarantee the condition globally and invalidate the code if that changes.
- "Profile-guided optimization and JIT specialization are the same thing at different times." They differ in kind. PGO cannot check its assumptions at run time, so it may only use the profile for heuristics; a JIT may use it for semantics because it can guard.
- "More profiling data is always better." Instrumentation is a tax on the unoptimized tier, feedback slots consume memory per site, and a profile that describes a finished execution phase is worse than no profile.
- "A megamorphic site means the code is badly written." It often means the code is genuinely general — a serializer, a dispatcher, a framework hook. The right response is to stop speculating there, not to restructure the program around the compiler.
Misconceptions
The claim, and what is actually true.
Go deeper
The same idea at increasing depth. Stop wherever it stops being useful.
overview
A compiler that runs before the program does has to handle everything the program might do. A compiler that runs while the program is executing can see what it is actually doing — which types are showing up, which way the branches go, which functions the calls really reach — and write code for that case specifically, with a quick check in front in case things change.
practical
The practical consequence is that consistency is worth more than cleverness. A function called with integers stays fast; the same function called with integers and occasionally with strings loses the specialization for everyone. Keep object shapes stable, avoid adding properties to objects after construction, and do not write one generic dispatcher that every call in the system funnels through — that site goes megamorphic and takes the inlining of everything above it with it. When something is unexpectedly slow, --trace-ic or the equivalent will usually show a site that went polymorphic at exactly the moment the slowdown started.
advanced
The deep framing is that a profile converts a *may* analysis into a *must, probably* analysis, and the entire engineering problem is converting the "probably" back into soundness at an affordable price. That price is a checkable predicate. Which is why the facts engines speculate on are all cheap predicates over representation — is this a small integer, is this object of this hidden class, is this call target still this function — and never expensive semantic properties, however stable the profile is. It also explains an asymmetry that looks arbitrary at first: engines will happily speculate that a property is a constant, because a global invalidation mechanism can watch for the write, but will not speculate that a loop bound is small, because there is no cheap check that establishes it in advance. The available checks, not the available observations, are what bound what a JIT can do with what it knows.
How much this depends on
Nothing in this domain is true of every compiler. These say how much.
If you were asked this in an interview
- Name three things a JIT knows that a static compiler cannot, and say what each one is worth.
- Why can profile-guided optimization not specialize the way a JIT does, given that it also has a profile?
- A site has seen forty different receiver types. What should the compiler do, and why is that the right answer rather than a failure?
- What property must a fact have for a compiler to be allowed to speculate on it?
Connections
- Programming Languages & Runtime Internals — Hidden classes, object shapes and the property-lookup path that type feedback is actually describingThe "type" a JavaScript engine records at a site is a hidden class — a runtime-managed description of an object's layout that the object model creates and mutates. What the profile records is meaningless without the object model that produced it, and that model is the runtime's design, not the compiler's.