Whole-Program Optimization
Seeing every function at once turns three transformations from impossible to routine — cross-module inlining, devirtualization and dead-function elimination — by supplying the one thing separate compilation deliberately withheld: the rest of the program.
What can a compiler do if it can see the whole program that it cannot do one file at a time?
The entire program as a single module: one call graph spanning every function, one namespace of definitions, one type hierarchy known to be complete. This representation exists to answer a question no single compilation unit can even phrase — "is this the only definition, the only caller, the only implementation?" — because a unit compiled alone must assume the answer is no.
An interprocedural transformation is legal only where the visible program really is the whole program. Deleting an unreferenced function requires that nothing outside the compiled set can name it: not an exported symbol, not a dynamic lookup by name, not a function pointer whose value escaped to code the compiler cannot see. Devirtualizing a call requires that the set of possible implementations is closed — no subclass can arrive later through dynamic loading, and no interposition may substitute a different definition at load time. When either premise is false, the transformation is a miscompilation, and the premise is a property of the linking and loading model rather than of the source.
Key points
- A separately compiled unit must treat every external call as an unknown black box; whole-program optimization removes that assumption by removing the boundary.
- Cross-module inlining is the engine — devirtualization, dead-function elimination and interprocedural constant propagation are largely what it enables.
- The costs are structural, not incidental: parallelism, incrementality, peak memory and stack-level debuggability were all being bought by the boundary.
- Devirtualization's legality is a property of the linking and loading model, not of the source: any mechanism that can introduce an implementation later invalidates it.
- Peak memory in one process is the limit teams hit first, and it is why the practical answer is usually a partitioned scheme rather than a true whole-program view.
- On a program that links much more library than it uses, dead-function elimination is where binary size comes from.
What the boundary was hiding
Separate compilation, from [[separate-compilation]], exists so that editing one file recompiles one file. The price is that every unit is compiled against a description of its neighbours rather than their bodies. A call to a function declared in a header is a call to an unknown black box: the optimizer knows its signature, and nothing about what it does, how large it is, whether it has side effects, or what it returns for these arguments.
That single ignorance blocks a family of transformations at once. It cannot inline the callee, so it cannot see the constants inlining would expose. It cannot prove the callee does not write memory, so [[alias-analysis]] gives up at the call and every value that might be visible to it has to be reloaded afterwards. It cannot see that no subclass ever overrides a method, so a virtual call stays indirect. It cannot know that a function is never called, so it stays in the binary.
Whole-program optimization removes the ignorance by removing the boundary — every definition in one place, one call graph, one optimizer run. Three consequences dominate in practice, and it is worth being precise that the first is the engine and the other two are largely its passengers: cross-module inlining is what exposes the facts that make everything else fire.
- Cross-module inlining. The body of a small accessor defined in another file is pasted in, and with it come the constants, the known types and the branch conditions that make the caller's code collapse — see
[[inlining]]. - Devirtualization. With the full class hierarchy visible, a virtual call whose receiver has exactly one possible implementation becomes a direct call, and then an inlinable one —
[[devirtualization]]. This is the transformation with the largest effect on idiomatic object-oriented code. - Dead-function elimination. A function reachable from no entry point is removed, along with the data it referenced. On a program that links a large library and uses a tenth of it, this is where binary size goes.
- Better interprocedural facts. Constant arguments propagated into callees, return values known to be non-negative, functions proved not to write memory, parameters proved not to escape — each unlocking transformations locally that could not be justified before.
What it costs, and why the cost is structural
dlopen, or a Java or .NET application with a custom class loader, a subclass can arrive at run time and the hierarchy is never closed — so those platforms devirtualize speculatively behind a guard at run time instead, which is [[speculative-optimization]] and a completely different mechanism with a completely different failure mode.The costs are not implementation shortcomings that a better optimizer would remove. They are the properties separate compilation was purchased with, being handed back.
Parallelism. Compiling a thousand files is a thousand independent jobs across every core you own. Optimizing one program is one job, and the middle-end is largely sequential. A build that saturated a 64-core machine now has one core doing the interesting work while the rest wait.
Incrementality. A one-character edit invalidates the whole-program view, so the entire optimization and code-generation step reruns. The proportionality that [[incremental-compilation]] and [[build-dependency-graph]] exist to protect is gone by construction: every rebuild is a full rebuild of the expensive half.
Memory. The IR for a whole program has to be held at once. This is the limit teams actually hit — a link step that needs tens of gigabytes and is killed by the OOM killer on a CI worker sized for compiles, not for links.
Debuggability and diagnosability. Aggressive cross-module inlining flattens the stack, so a crash reports frames that do not exist in the source, and a regression is harder to bisect because the transformation that caused it happened in a step with no natural unit boundary. See [[debugging-optimized-code]].
| Question | One unit at a time | Whole program |
|---|---|---|
| What does this callee do? | Unknown; assume the worst | Its body is available |
| Does this call write memory I care about? | Assume yes | Proved from the callee |
| How many implementations does this method have?typical | Assume any number | Exactly the ones in the program |
| Is this function ever called? | Assume yes | Answered by the call graph |
| Can I rebuild after a one-line edit cheaply? | Yes — one unit | No — the whole optimization step reruns |
| Can I use all my cores? | Yes — one job per unit | Mostly not; the middle-end is one job |
The transformation the boundary was blocking
One worked example carries the whole argument. A getter in a header, a virtual call in a hot loop, and a hierarchy with exactly one implementation in this program: separately compiled, all three survive to run time; seen together, they collapse into nothing.
What matters is that the legality precondition is not a property of the code shown. Both versions of the source are identical. The rewrite becomes legal because of what the *linking model* guarantees — and it becomes illegal again the moment the program can gain a new implementation after the compiler has finished.
// shape.h — Area() is virtual; only Circle exists in this program
double total(const std::vector<Shape*>& v) {
double s = 0;
for (auto* p : v) s += p->Area(); // indirect call, per element
return s;
}// after WPO: single implementation proved, devirtualized, inlined
double total(const std::vector<Shape*>& v) {
double s = 0;
for (auto* p : v) s += 3.14159265358979 * p->r_ * p->r_;
return s;
}Only if the set of types that can reach this call site is closed and contains exactly one implementation of Area. That requires the whole program to be visible, no subclass to be introducible by dynamic loading, and no symbol interposition able to replace the definition at load time — which on ELF platforms means the definition must not be preemptible.
The program calls dlopen on a plugin that registers a Square, or the class is exported from a shared library where another object can interpose a different Area. Then the direct call runs the wrong code for a receiver the compiler never saw, and the failure is a wrong number rather than a crash — see [[dynamic-linking]] and [[symbol-resolution-order]].
How it works
The steps, in the order the compiler takes them.
- Every compilation unit is compiled to a serialisable IR rather than to machine code, so its bodies survive to the point where they can be combined.
- A combining step merges the units into one module, resolving symbols and unifying types across them.
- The call graph is constructed over the merged module, and unreachable functions are identified from the declared entry points and exported symbols.
- Interprocedural analyses run: which functions may write memory, which parameters escape, which arguments are always constant, which types can reach each virtual call site.
- Inlining runs against the merged call graph with a size budget, which exposes the constants and types the local optimizer then folds.
- The ordinary intraprocedural pipeline runs over the result, now with far more established facts than a single unit could have supplied.
- Code generation and register allocation run per function, which is the one part of the process that parallelises again.
How it breaks
What the engineer observes when it goes wrong — not what goes wrong internally.
- The link step exhausts memory on a CI worker and is killed, with an error that names the linker rather than the optimization that caused it.
- Build wall-clock time collapses onto a single serial step, so adding cores to the build fleet stops helping and nobody can see why.
- A crash report shows a stack with functions missing, because they were inlined across modules, and the symbolised trace does not match any call chain in the source.
- A plugin loaded with
dlopenbehaves as though its overrides were ignored, because a call was devirtualized against a hierarchy that was complete at build time and is not at run time. - Binary size drops sharply and a feature reached only through a name-based lookup disappears, because nothing in the call graph referenced it.
- A one-line change triggers a fifteen-minute rebuild, and the team concludes incremental builds are broken rather than that they were traded away.
When it helps
- C++ and Rust codebases where small functions live in headers or generic code and the abstraction is expected to vanish at compile time.
- Object-oriented code with deep interface hierarchies that in practice have one implementation, where devirtualization then unblocks inlining.
- Size-constrained targets — embedded firmware, WebAssembly bundles — where dead-function elimination across the whole program is the dominant lever.
- Release builds of software shipped far more often than it is built, where the build-time cost is amortised over every user.
When it hurts
- Day-to-day development builds, where losing incrementality costs far more than the generated code gains.
- Programs whose extension model depends on dynamic loading, where the closed-world assumption is simply false and the transformations must be guarded instead.
- Very large monolithic binaries, where peak memory in the combining step becomes the binding constraint on the whole build.
- Code whose time is not in the generated instructions at all, where the entire cost buys nothing measurable —
[[compile-time-vs-runtime]].
What it costs
Every one of these is paid by something.
- A whole-program view buys interprocedural facts and pays with parallelism: a thousand independent compile jobs become one largely sequential optimization job.
- It buys cross-module inlining and pays with incrementality — the expensive half of the build reruns on any change, so a one-line edit costs a full rebuild.
- It buys dead-function elimination and pays peak memory, because the IR for the whole program is resident at once in a single process.
- It buys speed and pays in debuggability and bisectability: flattened stacks, missing frames and a regression that cannot be attributed to a translation unit.
- It buys closed-world reasoning and pays with a hard restriction on the deployment model — no plugin loading, no symbol interposition, or the reasoning is unsound.
What else you could do
What a different compiler or language does instead, and when that is better.
- Partitioned or summary-based schemes get most of the benefit while keeping parallelism and incrementality, at the cost of a less complete view — that is
[[link-time-optimization]]and specifically ThinLTO. - Speculative devirtualization behind a runtime guard, as a JIT does: it works in an open world because it can undo the assumption when a new implementation appears —
[[speculative-optimization]]and[[deoptimization]]. - Manual interface narrowing: sealing classes, marking functions
final, usingstaticor module-private visibility, and hiding symbols with-fvisibility=hidden. Cheap, and it gives the single-unit optimizer real facts instead of assumptions. - Monomorphization at the source level, as Rust and C++ templates do, which achieves cross-boundary specialisation in the frontend before the boundary exists — at the cost of code size and compile time, see
[[monomorphization]]. - Doing nothing and improving data layout or the algorithm, which is frequently a larger win and does not cost the build model anything.
See it for yourself
The flag, dump or tool that shows you this directly.
clang -flto -Wl,--plugin-opt=save-tempswrites the merged module so you can read the IR the optimizer actually saw.clang -Rpass=inline -Rpass-missed=inlinereports which inlining decisions fired and which were declined, with the reason — the reason column is the useful one.nm --defined-onlyon the binary before and after, to see which functions were eliminated.bloaty --domain=symbols old newattributes a binary size change to individual symbols, which is how you find out what dead-function elimination removed./usr/bin/time -von the link step reports peak resident set, which is the number that decides whether this is viable on your CI worker.objdump -don the hot function: a devirtualized call is a directcallto a named symbol where there used to be an indirect one through a register.
Plausible wrong readings
Stated the way a confident engineer states them.
- "Whole-program optimization means the compiler reads all my files." It means the *optimizer* sees all the bodies at once, which normally requires deferring code generation to link time. Reading the files is not the hard part.
- "Devirtualization works because the compiler is clever." It works because the world is closed. The same cleverness applied to a program that loads plugins is a miscompilation.
- "It only makes the binary bigger, because of inlining." It usually makes it smaller: dead-function elimination across the whole program typically removes more than inlining adds.
- "If it is slower to build, it must be faster to run." The build cost is certain and the runtime gain is not. On code that is not the bottleneck it is a pure loss.
Misconceptions
The claim, and what is actually true.
Go deeper
The same idea at increasing depth. Stop wherever it stops being useful.
overview
Normally a compiler sees one file at a time and has to assume the worst about everything defined elsewhere. If it can see the whole program at once, it can paste small functions in from other files, turn virtual calls into direct ones when only one implementation exists, and delete functions nobody calls. The catch is that this undoes the reason files were compiled separately: builds stop being parallel and stop being incremental.
practical
Enable it for release builds and leave it off for development, because the thing it costs — incremental rebuild — is the thing developers use constantly. Measure the link step's peak memory before enabling it in CI; that is what breaks first. If your program loads plugins or you ship shared libraries whose symbols may be interposed, understand the visibility rules before trusting devirtualization. And check binary size as well as speed: the size win from dead-function elimination is frequently the larger and more reliable of the two.
advanced
The interesting question is not whether to see the whole program but what a *sufficient* view is, because the full view is unaffordable at scale and, for most call sites, unnecessary. Almost all the value comes from a small neighbourhood of the call graph: the callee, its callees, and the types that actually reach a handful of virtual sites. That observation is the entire design of summary-based schemes — compute a compact description per module, combine only the descriptions, then import just the bodies the plan asked for. It is also why the closed-world premise can be relaxed rather than abandoned: a compiler that records *which* assumption it relied on can have a runtime check the assumption later, which is exactly the bridge from static whole-program reasoning to the guarded speculation a JIT performs.
How much this depends on
Nothing in this domain is true of every compiler. These say how much.
/GL with /LTCG; Go's linker performs dead-code elimination across the program but does not run a general interprocedural optimizer. What "whole program" contains differs correspondingly.-fno-semantic-interposition is required to make definitions in a shared library non-preemptible, and in the size budgets governing cross-module inlining. A result measured on one is not evidence about the other.If you were asked this in an interview
- What can an optimizer prove with the whole program that it cannot prove one translation unit at a time?
- Under what conditions is devirtualizing a call legal, and what invalidates those conditions after the build?
- Whole-program optimization made your build four times slower. What exactly got slower, and which part is not recoverable by adding cores?
Connections
- Programming Languages & Runtime Internals — Dynamic class loading, method dispatch and the runtime's ability to invalidate an assumptionWhether the world is closed is decided by the runtime's loading model, not by the compiler. A runtime that can add an implementation after the fact is what turns devirtualization from a static proof into a guarded speculation, and the guard and its undo path live on that side of the boundary.