Buildstypical

Compile Time versus Runtime

Every optimization is a purchase: build seconds now for execution seconds later. The exchange rate is set by how often the program runs against how often it is built — and there are whole classes of program where the purchase buys nothing at all.

The question

Is it worth turning on more optimization, and how would I know?

Source→Lexing→Tokens→Parsing→AST→Semantics→Typed→IR→Optimize→Codegen→Machine code→Link→Execute
What the program is here

Two budgets on the same program, spent by different people at different times. Compile time is work done once per build, on a developer's machine or a CI worker, over a representation the compiler has complete information about. Run time is work done once per execution, on a user's machine, over a representation that has none of that information left. The framing exists to answer a question neither budget answers alone: *who pays, how many times, and is the thing being paid for on the path that matters?*

What this phase may assume or do

Nothing in this lesson transforms a program, so the precondition is on the reasoning rather than the code: a claim that more compile time buys runtime speed is admissible only when the extra work targets a path the program actually spends time on, when that path is one the compiler can see through, and when the improvement has been measured on the workload that matters rather than inferred from the flag name. Absent those, the additional analysis is spent and nothing is bought. The transformations themselves remain bound by their own legality conditions — an optimizer given a larger budget does not get more permission, only more attempts.

Key points

  • More analysis usually costs build time and buys run time, but the curve flattens, and code-size effects can bend it the wrong way on a real workload.
  • The two budgets are paid by different people at different frequencies: engineers pay build time many times a day, users pay run time once per execution.
  • More compile time buys nothing when the program is not CPU-bound, when the bottleneck is algorithmic or in memory layout, when the hot code is behind an opaque boundary, or when the code is cold.
  • The largest cost of a slow build is behavioural: fewer and bigger changes, less local testing, and refactors that stop being attempted.
  • Splitting development and release configurations is the correct design, not a compromise — they are optimizing for genuinely different outcomes.
  • Every runtime win is paid for partly in debuggability, and that bill arrives during a production incident.

The trade, stated honestly

typicalThese are shapes, not measurements. A "CLI tool" that turns out to spend four seconds parsing a large input has a hot path like any other program, and a "service" whose cost is entirely database waits has none. The row you are on is decided by a profile, not by the category the software belongs to.

The standard story is a straight line: more analysis, slower builds, faster programs. The line is real — inlining exposes constants, which exposes branch folding, which exposes dead code; [[monomorphization]] and [[template-instantiation]] move work from run time to build time by generating specialised code; [[compile-time-evaluation]] moves computation itself. All of these genuinely cost build seconds and genuinely buy execution speed.

What makes it a decision rather than a setting is that the line flattens, and sometimes bends the wrong way. Past a point, additional passes find nothing left to prove. Past a further point, the transformations that do fire — aggressive inlining, loop unrolling, vectorization that widens a loop body — increase code size enough that instruction-cache pressure costs more than the removed work saved. That is not a rare pathology; it is why -Os exists, why some projects find -O2 faster than -O3 on their workload, and why the honest question is always "did it get faster on my program" rather than "is this level higher".

The second thing that makes it a decision is that the two budgets are paid by different people. Build time is paid by engineers, several times an hour, forever. Run time is paid by users, once per execution. For a library downloaded a million times, an extra minute of build time is free. For a CLI tool that runs for eighty milliseconds and exits, a minute of build time buys almost nothing, because there is almost no runtime to shorten.

Which budget dominates, by what the software istypical
SoftwareBuilds per dayExecutionsWhere the budget should go
A service that runs continuouslyTens, by CIContinuous, under loadRuntime. Compile time is amortised over a whole deployment.
A shipped library or SDKFew, at releaseEvery downstream user, foreverRuntime, decisively — and code size, because consumers pay for it.
A CLI tool with an 80 ms runMany, during developmentShort and frequentStartup and build time. Most middle-end work has nothing to shorten.
A large application under active developmentHundreds, by engineersIn production onlySplit: fast unoptimized dev builds, optimized release builds.
A batch job dominated by I/ORarelyNightlyNeither. The bottleneck is not the generated code.
A numeric kernel in a hot loopRarelyBillions of iterationsRuntime, and it is worth reading the assembly.

Where more compile time buys nothing

This is the half that is usually left out, and it is the more useful half, because it is where effort is most often wasted. In each of these cases the extra analysis runs, costs what it costs, and changes the program's wall-clock time by approximately zero.

Notice what unites the first four: the compiler is not on the critical path. It cannot make a network round trip shorter, it cannot turn a quadratic algorithm into a linear one, and it cannot optimize a call it is not allowed to look inside. Optimization level is a lever on the code the compiler generates, and only on the fraction of run time spent executing that code.

  • The program is not CPU-bound. Time is going to disk, network, a database or a lock. Shortening the instruction sequence between the waits changes nothing measurable — Performance owns this one under measure-before-optimizing, and it disposes of most optimization-level questions before they are asked.
  • The bottleneck is algorithmic. No optimizer converts an O(n²) scan into an O(n log n) one. It will produce a beautifully scheduled, perfectly allocated quadratic loop.
  • The bottleneck is memory layout. Pointer chasing and cache misses are decided by data structure, not instruction selection. The compiler is not permitted to change your struct layout.
  • The hot path is behind an opaque boundary. A dynamic library call, an FFI boundary, an indirect call through a function pointer the compiler cannot resolve. The optimizer stops at the boundary; a bigger budget does not let it see further. This is the whole reason [[link-time-optimization]] exists as a separate mechanism.
  • The work is already in someone else's optimized binary. If the time is inside a BLAS kernel or a system library, your flags do not apply to it.
  • Language semantics forbid the transformation. A C compiler that cannot prove two pointers do not alias will not vectorize the loop, however long you let it try. More budget does not manufacture the proof — see [[alias-analysis]].
  • The code is cold. Startup paths, error handlers, configuration parsing. Optimizing a function that runs once returns exactly one function-call's worth of savings.
  • The program is dominated by an interpreter or a runtime you did not compile. Optimizing the embedding application does nothing for the bytecode it executes.

The cost that nobody puts on the invoice

Build time is usually costed as machine time, and that is the smaller half. The larger half is what a slow build does to how engineers work. When the edit-build-test loop is ten seconds, people make small changes and run the tests. When it is fifteen minutes, they batch changes to amortise the wait, which makes each change larger and harder to review; they stop running the full test suite locally; and they stop doing speculative refactors, because a refactor is a change whose value is unknown until you see it and whose cost is now a quarter of an hour. The build time did not just slow the team down — it changed which work the team is willing to attempt.

This is why the split between development and release configuration is not a compromise but the correct design. Development builds optimize for turnaround: no or minimal optimization, full debug information, incremental compilation, dynamic linking. Release builds optimize for the shipped artifact. Trying to serve both with one configuration produces a build that is slow to run and slow to produce — see [[debug-vs-release]].

The corresponding runtime cost that nobody invoices is debuggability. Every optimization that pays off does so by destroying correspondence between the source and the running code: variables are "optimized out", line numbers jump backwards, frames are missing because functions were inlined. When a production crash needs diagnosing, that lost correspondence is the bill for the speed, and it arrives at the worst possible moment — [[debugging-optimized-code]].

Deciding, rather than defaulting

implementationWhat each optimization level contains is a per-compiler, per-version decision, not a standard. GCC and Clang both define -O2 and -O3, and the pass lists behind those names differ between the two and change between releases; MSVC uses /O2 and /Ox with different contents again, and rustc's opt-level=3 is not GCC's -O3. Comparing levels across toolchains, or carrying a benchmark result across a compiler upgrade, is comparing two things that share a spelling.

The decision is a short sequence of questions, and most candidates fail at the first two. It is worth doing in this order specifically because each step is cheaper than the one after it.

The answer this produces is frequently "neither" — spend the effort somewhere other than the toolchain — and that is a real answer rather than a failure of the process. The second most common answer is "split the configurations", which costs nothing at run time and returns build time to the people paying for it most often.

  • 1. Where does the time actually go? Profile the real workload. If the answer is I/O, a lock or a database, stop; nothing in this lesson applies.
  • 2. Is the hot code visible to the compiler at all? If it is behind a dynamic-library boundary, an indirect call or an FFI edge, the relevant lever is [[link-time-optimization]] or a devirtualization opportunity, not an optimization level.
  • 3. Who pays the build, and how often? Engineers many times a day, or a release pipeline once a week? This sets the exchange rate, and it usually settles the question by itself.
  • 4. Measure both sides of the specific change. Wall-clock build time before and after, and the runtime benchmark that represents the real workload. A flag that improves a microbenchmark and not the workload has bought nothing.
  • 5. Check code size and startup, not just throughput. Aggressive inlining and unrolling can grow the binary enough to cost instruction-cache locality and page-in time, which shows up as a slower program with a faster inner loop.
  • 6. Only then consider the expensive mechanisms. [[link-time-optimization]] and [[profile-guided-optimization]] both buy real speed and both make the build two-phase and harder to reproduce. They are worth it when steps 1 through 5 say the generated code is genuinely the bottleneck.

How it works

The steps, in the order the compiler takes them.

  • Profile the real workload to find where time is actually spent, before considering any flag.
  • Establish whether the hot code is even visible to the optimizer, or sits behind a dynamic, indirect or foreign-function boundary.
  • Determine the build-to-execution ratio: how many builds per day, against how many executions and for how long.
  • Change one thing, then measure both sides — wall-clock build time, and a runtime benchmark that represents the workload rather than a loop that fits in cache.
  • Check binary size and startup time alongside throughput, because inlining and unrolling trade one for the others.
  • Separate the development configuration from the release configuration so that neither has to compromise for the other.
  • Reach for whole-program and feedback-directed mechanisms only once the generated code is established as the bottleneck.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

  • A team raises the optimization level, build time doubles, and the service's latency percentiles do not move — because the service was waiting on a database the whole time.
  • A binary gets meaningfully larger at a higher level, and a workload with a large working set gets slower, which reads as an inexplicable regression because the inner loop did get faster.
  • The build reaches fifteen minutes, and six months later nobody can point to when the team stopped refactoring — the cost never appeared on any dashboard.
  • Optimization is enabled for release only, a heisenbug appears in production and not in development, and the debugger reports every interesting variable as optimized out.
  • A microbenchmark shows a 30% gain, the flag ships, and end-to-end timings are unchanged, because the benchmark exercised code the real workload barely touches.
  • A compiler upgrade changes what a level contains, an unrelated benchmark moves, and a week is spent looking for the change in the application code.

When it helps

  • CPU-bound code that the compiler can see all of — numeric kernels, parsers, serializers, compression, anything with a hot loop over data in memory.
  • Software distributed far more often than it is built, where build time is amortised across every user who will ever run it.
  • Deciding where to spend engineering effort: the framing frequently shows that the toolchain is the wrong place to look, which is the cheapest possible finding.
  • Justifying investment in build speed, by making the behavioural cost of a slow loop explicit rather than folding it into "builds are slow".

When it hurts

  • As a substitute for profiling. Reasoning about optimization levels before knowing where time goes is the most common way to spend a week and gain nothing.
  • As a universal rule. "Always ship at the highest level" and "always keep builds fast" are both defensible defaults and both wrong for some real project.
  • When it becomes a reason not to measure — the trade-off framing is a way to think about a measurement, not a replacement for taking one.

What it costs

Every one of these is paid by something.

  • Higher optimization levels buy generated-code speed and pay in build time, binary size, and the source-to-machine correspondence a debugger needs.
  • Fast unoptimized development builds buy iteration speed and pay in a development binary that behaves differently from the shipped one — different timing, sometimes different bug visibility.
  • Moving work to compile time — constant evaluation, monomorphization, template instantiation — buys runtime speed and pays in compile time, code size, and often in diagnostics quality when the generated code is what errors are reported against.
  • Moving work to run time — dynamic dispatch, interpretation, a JIT — buys build speed and deployment flexibility and pays in warmup, memory for the compiler itself, and less predictable latency.
  • Investing engineering time in build speed buys back everyone's daily iteration and pays for itself only past a team size and a build duration that need to be estimated honestly rather than assumed.

What else you could do

What a different compiler or language does instead, and when that is better.

  • A JIT defers the whole decision to run time, where the profile is a fact rather than a guess, and pays in warmup and memory — see [[jit-compilation]] and [[why-runtime-information-helps]].
  • Profile-guided optimization keeps the static compiler but gives it evidence, buying better decisions in exchange for a two-phase build — [[profile-guided-optimization]].
  • Tiered builds: compile most of the program cheaply and only the identified hot translation units aggressively. Cheap and effective where the hot set is small and stable; useless where it is diffuse.
  • Not optimizing the code at all and changing the algorithm or the data layout instead, which is regularly the larger win and is unavailable to any compiler.
  • Buying build speed with hardware or a shared cache rather than with lower optimization levels, which sidesteps the trade entirely when the money is available.

See it for yourself

The flag, dump or tool that shows you this directly.

  • time the build, and -ftime-report (GCC/Clang) or -Ztime-passes (rustc nightly) to see which passes are actually spending the compile time.
  • clang -S -o - at two levels and diff the assembly. If the hot function is identical, the level did nothing for it, and no benchmark is needed.
  • clang -Rpass=inline -Rpass-missed=inline and -Rpass-analysis=loop-vectorize report which transformations fired and, more usefully, which were attempted and declined and why.
  • size or bloaty on the binary at each level, to see the code-size side of the trade rather than only the speed side.
  • A profiler on the real workload — perf record / perf stat, or a sampling profiler for a managed runtime — as step one, before any of the above.
  • Compiler Explorer for a single function across levels and versions, where the diff between two flag sets is the whole point.

Plausible wrong readings

Stated the way a confident engineer states them.

  • "The highest optimization level is the right default for release." It is a defensible default, and on a program with a large working set it can be slower than a lower one. The workload decides.
  • "Slow builds are a machine problem." Machines are the cheap half. The expensive half is the changes engineers stop attempting when the loop is long.
  • "Compile time does not matter because CI is asynchronous." CI is not where the cost lands. The cost lands on the engineer waiting to know whether the last edit worked.
  • "If I turn on more optimization and nothing gets faster, the compiler is bad." The far more likely reading is that the generated code was not the bottleneck, which is a finding about your program, not the toolchain.

Misconceptions

The claim, and what is actually true.

Optimization level is a dial from slower to faster code.
It is a dial from fewer to more attempted transformations. Whether that produces a faster program depends on whether those transformations apply to code your workload executes, and code growth can make it slower.
Compile time only costs CI minutes.
It costs iteration count. A long loop makes changes larger, testing rarer and refactoring optional, and none of that appears in a build-time metric.
If the profile says a function is hot, more optimization will help it.
Only if the compiler can act on it. A hot function whose time is in an indirect call, an atomic or a cache miss is not waiting on instruction selection.
Development and release should build the same way to avoid surprises.
They optimize for different outcomes, and forcing one configuration to serve both yields a build that is slow to produce and slow to run. The surprises are managed by testing the release configuration, not by crippling the development one.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

Work has to happen somewhere. The compiler can do more of it while building — which makes the build slower and can make the program faster — or leave it for run time. Which side to spend on depends on how often the program is built compared with how often it runs, and on whether the generated code is where the time actually goes. Very often it is not.

practical

Profile first; the answer is frequently "the compiler is not the problem", which saves the entire investigation. If it is CPU-bound in code you compiled, measure the specific flag change on both axes — build wall-clock and the real workload, not a microbenchmark — and check binary size while you are there. Keep development and release configurations separate: fast, debuggable, incremental for the loop; optimized for the artifact. And treat build time as a product decision, because a fifteen-minute loop changes what your team is willing to attempt.

advanced

The deeper structure is that compile time buys *information*, and information has diminishing returns against what the language lets the compiler assume. Inlining is the clearest case: its value is not the removed call overhead but the constants and types it exposes, which is why its payoff is superlinear at first and then abruptly zero once nothing further is exposed. The same logic explains why LTO and PGO sit where they do on the curve — each buys a different kind of information, LTO by widening the visible scope and PGO by supplying evidence about frequency, and both hit the same wall when the remaining time is in code the compiler is not permitted to reason about. Which is the real reason a JIT can win: it does not buy more analysis, it buys facts, and it buys them after the language's static assumptions have stopped being the binding constraint.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

typicalThe rough shape — higher levels cost build time and usually help hot CPU-bound code — holds across mainstream toolchains. The magnitude does not transfer at all: a numeric kernel may gain several times over, a request handler dominated by allocation and I/O may gain nothing measurable, and the same flag change is worth a week of work in one repository and none in the next.
implementationOptimization level names are per-compiler conventions, not a specification. What -O2 contains differs between GCC and Clang, changes between their own releases, and has no equivalent in MSVC's /O2 or rustc's opt-level. A benchmark result is attached to one compiler at one version and does not survive an upgrade unmeasured.
targetThe balance shifts with the target. On a machine with a small instruction cache or tight memory, size-oriented settings frequently beat speed-oriented ones on the same source; on a large server core with deep caches the reverse usually holds. Cross-compiling means the machine you benchmark on is not the machine that pays.

If you were asked this in an interview

  • A team wants to raise the optimization level on a web service. What do you ask before agreeing?
  • Name three situations where spending more compile time buys no runtime speed at all.
  • What is the real cost of a fifteen-minute build, beyond the fifteen minutes?

Connections

Computer Architectureinstruction-cachecache-linesmemory-hierarchy
Performancemeasure-before-optimizingcpu-profilingmicrobenchmark-vs-systemperformance-tradeoffsbenchmarkingcpu-bound-vs-io-bound
Domains that do not exist yet
  • DevOps / Production Engineering — Build-time budgets, CI fleet sizing and the cost of developer wait time
    Turning "the build takes fifteen minutes" into a staffing and hardware decision — how many CI workers, what a queued build costs, when a remote cache pays for itself — is that domain's work. We stop at the compiler-side question of what the extra minutes were spent on and whether the program got faster.