Loweringspec

Closure Conversion

The transformation that turns a closure into an explicit pair of code and environment record. The whole design rests on one question — does the environment hold the values or the bindings — and the classic JavaScript loop bug is what that question looks like when you get it wrong.

The question

What does the compiler actually build when it sees a lambda that captures a variable?

Source→Lexing→Tokens→Parsing→AST→Semantics→Typed→IR→Optimize→Codegen→Machine code→Link→Execute
What the program is here

Before: a nested function whose body mentions names bound in an enclosing function. After: a top-level function taking an explicit environment parameter, plus a heap or stack record holding the captured variables, plus a two-word closure value pairing the two. The point of the after-form is that it contains no nesting and no free variables, so every phase downstream — instruction selection, register allocation, the linker — can treat it as an ordinary function and an ordinary struct.

What this phase may assume or do

The conversion preserves observable behavior only if every access through the environment observes what the language's scoping rules require. For a language that captures bindings, that means the environment holds one cell per captured variable and *both* the enclosing function and the closure are rewritten to access it through that cell — rewriting only the closure leaves two copies of one variable. For a language that captures values, it means the copy is taken at the moment the lambda expression is evaluated, not at first call and not at capture-set computation time. The environment may be stack-allocated only where the closure provably does not outlive the frame.

Key points

  • Closure conversion rewrites a nested function into a top-level function with an explicit environment parameter plus a record of captured variables.
  • After conversion there are no nested functions and no free variables, which is what object files and back ends can actually represent.
  • Whether the environment holds values or pointers to shared cells is the entire semantic difference between capture models.
  • Capturing bindings requires rewriting the enclosing function too, or the two sides end up with separate copies of one variable.
  • The JavaScript var loop bug is not a closure bug: three closures correctly share the one binding that var created.
  • Capture cost is reclaimed by escape analysis, by flat versus linked environments, and by special-casing closures that capture nothing.
  • A promoted variable leaves the register allocator's reach, so capturing a hot loop variable has a cost beyond the allocation.

Code plus environment, written out

The transformation is mechanical and short to state. Take the free-variable set the resolver computed. Build a record type with a field per free variable. Rewrite the lambda body so every reference to a free variable becomes a field access on an environment parameter. Lift the resulting function to the top level. At the point of the lambda expression, allocate the record, fill it, and produce a value pairing the function's address with the record's address.

Everything after this point is ordinary. Calling a closure is loading the code pointer and calling it with the environment as an extra first argument — an ordinary indirect call with an ordinary argument. There are no nested functions in the output, which is what lets the back end and the linker stay simple: a nested function is not a thing an object file can express.

The transform below is the by-value version, which is the simpler half. Read the legal and illegalWhen conditions carefully, because the by-reference version differs only in whether the field holds int or a pointer to a cell, and that one word is the entire subject of the next section.

Closure conversion of a lambda capturing one local
Before
fn make_adder(n: int) -> fn(int) -> int {
    return |x| x + n          // n is free in the lambda
}
After
struct Env_adder { n: int }

fn adder_body(env: *Env_adder, x: int) -> int {
    return x + env.n          // free variable is now a field access
}

fn make_adder(n: int) -> Closure {
    let e = alloc(Env_adder { n: n })     // captured at lambda-evaluation time
    return Closure { code: adder_body, env: e }
}
Legal only when

Only if n is never assigned after the lambda is created, or the language specifies capture by value so that later assignments are deliberately not observed. The copy must be taken when the lambda expression is evaluated — a lambda created inside a loop must copy the value that variable held on that iteration. The environment must outlive every reachable copy of the closure, which means heap allocation unless the compiler has proved otherwise.

Illegal when

If the language captures bindings rather than values and the enclosing function later writes n: the closure would keep reading the stale copy while the source says the two share one variable. It is equally wrong in the other direction — if the closure writes to n and the language specifies sharing, the write lands in the environment copy and the enclosing function never sees it. And it is wrong if the environment is stack-allocated while the closure is returned, which is the dangling-frame case the whole transformation exists to prevent.

Values or bindings: the one-word decision

If the environment field holds a copy of the value, the closure and the enclosing scope have two variables that happened to start equal. If the field holds a pointer to a heap cell — and the enclosing function is rewritten to use that cell too — they have one variable in two places. Everything a language says about closures follows from which of those it picked.

Capturing bindings costs more. The variable is *promoted*: it stops being a stack slot and becomes a heap cell, and the enclosing function pays an indirection on every access to it, not just the closure. Two sibling closures in the same scope share the cell, which is what makes a counter incremented by one lambda visible to another. And a captured variable no longer participates in register allocation, so a hot loop that also captures its induction variable gets measurably slower.

Capturing values costs nothing beyond the copy, and gives up the sharing. Java chose this and then closed the confusing half of the gap by requiring captured locals to be effectively final: if you cannot assign to the variable after capture, you cannot observe the difference. That is a language-design move rather than a compiler one, and it is the cheapest available answer.

C++ is the only mainstream language that asks the programmer per variable. [x] puts a copy in the closure object, [&x] puts a pointer to the frame slot in the closure object and leaves the variable exactly where it was. The second is free and unsafe: nothing checks that the closure dies before the frame.

The environment record under each capture rule
By value — the closure owns a copy
struct Env { n: int }

// read:  env.n
// write: env.n = v      (invisible to the enclosing scope)
// enclosing function keeps n in a register
By binding — one cell, two holders
▸struct Cell { v: int }
▸struct Env { n: *Cell }
▸
▸// read:  env.n->v
▸// write: env.n->v = v   (visible to everyone sharing the cell)
▸// enclosing function ALSO rewritten: every n becomes cell->v

Read it asThe right column costs an allocation, an indirection on every access, and — the part that is easy to miss — a rewrite of the *enclosing* function. A conversion that rewrites only the lambda and leaves the outer function reading its stack slot produces two variables that agree until the first write, which is a bug that passes every test written before someone assigns to the captured variable.

The loop bug, which is this decision in the wild

specThe per-iteration binding for let is required by the ECMAScript specification, which defines a CreatePerIterationEnvironment step for loops with lexical declarations in the head — it is not an engine optimization and it is not something a transpiler may skip. Python has no equivalent: a comprehension has its own scope but a for loop does not, so the late-binding behaviour persists and the default-argument idiom remains the workaround. Go changed the semantics in 1.22 so that loop variables are per-iteration; identical code compiled with an earlier Go produces the shared-binding result.

The most-reported closure bug in the history of programming is three lines of JavaScript, and it is entirely explained by "the environment holds the binding, and var gives the whole function one binding".

With var, i is a single function-scoped variable. The loop runs, three closures are created, and all three capture *that* variable — the same cell. The loop finishes with i equal to 3. Each closure then reads the cell and finds 3. Nothing is broken: the closures faithfully share one binding, and the binding's final value is 3.

With let, the specification creates a fresh binding per iteration and copies the previous iteration's value into it before the update. Now there are three cells, each closure captures a different one, and they hold 0, 1 and 2. The fix is not a fix to closures; it is a change to how many bindings the loop creates.

The pre-let workaround makes the mechanism visible: wrap the body in an immediately-invoked function taking i as a parameter. A parameter is a fresh binding per call, so each closure captures a different cell. Every language with binding capture and a shared loop variable has some version of this — Python's late-binding default arguments (lambda x, i=i: ...) are exactly the same trick, and Go fixed it in the language for 1.22 by giving each iteration its own variable.

One binding versus three
1for (var i = 0; i < 3; i++) {
2  setTimeout(() => console.log(i));   // 3, 3, 3
3}
4 
5for (let i = 0; i < 3; i++) {
6  setTimeout(() => console.log(i));   // 0, 1, 2
7}
8 
9// The pre-let workaround, which shows the mechanism:
10for (var i = 0; i < 3; i++) {
11  (function (i) {
12    setTimeout(() => console.log(i)); // 0, 1, 2 — i is a parameter, so a fresh binding
13  })(i);
14}

Count the bindings, not the closures. var creates one binding for the whole function; let in a loop head creates one per iteration; a parameter creates one per call. The closures behave identically in all three cases.

What it costs, and what takes the cost back

Naively, closure conversion means every lambda that captures anything is a heap allocation, and every access to a captured variable is a pointer chase. In a functional language where closures are the primary abstraction this is the dominant cost, and a great deal of compiler work exists to reduce it.

The reductions come in three shapes. Escape analysis proves a closure does not outlive its frame and stack-allocates the environment; this is the big one and it is what makes a comparator passed to sort free. Flat versus linked environments trade allocation count against access cost — a linked environment shares an existing parent record and allocates only the new frame, but a variable captured five levels up costs five pointer hops. And a closure that captures nothing needs no environment at all and can be a plain function pointer or even a static singleton, which is why compilers special-case it.

The other direction — making it cheaper by capturing less — is a language design lever. Rust's 2021 edition changed closures to capture individual fields rather than whole structs, so a closure mentioning config.timeout no longer borrows all of config. That is not a performance change so much as a change in what the borrow checker rejects, and it illustrates that the capture set itself is a design decision rather than a fact.

Environment representationstypical
RepresentationAllocationAccess costWhere it is used
Flat environmentOne record per closure, holding every captured variable directlyOne indirection regardless of nesting depthMost imperative languages; the default when nesting is shallow
Linked environmentOne record per scope, pointing at its parentOne hop per level of nesting between use and declarationInterpreters and languages with deep lexical nesting
Shared cellsOne cell per mutable captured variable, referenced by every closure that shares itTwo indirections: environment, then cellLanguages that capture bindings and permit mutation
Stack environmentimplementationNone — the record lives in the enclosing frameOne indirection, no allocationWhere escape analysis proved non-escape, or C++ [&]
No environmentNone — the closure is a bare code pointerNoneLambdas that capture nothing; often a compile-time singleton

How it works

The steps, in the order the compiler takes them.

  • Take the free-variable set from name resolution and construct an environment record type with one field per free variable.
  • Decide per variable whether the field holds a copy or a pointer to a shared cell, following the language rule or an explicit capture list.
  • For shared variables, promote the original: replace the stack slot with a heap cell and rewrite every access in the *enclosing* function to go through it.
  • Rewrite the lambda body so each free-variable reference becomes a field access on a new leading environment parameter, then lift the function to the top level.
  • At the lambda expression, allocate and populate the record and construct the two-word closure value; inside a loop this happens once per iteration.
  • Rewrite every call of a closure value into a load of the code pointer and an indirect call passing the environment as the first argument.
  • Run escape analysis; where the closure provably does not outlive the frame, place the record in the frame and delete the allocation.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

  • Every closure created in a loop reports the same final value, and the engineer concludes closures are broken rather than that the loop created one binding.
  • A mutation through a closure is invisible to the enclosing function: the conversion copied the value into the environment and left the outer function reading its original stack slot, so there are silently two variables.
  • A closure returned from a function reads plausible-looking garbage, because the environment was stack-allocated on the strength of an escape analysis that a later refactor invalidated.
  • Memory climbs in a long-running service: a linked environment chain retains every enclosing scope, so one small callback keeps several frames' worth of objects alive.
  • A hot loop slows down by a large factor after a lambda is added to it, because the induction variable was promoted to a heap cell and left the register allocator.
  • Two closures that were supposed to share a counter each increment their own, and the total is wrong by exactly a factor of the number of closures.

When it helps

  • Any language that permits nested functions to escape, which is most of them — the conversion is not optional, it is the implementation.
  • Targets that cannot express nesting at all: object files, WebAssembly, C as a compilation target. Conversion is what makes the program expressible.
  • Making the cost visible during optimization: once the environment is an ordinary record, escape analysis, scalar replacement and inlining apply to it like any other allocation.

When it hurts

  • In tight loops, where the per-iteration environment allocation and the indirection on every captured access dominate the actual work.
  • When a promoted variable was the hot one, since promotion removes it from register allocation entirely.
  • When the environment retains far more than the closure uses, turning a small callback into a large retention root.
  • When the language makes the value-versus-binding choice implicit, so a correct-looking refactor changes what the program computes.

What it costs

Every one of these is paid by something.

  • A flat environment buys constant-time access to every captured variable regardless of nesting, and pays by copying shared variables into every closure that captures them — more allocation, and more work to keep mutable ones in sync via cells.
  • A linked environment buys cheap creation, because it points at the parent instead of copying, and pays a pointer hop per nesting level on every access plus a retention chain that keeps ancestor scopes alive.
  • Capturing bindings buys semantics that match lexical scoping exactly, and pays a heap cell per mutable captured variable plus an indirection charged to the enclosing function as well as the closure.
  • Stack-allocating the environment buys the allocation back entirely and pays in fragility: the decision rests on a proof that any change to the surrounding code can invalidate, so performance becomes non-local and unstable across compiler versions.

What else you could do

What a different compiler or language does instead, and when that is better.

  • Lambda lifting: pass free variables as extra arguments instead of building a record. No allocation and no environment, and it only works when the closure does not escape — see [[lambda-lifting]].
  • Defunctionalization: replace every closure with a tagged variant and one apply function that switches on the tag. Whole-program only, and it removes indirect calls entirely, which suits targets where those are expensive or unavailable.
  • Require the programmer to write the environment, which is the C callback-plus-void* convention and the pre-Java-8 anonymous-class approach. Same machinery, visible costs, more code.
  • Forbid the escaping case. C has no nested functions in standard form, and the resulting language needs no conversion at all — the cost is moved entirely to the programmer.

See it for yourself

The flag, dump or tool that shows you this directly.

  • C++: put a lambda on Compiler Explorer and read the generated closure type. A [x] capture appears as a struct member; [&x] appears as a pointer; a capture-less lambda is convertible to a plain function pointer and compiles to nothing at all.
  • Go: go build -gcflags=-m prints func literal escapes to heap and moved to heap: x — the two halves of the decision, per site.
  • Python: f.__closure__ returns the tuple of cells and cell_contents reads one; two closures sharing a variable return cells that compare identical, which demonstrates the sharing directly.
  • Rust: rustc -Z unpretty=hir shows the inferred capture mode, and std::mem::size_of_val(&closure) at run time gives the environment size — a capture-less closure is zero-sized.
  • JavaScript: a DevTools heap snapshot shows the retained environment as a Closure object; comparing snapshots is how the retention-chain failure mode above is actually diagnosed.

Plausible wrong readings

Stated the way a confident engineer states them.

  • "The loop bug proves JavaScript closures are broken." The closures are correct. var created one binding for the whole function and all three closures share it, exactly as binding capture specifies.
  • "let fixes closures." let changes how many bindings the loop creates. The closure machinery is identical in both versions.
  • "Closure conversion means every lambda allocates." Every lambda that captures something allocates unless a proof removes it. Capture-less lambdas usually compile to a static value.
  • "The environment holds the enclosing frame." It holds the captured variables. Holding the frame is an implementation some interpreters use and it is why closure retention used to be much worse.

Misconceptions

The claim, and what is actually true.

Closure conversion is an optimization.
It is a lowering. Without it there is no way to express a nested function that escapes in an object file, a bytecode format or WebAssembly.
The closure holds the variables.
It holds a pointer to a record that holds them, or to cells that hold them. The distinction is what allows two closures to share one variable.
If I capture by value everything is simpler.
Simpler and different. A language that captures values must then decide what happens when someone assigns to a captured variable, and Java answered by forbidding it.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

A lambda that uses a variable from its surrounding function is compiled into two things: an ordinary top-level function that takes an extra argument, and a small record holding the variables it needed. Calling the lambda means calling the function with the record. That is all closure conversion is, and everything interesting is in whether the record holds copies of the variables or pointers to shared ones.

practical

When closures behave surprisingly, count bindings rather than closures. Ask how many separate variables the language created — one per function for var, one per iteration for let, one per call for a parameter — and the behaviour follows immediately. When closures are slow, look for allocation per iteration and for a captured variable that used to be in a register. And when a captured variable shows as unavailable in a debugger, that is the conversion: the variable is now a field of a generated record, and the debug information has to describe that indirection for the debugger to follow it.

advanced

Closure conversion is the point where a purely lexical property becomes an allocation, which makes it a good lens on the whole middle end. Once the environment is an ordinary heap record, every general optimization applies to it: escape analysis can stack-allocate it, scalar replacement can break it into individual values and delete it entirely, and inlining the call site can make the code pointer statically known so the indirect call becomes direct — after which the environment is often dead. That chain is why closures in a mature compiler can cost literally nothing at a call site the optimizer can see through, and cost an allocation plus an indirect call at one it cannot. The performance question about closures is therefore never about closures; it is about whether the optimizer could see the call site.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

specECMAScript specifies a fresh binding per iteration for let in a loop head, which is why the two loops in this lesson differ. Python specifies no such thing for for loops, so the late-binding behaviour is correct Python and the default-argument idiom remains the workaround. Go changed to per-iteration loop variables in 1.22, so the same source has different meanings under Go 1.21 and Go 1.22 — one of the few places a mainstream language deliberately changed the meaning of existing programs.
implementationWhether the environment is stack- or heap-allocated is an escape-analysis outcome, not a language guarantee. Go reports it per site with -gcflags=-m and the answers change between releases; HotSpot performs escape analysis only in C2 after a method is hot, so a lambda allocates during warmup and may stop afterwards. A measurement of allocation count is a measurement of one build.
typicalMainstream implementations promote only the captured variables to cells, share one environment among sibling closures in a scope, and represent capture-less lambdas without any environment. Naive implementations that allocate a record per lambda regardless, or that capture the whole enclosing scope, are correct and considerably more expensive — which is what a hand-written interpreter usually does first.

If you were asked this in an interview

  • Write out, in pseudocode, what the compiler generates for a lambda that captures one local.
  • Explain the for (var i...) closure result to someone who thinks closures are broken.
  • What does the compiler have to do to the *enclosing* function when a variable is captured by reference?

Connections

DSAhash-map
Performanceallocation-profilingmemory-leaks
OS & Networkingstack-framesstack-vs-heap
Domains that do not exist yet
  • Programming Languages & Runtime Internals — Object representation and collection of the environment record the conversion allocates
    The conversion decides that a record must exist and how large it is. Where it comes from, what header it carries, how the collector traces through the closure into it, and why a retained closure retains everything in the record are the runtime's side of the same object.