Toolingimplementation

Semantic Refactoring

A rename is not a text replacement. It is a query against the symbol table for every reference bound to one declaration, plus a check that the new name does not collide anywhere those references live — and the difference between those two operations is a class of silent bug.

The question

Why does a find-and-replace rename break code in ways an IDE rename does not?

Source→Lexing→Tokens→Parsing→AST→Semantics→Typed→IR→Optimize→Codegen→Machine code→Link→Execute
What the program is here

A symbol-keyed view of the program: every identifier occurrence resolved to the declaration it binds to, plus the reverse map from each declaration to every occurrence that binds to it, plus a full-fidelity tree per file so that edits can be applied without disturbing anything else. The critical part is that the unit of work is the *symbol*, not the string. Two occurrences of the text count in different scopes are two different objects here, and one occurrence of count inside a string literal is not an occurrence at all.

What this phase may assume or do

A refactoring is legal only if it preserves the program's observable behavior, which for a rename decomposes into two checkable conditions. Completeness: every reference bound to the renamed declaration is updated, including references in files not open and in code paths not exercised. No capture: after the rename, every one of those references still resolves to the same declaration, which fails if the new name is already bound in any scope containing a reference, or if the old name was shadowing something that would now become visible. A rename that satisfies the first and violates the second compiles cleanly and means something different — which is why the check, not the edit, is the hard part.

Key points

  • A safe rename operates on a symbol, not a string: the set of references bound to one declaration, obtained from the symbol table and a project index.
  • Text search gets five things wrong — different scopes, shadowing, strings, comments, and unrelated same-named symbols elsewhere.
  • The shadowing case is the dangerous one: it produces code that compiles, runs, and means something different.
  • The hard part is not finding the references but the validity check: the new name must not already be bound in any scope containing a reference.
  • Capture works in both directions — the new name colliding with an existing binding, and the old name having shadowed something the new one exposes.
  • Find-references is the reverse of resolution and requires a maintained project-wide index; a stale index gives an incomplete rename.
  • Edits must be applied atomically against the versions they were computed for, or a half-applied rename leaves the project broken.
  • The guarantee ends at reflection, string-based lookup, other languages, unindexed configurations and consumers outside the index.

What text search gets wrong

Find-and-replace on an identifier fails in five distinct ways, and it is worth separating them because they have different symptoms and different likelihoods of being noticed.

Different scopes, same name. A local i in one function and a local i in another are unrelated. Text replacement changes both. If the second one is a loop counter, the code still compiles and still works, and now one function has a variable named rowIndex for no reason. Harmless here; not harmless when the two are a field and a parameter with different meanings.

Shadowing. An inner declaration hides an outer one — see [[shadowing]]. Occurrences inside the inner scope refer to the inner declaration and must *not* be renamed with the outer one. Text replacement cannot see the boundary. This is the case that produces a silently wrong program rather than a compile error, because the renamed reference now binds to a different variable that happens to exist.

Strings and comments. "count" as a JSON key, an SQL column name, a log message, a reflective lookup. Replacing inside a string changes program behavior; replacing inside a comment is merely noise until the comment is a doc comment feeding a generated API. Neither is a reference.

The same name elsewhere, unrelated. size appears eleven thousand times in a large repository. Text search finds all of them and cannot tell you that eleven of them are yours.

Names that are not identifiers. Field names in serialized data, string keys in reflection, method names in a dependency-injection config, a symbol name referenced from another language over an FFI boundary. These are real references that a semantic rename *also* misses, which is why the safety guarantee has a documented edge — see the last section.

Four occurrences of `total`; exactly two are the same symbol
1function report(total) {          // [1] the parameter — a declaration
2  log("total is " + total);       // [2] a string, NOT a reference
3                                  // [3] the second `total` IS a reference to [1]
4  {
5    const total = recompute();    // [4] a NEW declaration, shadowing [1]
6    return total * 2;             // [5] a reference to [4], not to [1]
7  }
8}
9 
10// Renaming the parameter to `sum`:
11//   semantic:  [1] and [3] change. [2], [4], [5] do not.
12//   text:      all five change -> the string is corrupted AND
13//              the inner block now declares `sum` shadowing `sum`,
14//              which still compiles and still returns the same value,
15//              so nothing tells you the refactoring was wrong.

The dangerous line is not the string — that one breaks a test. It is [4] and [5]: renaming a shadowed declaration together with its shadower produces code that compiles, runs, and has quietly merged two distinct variables into one name. Reviewers reading the diff see a consistent rename and approve it.

How a real rename works

Four steps, and the third one is the one people forget exists.

Resolve the cursor. The position under the caret is mapped to a syntax node, and that node to the declaration it binds to. This is [[name-resolution]] run backwards from a point, and it is already available in a language server. If the cursor is on the declaration itself, that is the symbol; if it is on a reference, follow it.

Find every reference. The reverse of resolution, which no compilation computes, so it comes from a project-wide index. This is the step that requires the index to be complete and current — a stale index produces an incomplete rename, and an incomplete rename produces a project that does not compile, which is at least loud.

Check validity. For every scope containing a reference, does the new name already resolve to something? If so, the rename would capture. Does any reference sit inside a scope where the *old* name shadowed an outer declaration that the new name would now expose? Also capture, in the other direction. Would the new name collide with a method in a base class, change an override relationship, or clash with a keyword? Each of these is a refusal, and a good implementation reports which one and where rather than declining without explanation.

Apply atomically. Produce one edit set covering every file, applied together or not at all, against the document versions the analysis was computed for. A half-applied rename across forty files is worse than no rename.

Everything else — extract method, inline variable, change signature, move class — has the same shape with a harder validity check. Extract method must determine which local variables are read and written by the extracted region, turn the reads into parameters and the writes into a return value or out-parameters, and refuse if control flow leaves the region in a way a call cannot express. That is [[liveness-analysis]] doing exactly the job it does in a register allocator, applied to source.

Text replacement, structural rewriting and semantic refactoringtypical
ApproachKnows aboutBlind toExample tools
Text search and replaceCharactersEverything: scope, strings, comments, structuresed, editor find-replace
Structural pattern rewritingSyntax — matches trees, not textScope, types, which declaration a name binds tocomby, ast-grep, gofmt -r
AST codemod with manual scope handlingSyntax plus whatever the script implementsWhatever the script forgot; usually shadowingjscodeshift, libcst
Semantic refactoringDeclarations, references, scopes, types, overridesReflection, string-based lookup, other languages, unindexed filesIDE rename, gopls rename, cargo fix

The boundary of the guarantee

implementationHow much a given tool checks varies widely, and the failure is usually silent. gopls rename refuses when it detects a conflict and explains it; some IDE implementations warn and let you proceed; several codemod libraries perform no scope analysis whatsoever despite operating on an AST, which makes them structurally equivalent to text replacement for capture purposes. Before trusting a rename across a large codebase, rename something deliberately into an existing name in a nested scope and see whether the tool objects.

A semantic rename is safe within the program the analysis can see, and that qualifier is doing a great deal of work. Naming the ways it ends is the difference between trusting the tool appropriately and trusting it too much.

Reflection and dynamic lookup. getattr(obj, "count"), Class.getMethod("process"), a Spring bean named in XML, a serializer mapping a field name to a JSON key, an ORM column name derived from a property name. Every one of these is a real dependency on the identifier as a *string*, and no static rename can see it. This is why renaming a field in a language with runtime serialization is a semantic change to your data format, and why some frameworks require an explicit annotation to decouple the two.

Other languages. A symbol exported over an FFI boundary, a name referenced from a build script, a template referring to a variable, a shader uniform, a database column. The rename is complete within the language and incomplete within the system.

Public API. A rename is safe within the compilation closure the tool indexed. If the symbol is public and consumed by code you do not have, the rename is a breaking change, and the tool cannot tell you because the callers are not in the index. This is why [[abi-stability]] and deprecation cycles exist and why the good tools distinguish visibility levels before offering the refactoring.

Unindexed or unbuildable files. A file excluded by the build configuration, behind a feature flag that is off, or in a target the tool did not analyse, contains references the index does not have. The rename compiles for you and breaks for the person who builds with that flag on.

The practical discipline follows directly: check the scope of the rename before accepting it (every good implementation previews the affected files), be suspicious in proportion to how dynamic the language is, and run a full build across all configurations rather than the one you have open.

Codemods: refactoring at repository scale

The same machinery, driven by a script instead of a cursor, is how large-scale migrations happen: rename an API across a thousand call sites, change an argument order, replace a deprecated function with its successor, convert a callback API to promises. The tools split along exactly the line drawn above.

Syntactic codemods — comby, ast-grep, gofmt -r, jscodeshift used without scope analysis — match patterns in the tree and rewrite them. Fast to write, language-agnostic in some cases, and blind to binding. They are the right tool when the pattern is unambiguous: replacing foo.bar(x) with foo.baz(x) where bar exists on exactly one type in the codebase.

Semantic codemods — driven by the compiler frontend, as cargo fix, go fix, clang-tidy --fix and Roslyn analyzers do — resolve symbols before rewriting and can therefore distinguish bar on your type from bar on somebody else's. Much harder to write, and the only correct choice when the pattern is ambiguous.

One property matters more than which category you pick: the output must be reviewable. A codemod that reformats every file it touches produces a diff nobody can read, so the tool needs [[concrete-syntax-tree]]-level fidelity or a canonical formatter downstream — which is the underrated argument for [[formatters]] made concrete. And the migration should land as a mechanical commit separate from any hand-written change, so that a reviewer can verify the transformation rather than re-reading a thousand call sites.

  • Run the codemod, then the formatter, then the build, then the tests — in that order, and commit the mechanical change alone.
  • Prefer a semantic tool whenever the name being rewritten could exist on more than one type.
  • Preview the affected file list before applying; a rename that touches unexpected files is telling you something.
  • For public API, do a deprecation cycle instead: add the new name, alias the old, warn, remove later. A rename is a breaking change you cannot see.
  • Build every configuration afterwards — feature flags off, other targets, other platforms — because the index only covered one.

How it works

The steps, in the order the compiler takes them.

  • Map the cursor position to a syntax node and resolve that node to the declaration it binds to, following a reference back to its definition if necessary.
  • Query the project index for every occurrence bound to that declaration, across all files and all indexed build configurations.
  • For an overridable member, extend the set to the whole override hierarchy — renaming one override alone silently breaks the relationship without any error.
  • For each scope containing a reference, resolve the proposed new name; any successful resolution is a potential capture and must be reported rather than silently accepted.
  • Check the reverse direction: whether the old name was shadowing an outer declaration whose visibility the new name would change.
  • Check language-specific constraints — keywords, reserved names, case-sensitivity collisions on case-insensitive filesystems, visibility rules.
  • Produce a single edit set covering every affected file, each edit stamped with the document version it was computed against.
  • Apply the whole set atomically, or refuse; then re-run diagnostics, since the validity check is a static approximation and the compiler is the final authority.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

  • A find-and-replace rename changes an identifier inside a string that was a serialized field name, and the failure appears as data that no longer deserializes in production.
  • A rename merges a shadowed inner variable with the outer one it was hiding; the code compiles, the tests pass, and the two variables are now the same one.
  • A rename completes in the open files and misses a reference behind a disabled feature flag, so the build breaks only for the team that enables it.
  • Renaming one method of an override hierarchy silently severs the override, and dispatch quietly goes to the base implementation instead of the derived one.
  • A rename of a public symbol succeeds locally and breaks every downstream consumer, because the tool indexed the project and not its users.
  • A codemod reformats every file it touches, producing a twelve-thousand-line diff in which the actual change cannot be reviewed and is approved anyway.
  • An edit set computed against document version 12 is applied after the user typed, landing in the wrong offsets and corrupting a file.

When it helps

  • Any rename in a codebase large enough that you cannot verify the reference set by reading, which is most of them.
  • Large mechanical migrations — API deprecations, argument reordering, library replacements — where the alternative is a thousand hand edits and a thousand chances to be wrong.
  • Refactorings whose validity condition is genuinely hard: extract method, change signature, move class, inline function. The check is the value, not the edit.
  • Statically typed languages with a complete index, where the guarantee is at its strongest and the tool can be trusted to be exhaustive within the project.

When it hurts

  • In dynamic languages with pervasive reflection or string-based dispatch, where the safe-looking rename misses real dependencies and gives a false sense of completeness.
  • For public API, where completeness within the index is exactly the wrong guarantee — the consumers you care about are not in it.
  • Across language boundaries, where the symbol exists in a build file, a template, a schema or another language and the rename is silently partial.
  • When the index is cold or the project does not build in some configuration, since the reference set is then incomplete and the tool usually cannot tell you it is.

What it costs

Every one of these is paid by something.

  • A semantic rename buys correctness within the analysed program and pays a project-wide index that must be built, kept current and held in memory.
  • The validity check buys protection against silent capture and pays refusals — some of them conservative, on renames that would in fact have been fine, which trains users to bypass the tool.
  • Atomic multi-file edits buy consistency and pay the requirement that the whole edit set be computed before anything is applied, which is slow on a large reference set and must be cancellable.
  • A syntactic codemod buys the ability to write a migration in ten minutes and pays the possibility of rewriting a same-named symbol that belonged to something else entirely.
  • A semantic codemod buys binding-aware correctness and pays a much higher authoring cost, since it must be written against a specific compiler frontend's API.
  • Preserving formatting through a refactoring buys a reviewable diff and pays the full-fidelity tree machinery that makes minimal edits possible.

What else you could do

What a different compiler or language does instead, and when that is better.

  • Deprecation instead of renaming: introduce the new name, alias or delegate the old one, warn, and remove after a cycle. Slower, and the only correct option for an API with consumers you cannot see.
  • Structural search and replace (comby, ast-grep, gofmt -r) when the pattern is syntactically unambiguous. Far cheaper to write, blind to binding, and appropriate when you have verified the name is unique.
  • Compiler-driven migration: make the old name an error and let the type checker enumerate every call site. Crude, exhaustive within the compilation, and it works in any language with a compiler.
  • A type-level rename via an alias — keep the symbol, add a new spelling, migrate call sites gradually — which turns an atomic refactoring into an incremental one.
  • Not renaming. A name that is merely imperfect in a widely-consumed API is often cheaper to keep than to change, and recognising that is a legitimate engineering answer rather than a failure of nerve.

See it for yourself

The flag, dump or tool that shows you this directly.

  • Go: gopls rename -w -d ./... shows the diff before writing; it refuses on conflicts and explains them, which makes it a good tool to learn the validity rules from. gofmt -r 'a[b:len(a)] -> a[b:]' is the syntactic counterpart for comparison.
  • Rust: rename via rust-analyzer in the editor; cargo fix --edition is a compiler-driven semantic codemod you can watch operate on a real project.
  • C++: clang-rename -offset=N -new-name=foo file.cpp --, or clang-tidy --fix with a check that supplies fixes; both need a compile_commands.json, which is exactly the index-completeness requirement made explicit.
  • TypeScript/JavaScript: rename through tsserver in the editor; jscodeshift -t transform.js src/ and ts-morph for scripted codemods — ts-morph gives you the type checker, which is the difference between a semantic and a syntactic codemod here.
  • Python: libcst with its scope-analysis providers, or rope, which is the long-standing semantic refactoring library. Note that ast alone cannot do this correctly because it discards the information a review needs.
  • Any tool: test it deliberately. Create a nested scope that shadows a variable, rename the outer one to the inner one's name, and see whether the tool refuses. That one experiment tells you whether it is doing the validity check or only the reference search.

Plausible wrong readings

Stated the way a confident engineer states them.

  • "Renaming is a text operation." It is a query over the symbol table plus a capture check. The text edit is the last and easiest step.
  • "If it compiles after the rename, the rename was correct." Capture produces code that compiles and binds differently. Compilation success is necessary and not sufficient.
  • "An AST-based codemod is safe because it understands the code." It understands the syntax. Unless it resolves names, it cannot tell your size from someone else's, and it is structurally equivalent to text replacement for that purpose.
  • "The IDE renamed everything, so I am done." It renamed everything in its index. Reflection, string keys, other languages, disabled configurations and external consumers are all outside it.
  • "Rename is the simple refactoring." It is the one with the simplest edit and a validity condition that most tools implement incompletely. Extract method has a harder analysis and a much more obvious failure when it goes wrong.

Misconceptions

The claim, and what is actually true.

IDE rename and find-and-replace differ only in convenience.
They differ in what they operate on. One updates a set of references bound to a declaration; the other updates every matching character sequence, including strings and unrelated scopes.
The hard part of rename is finding all the references.
That is a lookup in an index. The hard part is proving the new name does not capture in any scope where a reference lives, in both directions.
Codemods are just scripted find-and-replace.
The useful ones resolve symbols before rewriting, which is what lets them distinguish your method from an identically named one on another type. The ones that do not are find-and-replace with extra steps.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

Your editor's rename and a find-and-replace look like the same operation and are not. Find-and-replace changes every matching run of characters — including the ones inside strings, inside comments, in unrelated functions that happened to pick the same variable name, and inside a nested block where the name refers to a different variable entirely. A real rename asks the compiler which declaration the name under your cursor refers to, finds the places that refer to that same declaration, and checks that the new name is not already taken anywhere those places live. The last check is the one that matters, because a rename that collides produces code that still compiles and quietly means something else.

practical

Use the semantic tool, always, and then check three things it cannot. Search for the old name as a *string* — serialized keys, reflection, config files, SQL, templates — because none of that is in the index. Build every configuration, not just the one you have open, since a reference behind a disabled flag was never analysed. And if the symbol is public, do not rename it at all: add the new name, keep the old one delegating with a deprecation, and remove it a release later. For repository-scale changes, land the mechanical codemod as its own commit, run the formatter after it, and never mix it with a hand-written change — a reviewer can verify a transformation and cannot re-read a thousand call sites.

advanced

The general form is worth seeing, because rename is the easy member of the family. Every refactoring is a program transformation with a precondition, exactly as every optimization is — the difference is only that the output is source rather than machine code, and the precondition is about binding and behavior rather than about observable effects. Extract method needs liveness to decide the parameter and return sets, and reachability to decide whether the region can be a call at all. Inline function needs to know the arguments are side-effect-free or to preserve their evaluation order. Change signature needs the whole override hierarchy plus every call site plus the default-argument rules. Move class needs visibility analysis in both the source and destination. Each carries the same obligation an optimization does: state the precondition, state a case where it fails, and refuse rather than proceed when you cannot establish it. Tools that skip the refusal are not faster refactoring tools; they are text editors with better autocomplete.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

implementationWhether a given tool performs the capture check, and how conservatively, varies enormously. gopls refuses and explains; several IDE implementations warn and proceed; most codemod libraries do no scope analysis at all even though they operate on a tree. The reference-finding step is nearly universal; the validity step is where implementations differ, and its absence is invisible until it produces a silently rebound program.
typicalThe claim that a semantic rename is complete describes statically typed languages with a full project index and no dynamic lookup — Go, Rust, Java, C# in ordinary code. It is materially weaker in Python, Ruby and JavaScript, where getattr, send and property access by computed string are idiomatic, and weaker again anywhere a framework maps identifiers to configuration, serialized keys or database columns. The tool cannot report what it could not see.
specThe correctness condition is a statement about binding, not about text: after the rename, every reference must resolve to the same declaration it resolved to before. That is checkable in any language with static scoping rules and is undecidable in the presence of dynamic name lookup, which is why the guarantee is language-shaped rather than tool-shaped. A language with lexical scoping and no reflection admits a complete check; one with eval does not admit one at all.

If you were asked this in an interview

  • List the ways a find-and-replace rename can go wrong, and say which one is silent.
  • What is the validity check in a rename, and why does it matter more than finding the references?
  • Where does the safety guarantee of a semantic rename end?

Connections

DSAhash-tablen-ary-treedfsdirected-graph
Securitysecure-software-design
Domains that do not exist yet
  • Testing & Reliability Engineering — Landing a repository-scale mechanical change safely
    A thousand-file codemod is a release-risk problem as much as an analysis problem: it needs a separate mechanical commit, a full build across every configuration, and a rollback plan for the case where the transformation was subtly wrong. That process is owned there; what the transformation can and cannot prove about binding is ours.
  • Programming Languages & Runtime Internals — Reflection and dynamic name lookup at runtime
    Every place a program resolves a name from a string at runtime is a reference that static refactoring cannot see, and that is where the guarantee ends. How reflective lookup works and what it costs is owned there; why it makes a rename incomplete is ours.