Real Worldimplementation

The CPython Pipeline

Running a `.py` file compiles it. CPython tokenizes, parses, builds a symbol table and emits bytecode into a code object before a single statement executes — and then a stack machine written in C runs that bytecode.

The question

What actually happens between python script.py and my first line of output?

Source→Lexing→Tokens→Parsing→AST→Semantics→Typed→IR→Optimize→Codegen→Machine code→Link→Execute

What the program is here

Five representations in sequence, all inside the process that will run the program: source bytes, a token stream in which indentation has become explicit INDENT and DEDENT tokens, an AST that Python itself can construct and inspect, a symbol table that fixes for every name whether it is local, global, free or a cell, and finally a code object holding a bytecode string plus its constants, names and stack-depth metadata. The code object is the only one of the five that survives compilation; everything else is discarded once it has been used.

What this phase may assume or do

The compiler is entitled to assume the module parsed and that the symbol table's binding decision for each name is final — whether x is local or global is fixed at compile time even though its value is not. It may assume nothing about types or operators: any global may be rebound between compilation and execution, + may be __add__ on a user class, and an attribute access may run arbitrary code. So a transformation is legal here only if it holds for every possible binding, which in practice restricts the compiler to folding operations over literals of built-in immutable types and deleting code no control-flow path can reach.

Key points

Running a Python file compiles the whole module to bytecode first; execution never reads source text.
The symbol table decides local versus global versus cell statically, and that decision is baked into which opcode is emitted.
The AST is a public, documented data structure, which is why Python's tooling ecosystem is built on the compiler's own representation.
CPython bytecode is an internal detail with no stability guarantee: opcodes, offsets and inline caches change between minor versions.
.pyc caches compilation of imported modules only. The entry-point script is compiled on every run.
PyPy, GraalPy and MicroPython implement the same language with entirely different pipelines, so "Python is X" is a claim about an implementation.

Python compiles. It compiles to bytecode.

implementationThis is CPython, roughly 3.11 through 3.13. The PEG parser replaced the old LL(1) parser in 3.9, the adaptive specialising interpreter arrived in 3.11, and a copy-and-patch JIT tier landed experimentally in 3.13 behind a build flag — so even within CPython the stage list has changed twice in five years. PyPy, GraalPy and MicroPython do not share this list at all.

The most useful thing to know about running Python is that a compiler runs first, every time, over the whole module. There is no line-by-line reading of source at execution time. By the time the first statement runs, the entire module has been tokenized, parsed, resolved and turned into a linear instruction sequence for a stack machine, and it is that instruction sequence — not the text — that executes.

Calling the result "not compiled" because no .exe appears is a category error the guide names explicitly: execution strategy is a property of an implementation, not of a language. What distinguishes this route is not the absence of compilation but *where it stops*. CPython stops at bytecode and hands it to an interpreter loop; a conventional ahead-of-time toolchain keeps going through [[instruction-selection]] and [[register-allocation]] to machine code. See [[execution-strategies]] for the full space of choices.

The stages below are CPython's. They are not Python's, and the distinction matters enough that the last section of this lesson is about nothing else.

CPython 3.11+, from a .py file to a running frameimplementation

Source textyou write it
A .py file: bytes plus an encoding, defaulting to UTF-8.
Tokensbuild time
A token stream from the C tokenizer, with positions.
Block structure. The tokenizer tracks an indentation stack and emits INDENT and DEDENT tokens that no character in the file spells — which is why a mixed-tabs file fails in the tokenizer rather than the parser.
Comments and all non-significant whitespace.
ASTbuild time
A tree of ast nodes, each with lineno and col_offset.
Grouping, precedence, and statement structure. This tree is a documented public data structure, not an internal detail.
Parentheses, formatting, and the exact tokens — an AST round-trip does not reproduce the file.
Symbol tablebuild time
A per-scope table classifying every name.
Whether each name is local, global, a cell (captured by a nested function) or free (captured from an enclosing one). This is decided statically, from assignments and global/nonlocal declarations — see [[lexical-scope]].
Code objectbuild time
A code object: a bytecode string, a constants tuple, name tuples, and stack/frame metadata.
A linear instruction sequence over a value stack, with an explicit evaluation order and a computed maximum stack depth.
The tree, and the identity of subexpressions. From here a traceback can only recover positions from a side table.
Cached code object (`.pyc`)load time
The marshalled code object under __pycache__/.
The ability to skip everything above on the next import of the same unchanged module.
Execution in the PVMrun time
Frames on a call stack, each with a value stack, locals array and a pointer into the bytecode.
Actual objects and actual types. Every question the compiler could not answer is answered here, one instruction at a time.

Read it asRead the when column: everything up to the code object happens before your program produces any output, and it happens again on every run for the entry-point script — which is never cached. Only imported modules get the .pyc shortcut, which is why a large application starts faster on its second run but its __main__ file never does.

The AST is a public data structure, and that is unusual

Most compilers keep their tree private. Python ships it: ast.parse returns the same node objects the compiler uses, compile() accepts a tree in place of source, and ast.unparse turns one back into code. The consequence is an entire ecosystem — formatters, linters, type checkers, coverage tools, refactoring engines — built on the compiler's own representation rather than on a reimplementation of it. That is [[ast-as-shared-infrastructure]] in its purest available form.

It is also the cheapest way to see the tree for yourself. python -m ast script.py prints an indented dump of the parse of any file, and comparing that dump against what you thought you wrote resolves most precedence arguments in one command.

The tree for one expression, as Python itself sees it

1$ python -m ast -c "x = a + b"
2Module(
3   body=[
4      Assign(
5         targets=[
6            Name(id='x', ctx=Store())],
7         value=BinOp(
8            left=Name(id='a', ctx=Load()),
9            op=Add(),
10            right=Name(id='b', ctx=Load())))],
11   type_ignores=[])

Notice ctx=Store() on the target and ctx=Load() on the operands. The tree already distinguishes reading a name from writing one, because that distinction changes which bytecode is emitted and, for a name never assigned in this scope, whether it is a local at all.

Reading the bytecode

implementationCPython bytecode is explicitly not a stable interface. BINARY_OP with an operator argument replaced the family of BINARY_ADD-style opcodes in 3.11, RESUME and the inline cache entries that pad these listings did not exist before 3.11, and offsets shift with almost every release. A .pyc from one minor version cannot be loaded by another, which is exactly what the magic number in its header enforces. Never write a tool that pattern-matches opcodes without pinning the interpreter version.

The dis module disassembles a code object into something readable. This is the single most valuable Python debugging tool nobody uses: when two spellings of the same operation differ in speed, the disassembly usually says why in four lines, and when a comprehension mysteriously does not see a variable, the disassembly shows which opcode is loading it.

The instruction set is a stack machine's — see [[stack-based-vm]]. Operands are pushed, an operator pops them and pushes a result, and there are no registers to allocate, which is a large part of why the compiler is so fast and the interpreter so slow relative to native code.

CPython 3.11 disassembly of a two-line function

1>>> import dis
2>>> def add(a, b):
3...     return a + b
4...
5>>> dis.dis(add)
6  1           0 RESUME                   0
7 
8  2           2 LOAD_FAST                0 (a)
9              4 LOAD_FAST                1 (b)
10              6 BINARY_OP                0 (+)
11             10 RETURN_VALUE

LOAD_FAST is an indexed read from the frame's locals array, decided by the symbol table long before the function ran. A global would be LOAD_GLOBAL, which is a dictionary lookup — and that difference, fixed at compile time by where the name was assigned, is the mechanical reason local variable access is cheaper.

What `.pyc` caches, and what it does not

On the first import of a module, CPython writes the marshalled code object to __pycache__/<name>.<tag>.pyc. On subsequent imports it checks a validity key and, if it matches, skips tokenizing, parsing, symbol-table construction and code generation entirely. This is the only caching in the pipeline, and it is a cache of *compilation*, never of execution: module-level code still runs on every first import in every process.

The default validity key is the source file's modification time and size, which is fast and wrong under two conditions engineers hit constantly — a checkout that resets mtimes, and a build that generates the source. PEP 552 added a hash-based mode for exactly this, and it is what you want in a container image where mtimes are meaningless. See [[hermetic-compilation]] for why this matters beyond Python.

The script you name on the command line is never cached. Neither is anything compiled by exec or eval. So a program whose work is in __main__ pays full compilation on every run, and moving that code into an imported module is a real, measurable startup improvement — one of very few in this pipeline that costs nothing.

What is and is not reused between runsimplementation

Thing	Cached across runs?	Why
Imported module bytecode	Yes, in `__pycache__/`	Validity checked by mtime+size, or by source hash under PEP 552.
The `__main__` script	No	The entry point is compiled fresh every invocation; there is nowhere for the import machinery to key it.
Module-level side effects	No	The `.pyc` holds a code object, not the result of running it. Import-time work happens every process.
`exec`/`eval` sources	No	Compiled on the spot, discarded with the frame.
Specialised (adaptive) instructionsimplementation	No	Quickening happens in memory as code warms up and is thrown away when the process exits.

CPython is one implementation of Python

implementationRelative performance between these depends entirely on the workload: PyPy's advantage is largest on long-running numeric loops and can invert on short scripts, where its warmup dominates, and on C-extension-heavy code, where cpyext costs more than it saves. Treat any single benchmark number here as a statement about one program, not about an implementation — see [[benchmarking]].

Everything above describes an implementation, and there are others that reach the same language semantics by very different routes. PyPy compiles Python with a tracing JIT and can, on numeric loops, be an order of magnitude faster while starting much more slowly. GraalPy runs on the JVM's Truffle framework and partially evaluates an AST interpreter into native code. MicroPython targets microcontrollers and makes different memory trade-offs throughout. Each of them is Python; none of them has CPython's bytecode.

This is why the domain refuses to say a language "is interpreted". The sentence has no truth value until you name an implementation, and the moment you do, the interesting content is in the differences — where native code appears, what warms up, what the C extension story is — rather than in the label.

The practical consequence for an engineer is a rule about what you may rely on. Language semantics are portable; bytecode, opcode names, frame internals, reference-counting timing and the exact moment __del__ runs are not. Code that depends on the second set works on CPython and breaks silently elsewhere.

Four implementations of the same languageimplementation

Implementation	Source becomes	Native code appears	C extension story
CPython 3.13	Stack bytecode in a code object	Only inside the C interpreter loop, plus an experimental JIT behind a build flag	The reference ABI; everything is built against it
PyPy	Its own bytecode, then traces	At runtime, for hot traces, via a tracing JIT	Emulated through `cpyext`, with a real performance penalty
GraalPy	A Truffle AST, specialised as it runs	Via Graal's partial evaluator on the JVM	Emulated; also interoperates with Java
MicroPython	A compact bytecode designed for small memory	Optionally, per-function, with a native code emitter	Not the CPython ABI; a different extension model

How it works

The steps, in the order the compiler takes them.

The C tokenizer scans source, maintaining an indentation stack, and emits tokens including synthetic INDENT and DEDENT.
A PEG parser (since 3.9) builds an ast tree with line and column positions on every node.
The symbol table pass walks the tree per scope, classifying every name from its assignments and any global/nonlocal declarations.
The compiler lowers the tree to a control-flow graph of basic blocks holding stack-machine instructions, folds constant literal expressions, then linearises the blocks and computes the maximum stack depth.
The result is marshalled into a code object, cached to __pycache__/ on import, and executed by the evaluation loop, which dispatches on opcode and manipulates a per-frame value stack.
At runtime the adaptive interpreter rewrites hot instructions in memory into specialised forms — a LOAD_ATTR that always saw the same type becomes a version that checks and then indexes directly — with a guard that falls back if the assumption breaks. That is [[guards]] and [[deoptimization]] inside an interpreter.

How it breaks

What the engineer observes when it goes wrong — not what goes wrong internally.

A syntax error in a function that is never called still stops the program before any output, because the entire module is compiled before any of it runs — surprising to anyone expecting line-by-line execution.
A stale .pyc is used after a source file is restored from a backup or checkout with an older timestamp, and the code that runs is not the code on disk. The symptom is a bug that is fixed in the file and present in the process.
A tool that parses dis output or matches opcode names breaks on a Python upgrade with an exception from deep inside the tool, weeks after the upgrade, when a rarely taken path finally hits the changed opcode.
A variable assigned anywhere in a function is a local everywhere in it, so reading it before that assignment raises UnboundLocalError on a line that looks like a perfectly ordinary global read.
Startup time grows steadily as import-time work accumulates in module bodies; nothing profiles as slow because the cost is spread across dozens of modules, and .pyc caching does not help because it caches compilation, not execution.

When it helps

Explaining a performance difference between two spellings of the same operation: the disassembly usually shows a different opcode, and the reason is in the symbol table.
Debugging import-time behaviour, circular imports and module-level side effects, all of which follow directly from "the module body is code that runs".
Deciding what a Python-based tool can safely rely on — the ast module yes, frame internals and opcodes no.
Reducing startup latency in a CLI or serverless handler by understanding exactly which work is cached and which repeats — see [[startup-time-and-cold-start]].

When it hurts

Reasoning about performance from the bytecode. Instruction counts do not predict time in an interpreter where a single BINARY_OP may invoke arbitrary user code.
Carrying CPython facts to another implementation. Reference-counting-driven deterministic destruction, in particular, is a CPython property that PyPy and GraalPy do not share.
Assuming the compiler will optimize anything. It will not remove your redundant attribute lookup, because it cannot prove the attribute access has no side effects.

What it costs

Every one of these is paid by something.

Compiling to bytecode rather than machine code buys instant startup, trivial portability across operating systems and a compiler simple enough to run on every import — and pays with an interpretive dispatch overhead on every single operation that a native compiler would not have.
Making the AST a public API buys an enormous tooling ecosystem and pays in compatibility obligations: every node shape is now something third-party code depends on, so adding a language feature means breaking or extending a published interface.
Deciding name binding statically buys fast local variable access via an array index, and pays with a rule that surprises people — one assignment anywhere makes the name local everywhere in the scope, UnboundLocalError included.
Caching by mtime and size buys a check that costs one stat call, and pays with silent staleness in exactly the environments where mtimes are unreliable: containers, generated code and version-control checkouts.

What else you could do

What a different compiler or language does instead, and when that is better.

PyPy replaces the interpreter with a tracing JIT, buying large speedups on hot loops and paying with warmup time, higher memory use and a slower C extension path — see [[jit-compilation]].
Cython and mypyc compile a typed subset of Python ahead of time into C extension modules, trading Python's dynamism for native speed on the parts you annotate.
A tree-walking interpreter over the AST would skip bytecode generation entirely — simpler, and materially slower per operation. See [[tree-walk-interpreter]].
Nuitka compiles whole Python programs to C, keeping semantics by embedding the runtime, which trades build time and binary size for a single distributable artifact.

See it for yourself

The flag, dump or tool that shows you this directly.

python -m ast script.py prints the parse tree; add -c "expr" for a one-liner. This is the fastest way to settle a precedence question.
python -m dis script.py, or import dis; dis.dis(fn) in a REPL, prints the bytecode with the source line numbers it came from. dis.dis(fn, adaptive=True) (3.12+) shows the specialised forms after warmup.
import symtable; symtable.symtable(src, "f", "exec") shows the local/global/cell classification directly, which is the ground truth for any UnboundLocalError.
python -X importtime script.py attributes startup cost per imported module — the right tool when startup, not throughput, is the problem.
PYTHONDONTWRITEBYTECODE=1 and python -m compileall --invalidation-mode checked-hash control the cache; python -m py_compile produces one .pyc so you can inspect its header.

Plausible wrong readings

Stated the way a confident engineer states them.

"Python reads my file line by line as it runs." It compiles the entire module first. A syntax error on the last line prevents the first line from executing.
"There is no compiler, so there is nothing to inspect." There are four intermediate representations, three of which have a standard-library module for dumping them.
"The .pyc makes the program run faster." It makes it *start* faster, once, by skipping compilation. Execution speed is identical.
"Bytecode is a Python-level specification, so I can target it from another language." It is a CPython implementation detail that changes between minor releases; the specified interface is the language, not the instruction set.
"CPython optimizes the obvious things like hoisting a loop-invariant attribute lookup." It cannot: the attribute lookup may run arbitrary code, so removing it would change observable behavior — see [[loop-invariant-code-motion]] for what would be required.

Misconceptions

The claim, and what is actually true.

Python is an interpreted language, so there is no compilation step.

Execution strategy belongs to an implementation, not a language. CPython compiles every module to bytecode before running any of it, and PyPy compiles further, to native code, at runtime.

The .pyc file is a compiled executable.

It is a marshalled code object plus a validity header. It cannot run without the interpreter, and it is not portable across interpreter versions.

Deleting __pycache__ will fix my import problem.

Occasionally, when the cache is genuinely stale. Far more often the problem is sys.path or a shadowing module name, and clearing the cache changes nothing except startup time.

Because Python is dynamic, the compiler decides nothing.

It decides scope. Whether a name is local, global or captured is fixed at compile time from the assignments it sees, which is why UnboundLocalError exists at all.

Go deeper

The same idea at increasing depth. Stop wherever it stops being useful.

overview

When you run a Python file, the whole file is compiled to bytecode — a simple instruction list for a stack machine — and then a loop written in C executes those instructions. That is why a syntax error anywhere stops everything, and why the first import of a big library is slower than the second: the second reuses a cached .pyc.

practical

Three tools pay for themselves. python -m ast when you are arguing about precedence or writing a code transform. dis.dis when two versions of the same code differ in speed — the answer is usually LOAD_FAST versus LOAD_GLOBAL, or an attribute lookup inside a loop that nothing is allowed to hoist. python -X importtime when startup is the complaint, because import-time work is not cached and accumulates invisibly.

advanced

The interesting design constraint is that CPython's compiler must be fast enough to run on every import, which caps how much analysis it can do, and its language forbids most of the analysis anyway: with names rebindable and operators overloadable, almost no classical transformation is legal without a runtime check. The 3.11+ adaptive interpreter is the response — instead of proving anything statically, it observes what actually happened, rewrites the instruction into a specialised form, and guards the assumption. That is a JIT's strategy implemented inside an interpreter loop, and it is the same argument as [[why-runtime-information-helps]], applied where a full compiler was too expensive.

How much this depends on

Nothing in this domain is true of every compiler. These say how much.

implementationEvery stage list, opcode and cache detail here is CPython, versions 3.11–3.13. The parser changed in 3.9, the interpreter gained adaptive specialisation in 3.11, and 3.13 added an experimental JIT. PyPy, GraalPy and MicroPython share the language and none of the machinery.

implementationCPython bytecode carries no compatibility guarantee across minor versions: opcodes are added, merged and renumbered, and the .pyc magic number exists to make a mismatch a hard error rather than a crash. Tools that inspect opcodes must pin the interpreter version they support.

typicalThe claim that CPython performs almost no optimization is true of the released compiler but is a moving target: constant folding, dead-code removal and, more recently, a limited specialising tier all exist. What has not changed is the constraint behind it — with types unknown and every operator overloadable, most classical optimizations are simply not legal.

If you were asked this in an interview

Is Python compiled? Defend whichever answer you give.
A function raises UnboundLocalError on a line that reads a module-level name. What happened, and at which stage was it decided?
What exactly does a .pyc file contain, and what makes one invalid?

Connections

DSAstack hash-table

Performancejit-and-warmup garbage-collection

Cloudstartup-time-and-cold-start

Domains that do not exist yet

Programming Languages & Runtime Internals — Reference counting, the cycle collector, and the object model the bytecode manipulates
This lesson stops when the code object reaches the evaluation loop. What LOAD_FAST actually pushes — a pointer to a heap object with a refcount and a type pointer — and when that object is freed belong to the runtime's half of the story, and every opcode here assumes it.

Runtime gain2

Compile cost1

Debuggability5

Implementation2

Context dependence5

Compiling to bytecode is a large gain over re-parsing text but a small one against native compilation, so runtime gain is low. Compile cost is deliberately near zero — the compiler runs on every import and must be fast, which is precisely why it optimizes so little. Debuggability is excellent: tracebacks carry exact positions and the tree, symbol table and bytecode all have dump tools. Context dependence is maximal because nothing in this lesson except the language semantics is portable to another implementation.

Try it

four language pipeline →pipeline explorer →vm stepper →