6 lessons

Runtime Performance

Garbage collection, event-loop lag, interpreter overhead, allocation cost and warm-up — labelled per runtime, because none of this generalizes across JS, Python, Go, the JVM and C++.

Symptom→Signal→Measurement→Hypothesis→Evidence→Root Cause→Change→Validation→Regression Check

Every lesson below starts from an observable symptom and ends with the measurement that proves the fix worked. Numbers carry a label saying whether they were measured, estimated, simulated or invented to show a shape.

Garbage Collection: Pause, Throughput, Footprint — Pick Two

▶ lab

A collector trades pause time against throughput against memory footprint, and no tuning flag escapes the triangle. The lever you actually control is not the collector — it is how much garbage your code produces per request.

Symptom · p50 latency is flat and healthy. p99 shows regular spikes of tens or hundreds of milliseconds, on a rhythm rather than at random, and no trace span accounts for the gap.

Event-Loop Lag: One Callback, Everybody Waits

A single-threaded event loop runs one callback at a time. A 200ms JSON parse does not just make that request slow — it delays every other pending task by 200ms, including the health check that is about to fail.

Symptom · All endpoints degrade together, including trivial ones like `/health`. The slowdown does not correlate with any single route's traffic, and per-request CPU looks unremarkable.

JavaScript Runtime Performance: V8 Where It Costs

Serialization, allocation and shape changes dominate real server-side JavaScript cost far more often than algorithmic choices. The engine optimizes aggressively for predictable code and deoptimizes quietly when you surprise it.

Symptom · CPU high, latency rising with load, and a profile whose top frames are engine internals and serialization rather than any function the team wrote.

CPython Performance: The Interpreter Tax and the GIL

CPython pays a per-operation interpreter cost that no algorithm change removes, and its global lock means CPU-bound threads do not run in parallel. Neither fact makes Python slow at the thing most services actually do, which is wait.

Symptom · CPU pinned at roughly one core's worth regardless of how many worker threads are configured. Throughput flat. Adding threads increases memory and context switching and nothing else.

C++ Memory Performance: Allocation, Copies and Locality

No collector means no pauses and no free lunch: cost moves to allocator behaviour, fragmentation, and copies the language will make for you silently. And on modern hardware, where your data sits usually matters more than how many instructions you execute.

Symptom · Tail latency spikes with no collector to blame, resident memory that grows and never returns under stable load, and a profile where `malloc`, `free` and copy constructors appear above business logic.

JIT and Warm-Up: The First Thousand Requests Are a Different Program

A JIT-compiled runtime starts interpreted and speeds up as it observes what the code actually does. That makes early requests slower, benchmarks without warm-up meaningless, and freshly-scaled instances a source of tail latency nobody attributes correctly.

Symptom · Latency spikes right after a deploy or a scale-out event, decaying over seconds to minutes. Benchmarks report numbers production never reproduces, in either direction.