SMT: Two Contexts, One Core
Simultaneous multithreading gives one physical core a second register set so it can switch instruction streams instantly and fill cycles the first stream would waste. It does not add execution units, and it does not add a core. On the right workload it is a solid gain; on the wrong one it is negative.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
What is duplicated and what is not
SMT duplicates the small, cheap state that defines an instruction stream: architectural registers, the program counter, and some control state. It does not duplicate the expensive parts: the ALUs, the load/store units, the vector units, the L1 and L2 caches, the TLB, or usually the branch predictor tables. Those are shared, and shared means contended.
So the gain is entirely about filling idle issue slots. A single stream frequently cannot keep a wide superscalar core busy — it stalls on a cache miss, or hits a dependency chain that leaves execution ports empty. With a second stream resident, those cycles get used. On a workload that stalls a lot, that is a genuine and worthwhile win.
The corollary is exact and unforgiving: if the first stream was already keeping the execution units busy, there are no idle slots to fill, and the second stream can only take resources away from the first. Well-optimised, cache-resident, vectorized code is precisely the code that gets nothing from SMT and can lose from it.
| Resource | Duplicated per hardware thread? | Consequence |
|---|---|---|
| Architectural registers | Yes | Instant switching with no save/restore |
| Program counter and control state | Yes | Two independent instruction streams resident |
| ALUs, load/store, vector units | No — shared | Two busy streams contend for issue slots |
| L1 data and instruction cache | No — shared | Each stream effectively gets less cache |
| L2 cache and TLB entries | No — shared | Working sets can evict each other |
| Branch predictor state | Usually shared | Streams can pollute each other's prediction history |
When it helps, when it hurts
The rule follows directly from the mechanism. SMT helps when a stream leaves the core idle: workloads full of cache misses, pointer chasing, dependent loads or unpredictable branches. It hurts when streams compete for something scarce: execution-unit-saturating numeric code, or working sets that already only just fit in L1 or L2, where a sibling halves the effective capacity and both streams start missing.
For latency-sensitive services there is a second consideration beyond throughput. Sharing a core makes a request's completion time depend on what the sibling is doing, which widens the latency distribution. A service can show slightly better aggregate throughput with SMT enabled and materially worse p99 — a trade that is often not worth making, and one that aggregate benchmarks hide completely.
None of this generalises into "enable" or "disable". It generalises into: measure your workload both ways, and measure the tail, not just the mean.
1// Dense, cache-resident, vectorized numeric work.2// The single stream already saturates the vector units;3// there are no idle issue slots for a sibling to use.4for (i = 0; i < n; i += 8) {5 acc = vector_fma(acc, a[i..i+8], b[i..i+8]);6}7// A sibling here competes for the same vector ports8// AND halves the effective L1. Throughput can fall.1// Pointer chasing: each load depends on the previous one.2// The core stalls waiting on memory for most of its cycles.3while (node) {4 sum += node->value;5 node = node->next; // stalls here, repeatedly6}7// A sibling stream issues during exactly those stalls,8// using cycles the first stream cannot.SMT converts *stall cycles* into useful work. Its benefit is therefore proportional to how much the resident stream was already wasting — which makes well-optimised code the worst candidate and memory-stalled code the best. This is the same dependency-chain effect described in Pointer Chasing: The Address You Do Not Have Yet and When You Cannot Ask the Next Question Yet.
The claim to stop making
The headline misconception — that SMT doubles the CPU — deserves a direct answer. It does not, and it never did. The realistic range on workloads that suit it is a modest fractional gain in throughput; on workloads that do not suit it, the gain is zero or negative. Any capacity plan that treats logical CPUs as cores is overcommitted from the start.
There is also a security dimension worth knowing about, though the depth belongs elsewhere: because SMT siblings share caches, predictors and execution ports, they can observe each other's timing. That makes co-resident SMT threads a cross-tenant concern in shared environments, and some operators disable SMT for that reason alone rather than for performance. Side Channels: When Performance Optimisations Leak covers the mechanism.
- Never size CPU-bound capacity from logical CPUs without measuring — start from physical cores.
- Measure the tail, not the mean, on latency-sensitive services: SMT widens the distribution.
- Expect a fractional gain at best, and accept that some workloads lose.
- Treat siblings as a shared trust boundary in multi-tenant environments, not as isolated CPUs.
Key points
- SMT duplicates registers and control state only; execution units, caches and TLB are shared and contended.
- The entire benefit is filling issue slots a stalled stream would waste — so stall-heavy code gains and saturating code does not.
- SMT does not double the core. Realistic gains are fractional, and can be negative on cache-tight or unit-saturating work.
- It widens the latency distribution because completion time depends on what the sibling is doing.
- Siblings share microarchitectural state, which makes them a security consideration in multi-tenant environments.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Core → two register files: both hardware threads keep their architectural state resident, so switching needs no save or restore.
- 2Front end → shared decode: instructions from both streams enter the same fetch and decode resources, interleaved by the core.
- 3Scheduler → shared execution ports: micro-operations from either stream issue to whichever port is free.
- 4Stream A stalls → stream B issues: the cycles A would have wasted waiting on memory are used by B, which is the whole gain.
- 5Both streams → one L1/L2/TLB: their working sets share fixed capacity, so each effectively gets less than a solo thread would.
- • "Hyper-threading doubles the cores" — it adds contexts, not execution units, and the gain is fractional at best.
- • "SMT made throughput slightly better, so enable it everywhere" — check the tail; the p99 may have got materially worse.
- • "SMT is off, so this core is wasted" — a saturating single stream uses a core perfectly well.
- • "The regression must be the scheduler" — a cache-tight thread sharing L1 with a sibling regresses for hardware reasons.
Consequences, controls and cost
- • CPU-bound pools sized to logical CPUs run more threads, each slower, with a worse tail.
- • Cache-tight workloads regress when a sibling is scheduled onto the same core and halves effective L1.
- • Throughput benchmarks and latency benchmarks disagree about whether SMT helped.
- • Cloud capacity based on vCPU counts overstates CPU-bound headroom.
- • Benchmark your real workload with SMT enabled and disabled, measuring p99 as well as throughput, and decide from that.
- • Size CPU-bound thread pools from physical cores, treating SMT gains as a measured bonus rather than an assumption.
- • Pin latency-critical threads so they do not share a core with a noisy sibling, accepting the scheduling cost.
- • In multi-tenant environments, treat co-resident siblings as a shared boundary and consider disabling SMT for isolation.
- • Run the workload pinned to N physical cores, then to 2N logical CPUs, and compare throughput *and* p99.
- • Watch per-thread IPC as siblings are added: falling IPC with rising thread count is sibling contention, not scheduling.
- • Watch L1 miss rate with and without a sibling active — a jump is the shared-capacity effect.
- • Where the platform allows, disable SMT and re-run; it is the cleanest possible A/B.
- • Disabling SMT gives more predictable latency and loses whatever throughput the stall-filling was providing.
- • Pinning to avoid siblings costs scheduler flexibility and can leave capacity idle under load imbalance.
- • Sizing pools to physical cores leaves throughput on the table for genuinely stall-heavy workloads.
Scope
§224 — what these claims are specific to.
- MICROARCH-SPECIFICTwo-way SMT is typical on mainstream x86-64; some architectures use four or eight-way, and several core designs omit SMT entirely. What is shared versus duplicated varies with the design.
- PLATFORM-SPECIFICWhether SMT is exposed, and whether a cloud vCPU corresponds to a logical or physical CPU, is a platform and instance-family decision.