NUMA: Not All Memory Is Equally Far
On a multi-socket machine, memory is attached to sockets. A core reaching its own socket's memory is on a short path; reaching the other socket's memory crosses an inter-socket link. Same instruction, same address space, materially different cost — and the allocator decides which you get.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
Local and remote memory
A two-socket machine is really two nodes glued together. Each node has cores, caches, memory controllers and DRAM. The nodes are joined by an inter-socket link, and cache coherence spans both — the address space is shared and coherent, so nothing about correctness changes. What changes is cost.
A core accessing memory attached to its own socket goes through the local memory controller. A core accessing memory attached to the other socket goes across the link, through the remote controller, and back. The access is transparent; there is no error, no fault and nothing in the code to indicate it happened. There is just more latency, and consumption of a link that every other core is also using.
The ratio between local and remote is modest compared with, say, a cache miss versus a hit — but it applies to *every* access to that memory, and the inter-socket link is a shared resource that saturates. On a large machine with badly placed data, remote traffic can dominate.
Where the placement decision actually happens
The detail that catches people is that allocation does not decide placement — first touch usually does. A common OS policy allocates a physical page on the node of the core that first writes to it, not the core that called the allocator. So a program that allocates a large buffer in a startup thread and then processes it from many threads may find the entire buffer sitting on one node, with every other socket's cores reaching across the link for all of it.
This produces one of the more confusing performance signatures in systems work: a parallel program that scales acceptably up to the core count of one socket and then degrades as it spreads onto the second. Nothing about the algorithm changed at that boundary; the memory simply became remote for half the workers.
The corresponding fix is to touch memory from the thread that will use it — parallel-initialise the buffer with the same decomposition used for processing, so each node's pages are allocated locally. Where the access pattern genuinely spans nodes, explicit interleaving spreads pages across nodes so no single link saturates, trading uniform mediocre latency for the avoidance of a hotspot.
When it matters and when it does not
NUMA is irrelevant on a single-socket machine, which is most laptops and many cloud instances — there is one node and every access is local. It becomes important on multi-socket servers, and it is a first-order concern for databases, in-memory caches and any large parallel workload, which is why those systems ship with explicit NUMA configuration.
The uncomfortable part is that virtualisation can hide the topology or present a misleading one. A VM may span sockets without the guest knowing, so a guest that believes it has a flat sixteen-core machine is actually running across two nodes with its memory arbitrarily placed. This is a real cause of "the same instance type performs differently on different days" — the underlying placement changed. The Hardware That Makes Virtual Machines Possible covers the mechanism; the practical advice is to check the topology the guest actually sees rather than assuming it is flat.
| Situation | Does NUMA matter? | What to do |
|---|---|---|
| Single-socket machine or small VM | No — one node, all local | Ignore it; verify the node count once |
| Multi-socket server, small working set | Barely — data fits in cache | Nothing; caches absorb it |
| Multi-socket, large working set, parallel | Yes — often first-order | First-touch initialisation matching the work decomposition |
| Database or in-memory cache on a big server | Yes — a headline tuning parameter | Explicit node pinning and memory policy |
| VM that may span sockets | Yes, and it is invisible | Check the topology the guest sees; expect variance |
Key points
- On multi-socket machines memory is attached to sockets; local access is cheaper than remote, transparently.
- Placement is usually decided by first touch, not by allocation, so the initialising thread determines locality.
- The classic signature is a parallel program that scales within one socket and degrades when it spreads to the second.
- The fix is to initialise memory from the threads that will use it, matching the processing decomposition.
- NUMA is irrelevant on single-socket machines and first-order on large servers — and virtualisation can hide the topology entirely.
NUMA Locality
Change an input and watch which number moves — and which one refuses to.
The ratio between local and remote depends entirely on the machine, so only its shape is shown. What matters is the mechanism: memory is allocated where it is first touched, so a thread that allocates on one socket and runs on another pays the remote cost on every access — and a scheduler that migrates threads can turn a fast run into a slow one without any code changing.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Thread → allocator: a virtual allocation succeeds without any physical page being assigned yet.
- 2First write → page fault: the OS assigns a physical page, commonly on the node of the touching core.
- 3Core on node 0 → local controller: accesses to that page from node 0 take the short path.
- 4Core on node 1 → inter-socket link: the same accesses from node 1 traverse the link to node 0's controller.
- 5Link → saturation: many remote-accessing cores contend for finite inter-socket bandwidth, and cost rises further.
- • "One address space means uniform cost" — it means uniform *addressing*; cost depends on which node holds the page.
- • "The allocator decides placement" — first touch usually does, which is often a different thread entirely.
- • "Scaling stopped at 16 threads, so 16 is the parallelism limit" — check whether 16 is also the per-socket core count.
- • "My VM has 32 cores, so it is one machine" — it may span sockets, with memory placed arbitrarily.
Consequences, controls and cost
- • Parallel scaling that stops or reverses at the socket boundary.
- • Identical binaries performing differently depending on which thread initialised the data.
- • Cloud instances of the same type showing run-to-run variance from differing physical placement.
- • Inter-socket link saturation acting as a hidden global bottleneck.
- • Initialise data in parallel from the threads that will process it, so first touch places pages locally.
- • Pin worker threads to nodes and partition data per node, so each node's work stays local.
- • Where access genuinely spans nodes, interleave pages across nodes to avoid saturating one link.
- • Verify the topology the process actually sees before tuning — especially inside a VM, where it may not be what you assume.
- • Read the node topology and per-node memory the process actually sees; confirm the node count before anything else.
- • Compare throughput with threads pinned to one node against threads spread across nodes on identical work.
- • Watch remote-memory-access counters where available; a high remote fraction is the direct evidence.
- • Test whether parallel first-touch initialisation changes throughput — a large delta confirms placement was the issue.
- • Node pinning improves locality but reduces scheduling flexibility and can leave a node idle under imbalance.
- • Interleaving avoids hotspots at the cost of making every access uniformly slightly remote.
- • Parallel first-touch initialisation complicates startup code and must match the processing decomposition to help.
Scope
§224 — what these claims are specific to.
- PLATFORM-SPECIFICNUMA exists on multi-socket systems and some large single-socket parts with multiple memory controllers. Single-socket desktops and small cloud instances are uniform, and local/remote ratios differ by interconnect generation.
- SIMPLIFIEDThe two-node picture omits multi-hop topologies on four-socket-and-larger systems, and sub-NUMA clustering where one socket is partitioned into several nodes.