What Happens When I Allocate Memory?

The request passes through three actors with three different units. The language runtime deals in objects: it knows the type, the size, and who will free it (you, a refcount, or a collector). The allocator — libc malloc, jemalloc, V8’s heap, CPython’s pymalloc — deals in blocks: it keeps large regions carved into pieces and hands out a piece of the right size. The kernel deals in pages: it extends the process’s address space by whole pages on request and backs them with physical frames only when they are touched. Almost every allocation is satisfied by the allocator without involving the kernel; the kernel is called only when the allocator’s regions are exhausted.

A general-purpose allocator has to answer malloc(64) fast, free(p) fast without being told the size, and do both from many threads. The universal design: round each request up to a size class (glibc uses 16-byte steps for small sizes; jemalloc and tcmalloc use ~40 classes from 8 bytes to a few kB), keep a free list per class, and serve a request by popping the list head. Freed blocks are pushed back onto their class’s list; the size is recovered from a header before the block or from which page it lives in. Large requests (glibc: above 128 kB by default) skip the classes and get their own mmap, so free can return them to the kernel immediately.

Memory for the lists comes from arenas: regions of a few megabytes obtained from the kernel and sliced into runs of one size class each. Multi-threaded allocators give each thread its own cache of free blocks (glibc’s tcache, jemalloc’s tcache, tcmalloc’s per-CPU caches) so that a thread’s malloc/free never takes a lock in the common case, and fall back to per-arena locks only when the cache is empty. This is why replacing glibc malloc with jemalloc or mimalloc can change a multi-threaded server’s throughput and fragmentation noticeably: the size-class boundaries, the arena count and the return-to-OS policy differ.

What the allocator returns to the kernel matters for what top shows. A freed block goes back to a free list, not to the OS; the process’s RSS stays where it was. Allocators return memory only when whole pages or arenas are empty (madvise(MADV_DONTNEED) or munmap), and glibc’s heap only shrinks from the top. A process that allocated 2 GB, freed it all, and now uses 10 MB may still show 2 GB resident — not a leak, but not free either.

The Unix heap has two doors to the kernel. brk/sbrk moves the program break — the end of the data segment — upward, growing one contiguous region; it is the classic heap and glibc still uses it for the main arena. mmap(NULL, len, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) asks for a fresh region anywhere in the address space; secondary arenas, large blocks and thread stacks come from it. Both return address space, not memory: the kernel records a new virtual memory area with its permissions and returns immediately. No frames are allocated, no zeroing happens, and the syscall costs about a microsecond regardless of whether you asked for 4 kB or 4 GB.

On Windows the equivalent is VirtualAlloc, with an explicit two-step model that Unix hides: MEM_RESERVE claims address space, MEM_COMMIT promises backing (charged against the system commit limit) and only a first touch allocates a physical page. The Windows heap (HeapAlloc, and the CRT malloc on top of it) sits above VirtualAlloc in the same way glibc sits above brk/mmap. The distinction matters because Windows does not overcommit: a commit that exceeds RAM + pagefile fails immediately, where a Linux mmap of the same size would succeed and fail later — see Memory Pressure, Swap and the OOM Killer.

After mmap returns, the region is mapped in the page table as not present. The first load or store to each 4 kB page traps into the kernel (Page Faults), which finds the address inside a valid area, allocates a physical frame, zeroes it (the kernel never hands one process another’s old bytes), installs the mapping and resumes the instruction. This is first-touch or demand allocation, and it costs roughly 0.5–2 µs per page including the zeroing — so touching a fresh 1 GB buffer costs ~250,000 faults and a quarter of a second before any useful work, which is why calloc of a huge buffer is fast (it is all untouched zero pages) and the first pass over it is slow.

It is also why "virtual size" and "resident size" diverge: a process may have 40 GB mapped and 2 GB resident, and only the resident part is real. Applications that need the memory to be there now — databases, real-time systems, anything measuring latency — pre-fault it deliberately: MAP_POPULATE, mlock, or simply writing one byte per page at startup. Allocators that know a region will be hot do the same.

Two refinements change the numbers. Transparent huge pages (Linux) may back a first touch with a 2 MB page instead of 4 kB — 512× fewer faults, at the cost of committing 2 MB for a single byte and of latency spikes when the kernel compacts memory to find contiguous 2 MB frames; databases (Redis, MongoDB, Oracle) document whether to turn it off. And MAP_HUGETLB/hugetlbfs reserve true huge pages up front for the cases where the TLB benefit (The TLB) is worth the management.

C++ (new, std::make_unique, containers): the compiler emits a call to operator new, which calls malloc, which is the path above. The runtime adds nothing between you and the allocator except the constructor. Freeing is your responsibility, made deterministic by RAII: a std::vector going out of scope runs its destructor, which calls operator delete → free at that exact moment. The cost model is the allocator’s: ~20–50 ns per small allocation, no pauses, and fragmentation as the long-run risk. Custom allocators (arenas, pools, PMR) are common exactly because the general-purpose path is the only overhead there is.

JavaScript/TypeScript (V8): new Object() does not call malloc. V8 owns a large heap it obtained from the OS in advance, and allocates by bump pointer in the young generation (new space, a few MB of semi-spaces): increment a pointer, write the object header — a handful of instructions, faster than malloc. When new space fills, the scavenger copies the live objects (usually a small minority) into the other semi-space and, after surviving twice, into the old generation; the dead ones are never touched. Old space is collected by a concurrent mark-sweep-compact; large objects (≥ ~500 kB) get their own large-object space. Cost model: allocation is nearly free, freeing is batched into GC work proportional to *live* data, and the failure modes are pauses and the --max-old-space-size limit (defaults on the order of 2–4 GB), not fragmentation.

Python (CPython): obj = SomeObject() allocates a PyObject (16-byte header holding a refcount and a type pointer, plus instance data, plus a __dict__ unless __slots__) via pymalloc for objects up to 512 bytes and via malloc above that. pymalloc keeps 256 kB arenas divided into 4 kB pools, one size class (8-byte steps) per pool, with free lists per pool — a small-object allocator tuned for CPython’s pattern of many tiny, short-lived objects. Freeing is reference counting: when the last reference dies the object is freed immediately and deterministically, so a loop’s temporaries never accumulate. The cyclic GC runs only to find garbage cycles, in three generations triggered by allocation counts (700 net young allocations by default; 3.12+ uses a two-generation incremental scheme), and is the source of the occasional pause. Cost model: every value is an allocation, refcounting adds writes on every assignment, and the GIL serialises the allocator anyway.