GPUquantizationmemorykv cacheprecisioncapacity

Model Memory, and Why the Naive Number Is Always Too Low

Parameters times bytes per parameter gives a floor, not an answer. Activations, the KV cache that grows with context and concurrency, and runtime overhead all sit on top — which is why a model that "fits in memory" by the simple calculation frequently does not.

Follow the mechanism

Software view, hardware view

The gap between what you wrote and what the machine does is where this whole domain lives.

The question
How much memory does serving a model actually need, and what does reducing precision buy in hardware terms?
What you wrote
A parameter count and a precision, multiplied together to decide whether a model fits on a device.
What the hardware does
Device memory holding several things at once: the weights, the activations in flight, a KV cache that grows with every token of every concurrent sequence, and the runtime's own allocations — all competing for a fixed pool.
Capacity decides whether a model can be served at all, and bandwidth decides how fast. Both are driven by bytes, which is why precision reduction is the lever that moves both at once — and why the naive fitting calculation strands deployments that looked fine on paper.
SourceCompilerInstructionsFront EndExecutionRegistersCachesMemoryI/OBehavior

Where the memory actually goes

SIMPLIFIEDA budgeting sketch, not a formula for any specific runtime. Exact KV cache size depends on the attention variant — grouped-query and multi-query attention reduce it substantially — and on layer count, head dimensions and cache precision, all of which differ per model.

The weights are the part everyone counts: parameter count multiplied by bytes per parameter. That number is a floor and it is usually the largest single term, but treating it as the requirement is what produces deployments that fail under load rather than at startup.

The KV cache is the term that surprises people, because it scales with things the model card does not mention: it grows linearly with context length *and* with the number of concurrent sequences. A deployment sized for the weights alone will start fine and run out of memory once real traffic arrives with long contexts, which is a failure that looks like a load problem and is actually a capacity arithmetic problem.

Activations are transient but real, and their peak depends on batch size and the widest layer. The runtime itself holds workspace, fragmentation and its allocator's reserve. In practice the usable fraction of device memory for weights is meaningfully below the physical capacity, and the gap has to be budgeted rather than discovered.

Budgeting device memory — every term, not just the first
1// 1. Weights: the floor everyone computes.
2weights = parameters * bytes_per_parameter
3
4// 2. KV cache: grows with BOTH context and concurrency.
5// This is the term that turns a working demo into an
6// out-of-memory failure once real traffic arrives.
7kv_per_token = 2 * layers * kv_heads * head_dim * bytes_per_element
8kv_cache = kv_per_token * context_length * concurrent_sequences
9
10// 3. Activations: transient, peaks with batch and widest layer.
11activations = f(batch_size, hidden_size, layers)
12
13// 4. Runtime: workspace, fragmentation, allocator reserve.
14overhead = runtime_workspace + fragmentation
15
16total = weights + kv_cache + activations + overhead
17
18// The common mistake is stopping after line 1 and concluding
19// the model fits. It fits at concurrency 1 and context 0.

What lowering precision actually does

Quantization stores parameters in fewer bits. In hardware terms this does two things at once, and both matter. It reduces capacity consumption proportionally, so a larger model fits on the same device. And it reduces bytes moved per weight sweep, which — because decode is bandwidth-bound, as LLM Inference Is a Memory Bandwidth Problem establishes — raises the token rate close to proportionally.

That second effect is the one that is easy to miss. Halving the bytes per parameter does not merely let a bigger model fit; on a bandwidth-bound decode phase it roughly halves the time to stream the weights, so tokens arrive faster. This is why quantization is the highest-leverage single change available for inference performance: it moves capacity and throughput with one decision.

The cost is output quality, and its magnitude is genuinely workload-dependent rather than a fixed penalty. Some tasks tolerate aggressive quantization with no measurable degradation; others degrade noticeably at the same setting. There is also a hardware dependency: a device with native support for a given precision executes it at full rate, while one without may emulate it and lose the benefit — so the right choice is a property of the model, the task *and* the hardware together.

Precision reduction: what changes, and what to watch
WhatEffect on capacityEffect on decode rateRisk
Halving bytes per parameterRoughly halves weight memoryRoughly halves weight streaming timeQuality loss, workload-dependent
Quantizing the KV cacheCuts the term that grows with context and concurrencyHelps once KV traffic is significantQuality loss concentrated in long contexts
Below native precision supportStill saves capacityMay save nothing if the device emulates itCheck hardware support before assuming a gain
Quantizing activations tooSmaller transient peakModest additional gainUsually more damaging than weight-only
No quantizationBaselineBaselineNone, but capacity and rate are what they are

Capacity and rate are the same lever

It is worth stating the unifying point directly, because the two constraints are usually discussed separately. Whether a model *fits* is a bytes question. How fast it *generates* is also a bytes question, because decode streams the weights. The same reduction improves both, and no other single change does that.

This reframes model selection. "Which model fits on this device" and "which model is fast enough" are not independent questions to be answered in sequence — they are the same question asked about the same number. A smaller model at higher precision and a larger model at lower precision can occupy identical memory and stream at identical rates, and then the choice between them is purely about output quality on your task.

It also explains why the KV cache deserves attention out of proportion to its share at low concurrency. At batch size one and a short context it is a rounding error. At production concurrency with long contexts it can rival or exceed the weights, and at that point it is simultaneously eating the capacity that would have allowed more concurrent sequences and the bandwidth that would have generated tokens.

  • Capacity and rate are both bytes — the same reduction moves both, which is what makes precision the highest-leverage lever.
  • Budget the KV cache at production concurrency and context, not at the demo's batch size of one.
  • Check native precision support before assuming a lower precision will be faster rather than merely smaller.
  • Evaluate quality on your own task — the degradation from a given setting is not a published constant.
  • Equal-memory choices are quality choices — a small model at high precision and a large one at low precision may be interchangeable on hardware.

Key points

  • Parameters times bytes per parameter is a floor; activations, KV cache and runtime overhead sit on top of it.
  • The KV cache grows with context length and concurrency, which is why capacity failures appear under load rather than at startup.
  • Reducing precision cuts capacity and weight-streaming time together, moving both constraints with one change.
  • The quality cost of quantization is workload-dependent and must be evaluated on the actual task.
  • A precision the hardware does not natively support may save memory while saving no time at all.

Follow the mechanism

The path through the machine, hop by hop — and the conclusions it invites that are wrong.

  1. 1
    Model → device memory: weights are loaded once and stay resident for the life of the process.
  2. 2
    Sequence → KV cache: every token of every concurrent sequence adds state that persists for that sequence.
  3. 3
    Decode step → memory bandwidth: the weight sweep plus KV reads set the achievable token rate.
  4. 4
    Precision reduction → fewer bytes: both the resident footprint and the per-step traffic shrink together.
  5. 5
    Capacity exhaustion → rejected work: concurrency is capped by what remains after weights and cache.
What people conclude from this — wrongly
  • "The model is 14 GB and the device has 24 GB, so it fits" — it fits at concurrency one with an empty cache, which is not the deployment.
  • "Quantization only saves memory" — while decode is bandwidth-bound it also raises the token rate, which is often the larger benefit.
  • "Lower precision is always faster" — only where the hardware executes it natively; otherwise it saves capacity and no time.
  • "Quality loss from quantization is a known constant" — it varies by model and by task, and has to be measured on yours.

Consequences, controls and cost

What it causes
  • • Deployments sized on weights alone fail under production concurrency and long contexts.
  • • Token rate improves roughly in proportion to reduced bytes per parameter while decode is bandwidth-bound.
  • • Maximum concurrency is set by leftover memory after weights, not by arithmetic capacity.
  • • Two very different model-and-precision combinations can be indistinguishable on hardware and differ only in quality.
What you can do
  • • Budget all four terms at production context length and concurrency before selecting hardware.
  • • Quantize weights to the lowest precision that passes evaluation on your own task, checking native hardware support.
  • • Quantize or compress the KV cache when long contexts or high concurrency make it a leading term.
  • • Re-evaluate quality after any precision change rather than assuming a published result transfers.
How to see it
  • • Instrument device memory occupancy split by weights, KV cache and activations at realistic concurrency.
  • • Measure tokens per second before and after a precision change to confirm the bandwidth benefit materialised.
  • • Run a task-specific evaluation at each candidate precision rather than relying on published degradation figures.
  • • Push concurrency until allocation fails and record the ceiling; that number is the real capacity limit.
What it costs
  • • Lower precision trades output quality for capacity and speed, at a rate that differs by task.
  • • Quantizing the KV cache helps most where quality is most sensitive — long contexts.
  • • Reserving memory for high concurrency limits the model size you can host, and vice versa.

Scope

§224 — what these claims are specific to.

What these claims are specific to
  • SIMPLIFIEDA budgeting sketch. Real KV cache size depends on the attention variant — grouped-query and multi-query attention reduce it substantially — plus layer count, head dimensions and cache precision.
  • GPU-SPECIFICWhich precisions execute at full rate is a hardware property; a device without native support for a format may emulate it, saving capacity but not time.

Misconceptions

Claim
“If the weights fit in device memory, the model can be served.”
Reality
Weights are one of four terms. The KV cache grows with context and concurrency and frequently becomes the binding constraint under real traffic, which is why these failures appear in production rather than in a single-user test.
Claim
“Quantization is purely a memory optimisation.”
Reality
While decode is bandwidth-bound, fewer bytes per weight means less time streaming them, so it raises the token rate too — often the more valuable of the two effects.
Claim
“Any lower precision will run faster.”
Reality
Only if the hardware supports that format natively. Otherwise the runtime may emulate it, keeping the capacity saving and losing the throughput gain entirely.