Request Bodies and Streaming
A body is bytes arriving over time at a rate the client controls, which makes buffering a memory decision and limits a survival requirement.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
Should this body be buffered into memory or streamed, and what does either choice cost me under load?
Users upload documents and submit large payloads. The service must accept them without falling over when several arrive at once.
Let the framework parse the body, then work with the resulting object or buffer. It is one line of middleware and everything downstream gets a normal value.
Memory use is request size times concurrent uploads. Twenty concurrent 50 MB uploads is a gigabyte held in one process, and the process is killed by the container memory limit rather than returning an error (Containerizing a Backend).
- Memory use is request size times concurrent uploads. Twenty concurrent 50 MB uploads is a gigabyte held in one process, and the process is killed by the container memory limit rather than returning an error (Containerizing a Backend).
- A body-size check that runs after parsing has already spent the memory it was meant to protect.
- A client that sends one byte per second holds a request slot indefinitely at almost no cost to itself (Accepting Connections).
- You return a 413 early and the request hangs anyway, because the client is still writing a body nobody is reading and the socket buffers fill.
- The body is read once, so a handler that parses JSON and then wants the raw bytes for a signature check finds them gone (Request and Response Objects).
What is actually happening
- The body is not present when the handler starts. It arrives as a sequence of chunks over the connection, at whatever rate the client sends and the network allows.
- Framing tells you when it ends:
Content-Lengthgives an exact count, chunked encoding ends with a zero-length chunk.Content-Lengthis a claim by the client, not a guarantee — it can be absent, wrong, or larger than what actually arrives (Parsing HTTP). - Buffering accumulates the whole body in memory before your code runs. Simple; costs memory proportional to size times concurrency.
- Streaming processes chunks as they arrive and writes them onward — to disk, to object storage, to a hash function — so peak memory is one chunk rather than one body.
- Backpressure is the mechanism that keeps streaming honest: if you read faster than you can write onward, you must stop reading, or you have reinvented buffering with extra steps (Backpressure).
- If you respond before consuming the body, the client is still sending. Depending on the runtime and the client, you must destroy the connection rather than expect the write to just stop — a 413 with the socket left open can hang.
- Responses stream too, and the same rules invert: a response body written faster than a client reads is buffered somewhere, in your process or in the kernel.
Buffer or stream, and what each multiplies
The decision is not stylistic. Buffering makes peak memory a product of two numbers you do not fully control — payload size and concurrency — and both of them move during exactly the traffic you were worried about.
Streaming keeps memory at roughly one chunk per request and moves the difficulty to correctness: you may have already written half a file when you learn the request should have been rejected.
How large can it be, and what has to happen to it?
when JSON and form payloads measured in kilobytes.
cost Memory equal to limit times concurrency; you must set the limit deliberately rather than inherit it.
when The bytes must pass through your process — hashing, virus scanning, transcoding, or an on-the-fly transform.
cost Backpressure handling, partial-write cleanup, and validation that can only happen after the fact.
when File-sized objects that only need to be stored.
cost A second round trip, orphaned objects when the client vanishes, and a reconciliation job (Presigned URLs).
when Someone wants to send a 500 MB JSON array.
cost The caller must paginate or batch, which is a contract change (Pagination That Survives a Large Table).
Limits are enforced while reading, not after
This is the single most transferable rule in the lesson. A check on the size of an object you already have in memory has not protected the memory. The check has to be in the loop.
The same applies one level up: reject on Content-Length before the body arrives when the client provides it, and still enforce while reading, because the header is a claim.
1import hashlib2 3MAX_BYTES = 100 * 1024 * 10244CHUNK = 64 * 10245 6def receive_upload(body_stream, declared_length, sink):7 """body_stream: a file-like object over the request body.8 sink: something with .write(bytes) -- a storage multipart upload,9 a temp file, whatever the bytes are going to.10 """11 # 1. Reject early when the client tells us it is too big.12 # Cheap, and saves the transfer entirely.13 if declared_length is not None and declared_length > MAX_BYTES:14 raise PayloadTooLarge()15 16 total = 017 digest = hashlib.sha256()18 19 while True:20 chunk = body_stream.read(CHUNK) # one chunk in memory, not one body21 if not chunk:22 break23 24 total += len(chunk)25 # 2. Enforce WHILE reading. Content-Length was a claim.26 if total > MAX_BYTES:27 raise PayloadTooLarge()28 29 digest.update(chunk)30 sink.write(chunk) # backpressure lives here: if the31 # sink is slow, this call blocks32 # and we stop reading. That is the33 # mechanism working, not a stall.34 35 # 3. The client may have disconnected mid-body.36 if declared_length is not None and total != declared_length:37 raise IncompleteBody(received=total, declared=declared_length)38 39 return total, digest.hexdigest()Peak memory here is one chunk, whatever the file size. The three checks are the whole lesson: reject on the claim, enforce on reality, and verify the body actually finished.
The failures, and what each one really is
Body handling produces failures that present as infrastructure problems — a killed container, a full disk, a saturated worker pool — which is why they are so often mis-assigned. Each row below is an application decision wearing an operations costume.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Several large uploads arrive at once | Process killed by the memory limit; unrelated requests die with it | Buffering: peak memory is size times concurrency | Stream, or move uploads to pre-signed URLs; cap concurrent uploads explicitly (Resource Limits) |
Client sends Content-Length: 50000000 at one byte per second | Connection slots or workers occupied, bandwidth near zero | No body-read timeout | Separate body timeout; a minimum-throughput rule; terminate at a proxy that enforces both |
| Server returns 413 mid-body | Request appears to hang; the client eventually times out | The client is still writing and nobody is reading | Respond, then destroy the connection rather than waiting for the write to finish |
| Upload fails after a temp file is written | Disk fills over days or weeks; the instance fails much later | Cleanup on the success path only | Clean up in a finally; prefer streaming to storage over local temp files (Stateless Services) |
| Small gzip body, huge decompressed body | Memory or CPU spike far beyond the request size | Limit applied to wire bytes, not decompressed bytes | Cap the decompressed size and stop decompressing when it is exceeded |
| JSON parsed before the signature check | Valid webhooks rejected intermittently | Re-serialised bytes differ from the bytes signed | Verify over the raw body, then parse (Webhook Signature Verification) |
How to build it
Most important first.
- Set a maximum body size per route, not globally. A JSON endpoint that accepts 100 KB and an upload endpoint that accepts 100 MB should not share one limit (Transport Validation).
- Enforce the limit while reading, and stop reading when it is exceeded.
Content-Lengthlets you reject early — before any bytes arrive — which is strictly better when the client provides it honestly. - For anything file-sized, do not route the bytes through your application at all: hand the client a pre-signed URL and let object storage take the upload (Presigned URLs, Choosing an Upload Path).
- When you must stream through the process, stream to a destination rather than to memory, and respect backpressure — pipe, do not accumulate.
- Give the body its own timeout, separate from the header timeout and from the handler timeout. Three different things can be slow (Timeouts).
- Keep the raw bytes only when something needs them — signature verification does; JSON parsing does not (Webhook Signature Verification).
- When rejecting mid-body, respond and then destroy the connection so the client stops sending.
What can go wrong
- Out-of-memory during a burst of concurrent uploads, killing in-flight requests that had nothing to do with uploading (Memory Leaks in Backend Services is a different shape; this one is instantaneous).
- A temporary file written to local disk and never cleaned up on the error path, filling the disk over weeks and taking down the instance (Stateless Services).
- Slow-loris on the body rather than the headers:
Content-Length: 10000000followed by a trickle of bytes. - A multipart parser that allocates per part, so a request with ten thousand tiny parts costs far more than its byte count suggests.
- The mitigation failing: a size limit enforced in application middleware while a proxy in front already buffered the whole body to satisfy its own limit — you protected the wrong process.
- A client can disconnect mid-upload, so a stream can end early with no error until you check that the received byte count matches what was claimed.
- Concurrent uploads to the same destination key overwrite each other; generate unique keys server-side rather than deriving them from client input (Object Storage).
- Responding while the body is still arriving is a race between your write and the client's write, and the loser is usually a connection stuck until a timeout.
- Body limits are an availability control against unauthenticated traffic, because parsing happens before authentication (Parsing HTTP).
- Never trust
Content-Typeor a filename from a multipart part: both are client-supplied strings. Determine type from content, and never use a client-supplied name as a filesystem path (File Uploads Through the Backend). - Decompression bombs: a small gzip-encoded body can expand enormously. Limit the *decompressed* size, not just the bytes on the wire.
- Streaming an upload directly to a destination path derived from client input is path traversal with extra steps. Generate the key server-side (Object Storage).
- Nested-structure limits matter too: deep JSON nesting and enormous arrays are parser-cost attacks even within a modest size limit (Deserialization: Bytes to Objects).
- "
Content-Lengthtells me how big the body is." It tells you what the client claims. Enforce while reading regardless. - "Streaming is more efficient, so it is the better default." Streaming is more efficient in memory and worse in almost every other way. Buffer small bodies deliberately.
- "I returned 413 so the upload stopped." The client keeps sending until it notices. Destroy the connection or expect the socket to stay busy.
- "The framework limits body size." Some do, some do not, and a proxy in front may have already buffered the body before your limit was consulted. Check all of the hops (The Request Lifecycle).
- "Big requests are a bandwidth problem." They are a memory and concurrency problem. Bandwidth is rarely the thing that falls over first.
Operating it
- Record request body size as a histogram per route. It is the input to every limit you will set and the explanation for a whole class of latency (What Serialization Costs).
- Track 413s by route. A sudden appearance usually means a client feature shipped, not that anyone is attacking you.
- Watch process memory against concurrent in-flight requests. A correlation between the two is buffering, and it tells you the multiplier before it kills you.
- Time-to-first-byte of the request body versus time-to-last-byte separates "the client is slow" from "we are slow", which no single duration can.
- Alert on temporary-directory usage. Disk that only ever goes up is an error path that never cleans up.
- At 10x concurrency, buffering multiplies directly: peak memory is body size times concurrent requests, and it is the concurrency, not the request rate, that decides it (Little's Law as Working Intuition framing from Performance applies directly).
- At 100x, uploads through the application stop being viable on economics alone — you would be paying for compute to move bytes that object storage moves better (Presigned URLs).
- Streaming keeps memory flat as size grows and shifts the constraint to network and to the destination's write throughput, which is usually a much better constraint to have.
- Buffering is simple, retryable and debuggable: you have the whole body, you can validate it before acting, you can log it. It costs memory proportional to size times concurrency.
- Streaming keeps memory flat and makes everything else harder: you may act on the first half of a body whose second half turns out to be invalid, and rollback is now your problem.
- Pre-signed uploads remove the load entirely and add a second step, a second failure mode (the client uploads and never tells you), and a need to reconcile orphaned objects (Choosing an Upload Path).
- Low limits are safe and reject real users. There is no size limit that is both generous and cheap.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALBuffer-versus-stream and the memory multiplier are true of any server on any runtime; only the syntax for expressing backpressure differs.
- RUNTIME-SPECIFICNode streams are non-blocking with explicit backpressure, so a slow destination is signalled back to the source and memory stays flat if you pipe rather than accumulate. A synchronous Python worker reading a body blocks that worker entirely, so a slow uploader occupies one of a small number of workers and the failure is worker starvation rather than memory growth (Worker Processes).
- FRAMEWORK-SPECIFICDefaults differ and matter: Express's
express.json()has a modest default limit and is applied globally unless you scope it; many frameworks disable body parsing on routes you have not configured, which is whyreq.bodyis sometimes undefined for reasons that have nothing to do with the client.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.