MiddlewareGENERALFRAMEWORK-SPECIFICRUNTIME-SPECIFIC

The Middleware Pipeline

Middleware is function composition around a handler, with a phase on the way in and a phase on the way out — not a list of things that run first.

What actually happens How to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

What is middleware actually, once the framework's next() is taken away?

The requirement

Every request needs a correlation id, an access log line, a CORS decision, a body parsed, a caller authenticated and errors turned into responses. None of that belongs in a handler, and all of it must happen for every route.

The obvious build

Middleware is a list of functions that run before the handler. You register them in order, each does its bit, and then your route function runs.

Why it breaks

The access log needs the status code and the duration, both of which only exist after the handler returns. A "runs before" mental model has nowhere to put that.

How it breaks in production

The access log needs the status code and the duration, both of which only exist after the handler returns. A "runs before" mental model has nowhere to put that.
A middleware forgets to call next() on one branch. Nothing errors. The request hangs until a proxy times out, and the log line for it never appears because the logging middleware never got the response (Timeouts).
next() is called twice — usually a missing return before an early error response — and the handler runs after a response was already written. The framework throws Cannot set headers after they are sent, from a stack trace pointing at the wrong place.
A middleware is registered after the route it was meant to protect. On an order-dependent framework it simply never runs for that route, and the diff that introduced it looks correct (Route Precedence).
An async middleware rejects. On Express 4, an async function's rejection is not passed to the error handler at all — the request hangs; on Express 5 it is. Same code, different framework version, opposite failure.

Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

A middleware chain is nested function calls, not a queue. next() is a call into the rest of the chain; when it returns, you are back in your own frame with the response in hand. That is why the shape is usually called an onion.
This gives every middleware two phases for free: everything before next() runs on the way in, everything after it runs on the way out. Timing, access logging, response headers, response compression and metric recording all live in the second phase.
The outermost middleware runs first on the way in and last on the way out. In Go this is literal and unavoidable — Logging(Auth(RateLimit(mux))) is composition you can read — while frameworks with a registration API hide the nesting and make ordering look like a list.
Not every framework uses one linear chain. Fastify exposes named lifecycle hooks (onRequest, preParsing, preValidation, preHandler, onSend, onResponse), so ordering is by phase first and registration second; Spring has HandlerInterceptor with separate preHandle, postHandle and afterCompletion methods rather than a single function around the chain.
Short-circuiting is a first-class outcome. A middleware that responds without calling next() ends the request — that is how authentication, rate limiting and CORS preflight are supposed to work. The bug is not short-circuiting; it is short-circuiting and then continuing anyway.
The chain is per-request but the middleware functions are shared by every concurrent request in the process. Anything a middleware stores outside the request object is shared mutable state (Backend Races).

It is composition, not a list

The clearest way to see the structure is a framework that does not hide it. In Go, middleware is a function that takes a handler and returns a handler; the chain you write is literally nested calls, and the outermost wrapper is the first to see the request and the last to see the response.

Once you have that picture, framework-specific behaviour stops being surprising. next() is the call into the inner handler. Returning without calling it ends the request. Calling it twice calls the inner handler twice. Code after it runs with the response already written.

The same chain, three ways of saying it

1// Go — composition is visible; Logging is outermost, so it runs first in and last out
2//   handler := Logging(RequestID(Auth(RateLimit(mux))))
3 
4// Express — the nesting is hidden behind registration order
5app.use(logging)      // outermost
6app.use(requestId)
7app.use(auth)
8app.use(rateLimit)    // innermost
9app.get('/orders/:id', getOrder)
10 
11// What the logging middleware actually looks like as an onion
12const logging: RequestHandler = (req, res, next) => {
13  const started = process.hrtime.bigint()          // --- inbound phase
14  res.on('finish', () => {                         // --- outbound phase
15    const ms = Number(process.hrtime.bigint() - started) / 1e6
16    log.info({
17      method: req.method,
18      route: req.route?.path ?? 'unmatched',       // template, not raw path
19      status: res.statusCode,                      // only exists now
20      duration_ms: ms,
21      correlation_id: req.ctx?.correlationId,
22    })
23  })
24  next()                                           // call the rest of the chain
25}

The status code and duration are only knowable after the inner handler has finished. That is why access logging cannot be a "runs before" concern, and why it has to be registered outermost rather than nearest the route.

Two phases, one function

Reading the chain as a pipeline with an in-leg and an out-leg makes the ordering rules derivable instead of memorised. Anything that must observe the response goes outermost; anything that must run before an expensive step goes above it; anything that produces a value goes above everything that consumes it.

The failsBy column below is the useful part. Each of these failures is what you get when that specific step is placed wrongly relative to its neighbours — not a bug in the middleware itself.

Inbound on the way down, outbound on the way back up

1
Error boundary (in: nothing / out: catch)
Wraps everything so any throw below becomes a mapped response.
fails by Registered innermost, so it never sees errors raised by middleware above it (The Error Boundary).
2
Correlation id (in: generate / out: echo)
Accepts or creates an id and puts it on the context and the response header.
fails by Placed after logging, so every log line above it has no id to carry (Correlation Ids That Survive Every Hop).
3
Access log (in: start timer / out: write line)
Records method, route template, status and duration for every request that entered.
fails by Written on the inbound leg only, so it records a status that does not exist yet.
4
CORS (in: preflight / out: headers)
Answers OPTIONS without a handler and attaches headers to real responses.
fails by Registered below authentication, so a 401 goes back without CORS headers and the browser reports a CORS failure instead of an auth failure.
5
Rate limit (in: decide)
Rejects with 429 before expensive work happens.
fails by Placed after authentication, so a flood of invalid tokens still costs signature verification and a user lookup (Authenticate First, or Rate-Limit First?).
6
Body limit + parse (in: read)
Bounds and materialises the request body.
fails by Parsing before a webhook route that needs the raw bytes for signature verification (Webhook Signature Verification).
7
Authenticate (in: identify)
Establishes the principal on the context, or rejects.
fails by Applied per-route instead of globally, so a new route ships unprotected.
8
Handler
The one thing the request was for (What a Handler Is Responsible For).
fails by Everything else in this domain.
9
Compression / serialise (out)
Encodes the response body on the way out.
fails by Registered outside a middleware that sets headers later, producing headers already sent.

Four frameworks, four contracts

FRAMEWORK-SPECIFICExpress and ASP.NET Core disagree about whether the error boundary is registered first or last — and both are consistent, because Express's error middleware is selected by signature rather than by position in the onion, while ASP.NET's is an ordinary middleware that must therefore be outermost. Go has no exceptions at all: the "error middleware" recovers panics, and expected failures travel as return values. Do not port an ordering rule between stacks without re-deriving it.

The concept transfers completely; the details do not transfer at all. Before you rely on middleware for a security control, confirm three things in your framework and version: how an async rejection propagates, where the error handler must be registered, and whether an early response without calling next() reliably stops the chain.

The row that catches most people is the error one. "It throws, so the error handler gets it" is true in some of these and false in others, and it is false in a way that produces a hung request rather than an error.

Framework	Middleware shape	Outbound phase	Where the error boundary goes
Express 4/5	`(req, res, next)`; errors via a 4-argument `(err, req, res, next)`	`res.on('finish')` or wrapping `res.end`	Registered last, after all routes
Koa	`async (ctx, next)` with `await next()`	Ordinary code after the `await`	First middleware, using `try/catch` around `await next()`
Fastify	Named lifecycle hooks, not one chain	`onSend`, `onResponse` hooks	`setErrorHandler`, globally or per encapsulated scope
ASP.NET Core	`RequestDelegate` with `await next(context)`	Code after the await	`UseExceptionHandler` registered first, so it is outermost
Go `net/http`	`func(http.Handler) http.Handler`	Code after `next.ServeHTTP(w, r)`	Outermost wrapper with `recover()`; ordinary errors are returned, not thrown

How to build it

Most important first.

Think in onions. Ask of every middleware: what does it need to be true before it runs, and does it need to do anything after the response exists? The second question is the one that gets skipped (Middleware Ordering Is a Correctness Decision).
Register global cross-cutting concerns once, in one file, in a visible order, rather than sprinkling app.use() through the modules that happen to need them.
Keep per-request work in middleware proportional to what every request needs. A middleware doing a database lookup runs for health checks, static assets and 404s too (What Belongs in the Pipeline).
Always return when you write a response. The single most common middleware bug is a missing return before an early exit.
Attach values to a single, typed context object rather than scattering ad-hoc properties on the request (Request Context Propagation).
Put the error boundary at the outermost position, so it wraps every other middleware and not just the handler (The Error Boundary).

What can go wrong

Failure modes

Missing next(): the request hangs, holds a connection and a socket, and eventually times out upstream with no application log line.
Double next() or next() after a response: a second write attempt, and an exception raised in a frame unrelated to the cause.
Middleware that mutates the request in a way a later one does not expect — reassigning req.url, replacing req.body, or consuming the request stream so nothing downstream can read it (Request Bodies and Streaming).
A response-modifying middleware registered inside the one it needs to wrap, so its outbound phase runs too late to affect headers.
Per-route middleware registered after the route definition on an order-dependent framework — silently inert.
CPU work in middleware on a single-threaded runtime, which delays every other in-flight request rather than just this one (Blocking the Event Loop).

What can race

A middleware storing per-request state in a module-level variable is shared by every concurrent request in the process; the value observed downstream is whichever request wrote last (Request Context Propagation).
Work started in a middleware but not awaited continues after the response is sent, and can be terminated mid-flight by a deploy (Graceful Shutdown).
A cached authorisation decision populated in middleware can be stale by the time the handler acts on it if permissions changed concurrently (Backend Races).

Security

Anything enforced in middleware is enforced only for requests that reach it. A route registered above the middleware, mounted on another router, or served by a static handler bypasses it entirely (Route Precedence).
Prefer default-deny: apply authentication globally and mark public routes explicitly, rather than applying it per protected route and hoping nobody forgets one (Where the Check Belongs).
A middleware that reads a header to establish identity or client IP must know whether that header is attacker-controlled. X-Forwarded-For is client-settable unless the proxy overwrites it and you count hops correctly (Rate Limiting).
Middleware that logs the whole request object will eventually log an Authorization header, a cookie or a card number (Secrets in Logs).
An error thrown inside middleware that is not caught by the boundary can produce a framework default response containing a stack trace (Not Leaking Your Internals).

Misreads

"Middleware runs before the handler." Half of it does. The other half runs after, and that half is where logging, metrics and response headers belong.
"next() means continue to the next middleware." It means *call* the rest of the chain. When it returns, you are still executing, and the response already exists.
"Registration order equals execution order." On the way in, usually. On the way out it is reversed, and on a phase-based framework it is neither.
"If middleware handles it, every route is covered." Only routes that traverse that chain. Static handlers, mounted sub-routers and routes registered earlier may not.
"Async middleware errors go to the error handler." Framework- and version-dependent. Verify it with a test that throws inside an async middleware, in the version you deploy.

Operating it

How you see it in production

Time spent per middleware phase, sampled. It is usually invisible, and when it is not — a synchronous crypto call, a per-request lookup — the pipeline is exactly where the latency is.
A counter of requests that entered the chain and never produced a response. That number should be zero; anything else is a missing next() or an unhandled rejection.
Log the terminating middleware for short-circuited requests, so a 401 or 429 says which control produced it rather than just a status (Structured Logging).
Assert in a test that the assembled chain has the order you think it does. Printing the chain is cheaper than reasoning about it.

What changes at 10x and 100x

Every middleware cost multiplies by request count including the requests you do not care about — health checks, preflights, favicon requests, 404s. A 2-millisecond lookup in middleware is 2 milliseconds on all of them.
On a single-threaded runtime, middleware work is head-of-line work for the whole process, so the same code has a very different blast radius than on a thread-per-request runtime (Backend Runtime Models).
At high rates, moving the cheapest rejections upward — to the proxy or CDN — is worth more than optimising the middleware itself (Authenticate First, or Rate-Limit First?).
The chain length itself rarely matters; ten function calls per request is nothing next to a single database round trip.

What this costs

Middleware makes handlers short and makes them incomplete to read: a reviewer cannot see what has already happened to the request. That is a genuine, permanent cost paid for uniform enforcement.
Global registration guarantees coverage and forces every exception to be an explicit allowlist entry, which is more maintenance than per-route registration and far safer.
A framework with named lifecycle phases removes ordering guesswork and constrains you to the phases it defines.
Putting the error boundary outermost means it also catches errors from middleware you would rather see crash loudly in development.

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

GENERALComposition around a handler with an inbound and an outbound phase describes every server-side middleware system, including ones that call it filters, interceptors or hooks.
FRAMEWORK-SPECIFICExpress uses (req, res, next) with a separate four-argument error signature and requires the error handler to be registered last; Koa and ASP.NET Core await next() so the outbound phase is ordinary code after the await; Fastify replaces the single chain with named lifecycle hooks; Go composes func(http.Handler) http.Handler decorators explicitly. Rules about what happens on an unhandled async rejection differ between all of them and between Express 4 and 5.
RUNTIME-SPECIFICOn Node the entire chain shares one loop thread, so synchronous work in middleware delays every in-flight request in the process; on a thread-per-request runtime such as a servlet container the same work delays only that request but consumes a thread from a bounded pool. Both saturate — differently, and with different symptoms (Worker Processes).

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

API Designapi-gateway-policy request-ids

Architectureapi-gateway

Performancecorrelation-ids instrumentation

Concurrencyevent-loop-vs-threads

OS & Networkingproxies