The question this answers
My service is overloaded. How does the pressure actually get back to whoever is generating the work?
Bounded in-flight work *at this hop*, and a defined behaviour when the bound is reached. Backpressure does not guarantee bounded end-to-end latency, and it does not guarantee that the ultimate source of load slows down. It guarantees only that this node stops accepting; whether that helps depends entirely on what the rejected work does next.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A node knows its own queue depth, its own in-flight count, and whether its writes to a downstream socket are blocking. It does not know the aggregate offered load, how many peer instances are pushing the same dependency, whether its caller is still waiting for the answer, or whether the load it is rejecting is about to come back as a retry. Every backpressure decision is made on a local view of a global condition.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
The signal is a message, so it inherits every property of a message
In a single process a bounded channel gives you backpressure for free: queue.put() blocks, the producer thread stops, and the stop is instantaneous and reliable because the producer and consumer share memory. Across a machine boundary none of that holds. The consumer has to *tell* the producer, over the same network that is already congested, and the producer has to be listening, and the producer has to be willing.
So distributed backpressure is a protocol, and like any protocol it has a delivery semantics, a latency, and a failure mode. A 429 Too Many Requests can be dropped. A closed connection can be interpreted as "retry immediately". A credit-based scheme can leak credits when a peer crashes holding them. The mechanisms below differ mostly in *how fast the signal travels* and *what the sender does with it*.
| Mechanism | Signal travels | Stops what | Fails how |
|---|---|---|---|
| TCP flow control (zero window)protocol | One RTT, automatic | The socket writer, not the application | App keeps producing into its own send buffer; memory grows with no error |
| HTTP 429 / gRPC RESOURCE_EXHAUSTEDtypical | One round trip, per request | One request, if the client honours it | Client retries immediately; rejection rate and request rate both rise |
| Bounded concurrency at the callertypical | Zero — it is local | The caller, before it sends | Caller blocks on its own semaphore and its own queue grows instead |
| Credit / window from consumer to producertypical | One round trip, amortised over many messages | The producer, at the source | Credits stranded on a crashed peer; producer stalls forever without a lease |
Where the chain breaks: the buffer that says yes
A backpressure chain only works if every link passes the signal along. The link that breaks it is always the same shape: something that accepts work it cannot yet do, and returns success. A message broker with seven days of retention. A 202 Accepted that enqueues. An unbounded in-memory work queue. A connection pool with an unbounded wait queue. Each of these converts "the system is full" into "the system is fast", which is exactly backwards.
The distributed twist is that the buffer is usually somewhere nobody owns. The producers are healthy, the consumers are healthy, and the only sick thing is the number in the middle that nobody has an alert on. This is why queue age is a better signal than queue depth: depth tells you how much work is waiting, age tells you how long ago the person who asked for it gave up.
A chain that terminates in a durable log is not wrong — it is a deliberate choice to trade latency for absorbing a spike. It becomes wrong when nobody decided it, and when the log has no bound on how far behind it is allowed to fall before someone sheds.
Pushing back on a caller you do not control
Internal callers can be made to behave: you own their client library, so you can give it a bounded concurrency limit, a retry budget, and an honest interpretation of 429. External callers cannot. A public API faces clients that were written years ago by someone who is not on call tonight, and a meaningful fraction of them treat every non-200 as "try again right now".
For those, the only backpressure that works is the kind that costs the client something and costs you nothing: reject at the edge, as early as possible, with a Retry-After that a well-behaved client honours and a connection-level limit that punishes a badly-behaved one. Note the asymmetry — rejecting is your entire lever, and it only helps if rejecting is cheap. That is why Rejecting Work on Purpose — and Rejecting It Cheaply Enough to Help and Decide at the Door Whether the Capacity Exists are the load-bearing lessons of this module and this one is mostly diagnosis.
The uncomfortable conclusion: for a public surface, backpressure is not a way to slow clients down. It is a way to choose *which* work you drop, while the offered load does whatever it was going to do anyway.
api.requests.rate 12,400/s (normal: 12,000/s) <- offered load barely moved api.responses.2xx 12,390/s <- API is "healthy" api.latency.p99 18ms <- API is fast broker.queue.depth 4,100,000 ^ rising 3,000/s broker.queue.age.p50 41min ^ rising 1s per second <- the real signal worker.utilisation 100% worker.throughput 9,400/s <- capacity, not demand db.cpu 97%
Key points
- Backpressure inside a process is a blocking call; across machines it is a message, with delivery semantics and latency of its own.
- A chain of backpressure is only as strong as its first buffer — anything that accepts work and returns success absorbs the signal and hides it.
- Queue *age* is the signal, not queue depth: age tells you whether anyone is still waiting for the answer.
- The signal must terminate at a party that can and will reduce demand. For external clients, no such party exists — so shedding, not slowing, is the lever.
- TCP flow control gives you backpressure to the socket buffer, never to the application that is filling it.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • A node measures a local proxy for saturation: in-flight count, queue depth, queue age, or observed latency against a target.
- • When the measure crosses a threshold, the node stops accepting: it blocks, rejects with an explicit status, or withholds credit from its producer.
- • That refusal propagates upstream one hop, where the caller must choose: block, queue locally, shed, or fail the request.
- • The refusal propagates further only if the caller has no buffer of its own to hide it in.
- • The chain terminates when it reaches a party that can genuinely produce less — a batch job that pauses, a scheduler that defers, or a user-facing tier that returns a degraded response.
- • The rejection response is itself lost or delayed, so the caller times out and retries rather than backing off.
- • The network path that would carry the pressure signal is the congested path.
- • A producer crashes holding credits, and the consumer waits forever for capacity it already granted.
- • An intermediate broker or pool absorbs the signal silently and reports health.
- • The caller reacts to backpressure by increasing load — retrying immediately, or opening more connections.
- • Latency collapse with no errors: success rate stays at 100%, p99 climbs steadily to exactly the client timeout, queue age rises linearly while queue depth looks stable. Every dashboard is green except the one nobody built.
- • Absorbed signal: broker lag grows for hours, producers see
202 Acceptedthroughout, and the first customer-visible symptom is a webhook that fires 40 minutes late. - • Memory failure at the wrong tier: the ingress service OOM-kills because its unbounded local queue held the backpressure that should have been returned to the caller. The operator sees a healthy database and a crash-looping front door.
- • Inverted backpressure: rejection rate rises and *total* request rate rises with it, because every rejection becomes an immediate retry. The overload metric and the load metric move together, which is the fingerprint of a client that does not honour the signal.
- • None required for a single hop: the decision is local, which is exactly why it is cheap and exactly why it is blind to the aggregate.
- • Coordinating a *global* limit across instances (a shared token bucket) needs a shared store on the request path — one more dependency, one more round trip, and one more thing that fails during the incident it exists to handle.
- • The usual compromise is per-instance limits sized as
global / instance-count, which is uncoordinated and therefore wrong whenever instances scale or traffic is unevenly balanced. Say that out loud rather than pretending the limit is global.
- • Work already accepted is still owed: refusing new work does not shorten the queue you already took.
- • Correctness is unaffected — backpressure changes what is admitted, never what an admitted operation means.
- • The system continues to serve *some* requests correctly rather than serving all of them too slowly to matter. That is the whole guarantee.
- • Detect: alert on queue age and on the ratio of rejections to arrivals, not on CPU. Saturation shows up in the queue long before it shows up in a utilisation number.
- • Contain: bound every queue and every pool. An unbounded buffer is a decision to fail later and worse; see Decide at the Door Whether the Capacity Exists.
- • Recover: drain with the source paused if you can pause it, and with shedding if you cannot. Draining while still accepting at full rate does not converge.
- • Reconcile: work that was rejected must be visibly rejected — a dropped request that the caller believes succeeded is a data problem, not a load problem.
- • Verify: replay the load shape in a test and confirm the signal reaches the source. Most backpressure designs have never been observed working end to end.
- • Queue age at every hop (p50 and p99), which directly answers "is anyone still waiting for this?".
- • In-flight count against the configured limit, per dependency — the utilisation number that actually predicts rejection.
- • Rejections per second alongside arrivals per second on the same axis; divergence means the signal is working, convergence means clients are retrying.
- • Acceptance latency versus service latency: a gap between them is queueing, and queueing is where the backpressure should have been.
- • Any hop where the consumer has a hard capacity limit and the producer is code you control — internal service-to-service calls, worker pools, replication streams.
- • Pipelines with a pausable source: batch imports, backfills, reconciliation jobs. These are the only places where backpressure can genuinely reduce offered load rather than redistribute pain.
- • On a public API, where the client population will not honour the signal and rejections convert to retries — you need shedding and rate limits, not politeness.
- • When the "backpressure" is a blocking call in a thread-per-request server: the pressure lands as thread exhaustion, and one slow dependency stalls endpoints that never touch it.
- • When applied to work that must not be dropped and cannot be delayed. If neither is true, backpressure has no move to make and the answer is capacity.
- • A hard rate limit at the edge: cruder, uncoordinated with actual capacity, but it works against clients you do not control and it is cheap to evaluate.
- • Autoscaling on queue age, which raises capacity instead of lowering demand — good when the spike is sustained and the scale-up is faster than the queue grows; useless for a 30-second spike.
- • Absorb into a durable log on purpose, accept the latency, and alert on lag. Correct for asynchronous work whose value does not decay in minutes.
- • Do nothing and let requests time out. Sometimes honest, and always the baseline you should compare against — a timeout is a form of load shedding with terrible economics.
Backpressure: a bound converts one unbounded quantity into another
What people believe, and what is true
We have backpressure — the queue is bounded.
A bounded queue produces backpressure only if something upstream is blocked or rejected when it fills. If the producer catches the rejection and buffers locally, you moved the queue, you did not bound it.
TCP gives us backpressure for free.
It gives the socket backpressure. An application that reads from the socket into an in-memory list has removed it, and now the pressure is your heap.
Returning 429 slows the client down.
It slows down a client that implements backoff. A meaningful fraction of real clients retry immediately, so a 429 costs you a round trip and buys nothing — which is why it must be cheap to produce.
Backpressure and rate limiting are the same thing.
A rate limit is a fixed budget set in advance; backpressure is a reaction to measured saturation. The first works against strangers, the second adapts to actual capacity. Most systems need both, for different callers.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Backpressure means the busy end tells the sending end to stop. Across machines that message travels over the network, arrives late, and usually stops at the first queue that is willing to accept the work instead.
Practical
Bound every queue and pool, then decide explicitly what happens when the bound is hit: block, reject, or shed. Alert on queue age rather than depth. Then trace one hop at a time and find the buffer that turns a rejection back into a success — that buffer is where your backpressure design ends.
Advanced
Model the chain as a closed loop with delay. The pressure signal has a round-trip latency, and a control loop with too much delay relative to its gain oscillates: reject, clients back off, load collapses, limits reopen, load returns as a synchronised spike. This is why limits should move slowly (AIMD-style) while shedding moves fast, and why Without Jitter, Every Client That Failed Together Retries Together is a stability requirement, not a politeness convention.
Apply it
- 💬 A worker pool is saturated and its queue is a Kafka topic with seven days of retention. Where does backpressure reach, and what does the producer see?
- 💬 You add a 429 to an overloaded service and total request rate goes up. Explain.
- 💬 Why is queue age a better alert than queue depth?