Backpressure
When a producer emits 100,000 messages per second and the consumer handles 20,000, the queue grows by 80,000 per second and the oldest message is minutes old within minutes; backpressure is every mechanism that makes the producer feel the consumer's limit before memory, disk or latency does.
A queue between producer and consumer absorbs bursts, but it cannot absorb a sustained rate mismatch: the backlog grows without bound, memory or disk fills, and the latency of the oldest message climbs until the work is useless by the time it runs. Backpressure keeps the system inside its capacity by slowing, bounding, shedding or scaling — deliberately, rather than by crashing.
The arithmetic
Producer 100,000 msg/s, consumer 20,000 msg/s. Net growth 80,000 msg/s: 4.8 million after a minute, 288 million after an hour. At 500 bytes each that is 144 GB an hour of backlog on a broker, or an out-of-memory crash within seconds on an in-process buffer. Worse than the volume is the age of the oldest message: after ten minutes the consumer is processing messages that arrived ten minutes ago, and every message it processes has been waiting longer than the one before. If the message was "update the price on the product page", a ten-minute-old update is wrong by the time it lands. A queue does not fix a rate mismatch; it converts it from an error into latency, and hides it until the latency is catastrophic.
Little's law makes this precise: the number of items in a system equals arrival rate times time spent in it, L = λ × W. Rearranged, W = L / throughput: with 4.8 million messages queued and a consumer doing 20,000/s, the newest message will wait 4.8M / 20k = 240 s before it is touched, regardless of anything else. Queue depth divided by consumer throughput *is* your latency. That is why the two numbers to watch on any queue are depth (or consumer lag, in Kafka terms) and oldest-message age, and why a queue that "never drains" after a feature launch is a capacity problem with a queue in front of it.
time backlog oldest msg age W = backlog / 20k 10 s 800,000 ~10 s 40 s 60 s 4,800,000 ~60 s 240 s 10 min 48,000,000 ~10 min 40 min 1 h 288,000,000 ~1 h 4 h (144 GB at 500 B/msg)
Strategies
There are only a few honest responses to "the consumer cannot keep up", and every real system uses several. Which ones apply depends on one question: is every message required, or is the latest good enough?
| Strategy | Mechanism | What it costs | Use when |
|---|---|---|---|
| Slow the producer | Flow control: the consumer grants credits / the producer blocks on a full bounded buffer (TCP windows, reactive streams request(n)) | Producer latency rises; upstream must tolerate it | You control both ends and losing messages is not acceptable |
| Bounded buffer | Queue with a max size; enqueue blocks or fails when full | Bursts above the bound are refused | Always — an unbounded buffer is a crash with a delay |
| Rate limit at the source | Token bucket per producer / tenant; 429 + Retry-After | Some requests are refused at the edge | Multi-tenant ingress; one hot producer must not starve the rest |
| Drop / sample | Discard some messages; keep 1 in N; keep only the latest per key | Data loss, by design | Metrics, telemetry, position updates — latest wins |
| Load shedding | Refuse low-priority work when depth or age passes a threshold | Degraded features under load | Mixed workloads with a clear priority order |
| Batch | Consumer handles 500 messages per DB round trip instead of 1 | Per-message latency rises slightly | Per-item overhead dominates (DB writes, HTTP calls) |
| Scale consumers | More workers / partitions | Money; downstream may be the real limit | Consumer is genuinely CPU- or I/O-bound and downstream has headroom |
Flow control: making the producer feel it
The most robust form of backpressure is the one TCP has used for decades: the receiver advertises how much it can take, and the sender never sends more. Reactive streams formalise this for application code — a subscriber calls request(n) and the publisher emits at most n items until asked again. A bounded in-process channel does the same thing implicitly: send blocks when the channel is full, so a producer that is 5× faster than its consumer simply runs at the consumer's pace and the buffer stays small. This is the default you want inside one process, and it propagates naturally: a blocked producer is itself a slow consumer of whatever feeds it, so the pressure travels upstream until it reaches an ingress that can say 429 or 503 to a client — the only place where the system can honestly refuse.
Between services the same principle takes the form of bounded prefetch on consumers (do not pull 10,000 messages into memory to process 10), consumer lag alerts that fire before age matters (Kafka-Style Logs: Topics, Partitions, Offsets), and admission control at the API (Rate Limiting): if the queue is deeper than the consumer can clear within its latency budget, stop accepting work that would only join the backlog. Note the interaction with Reliability Patterns: retries under backpressure are a producer *increasing* its rate exactly when it should decrease it, which is how a slow consumer turns into an outage.
When a queue hides a capacity problem
A queue is supposed to smooth bursts: 5 min at 100k/s followed by an hour at 5k/s averages well under 20k/s and the backlog drains. A queue that never drains means the *average* arrival rate exceeds consumer throughput, and no amount of buffering changes that. Teams that see "the queue is growing" and respond by raising the retention limit are buying disk to postpone the conversation. The fix is either more consumer throughput (workers, batching, a faster downstream — but check that the database is not the actual bottleneck, because ten more workers against one saturated database just move the queue into connection waits) or less arrival (rate limits, sampling, dropping work that is stale by the time it would run).
The engineering signal: put an SLO on oldest-message age (say, 95% of messages processed within 30 s) and alert on it. Depth alone lies — a deep queue with a fast consumer is fine; a shallow queue with a stalled consumer is not. Age measures what the user experiences.
Key points
- A queue turns a rate mismatch into latency, not into capacity;
W = L / throughputis the wait time of the newest message. - An unbounded buffer is a crash with a delay. Bound every queue and decide what happens when it is full.
- Strategies: slow the producer (flow control), rate limit at the edge, drop or sample where latest-wins, shed low priority, batch, scale consumers.
- Pressure should travel upstream to the ingress, where
429/503is an honest answer; retries under pressure make it worse. - Alert on oldest-message age, not just depth; a queue that never drains is a capacity problem wearing a queue.
Producer 100k/s, consumer 20k/s
depth over time ▁▂▃▄▅▅▆▇██ max 800 k Little's law L = λ × W → 800 k in queue = 20 k/s × 40.01 s wait
A queue absorbs bursts. It cannot fix a sustained producer > consumer imbalance — the excess has to go somewhere: into depth (latency, then memory), back to the producer (blocking), onto the floor (dropping), or into more consumers.
How data moves through it
One request or event, hop by hop.
- 1Producer → Bounded queue:
sendblocks (or fails fast) when the queue is at its bound. - 2Queue → Consumer: pulls in batches with a small prefetch; acknowledges after the batch is durable.
- 3Consumer → Database: one batched write per 500 messages instead of 500 writes.
- 4Queue metrics → Gateway: depth and oldest-message age drive admission control; the gateway returns
429withRetry-Afterwhen the age SLO is at risk. - 5Gateway → Client: the client backs off; the pressure has reached the only place that can refuse honestly.
When to use — and when not
- Any producer/consumer pair whose rates are not guaranteed to match — which is every queue, stream and channel.
- Ingress from many tenants where one hot client can flood a shared consumer.
- Telemetry, metrics and position streams where sampling or latest-wins is acceptable and volume is enormous.
- Rates are provably matched and bounded — a scheduled batch that processes exactly what one nightly job produced.
- Dropping is not acceptable *and* you cannot slow the producer: then the only option is capacity, and the backpressure conversation is really a scaling conversation.
Tradeoffs
Bounded buffers and flow control are cheap to add and cost almost nothing at runtime; deciding what to refuse or drop is a product decision that engineering cannot avoid by adding disk.
How it fails
- Unbounded in-memory buffer: the producer runs fine for minutes, then the process dies of OOM and loses everything in the buffer.
- Depth alarm without an age alarm: the queue looks "only" 50,000 deep while the oldest message is 40 min old and every update is stale.
- Consumers scaled up against a saturated database: throughput does not rise, connection waits do, and the queue keeps growing.
- Retries from producers when the consumer is slow: effective arrival rate goes *up*, the exact opposite of flow control.
- Prefetch set to thousands: one consumer holds messages it will not process for an hour while others sit idle.
How it scales
- Partition the stream by key so consumers scale horizontally; per-partition lag tells you which key is hot.
- Batch writes downstream so consumer throughput is bounded by round trips, not by item count.
- Move sampling and aggregation as close to the producer as possible: 1% of the volume at the edge is cheaper than 100% in the broker.
- When the consumer's database is the ceiling, the next step is Scale This System for the database, not more consumers.
How it interacts with databases, queues, caches, APIs and external systems
- Queue/log: bounded queue size (
x-max-length), Kafka consumer lag, SQS approximate age of oldest message — the metrics that drive everything else. - Cache/Redis: shared token buckets for producer rate limits;
XADD … MAXLENgives a bounded stream (see Redis: Data Structures, Not a Cache). - Database: batched inserts and
COPY-style bulk loads raise consumer throughput far more than more consumers do. - API gateway: admission control and
429based on downstream health — the end of the backpressure chain. - External APIs: their rate limits are backpressure applied to *you*; honour
Retry-Afterand cap concurrency per provider.