advancedBackground Jobs & Queues

Producers Outrunning Consumers

Decide what you would build from the brief alone. Everything below it is available, but the exercise stops working if you open it first.

The brief you were given

An ingest endpoint accepts events and enqueues them. During a traffic surge the queue depth grows continuously and processing falls hours behind. Design the response.

The trap — the wrong fix that looks right

Scaling the workers up sharply because the queue is deep and the workers are the visible consumer. If the drain is limited by the shared database, the new workers add write contention and connection pressure — the drain rate barely moves, and the API path slows because it is competing for the same pool. The queue keeps growing, now with a larger bill and a slower service, and the graph looks exactly the same as before.