Request or Background?
Five questions that decide where work runs — and the reminder that deferring is a cost, not a default.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
How do I decide whether this piece of work belongs in the request or in a queue?
Checkout does six things. Some of them the user is waiting for, and some of them they are not — but nobody has written down which is which.
Anything slow goes to a queue. Speed is what users notice, so move everything that takes time out of the request path.
The user was waiting for that answer. Moving the payment authorization to a queue means the response cannot say whether the card was accepted, so the UI has to poll — and the checkout that was slow is now slow *and* uncertain.
- The user was waiting for that answer. Moving the payment authorization to a queue means the response cannot say whether the card was accepted, so the UI has to poll — and the checkout that was slow is now slow *and* uncertain.
- Validation deferred to a worker means the API accepts requests it will later reject, with no way to tell the caller. The 202 was a lie and support finds out first (Business Validation).
- The work was 40ms. It now costs a broker write, a queue hop, a worker pickup, a retry policy and a dashboard, in exchange for saving 40ms nobody could perceive.
- Immediately after the response, the user's next page load reads state the job has not written yet, and it looks like a bug in the write they just made (Eventual Consistency in Practice).
- The work was not retryable — a non-idempotent charge — and the queue retried it (Job Idempotency).
What is actually happening
- The decision is not about duration. It is about who is waiting for the answer and what happens if it fails later. A 2-second call the user is waiting for stays in the request; a 50ms call nobody is waiting for can be deferred if it has other reasons to be.
- Five questions settle almost every case: must the user see the result, is the work slow or variable, is it safe to repeat, is immediate consistency required, and can a failure be handled after the fact?
- Note that "slow" is the weakest of the five. It is a reason to *want* to defer, never on its own a reason that you *can*.
- There is a middle option people forget: keep it in the request but make it fast — batch the queries, drop the payload size, parallelise the independent calls. Deferring is not the only way to shorten a handler (Eager Loading and Batching).
- And there is a hybrid: do the part the user is waiting for synchronously (authorize the payment, validate the address) and defer the part they are not (the receipt email, the analytics event, the index update). Most checkouts end up here.
- Deferring changes the contract, so it is an API decision as much as an implementation one: 201 with a resource becomes 202 with a job id and a way to check on it (The Async Job Pattern).
The five questions
Ask them in order, of each piece of work separately. The first "no" that matters usually settles it — and note that a "yes" to the first question ends the discussion regardless of how slow the work is.
The criteria are the lesson here; there is no winning option. A checkout typically produces different answers for the payment authorization, the stock decrement, the receipt email and the analytics event, and the design that results is a split, not a choice.
Answer these five, in order, for this specific step
when Yes — payment authorization, validation, the id of the thing created.
cost It stays in the request. Now your latency is bounded by this dependency, so it needs a timeout and a fallback (Timeouts).
when Yes — image processing, export generation, fan-out to many recipients.
cost It wants to be deferred, but wanting is not permission. Question 1 outranks this one.
when Yes — idempotent, or protected by a key.
cost If no, it cannot go on a queue as-is: at-least-once delivery means it will eventually run twice (Job Idempotency).
when Yes — the user is redirected to a page that displays this state.
cost Keep it synchronous, or accept a visible consistency gap and design the UI for it (Eventual Consistency in Practice).
when Yes — retry, dead-letter and a human, or a compensating action.
cost If no — nothing can undo it and nobody will notice it missing — it belongs in the request where the caller sees the error.
when Side effects on other systems: email, search index, webhooks, analytics, thumbnails.
cost A broker, workers, retries, a DLQ, dashboards and an eventual-consistency window you must state.
One checkout, six answers
Applying the questions to a real operation is what makes the split obvious. The user is waiting for two of these six. Two more must be consistent with the next page they will see. The remaining two are side effects on other systems that nobody is watching in real time.
The result is not "checkout is async" or "checkout is sync". It is a short synchronous core and four deferred effects, with the API returning the order and its confirmed payment state.
| Step | User waits? | Repeat-safe? | Where it goes | Why |
|---|---|---|---|---|
| Validate the cart | Yes | Yes | Request | A rejection must be an error the caller sees (Business Validation) |
| Authorize the payment | Yes | Only with a key | Request | The response has to say whether the card was accepted; the key makes the retry safe (Idempotency Keys) |
| Write the order and decrement stock | Yes | Within one transaction | Request | The next page reads it, and stock is disputable — it cannot be eventually consistent (Where the Transaction Boundary Goes) |
| Send the confirmation email | No | Needs a dedupe key | Background | A third-party call with unbounded latency, and nothing about the response depends on it |
| Update the search index | No | Yes — a write of current state | Background | Consistency within seconds is acceptable (Keeping a Search Index in Sync) |
| Notify the warehouse | No | Needs an idempotency key | Background | Must survive a client disconnect and must retry on their outage (Retries) |
The third option: make it fast instead
Before a queue is designed, it is worth asking whether the handler is slow for a reason a queue does not address. Work that is slow because of an N+1, an unbounded result set or three sequential independent calls does not become cheaper on a worker — it becomes invisible, which is worse.
The version on the right keeps the precise, synchronous contract the caller wanted. No broker, no worker fleet, no dead-letter queue and no consistency window. When it is achievable it is strictly the better outcome; deferring should be what you do when it is not.
// 1 query for the order + 1 per line item + 3 sequential API calls
await queue.enqueue('build-order-summary', { orderId })
return res.status(202).json({ jobId })
// The user now polls for something they used to get directly,
// and the N+1 still runs — on the worker fleet, unmonitored.const [order, tax, shipping, promo] = await Promise.all([ db.orderWithItems(orderId), // one query, joined — no N+1 taxApi.quote(cart), // the three calls were shippingApi.quote(cart), // independent all along promoApi.check(cart), ]) return res.status(200).json(buildSummary(order, tax, shipping, promo))
Deferring does not make work cheaper; it makes it later and less visible. Fixing the query and parallelising independent calls removes the latency at its source and keeps a synchronous contract that needs no polling, no worker fleet and no eventual-consistency window. Defer when the work is genuinely unbounded or genuinely nobody's business but yours — not because a handler is slow.
How to build it
Most important first.
- Split the operation before deciding. "Checkout" is not one thing; it is six, and the answer is usually different for each.
- Keep in the request: anything whose result the response must carry, anything whose failure must become an error the caller sees, and anything that must be consistent with the read the user makes next.
- Defer: side effects on other systems (email, search index, webhooks, analytics), work whose duration is unbounded or driven by data size, and anything that must survive a client disconnect (What Happens After the Bytes Land).
- Before deferring, check retry safety. If the work is not idempotent, make it idempotent first — that is a prerequisite, not a follow-up (Idempotency Keys).
- When you defer, give the caller an observable outcome: a job id, a status field on the resource, or a notification. Silence is not an acceptable async contract.
- Prefer "fast in the request" over "deferred" whenever it is achievable. A synchronous path that is quick has no queue, no worker fleet, no backlog and no eventual-consistency window.
What can go wrong
- Deferring something the user is waiting for, producing a polling UI and a support queue.
- Deferring validation, so the API accepts what it will later reject and the rejection has nowhere to go.
- Keeping an unbounded operation in the request path, so it is killed by an upstream timeout partway through, leaving half-applied state (Timeouts).
- Deferring a non-idempotent side effect and discovering duplicates only after the first retry storm.
- Splitting so finely that a single user action becomes eight jobs with eight independent failure modes and no overall status.
- Deferring to avoid fixing an N+1, so the same bad query now runs on the worker fleet where nobody is looking at latency (The N+1 Query Problem).
- Read-your-own-writes: the user is redirected to a page whose data the deferred job has not written yet.
- Two deferred jobs for one user action completing out of order, so a later state is overwritten by an earlier one (Queue Semantics).
- A synchronous step and its deferred follow-up racing against a user's immediate second request — cancel-before-fulfil is the canonical example.
- Deferred cache invalidation losing a race with the repopulating read (Cache Invalidation).
- A deferred authorization check is not a check. Evaluate permission at request time *and* re-evaluate in the worker, because the actor's rights can change between the two (Authorization in Backends).
- Anything user-visible that is deferred must not leak through timing: a job that reveals whether an account exists is an enumeration oracle whether it is sync or async.
- Rate limits belong on the enqueue, not on the worker. Limiting the consumer lets an attacker fill the queue for free (Rate Limiting).
- Deferring audit logging is a mistake: the audit record for a security-relevant action should commit with the action, not depend on a worker that might dead-letter (Agent Audit Logs).
- "Slow means async." Slow means *investigate*. Sometimes the answer is an index, a batch or a smaller payload, none of which add a queue (Should I Add an Index?).
- "Async is always more scalable." It relocates load. Enough deferred work outruns your workers and becomes a backlog, which is a worse failure than a slow endpoint because it is invisible until it is large (Queue Backlog).
- "Retryable means safe to retry." Those are different properties. Retryable means the transport permits another attempt; safe means the business effect does not repeat. Only the second one lets you defer a charge (At-Least-Once Delivery).
- "202 is a more modern response." 202 is a promise to do something later. If you cannot tell the caller how to find out whether it happened, it is a worse contract, not a more modern one.
- "We can add the queue later." The API shape, the idempotency requirement and the consistency window are all decided by this choice. Retrofitting means changing the contract clients already depend on.
Operating it
- Measure end to end, from the user's click to the effect being visible — not just handler duration. Deferring makes the handler metric better and the user experience possibly worse (Latency Budgets: Spending 200 Milliseconds on Purpose).
- Track the eventual-consistency window per job type: enqueue to completion, at p50 and p99. That number is what a user can trip over.
- For anything the caller polls, count polls per job. A high number means the async contract is being used as a slow synchronous one.
- Break down handler time by step before deciding what to defer. The step you assume is slow is frequently not (Why Is My API Slow?).
- At 10x the case for deferring strengthens for side effects, because variance in third-party latency stops being absorbed by the request path.
- At 10x the case for deferring *weakens* for anything the user waits on, because a longer queue makes the wait less predictable, not more.
- At 100x, split queues by job class so a slow bulk job cannot delay a latency-sensitive one (Bulkheads).
- Sometimes nothing changes: an operation that is fast and must be synchronous stays synchronous at every scale, and that is the correct answer rather than a missed optimisation.
- Deferring buys a fast, predictable response and costs an eventual-consistency window plus a second system to operate and watch.
- Keeping work synchronous buys a precise answer and couples your latency to your slowest dependency.
- The hybrid split buys most of both and costs the most design effort: two paths, two failure stories, and an API that must describe both.
- Making the synchronous path fast instead of deferring is the cheapest outcome to operate and often the most work to achieve.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- GENERALThe five questions apply to any backend regardless of stack or broker.
- RUNTIME-SPECIFICWhat "slow in the request path" costs depends on the concurrency model. On a single-threaded event loop, CPU-bound work in a handler blocks every other in-flight request in that process, so it must move even if it is short (Blocking the Event Loop). On a thread- or process-per-request model the same work occupies one worker and degrades throughput gradually instead.
- CLOUD-SPECIFICManaged load balancers and API gateways impose maximum request durations you cannot exceed from inside the application. Where that ceiling sits differs by provider and product, and work that might exceed it has to be deferred regardless of what the five questions say (Serverless Trade-offs).
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.