The question this answers
When this operation takes too long, what exactly happens — to the caller, and to the operation?
A checkout handler calling an inventory service with a 500 ms timeout. Inventory is degraded and responding in 2.5 seconds. Checkout traffic is 400 requests per second.
The connection or task the operation holds, and — once the caller has given up — nothing the caller can see. That invisibility is the entire problem.
The caller returns within its budget, and every operation the caller abandoned releases its resources within a bounded time rather than accumulating.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
A timeout is two decisions pretending to be one
Writing timeout: 500ms feels like one decision. It is two. The first is how long the caller waits — a latency decision, driven by what the caller's own budget allows. The second is what happens to the operation when the caller stops waiting — a resource decision, and one that most timeout APIs quietly answer as "nothing".
Promise.race([work, timeout]) is the clearest illustration. It resolves after 500 ms with a timeout error, and work is entirely unaffected: still connected, still consuming a connection from the pool, still going to complete in 2.5 seconds and resolve a promise nobody holds. The caller has moved on and probably retried. A timeout without cancellation just stops waiting; the work continues.
At 400 requests per second with a 500 ms timeout against a 2.5 second dependency, each request abandons work that lives for two more seconds. That is roughly 800 abandoned operations alive at any moment, each holding a connection, on top of the 200 live ones. The pool is exhausted, so new requests fail to acquire, so latency rises, so more requests time out, so more work is abandoned. The timeout that was supposed to protect the system is what is destroying it (Retry Storms: The Load You Generated Yourself and The Bottleneck Moves After Every Fix in Performance).
Wiring it so the work actually stops
The fix is that the timeout and the cancellation must be the same event. Create the deadline as a cancellation source, pass its signal into the operation, and let the operation's own abort path release the resource. Then "deadline exceeded" is not a race the caller wins against the work — it is a message the work receives.
Everything from Cancellation applies here, including the limits: if the operation is a CPU loop with no check points, or a blocking read the driver will not abort, the signal changes nothing and you are back to abandonment. In that case be honest about it — you have a bounded *wait* and an unbounded *work*, and the protection you need is a concurrency limit so abandoned work cannot exceed a known share of capacity (Bounding Concurrency).
The other half of wiring is the retry. A retry after a timeout is a second copy of an operation that may still be running, so retries and timeouts compose into load multiplication. Only retry when the operation is idempotent, only with backoff and jitter, and only within the caller's remaining budget — retrying at 480 ms into a 500 ms budget accomplishes nothing except doubling downstream load (Retries and Timeouts as Contract Guidance and Idempotency Keys: The Mechanism in API Design).
1async function getInventory(sku: string) {2 return Promise.race([3 inventoryClient.get(sku), // still running afterwards4 new Promise((_, rej) =>5 setTimeout(() => rej(new Error('timeout')), 500)),6 ])7}8// At 400 rps against a 2.5s dependency this leaves ~800 abandoned calls9// alive at all times, each holding a pool connection. The pool is the10// resource that fails, not the dependency.11// The setTimeout is also never cleared on the success path: a pending12// timer per call, which is its own slow leak.1async function getInventory(sku: string, parent: AbortSignal) {2 const signal = AbortSignal.any([3 parent, // caller went away4 AbortSignal.timeout(500), // our own budget5 ])6 try {7 return await inventoryClient.get(sku, { signal }) // client aborts the request8 } catch (e) {9 if (signal.aborted) {10 metrics.timeouts.inc({ dep: 'inventory' })11 throw new DependencyTimeout('inventory', 500) // distinct from a 50012 }13 throw e14 }15}16// The abort reaches the transport: the socket is closed, the pool slot17// returns immediately, and the inventory service can stop too if it18// notices the disconnect.The race version makes the timeout a property of the caller. The signal version makes it a property of the operation. Only the second one bounds resource usage — the first bounds only how long a human waits for the error message.
Choosing the number, and what a timeout cannot do
A timeout is not "a bit more than usual". Set it from the caller's budget working downward: the user-facing target is 800 ms, the handler needs 100 ms of its own work, there are two sequential dependency calls, so each gets roughly 350 ms — and if that is below the dependency's own p99, the design is wrong and no timeout value fixes it. Setting a timeout above your own budget is pointless: the caller upstream has already given up (Latency Budgets: Spending 200 Milliseconds on Purpose in Performance).
Two common values are both wrong for the same reason. A timeout set at the dependency's p50 turns normal variance into constant failure. A timeout set at ten times its p99 never fires until the system is already dead, which means it protects nothing and merely converts a hang into a very slow hang. And the tempting "no timeout" is a decision to wait forever, which is how one degraded dependency stalls every thread in a pool (The Thread Pool Server in Operating Systems).
Finally, be clear about what a timeout is not. It is not a health check — it fires on one operation and says nothing about the dependency's general state; that is what a circuit breaker is for (Circuit Breaker in Architecture). It is not a correctness mechanism — a write that timed out may have succeeded, and the only way to know is an idempotency key and a reconciliation. And it is not a substitute for capacity: timeouts change how you fail, never whether the work fits.
| # | Checkout handler | Payment service | Payment ledger | State |
|---|---|---|---|---|
| 1 | POST /charge (500 ms timeout), no idempotency key | · | · | attempts=1 charged=0 |
| 2 | · | receives request; begins charge | · | attempts=1 charged=0 |
| 3 | 500 ms elapsed → timeout error | · | · | attempts=1 charged=0 caller=gave up |
| 4 | · | charge succeeds at 640 ms; writes ledger row | · | attempts=1 charged=1 |
| 5 | retries POST /charge | · | · | attempts=2 charged=1 |
| 6 | · | no idempotency key — treats it as a new charge | · | attempts=2 charged=1 |
| 7 | · | · | second ledger row written | attempts=2 charged=2 ✕ The customer was charged twice. No concurrency bug occurred: a timeout plus a retry on a non-idempotent write is sufficient on its own. |
| 8 | with an idempotency key, the retry would have matched the first row and returned it | · | · | attempts=2 charged=1 |
Key points
- A timeout is two decisions: how long the caller waits, and what happens to the work. Most APIs answer the second one as "nothing".
- A timeout without cancellation just stops waiting — the work keeps its connection and its CPU while the caller has already retried.
- At high rates, abandoned work accumulates until the connection pool, not the dependency, is the thing that fails.
- Make the deadline a cancellation source and pass its signal into the operation, so "deadline exceeded" reaches the work instead of racing it.
- Derive the value from your own latency budget downward, not from the dependency's typical latency upward.
- A timeout on a write turns failure into ambiguity; only an idempotency key makes the retry safe.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • A deadline is computed at the start of the operation, ideally as an absolute instant rather than a duration (Deadlines vs Timeouts).
- • A cancellation source is created that fires at that instant, and its signal is passed into the operation.
- • The operation performs its work, checking the signal at suspension points and passing it to anything it calls.
- • If the work completes first, the timer is cancelled — an uncleared timer per call is a slow leak in long-lived processes.
- • If the deadline fires first, the signal aborts the operation: the transport closes the connection, cleanup runs, and the caller receives a distinct timeout error.
- • The caller decides whether to retry based on idempotency and on how much of its own budget remains.
- • Work completes at 480 ms; the timer fires at 500 ms into a resolved promise and does nothing. Harmless, except the timer object lives until it fires.
- • Work completes at 505 ms; the caller has already returned an error at 500 and retried. The system now has two operations for one request, and if the operation is a write, possibly two effects.
- • Race-based timeout: caller returns at 500 ms; work runs to 2 500 ms holding a connection. At 400 rps this accumulates ~800 abandoned operations, and the pool fails before the dependency does.
- • Signal-based timeout: the abort reaches the transport at 500 ms, the socket closes, the pool slot returns at 502 ms. Steady-state abandoned work is zero.
- • Timeout fires during cleanup: the operation is aborting, the connection release is slow, and a second deadline elsewhere fires on the same resource. Cleanup must be idempotent (Cancellation Propagation).
- • Write ambiguity: charge succeeds at 640 ms, response lands on a closed socket, caller retries without an idempotency key, customer is charged twice.
- • A timeout guarantees the caller returns within a bounded time. That is its only unconditional guarantee.
- • With cancellation wired in, it additionally guarantees the operation is *asked* to stop and, for cancellable operations, that its resources are released promptly.
- • It does NOT guarantee the work stopped — cooperative cancellation and uncancellable leaves both apply (Cancellation).
- • It does NOT tell you whether the operation succeeded. After a timeout the outcome is unknown, which is strictly worse than known failure.
- • It does NOT protect the dependency. It protects the caller; the dependency is still receiving the load, plus any retries.
- • It does NOT compose across hops on its own. Each hop timing out independently produces a total far longer than any single value (Deadlines vs Timeouts).
- • Abandoned operations hold connections, threads or tasks, so an uncancelled timeout converts a latency problem into a pool-exhaustion problem (Pool Saturation).
- • Timers themselves are a shared structure; at very high rates, timer wheel insertion and cancellation are measurable, and uncleared timers accumulate.
- • Synchronised timeouts create correlated bursts: everything started at the same moment times out at the same moment and retries at the same moment, which is a thundering herd unless retries are jittered (Thundering Herd).
- • Cleanup after mass timeout contends on the resources being released, exactly when they are scarcest.
- • Abandonment: the caller returns and the work continues, invisibly, until the pool is exhausted.
- • Retry amplification: each timeout produces another copy of an operation that is still running, multiplying load on a struggling dependency.
- • Duplicate side effects on non-idempotent writes.
- • Timer leak from timers never cleared on the success path.
- • Cascading timeout: a slow dependency causes upstream timeouts, whose retries slow it further, converting degradation into failure (The Bottleneck Moves After Every Fix in Performance).
- • Useless work: with per-hop timeouts and no deadline, an operation completes at hop four for a caller who gave up at hop two.
- • On every outbound call. An unbounded wait means one degraded dependency can consume every thread or task in the process.
- • When the caller has a real budget to protect — user-facing paths, anything with an SLO (Latency Budgets: Spending 200 Milliseconds on Purpose in Performance).
- • On read paths, where abandoning is cheap and safe and the only cost is a wasted query.
- • As the trigger for a fallback: a timeout with a cached or degraded response is often far better for the user than a slow correct answer (Reliability Patterns in Architecture).
- • On non-idempotent writes without an idempotency key, where the timeout creates an ambiguity the retry then turns into a duplicate.
- • Without cancellation at high request rates, where it actively accelerates the failure it was meant to prevent.
- • When set below the dependency's normal p99, turning ordinary variance into a constant error rate and a constant retry load.
- • When each hop has its own generous timeout and there is no overall deadline, so total latency is the sum and the user waits for all of it.
- • Timeout rate per dependency, as its own counter — never folded into a generic error rate, because the remediation is completely different.
- • The dependency's latency distribution against your timeout value, plotted together. If the timeout sits inside the normal distribution, it is misconfigured.
- • In-flight operation count versus caller-waiting count. A persistent gap is abandoned work, and it is the number that proves the race-based timeout is hurting you.
- • Pool acquisition wait time during timeout bursts, which is where abandonment surfaces first (Connection Pool Saturation: Waiting in Front of an Idle Database in Performance).
- • Duplicate-effect rate on write paths after timeouts — the direct measure of whether idempotency is actually working.
- • Every outbound call site gains a value that must be justified, reviewed and maintained as latencies drift.
- • Doing it properly means threading a signal through, which is the whole plumbing cost of Cancellation.
- • Retry policy becomes entangled with timeout policy and with idempotency; the three must be designed together or they compose into duplication.
- • Testing needs a controllably slow dependency, which most test suites do not have, so timeout behaviour is usually first exercised in production.
- • A deadline propagated through the whole call chain, which composes correctly across hops where per-hop timeouts do not (Deadlines vs Timeouts).
- • A circuit breaker, when the dependency is failing rather than merely slow — it stops sending requests at all instead of timing each one out (Circuit Breaker in Architecture).
- • A bounded concurrency limit per dependency: even if calls are abandoned, no more than N can exist, so the pool cannot be exhausted (Bounding Concurrency).
- • Make the call asynchronous: accept the request, return a job id, and let the client poll. Removes the timing coupling entirely (The Async Job Pattern in API Design).
The deadline expired. What happened to the work?
try:
result = await wait_for(call(req), 300ms) # only the *wait* is bounded
except Timeout:
return 504 # call() is still running, on a worker, right nowWhat people believe, and what is true
The operation timed out, so it stopped.
Unless the timeout is wired to cancellation, only the waiting stopped. The operation keeps its connection and finishes into a void.
A timeout means the operation failed.
It means the outcome is unknown. On a write, "unknown" plus "retry" equals "done twice" unless the write is idempotent.
Timeouts protect the downstream service.
They protect the caller. The downstream is still doing all the work, and now also handling the retries the timeout triggered.