AgenticGENERALSCALE-SPECIFICSIMPLIFIED

Budgets, Deadlines and Step Limits

An agent loop terminates because you made it terminate — bounded in time, in steps, in tokens and in money.

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

What stops an agent loop from running forever, and from spending an unbounded amount of money while it does?

The requirement

The assistant should keep working until it has an answer. Finance would like the monthly bill to be predictable, and the API should not hold a connection open for four minutes.

The obvious build

Loop until the model returns a final answer instead of a tool call. The model knows when it is done, and a per-call timeout on the model API is enough.

Why it breaks

The model calls a failing tool, reads the error, tries a variation, fails again, and continues — a loop with no exit condition that your code controls (Retry Storms).

How it breaks in production

The model calls a failing tool, reads the error, tries a variation, fails again, and continues — a loop with no exit condition that your code controls (Retry Storms).
A per-call timeout bounds one call, not the sequence. Twenty calls of eight seconds each is a request that takes over two minutes.
Cost is unbounded per request: context grows with every tool result, so later steps are more expensive than earlier ones and the growth is superlinear.
One user, or one bug, can consume the account-level rate limit and take the feature down for everyone (Rate Limiting).
Long requests hold HTTP connections, worker slots and often pooled database connections for the whole run (Connection Pool Exhaustion).
The monthly billing alert is the first signal, which arrives days after the incident.

Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

An agent loop is a while loop whose termination condition is produced by a probabilistic process. Nothing in that guarantees termination, so termination must be imposed from outside.
Four independent quantities need bounds, and each fails differently if it is the only one: wall-clock time (user experience and resource holding), steps (loop convergence), tokens (context growth), and money (aggregate spend).
They are not substitutes. A step limit does not bound cost, because context grows within the allowed steps; a token limit does not bound time, because a slow provider can take minutes within budget.
Cost per step rises as the conversation grows: each turn resends accumulated context, so the tenth tool result is charged alongside everything before it (Token Budgets in the Agentic AI domain).
Budgets are needed at several levels — per request, per user or tenant, and per service — because bounding one request does not bound a thousand of them (Resource Limits).
What happens at the limit is a product decision: return the best partial answer, escalate to a human, fall back to a non-agent path, or fail. Silence is the one unacceptable option.

Four bounds, four different failures

Teams usually add one bound, discover it was insufficient, and add another after an incident. Adding all four at the start costs a few lines, and the table makes clear why none of them substitutes for the others.

Bound	What it protects	What it does not protect	If it is the only one
Wall-clock deadline for the run	User experience, connection and worker holding	Money — a fast loop can be expensive	Cheap runs that finish quickly and cost more than expected
Maximum steps	Loop convergence, internal service fan-out	Cost, because context grows per step	Terminating at step 20 having spent far more than 20 early steps would
Token ceiling per run	Context growth and per-request cost	Time — a slow provider stays within budget	A request that is affordable and takes four minutes
Cost ceiling per run and per tenant	Aggregate spend, one tenant starving others	A single very slow or very deep run	Predictable bills and unpredictable latency

The loop, with the bounds in it

SIMPLIFIEDCost is estimated from token usage for illustration; real accounting differs per model and per input/output split. The structure — check before spending, record why you stopped — is what transfers.

The important property of the code below is that every bound is checked inside the loop, before doing anything expensive, and that the reason for stopping is a value the caller and the metrics both receive.

A bounded agent loop

1type Stop = 'answered' | 'steps' | 'deadline' | 'tokens' | 'cost' | 'stalled'
2 
3async function runAgent(ctx: RunContext, input: string) {
4  const deadline = Date.now() + ctx.limits.wallClockMs
5  let steps = 0, tokens = 0, cents = 0
6  const recent: string[] = []
7  let messages = seed(ctx, input)
8 
9  const finish = (stop: Stop, answer?: string) => {
10    metrics.inc('agent.run', { stop })
11    metrics.observe('agent.steps', steps)
12    metrics.observe('agent.cost_cents', cents)
13    return { stop, answer, steps, tokens, cents }
14  }
15 
16  for (;;) {
17    if (steps >= ctx.limits.maxSteps) return finish('steps')
18    if (Date.now() >= deadline) return finish('deadline')
19    if (tokens >= ctx.limits.maxTokens) return finish('tokens')
20    if (cents >= ctx.limits.maxCents) return finish('cost')
21    // tenant-wide ceiling: atomic, shared across instances
22    if (!(await tenantBudget.tryConsume(ctx.tenantId, ESTIMATE_CENTS))) {
23      return finish('cost')
24    }
25 
26    steps++
27    const remaining = deadline - Date.now()
28    const turn = await model.complete(messages, {
29      tools: ctx.allowedTools,            // allowlist, from the user's role
30      timeoutMs: Math.min(remaining, PER_CALL_MS),
31    })
32    tokens += turn.usage.total
33    cents += price(turn.usage)
34 
35    if (turn.kind === 'answer') return finish('answered', turn.text)
36 
37    // non-convergence: the same call repeated is not progress
38    const sig = `${turn.tool}:${stableHash(turn.args)}`
39    if (recent.filter((s) => s === sig).length >= 2) return finish('stalled')
40    recent.push(sig)
41 
42    const result = await dispatchTool(ctx, turn, {
43      timeoutMs: Math.min(deadline - Date.now(), TOOL_MS),
44    })
45    messages = append(trim(messages, ctx.limits.contextTokens), turn, result)
46  }
47}

Three details carry most of the value: the deadline is computed once and every downstream call receives only the time remaining; the tenant budget is consumed atomically so instances cannot each grant the same allowance; and the stall check terminates a non-converging loop long before the step limit would.

What to do at the limit

Hitting a budget is a normal outcome, not an error, and the handling decides whether the feature feels reliable or broken. The worst option is the one that happens by default: an exception, a generic 500, and no indication of what was already done.

Reaching a bound: symptom, cause, response

Trigger	Symptom	Cause	Response
Step limit reached	User gets nothing after a long wait	Plan did not converge within the allowance	Return partial progress and what remains; log the trajectory for analysis (Agent Audit Logs)
Deadline exceeded mid-tool	Response returns but the side effect lands afterwards	Cancellation not propagated to the in-flight call	Pass the cancellation signal into every tool; make writes idempotent (Idempotency Keys)
Token ceiling reached early	Runs stop after very few steps	Tool results are large and never trimmed	Return ids and summaries; let the model fetch detail on demand
Tenant cost ceiling reached	One tenant's feature stops working mid-day	Aggregate budget consumed, correctly	Surface it as a quota condition with a reset time, not as an error (Quotas vs Rate Limits)
Provider rate limit hit	429s from the model API across all tenants	Aggregate demand exceeded the account quota	Queue and shed at your edge; fair-share across tenants (Backpressure)
Stalled loop detected	The same tool called repeatedly with identical arguments	The model cannot interpret the tool's error response	Improve the error message the tool returns; terminate and escalate (Tool Errors, Retries and Timeouts in the Agentic AI domain)
Terminated after irreversible action	Refund issued, ledger entry missing	The bound fired between two steps of one logical operation	Make steps individually safe, or gate the irreversible one behind approval

How to build it

Most important first.

One deadline for the whole run, established at the start and propagated to every model call and tool call, so each one gets only the time remaining (Timeouts).
A hard step limit, with the termination reason recorded as a first-class label.
A token ceiling per run, checked before each model call using the accumulated count — not only the provider's per-call maximum.
A cost ceiling per run and per tenant, enforced in code. A monthly alert is a report, not a control (Cost per Request: The Other Performance Metric in the Observability domain).
Trim the context deliberately — summarise or drop old tool results, return ids rather than blobs — so the budget buys more steps (Context Selection & Compression in the Agentic AI domain).
Define the behaviour at each limit and make it visible to the user: a partial answer with what was completed, or an explicit handoff.
Detect non-convergence early: repeated identical tool calls, or the same error twice, should terminate before the step limit does.
Move long runs into a job so the deadline is not also an HTTP connection lifetime (Background Jobs).
Cap concurrent runs per user, because a per-run budget multiplied by unlimited runs is not a budget (Unbounded Concurrency).

What can go wrong

Failure modes

Budgets defined but not enforced — computed for the dashboard and never checked in the loop.
A deadline that is not propagated, so a tool started at the last second runs long past the run's end and leaves side effects behind.
Terminating mid-plan after irreversible side effects, leaving state that needs reconciliation (The Dual Write Problem).
Limits so tight that legitimate multi-step tasks never complete, which teams then relax globally instead of per task type.
Per-instance budget counters, so the effective limit multiplies by fleet size (Stateless Services).
Retries around the whole agent run, multiplying the budget by the retry count.
Cancellation that returns to the caller but does not actually stop in-flight tool calls, so work and spend continue invisibly.

What can race

Concurrent runs sharing a tenant budget race on the counter; use an atomic increment rather than read-modify-write (Atomic Operations).
A deadline can fire while a tool call is in flight, so cancellation must be propagated or the side effect lands after the run ended (Graceful Shutdown).
Two runs can each pass a check against the same remaining budget and jointly exceed it.

Security

Unbounded cost is a denial-of-wallet vulnerability: an attacker who can trigger agent runs can convert traffic directly into your money (Rate Limiting).
Per-user and per-tenant budgets are what stop one caller from consuming a shared provider quota and denying service to everyone else (Multi-Tenancy).
Prompt injection can aim for the budget as well as the data — instructing the model into a long tool loop is a cheap resource-exhaustion attack (Prompt Injection in the Security domain).
Budget-exhaustion messages should not disclose internal limits, model names or cost structure to end users (Not Leaking Your Internals).

Misreads

"The model stops when it is done." It stops when it emits a final answer, which is a probabilistic event, not a guarantee.
"We set a timeout on the model API." That bounds one call. The loop is the thing that needs a deadline.
"A step limit bounds cost." Context grows within those steps; ten steps late in a long conversation can cost more than fifty early ones.
"We alert on monthly spend." An alert is a report after the fact. The control has to be inside the loop.
"Budgets hurt quality, so we should be generous." Generous per-run budgets with unlimited concurrent runs is not a budget at all.

Operating it

How you see it in production

Termination reason as a label on every run: answered, step limit, deadline, token ceiling, cost ceiling, tool error, cancelled. This single distribution explains most agent production behaviour.
Steps per run and tokens per run as distributions, watching the tail rather than the mean.
Cost per run, per user and per tenant, with alerting on rate of change rather than absolute monthly total.
Budget-exhaustion rate per feature: rising exhaustion means either budgets are too tight or plans are not converging.
Time to first token and total run duration separately — one is the user's perception, the other is your resource holding (Inside One Model Call: Queue, First Token, Generation in the Observability domain).
Repeated-identical-tool-call counts, the earliest signal of a non-converging loop.

What changes at 10x and 100x

Provider rate limits become the binding constraint before your own capacity does; queueing and shedding have to happen at your edge (Backpressure).
Budget counters must be shared across instances to mean anything, which makes them a shared store on the request path (Atomic Operations).
Cost per request stays roughly constant as you scale, which makes agent features one of the few backend components whose marginal cost does not improve with volume.
At small scale a step limit and a deadline are enough; token and cost ceilings become essential when many tenants share one provider account.

What this costs

Tighter budgets mean more truncated answers. The right limits differ per task, so a single global number is always wrong somewhere.
Context trimming reduces cost and can remove information the model needed, producing worse answers for less money.
Shared budget counters add a dependency and latency to every step (Calling Something You Do Not Control).
Moving runs into a job system fixes resource holding and adds a queue, a state store and a progress protocol (Job Queues).

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

GENERALApplies to any agent loop regardless of provider or framework.
SCALE-SPECIFICA single-tenant internal tool can live with a step limit and a deadline. A multi-tenant product needs per-tenant token and cost ceilings enforced in a shared store, because one tenant can otherwise consume the account quota.
SIMPLIFIEDCost is treated here as proportional to tokens. Real pricing varies by model, by input versus output tokens, and by features such as cached prompt prefixes — none of which changes the requirement for an in-loop ceiling.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

API Designquotas-vs-rate-limits rate-limiting-contract

Architecturerate-limiting backpressure

Performanceagent-cost cost-per-request llm-latency timeouts-and-latency

Cloudcost-engineering

Concurrencyconcurrent-backpressure

Agenticbudgets-limits-termination token-budgets tool-errors-retries-timeouts fallbacks-caching-routing reliability-overview context-selection-compression

Domains that do not exist yet

— System Design — deciding the latency and cost envelope a feature is allowed to occupy before choosing an architecture that fits inside it.