What One Agent Run Costs, and Which Term Dominates
Cost per run is tokens times price times steps — and the tokens term grows every step, because each tool result is appended to a context that every subsequent call must pay for again. That quadratic-ish growth is why long runs cost far more than their step count suggests.
Frame the diagnosis
Performance work starts from a symptom and a signal — never from a resource dashboard.
The cost model, and the term everyone underestimates
Per-run cost is the sum over steps of (input tokens × input price) + (output tokens × output price). Two things about this surprise people. First, input tokens usually dominate the *count* even though output tokens dominate the *unit price* — a run sending 48k input tokens and producing 2k output tokens can be input-dominated even at a 3–5x price ratio. Second, input tokens are not constant across steps: every tool result is appended, so step six pays for everything steps one through five accumulated.
That accumulation is the crux. If each step adds roughly a fixed amount of context, then total input tokens across a run grow with the *square* of step count rather than linearly — step one pays for the base prompt, step two for base plus one result, step three for base plus two results, and so on (Token Budgets). A run that takes twice as many steps can cost close to four times as much, which is why step-count outliers are so disproportionately expensive.
This is also why the highest-leverage cost work is identical to the highest-leverage latency work: reducing step count and trimming context improve both at once (Where an Agent Run Actually Spends Its Time). Cost and latency in agent systems are unusually well aligned, which is a pleasant change from most performance work where they trade against each other (Every Optimization Buys Something and Sells Something).
1// ILLUSTRATIVE — substitute your provider's current prices.2type Step = { inputTokens: number; outputTokens: number }3 4const PRICE_PER_1K_IN = 0.003 // placeholder5const PRICE_PER_1K_OUT = 0.015 // placeholder6 7function runCost(steps: Step[]) {8 let inTokens = 09 let outTokens = 010 for (const s of steps) {11 inTokens += s.inputTokens12 outTokens += s.outputTokens13 }14 return {15 inTokens,16 outTokens,17 usd: (inTokens / 1000) * PRICE_PER_1K_IN + (outTokens / 1000) * PRICE_PER_1K_OUT,18 }19}20 21// Context accumulates: each step resends everything before it.22// base prompt 4k, each tool result ~3k, output ~300 per step.23function simulate(stepCount: number) {24 const steps: Step[] = []25 let context = 400026 for (let i = 0; i < stepCount; i++) {27 steps.push({ inputTokens: context, outputTokens: 300 })28 context += 3000 // the tool result is appended for every later step29 }30 return runCost(steps)31}32 33simulate(3) // in ~21k → the linear intuition34simulate(6) // in ~69k → 2x the steps, ~3.3x the input tokens35simulate(12) // in ~246k → 4x the steps, ~12x the input tokensWhere the money actually goes
Break spend down by task type, then by step count within task type. Almost always a small fraction of runs consumes a large fraction of the budget, and those runs share a property: high step count, usually from exploration or a retry loop. Fixing the long tail of runs is more valuable than shaving tokens off the common case, and it is the same work that fixes your latency tail (Tail Latency: Why p50 Being Fine Does Not Help).
Model choice is the other large term and the one with the most obvious lever. Routing simple requests to a smaller model, or using a small model for classification and routing while reserving the large one for the actual reasoning, can move costs substantially (Router Architecture). The constraint is quality, and the only honest way to make that trade is to measure it — swap the model, run the eval suite, compare accuracy against cost, and decide with both numbers in front of you (Eval Metrics: What to Measure and How).
Retries deserve a specific line item. A retry after a schema validation failure costs a full round trip of input tokens — the entire accumulated context, resent. A run with three retries can cost double a run with none, and the retries are usually fixable at their source: a clearer tool schema, a stricter structured-output constraint, better argument validation (Tool Schemas, Structured Outputs).
| Signal | Value | What it tells you | Verdict |
|---|---|---|---|
| Total model spend | $41,200/month | The headline. Says nothing about where to act. | suspect |
| Runs above 10 steps | 4% of runs, 38% of spend | The long tail dominates. Fixing thrashing runs is the single highest-value action. | smoking gun |
| Input vs output tokens | 94% input / 6% output by count | Even at a 5x price ratio, input dominates the bill. Context is the lever, not verbosity. | smoking gun |
| Retry-attributable tokens | 11% of all input tokens | Schema failures resending full context. Fixable at the tool contract (Tool Schemas). | smoking gun |
| Spend by task type | Support triage 62% | One task type dominates; a router could send its simple cases to a smaller model. | suspect |
| Cost per successful task | $0.34 (vs $0.21 last quarter) | The metric that matters — spend per outcome, not spend per run (Cost per Request: The Other Performance Metric). | smoking gun |
Latency, cost and quality: no free corner
The three properties trade against each other, and every lever moves at least two. A smaller model is cheaper and faster and usually less accurate. More retrieved context often improves accuracy while raising both cost and TTFT. More reasoning steps can improve quality on hard problems and multiply cost quadratically. Aggressive context trimming cuts cost and latency together and risks removing what the model needed (Context Selection & Compression).
What makes this tractable is that the trade is *measurable*. An eval suite turns "will a smaller model be good enough?" from an opinion into an experiment: run the golden dataset against both models, compare accuracy, compare cost, and decide (Golden Datasets). Without evals, cost optimization in agent systems is guesswork with a quality risk nobody can quantify, which is why teams either overspend indefinitely or cut costs and discover the quality regression from customer complaints.
One structural approach worth knowing: tier the work. Route by difficulty, so that easy requests take a cheap fast path and hard ones take the expensive careful one. This beats picking a single point on the trade-off curve, because most workloads are heavily skewed toward easy requests — and it means the expensive path only runs when it is earning its cost (Fallbacks, Caching and Model Routing).
| Lever | Cost | Latency | Quality | Measure it with |
|---|---|---|---|---|
| Reduce step count | Large reduction (super-linear) | Large reduction | Neutral to slightly worse | Step histogram + eval accuracy |
| Trim / compress context | Large reduction | Reduces TTFT | Risk of loss | Eval accuracy before/after (Evaluating Agents: Testing Probabilistic Systems) |
| Smaller model | Large reduction | Reduction | Usually worse | Head-to-head eval on golden dataset |
| Route by difficulty | Large reduction on the common case | Reduction on the common case | Neutral if routing is accurate | Routing accuracy + per-tier eval |
| Fix retry-causing schemas | Reduction | Reduction | Improves (fewer failures) | Retry rate by cause |
| Prompt caching (where supported) | Provider-dependent reduction | Reduces TTFT | Neutral | Cache hit rate; verify billing behavior |
| More retrieved context | Increase | Increases TTFT | Often improves | Retrieval relevance + eval (RAG Evaluation) |
| More reasoning steps | Large increase (super-linear) | Large increase | Improves on hard tasks only | Eval accuracy by task difficulty |
Key points
- Input tokens usually dominate the bill by count even when output tokens cost more per token — context is the lever, not verbosity.
- Context accumulates across steps, so total input tokens grow roughly with the square of step count; twice the steps can cost four times as much.
- A small fraction of high-step-count runs typically consumes a large fraction of spend, and they are the same runs that dominate the latency tail.
- Retries resend the entire accumulated context, so schema and validation failures are a direct and fixable line item.
- Cost, latency and quality trade against each other, and an eval suite is what converts that trade from an opinion into an experiment.
Follow the diagnosis
The causal chain, hop by hop — and the readings that invite the wrong conclusion.
- 1Step → context: each tool result is appended to the conversation, permanently increasing the input size of every subsequent call.
- 2Context → per-step cost: step N pays for the base prompt plus all N−1 accumulated results, so per-step cost rises monotonically through the run.
- 3Step count → total: because per-step cost rises, total input tokens grow super-linearly in step count rather than proportionally.
- 4Exploration → step count: ambiguous requests and unclear tool affordances cause exploration, which is exactly what drives step count up.
- 5Long tail → budget: a few percent of runs at high step counts consume a disproportionate share of spend, invisible in an average-cost-per-run metric.
- • "Output tokens cost more, so we should shorten responses" — output is usually a small fraction of the token count; check the input/output split before optimizing verbosity.
- • "Average cost per run is $0.08, that is fine" — the average hides a tail where a few percent of runs cost dollars each (The Average Was Fine and Users Were Not).
- • "A bigger context window lets us include more" — every included token is billed on every call that carries it, so the window raises the ceiling and not the affordability.
- • "Switching to a cheaper model will cut costs proportionally" — only if quality holds; without an eval comparison this trades an unmeasured amount of accuracy for a measured amount of money.
- • "Retries are cheap" — a retry resends the full accumulated context, so late-run retries are among the most expensive events in the system.
Measure, fix, validate
An optimization is not finished until the metric that motivated it has moved.
- • Cumulative input and output tokens per run, attributed per step, so context growth is visible as a curve rather than a total.
- • Spend by task type and by step-count bucket — the distribution identifies where to act far better than the total does.
- • Retry-attributable tokens as a separate line, split by retry cause (schema failure, tool error, timeout).
- • Cost per *successful* task rather than cost per run, so failed and thrashing runs are counted as the waste they are ([[cost-per-request]]).
- • Quality metrics from an eval suite alongside every cost figure, so a cost reduction that degrades accuracy is immediately visible ([[eval-metrics]]).
- • Attack step count first: it reduces cost super-linearly and latency linearly at the same time.
- • Trim context between steps — drop tool results that are no longer relevant, summarise earlier turns, retrieve less and rerank better ([[reranking]]).
- • Fix retry causes at the source with clearer tool schemas and structured-output constraints, rather than accepting retries as a cost of doing business.
- • Route by difficulty so the expensive path runs only when it earns its cost, and measure routing accuracy as its own metric.
- • Set per-run token and cost budgets with a graceful degraded answer at the ceiling, so no single run can consume an unbounded amount ([[budgets-limits-termination]]).
- • Cost per successful task falls — the metric that accounts for failed and thrashing runs, not cost per run.
- • The token-per-run distribution's right tail shortens, which is where the spend concentrated.
- • Eval accuracy on the golden dataset is unchanged or improved after the change; a cost win with an accuracy loss is a trade that needs explicit approval ([[golden-datasets]]).
- • Latency improves alongside cost — in agent systems these move together, so a cost reduction with no latency change suggests the mechanism was not what you thought.
- • Context trimming reduces cost and latency and risks removing information the model needed — only evals can quantify the accuracy cost.
- • Smaller models and difficulty routing save substantially and introduce a quality cliff on the requests the router misclassifies.
- • Token budgets guarantee bounded spend and produce incomplete answers at the ceiling; the degraded response has to be designed rather than defaulted.
- • Building and maintaining an eval suite is real ongoing work, and it is the prerequisite for every cost decision here being anything other than a guess.
- • Alert on p95 tokens per run and on the rate of runs hitting the token ceiling.
- • Add token count and cost assertions to the eval suite so a prompt or retrieval change that inflates context fails a gate (Regression Gates and Online Evaluation).
- • Track cost per successful task as a continuously reported metric, reviewed with the same seriousness as latency.
- • Re-verify provider pricing and caching behavior periodically — both change, and a pricing change silently invalidates every cost model you built.
Accuracy
Performance numbers are conditional. These are the conditions.
- ILLUSTRATIVEAll prices, token counts, spend figures and percentages are invented placeholders. Provider pricing changes frequently and varies by model, region and commitment tier — substitute current numbers before using this model for anything real.
- WORKLOAD-SPECIFICThe input/output token ratio and the step-count distribution depend entirely on your task mix and prompt design. Measure both rather than assuming this example's shape.