The question this answers
Given a set of tool calls an agent wants to make, which of them are allowed to overlap?
One agent turn that wants to call: search(query), db.getCustomer(id), db.getOrders(id), api.createRefund(orderId), and email.send(customer).
The agent's context (every result is appended to it), the database rows the tools read and write, the third-party rate-limit budget, and the spend budget for the run.
No tool call executes before the input it depends on exists, no write executes twice for one logical intent, and the run stays within its rate and spend budgets.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Four filters, applied in order
The question "can these run concurrently?" decomposes into four independent checks, and a call must pass all of them. Data dependency: does this call need the output of another? createRefund(orderId) cannot start before getOrders returns the order id. That is a dependency graph, and it is the same reasoning as Dependency Graphs and Work and Span — the span of the graph, not the number of tools, bounds how fast a turn can be.
Side effects: does this call change the world? Two reads overlapping is free. A read and a write overlapping may return a value that is immediately stale. Two writes overlapping is where money moves twice — see Two Agents, One Document. Reads and writes are not symmetric and the distinction belongs in the tool schema, not in the model's judgement.
Shared limits: do these calls draw on the same budget? Five parallel calls to one API is five times the instantaneous rate against a quota that is usually per-key, not per-call. Ordering requirements: does the *business* require an order even without a data dependency? Charging before shipping has no data dependency and a very strong ordering requirement. That last one is invisible to any automatic analysis and has to be declared.
| Filter | Question | How you know | If violated |
|---|---|---|---|
| Data dependency | Does this call consume another call's output? | Argument references a prior result — visible in the plan | The call runs with a missing or hallucinated argument |
| Side effects | Does this call write, charge, send or delete? | Declared in the tool schema as read-only or mutating | Two writes for one intent; a read that is stale on arrival |
| Shared limits | Do these calls draw on the same quota, pool or key? | Group tools by the backend they hit, not by tool name | 429s, throttling, or a connection pool exhausted by one turn |
| Ordering requirement | Does the business require A before B, absent any data flow? | Only from a human. No analysis can infer it. | Shipped before charged; emailed before approved |
| Cost | Does running all of these speculatively cost more than the latency saved? | Per-call price times fan-out against the latency budget | The run is faster and three times more expensive |
The dependency graph decides the shape
Once the filters are applied, what remains is an ordinary dependency graph, and the scheduling reasoning is identical to any other parallel decomposition: independent nodes may run together, the critical path sets the floor, and adding parallelism beyond the graph's width buys nothing — Fork/Join, Parallelism Moves the Load Downstream.
For the example turn, search, getCustomer and getOrders are mutually independent reads and can overlap. createRefund depends on getOrders and is a write. email.send depends on createRefund succeeding and is also a write with an irreversible effect. So the shape is: a three-wide read fan-out, then a serialized write chain of length two. The turn cannot be faster than one read plus one refund plus one email, no matter how many tools exist.
This is worth drawing because it makes an important point visible: agent latency is usually dominated by a few sequential steps, and parallelising the reads helps exactly once. The remaining time is the write chain and the model calls between steps — the latter of which is often the largest single term. See Where an Agent Run Actually Spends Its Time and Inside One Model Call: Queue, First Token, Generation before optimizing tool parallelism.
What the fan-out costs downstream
A single agent that fans out three calls is unremarkable. The same agent running for a hundred users, each fanning out three calls, is three hundred concurrent requests at a single instant, arriving as a burst rather than as a rate. The downstream sees the burst, not your intent — and this is Fan-Out / Fan-In: One Request Becomes N and Thundering Herd with a model deciding the fan-out width.
The specific hazard is that the width is *model-determined*. A prompt change, a different question, or a longer document can turn a three-way fan-out into a fifty-way one with no code change and no deploy. Unbounded concurrency chosen by a non-deterministic component is exactly the shape Unbounded Concurrency warns about, and the answer is the same: an explicit semaphore bounding concurrent tool calls per run and across runs — Bounding Concurrency and Semaphores: Counting Permits as a Resource Limit.
The timeline below shows one turn against a rate-limited API. Note that the three parallel reads finish faster than sequential, and the refund still dominates — and that at ten concurrent runs the same pattern produces 429s that a single-run test would never surface.
Key points
- Four independent filters decide whether tool calls may overlap: data dependency, side effects, shared limits, and business ordering. A call must pass all four.
- Business ordering — charge before ship — has no data dependency and cannot be inferred by any analysis. It must be declared.
- Read-only versus mutating belongs in the tool schema, not in the model's judgement, because that is the filter that prevents duplicate writes.
- What remains after filtering is an ordinary dependency graph: the critical path sets the floor, and parallelism beyond the graph's width buys nothing.
- Fan-out width is model-determined, so a prompt change can turn three concurrent calls into fifty with no code change — bound it explicitly.
- Model call latency is frequently the largest term in a turn; parallelising tools optimizes the smaller half.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • The model proposes a set of tool calls; the runtime does not execute them as proposed but first builds a dependency graph from argument references.
- • Each tool's schema declares whether it is read-only or mutating, and which backend or quota group it belongs to.
- • Calls with no unmet dependency, no mutation conflict and available quota are dispatched concurrently, under a semaphore bounding total in-flight calls.
- • Results are appended to context as they arrive; the runtime must decide whether context order is completion order or a canonical order, because the model will condition on it.
- • Mutating calls are serialized against each other unless explicitly declared independent, and each carries an idempotency key so a retry is not a second effect.
- • search, getCustomer and getOrders dispatched together: three round trips overlap, the turn is ~12 ticks shorter, and nothing shared is written. The safe case.
- • createRefund dispatched in parallel with getOrders because the model guessed an order id: the refund executes against a hallucinated id and either fails or refunds the wrong order. The data-dependency filter exists to prevent exactly this.
- • Two tool calls both call
db.getOrdersfor the same customer through a pool of 5 while 40 runs are active: 80 concurrent queries against 5 connections, and every run's latency rises. The tools were independent; the pool was not — Connection Pool Saturation: Waiting in Front of an Idle Database. - • email.send dispatched concurrently with createRefund: the email arrives, the refund fails, and the customer has written confirmation of a refund that does not exist. No data dependency existed; the ordering requirement was never declared.
- • Dependency analysis guarantees a call does not start before its inputs exist — but only for dependencies expressible as argument references. Implicit dependencies are invisible to it.
- • Schema-declared read-only status guarantees the runtime will not serialize unnecessarily; it does not guarantee the tool is actually read-only, which is a review question.
- • A concurrency semaphore guarantees a bound on in-flight calls per run. It guarantees nothing across runs unless the bound is shared — which is A Mutex on Server A Does Nothing About Server B again.
- • Nothing guarantees the model proposes a sensible set of calls. The filters constrain execution, not proposal, and that is the correct division.
- • Parallel execution guarantees no ordering between results, so anything that depends on order must impose it explicitly rather than relying on arrival.
- • The shared quota per API key is the usual first ceiling, and it is consumed by every concurrent run simultaneously.
- • Database connection pools are shared across all runs, so per-run fan-out multiplies into pool exhaustion at modest run counts.
- • The model provider itself has rate limits, and parallel sub-agent calls contend for the same token budget — Token Budgets.
- • The agent's context is a shared accumulator: concurrent results all append to it, and its size affects the cost and latency of every subsequent call.
- • Executing a dependent call with a hallucinated argument because the dependency was not detected.
- • Duplicate writes from concurrent mutating calls for one logical intent — Two Agents, One Document.
- • Rate-limit rejection from a fan-out whose width the model chose, appearing only under concurrent runs.
- • Ordering violation on effects with no data dependency: emailed before charged, shipped before paid.
- • Context ordering nondeterminism, where results appended in completion order make the run non-reproducible — Nondeterminism: Same Input, Different Output.
- • Cost blowout from speculative parallel calls whose results are discarded when an earlier one determines the path.
- • Turns dominated by several independent reads, where parallelising converts N round trips into one and is a clear, safe win.
- • Research and retrieval patterns, where breadth is the point and every call is read-only — the ideal case for concurrency here.
- • Multi-agent designs where sub-agents work on disjoint inputs, which is the same independence argument one level up — Multi-Agent Systems Overview.
- • Any turn dominated by model latency, where tool parallelism optimizes the smaller term and adds failure modes for a small gain.
- • Write-heavy turns, where serialization is required anyway and concurrency adds only risk.
- • Rate-limited or expensive tools, where three parallel calls cost three times as much and finish only slightly sooner.
- • When correctness depends on an ordering nobody wrote down, which is common and is discovered in production.
- • Critical-path length of the turn against total tool time — if they are close, there was little parallelism to extract.
- • Concurrent in-flight tool calls per run and across runs, which is the number that actually hits downstream limits.
- • Rate-limit rejection counts attributed to agent-initiated calls specifically.
- • Fan-out width distribution over time, which reveals when a prompt change altered it.
- • Cost per turn split by tool calls versus model calls, so optimization effort goes where the money is — What One Agent Run Costs, and Which Term Dominates.
- • A dependency graph builder and a scheduler in the agent runtime, plus schema fields for effect class and quota group.
- • A declared ordering mechanism for business rules that no analysis can infer, and a process for keeping it accurate.
- • Concurrency bounds at two levels — per run and across runs — with the second requiring shared state.
- • Determinism becomes harder: parallel completion order varies, and reproducing a run requires canonicalizing it.
- • Run tools sequentially. Simple, deterministic, and correct — and for turns dominated by model latency, barely slower. See Parallel vs Sequential Tool Calls.
- • Batch at the tool level: one
getCustomerBundle(id)that returns customer and orders together, removing the concurrency question entirely. - • A fixed workflow rather than model-chosen tool calls, where the dependency graph is written by a human and the model fills in arguments — Workflow State Graph.
- • Cache read results across turns so repeated reads do not become repeated calls, which reduces fan-out without any scheduling.
What people believe, and what is true
The model decided to call these three tools, so they are independent.
The model proposed them. Independence is a property of data flow, effects, quotas and business rules, and the runtime must check it — the proposal is not evidence.
Parallel tool calls make agents faster.
They shorten the tool portion of a turn. When model latency dominates, the end-to-end gain is small and the added failure modes are not.
Read and write calls can be treated the same for scheduling.
Overlapping reads is free. Overlapping writes is where duplicates and lost updates come from, and the asymmetry has to be encoded in the tool schema.