Concurrency in Agent Systems
Parallel tool execution and what it costs in load, money and determinism; agents racing on the same document; and cancelling a long-running agent run without leaving tool calls, queue entries and side effects behind.
An agent holding a search tool, a database tool and a payments API decides, each turn, what to invoke. Some of those calls are independent and some are not, and the model does not know which. Deciding what may overlap is a dependency question, a rate-limit question, a cost question and — for anything that writes — a safety question.
Q · Given a set of tool calls an agent wants to make, which of them are allowed to overlap?
Running three tool calls at once instead of in sequence turns three round trips into one. It also triples the instantaneous load on whatever they hit, opens the possibility of conflicting writes, raises the spend for calls whose results get discarded, and makes the run nondeterministic. All four costs are real; the latency win usually still wins for reads.
Q · What exactly am I buying and paying for when I let the agent run tool calls in parallel?
A summarizer and a fact-checker both read version 4 of a document, both edit it, both write. One edit survives. This is the lost update, unchanged since the first bank-balance example — except the writers are nondeterministic, the schedule is chosen by a model, and the losing edit is plausible enough that nobody notices it went missing.
Q · Two agent tasks are editing the same document. What stops one of them from silently erasing the other?
The user presses stop. The model call can be aborted, the queued tool calls can be dropped, the in-flight ones will finish whether you want them to or not — and the three emails already sent cannot be un-sent. Cancellation is a request to stop future work, never an undo of work already done.
Q · A user cancels a long agent run. What stops, what finishes anyway, and what is already irreversible?