Concurrency in Agent Systems

Two Agents, One Document

A summarizer and a fact-checker both read version 4 of a document, both edit it, both write. One edit survives. This is the lost update, unchanged since the first bank-balance example — except the writers are nondeterministic, the schedule is chosen by a model, and the losing edit is plausible enough that nobody notices it went missing.

The question this answers

The question

Two agent tasks are editing the same document. What stops one of them from silently erasing the other?

The work

Two agent tasks operating on document doc-77: task A summarizes the introduction, task B corrects factual errors in the body. Both read the whole document, both produce a full replacement, both write it back.

What is shared

The document — its content and its version number — stored in a database that both agents reach through a tool.

The invariant — what must stay true under every interleaving

Every accepted edit is applied to the version its author actually read, and no accepted edit is silently discarded.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

WorkCan it overlap?Can it parallelise?What is shared?What ordering?What synchronization?Where is contention?What can deadlock?What can race?What is gained?What complexity?

The oldest bug, with a new set of writers

Nothing about this is novel. Read-modify-write from two actors with no coordination loses one of the writes, and that is Shared Mutable State and the lost update from the first week of this domain. What is different is the writers.

First, the payload is a *whole document replacement* rather than a field update, so the lost write is total rather than partial. Second, the actors are nondeterministic: rerunning the pair produces a different interleaving and possibly different content, so the failure is not reproducible. Third, and most importantly, the surviving document is *plausible*. A lost bank transaction produces a balance that does not reconcile; a lost paragraph edit produces a document that reads perfectly well and is simply missing a correction someone asked for. There is no alarm because nothing looks wrong.

That last property is why this needs explicit handling rather than vigilance. A silent failure that produces valid-looking output will not be caught by review, by tests, or by the agent itself — which will report success, because from its perspective the write succeeded.

Two agent tasks, one document. Both tools return success.ILLUSTRATIVE
Invariant · Every accepted edit is applied to the version its author read; no accepted edit is discarded.
#Task A — summarizerTask B — fact-checkerDocument storeState
1readDocument("doc-77") -> v4, 2,400 words··version=4 A read=v4 B read=-
2·readDocument("doc-77") -> v4, 2,400 words·version=4 A read=v4 B read=v4
3model rewrites the introduction (18s)··version=4
4·model corrects three factual errors in the body (26s)·version=4
5writeDocument("doc-77", fullText) -> ok··version=5 content=v4 + new intro
6reports success to the user··version=5
7·writeDocument("doc-77", fullText) -> ok·version=6 content=v4 + fact corrections
✕ No accepted edit is discarded. B wrote a full document derived from v4, so A's new introduction is gone. Both tools returned success.
8··stores v6; no conflict detected because none was checked forversion=6 A edit=LOST B edit=present
9·reports success to the user·version=6
One document, two successful writes, one silently discarded edit, and no error anywhere. The interleaving is chosen by whichever model call finished first, so a rerun may lose B's edit instead. The fix is a version check on write — the store must reject a write whose base version is no longer current. See Optimistic Concurrency Control.

Optimistic concurrency: read a version, write against it

The mechanism is small: every read returns the current version; every write states the version it was based on; the store applies the write only if that version is still current, and rejects it otherwise. The rejected caller re-reads and retries against the new version. This is Optimistic Concurrency Control exactly, and on the HTTP side it is the conditional-request contract — Optimistic Concurrency: Versions and If-Match and Conditional Requests: ETags, 304 and 412 in API Design.

It is called optimistic because it assumes conflicts are rare and pays only on the rare path. That assumption is usually correct for documents: two agents editing the same document in the same twenty-second window is unusual. When it stops being rare — a popular shared document, or a fan-out of ten agents onto one artefact — retries start to dominate and pessimistic locking becomes cheaper, which is the threshold in Optimistic vs Pessimistic.

The critical detail for agents specifically is what the retry *does*. A naive retry re-runs the model call, which costs money and time and may produce different output. A better retry re-reads the current version and re-applies the same *edit* rather than the same *text* — which is only possible if the agent emitted a patch or a structured operation rather than a full replacement. That is the single highest-leverage design change here: make the tool accept an edit, not a document.

1// 1. UNSAFE -- the schedule above. Last writer wins, silently.
2async function writeDocument(id: string, text: string) {
3 await db.none('UPDATE docs SET content = $1 WHERE id = $2', [text, id])
4 return 'ok'
5}
6
7// 2. SAFE, EXPENSIVE -- version check, but retry re-runs the model.
8async function writeDocumentChecked(id: string, text: string, baseVersion: number) {
9 const n = await db.result(
10 'UPDATE docs SET content = $1, version = version + 1 ' +
11 'WHERE id = $2 AND version = $3', // <-- the whole fix
12 [text, id, baseVersion],
13 )
14 if (n.rowCount === 0) return { status: 'CONFLICT', current: await readVersion(id) }
15 return { status: 'ok' }
16}
17// On CONFLICT the agent must re-read and regenerate: another model call,
18// another 26 seconds, another bill, and possibly different content.
19
20// 3. SAFE AND CHEAP -- the tool takes an EDIT, not a document.
21type Edit =
22 | { op: 'replace_section'; heading: string; text: string }
23 | { op: 'replace_span'; find: string; text: string }
24
25async function applyEdits(id: string, edits: Edit[], baseVersion: number) {
26 for (let attempt = 0; attempt < 3; attempt++) {
27 const doc = await readDocument(id)
28 if (attempt === 0 && doc.version !== baseVersion) { /* fall through and rebase */ }
29
30 const next = edits.reduce(applyEdit, doc.content) // deterministic, no model call
31 const n = await db.result(
32 'UPDATE docs SET content = $1, version = version + 1 WHERE id = $2 AND version = $3',
33 [next, id, doc.version],
34 )
35 if (n.rowCount === 1) return { status: 'ok', version: doc.version + 1 }
36 // conflict: loop, re-read, re-apply the SAME edits to the NEW base.
37 // No model call. No new cost. The other agent's edit survives.
38 }
39 return { status: 'CONFLICT_EXHAUSTED' } // escalate to a human, do not overwrite
40}
41
42// The design decision that matters is in the SIGNATURE, not the retry loop:
43// a tool that takes a full document cannot merge. A tool that takes edits can.
Three versions of the same tool. Only the third is safe and cheap.

Choosing a strategy, and what each one costs

Optimistic control is the default and the right one for most agent work. But it is not the only option and the alternatives are genuinely better in specific cases. Partitioning — give each agent a disjoint section, and there is no conflict to resolve — is the strongest answer when the work divides cleanly, because it eliminates the problem rather than detecting it. Serialization through a single-writer queue per document turns concurrent edits into ordered ones and is simple to reason about, at the cost of throughput on hot documents — Message Passing and The Actor Model.

Append-only models sidestep the whole question: each agent writes a new suggestion or annotation rather than mutating the document, and a separate step merges. This is often the correct design for agent output regardless of concurrency, because it preserves provenance and makes human review possible — and it makes the concurrency question disappear as a side effect.

What does not work: a process-local lock in the agent runtime (see A Mutex on Server A Does Nothing About Server B), telling the model in the prompt not to conflict, or retrying without a version check. The last one is the most dangerous because it looks like error handling and provides no protection at all.

StrategyHow conflicts are handledCostBest when
Nothing (last writer wins)They are not. One edit is silently discarded.Zero, until it costs a customer's correctionNever, for anything a human asked for
Optimistic: version check + retryRejected on stale base; caller re-reads and retriesA retry path; possibly a re-run of the model callConflicts are rare — the usual case
Optimistic with structured editsRejected on stale base; edits re-applied to the new base with no model callTool must accept edits rather than documentsConflicts are rare and re-running the model is expensive
Pessimistic: lock the documentThe second agent waits or is refusedSerializes work on hot documents; needs lease + fencing across processesConflicts are frequent, or edits are very expensive to redo
Partition by sectionNo conflict existsRequires the work to divide cleanly, which it often does notIndependent sections with a clear owner each
Append-only suggestionsNo conflict exists; a merge step resolves laterA merge step, and someone to run itHuman review is wanted anyway — usually the best agent design
Five strategies for concurrent agent edits, with their real costs.

Key points

  • This is the classic lost update, with two differences: the payload is a whole-document replacement, and the losing edit produces output that looks entirely correct.
  • Both tool calls return success. No error is raised anywhere, and the agent truthfully reports that its write applied.
  • The fix is a version check on write: the store rejects any write whose base version is no longer current.
  • The design decision that matters is the tool signature. A tool that takes a full document cannot merge; a tool that takes structured edits can re-apply them to a new base with no model call.
  • Naive optimistic retry re-runs the model — expensive, slow, and nondeterministic. Structured edits make the retry free.
  • Partitioning and append-only suggestions eliminate the conflict rather than detecting it, and are frequently the better agent design for independent reasons.
  • Retrying without a version check looks like error handling and provides no protection whatsoever.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • Every read returns the document plus a version identifier — a counter, a hash, or an entity tag.
  • Every write carries the version it was based on, and the store applies the update only where the stored version still matches.
  • A non-matching version means someone else wrote in between; the store rejects the write and returns the current version rather than overwriting.
  • The caller re-reads and either re-applies structured edits deterministically, or re-runs the generation against the new base.
  • Retries are bounded; on exhaustion the operation escalates to a human rather than forcing the write, because forcing is the original bug with extra steps.
Interleavings that matter
  • A reads v4, B reads v4, A writes (v5), B writes (v6) — B's document was derived from v4, so A's introduction is gone and both tools reported success.
  • With a version check: A writes with base=4 and succeeds (v5); B writes with base=4, the store finds version 5, and B is rejected. B re-reads v5 and re-applies its corrections. Both edits survive.
  • With a version check but a whole-document tool: B's rejection forces a full model re-run at 26 seconds and full cost, and the regenerated text may differ from what B produced the first time.
  • Three agents fan out onto one document: A succeeds, B and C are both rejected, both retry, one succeeds and one is rejected again. Retry amplification under contention — the point at which pessimistic locking becomes cheaper.
  • Retry without a version check: B's write fails on a transient network error, B retries and succeeds, and A's edit is lost exactly as before. The retry logic protected nothing.
What it guarantees — and does not
  • A version check guarantees no write is applied to a base other than the one its author read. That is the whole guarantee, and it is exactly what prevents the lost update.
  • It does not guarantee that both edits end up in the document. It guarantees the conflict is *detected*; merging is a separate decision.
  • Structured edits guarantee a cheap retry only when the edits still apply to the new base — a section that no longer exists is a real conflict a human must resolve.
  • Bounded retries guarantee termination, not success. Exhaustion must escalate rather than force.
  • Nothing here guarantees the model produces a sensible edit. This mechanism protects against concurrency, not against bad output.
Where contention appears
  • Hot documents concentrate conflicts: the more agents fan out onto one artefact, the more retries, and retry cost grows with contention — What Contention Actually Costs.
  • Each retry that re-runs a model call consumes tokens and provider concurrency, so contention converts directly into spend — Token Budgets.
  • A pessimistic lock on a document serializes all agent work on it, which is correct and turns throughput into a queue.
  • Retry storms are possible when many agents conflict and all retry immediately; jittered backoff applies here as everywhere — Retry Storms: The Load You Generated Yourself.
How it fails
  • Lost update: an edit silently discarded, with a plausible-looking document and two success reports.
  • Retry amplification: high contention turning every write into several model calls and several times the cost.
  • Forced overwrite on retry exhaustion, which converts a detected conflict back into a silent loss.
  • Stale edits that no longer apply after rebasing — a structural conflict that automatic merging cannot resolve and must escalate.
  • False safety from process-local locking in a multi-instance agent runtime — A Mutex on Server A Does Nothing About Server B.
  • Conflict detection with no user-visible surface, so conflicts are counted and never acted on.
When it helps
  • Any multi-agent or multi-task design where more than one actor can write the same artefact — which includes almost every non-trivial agent system.
  • Human-plus-agent editing, where a person may be editing the same document the agent is rewriting.
  • Retries in general: a version check makes a retry safe, which is what allows aggressive retry policies elsewhere.
When it hurts
  • On very high contention, where retries dominate and a lock or a queue would be cheaper.
  • When conflicts are structural rather than positional, where automatic rebasing produces a merged document that satisfies neither agent's intent.
  • When it becomes the only mechanism and the deeper design question — should two agents be editing one document at all? — goes unasked.
How you would know
  • Conflict rate: rejected writes as a fraction of attempted writes. Rising past a few percent is the signal to consider partitioning or locking.
  • Retry count distribution per write, and how often retries are exhausted.
  • Cost attributable to retries specifically, since that is where optimistic control becomes expensive.
  • Whether edits are being lost: sample documents and check that every requested change is present. This is the only measurement that catches the unguarded case.
  • Time between read and write per agent task — the wider that window, the higher the conflict probability.
Complexity it introduces
  • Version identifiers must be threaded through every read and write path, including tool schemas and model-visible arguments.
  • A conflict is now a case the agent must handle, which means the prompt and the tool contract both need to express it.
  • Structured edits are a substantially richer tool interface than "write this text", and the model has to produce them reliably.
  • Escalation to a human needs a surface: a queue, a notification, a diff view. Detecting conflicts and dropping them is not better than not detecting them.
Simpler alternatives
  • Partition the work so each agent owns a disjoint section — eliminates the conflict rather than detecting it.
  • A single-writer queue per document, converting concurrent edits into ordered ones — Message Passing, The Actor Model.
  • Append-only suggestions with a separate merge step, which preserves provenance and enables human review — usually the better agent design regardless.
  • Pessimistic locking with a lease when contention is high and re-running the model is expensive — with the caveats in What Changes When the Shared State Is on Another Machine.
  • Not running two agents on one document. The simplest fix, and the one that gets skipped.

What people believe, and what is true

Claim

The tool returned success, so the edit was applied.

Reality

Both writes succeeded. Success means "my write landed", not "my write survived". Without a version check, the second write erases the first and both are truthful.

Claim

We retry on failure, so we are safe.

Reality

Retrying without a version check protects against transient errors and nothing else. The lost update is not an error, so there is nothing to retry.

Claim

The agents are told not to conflict.

Reality

A prompt is not a synchronization primitive. Exclusion is enforced by the store rejecting a stale write, or it is not enforced.

Apply it