Toolsidempotency keysat-least-onceside effectsdeduplicationexactly-once

Idempotency

In an at-least-once world, a tool with side effects must be safe to call twice with the same arguments — idempotency keys make retries and re-runs harmless.

Interview question
Progress

Why agents execute at least once

Between the model deciding to call send_email and the email leaving the server there are many places to fail: the HTTP request times out after the mail provider accepted it; the dispatcher crashes after execution but before recording the result; the retry logic from Tool Errors, Retries and Timeouts fires on an ambiguous error; the model itself, not seeing a result, calls the tool again "to be sure". Agent runs are also replayed — for debugging, after a deploy, from a checkpoint in a Workflow State Graph.

Every one of these gives at-least-once execution. Exactly-once is not achievable across a network in general; what you can achieve is "executed at least once, with effects applied exactly once". That property is idempotency, and it has to be designed into each tool.

An operation is idempotent if performing it N times has the same effect as performing it once. GET /order/42 is naturally idempotent. POST /charge is not — unless you make it so.

Idempotency keys

The standard mechanism: the caller attaches a unique idempotency key to the request, and the server stores key → result the first time it sees it. Any later request with the same key returns the stored result without re-executing. Stripe, AWS and most payment and messaging APIs support this natively; for your own tools you implement it with a small table or a cache with TTL.

The key must be stable across the retries you want to dedupe and different across the calls you want to distinguish. In an agent, the natural choice is the tool call id the provider assigns (each call in a response has one), or a hash of (run_id, step, tool_name, canonical_args). Do not derive it only from arguments — the user may legitimately want to send the same email twice tomorrow.

Idempotent wrapper around a side-effecting tool, keyed on the tool call id.
1import json, hashlib
2 
3class IdempotentStore:
4    def __init__(self):
5        self._done: dict[str, dict] = {}   # in prod: Redis / DB with TTL
6 
7    def get(self, key):  return self._done.get(key)
8    def put(self, key, result): self._done[key] = result
9 
10store = IdempotentStore()
11 
12def idempotent(tool_name: str):
13    def wrap(fn):
14        def run(args: dict, ctx) -> dict:
15            key = ctx.tool_call_id or hashlib.sha256(
16                f"{ctx.run_id}:{ctx.step}:{tool_name}:{json.dumps(args, sort_keys=True)}".encode()
17            ).hexdigest()
18            if (cached := store.get(key)) is not None:
19                return {**cached, "deduplicated": True}
20            result = fn(args, ctx)          # executes exactly once per key
21            store.put(key, result)
22            return result
23        return run
24    return wrap
25 
26@idempotent("send_email")
27def send_email(args, ctx):
28    msg_id = mail.send(to=args["to"], subject=args["subject"], body=args["body"],
29                       idempotency_key=ctx.tool_call_id)  # pass it downstream too
30    return {"message_id": msg_id}

Worked examples: send email, create ticket

Send email. Without a key, a timeout on the provider's side plus one retry sends two emails. With the key passed to the provider, the retry returns the original message_id. If the provider has no key support, dedupe locally: store key → message_id before the send attempt as "in flight", and on retry check the provider's sent log for the same Message-ID header you generated.

Create ticket. The naïve version creates duplicates on every retry and every replay. Two designs fix it: (a) an idempotency key on the create endpoint; (b) a natural key — (customer_id, normalized_subject, day) — checked with "find-or-create" semantics. Option (b) also protects against the model calling create_ticket twice in the same run with slightly different wording.

Reads are free. Updates should be expressed as absolute state (set_status(ticket, "closed")) rather than deltas (increment_priority(ticket)); the former is idempotent by construction, the latter is not.

  • Prefer PUT-style "set to X" over POST-style "add one".
  • Return the same response for a deduplicated call as for the original, plus a deduplicated flag for tracing.
  • Expire keys after a window (24 h is common) so storage stays bounded.
  • For multi-step side effects, store progress under the key so a replay resumes rather than restarts.

Side-effect safety beyond dedupe

Idempotency handles the "same call twice" case. It does not handle the "wrong call once" case — that is Argument Validation and Tool Permissions and Least Privilege. Nor does it handle the model issuing two different destructive calls that together are harmful. Layer the controls: validate, authorize, make idempotent, and for irreversible actions gate behind confirmation (Approval Gates and Risk Classes).

A useful classification for every tool in your registry: read (always safe to retry), idempotent write (safe to retry with key), non-idempotent write (retry at most once, never replay), irreversible (approval required). Tag each tool with its class and let the dispatcher's retry policy read the tag — the policy then cannot be forgotten on a new tool.

Key points

  • Network failures, retries, replays and model repetition all yield at-least-once execution.
  • Idempotency means N executions have the effect of one; design it into every write tool.
  • Use an idempotency key stable across retries (tool call id or run/step/args hash) and pass it downstream.
  • Express updates as absolute state, not deltas; use find-or-create for entity creation.
  • Tag tools read / idempotent-write / non-idempotent-write / irreversible and drive retry policy from the tag.
  • Idempotency prevents duplicate effects, not wrong effects — layer validation and permissions on top.

When to use — and when not to

Use it when
  • Any tool that sends, creates, charges, posts or deletes.
  • Agents with retry logic, checkpoint replay, or human-approved re-runs.
  • Workflows where a step may be re-executed after a crash.
Avoid it when
  • Pure reads — adding keys to get_weather is wasted complexity.
  • As a replacement for confirmation on irreversible actions; a deduplicated delete_account is still a deletion.
  • Keying purely on arguments when the same arguments can legitimately recur.

Failure modes

  • Retry after an ambiguous timeout sends the customer two invoices.
  • Checkpoint replay re-creates 40 tickets that already exist.
  • Key derived from arguments only; a legitimate second reminder email is silently dropped as a duplicate.
  • Key stored locally but not passed to the provider; local dedupe misses the crash between send and store.
  • Delta-style update (add_credit(10)) retried three times grants 30.

Tradeoffs

Complexity
low → high
Latency
low → high
Cost
low → high
Reliability
poor → strong
Debuggability
hard → easy

One lookup per call; the reliability gain is disproportionate to the code.

Don't delegate understanding
The manifesto →
← Tool Errors, Retries and TimeoutsParallel vs Sequential Tool Calls →