Handling tool errors and retries
“A tool called by your agent fails intermittently. How do you handle errors, retries, and timeouts?”
What this tests
- Classifying errors as retryable vs not
- Knowledge of backoff, jitter, timeouts, and circuit breakers
- Understanding what the model should and should not see
- Idempotency awareness
Answers by level
Read the beginner answer first and notice what is missing.
First classify: transient errors (timeouts, 429, 503) get retried with exponential backoff and jitter, capped at a small number of attempts; permanent errors (400, not found, validation failure) are returned to the model immediately as a structured error so it can change its approach; auth or configuration errors abort the run and alert, because retrying cannot help. See Tool Errors, Retries and Timeouts.
Retries happen in code, not in the model loop, because a model retry costs a full LLM call and is unpredictable. Timeouts are per-tool and tight (a few seconds for a lookup), with the overall run bounded by a wall-clock budget. Retries are only safe for idempotent operations (see Idempotency); for anything that changes state, use an idempotency key so the server deduplicates.
The model sees a clean, structured outcome: success with data, or an error object with a type and a hint. It should not see stack traces, and it should not be asked to decide whether to retry a 503.
Green flags · Red flags
- Classifies errors into transient, permanent, and fatal
- Backoff with jitter, capped attempts, retries in code
- Per-tool timeouts plus an overall run budget
- Idempotency keys for state-changing calls
- Circuit breaker and fallback
- Structured error returned to the model, no stack traces
- Lets the model decide retries for transient errors
- Retries everything including 400s and non-idempotent writes
- Generous timeouts with no run budget
- No metrics on error rates