Guardrails & Security

An agent that reads untrusted text and can call tools is a confused deputy by construction; security is layered checks in code around the model, not a paragraph in the prompt.

Direct prompt injection is a user supplying text that overrides the developer’s instructions; it cannot be fully prevented by prompting because instructions and data share one channel.

Indirect injection is when content the agent reads (a web page, document, email or tool result) carries instructions; it is the dominant real-world attack against tool-using agents.

The damage from a compromised or confused agent flows through its tools: destructive writes, data leaving via URLs, emails and files, and runaway call volumes.

An agent should act as the user, with the user’s scoped credentials, and every tool must check authorisation in code; the prompt is not an access control list.

Guardrails are cheap checks in code before and after the model: classifiers, allow and deny lists, schema validation, PII detection; they reduce risk but cannot enforce policy on their own.

Secrets must never enter the model’s context, traces or logs, and every byte returned by a tool must be treated as untrusted data and labelled before it re-enters the loop.