AI & Agent Security
Prompt injection, untrusted tool output, over-privileged tools, poisoned retrieval and memory — and the one rule: the model is never the authorization layer.
Every lesson below identifies the asset, attacker capability and boundary before naming the vulnerability. Controls are split into prevention, detection and recovery; residual risk is explicit.
Agents combine untrusted language, retrieved data, memory, models and tools; capability boundaries—not model obedience—control the outcome.
User, model, retrieval, memory, tools and external content have different trust and privilege; mark every flow explicitly.
A user or retrieved document supplies language that the model may confuse with authority; the robust defense is to constrain capabilities and decisions outside it.
A tool schema is a capability interface: make it narrow, bind it to a principal, enforce policy outside the model and record the result.
The agent proposes an action; deterministic code evaluates principal, action, resource and policy. Natural-language confidence is never permission.
Money transfer, deletion, external communication and permission changes should pause at an explicit risk gate with a comprehensible diff.
A website, API or integration can return text that is false or malicious; tool data must not become higher-priority authority simply because a tool fetched it.
Retrieval and memory add durable, searchable copies of data where poisoning, tenant-filter mistakes, retention and sensitive recall become security boundaries.
Code execution gets limited files, network, CPU, memory and scoped credentials; the sandbox must constrain the capability, not just the process tree.