Input and Output Guardrails
Guardrails are cheap checks in code before and after the model: classifiers, allow and deny lists, schema validation, PII detection; they reduce risk but cannot enforce policy on their own.
What a guardrail is
A guardrail is a deterministic or small-model check placed on the boundary of the LLM call: input guardrails decide whether a request should reach the model and in what form; output guardrails decide whether a model result may proceed to a side effect or a user. They run in ordinary code, are fast, and produce a binary or scored decision you can log and test.
The value of a guardrail is proportional to how cheaply it catches the common case. It is the equivalent of input validation on a web form: it stops the obvious, keeps the logs clean, and makes the rare sophisticated attack stand out. It is not the equivalent of authentication.
Input guardrails
Input checks run before the expensive call and often before any context is assembled. They are a good place to reject cheaply and to strip things the model should never see.
- Injection classifier: a small fine-tuned model or a hosted moderation endpoint scoring “is this an attempt to override instructions?”. Tune the threshold on your own traffic; false positives annoy users.
- Topic and scope filter: a support bot for invoices should refuse medical advice; a cheap embedding-similarity check against allowed topics works.
- PII detection and redaction: regex plus NER for emails, phone numbers, card numbers, national ids. Redact before the model sees them if the task does not need them.
- Length and format limits: cap input size; reject binary blobs and obviously encoded payloads.
- Allow and deny lists: known-bad phrases, blocked domains in URLs, banned file paths.
Output guardrails and validation before side effects
Output checks are where guardrails earn their keep, because they sit between the model and the world. The most important rule is ordering: validate before the side effect, not after. A check that runs on the final chat message is useless if the email was already sent three steps earlier.
Structured output makes this tractable. If a tool call is a JSON object, schema validation (Structured Outputs, Argument Validation) rejects malformed and out-of-range arguments deterministically. If the final answer must contain citations, a validator checks that each cited id exists in the retrieved set (Citations).
- Schema validation on every tool call argument object; reject and let the model retry with the error message.
- Destination allow-lists on recipients, URLs, file paths, before the tool runs.
- Secret and PII scan on the final answer and on any text about to leave the system.
- Grounding check: does the answer cite only retrieved content? Flag or refuse otherwise.
- Policy classifier on the final answer for the categories your product forbids.
1import { z } from 'zod'2 3const SendEmail = z.object({4 to: z.string().email().refine((a) => allowedDomains.has(a.split('@')[1]), 'recipient domain not allowed'),5 subject: z.string().max(200),6 body: z.string().max(20_000),7})8 9export async function guardedSend(rawArgs: unknown, session: Session) {10 const parsed = SendEmail.safeParse(rawArgs)11 if (!parsed.success) return { error: parsed.error.issues.map((i) => i.message).join('; ') }12 const leak = secretScanner.scan(parsed.data.body) // API keys, tokens, card numbers13 if (leak.found) return { error: 'body contains sensitive data: ' + leak.kinds.join(', ') }14 if (session.sawUntrustedContent) await approvals.require(session, 'send_email', parsed.data)15 return mail.send(parsed.data, { actor: session.user.id })16}What guardrails cannot catch
Be honest about the limits so nobody mistakes a classifier for a control. Guardrails are pattern matchers; attackers are not.
- Paraphrased and translated attacks: “disregard prior guidance” in Portuguese passes an English keyword list.
- Encoded payloads: base64 or homoglyphs pass the input check and are decoded by the model.
- Multi-turn attacks: each message is benign; the combination is not.
- Semantic misuse: a perfectly valid
refund(order, 49.99)call that should not have happened. Only authorisation and approval gates catch that (Permissions, Authentication and Authorisation). - Novel attacks: a classifier trained on last year’s jailbreaks has not seen this year’s.
Key points
- Guardrails are cheap deterministic or small-model checks at the model boundary; they filter, they do not authorise.
- Run output validation before the side effect; a check on the final message is too late.
- Schema validation on tool arguments is the highest-value output guardrail and is fully deterministic.
- Return guardrail failures to the model as tool errors so it can self-correct.
- Guardrails miss paraphrase, encoding, multi-turn and semantically valid misuse; pair them with least privilege and approval gates.
When to use — and when not to
- Every user-facing LLM product needs at least an input classifier and an output secret scan.
- Every tool call needs schema validation.
- Products with compliance obligations need PII redaction on both sides.
- Do not run a 500 ms classifier on every step of an internal batch pipeline with no untrusted input.
- Do not use a guardrail as the only protection for a destructive tool.
- Do not tune the classifier so aggressively that legitimate users are refused; measure the false positive rate.
Failure modes
- Output check runs on the final message after the harmful tool call already executed.
- Keyword deny-list is bypassed by translation or base64.
- Guardrail failures are silently dropped instead of returned to the model, so the agent loops or gives up.
- PII regex misses locale-specific formats and the compliance report is wrong.
- Classifier false positives block a large fraction of real support requests and nobody measures it.
Tradeoffs
Deterministic guardrails are the most debuggable component in the stack; model-based ones add latency and their own false positives.