HITLhitlapprovalrisksafetyoversight

Human-in-the-Loop Overview

Between an agent proposing an action and executing it sits a risk assessment that routes high-impact actions to a human for approval — the architectural control that makes autonomous systems deployable.

▶ InteractiveInterview question
Progress

The gate

An agent that can send email, move money, delete data, publish content or change production will eventually do one of those wrongly. Prompting it to "be careful" is not a control. The control is a gate in the execution path: the agent proposes an action, code classifies its risk, and above a threshold the action waits for a human decision before anything runs.

The gate lives in the tool-execution layer, not in the model. The model emits send_email(to, subject, body) exactly as before; the executor intercepts, computes a risk class, and either runs it, asks, or refuses. This means the control holds even under Prompt Injection — an injected instruction can make the model *propose* a transfer, but cannot skip the gate.

Approval gate in the execution path
yesnoapprovedAgentAction proposedRisk assessment (code)Approval required?Human approves / edits / rejectsExecuteAudit logResult to agent
UserLLMAgentToolDataDecisionHumanGuardrail

Which actions

The classic set: sending communications (email, Slack, SMS — reputational, unrecallable), moving money (refunds, payouts, purchases), deleting or overwriting data, publishing content (public pages, app store listings), and modifying production (deploys, config, IAM). What they share is that the cost of a wrong action is far higher than the cost of a 30-second pause.

Read-only actions almost never need a gate; the exception is reading data whose exposure is itself the harm (PII exports), which Tool Misuse and Data Exfiltration covers.

  • Draft the email, show it, send on click — the approval is the send, not the draft.
  • Refund under 20 EUR: auto; 20–500 EUR: approval; over 500 EUR: two approvers (Approval Gates and Risk Classes).
  • DELETE and DROP are gated regardless of who asked; SELECT is not.

Cost of the gate

Gates add latency (human response time — minutes to hours) and require a UI, a queue, timeouts and escalation (In-the-Loop vs On-the-Loop and Escalation). If every action needs approval, the agent is a slow form; if none does, it is a liability. Calibrate by measuring: what fraction of approvals are rubber-stamped? Those can move to auto-with-audit. Which auto actions were later reverted? Those need a gate.

Persist the agent's state while it waits. A run paused for approval must survive a process restart, which is why workflow graphs with checkpointing (Workflow State Graph) pair naturally with HITL.

Key points

  • The gate is code in the executor, not instructions in the prompt.
  • Gate actions whose wrong execution costs far more than a pause: communications, money, deletion, publishing, production.
  • Read-only actions usually run free; data exposure is the exception.
  • Calibrate thresholds from approval and reversal rates, not guesses.
  • Paused runs must be durable across restarts.
  • The gate holds even when the model is manipulated by injected content.

Approval gate simulator

Approval gate simulator
Pick an action; tune the policy (auto-approve limit, confidence threshold). The gate decision is made in code from risk, not by asking the model whether it feels sure.
yesnoapprovedAgentAction proposedRisk assessmentApproval required?Human approvalExecute
Reversible
no
Blast radius
a team / customers
Risk score
6
Approval required. Above the auto-refund limit → approval. Below it, policy in code approves automatically.

In-the-loop: a human approves before execution. On-the-loop: it executes and a human can intervene or roll back. Escalation: when no approver responds within the SLA, fail closed for irreversible actions.

When to use — and when not to

Use it when
  • Any agent with tools that have external side effects.
  • Early deployment of a new agent, before evals prove its action accuracy.
  • Regulated domains where a human must be accountable for the decision.
Avoid it when
  • Purely read-only or advisory agents — an approval on a summary is theatre.
  • Batch tasks with thousands of low-risk actions — gate the batch, not each item.
  • As a substitute for fixing an agent that proposes bad actions 30% of the time.

Failure modes

  • Gate implemented as a prompt instruction; bypassed by injection.
  • Approval fatigue: humans click approve without reading (Approval Gates and Risk Classes on UI design).
  • No timeout — runs pile up waiting for someone who is off shift.
  • State lost on restart; approved action executes twice or never.
  • Risk classifier keyed on tool name only, missing a high-risk argument (amount=50000).

Tradeoffs

Complexity
low → high
Latency
low → high
Cost
low → high
Reliability
poor → strong
Debuggability
hard → easy

Latency is dominated by human response time; reliability rating reflects prevented incidents, not throughput.

Approval Gates and Risk Classes →