Multi-Agent Systems Overview
Multiple agents with distinct roles can be arranged as supervisor, pipeline, hierarchy or swarm; a role buys separate context, tools and evals, at the price of coordination.
What a role actually buys you
A "second agent" is not magic. It is a second LLM loop with its own system prompt, own context window, own tool set and own evaluation suite. Those four separations are the entire value proposition; if you would not use any of them, you do not need a second agent.
Separate context prevents a 60k-token research transcript from crowding out the writing instructions (Context Ordering & Lost in the Middle). Separate tools let the reviewer agent have zero write access while the executor has some (Tool Permissions and Least Privilege). Separate evals let you measure "did research find the right sources" independently of "was the prose good" (Evaluating Agents: Testing Probabilistic Systems).
The cost is coordination: every hand-off is a lossy serialisation of state into text, another model turn of latency, and another place for errors to compound. When Not to Use Multi-Agent makes the full case; this lesson maps the shapes.
The four topologies
Supervisor: one orchestrator decomposes a task, delegates to specialists, merges results. Central control, central bottleneck.
Pipeline: fixed stages, each agent transforms the previous output — Research → Analyze → Write → Review. Predictable, easy to eval per stage, no dynamic routing.
Hierarchical: supervisors of supervisors. A manager assigns to team leads who assign to workers. Scales breadth; each level adds latency and summarisation loss.
Swarm / peer: agents hand off to each other directly based on capability; no central controller. Flexible, hardest to trace and bound.
Pipeline and hierarchy
A pipeline is a DAG of agents where edges are data. Because each stage has a typed input and output, you can replace any stage with deterministic code or a cheaper model without touching the others — the strongest argument for starting here when a task really is sequential.
A hierarchy is a tree. Information flows down as tasks and up as summaries; every level compresses. Three levels means the top agent sees a summary of a summary — fine for status, dangerous for details like "the migration drops a column".
Hierarchy and swarm
In a swarm, control moves with the conversation: a triage agent hands the thread to billing, billing hands to refunds. Each hand-off transfers a bounded context plus the active tool set. Without a hop limit, two agents can hand off to each other forever (Budgets, Limits and Termination).
Key points
- A role = separate context + separate tools + separate evals + separate prompt; nothing else.
- Supervisor centralises control; pipeline fixes order; hierarchy scales breadth; swarm decentralises.
- Every hand-off is lossy serialisation plus latency plus a new failure point.
- Pipelines are the easiest to eval and to partially replace with deterministic code.
- Choose the topology from the task's shape, not from what a framework demos.
- Ask When Not to Use Multi-Agent before choosing any of these.
Multi-agent architectures
When to use — and when not to
- Subtasks need genuinely different tool sets or permission levels.
- A single context cannot hold all necessary material without degrading.
- You need independent, per-role quality metrics.
- Subtasks are parallelisable and wall-clock latency matters.
- One well-prompted agent with good tools already hits the quality bar.
- The task is a fixed sequence a workflow graph can express (Workflow State Graph).
- You cannot yet evaluate the single-agent version — you will not be able to evaluate the multi-agent one either.
Failure modes
- Context lost at hand-off: the writer never sees the caveat the researcher found.
- Supervisor becomes a serial bottleneck (
supervisor-bottleneck). - Agents disagree and loop; no termination condition.
- Cost multiplies: 4 agents × 3 turns each ≈ 12 model calls for one user request.
- Blame is diffuse; traces span agents and nobody owns the regression.
Tradeoffs
Ratings for a generic multi-agent system; pipelines score better on reliability and debuggability than swarms.