Compare Architectures
Side-by-side: use case, requirements, latency, cost, strengths, weaknesses, failure modes, example — and when to choose each.
Workflow vs AgentSingle agent vs Multi-agentRAG vs Fine-tuningVector search vs BM25Function calling vs MCPDirect API integration vs MCPPlan-then-execute vs ReActShort-term memory vs Long-term memoryLLM-as-judge vs Deterministic evaluatorsHuman-in-the-loop vs Human-on-the-loopSupervisor vs PipelinePrompt caching vs Semantic cachingRouter vs SupervisorFramework vs No framework
| LLM-as-judge | Deterministic evaluators | |
|---|---|---|
| Use case | Open-ended quality: helpfulness, faithfulness, tone | Verifiable properties: schema, exact match, tool called, length |
| Requirements | Rubric, judge prompt, calibration set graded by humans | Code assertions and reference values |
| Latency | One extra model call per sample | Milliseconds |
| Cost | Tokens per evaluation; expensive at scale | Near zero |
| Strengths | Scales human-like judgement to thousands of samples | Reproducible, cheap, no drift |
| Weaknesses | Biased (position, verbosity, self-preference); noisy | Cannot assess quality beyond what is checkable |
| Failure modes | Judge agrees with confident wrong answers | Passing tests while answers are useless |
| Example | Is this summary faithful to the source? | Did the output parse as valid JSON with required fields? |
| Choose this when | Choose an LLM judge for qualities you cannot express as code, and calibrate it against human labels first. | Choose deterministic evaluators for everything checkable; run them first, on every change. |