Toolsfunction callingdispatch loopJSON argumentsobservations

Tool Calling Basics

The model emits a structured request to call a function; your application executes it and feeds the result back — the model never runs anything itself.

▶ InteractiveInterview question
Progress

What tool calling actually is

Tool calling (also "function calling") is a contract between the model and your code. You send the model a list of tool definitions — a name, a description, and a JSON schema for parameters. Instead of answering in prose, the model may respond with a tool call: the name of a tool plus arguments that conform to the schema.

That is the entire mechanism. The model produces text that happens to be a well-formed JSON object; the provider parses it into a typed tool_call block. Nothing has been executed yet. Your application inspects the call, decides whether to run it, runs it, and returns the result to the model as a tool result message. The model then continues — either calling another tool or producing the final answer.

Compare this with the escalation in Choosing the Right Abstraction: tool calling sits one step above structured output. Use it when the model must decide which of several actions to take and with which arguments. If there is only one action and it always runs, call the function yourself and skip the model.

The round trip

A single tool-using turn is a fixed sequence of seven steps. Every framework, from raw HTTP to LangGraph, is a wrapper around this loop.

Notice that the model appears twice and the executor appears once. The executor is your code: it owns credentials, network access and side effects. The model only ever sees strings going in and strings coming out.

One tool-calling round trip
yesnoUser requestLLM reads request + tool listSelects toolEmits structured arguments (JSON)Final responseApp validates argumentsApp executes functionTool result appended to contextLLM reads result
UserLLMAgentToolDataDecisionHumanGuardrail

The model never executes anything

This is the single most important idea in the module. A tool call is a request, not an action. The model has no socket, no filesystem, no database handle. When people say "the agent deleted the table", what happened is: the model emitted {"name":"run_sql","arguments":{"query":"DROP TABLE users"}} and an application, written by a human, ran it without checking.

Consequences follow directly. Argument validation (Argument Validation) is your job. Permission checks (Tool Permissions and Least Privilege) are your job. Retries and timeouts (Tool Errors, Retries and Timeouts) are your job. The model can be asked to be careful, but "asked" is not a control — it is a suggestion to a probabilistic process.

The upside is symmetrical: because you own execution, you can put any policy you like between the model and the world — dry-run mode, approval gates, rate limits, sandboxing — without changing the model or the prompt.

  • Tool call = model output that parses as {name, arguments}. Nothing more.
  • Execution, side effects and credentials live exclusively in the application.
  • Every safety property of a tool-using system is enforced at the dispatch boundary, not in the prompt.

A minimal dispatch loop

Below is the whole thing with no framework. TOOLS maps names to Python callables; SPECS is what the model sees. The loop runs until the model stops requesting tools. In production you add a step budget, validation, tracing and error handling — see Budgets, Limits and Termination — but the skeleton does not change.

Framework-free tool dispatch loop (provider call abstracted as `chat`).
1import json
2 
3def get_weather(city: str) -> dict:
4    return {"city": city, "temp_c": 21}
5 
6TOOLS = {"get_weather": get_weather}
7SPECS = [{
8    "name": "get_weather",
9    "description": "Current temperature for a city.",
10    "parameters": {
11        "type": "object",
12        "properties": {"city": {"type": "string"}},
13        "required": ["city"],
14    },
15}]
16 
17def run(messages: list[dict], max_steps: int = 8) -> str:
18    for _ in range(max_steps):
19        reply = chat(messages, tools=SPECS)  # provider API call
20        messages.append(reply)
21        if not reply.get("tool_calls"):
22            return reply["content"]           # final answer
23        for call in reply["tool_calls"]:
24            fn = TOOLS[call["name"]]           # KeyError => unknown tool
25            args = json.loads(call["arguments"])
26            result = fn(**args)                # the APP executes, not the model
27            messages.append({
28                "role": "tool",
29                "tool_call_id": call["id"],
30                "content": json.dumps(result),
31            })
32    raise RuntimeError("step budget exhausted")

Key points

  • A tool call is structured text the model emits; the application parses, validates and executes it.
  • The round trip is request → model → tool selection → arguments → execution → result → model → answer.
  • Tool results go back into context as messages; the model reasons over them like any other input.
  • All safety, permission and reliability controls live at the dispatch boundary in your code.
  • The loop needs a step budget; without one a confused model can call tools forever.
  • Tool calling is one step above structured output on the abstraction ladder — use it only when the model must choose an action.

Tool-calling simulator

Tool-calling simulator
Pick a scenario and step through it. Notice how much happens in application code around the model.
User requestLLMTool selectionStructured argsValidateExecuteTool resultLLMFinal response
User request
What’s the weather in Berlin right now, in Celsius?
Request. The request mentions a city and present conditions — a good match for the tool description.
Tool schema the model sees
{
  "name": "get_weather",
  "description": "Current weather for a city. Use for questions about
                  present conditions, not forecasts.",
  "parameters": {
    "type": "object",
    "properties": {
      "city":  { "type": "string", "description": "City name, e.g. 'Berlin'" },
      "units": { "type": "string", "enum": ["celsius", "fahrenheit"] }
    },
    "required": ["city"]
  }
}
1/9 · Request

When to use — and when not to

Use it when
  • The model must pick between several actions or data sources based on the request.
  • Arguments depend on natural-language input that deterministic parsing cannot handle reliably.
  • You need a typed interface between the model and existing APIs instead of parsing free text.
  • The task requires live data (weather, inventory, tickets) the model cannot know.
Avoid it when
  • There is exactly one action and it always runs — call it directly and pass the result in the prompt.
  • The output is a fixed-shape record with no side effects — use Structured Outputs instead.
  • Latency budget is tight: every tool call adds at least one extra model round trip.
  • The action is irreversible and there is no approval path — fix the control plane first.

Failure modes

  • Model picks the wrong tool because two descriptions overlap (see Tool Schemas).
  • Arguments parse as JSON but violate semantics — negative quantity, path outside the sandbox.
  • Tool result is huge (a whole CSV) and blows the context window on the next turn.
  • No step budget: the model alternates between two tools indefinitely.
  • Executor trusts call["name"] and dispatches via eval or getattr on arbitrary objects.

Tradeoffs

Complexity
low → high
Latency
low → high
Cost
low → high
Reliability
poor → strong
Debuggability
hard → easy

Cheap to build, but each call is a full model round trip; debuggability is high because every call and result is a discrete message you can log.

Don't delegate understanding
The manifesto →
AI should amplify judgment, not replace it.
Tool Schemas →