Twenty questions, tools switched off, and a ceiling of five. The first eleven posts in this series have been circling one claim without proving it: a model on its own cannot do the job, and tools are what close the gap.
Time to prove it, and then build the thing that uses them.
Start with the questions it could not reach
Fifteen of the twenty were unreachable by design. Rather than treating that as one number, go back
through those fifteen and label each with the capability it needed. The needs field from your
question set already holds the answer.
from collections import Counter
unreachable = [c for c in CASES if c["needs"] != ["reasoning"]]
Counter(n for c in unreachable for n in c["needs"])
# Counter({'web': 12, 'file': 5, 'calculation': 4})
In a set of twenty written about real work, two categories dominate.
| Missing capability | Count | Why the model cannot supply it |
|---|---|---|
| Fetch something current | 12 | The value changed after training ended |
| Read a supplied file | 5 | The file was never in the training data |
| Both | 2 | Fetch a rate, then apply it to your numbers |
Read the right-hand column, because it is the whole argument. Neither gap is a shortcoming in the model's reasoning. A price that changed last Tuesday is unavailable to a system whose knowledge finished a year ago, however well it reasons. A spreadsheet you exported this morning is unavailable to a system that has never seen it, at any level of capability.
So a bigger, cleverer model moves the ceiling by a question or two. It leaves the structure exactly where it is.
Take the running question apart
Our support team closed 412 tickets last quarter. If we route the routine ones to an agent, how many staff-hours would that free, and does the saving cover what the model calls cost?
Break it into what answering actually requires:
- Read the ticket export and count the routine categories. A file the model has never seen.
- Find the average handling time for those categories. Either in the file or looked up.
- Look up what the model costs per million tokens today. Current, external, changes.
- Estimate tokens per ticket, multiply, compare against the staff-hours saved. Arithmetic.
Three of those four need something from outside the model. No prompt improves that, because the missing element is information rather than skill.
Three of four steps need something the model was never given.
What a tool actually is
A tool is a function you wrote, plus a description the model can read.
def read_tickets(quarter: str) -> str:
"""Return the ticket export for a quarter as CSV text."""
return open(f"exports/tickets-{quarter}.csv").read()
TOOLS = [{
"type": "function",
"function": {
"name": "read_tickets",
"description": "Read the support ticket export for a given quarter.",
"parameters": {
"type": "object",
"properties": {"quarter": {"type": "string", "description": "e.g. 2026-Q2"}},
"required": ["quarter"],
},
},
}]
Two halves, and the split matters. The Python function does the work. The JSON schema is documentation written for the model: what the tool is called, when it applies, and what arguments it takes.
The model never runs your function. It reads the schema, decides the tool applies, and asks you to run it.
The reply nobody shows you
Send a question with tools attached and print the whole reply, rather than just .content.
reply = client.chat(model=MODEL, tools=TOOLS,
messages=[{"role": "user", "content": "How many Q2 tickets were password resets?"}])
print(reply.stop_reason) # "tool_use" <- it stopped to ask for something
print(reply.tool_calls) # [{"name": "read_tickets", "arguments": {"quarter": "2026-Q2"}}]
print(reply.content) # "" <- no answer yet, and that is correct
Three lines of output hold the whole mechanism, and seeing them once removes most of the mystery.
stop_reason is tool_use instead of end_turn. The model stopped mid-task. It filled in the
argument itself, turning "Q2" in the question into "2026-Q2" to match the schema. And content is
empty, because the model has no answer yet and correctly declines to invent one.
A tool call is the model saying: I know what I need, I know which of your functions provides it, here are the arguments, please run it and tell me what came back.
The loop, which is the agent
The model asked for something. You run it and hand the result back, and the fact that the model may ask again is what forces the shape of the code.
messages = [{"role": "user", "content": QUESTION}]
while True:
reply = client.chat(model=MODEL, tools=TOOLS, messages=messages)
if reply.stop_reason != "tool_use":
break # it has an answer
messages.append(reply) # what it asked for
for call in reply.tool_calls:
result = DISPATCH[call.name](**call.arguments) # you run YOUR function
messages.append({"role": "tool", "content": result}) # what came back
print(reply.content)
Twelve lines, and every earlier post in this series is now load-bearing. The list you keep and re-send
is messages. The growing cost is every one of those tool results riding along on the next call. The
prompt work decides whether the model reaches for a tool or asks your permission first.
Watch the loop run on the running question and the sequence is unremarkable:
turn 1 tool_use read_tickets(quarter="2026-Q2")
turn 2 tool_use web_search("input token price <model> per million")
turn 3 tool_use calculate("118 * 6 / 60")
turn 4 end_turn "Password resets consumed about 11.8 staff-hours..."
Four passes for this question. A different question takes two, or nine. Nothing in the code above states how many, and nothing could.
The chain, end to end
The unknown step count is the point everything has been building toward.
Most real questions need information the model was never given. Counted, not asserted: fifteen of twenty, and no model upgrade changes it.
So the model needs tools, because a tool is the only route from a system that reasons over training data to a system that can read your file and check today's price.
So something has to choose which tool, and when. Your questions vary, so the choice varies. The model is the only component holding enough context to make it.
So the code has to loop, because the model chooses one step at a time and each result changes what it wants next. You cannot write the sequence in advance, so you write a loop that keeps asking.
That loop is the agent. Not a bigger model. The same model, handed capabilities and asked repeatedly what it wants next.
No tools, so most questions are unreachable. Add tools, and something must choose which one, every turn.
What tools do not fix
A tool changes where an answer comes from. It leaves plenty untouched, and being clear about that saves disappointment later.
The model still chooses badly sometimes: the wrong tool, a malformed argument, a search phrased so poorly it returns nothing. It can read a correct tool result and still draw the wrong conclusion. And every tool you add widens what the system can do when it goes wrong, which is a sentence worth sitting with before you hand one an API key.
What a tool genuinely provides is a source. After the run you can point at the file that was read and the page that was fetched. An answer from training data offers nothing to point at, and a confident guess and a real answer arrive in the same format.
Traceability is why tools sit at the centre of agent design. They move an agent from recalling to fetching, and fetching leaves a trail.
What to take from this
- Count what your unreachable questions actually need. The gap is missing information, not missing intelligence, so no model upgrade closes it.
- A tool is your function plus a schema the model reads. The model never runs it; it asks you to.
- Handing over tools forces a loop, because the step count depends on what each result turns up. That loop is what makes it an agent.
A model answers. An agent reads, fetches, calculates and decides when it is finished. The distance
between those two is twelve lines of while loop and a set of functions you already know how to
write.