Introduction
A person can read a friendly paragraph, but a program cannot. If your app needs the city, the date and the price from a customer message, it needs those three values in a fixed shape, not a sentence. Structured output means asking the model to reply in a format such as JSON, so that code can read the answer. A second need is action: a model cannot check the weather or send an email, but it can ask your program to do so. That request is called a tool call. Both ideas depend on one habit: never trust the model's reply until your code has checked it.
How it works
For structured output you describe the schema, the fields and their types, in the prompt, for example "Reply only with JSON that has a string field name and an integer field age". A model usually obeys, but it can add friendly words around the JSON, leave out a field or give a number as text. So the code does three things in order: it parses the text with json.loads, which raises an error for text that is not JSON, then validates the result (is it an object, are the fields there, do they have the right types), and finally uses it. When a check fails, the program does not crash: it records the reason and often retries, sending the reason back to the model, up to a small limit such as three tries.
Tool calling builds on this. You tell the model which tools exist and what arguments each takes. When it wants one, it replies with a structured message such as {"tool": "add", "args": {"a": 19, "b": 23}}. Your code, not the model, validates the message, runs the function and sends the result back as a new message. The model then continues, and this cycle is the agent loop:
- Send the conversation to the model.
- If the reply is a tool call, validate it, run the tool and add the result to the conversation.
- If the reply is a final answer, stop.
- Stop after a maximum number of steps, so a confused model cannot loop forever.
Because the model's words decide what your code runs, every tool call is checked: the tool must be on your list, and the arguments must have the right count and types. Tools that change things, such as sending money, need extra care and often a person's approval.
Real libraries wrap this loop for you. A typical call looks like this (shown, not run, because it needs an account and a network):
reply = client.chat(model="some-model", messages=history, tools=tool_list)
In this lesson a plain function and a fixed script stand in for the model, so the whole loop runs in your browser.
Example
import json
# A stand-in for a language model: it replies from a fixed script.
# A real model would read the whole conversation before each reply.
script = [
"Sure! I will add the numbers first.",
'{"tool": "add", "args": {"a": 19, "b": 23}}',
'{"tool": "multiply", "args": {"a": 42, "b": "two"}}',
'{"tool": "multiply", "args": {"a": 42, "b": 2}}',
'{"answer": "19 + 23 = 42, and 42 x 2 = 84."}',
]
tools = {
"add": lambda a, b: a + b,
"multiply": lambda a, b: a * b,
}
def check(message):
"""Returns what is wrong with the model's message, or None if it is a valid tool call or answer."""
if not isinstance(message, dict):
return "not a JSON object"
if "answer" in message:
return None
if message.get("tool") not in tools:
return "unknown tool"
args = message.get("args", {})
if not all(isinstance(args.get(k), (int, float)) for k in ("a", "b")):
return "a and b must be numbers"
return None
history = ["user: What is (19 + 23) x 2?"]
for step, reply in enumerate(script):
try:
message = json.loads(reply)
except json.JSONDecodeError:
message = reply
problem = check(message)
if problem:
print(f"{step}: rejected ({problem})")
history.append(f"error: {problem}. Reply with JSON only.")
continue
if "answer" in message:
print(f"{step}: final answer: {message['answer']}")
break
a, b = message["args"]["a"], message["args"]["b"]
result = tools[message["tool"]](a, b)
print(f"{step}: {message['tool']}({a}, {b}) = {result}")
history.append(f"tool: {result}")
print(len(history), "messages in the conversation")Line by line
scriptis the fake model: five replies in order. The first is chatty text, the third has a bad argument, and the last is the final answer.toolsmaps each tool name to a function. Only names in this table can ever run.checkreturns the reason a message is wrong, orNonewhen it is fine. It tests the shape first, then the tool name, then the argument types.- For each reply the loop tries
json.loads. Text that is not JSON stays a string, andcheckrejects it. - A rejected reply adds an error message to the history, which a real model would read before answering again.
- A good tool call is run, and its result is added to the history. The final answer ends the loop.
Expected result
0: rejected (not a JSON object)
1: add(19, 23) = 42
2: rejected (a and b must be numbers)
3: multiply(42, 2) = 84
4: final answer: 19 + 23 = 42, and 42 x 2 = 84.
5 messages in the conversation
In the real world
Assistants that book meetings, look up orders or search the web all work this way: the model decides which tool to call, and the application runs it. Apps that extract data, such as invoices into table rows, use JSON schemas so that every answer can go straight into a database. Because models sometimes break the format, production systems always validate, log the failures and retry.
Common mistakes
- Using the reply without parsing it safely, so one chatty answer crashes the program.
- Checking only that the text is JSON, not that the fields and types are right.
- Running any tool name the model asks for, instead of only the tools on your list.
- Retrying forever when the model keeps failing.
- Letting a model's tool call run a risky action without any approval.
Best practices
- Describe the schema in the prompt, and validate it again in code.
- Return a short, specific error message when a reply is wrong.
- Limit the number of retries and the number of loop steps.
- Keep a list of allowed tools, and ask a person to approve actions that cannot be undone.
Summary
Structured output asks a model for JSON that matches a schema, and code must parse and validate it because the format can break. Tool calling lets a model ask for functions that your code runs, in a loop of reply, check, run and result until a final answer. Checking every message, allowing only listed tools and limiting retries keep the loop safe.
Quick check
Key takeaways
- Free text is hard for code to use; ask for JSON and always validate it, because a model can break the format.
- A model never runs anything itself: it only asks for a tool call, and your code decides whether to run it.
- Send clear error messages back to the model so that it can fix its reply, and stop after a few tries.
Practice problems for this lesson
My notes
Write down what you want to remember from this lesson. Notes are saved on this device as you type, and appear on your dashboard.