What is function calling?
Function calling (also called tool use) lets you expose typed functions to the LLM. Instead of asking the model to produce text, you ask it to produce a structured call to one of your functions — with arguments that your code then executes.
{"function": "get_weather", "args": {"city": "Tokyo"}}
get_weather("Tokyo") → "22°C, cloudy"
Writing a tool schema
You describe each tool as a JSON schema — the model uses this to decide when and how to call it. Three parts matter most: the name, the description, and the parameters.
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city. "
"Use this when the user asks about weather "
"or temperature in a specific location.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name, e.g. 'Tokyo' or 'New York'"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit. Default: celsius"
}
},
"required": ["city"]
}
}
}
]Use snake_case, descriptive verbs: get_weather, search_products, send_email. The name is the model's primary signal for when to call it.
This is your most important field — explain what the function does AND when to use it. Include example triggers: "Use this when the user asks about…"
Describe each parameter clearly. Use enum to constrain values where possible. Mark required vs optional explicitly.
Building the tool loop
Function calling is a multi-turn process. Here's the complete pattern using the OpenAI API:
import json
from openai import OpenAI
client = OpenAI()
messages = [{"role": "user", "content": "What's the weather in Tokyo?"}]
while True:
response = client.chat.completions.create(
model="gpt-4o-mini",
tools=tools,
messages=messages,
)
msg = response.choices[0].message
# No tool call — model has final answer
if not msg.tool_calls:
print(msg.content)
break
# Execute each tool call
messages.append(msg) # add assistant message with tool_calls
for tc in msg.tool_calls:
args = json.loads(tc.function.arguments)
result = get_weather(**args) # your function
messages.append({
"role": "tool",
"tool_call_id": tc.id,
"content": json.dumps(result),
})
# loop continues — model will now generate final answerTools vs pure prompting
| Situation | Use |
|---|---|
| Need real-time or live data (weather, prices, search) | Tools |
| Need exact arithmetic or guaranteed correct computation | Tools |
| Writing, summarisation, content generation | Pure prompting |
| Need to take side-effecting actions (send email, write DB) | Tools |
| Reasoning and analysis over provided text | Pure prompting |
| Classification, extraction from text in the prompt | Pure prompting |
Rule of thumb: use tools when the model needs to do something beyond reasoning over text. Use prompting when it just needs to think about text already in the context.
Writing better tool descriptions
Weak description
"description": "Search for products."No context on when to call it. The model may call it for everything, or never.
Strong description
"description": "Search the product catalogue by keyword or
category. Use this when the user is looking for a specific
product, asking what's available, or comparing options.
Do NOT use this for order history or returns."Tells the model when to call it AND when NOT to. Negative instructions prevent over-calling.
Notes
Parallel tool calls can create race conditions in shared state
GPT-4o and later OpenAI models support parallel tool calls by default. If two of your tools read from and write to shared state (a database row, a file), parallel execution can produce inconsistent results. Set parallel_tool_calls=False in the API call when tools share mutable state.
Tool errors must be returned as tool messages, not raised exceptions
If your tool function raises an uncaught Python exception instead of returning an error string, the API call fails entirely and the model receives no feedback. Always wrap tool function bodies in try/except and return a descriptive error string as the tool result. The model can then reason about the error and decide how to recover.
Enable strict=True to eliminate hallucinated parameters
OpenAI's Structured Outputs for tool calls (strict=True in the tool schema) forces the model to pass only arguments defined in the schema. Without it, the model occasionally passes extra keyword arguments that break your function signature. Enable strict mode in production for all tool schemas.
Each tool definition adds to input token cost on every call
With 10+ tools in the payload, schema definitions alone can add 500–1,000 input tokens to every API request. At scale this is significant. Group related tools into a single tool with a type enum parameter (e.g. one search_product tool with a category enum) to reduce schema size.
Tool Use & Function Calling FAQ
What is function calling in LLMs?
Function calling (also called tool use) lets you expose typed functions to an LLM. The model produces a structured JSON call specifying which function to invoke and with what arguments — your code then executes the function and feeds the result back to the model. The LLM never executes code itself.
How do I write a good tool schema for an LLM?
A good tool schema has three key parts: a descriptive snake_case name (e.g. get_weather), a detailed description that explains what the function does AND when to call it (include trigger phrases like "use this when the user asks about..."), and clearly described parameters with enum constraints where applicable.
What is the difference between tool use and pure prompting?
Use tools when the model needs to act beyond reasoning over text — live data, exact arithmetic, or side-effecting actions like sending email. Use pure prompting when the model just needs to reason, write, summarise, or classify text already in the context window.
How does the tool loop work in a multi-turn conversation?
The tool loop is: (1) send a message with tool definitions, (2) check if the response contains tool_calls, (3) execute each tool call in your code, (4) append the tool result as a tool-role message, (5) call the API again. Repeat until the model returns a response with no tool_calls — that is the final answer.
How do I write better tool descriptions so the LLM uses them correctly?
Include both positive triggers ("use this when the user asks about product availability") and negative exclusions ("do NOT use this for order history"). Without negative instructions, models tend to over-call tools. Describe each parameter's purpose and valid values using enum constraints to reduce invalid arguments.
Quick summary
- Function calling: LLM produces a structured call → your code executes it → result fed back
- Tool description is your most important field — include when to call it AND when not to
- The tool loop: call API → check for tool_calls → execute → append result → repeat
- Use tools for live data, exact computation, and side effects. Use prompting for reasoning over text
- Use
enumin parameters to constrain values the model can pass