How tool calling works: the model asks, your code runs

You will be able to describe each step of a function calling exchange between your code and a model.

Marcus runs a small online shop in Singapore selling coffee equipment, with buyers in Malaysia and Indonesia too. Most customer messages are the same three questions: where is my order, how much is this in ringgit, and is the grinder back in stock. He tried a chat assistant with his product list pasted in. It answered the stock question from a list that was already out of date, invented a tracking status, and got the ringgit price wrong by working out the exchange rate from memory.

The model was not the problem. It had no way to look anything up. To answer these questions properly, it needs to ask Marcus's own systems, and that is what tool calling is for.

The model can ask for a function

In AI fundamentals, lesson 7.3, Tool use: when a model asks software to act, you met the idea from the outside. Now you build it. Tool calling, also called function calling, lets you describe functions your code can run, and lets the model reply with a request to run one of them instead of answering directly.

You describe each tool in the request, alongside the messages. A tool description has three parts: a name, such as get_order_status; a description in plain words of what it does and when to use it; and a JSON schema for its arguments, in the same schema format you used in lesson 2.2, Ask for JSON and enforce it with a schema. For get_order_status, the schema might say there is one required argument, order_number, which is a string.

The tools are sent with every request, much like the system prompt, and they count towards your input tokens.

Step by step through one exchange

Here is the full sequence for one customer question. Follow who sends what.

First, your code sends a request containing the system prompt, the customer's message "Where is my order 10482?", and the list of tools.

Second, the model replies. But instead of a text answer, the reply contains a tool call: the name get_order_status and the arguments, with order_number set to "10482". The response also marks that it stopped in order to call a tool, so your code can tell this apart from a final answer. Each tool call carries an ID.

Third, your code reads the tool call, checks the arguments and runs the real function. That function queries Marcus's order database and gets back a result: shipped on Tuesday, with a tracking number from the courier.

Fourth, your code sends a new request. It contains everything from before, plus the model's tool call, plus a new message holding the tool's result, labelled with the same ID so the model knows which call it answers. Remember from lesson 1.1 that the model keeps no memory between calls, so the whole exchange has to be resent.

Fifth, the model reads the result and replies with a normal text answer: your order shipped on Tuesday, and here is the tracking number.

That is two model calls and one function call for one question. If a question needs two tools, such as checking stock and converting a price, the model may ask for both at once or one after the other, and the cycle repeats until it gives a final answer. Lesson 5.1 turns that repetition into an agent.

The model only asks

This is the point to hold onto through the rest of the course. The model never runs anything. It has no access to Marcus's database, his courier account or the internet. All it can do is produce text that says "please call get_order_status with these arguments". Your code receives that request and decides what to do with it.

That makes your code the gatekeeper. It can run the function, refuse, ask a person first, or return an error. The model's tool call is a suggestion, written by a system that can be wrong or can be manipulated by text it has read, and lesson 4.3, Check every argument before you act on it, treats it that way.

It also means the tools you define set the limit of what the model can do. A model given only get_order_status and convert_currency cannot issue refunds, however a customer asks, because no such tool exists. Choosing which tools to provide is one of the most important safety decisions you make, and lesson 8.2 returns to it.

Why this fixes Marcus's problems

Each of Marcus's three failures came from the model answering from memory. With tools, stock comes from the live inventory table, tracking comes from the order system, and the ringgit price comes from code that multiplies the price by a current exchange rate. The model's job shrinks to understanding the question, picking the right tool and writing the reply. Those are the parts it is good at.

Before you write any tool code, it helps to see the exchange laid out. The activity below asks you to draw the message sequence for one question that needs one tool.

Draw the message sequence for a question that needs one tool call, labelling who sends each message and what it contains.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).