How tool calling works: the model asks, your code runs

You will be able to describe each step of a function calling exchange between your code and a model.

Marcus sells coffee equipment through a small online shop in Singapore, and some of his buyers are in Malaysia and Indonesia. Nearly every customer message he gets asks one of three things: where an order has got to, what an item costs in ringgit, and whether the grinder is back in stock.

To save time, he set up a chat assistant and pasted his product list into it. It told customers about stock using a list that had already gone out of date, it invented a tracking status, and it quoted the wrong ringgit price after working out the exchange rate from memory. The model itself was working fine. Its trouble was that it could only answer from memory, with no means of checking anything.

Moving the lookups into your own systems

Tool calling, which you will also see called function calling, fixes this. You list functions your code is able to run, and the model can respond with a request for one of them in place of a direct answer. The AI fundamentals course showed you tool use from the outside in lesson 7.3, "Tool use: when a model asks software to act". Now you build the pieces yourself.

Once tools are in place, the facts come from systems that hold them. Stock levels are read from the live inventory table. Tracking details come out of the order system. A ringgit price is calculated by code that multiplies the price by a current exchange rate. The model is left with three jobs it does well: making sense of the question, choosing the right tool, and writing the reply.

What goes into a tool description

Tools are described in the request, next to the messages. Each tool description has 3 parts: a name, a plain-language description saying what the tool does and when to use it, and a JSON schema for its arguments. For the arguments, you write the same kind of JSON schema you used in lesson 2.2, "Ask for JSON and enforce it with a schema".

Marcus defines two tools. The first, get_order_status, looks up an order, and its description tells the model to use it when a customer asks where their order is. It takes a single required string argument, order_number. The second, convert_currency, handles the ringgit question.

The tool list goes out with every request, in the same way the system prompt does, and the descriptions count towards input tokens.

The five steps of one order lookup

Suppose a customer messages Marcus asking where order 10482 is. Five steps pass between that message and the answer.

Step 1. Your code sends a request containing the system prompt, the customer's message and the list of tools.

Step 2. The model does not answer in text. It sends back a tool call: a reply that names a tool and supplies arguments for it. In this case it names get_order_status with order_number = "10482". The response is marked as a stop to call a tool, which lets your code distinguish a tool call from a final answer, and the call comes with an ID.

Step 3. Your code reads the call, checks the arguments and runs the actual function. For Marcus, that means a query on his order database, which returns "shipped on Tuesday" along with a courier tracking number.

Step 4. Your code sends a fresh request. It includes everything sent the first time, the model's tool call, and a new message containing the tool result. That message carries the same ID as the call it answers, so the model can pair each result with its call. The whole exchange has to be sent again because the model keeps no memory between calls, which you saw in lesson 1.1.

Step 5. The model reads the result and writes to the customer in ordinary text, saying the order shipped on Tuesday and giving the tracking number.

Counted up, one question needing one tool costs 2 model calls and 1 function call. A question that needs two tools can lead the model to request both together (in parallel) or one and then the other (in sequence). Steps 2 to 5 keep repeating, in either pattern, until the model produces a final answer. In lesson 5.1, that repeating cycle becomes an agent.

Your code decides what actually runs

Keep this in mind for the rest of the course: the model runs nothing. What it produces is text requesting a function call with particular arguments. It cannot reach the database, the courier account or the internet. Its only way to affect any of them is to ask your code to act.

That makes your code the gatekeeper. For each tool call that arrives, your code can run the function, refuse, ask a person first, or return an error. Treat every tool call as a suggestion from a system that can be mistaken, or that can be manipulated by text it has read. Lesson 4.3, "Check every argument before you act on it", deals with calls that are wrong or manipulated.

The set of tools you define is the limit of what the model can do. An action with no tool behind it is out of reach, however a customer asks. Marcus gave his assistant no refund tool, so it cannot issue a refund, whatever a customer says. Deciding which tools to provide is one of the most important safety decisions you will make, and lesson 8.2 comes back to it in more detail.

Before you write any tool code, it helps to see the exchange laid out. The activity below asks you to draw the message sequence for one question that needs one tool.

Pick a question that needs one tool call and draw the message sequence for it. Label who sends each message and what is in it.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).