You will be able to check tool arguments, and add permission checks and confirmations to actions that change data.
Pick one tool your assistant can call and trace the path from the moment the model's request reaches your code to the line that fetches or changes something. Write down every check that sits on that path. If the list is empty, the tool runs on the model's say-so.
Lesson 4.1, "How tool calling works: the model asks, your code runs", showed that a model can only request a tool. Your code decides whether the tool runs, and it can only decide sensibly after it has examined the request. The checks on your list do that examining. An empty list means your code takes whatever the model asked for and acts on it.
Marcus sells products such as a burr grinder and is building an AI assistant to answer his customers. The assistant already has two tools. To handle returns, he adds a third, cancel_order, which takes an order number.
A tester asks about order 10482 (an invented example, as are all the order details here). Several messages later, the same tester says order 10377 is theirs too and asks to cancel it. The model passes 10377 to cancel_order. That order belongs to someone else, and the tool cancels it.
The model was not at fault here. It got a believable message and had a tool that could cancel any order. The fix belongs in Marcus's own code, and he makes four changes, which the sections below walk through.
Treat the arguments in a tool call as untrusted input. Handle them the way you would handle a form field filled in by an anonymous visitor to your website. A bad argument can come from three places. The model may have misunderstood. It may be passing on something the user said, as with order 10377. Or text hidden in a document it read may have steered it.
Before any tool does its work, run three checks in this order.
The first is type and format. An order number is a string of five digits, an amount is a number, and a date is a real date in the format you expect. Schemas and structured output cut down on wrong types, but check anyway, because, as lesson 2.2 showed, a value with the right shape can still be the wrong value.
The second is range. Amounts are positive and under a sensible ceiling, a date that must be in the future is in the future, and quantities are whole numbers.
The third is permission: is the person in this conversation allowed to access this record? Take the customer's identity from the login session, never from the model. When the check fails, the tool returns a refusal and takes no action.
Developers tend to skip the permission check because the model feels like an ally. The model is on nobody's side. It builds arguments from the conversation, and the conversation holds anything the user chose to type. Marcus's first fix is this check. His code reads the logged-in customer's ID from the session and confirms the order is on that customer's account. If it is not, the tool replies "This order is not on your account" and stops.
Some tools carry more risk than others, so split them into two groups. Read-only tools fetch information without changing anything, for example order status, price conversion and document search. If one is called wrongly, you get a wrong answer, which is bad but fixable. Write tools act on the world: sending emails and messages, taking payments, issuing refunds, cancelling orders, editing or deleting records. A mistaken write call can lose money or trust, and is often hard to undo.
Every write tool gets a confirmation step. Instead of acting, the tool returns a precise description of what it would do. Your app shows that description to a person, either the customer or a staff member depending on the action, and that person must approve it. The approval, for example a button press, goes through your code and never through the model, and nothing runs until it arrives.
Marcus's second fix applies this to cancellation. The cancel tool now replies "Cancel order 10482, one burr grinder, S$289, refund to the card ending 4471. Confirm?" with a button. The order is cancelled only when the button is pressed, and his code handles the press. Lesson 5.4, "Keep a person in the loop for actions that matter", covers confirmation in more depth.
Apply least privilege: each tool reaches only what it needs, and you enforce that both in the tool's code and in the credentials it uses. An order lookup that signs in to the database as a user with read access to the orders table only cannot touch the customers table, even through a bug or a manipulated call. Least privilege limits the harm when something fails, instead of relying on nothing failing.
Marcus's third fix follows this. His order lookup can only read the orders table. His receipt sender writes only to the email address on the customer's account, regardless of any address mentioned in the chat.
Marcus's fourth fix is to log every tool call, recorded with the user it was made for. Each entry holds six fields:
the time the conversation the tool name the arguments whether the checks passed what the tool returned
With these fields you can diagnose a problem quickly. Say a customer reports that the assistant quoted the wrong delivery date. Within a minute, the log shows whether the model picked the wrong tool, sent the wrong order number, or got bad data from the courier. Without a log you can only guess at the cause. Logs also reveal patterns, such as one user trying order number after order number that belong to other people.
Logs hold personal data, so they need protecting. Lesson 8.3, "Secrets, personal data and logs", explains how to do that.
The activity below asks you to add these checks to the two tools you defined in lesson 4.2 and to write down, for each check, the mistake or attack it stops.
Add argument checks and a confirmation step to your two tools. Then write down what each check stops from happening.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).