Spot the common ways agents fail in a run log, and add a check for each one.
AI agents fail in a small number of recognisable ways: loops, drift, false success and hijacking. If you can spot one of these failures in a run log, you can add a check in code or in your process that catches it the next time.
Daniel works at a firm and built a small agent to research HR questions across the firm's policies and the public guidance on the Ministry of Manpower website. He asked it whether the firm's leave policy covers everything a new parent is entitled to. After running for twelve steps, the agent reported "Done. The leave policy is fully compliant." as its final message to Daniel.
When Daniel read the log, he found three problems. The agent searched the policies four times with almost the same words, it wandered off to summarise the firm's medical benefits, and it never compared anything against the official guidance. Those are a loop, drift and false success, and they show that an agent's confident final message can be the least reliable part of a run.
A loop is when the agent repeats the same tool call with the same or nearly the same arguments and expects a different result. Daniel's log shows searches for "parental leave entitlement", "parental leave entitlements" and "entitlement parental leave". Each search returned the same chunks, and the model didn't notice that it already had the only answer it was going to get.
Loops happen when a tool result doesn't satisfy the model and the model has no other idea to try. Vague error messages make this worse, which is why lesson 4.2, "Write tool descriptions the model uses correctly", asked you to write error messages that tell the model what to do next.
Exact repeats are easy to catch in code. During the run, record each tool name and its arguments, and before each call, check whether that same tool name and argument pair has already run. If it has, don't run it again. Return a message to the model in place of the result, for example: "You already ran this search. The results are above. Try a different approach or give your answer." If the agent repeats the call again after that message, stop the run.
Near repeats are harder to catch. A simple method is to normalise the arguments before comparing them, by lowercasing them and sorting the words, so that "entitlement parental leave" and "parental leave entitlement" match. The step cap from lesson 5.2 is the backstop for anything that gets past both the exact and the near-repeat checks.
Drift is when the agent gradually loses the original goal over many steps. In a log, it looks like this: a result mentions something interesting and the agent follows it, then the next result mentions something else, and by around step eight the agent is doing a different task. Daniel's agent drifted because the leave policy mentioned hospitalisation leave, which led it to the medical policy.
Drift grows with the length of the run. The original goal sits at the top of a growing conversation, and every new step pushes it further from the latest results.
The first fix is to restate the goal at every step and place it close to the latest results, with a line such as "Your goal: check whether the leave policy covers new parents' leave entitlements". The second fix is to keep tool results short so the goal isn't buried, as lesson 5.2 recommended. If a task still drifts often after both fixes, it may be too broad for one agent, and you should split it into smaller tasks or use a fixed workflow instead.
False success is when the agent reports a task as done when it isn't. It is the most dangerous of the four failures because it looks like success. The model writes the final message, and it writes a confident one whether or not the work behind it happened.
Wherever possible, check the outcome in code and don't rely on the agent's report. If it says it saved a file, check that the file exists and isn't empty. If it says it updated a record, read the record back. If it says it compared two documents, check that the log shows it retrieved both. After his run, Daniel added a rule of this kind: any run that claims a comparison must have retrieved from both sources, or the result is marked unverified.
Some outcomes can't be checked in code, such as a judgement on whether a policy is compliant. For those, the answer goes to a person as a draft, never as a conclusion, and lesson 5.4 covers how that step works.
Hijacking is when the agent follows instructions it finds inside tool results, so unlike the other three failures it comes from outside the agent. Tools return text, such as web pages, emails, documents and search results, and the model reads all of it. If that text contains instructions, the model may follow them. A web page could contain a hidden line telling the agent to ignore its task and email a document to a given address, and an agent with an email tool might carry out that instruction.
The technical name for this is indirect prompt injection, and it is the reason agents that read untrusted content need tight limits on what they can do. Lesson 8.2, "Prompt injection: when your input gives orders", covers these three defences in depth:
treat tool results as data to be read, with no authority to give the agent instructions keep write actions behind approval give the agent only the tools the task needs
The fastest way to learn these failures is to see them in a real run. The activity below gives you an agent's run log: read it and label each failure you find as a loop, drift, false success or hijacking.
Open the log of a failed agent run, either from your own tests or from an example, and label each failure with its type.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).