Run a short check on any AI answer before you use it, the same way every time.
Open the most recent AI answer you used or sent to someone, and underline each number, name, date, quote and rule in it. You have just done the first of four moves in a check that takes a few minutes, works on any AI answer and needs no special software. Together the four moves catch the mistakes that would cost you.
Most people react to a smooth AI answer in one of two ways. They trust it because it sounds right, or they feel uneasy and send it on anyway. Both leave any error in place, and the four moves give you something better to do with the unease.
Take Wei Jie (an invented example), who does freelance design jobs on the side. He wants to know whether he must register for GST, so he asks an AI assistant. The reply is clear, friendly and about 200 words long. It gives a turnover figure, a time period and a rule on when registration becomes compulsory, and it ends with a link to an IRAS page. Because it reads smoothly, he is about to send it to an accountant friend with the note "looks like I'm fine".
A checkable claim is any statement that could be shown right or wrong. Most are numbers, names, dates, quotes and rules. In an answer like Wei Jie's they often take the form of a rule with a number attached ("must register if turnover exceeds..."), a rule about a date ("applies from the start of the following quarter"), or a claim about what a named body says ("IRAS recommends").
On the first pass, skip the general explanation and go to the specific details. Lesson 6.2 showed that invented content tends to appear in those details. An answer of about 200 words usually holds somewhere between 3 and 10 checkable claims. Wei Jie underlines five.
Once the claims are underlined, the answer reads as a set of separate claims instead of one solid block. Each claim can be right or wrong independently of the others, so you can check them one at a time.
Claims do not all need equal care. For each one, ask whether you will act on it and whether an error in it would cost you. Wei Jie ranks the turnover figure and the compulsory-registration rule highest. An error about filling in the form ranks lower, because he will see the real form when the time comes.
Check the important claims against a primary source, meaning the organisation that makes the rule or the original document. In Singapore that is usually an agency's own website, such as IRAS, CPF Board or the Ministry of Manpower, or the wording of the law on Singapore Statutes Online. If a figure comes from a study, read the study itself, and skip the article about it. For a company's results, read the report the company published.
News stories, blogs, forum threads and other AI answers are each a secondary source. They may be correct, but each is one step further from the truth and may be out of date.
For every link or source the answer names, open it and check three things:
The page exists. A model can produce a URL that looks like a genuine agency address but leads nowhere. It is the right sort of page: official, current, and about Singapore if your question was about Singapore. It says what the answer claims. Use your browser's find function to search the page for the figure or phrase.
A link to a genuine page can still hide a problem. Wei Jie's link leads to a real IRAS page on GST registration, and that page's turnover rule carries a condition the assistant left out. A glance at the citation would not have shown him the condition. He found it by reading the page.
The two New York lawyers in lesson 6.1 did not open their sources, and lesson 6.1 calls that the whole difference in their case.
Ask the assistant: "Which parts of your answer are you least sure about, and why?" This takes about ten seconds. The reply often flags something useful, such as a figure that changes every year or a rule with exceptions.
Treat what it says as leads to follow up. As lesson 6.1 explained, a model produces everything by prediction, including its view of its own uncertainty. It may express doubt about parts that are correct and stay silent about parts it invented. The rule for this move: anything it flags goes onto your checking list, and nothing comes off the list because of what it says.
For small jobs, such as rewording an email, you can skip most of the check. Run the full check on anything you will act on, sign, send to a client, or hand to someone who trusts you. In those cases the check usually takes less time than fixing the mistake afterwards.
The routine only works if you use it without having to think about it. The four moves above are one version. Write yours in your own words, short enough to remember, and put it where you will actually see it the next time an answer looks good enough to forward.
Write the routine as a five-line checklist in your own words. Keep it somewhere you will see it the next time you use an assistant.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).