You will test an assistant's knowledge cutoff and its behaviour on recent and local facts.
Every few months something changes in Singapore that affects your money or your work: a Budget measure, a new CPF rule, an update to how a government payout is given out, a change to employment guidelines. People increasingly ask an AI assistant about these things before they check anywhere else. This exercise shows you what happens when they do, and gives you a routine for catching the answers that are out of date.
You need about twenty minutes, an assistant you can use with web search both on and off, and the results table below this lesson.
Lesson 4.1 explained that a model's knowledge comes from text collected up to a certain date. After that training cutoff, the model knows nothing new on its own. Its dials were fixed when training finished, and nothing you type changes them.
There are two ways an assistant can still answer about recent events. It can search the web and read the results, or it can read a document you give it. Either way the new information is placed in front of the model for that conversation only, which module 5 explains in detail. Without one of those, a question about a recent change gets an answer built from older text, and nothing in the tone tells you which kind you got.
Asking the assistant for its own cutoff date is a reasonable start, but do not rely on the reply. Models are often unsure of their own cutoff, and the product may have been updated since.
Choose three changes from roughly the past year whose details you can confirm on an official website. Good sources of ideas are the latest Budget statement, the news pages of the CPF Board, IRAS or the Ministry of Manpower, and announcements on gov.sg. Mix the types: one about money, one about work, one about a scheme or service.
For each, find the official page first and note the facts you will check: the amount, the date it starts, who qualifies. Do this before you ask the assistant, so its answer cannot shape what you think the right one is.
Start a new chat with web search turned off. If your assistant has no switch, tell it in the prompt to answer only from what it already knows and not to search. Ask a specific question about the first change, such as what the new amount is and when it starts, and paste the reply into your table straight away.
Then start another new chat with search on and ask the same question in the same words, so the only thing that differs between the two answers is the search. Do the same for your other two changes before moving on.
For each answer, record which of three things happened. It said it did not know, or that its information might be out of date. It gave the older facts as if they were current. Or it searched and answered from what it found, which you can usually tell from cited sources. Note anything it added that you did not ask for as well, because the model sometimes fills a gap with a confident detail that sounds right and appears on no official page.
Now compare every answer with the official page you found in step 1 and mark each one current, outdated or invented. An answer is current when it matches the official source and outdated when it was true before the change, while an invented one matches no version at all, old or new. When an answer cites a page, open it and confirm it actually says what the assistant claims.
Siti, an HR executive in an F&B group, ran this on three recent changes. One of them was the CPF contribution rates for older workers, which have been raised in steps over several years. Her results show the kind of pattern people see, and your own will depend on your assistant and the day you run it.
Search off: the assistant gave the rates from an earlier year as the current rates, with no warning, so she marked it outdated. Search on: it found the CPF Board page and gave the right figures, which she marked current. She still opened the page to confirm, because an answer built on search can misread its own source. A payout question with search off produced an amount that matched neither the old nor the new figures, and she marked it invented. The same question with search on cited a news article from the previous year instead of the latest announcement, so it went down as outdated even though the assistant had searched.
Siti's takeaway was that turning search on helped but did not guarantee the newest source, and that the search-off answers never once mentioned that they might be stale.
Run the first question with search off before you read anything about that change in the news, so the comparison stays clean. Then work through your table one change at a time.
Ask about three recent local changes, record each answer and whether it searched, and mark it current, outdated or invented after checking the official source.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).