Search answers, research reports and plain recall

You will be able to choose the right kind of AI research for a question and know what each one can get wrong.

Before you ask an AI assistant a research question, decide which kind of research you want it to do: plain recall, search, or a research mode. Whether the answer is right depends on which one you used. A well-written answer is not necessarily a correct one.

Most people never notice that there is more than one kind. Take Kavitha, an invented example, who works in business development for a company that runs corporate wellness programmes. Her director tends to ask questions on Friday afternoons. Examples are how competitors are pricing now, how many companies of the target size there are in Singapore, and whether anything has changed in the rules on workplace health. She used to spend Monday morning on searches and phone calls to answer them. Now she types the question into an assistant and gets an answer in seconds. The answers read well, but how accurate they are depends on which kind of AI research she used.

Plain recall and the training cutoff

With plain recall, the assistant answers without searching, using patterns it learned in training. AI fundamentals lesson 4.4, "Find the edges of what a model knows", explains how this works. Plain recall has two limits that matter for research at work.

The first limit is the training cutoff date, which is the date the training data stops. The model cannot see anything that happened or changed after that date, and it does not always say so. If you ask about a rule that changed last year, you may get a confident description of the old rule.

The second limit is that plain recall gives no sources you can check. The model can produce text that looks like a citation, such as a report title or a statistic with a year attached. It generates these the same way as the rest of its text, so they are not real sources. Specific figures, names and references are the parts most likely to be made up. AI fundamentals lesson 6.2, "Where invented answers are most likely", explains why.

Use plain recall to explain a concept, to suggest what to look for, or to draft a first list of questions. Don't use it for facts you will repeat to your director.

Search and the pages behind the answer

Most assistants can now search the web. Some search when you switch it on, and others search when they judge that a question needs it. The assistant runs searches, retrieves some pages, writes an answer based on what it found, and usually includes links. AI fundamentals lesson 5.3, "Retrieval: how an assistant looks things up", describes the process.

Search fixes two problems with plain recall: the information can be current, and you get links you can open. It also means the answer depends on the pages the search found, and it is only as good as those pages. Say the top results are a two-year-old blog post and a vendor's marketing page. The summary will reflect them faithfully, including the outdated claims and the sales angle. The model can also misread a page, mix up two sources, or attach a link to a claim the page does not make. When you use search, look at which pages the answer drew on before you rely on it.

Research modes and long reports

Many assistants now offer a longer research mode, and different assistants give it different names. It plans a set of questions, reads many sources, and writes a structured report with citations. It takes minutes rather than seconds, sometimes longer.

A research mode helps you get oriented on an unfamiliar topic quickly, and it often surfaces sources you would not have found yourself. A long, polished report is more persuasive, but length and polish do nothing for accuracy, so don't count them as evidence that the report is right. A report with forty citations is harder to check than a paragraph with three. Errors inside it are easier to miss because everything around them looks thorough.

Kavitha's test across all three

Kavitha asked the same question about workplace health rules in all three modes. Plain recall gave a neat answer with no sources, and one detail later turned out to be out of date. Search gave three links. Two of them were from a consultancy's blog and one was from the Ministry of Manpower (MOM) website, the Singapore government source for workplace rules. The research mode gave a long report citing more than twenty sources, including the MOM page and several news articles. It also contained one figure she could not find in any of those sources.

No mode removes the need to check. All three can misquote a source, describe an outdated rule as current, or invent a detail that sounds right. The modes differ in how you check and how much.

Matching the checking to the stakes

Decide how the answer will be used, and scale the checking to match. Suppose Kavitha wants a quick sense of what wellness programmes typically include before a brainstorm. A search answer skimmed for plausibility is enough for that. If her director will quote a figure to a client, she needs to trace every number to the page it came from and confirm it there. Lesson 4.2 gives the method for doing that.

Try this with a question of your own. Pick something about a Singapore market or regulation that matters in your job, and ask it in each mode your assistant offers so you can compare where the answers came from.

Ask the same question about a Singapore market or regulation in plain mode, with search and in a research mode if available, and compare the sources each used.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).