Run a few small tests that show token-by-token generation and sampling at work in a real assistant.
Marcus is an admin executive at a clinic in Tampines. He ran four short tests in the AI assistant he uses at work, and his results appear below as an example only.
You have now met three mechanisms. Text is cut into tokens (Lesson 3.1), answers are built one predicted token at a time (Lesson 3.2), and sampling makes each attempt slightly different (Lesson 3.3). The four tests show each one at work in whichever assistant you normally use. They take about twenty-five minutes, and you need only the assistant and the results table below. The habit to build is simple: when an assistant does something odd, ask which part of the machinery caused it instead of calling it random.
Open the results table. For every test, follow the same four rules:
start a new chat, so earlier messages do not affect the answer if the assistant has a memory or personalisation feature, note whether it is on copy each prompt exactly write down what came back before you try anything else
Results differ by assistant and by day, so yours will not match Marcus's exactly.
Choose a long word with a repeated letter, such as "accommodation" or "Mississippi". Count the target letter on paper first so you know the right answer. Then ask the assistant how many times that letter appears, for example how many m's are in "accommodation". Record its answer and whether it is correct.
Next, open a new chat and ask the assistant to spell the word one letter per line and only then count the letter. Put the two results side by side in your table.
Marcus asked how many m's are in "accommodation" and got 2, which is correct. He asked how many s's are in "Mississippi" and got 3, which is wrong. When the assistant spelled "Mississippi" out letter by letter first, it counted 4, which is correct. He tagged the result as tokens. In the first attempt the letters sit hidden inside chunks. Spelling the word out turns each letter into its own token that the model can see.
Newer assistants often get both attempts right, sometimes because they quietly run a small program to do the counting. If yours appears to do that, note it in the table.
Ask for ten name ideas for something specific, for example a home-based bakery in Punggol that sells kueh. Copy the list somewhere safe. Open two more fresh chats and ask exactly the same prompt in each. With the three lists in front of you, count how many names appear in more than one list.
Marcus got 30 names across his three lists. 4 appeared twice and none appeared three times. He tagged it as sampling, since each attempt is a separate draw from the same probabilities. Some overlap is normal because the most likely names come up again and again, but you should also find plenty of differences between the lists.
Pick a local event from the last week or two whose facts you know, such as a new hawker centre opening, a transport announcement or a result in a sport you follow. Ask the assistant a specific question about it, and keep the answer out of your prompt. Record which of three outcomes you got: (a) it answered from its own knowledge, which may be confidently wrong or out of date; (b) it said it does not know or cannot see recent events; or (c) it searched the web and cited sources.
Marcus asked about a hawker centre that had opened ten days earlier. With search on, the assistant found a news article and gave the correct opening date. In a second chat he asked it not to search, and it described a different hawker centre with a similar name. He tagged it as prediction, with search filling the gap.
Without search, the model can only continue the text with what is likely given its training. An event that happened after its training data was collected was never in that data, so the best the model can produce is an answer that sounds likely. Module 4, lesson 4.4, "Find the edges of what a model knows", looks directly at this limit.
Marcus remembered this test most, because the made-up answer was written in exactly the same calm tone as the correct one.
Go back through your table. Next to each test, write which mechanism explains what you saw: tokens, prediction or sampling. Some results have more than one cause, as Marcus found with prediction and search in Test 3, and you can write down both when that happens.
Open the results table now and run the first test before reading any further, so that what you have read here does not shape how you word the prompt.
Run the four tests. Fill in a table with the prompt, what happened and the mechanism that explains it: tokens, prediction or sampling.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).