You will see how an assistant handles a long document and practise a method that keeps answers accurate.
Sooner or later someone at work forwards you a hundred-page document with "can you pull out what matters for us by Friday". It might be a tender document, a supplier's annual report, or a long government guide. Uploading it to an assistant and asking questions is the obvious move. This exercise shows you how far you can trust the answers, and gives you a method that keeps them honest.
You need about thirty minutes, an assistant that accepts file uploads or long pasted text, and the scoring table below this lesson. Pick a public document for this, since lesson 8.3 comes back to what you should never upload, but the short version is that anything confidential stays out of this test.
Choose a long public document you can read yourself, ideally fifty pages or more. Good options are the annual report of a company listed on SGX, which you can download from the company's investor relations page, or a detailed guide from a government agency such as IRAS, HDB or the Ministry of Manpower.
Before you open the assistant, write five questions whose answers you can find in the document yourself, and note the page where each answer sits. Shape them like this:
Two whose answers sit in the first few pages. A third answered near the end. A fourth answered in the middle, ideally in a table or a paragraph that is easy to miss. A fifth the document does not answer at all, but that sounds as if it might.
Finding the answers first matters. If you read the assistant's reply before you know the right answer, its version tends to become yours.
Start a new chat, upload the whole document and ask your five questions one at a time. For each one, add the same request: "Quote the exact passage you relied on, and give the page number. If the document does not answer this, say so."
Paste each reply into your table straight away. Lesson 5.2 explained why long material gets uneven attention, and this step tests that on a real file. Watch the middle question and the one with no answer most closely.
Now go back to the document. For each answer, use your PDF reader's search function to look for a distinctive phrase from the quote. You will find one of three things. Either it is there, word for word, on the page given, or a close but altered version turns up, sometimes on a different page, or the phrase is nowhere in the document at all, which means the model wrote text that only looks like a quote.
Then score the answer itself against the page you found in step 1. Right means correct and complete. Partly right means correct but missing a condition, a figure or a qualification that changes how you would use it. Wrong means wrong. A confident answer to the question the document does not cover counts as wrong too.
Open a new chat. This time upload or paste only the section that contains your middle question's answer, a few pages at most. Ask the same question with the same request for a quote. Compare the result with step 2.
If you have time, do the same for any other question that scored partly right or wrong. You are testing a simple idea: does giving the model less to read, but the right part, improve the answer?
Here is one way a run can turn out. The figures are an example, and your own results will depend on the assistant and the document. Hafiz, a procurement executive at a manufacturing firm, tested an 80-page annual report from a listed logistics company that is one of his suppliers.
His two opening questions, on total revenue and the number of countries the firm operates in, came back right, with real quotes on the right pages. The question near the end, about the auditor's name, was also right. The middle question asked how many staff the company employed in Singapore, a figure sitting in a table on page 41, deep in the section on the company's workforce. The assistant gave the group's total headcount from the opening pages instead, with a quote that was real but answered a different question, so he scored it wrong.
His unanswerable question asked about plans to open a Vietnam office. The assistant replied that the company "intends to expand selectively in Southeast Asia, including Vietnam", with a quote. A search of the document found "expand selectively in Southeast Asia" but no mention of Vietnam. The model had extended a real phrase to fit his question.
In step 4 he uploaded pages 38 to 44 alone. The Singapore headcount came back right, with an exact quote from the table.
The rule Hafiz wrote for himself: for any figure from a long file, give the assistant only the section it is in, and do not use a quote I have not found with search. Your own rule may come out differently, because it should come from what your five answers show. Score them honestly, including the ones that were nearly right, since partly right is where most of the expensive mistakes hide.
Score five answers as right, partly right or wrong, record whether the quotes were real, and write one rule you will follow when working with long files.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).