Ask for a critique, and know its limits

Get a useful critique from an assistant and know what its self-checking will miss.

Take the last piece of work you asked an assistant to review and find the chat where you asked for the review. If it's the same chat you used to write the draft, the feedback was probably kinder than it should have been. A critique from an assistant only helps when you word the request well and you know what it can't spot.

Ravi's loyalty app proposal

Ravi works for a cafe group and wanted management to replace its paper stamp cards with a loyalty app. He wrote a one-page proposal with an assistant's help over one afternoon. When he finished, still in that chat, he typed "Is this good?" The assistant liked the structure, said the argument was persuasive and suggested adding a call to action. He sent the proposal.

At the meeting, the finance director asked what the app would cost to run each year. Ravi's page gave a one-off development cost. It said nothing about the annual running cost, and the assistant's feedback hadn't pointed out the gap either.

Later he copied the proposal into a fresh chat and asked three pointed questions. The reply made several points. One was that the page gives the development cost but leaves out yearly costs such as hosting, maintenance and app store fees, and that anyone reading it with a finance eye will want that figure. That was the same question he'd been asked in the meeting.

Asking three pointed questions

A vague "Is this good?" gets a polite reply. Assistants tend to say what they think you want to hear. AI fundamentals lesson 4.3, "Human feedback and why assistants sound so agreeable", explains why. If you ask broadly for feedback, you usually get general praise and a couple of small tweaks, which is what happened to Ravi. Narrow questions get narrow, usable answers. These three work for almost anything you write:

What are the three weakest points, and why? Which claims are most likely to be wrong or need checking? What would a sceptical reader object to?

Asking for the three weakest points makes the model rank the problems, so it can't report that everything is fine. The question about claims turns its attention to facts and away from style. The third question asks it to read the piece the way your actual audience would. That is the question that would have caught the running cost Ravi left out.

A sceptical reader is the person who actually has to be convinced, and you ask the model to take their point of view. The question gets much sharper when you say who that person is. A finance director, a cautious client and a new customer each care about different things, so each would push on a different part of the page. Put the reader into the wording: "What would a finance director who has to approve the budget object to?"

Why the chat you use matters

A fresh chat is one that has none of the conversation that produced your draft. It sees only the draft, which is also all your reader will see. The chat you wrote in remembers every reason you gave for every choice. Because those reasons are still in view, it tends to accept them. The text it's judging also grew out of its own earlier replies, and that pulls it towards defending the draft as it stands.

The steps are: start a new chat, paste in the finished work and add one line saying who it's for. You can also remove anything that shows how much effort went into it, which gets you closer to how a cold reader would react. Then ask the three questions, naming the reader in the last one.

Facts a critique can't verify

When a model reviews the kind of output it produces, it has the same blind spots it had when writing. Say the draft includes a statistic on loyalty app adoption that the model made up, stated with confidence, and that sounds believable. When the same model reads it back later, it will probably find it believable too. Nothing in an assistant checks claims against a source, as AI fundamentals lesson 6.1, "Why a fluent answer can still be false", explains, and asking for a critique doesn't add that step.

Critique is good at catching gaps in reasoning, missing sections, muddled structure, weak arguments and questions a reader is likely to raise. It is much less reliable on factual mistakes, especially specific ones that sound right. The second question is useful because it gives you a list of claims worth checking. You still have to do the checking yourself, using the methods in lesson 7.3, "Checking numbers, quotes, sources and code".

If the work matters, you can get a second opinion cheaply and quickly: paste the same draft into a different assistant and ask the same three questions. Models are trained in different ways and have different weak spots, so the second one sometimes catches what the first missed. If both say a claim is right, treat that as weak evidence. They could have learnt the same wrong fact from similar sources, or both could fill a gap with the same plausible guess. One check against a primary source counts for more than two models agreeing. Lesson 7.1, "Match the check to what is at stake", says agreement between models should never replace that check when the stakes are high.

Think of a critique as a reader who never gets tired and never gets offended, but who can't look anything up for you. To see what that reader is worth on your own work, take a draft you have finished recently, ideally one you produced with an assistant, and open a fresh chat to put it in.

Paste one of your drafts into a fresh chat and ask the three critique questions. Note which points helped and what it missed.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).