You will test an assistant on image, audio and tool tasks and record where it helped and where it failed.
The last four lessons explained how assistants read images, handle audio and call tools. This exercise finds out how well yours does on the material you actually deal with at work. By the end you will have a short results table that tells you which of these tasks you can hand over with a light check and which you would be better off doing yourself.
You need about twenty-five minutes, an assistant that accepts images (and audio, if it does), and the results table below this lesson. Use material that is not confidential. A whiteboard from a brainstorm about the office party is fine, while one covered in client names is not, and lesson 8.3 explains why.
Collect three pieces of real material, each a different type.
The first is a photo of handwriting: a whiteboard after a meeting, a page of notes, or a sticky note with a list. Take it the way you normally would, on your phone, without staging it.
For the second, take a screenshot of a spreadsheet with numbers in it, at least ten rows. A household budget, a sales tracker or a roster works. If it has totals, even better.
The third is a short audio clip of a minute or two, if your assistant accepts audio. A voice note to yourself listing three or four tasks is ideal, or a short recording of a conversation where everyone has agreed to be recorded. If your assistant cannot take audio, use a photo of a printed document with small text instead, such as a receipt or a page of terms and conditions.
For each input, ask for something with a right answer you can check. Avoid vague requests like "what do you think of this?", because there is nothing to mark.
For the whiteboard, ask it to type out everything written on it, word for word, in the same order. For the spreadsheet, ask it to list every value in one column and give the total. For the audio, ask for the action items, with who does what and by when. For a printed document, ask it to type out one paragraph exactly.
Start a new chat for each input, so that one task does not colour the next. Paste each reply into your table.
Now go through each output line by line against the original. This is the slow part, and it is the point of the exercise.
Count two things for each input: how many items there were, and how many the assistant got wrong. An item is one word, one number, one action. Lesson 7.1 showed that image readers do not leave gaps when something is hard to read. They fill it with something plausible, so look hardest at the smudged corners and the small print.
Work out a rough error rate. Two wrong out of twenty numbers is one in ten. Also note the kind of error, because it matters: a misspelt name in brainstorm notes is harmless, while a wrong figure in a total is not.
Finally, give each task one of three verdicts.
Use: few or no errors, and the ones that turned up are cheap and easy to spot. A light check is enough. Check: useful, but errors turned up in places that matter. You would use it as a first draft and check every important item against the original. Avoid: so many errors, or such costly ones, that fixing the output takes longer than doing the task yourself.
Here is one example of how the table can turn out. Your own assistant and material will give different results.
Grace, a sales coordinator at an events company in Ubi, ran the exercise on three inputs. Her whiteboard photo had 34 words. The assistant got 31 right, misread one handwritten name and invented a word where the marker had faded. She gave it use, because she could fix both errors at a glance.
Her spreadsheet screenshot had 15 values in the column she asked about. The assistant read 14 correctly but turned S$1,280 into S$1,230, which made its total S$50 short. She marked it check, since a total she cannot trust needs every line compared.
Her two-minute voice note mixed English with a few Malay and Hokkien words and named three colleagues. The assistant got the tasks broadly right but gave one of them to the wrong person and changed "Thursday" to "Tuesday". Two errors in four action items was too many for something she would forward to the team, so she marked it avoid for now.
Grace's table told her something she had not expected: the task she thought hardest was the one it handled best. Fill in your own table the same way, one input at a time, and keep a note of the errors that would have cost you most.
Complete a results table for at least three inputs with what you asked, what it returned, the errors you found and a verdict of use, check or avoid.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).