You will build a small script that reads sample emails and returns validated JSON for each one.
Every week, customers email Priya's team to ask for a change to a delivery (Priya and her team are an invented example). One of her colleagues opens each message and copies the order number, the new date, the new address and any special instructions into the job system by hand.
That copying is slow, dull work. Pulling fixed fields out of loose text into a set structure is what a schema-based extractor is built for. You'll build one that turns emails into JSON, either for Priya's delivery changes or for a kind of email that lands in your own inbox, and then check every result yourself. Set aside about 35 minutes. You need an AI coding assistant and the script you got working in lesson 1.4, "Send your first request and read the response".
Create a folder named "samples" and put 10 emails into it, each saved as a plain text file. Real emails make the best test material, as long as you change every name, phone number and address to an invented one first. A practice folder should hold no personal data.
What counts is how different the emails are from each other, more than how many you have. Make sure at least one is written in another language and at least one contains nothing worth extracting. These two kinds trip up most extractors, and it is better to learn how yours handles them now than when a customer's email arrives.
Priya's set (an invented example) has 4 routine change requests. The rest each cause a different problem: 1 that covers two orders, 1 that gives the new date as "next Thursday", 1 that asks for "same address as last time", 1 written in Chinese, 1 forwarded thread where the actual request sits beneath 3 replies, and 1 that says only "Thanks, received". Together that makes 10.
Settle the schema before you ask the assistant for a single line of code. Priya's has 5 fields:
order_number: string, or null if none is given new_date: string in YYYY-MM-DD format, or null if the email does not give one clearly new_address: string, or null request_type: enum with values change_date, change_address, change_both, cancel, no_request notes: short string for anything else the person handling the email should know
Two choices in it are deliberate. First, each field can be null. When an email gives no order number, the model can say so with null instead of making one up. Second, request_type takes 5 values, and no_request is one of them, so a reply such as "Thanks, received" still has a valid category. Lesson 2.2, "Ask for JSON and enforce it with a schema", suggested including that value.
Dates written in relative terms are a common source of mistakes, since the model has no idea what today's date is unless you supply it. Give each email's received date in the system prompt, and instruct the model to return null when the date isn't clear instead of guessing. Priya's system prompt does both: it carries the received date for each email and the instruction to return null.
Hand the assistant your schema along with the structure you want. The script should go through every file in samples and send each one, together with the system prompt and schema, using your provider's structured output mode. It should validate each reply against the schema and check the stop reason. If a call fails, it tries once more, passing the validation error back with the second attempt. Each email's result goes to an output file, and every call adds one line to a log: the file name, whether validation passed, whether there was a retry, and how many tokens were used.
Before running anything, read the code the assistant wrote. Confirm three things: the API key is read from an environment variable, there is exactly one retry with no loop around it, and if one email fails the script moves on to the next. Then run it on all ten samples.
Open the log first, before any output. Count the calls that failed validation on the first attempt and the ones that recovered on retry, and look for any call that hit the output limit. In Priya's first run, 1 of 10 calls failed validation, on the forwarded thread, and the retry recovered it.
Next, put each email next to its JSON and give it one of three marks. Correct means every field matches what a careful person would have typed. Wrong means the JSON is valid but a value in it is wrong. Failed means the extractor gave up and passed the email to the fallback.
A response can be valid JSON and still hold the wrong value, which lesson 2.2 warned about. For now, reading each one by hand is the only way to find those, and it's also where you'll learn the most. Priya's run produced valid JSON for all 10 emails, yet 2 were wrong. The "next Thursday" email came back with a date one week too late, and the Chinese email had the new address without the unit number.
Make a results table with one row per email showing its mark, and beside each wrong row write a note on what went wrong. When you finish, you'll have the samples folder holding ten emails, the script, one JSON output for each email, a log covering every call and retry, and the results table with its notes on the wrong rows.
Leave the problems unfixed for now. Your short list of the ways the extractor fails is worth more at this stage than a patched prompt, because module 7 turns these exact cases into tests.
Run your extractor on ten sample emails, save the JSON it returns, and mark each one correct, wrong or failed.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).