You will train and test a small model in a free browser tool to see learning, overfitting and data gaps first hand.
Reading about training only gets you so far. In this exercise you train a real model in your browser, break it on purpose, and fix it, all in about half an hour and without writing a line of code. Everything from lessons 2.2 and 2.3 shows up on your own screen: the training loop, the held-back test, overfitting and the gaps in the data.
You need a laptop with a webcam, two small objects you can hold up to the camera, and the test log template below this lesson. Pick objects that are clearly different to you but not wildly so, such as a coffee mug and a water bottle, or an EZ-Link card and a name card.
Search for Teachable Machine, a free tool from Google that runs in the browser. Start a new image project and choose the standard image model. You will see two boxes, one for each class, which is the tool's word for the groups you want the model to tell apart. Rename them after your objects.
You do not need an account to try it, but read the tool's own notes on how your webcam images are handled before you start, and do not point the camera at anything private.
This first round is deliberately poor. Sit in one spot, with one background behind you and the same lighting. Use the webcam button to record about twenty images of the first object, held the same way each time. Do the same for the second object. Then press the train button and wait. This is the guess, measure and adjust loop running on your examples.
When training finishes, the preview panel shows the model's live guess, with a bar for how confident it is in each class. Hold up each object in the same spot you trained in. It will probably get both right, with high confidence. That is the equivalent of a student acing the practice paper.
Now give it the exam it has never seen. Test it under three conditions and note what happens in your log each time.
First, change the background. Move to another room, or hold the object in front of a different wall. Second, change the lighting: switch off the main light, or sit facing the window. Third, change the object's position. Hold it at an angle, closer to the camera, or partly covered by your hand.
Watch the confidence bars as well as the label. A model that flips between classes, or that is barely more confident in the right answer than the wrong one, has not really learned the objects. Very often you will find it has learned something you did not intend, such as the colour of the wall behind you, because in the training images the background never changed.
That is overfitting to your training set, and it is also a data gap of the kind lesson 2.3 described. The model only knew one room, so it treated the room as part of the answer.
Go back to each class and add more examples, but make them varied: different rooms, different lighting, different angles, sometimes in your hand and sometimes on the table. Train again and repeat the three tests.
Then try the tempting alternative for comparison. Add twenty more images taken in the original spot and retrain. Most people find that more of the same barely helps, while ten varied images help a lot. Write down which change fixed which error.
Here is the log from Darren, a procurement officer who ran this during a lunch break with a red mug and a clear water bottle. The results are an example of what people commonly see, and yours will differ.
Original spot, desk lamp on: both objects right, confidence above 90 percent. Different room, white wall behind him: the bottle was labelled mug about half the time. The model seemed to be using his blue cubicle partition as a clue. Lights off, screen glow only: both objects drifted towards mug, and confidence dropped close to 50 percent for each. Bottle held sideways: labelled mug. After adding 20 more images in the same spot: almost no change. After adding 10 varied images per class, in two rooms and at different angles: right in all three conditions, with lower but steadier confidence in dim light.
Darren's note at the bottom said the model had been learning his cubicle, and that he had fixed it with variety rather than volume.
Keep your first model's results before you add any images, because the comparison is the interesting part. Open the template and record your first round before you start changing anything, so you have something to compare against.
Train a two-class model, test it under three conditions, and write a short log of what it got wrong and which change to the training data fixed it.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).