Train and test a small model in a free browser tool, and see learning, overfitting and data gaps for yourself.
Reading about how a model is trained only gets you so far. You can describe the training loop and the held-back test in words, but they are easier to understand once you have watched a model learn the wrong thing on your own screen. You need no code for this, and it takes about half an hour. You will see four ideas from lessons 2.2 and 2.3 in practice: the training loop, the held-back test, overfitting, and gaps in the data.
You need a laptop with a webcam, two small objects you can hold up to the camera, and the test log template provided below the lesson. Choose two objects that are clearly different from each other, but not wildly different. A coffee mug and a water bottle work well, and so do an EZ-Link card and a name card.
The tool is Teachable Machine, a free browser-based tool from Google, and you don't need an account to try it. Search for Teachable Machine, start a new image project and choose the standard image model. The screen shows two boxes, one for each class, which is the tool's word for the groups you want the model to tell apart. Rename each class after one of your objects. Before you record anything, read the tool's own notes on how webcam images are handled, and keep the camera away from anything private.
The first round is meant to be poor. Stay in one spot, with one background and the same lighting throughout. Use the webcam button to record about twenty images of the first object, holding it the same way each time, then do the same for the second object.
Press train and wait. While you wait, the tool is running the guess, measure and adjust loop on your examples. When it finishes, the preview panel shows the model's live guess, with a confidence bar for each class. Hold up each object in the same spot and check the label and the bars. The model will probably get both objects right with high confidence, the way a student can ace the practice paper.
Next, give the model the exam it has never seen. Run three tests and note each result in your log:
Background: move to another room, or use a different wall. Lighting: switch off the main light, or face the window. Position: hold the object at an angle, closer to the camera, or partly covered by your hand.
Watch the confidence bars as well as the label. If the model flips between classes, or is only slightly more confident in the right answer than the wrong one, it has not really learned your objects. Often it has learned something you never meant to teach it, such as the colour of the wall behind you, because the background never changed in the training images. This is overfitting to your training set: the model learned the training conditions instead of the objects, so it fails when those conditions change. It is also a data gap of the kind described in lesson 2.3. The model only knew one room, so it treated the room as part of the answer.
Darren, a procurement officer, ran the exercise during a lunch break with a red mug and a clear water bottle. In his original spot, with the desk lamp on, he got both objects right with confidence above 90 percent. In a different room with a white wall behind him, the bottle was labelled mug about half the time, and the model seemed to be using his blue cubicle partition as a clue. With the lights off and only the screen glow, both objects drifted towards mug, and confidence dropped close to 50 percent for each. When he held the bottle sideways, it was labelled mug.
Before you change anything, write the first round's results into the template. Once you start adding images you can't get the first model back.
Then add varied examples to each class: different rooms, different lighting, different angles, sometimes held in your hand and sometimes on the table. Retrain and repeat the three tests. For comparison, also add twenty more images taken in the original spot and retrain. Adding more images from the same spot is the tempting fix, but most people find it barely helps, while a small number of varied images, around ten, helps a lot. Write down which change fixed which error.
Darren's results are an example of what people commonly see, and yours will differ. When he added 20 more images in the same spot, there was almost no change. When he added 10 varied images per class, taken in two rooms and at different angles, the model got both objects right in all three conditions, with lower but steadier confidence in dim light. At the bottom of his log he noted that the model had been learning his cubicle, and that he had fixed it with variety rather than volume.
Keep your first model's results before you add any images, because the comparison is the interesting part. Open the template and record your first round before you start changing anything, so you have something to compare against.
Train a two-class model and test it under three conditions. Write a short log of what it got wrong and which change to the training data fixed it.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).