Lesson 3: Training vs Testing

45–60 minutes Windows PC, Browser, Paint

Today you will…

Equipment Needed

Key Vocabulary

Training DataThe images you use to teach the model. The model sees these during training.
Testing DataImages the model has NEVER seen before. You use these to check how well it works.
AccuracyHow often the model gets the right answer. Calculated as: correct predictions ÷ total predictions × 100.
OverfittingWhen a model memorises its training data but can't handle new images it hasn't seen.

Starter Activity (5 minutes)

Think about it: Imagine you revise for a maths test by memorising the exact questions from the textbook. Then the test has completely different questions. Would you do well? Why?

Key point: Just memorising isn't the same as understanding. This is like a model that only works on images it's seen before (overfitting) vs one that truly learned the pattern.

Main Tasks

Part A: Organise Your Data

  1. Create a folder called TM_Lesson3 on your desktop. Inside, create this structure:
    • TRAIN/Circles/ — put 10 circle images here
    • TRAIN/Triangles/ — put 10 triangle images here
    • TEST/Circles/ — put 5 different circle images here
    • TEST/Triangles/ — put 5 different triangle images here
  2. Important rule: The TEST images must be completely different from the TRAIN images. Do NOT copy the same images into both folders! Draw new ones for the test set.
  3. If you don't have enough images, draw new ones in Paint now. Or use the demo dataset.

Why Separate TRAIN and TEST?

If you test your model with images it already trained on, it might just "remember" them. That doesn't tell you if it really learned. Testing with new images shows you how well the model will work in the real world.

Part B: Train and Test Your Model

  1. Open Teachable Machine and start a new Image Project.
  2. Upload only your TRAIN images: Circle TRAIN images to the Circle class, Triangle TRAIN images to the Triangle class.
  3. Train your model.
  4. Now test with your TEST images — one at a time. For each test image, record the prediction and whether it was correct.

Part C: Record Your Results & Calculate Accuracy

Test all 10 test images and record results below:

# Actual Shape Prediction Confidence Correct?
1
2
3
4
5
6
7
8
9
10

Calculate Your Accuracy

Use this formula: Accuracy = (Number Correct ÷ Total Tested) × 100

Example: If the model got 8 out of 10 correct: 8 ÷ 10 × 100 = 80% accuracy. Most students should get 80–100% with clean data.

Common Mistakes & Quick Fixes

Challenge / Stretch Task

What do you think would happen if you trained with only 3 images per class instead of 10? Try it! Compare the accuracy.

Exit Ticket

Expected answers:

  1. Training data is used to teach the model; testing data is new images the model has never seen, used to check how well it works.
  2. Because the model might have just memorised them (overfitting), and it won't tell you if it really learned the pattern.
  3. 7 ÷ 10 × 100 = 70%.

Lesson Downloads

Demo Datasets

Pre-sorted TRAIN/TEST folders

Download ZIP
← Lesson 2 Lesson 4 →