AI foundations · 02

How can examples become a rule?

By letting a computer search for the rule instead of writing it yourself. You collect examples, measure the same things about each one and, usually, write down the right answer. A learning method adjusts a model until its answers match. Then comes the part that matters: checking the rule on examples it has never seen.

Grades 6+15 minutesCounting and percentagesGrades 3–5 version below

What you will be able to do

Point to the features and the label in a table, choose classification, regression or clustering for a new problem, name four learning settings, and explain why a rule must be tested on examples that were not used to find it.

Before you start

If “machine learning” is a new phrase, read Is every AI a chatbot? first. You need to read a small table and work out a percentage.

1 · You have already seen this

The spam folder that fills itself

Some junk email lands in a separate folder before you ever open it.

A spam filter can be trained on emails that people marked as “spam” or “not spam.” From those examples it finds a rule that separates the two kinds, then uses that rule on new emails as they arrive. The filter is only worth having if the rule also works on emails it has never seen.

That is machine learning: you collect examples, write down the right answers, and let a program search for the rule instead of writing it yourself. This page uses a smaller case, mandarins and oranges from a market stall. It shows how that works, lets you try it, and shows where it goes wrong.

2 · What happens, step by step

From examples to a tested rule

Machine learning in four steps. The three sections after this one go deeper into steps 1, 2 and 3, and the lab lets you do steps 3 and 4 yourself.

How a model is made

Four steps

  1. 1 · Write down examplesEach example becomes one row in a table, with the same things measured for every row. For the fruit, these features are width and weight.
  2. 2 · Choose the answerPick the column you want to predict, the label. Usually a person has to fill it in, by looking at each fruit and writing “mandarin” or “orange.”
  3. 3 · Find the ruleA learning method adjusts the model until its answers match the labels. In the lab the rule is one cut on width, and the computer tries every cut from 5.0 to 9.0 cm and keeps the best.
  4. 4 · Test on new examplesScore the rule on examples that played no part in choosing it. A cut at 6.8 cm gets 9 of 10 training fruit right and 5 of 6 new fruit, and the 5 of 6 is the honest score.

Going deeper · step 1

Rows, features and a label

Machine learning starts with examples written down in a consistent way. A market stall measured some fruit.

Each row is one example: one fruit. Each measured column is a feature, an input the model is allowed to use. Here the features are width and weight. The column you want to predict is the label, also called the target. Someone had to supply it by looking at each fruit and writing “mandarin” or “orange.”

Machine learning means using a program to find a rule that links the features to the label. What it ends up with is the model: the rule plus any numbers it adjusted along the way. A model is useful only if it also works on fruit it has never seen, and that has to be checked, not assumed.

Six fruit from a made-up market stall
FruitWidth (cm) featureWeight (g) featureKind label
15.575mandarin
26.3110mandarin
37.2165mandarin
46.9150orange
57.9220orange
68.8305orange

Made-up measurements for teaching. Six of the ten training fruit used in the lab below.

Going deeper · step 2

A category, a number or a group?

Which column you treat as the answer decides the task. The table itself does not change.

One table, three different questions
TaskQuestion about the fruitWhat comes outNeeds a label column?
ClassificationIs this fruit a mandarin or an orange?A category (often with a probability)Yes: the kind
RegressionHow heavy is a fruit 7.5 cm wide?A numberYes: the weight
ClusteringWhich of these fruit are alike?Groups, with no names attachedNo

Classification predicts a category. Width alone does a decent job here, because the oranges in this table tend to be wider. You will test exactly that rule in the lab.

Regression predicts a number. Make weight the label and width the feature: the same column can be an input in one task and the answer in another. One simple method finds the fruit closest in width and averages their weights. For a fruit 7.5 cm wide, the two closest are fruit 3 (7.2 cm, 165 g) and fruit 5 (7.9 cm, 220 g), so the prediction is (165 + 220) ÷ 2 = 192.5 g. This method is called k-nearest neighbors, here with k = 2.

Clustering drops the label column and asks which fruit are alike. A clustering method would probably put fruit 3 and fruit 4 together, since they are the closest pair in both width and weight, even though one is a mandarin and one is an orange. Groups found this way are suggestions. A person still has to decide what, if anything, they mean.

Predict first

A shop wants to predict how many days each fruit will stay fresh. Classification, regression or clustering? What is the label?

Show the answer

Regression, because the answer is a number of days. The label is “days until it spoiled,” which someone has to record by watching earlier fruit actually go bad. Change the question to “Will it still be fresh on Friday, yes or no?” and the same data becomes a classification task.

Going deeper · step 3

Four learning settings, plus one

Supervised, unsupervised, self-supervised and reinforcement learning name where the learning signal comes from. They are learning settings, not products or brands.

Where the learning signal comes from
Learning settingWhere the feedback comes fromFruit versionA real example
SupervisedExamples paired with the right answers (labels)Learn mandarin or orange from fruit that a person labeledA spam filter trained on emails people marked as spam or not spam
UnsupervisedNo answers at all; the method looks for structure, such as groupsGroup unlabeled fruit by size and weightGrouping shoppers with similar buying habits
Self-supervisedThe data supplies its own answers: hide part of it and predict the hidden partCover part of each fruit photo and learn to fill in the missing pixelsLanguage models trained to predict the next word of real text (next word)
ReinforcementRewards or penalties after actions, learned by trial and errorA sorting robot gets +1 for each fruit in the right bin and −1 for each mistakeGame-playing programs that improve by playing many games against themselves
Semi-supervised extensionA few labeled examples plus many unlabeled ones20 labeled fruit photos plus 2,000 unlabeled onesProjects where labeling is slow or expensive, so most examples never get a label

Classification and regression with labels are supervised. Clustering is unsupervised. Self-supervised learning sounds like magic but is simple: text contains its own answers. Hide the next word, and the real text tells you what it was, so no one has to label anything. Reinforcement learning has no answer key at all, only a score for how things turned out after acting.

These settings are not rival teams. One system can use several, one after another. Many chatbot models are built in stages: self-supervised pretraining on huge amounts of text, then supervised fine-tuning on example conversations written by people, then reinforcement learning from human feedback, where people compare answers and the model is trained toward the kind they preferred. Not every chatbot is trained exactly this way, and companies do not always publish the details. The assistant training stages on the transformers page show this path.

Predict first

A friend says, “Chatbots use unsupervised learning, because nobody labeled the internet.” What would you add?

Show the answer

Half right. Nobody labeled the text, but the training still has an answer key: the next word in the real text. That is why it is called self-supervised. Most chatbots then add supervised fine-tuning and reinforcement learning from people’s feedback. One product, several learning settings.

3 · Try it

Find the rule, then test it

You play the learning method. Move the cut along one feature, width, to separate mandarins from oranges. Then see how your rule does on fruit it has never met.

Lab

Where would you draw the line?

Pick a cut and read the training score. Press “Try new fruit” to score the same rule on six fruit you did not use. Then let the computer pick the cut and compare.

Simulated in your browser with made-up fruit. No AI model runs here.

Left of the cut, the rule says M (mandarin). Right of the cut, it says O (orange). A red ring marks a fruit the rule gets wrong.

Do this on paper. Draw a line from 5 to 9 cm and mark the ten training fruit (all made up):

  • Mandarins: 5.5, 5.9, 6.3, 6.7, 7.2 cm
  • Oranges: 6.9, 7.4, 7.9, 8.4, 8.8 cm

Choose a cut: fruit at least that wide are called orange, narrower ones mandarin. Count how many training fruit your rule gets right. No cut gets all ten, because mandarin 7.2 cm is wider than orange 6.9 cm. The best cuts (6.8 to 6.9 cm, or 7.3 to 7.4 cm) get 9 of 10.

Only then uncover the six new fruit: mandarins 5.8, 6.6, 7.0 cm and oranges 7.1, 7.7, 8.2 cm. A cut at 6.8 cm gets 5 of 6 right (it calls mandarin 7.0 cm an orange). That second score is the honest one, because these fruit played no part in choosing the cut.

Worked example

Scoring the rule “6.8 cm or wider means orange”

  1. The ruleWidth 6.8 cm or more → orange. Less than 6.8 cm → mandarin.
  2. Training fruitMandarins 5.5, 5.9, 6.3 and 6.7 cm are below the cut: right. Mandarin 7.2 cm is above it and gets called orange: wrong. All five oranges (6.9 to 8.8 cm) are at or above the cut: right.
  3. Training score9 ÷ 10 = 90%
  4. New fruitMandarins 5.8 and 6.6 cm: right. Mandarin 7.0 cm gets called orange: wrong. Oranges 7.1, 7.7 and 8.2 cm: right.
  5. New-fruit score5 ÷ 6 ≈ 83%
  6. What to reportThe 83%, because those fruit had no say in choosing the cut. With only six new fruit, one mistake moves the score by 1 ÷ 6 ≈ 17 percentage points, so a bigger test set would give a steadier number.

All measurements are made up for teaching.

Choosing the cut by hand is a teaching analogy. In real machine learning, a training algorithm sets the model’s parameters automatically. Here the only parameter is the cut, and “Let the computer fit it” simply tries every cut from 5.0 to 9.0 cm and keeps the best one. A decision tree learns in a similar way: it tries cuts on each feature, keeps the one that best separates the labels, then repeats on each side. Trained on this data, it might ask about width first and weight second. Neural networks instead nudge millions or billions of parameters in small steps; see how AI works.

Notice where each set of fruit came in. The training fruit were used to choose the rule. The new fruit were kept aside and used only to check it. Testing on the training fruit would be an unfair test: a rule that simply memorized all ten would score 100% and tell you nothing about the next fruit. Moving the cut after seeing the new fruit is unfair too, because then they have quietly become training fruit. The next stop, What did we teach it by accident?, shows how to split data so the test stays honest.

Already teaching this? Train the Machine is the same idea with made-up creatures: students build a rule from 12 labeled examples, test it on creatures it has never seen, and discover that a rule can fit the training set for the wrong reason.

Honest choices

When not to use machine learning

Machine learning earns its place when the pattern is hard to write down, you have plenty of good examples, and you can test the result on new ones. Often something simpler is better.

Five jobs, five different best tools
SituationBetter choiceWhy
Pack fruit into size bands a buyer already defined (under 6 cm, 6 to 7 cm, over 7 cm)A simple ruleThe rule is known and fixed. Measuring and comparing is exact, cheap and easy to explain.
Find the shortest delivery route through ten shopsSearch and planningThere is no pattern to learn from past examples. A planner explores possible routes and checks them. Try the robot pathfinder.
Show today’s price of mandarinsLook it upThe answer is already stored. Predicting it would only add errors.
Make an important decision about a person from 30 past casesNo automation, or a person with clear guidelinesToo few examples to test fairly, and the people affected deserve a reason they can question.
Spot ripe fruit in camera photos on a sorting lineMachine learning fits wellHard to write as rules, many photos are available, and you can test on new photos.

An application: on the farming page, a sprayer’s camera classifies each plant as weed or crop, the same kind of task as mandarin or orange. A limitation: a rule learned from one field’s photos can fail in different light or soil, because the new examples no longer look like the training ones. A connection: a route planner uses search, not learning, and many real machines combine both.

4 · Common mix-ups

Four ideas that are easy to get backwards

Each one comes with what to think instead.

  • Mix-up: Machine learning means the computer understands fruit.

    Better: It found a rule that fits the examples. It has no idea what a mandarin is, and it will happily apply its width rule to a tennis ball.

  • Mix-up: A high score on the training examples proves the model works.

    Better: Only examples kept aside from training can show that. A model can memorize its training set.

  • Mix-up: Supervised, unsupervised and reinforcement learning are kinds of AI product.

    Better: They are learning settings: where the feedback comes from. One product can use several.

  • Mix-up: The label is always a category.

    Better: In regression the label is a number, such as a weight in grams.

5 · Check yourself

Three quick questions

Answer in your head or on paper before you open each one.

  1. A library table has one row per book, with the columns pages, year printed, genre and borrowed this month (yes or no). A model will predict the last column. Name one feature, the label and the task.

    Check your answer

    Features: pages, year printed or genre (any one of them). Label: borrowed this month. Task: classification, because the answer is a category, yes or no.

  2. Choose the task for each: (a) predict tomorrow’s highest temperature in °C; (b) sort plant photos into “has flowers” and “no flowers”; (c) arrange 500 unlabeled songs into groups that sound alike.

    Check your answer

    (a) Regression: the answer is a number. (b) Classification: the answer is one of two categories. (c) Clustering: there are no labels, so this is unsupervised, and someone will still have to interpret the groups.

  3. A team says its fruit sorter is “100% accurate.” They measured that on the same 200 fruit they trained it on. What is unfair about the test, and what should they do instead?

    Check your answer

    The model has already seen every one of those fruit, so it could score 100% just by memorizing them. They should keep some fruit aside before training, never use them to choose or tune the model, and report the score on those. Fruit picked on a different day or from a different farm would make the test even more convincing.

Honest limits

What this page leaves out

What this page leaves out

How training algorithms actually choose parameters (loss functions and gradient descent), how to measure errors in detail, why flexible models can overfit, and the many model families. Those come in later stops: training and inference, evaluating AI and classical models. Real fruit sorting also uses many more features than width.

For younger learners

A grades 3–5 version

No vocabulary list: real fruit, a ruler and a bag.

20 minutes · teacher-led

Ruler and a ribbon

  1. Bring about ten fruit of two kinds that differ in size, or paper cut-outs. Hide three of them in a bag.
  2. Students measure the width of the rest and place each fruit along a long ruler or a line on the floor.
  3. Put a ribbon where the class thinks “small kind” ends and “big kind” begins. Count how many fruit the ribbon gets right.
  4. Now take the fruit out of the bag one by one. Does the ribbon still work?

Exit question

Why did we keep fruit in the bag?

Good answers say that the ribbon was chosen to fit the fruit we could see, so only the hidden fruit show whether it works on new ones.

Next activity: Train the Machine (grades 3 and up), with a printable card version.