AI foundations · 06

Good at what, for whom, and compared with what?

A single score never proves that an AI system is good. You need to know which task it was tested on, on whose data, which mistakes it made, and whether it beats a simple alternative. A model can be “90% accurate” and still be useless for the job you have in mind.

Grades 7+20 minutesFractions and percentagesOptional: error and calibration

What you will be able to do

Calculate accuracy, precision and recall from a confusion matrix, compare a model with a simple baseline, and explain why a high score can still fail a real use.

Before you start

Read How can examples become a rule? (features, labels, classification) and, if you can, Training vs using. You need to be comfortable with fractions.

1 · You have already seen this

A score in a headline, with nothing to compare it to

You have probably seen an app or a news story say a model is “90% accurate.”

A number like that is easy to remember and hard to judge. Ninety percent of what, tested how, and compared with what? When the thing a model looks for is rare, even a model that never finds it can score high.

Working out whether a score means something is called evaluating AI. You ask what task it was tested on, on whose data, which mistakes it made, and whether it beats a simple alternative. The rest of this page shows how that works, lets you try it, and shows where it goes wrong.

2 · What happens, step by step

From a claim to a fair test

Testing a yes-or-no model in four steps. The next section goes deeper into steps 1 and 2, and the one after it into steps 3 and 4.

How a fair test works

Four steps

  1. 1 · Pick a baselineChoose the simplest reasonable alternative, such as always guessing the most common answer. A score only means something next to it.
  2. 2 · Test on new examplesCheck the model on examples it never saw while learning. If test examples leak into training, the score is too good to be true.
  3. 3 · Count the outcomesSort every case into four boxes: caught, false alarm, missed and correct pass. That table is called a confusion matrix.
  4. 4 · Choose what mattersDecide which mistake costs more before you pick a number. A smoke alarm should rarely miss a fire, so it cares most about recall; a spam filter that hides real email cares most about precision.

Going deeper · steps 1 and 2

A baseline and examples it has never seen

Two rules come before every metric. Without them, even a correct calculation can tell you nothing.

Compare with a baseline. A baseline is the simplest reasonable alternative: always guess the most common answer, predict that tomorrow will be like today, or use the rule people already follow. A score only means something next to it. If a weather model is right 80% of the time but “same as today” is right 78% of the time, the model adds very little.

Test on held-out data. A model can memorize its training examples, so it is tested on examples it never saw while learning, usually with the three-way split in the box. If test examples leak into training, the score is too good to be true. Data and datasets shows how leaks happen.

Then ask the three questions in the title. Good at what? Name the task and the metric. For whom? Check the people, places and conditions where it will be used. Compared with what? Name the baseline.

Three-way split

Who sees which data

  • Training set: the model learns from it.
  • Validation set: people use it to choose between versions and settings.
  • Test set: used once at the end, to estimate how the chosen model does on new examples.

If you keep checking the test set and tweaking, it slowly turns into a validation set and stops being a fair test.

Going deeper · steps 3 and 4

Count the four outcomes

When a model answers yes or no, every case lands in one of four boxes. That table is called a confusion matrix.

Predict first

A fictional sensor checks 100 cases and 10 of them are positive. A lazy “model” always answers negative. How accurate is it?

Show the answer

90%. It is right about all 90 negatives and wrong about all 10 positives. High accuracy, zero usefulness. Keep that number in mind for the worked example.

Worked example

One hundred made-up cases, one model

  1. The situationA fictional sensor checks 100 cases. In truth, 10 are positive (the thing we want to find is there) and 90 are negative.
  2. What the model didIt caught 8 positives (true positives), missed 2 (false negatives), raised 9 false alarms (false positives) and correctly passed 81 negatives (true negatives). Check: 8 + 2 + 9 + 81 = 100.
  3. AccuracyRight answers ÷ all cases = (8 + 81) ÷ 100 = 89%.
  4. PrecisionOf the 17 cases it flagged (8 + 9), how many were really positive? 8 ÷ 17 ≈ 47%. More than half of its alarms were false.
  5. RecallOf the 10 real positives, how many did it find? 8 ÷ 10 = 80%.
  6. The baselineA “model” that always says negative is right 90 times out of 100: 90% accuracy, higher than the real model. Its recall is 0 ÷ 10 = 0%. It never finds anything.

The numbers are invented for teaching. The lesson is real: when one answer is rare, accuracy rewards ignoring it.

The confusion matrix for the worked example
Really positive (10)Really negative (90)Total
Model says positive8 caught (true positives)9 false alarms (false positives)17 flagged
Model says negative2 missed (false negatives)81 correct passes (true negatives)83 not flagged
Total1090100

Accuracy divides by everything. Precision divides by what the model flagged: “when it raises an alarm, how often is it real?” Recall divides by what was really there: “of all the real cases, how many did it find?” Always say which denominator you used.

Which number matters depends on which mistake costs more. A smoke alarm should rarely miss a fire, so recall comes first and some false alarms are acceptable. A spam filter that hides real email annoys everyone, so its false alarms matter more and precision comes first. Ask which errors matter before choosing a metric, not after you see which one looks best. (The F1 score combines precision and recall into one number, but it still hides which mistake happened.)

3 · Try it

Signal or false alarm?

Most classifiers give a score, and a person chooses the threshold that turns the score into a yes or no.

Lab

Where should the leaf checker draw the line?

A made-up greenhouse camera gives 20 leaves a leaf-spot score from 0 to 100. A gardener then checked every leaf by hand. Move the threshold and watch the four outcomes, the three metrics and the baseline change.

Simulated in your browser with 20 made-up leaves. No real plants or real model.

leaf spot (found by the gardener) · healthy · Flagged leaves have a bright border. Sorted from highest score to lowest.

Three challenges

  1. Find the highest threshold that still catches all 5 sick leaves. How many false alarms come with it?
  2. Find a threshold with the highest accuracy. What is the recall there? Would a gardener be happy?
  3. Set the threshold to 100. Compare the result with the baseline. What does that tell you about accuracy?

The 20 made-up leaves, from highest score to lowest, with what the gardener found: 94 (spot), 83 (spot), 78 (healthy), 71 (spot), 66 (healthy), 57 (spot), 52 (healthy), 45 (healthy), 41 (healthy), 36 (spot), 33 (healthy), 28 (healthy), 24 (healthy), 19 (healthy), 15 (healthy), 12 (healthy), 9 (healthy), 7 (healthy), 4 (healthy), 2 (healthy).

A leaf is flagged when its score is at least the threshold. Here is what six thresholds do. The “always say healthy” baseline gets 15/20 = 75% accuracy and catches 0 of 5 sick leaves.

Signal or false alarm: six thresholds
ThresholdCaughtFalse alarmsMissedCorrect passesAccuracyPrecisionRecall
0515005/20 = 25%5/20 = 25%5/5 = 100%
355501015/20 = 75%5/10 = 50%5/5 = 100%
504311216/20 = 80%4/7 = 57%4/5 = 80%
554211317/20 = 85%4/6 = 67%4/5 = 80%
802031517/20 = 85%2/2 = 100%2/5 = 40%
1000051515/20 = 75%nothing flagged0/5 = 0%

Same idea, different stakes. The ARQIV health threshold lab uses this trade-off for screening. When the scores overlap, no line removes every mistake; moving it trades misses for false alarms. Choosing where to put it is a decision about costs, and people, not the model, should make it.

Predicting numbers

How far off, on average?

When the output is a number, such as a temperature, there is no “right or wrong,” only how far off.

For a numeric prediction, the error is the distance between the prediction and what really happened. The mean absolute error (MAE) averages those distances, ignoring whether the guess was too high or too low. It is in the same units as the prediction, so “MAE 1 °C” is easy to picture.

The made-up forecast in the table has an MAE of 1.0 °C. Is that good? Compare it with the baseline “tomorrow will be like today,” which has an MAE of 1.75 °C on the same days. Weather forecasters compare against baselines like this one; see AI in weather. A cousin of MAE, RMSE, punishes big misses more strongly.

Made-up temperatures (Sunday was 18 °C)
DayActualModelModel error“Same as today”Baseline error
Mon17 °C18 °C118 °C1
Tue20 °C22 °C217 °C3
Wed19 °C19 °C020 °C1
Thu21 °C20 °C119 °C2
MAE4 ÷ 4 = 1.07 ÷ 4 = 1.75

Beyond the average

For whom, how sure, and for how long?

A model can pass every test on average and still fail a particular group, a particular confidence level or a new season.

Subgroup checks

Fine on average, poor for some

A made-up leaf checker finds 40 of 50 sick leaves: 80% recall. Split by greenhouse, it finds 36 of 40 in the sunny one (90%) but only 4 of 10 in the shady one (40%). The average hid the problem because the shady greenhouse had fewer leaves. Always break results down by the groups that matter, especially groups of people.

Calibration

When it says 80%, is it right 80% of the time?

Many models output a confidence. A calibrated model that says “80% likely” is right about 80% of the time it says so. If 50 leaves each got “80%” and only 28 really had leaf spot (56%), the model is overconfident. A score like the one in the lab is not a probability unless someone has checked it this way.

Distribution shift

The world changes after the test

A model tested on summer photos from one camera may do worse with winter light or a new camera. When the data in use drifts away from the data it was tested on, old scores stop applying, so systems in use need checking again. The farming weed spotter asks the same question: what changes with lighting?

Chatbots and generators

When there is no single right answer

A chatbot’s answer, a summary or an image cannot be checked against one label, so evaluation becomes a set of separate checks.

Separate panel · generative systems

Five checks instead of one score

Five things to measure in a generative system
What to checkAn example testWatch out for
Factual supportFor 30 answers, does every claim match a source you can open?Fluent, confident text with no support (hallucination)
Task successDid the summary keep the five key points? Does the code pass its tests?Judging by how good it sounds instead of what it did
Safety failuresHow often does it give harmful help, or refuse a harmless request?Testing only polite, easy prompts
Latency and costSeconds per answer, and price per 1,000 answersA small quality gain that costs ten times more
ConsistencyAsk the same question five times: do the answers agree?Showing your best run instead of a typical one

Benchmarks are shared test sets that let people compare models, but a high benchmark score is not your task: the questions may differ from yours, and test questions can leak into training data. Run a small, fair test on your own task, with the same prompts for every model. Test the models shows how to do this with a class.

An AI judge gives you evidence, and it can be wrong. It is common to ask one language model to grade another’s answers. This is fast and often agrees with people, but judges have known biases: they can prefer longer answers, the answer shown first, or answers written in their own style, and they can miss the same false facts. Check a sample of judgments yourself, and verify claims with sources, as in the Verify lab.

4 · Common mix-ups

A high score can still mislead

Four ideas that trip people up, and what to think instead.

  • Mix-up: High accuracy means a good model.

    Better: Accuracy can be high just because one answer is rare. Compare it with the “always the common answer” baseline and look at precision and recall.

  • Mix-up: Precision and recall are two names for the same thing.

    Better: They divide by different groups: precision by everything the model flagged, recall by everything that was really there.

  • Mix-up: A score of 0.8 means an 80% chance.

    Better: Only if the model is calibrated. Many scores are just rankings.

  • Mix-up: If an AI judge gave it 9 out of 10, it is good.

    Better: An AI judge is one more model with its own biases. Treat its grades as evidence to check.

5 · Check yourself

Three quick questions

Answer in your head or on paper before you open each one.

  1. In the worked example, the model scores 89% accuracy and “always negative” scores 90%. Why does accuracy alone mislead here?

    Check your answer

    Because only 10 of 100 cases are positive, ignoring them all still gives 90% accuracy. Accuracy does not show that the baseline finds 0 of 10 positives while the model finds 8 of 10 (80% recall). You need recall and precision, and a decision about which mistakes matter.

  2. A fictional fruit sorter flags 20 apples as bruised. 15 of them really are bruised, and 10 bruised apples were not flagged. What are its precision and recall?

    Check your answer

    Precision = 15 ÷ 20 = 75% (of the flagged apples, three quarters were bruised). Recall = 15 ÷ (15 + 10) = 15 ÷ 25 = 60% (of the 25 bruised apples, it found 15). Same 15 on top, different denominators.

  3. A school tests a homework chatbot on 50 questions and an AI judge scores it 92%. Name two more things you would check before trusting it.

    Check your answer

    Good answers include: check a sample of the judge’s grades yourself; compare with a baseline (another tool, or a textbook); check factual claims against sources; test on questions from the subjects and grade levels your students actually have; check for harmful or wrong answers; and see whether the score holds on new questions it could not have seen.

For younger learners

A grades 4–6 version

No percentages needed: just counting four kinds of outcomes.

20 minutes · teacher-led

The ripe-fruit helper

  1. Make 12 picture cards of fruit; 3 are ripe.
  2. A pretend “helper” (the teacher) calls each card ripe or not ripe, making a few deliberate mistakes.
  3. Students sort the cards into four piles: caught, false alarm, missed, correct pass.
  4. Ask: a helper that always says “not ripe” is right 9 times out of 12. Is it a good helper?

Exit question

Which mistake is worse?

For a fruit picker, is it worse to miss a ripe fruit or to pick an unripe one? Good answers give a reason, and notice that the answer changes with the job.

Honest limits

What this page leaves out

What this page leaves out

Many metrics get no space here: F1 in detail, ROC curves and AUC, RMSE, ranking metrics, and fairness metrics that compare error rates between groups. Real evaluations also need enough test cases: with 20 leaves, one leaf moves accuracy by 5 points. Measuring a system in use (monitoring) is a topic of its own.