Decide & Adapt · Week 2 · Judgment · calibration

How Sure Are You?

Students answer fifteen true-or-false questions with a confidence number, build a class calibration chart and find out whether their “90% sure” really means 90%.

Level 0 No AIGrades 8–1245 minutesNone needed (optional lab on student devices or one projector)

The big idea

Being sure and being right are different things. You can measure the gap, and you can shrink it.

Students will be able to:

  • Give an honest confidence number from 50% to 100%
  • Read a calibration chart
  • Explain over- and underconfidence with an example
  • Name one habit that improves their calibration

Lesson flow · the 5 P’s in decision mode

Think first. Then the tool.

Most of the lesson happens on paper. AI is optional here: it appears once, as an example of a confident tone, and the teacher can run it from one screen. How the 5 P’s work in decision mode →

1

Ponder

5 min○ Screens off · paper only
  1. Ask the class: “When people say they are 90% sure, how often do you think they are right?” Take three guesses and write them on the board.
  2. Explain the scale: 50% means a pure guess, 100% means you would bet anything on it.
Say to students“Today nobody is graded on being right. You are graded on being honest about how sure you are.”
2

Plan

12 min○ Screens off · paper only
  1. Students answer the 15 statements alone: circle True or False, then write a confidence of 50, 60, 70, 80, 90 or 100%.
  2. No talking and no devices. Remind them that 50% is a fine answer when they truly don’t know.
Say to students“If you catch yourself writing 100% a lot, ask: would I bet my lunch on this?”
3

Partner

5 min● AI on · teacher screen, optional
  1. Optional, on the teacher’s screen: ask a chatbot one of the hard statements and ask it how sure it is.
  2. Discuss: does the chatbot’s confident tone tell you its real chance of being right? (Its tone is not a probability.)
Is this statement true or false: [statement]? Then tell me how confident you are as a percentage, and what would make you less sure.
Say to students“A fluent answer sounds just as sure when it is wrong. That is true for people and for chatbots.”
4

Probe

15 min○ Screens off · paper only
  1. Read the answers. Students mark each item right or wrong.
  2. Build the class chart: for each confidence level, count hands for “answers I gave at this level” and “of those, how many I got right.” Write the percentage right for each level.
  3. Draw the dashed “perfectly calibrated” diagonal. Points below it are overconfidence; points above it are underconfidence.
Say to students“Look at the 90% column. If we were calibrated, nine in ten of those answers would be right. Were they?”
5

Prove

8 min○ Screens off · paper only
  1. Each student writes: “I was over- or underconfident on ___ because ___,” and one habit they will use next time (for example: “before I say 100%, I name how I know”).
  2. Optional homework: play two rounds of the online lab and compare.
Say to students“The goal is not to be sure. The goal is to be as sure as your evidence.”

Materials

What you need

Everything here is hand-authored and fictional unless a source is named.

Checklist

  • The 15-statement paper round (below and in the printable kit)
  • Answer key (teacher only)
  • Board or chart paper with a large calibration grid
  • Optional: the How Sure Are You? lab

Hand-authored example

The 15-statement paper round

  1. Earth is closest to the Sun in January, during winter in the Northern Hemisphere.
  2. The sky looks blue because air molecules scatter blue light more than other colors.
  3. Most of Earth’s fresh water is found in lakes and rivers.
  4. The Wright brothers’ first powered flight in December 1903 was piloted by Wilbur Wright.
  5. Measured from Earth’s center, the summit of Mount Everest is the farthest point on Earth’s surface.
  6. A light-year measures distance, not time.
  7. A day on Venus is longer than a year on Venus.
  8. A lightning bolt can heat the air to about five times hotter than the surface of the Sun.
  9. Ice floats because it is denser than liquid water.
  10. Yellowstone was the world’s first national park.
  11. The highest tides in the world are found in the Bay of Fundy in Canada.
  12. Mercury is the hottest planet because it is the closest planet to the Sun.
  13. Sunlight takes about 8 minutes to reach Earth.
  14. Scientists can now predict the date, place and size of a major earthquake ahead of time.
  15. Your eardrum turns sound into the electrical signals that travel to your brain.
Answer key (teacher)
  1. True. Earth is closest to the Sun in early January and farthest in July, so distance does not cause the seasons. Source
  2. True. Blue light travels in shorter waves, so the gases in the air scatter it in all directions more than red or yellow light. Source
  3. False. About 69% of fresh water is locked in ice caps and glaciers and about 30% is underground. Lakes and rivers hold only a tiny share. Source
  4. False. Orville flew first, on December 17, 1903, about 120 feet. Wilbur won the coin toss but his attempt three days earlier failed. Source
  5. False. Earth bulges at the equator, so Mount Chimborazo in Ecuador reaches farther from Earth’s center, over 2 km more than Everest. Source
  6. True. A light-year is the distance light travels in one year, about 9 trillion kilometers. Source
  7. True. Venus spins so slowly that one day lasts 243 Earth days, while one trip around the Sun takes only 225 Earth days. Source
  8. True. Lightning can heat the air it passes through to about 50,000 °F, roughly five times hotter than the Sun’s surface. Source
  9. False. Ice floats because it is less dense than liquid water. Its molecules lock into a spread-out pattern. Source
  10. True. Yellowstone was established as the world’s first national park in 1872. Source
  11. True. Canada’s Bay of Fundy, between New Brunswick and Nova Scotia, has the highest tides in the world. Source
  12. False. Venus is the hottest planet. Its thick atmosphere traps heat, while Mercury has almost no atmosphere and gets very cold at night. Source
  13. True. Light from the Sun takes about 8.3 minutes to reach us, so we always see the Sun as it was a few minutes ago. Source
  14. False. No scientist has ever predicted a major earthquake. They can only estimate the chance of one in an area over many years. Source
  15. False. The eardrum vibrates and passes sound to three tiny bones. Hair cells in the cochlea, in the inner ear, make the electrical signals. Source

Printable

Student worksheet

Print one per student. Everything students hand in is handwritten, so their thinking is visible.

How Sure Are You?

Decide & Adapt · Week 2. Write your own thinking; it is what gets graded.

Name:Date:Class:
1 · Fifteen statements: circle T or F, then write your confidence (50–100%) for each. (Use the paper round printed with this lesson.)
1.  T  /  F   ____%2.  T  /  F   ____%3.  T  /  F   ____%4.  T  /  F   ____%5.  T  /  F   ____%6.  T  /  F   ____%7.  T  /  F   ____%8.  T  /  F   ____%9.  T  /  F   ____%10.  T  /  F   ____%11.  T  /  F   ____%12.  T  /  F   ____%13.  T  /  F   ____%14.  T  /  F   ____%15.  T  /  F   ____%
2 · How many did I get right at 90–100%? At 50–60%?
3 · I was over- or underconfident on ___ because ___
4 · One habit I’ll use next time I feel sure

Assessment

Grade the process, not the answer

Every row scores something students did themselves. A lucky answer with no reasoning scores low; a careful decision that turned out wrong can score high.

Criteria4 · Excellent3 · Solid2 · Developing1 · Beginning
Honest confidenceUses the whole scale, with 50% where truly unsureUses at least three levelsMostly one levelNo confidence numbers
Reading the chartExplains over- and underconfidence using the class chart and their own answersExplains one of them correctlyReads values without interpretingCannot read the chart
ReflectionNames a specific item and the reason their confidence was offNames a specific itemGeneral statement onlyMissing
Next-time habitA concrete, checkable habit tied to their mistakeA concrete habitA vague intentionMissing

Why this lesson

Documented · adult forecasters

A probability and debiasing training module lasting under an hour improved forecasters' accuracy (Brier scores) by 6 to 11% over a control group, across four years of forecasting tournaments with adult volunteers.

The training study involved adult forecasters, not school students. We expect the habit to transfer, but that has not been tested in classrooms.

Adjust it

Younger students: Use 10 statements and three confidence levels only: “guessing,” “fairly sure,” “certain.”

Older students: Compute a Brier score by hand for five items (see the lab’s explanation) and compare it with a coin-flip score of 0.25.

No devices at all? Already device-free. Skip the Partner step, or read this aloud: a chatbot answered a hard statement correctly once and wrongly the next time, in the same confident tone both times. What does that tell you?

The AI step

Level 0 No AI

Optional and short. The lesson itself needs no AI. If you use this step, ask a chatbot one hard statement on the teacher’s screen, to show that a confident tone is not a probability. Prompt:

Is this statement true or false: [statement]? Then tell me how confident you are as a percentage, and what would make you less sure.

Exit ticket

Finish the sentence: “When I feel 100% sure, I should…”

Why this is hard to cheat

The score that matters is the match between confidence and correctness, so looking answers up would only hide what students need to see. Everything is on paper, in class.

Go further

How Sure Are You? lab →

Honest limits and sources

Where this comes from

Open the source before quoting it.

Honest limits

What this page leaves out

This is an initial edition, written by ARQIV in October 2026. It has not been classroom-piloted or independently reviewed, so its scenarios and timings may change after classroom use. The evidence named above supports the core move of the lesson, not this lesson as a whole. We do not claim the lesson measures or certifies a student’s judgment or “learning agility.”

Review status

Written by ARQIV. Initial edition, October 2026. This lesson has not been classroom-piloted or independently reviewed yet, and every question and scenario is hand-authored and checked against its sources by ARQIV. Tell us about anything wrong or unclear.

  1. Welton Chang, Eva Chen, Barbara Mellers and Philip Tetlock, "Developing expert political judgment," Judgment and Decision Making 11(5), 2016, pp. 509–526.
  2. ARQIV, How Sure Are You? question bank (each statement links its own source in the answer key).

All links opened and checked October 21, 2026 (accessed October 2026).