Decide & Adapt · Week 2 · Judgment · calibration
How Sure Are You?
Students answer fifteen true-or-false questions with a confidence number, build a class calibration chart and find out whether their “90% sure” really means 90%.
The big idea
Being sure and being right are different things. You can measure the gap, and you can shrink it.
Students will be able to:
- Give an honest confidence number from 50% to 100%
- Read a calibration chart
- Explain over- and underconfidence with an example
- Name one habit that improves their calibration
Lesson flow · the 5 P’s in decision mode
Think first. Then the tool.
Most of the lesson happens on paper. AI is optional here: it appears once, as an example of a confident tone, and the teacher can run it from one screen. How the 5 P’s work in decision mode →
Ponder
5 min○ Screens off · paper only- Ask the class: “When people say they are 90% sure, how often do you think they are right?” Take three guesses and write them on the board.
- Explain the scale: 50% means a pure guess, 100% means you would bet anything on it.
Plan
12 min○ Screens off · paper only- Students answer the 15 statements alone: circle True or False, then write a confidence of 50, 60, 70, 80, 90 or 100%.
- No talking and no devices. Remind them that 50% is a fine answer when they truly don’t know.
Partner
5 min● AI on · teacher screen, optional- Optional, on the teacher’s screen: ask a chatbot one of the hard statements and ask it how sure it is.
- Discuss: does the chatbot’s confident tone tell you its real chance of being right? (Its tone is not a probability.)
Probe
15 min○ Screens off · paper only- Read the answers. Students mark each item right or wrong.
- Build the class chart: for each confidence level, count hands for “answers I gave at this level” and “of those, how many I got right.” Write the percentage right for each level.
- Draw the dashed “perfectly calibrated” diagonal. Points below it are overconfidence; points above it are underconfidence.
Prove
8 min○ Screens off · paper only- Each student writes: “I was over- or underconfident on ___ because ___,” and one habit they will use next time (for example: “before I say 100%, I name how I know”).
- Optional homework: play two rounds of the online lab and compare.
Materials
What you need
Everything here is hand-authored and fictional unless a source is named.
Checklist
- The 15-statement paper round (below and in the printable kit)
- Answer key (teacher only)
- Board or chart paper with a large calibration grid
- Optional: the How Sure Are You? lab
Hand-authored example
The 15-statement paper round
- Earth is closest to the Sun in January, during winter in the Northern Hemisphere.
- The sky looks blue because air molecules scatter blue light more than other colors.
- Most of Earth’s fresh water is found in lakes and rivers.
- The Wright brothers’ first powered flight in December 1903 was piloted by Wilbur Wright.
- Measured from Earth’s center, the summit of Mount Everest is the farthest point on Earth’s surface.
- A light-year measures distance, not time.
- A day on Venus is longer than a year on Venus.
- A lightning bolt can heat the air to about five times hotter than the surface of the Sun.
- Ice floats because it is denser than liquid water.
- Yellowstone was the world’s first national park.
- The highest tides in the world are found in the Bay of Fundy in Canada.
- Mercury is the hottest planet because it is the closest planet to the Sun.
- Sunlight takes about 8 minutes to reach Earth.
- Scientists can now predict the date, place and size of a major earthquake ahead of time.
- Your eardrum turns sound into the electrical signals that travel to your brain.
Answer key (teacher)
- True. Earth is closest to the Sun in early January and farthest in July, so distance does not cause the seasons. Source
- True. Blue light travels in shorter waves, so the gases in the air scatter it in all directions more than red or yellow light. Source
- False. About 69% of fresh water is locked in ice caps and glaciers and about 30% is underground. Lakes and rivers hold only a tiny share. Source
- False. Orville flew first, on December 17, 1903, about 120 feet. Wilbur won the coin toss but his attempt three days earlier failed. Source
- False. Earth bulges at the equator, so Mount Chimborazo in Ecuador reaches farther from Earth’s center, over 2 km more than Everest. Source
- True. A light-year is the distance light travels in one year, about 9 trillion kilometers. Source
- True. Venus spins so slowly that one day lasts 243 Earth days, while one trip around the Sun takes only 225 Earth days. Source
- True. Lightning can heat the air it passes through to about 50,000 °F, roughly five times hotter than the Sun’s surface. Source
- False. Ice floats because it is less dense than liquid water. Its molecules lock into a spread-out pattern. Source
- True. Yellowstone was established as the world’s first national park in 1872. Source
- True. Canada’s Bay of Fundy, between New Brunswick and Nova Scotia, has the highest tides in the world. Source
- False. Venus is the hottest planet. Its thick atmosphere traps heat, while Mercury has almost no atmosphere and gets very cold at night. Source
- True. Light from the Sun takes about 8.3 minutes to reach us, so we always see the Sun as it was a few minutes ago. Source
- False. No scientist has ever predicted a major earthquake. They can only estimate the chance of one in an area over many years. Source
- False. The eardrum vibrates and passes sound to three tiny bones. Hair cells in the cochlea, in the inner ear, make the electrical signals. Source
Printable
Student worksheet
Print one per student. Everything students hand in is handwritten, so their thinking is visible.
How Sure Are You?
Decide & Adapt · Week 2. Write your own thinking; it is what gets graded.
Assessment
Grade the process, not the answer
Every row scores something students did themselves. A lucky answer with no reasoning scores low; a careful decision that turned out wrong can score high.
| Criteria | 4 · Excellent | 3 · Solid | 2 · Developing | 1 · Beginning |
|---|---|---|---|---|
| Honest confidence | Uses the whole scale, with 50% where truly unsure | Uses at least three levels | Mostly one level | No confidence numbers |
| Reading the chart | Explains over- and underconfidence using the class chart and their own answers | Explains one of them correctly | Reads values without interpreting | Cannot read the chart |
| Reflection | Names a specific item and the reason their confidence was off | Names a specific item | General statement only | Missing |
| Next-time habit | A concrete, checkable habit tied to their mistake | A concrete habit | A vague intention | Missing |
Why this lesson
Documented · adult forecasters
A probability and debiasing training module lasting under an hour improved forecasters' accuracy (Brier scores) by 6 to 11% over a control group, across four years of forecasting tournaments with adult volunteers.
The training study involved adult forecasters, not school students. We expect the habit to transfer, but that has not been tested in classrooms.
Adjust it
Younger students: Use 10 statements and three confidence levels only: “guessing,” “fairly sure,” “certain.”
Older students: Compute a Brier score by hand for five items (see the lab’s explanation) and compare it with a coin-flip score of 0.25.
No devices at all? Already device-free. Skip the Partner step, or read this aloud: a chatbot answered a hard statement correctly once and wrongly the next time, in the same confident tone both times. What does that tell you?
The AI step
Level 0 No AI
Optional and short. The lesson itself needs no AI. If you use this step, ask a chatbot one hard statement on the teacher’s screen, to show that a confident tone is not a probability. Prompt:
Exit ticket
Finish the sentence: “When I feel 100% sure, I should…”
Why this is hard to cheat
The score that matters is the match between confidence and correctness, so looking answers up would only hide what students need to see. Everything is on paper, in class.
Go further
Honest limits and sources
Where this comes from
Open the source before quoting it.
Honest limits
What this page leaves out
This is an initial edition, written by ARQIV in October 2026. It has not been classroom-piloted or independently reviewed, so its scenarios and timings may change after classroom use. The evidence named above supports the core move of the lesson, not this lesson as a whole. We do not claim the lesson measures or certifies a student’s judgment or “learning agility.”
Review status
Written by ARQIV. Initial edition, October 2026. This lesson has not been classroom-piloted or independently reviewed yet, and every question and scenario is hand-authored and checked against its sources by ARQIV. Tell us about anything wrong or unclear.
- Welton Chang, Eva Chen, Barbara Mellers and Philip Tetlock, "Developing expert political judgment," Judgment and Decision Making 11(5), 2016, pp. 509–526.
- ARQIV, How Sure Are You? question bank (each statement links its own source in the answer key).
All links opened and checked October 21, 2026 (accessed October 2026).