Lab · Judgment · Calibration

How sure are you?

Answer ten true-or-false questions, and for each one say how sure you are. Then see whether your “90% sure” really means 90%. This skill is called calibration: matching your confidence to how often you are actually right.

Grades 7+ and adults10–15 minutesSolo or one projectorFree · No sign-up

What you will be able to do

Give an honest confidence number, read a calibration chart, and explain what a Brier score says about your guesses.

How this lab is built

Hand-authored questions and logic that run in your browser. No AI writes or marks anything, the lab sends nothing to a server, and every statement links its source.

01 · Predict first

When people say “90% sure,” how often are they right?

Make a guess before you play.

Predict first

Out of every ten answers people give with 90% confidence, how many do you think are right?

Show the answer

Often fewer than nine. In many studies of general-knowledge questions, people’s high-confidence answers are right less often than their confidence suggests. This is called overconfidence. The good news: with feedback, people can get better at matching their confidence to the evidence. Adult forecasters improved their accuracy by 6 to 11% after under an hour of training. Documented · adult forecasters

02 · Play

Ten questions, one number each

Pick True or False, then how sure you are. 50% means a pure guess. 100% means you would bet anything.

Lab

How Sure Are You?

Answer, rate your confidence, read the explanation, then see your calibration chart and Brier score.

Hand-authored questions. Simplified scoring in your browser.

Text version. On paper, answer each statement True or False and write how sure you are (50–100%). Then check the answers below and count, for each confidence level, how often you were right.

  1. Earth is closest to the Sun in January, during winter in the Northern Hemisphere. True. Earth is closest to the Sun in early January and farthest in July, so distance does not cause the seasons.
  2. The sky looks blue because air molecules scatter blue light more than other colors. True. Blue light travels in shorter waves, so the gases in the air scatter it in all directions more than red or yellow light.
  3. Most of Earth’s fresh water is found in lakes and rivers. False. About 69% of fresh water is locked in ice caps and glaciers and about 30% is underground. Lakes and rivers hold only a tiny share.
  4. The Wright brothers’ first powered flight in December 1903 was piloted by Wilbur Wright. False. Orville flew first, on December 17, 1903, about 120 feet. Wilbur won the coin toss but his attempt three days earlier failed.
  5. Measured from Earth’s center, the summit of Mount Everest is the farthest point on Earth’s surface. False. Earth bulges at the equator, so Mount Chimborazo in Ecuador reaches farther from Earth’s center, over 2 km more than Everest.
  6. A light-year measures distance, not time. True. A light-year is the distance light travels in one year, about 9 trillion kilometers.
  7. Venus takes longer to spin once than to go once around the Sun. True. Venus spins so slowly that one full spin takes 243 Earth days, while one trip around the Sun takes only 225 Earth days.
  8. A lightning bolt can heat the air to about five times hotter than the surface of the Sun. True. Lightning can heat the air it passes through to about 50,000 °F, roughly five times hotter than the Sun’s surface.
  9. Ice floats because it is denser than liquid water. False. Ice floats because it is less dense than liquid water. Its molecules lock into a spread-out pattern.
  10. Yellowstone was the world’s first national park. True. Yellowstone was established as the world’s first national park in 1872.

If your 90% answers were right much less than 9 times in 10, you were overconfident. The Brier score is the average of (probability you gave to “true” minus 1 if true, or 0 if false) squared: 0 is perfect, 0.25 is what saying 50% every time scores, and 1 is sure and wrong every time.

03 · The score

What a Brier score means

One number that rewards being right and being honest about it.

How it works

For each question, turn your answer into the chance you gave that the statement is true. “True, 80% sure” is 0.8. “False, 80% sure” is 0.2. Subtract what happened (1 if it was true, 0 if false), square it, and average over the round.

A confident right answer costs almost nothing. A confident wrong answer costs a lot. Saying 50% every time scores exactly 0.25, so a score under 0.25 means your confidence told you something. The worst score is 1: sure and wrong every time.

A worked example

You say “True, 90%” and it is true: (0.9 − 1)² = 0.01. You say “True, 90%” and it is false: (0.9 − 0)² = 0.81. You say “False, 60%” and it is false: (0.4 − 0)² = 0.16. Average of the three: 0.33.

Weather forecasters and forecasting tournaments use this score. ARQIV’s Probability page shows the same idea for AI forecasts.

04 · For teachers

Run it in class

Use one projector, student devices, or paper. It pairs with Week 2 of Decide & Adapt.

Three ways to run it

  • Paper (no devices): print the 15-statement round from the Week 2 lesson and build a chart on the board. If students play online afterwards, they tick “My class did the paper round” so the lab leaves out those 15 statements.
  • One projector: switch to Class mode. Vote by show of hands, enter the majority and the average confidence.
  • Student devices: each student plays solo. Progress stays on their device; ask them to bring one sentence about their chart, not their score.

Timing: 2 min predict · 8 min round · 5 min chart talk.

Use your own questions

One per line: T | statement | explanation or F | statement | explanation. Your questions stay in this browser tab only: they are not saved or sent. A round on your own questions is labelled on its result screen and is not saved.

Honest limits

What this lab leaves out

Read your chart with these in mind.

Not measured here

Ten questions are too few to judge anyone’s calibration; you need many rounds. True-or-false trivia is not the same as real decisions, where you choose what to find out. The lab does not measure or certify your judgment, and it never compares you with anyone else. The training research we cite was done with adult forecasters, not students. The questions are hand-authored and checked against their sources; if you find an error, tell us.

Review status

Written by ARQIV. Initial edition, October 2026. These pages have not been classroom-piloted or independently reviewed yet, and every question and scenario is hand-authored and checked against its sources by ARQIV. Tell us about anything wrong or unclear.

Sources

Where this comes from

Each question also links its own source after you answer it.

  1. Welton Chang, Eva Chen, Barbara Mellers and Philip Tetlock, "Developing expert political judgment," Judgment and Decision Making 11(5), 2016, pp. 509–526.
  2. ARQIV, “How sure should an AI be?” (calibration of AI forecasts).
  3. NASA Space Place, “What Causes the Seasons?”
  4. NASA Space Place, “Why Is the Sky Blue?”
  5. USGS Water Science School, “How Much Water is There on Earth?”
  6. National Park Service, “1903: The First Flight”
  7. NOAA National Ocean Service, “What is the highest point on Earth as measured from Earth’s center?”
  8. NASA Space Place, “What Is a Light-Year?”
  9. NASA Science, “Venus: Facts”
  10. National Weather Service, “Understanding Lightning Science”
  11. USGS Water Science School, “Water Density”
  12. National Park Service, “Yellowstone: History and Culture”
  13. NOAA National Ocean Service, “Where is the highest tide?”
  14. NASA Science, “Mercury: Facts”
  15. USGS, “Can you predict earthquakes?”
  16. NIDCD, “How Do We Hear?”
  17. NASA Space Place, “What Are the Moon’s Phases?”
  18. NASA Science, “Jupiter Facts”
  19. NASA Science, “Uranus: Facts”
  20. NASA Space Place, “What Is a Black Hole?”
  21. NASA Space Place, “Asteroid or Meteor: What’s the Difference?”
  22. NASA Science, “Sun: Facts”
  23. NASA Science, “Moon Facts”
  24. NASA, “Laser Beams Reflected Between Earth and Moon Boost Science”
  25. NIST, “SI Units”
  26. NIST, “Meter”
  27. NIST, “Second: Introduction”
  28. NASA Space Place, “How Does GPS Work?”
  29. NASA Science, “Facts About Earth”
  30. NOAA JetStream, “Layers of the Atmosphere”
  31. NOAA National Ocean Service, “Why is the ocean salty?”
  32. NOAA National Ocean Service, “How deep is the ocean?”
  33. NOAA National Ocean Service, “How many oceans are there?”
  34. National Park Service, “Death Valley: Weather”
  35. NASA, “Apollo 11”
  36. National Park Service, “The Liberty Bell”
  37. National Park Service, “Statue of Liberty: Statue Statistics”
  38. ARQIV, “What is a transformer?”
  39. ARQIV, “Training and inference”
  40. ARQIV, “Machine learning”
  41. ARQIV, “Probability and uncertainty”

All links opened and checked October 21, 2026 (accessed October 2026). The NASA laser-ranging article was added and checked November 15, 2026.

AI words on this page

Calibration, Base rate. A Brier score entry is being added.