# Computer vision

- Link: https://arqiv.si/ai/computer-vision/
- Published: 2026-10-10T15:12:34-04:00
- Author: Steven Ryan

[Skip to content](https://arqiv.si/ai/computer-vision/#main-content)

[arqiv](https://arqiv.si/)

[For educators](https://arqiv.si/teachers/)

Teach

 * [Start here](https://arqiv.si/teachers/)
 * [Lessons & printable kits](https://arqiv.si/teachers/resources/)
 * [Classroom labs](https://arqiv.si/teachers/labs/)
 * [Free 20-minute lesson](https://arqiv.si/teachers/free-ai-lesson/)

Plan

 * [Curriculum guide](https://arqiv.si/teachers/curriculum/)
 * [AI Learning Center](https://arqiv.si/teachers/ai-literacy/)
 * [Future of work unit](https://arqiv.si/future-of-work/)
 * [What the research says](https://arqiv.si/teachers/what-works/)
 * [AI education worldwide](https://arqiv.si/teachers/around-the-world/)

Understand

 * [Think First method](https://arqiv.si/teachers/think-first/)
 * [Our approach](https://arqiv.si/research/#our-method)
 * [AI vocabulary](https://arqiv.si/teachers/ai-vocabulary/)

By role

 * [Counselors](https://arqiv.si/counselors/)
 * [School leaders & PD](https://arqiv.si/schools/)
 * [Speaking & workshops](https://arqiv.si/speaking/)
 * [Products](https://arqiv.si/products/)

[Learn AI](https://arqiv.si/ai/topics/)

Start

 * [All AI topics](https://arqiv.si/ai/topics/)
 * [The AI journey, from zero](https://arqiv.si/ai/)
 * [Using AI well](https://arqiv.si/ai/talking-to-ai/)
 * [AI builds](https://arqiv.si/ai-builds/)

AI in the world

 * [Space](https://arqiv.si/space/)
 * [Weather](https://arqiv.si/weather/)
 * [Health](https://arqiv.si/health/)
 * [Engineering](https://arqiv.si/engineering/)
 * [Farming](https://arqiv.si/farming/)
 * [Quantum](https://arqiv.si/quantum/)

What it means

 * [Future of work](https://arqiv.si/future-of-work/)
 * [Your AI path](https://arqiv.si/ai/your-path/)
 * [Research, explained](https://arqiv.si/research/)

[Games & labs](https://arqiv.si/games/)[Research](https://arqiv.si/research/)[Blog](https://arqiv.si/blog/)
[About](https://arqiv.si/about/)

[Search or ask](https://arqiv.si/ai/topics/)[Newsletter](https://arqiv.si/newsletter/)
[ARQIV.FM ↗](https://arqiv.fm/)

Menu

For educators

[For educators: start page →](https://arqiv.si/teachers/)

Teach

 * [Start here](https://arqiv.si/teachers/)
 * [Lessons & printable kits](https://arqiv.si/teachers/resources/)
 * [Classroom labs](https://arqiv.si/teachers/labs/)
 * [Free 20-minute lesson](https://arqiv.si/teachers/free-ai-lesson/)

Plan

 * [Curriculum guide](https://arqiv.si/teachers/curriculum/)
 * [AI Learning Center](https://arqiv.si/teachers/ai-literacy/)
 * [Future of work unit](https://arqiv.si/future-of-work/)
 * [What the research says](https://arqiv.si/teachers/what-works/)
 * [AI education worldwide](https://arqiv.si/teachers/around-the-world/)

Understand

 * [Think First method](https://arqiv.si/teachers/think-first/)
 * [Our approach](https://arqiv.si/research/#our-method)
 * [AI vocabulary](https://arqiv.si/teachers/ai-vocabulary/)

By role

 * [Counselors](https://arqiv.si/counselors/)
 * [School leaders & PD](https://arqiv.si/schools/)
 * [Speaking & workshops](https://arqiv.si/speaking/)
 * [Products](https://arqiv.si/products/)

Learn AI

[Learn AI: start page →](https://arqiv.si/ai/topics/)

Start

 * [All AI topics](https://arqiv.si/ai/topics/)
 * [The AI journey, from zero](https://arqiv.si/ai/)
 * [Using AI well](https://arqiv.si/ai/talking-to-ai/)
 * [AI builds](https://arqiv.si/ai-builds/)

AI in the world

 * [Space](https://arqiv.si/space/)
 * [Weather](https://arqiv.si/weather/)
 * [Health](https://arqiv.si/health/)
 * [Engineering](https://arqiv.si/engineering/)
 * [Farming](https://arqiv.si/farming/)
 * [Quantum](https://arqiv.si/quantum/)

What it means

 * [Future of work](https://arqiv.si/future-of-work/)
 * [Your AI path](https://arqiv.si/ai/your-path/)
 * [Research, explained](https://arqiv.si/research/)

 * [Games & labs](https://arqiv.si/games/)
 * [Research](https://arqiv.si/research/)
 * [Blog](https://arqiv.si/blog/)
 * [About](https://arqiv.si/about/)

[Get the newsletter](https://arqiv.si/newsletter/)

 1. [ARQIV / AI AT WORK](https://arqiv.si/ai/topics/)
 2. [**01**Computer vision](https://arqiv.si/ai/computer-vision/)
 3. [**02**Language & speech](https://arqiv.si/ai/language-and-speech/)
 4. [**03**Recommendations](https://arqiv.si/ai/recommendations/)
 5. [**04**Forecasts & anomalies](https://arqiv.si/ai/forecasting-and-anomalies/)

Go deeper · Ai at work

# What does a camera-based AI _actually measure?_

Light. A camera records how much light reached each tiny spot on its sensor, and
stores it as numbers. Computer vision is the part of AI that turns those numbers
into useful outputs: a label, boxes around objects, a map of every pixel, or the
words in a picture. The camera does not understand anything, and when the light 
changes, the numbers change too.

Grades 6+15 minutesCountingYounger version for grades 3–5

What you will be able to do

Tell apart classification, object detection, segmentation and OCR by their outputs,
explain why a change in lighting can break a vision system that worked before, and
name two privacy or fairness questions to ask about cameras that recognize people.

Before you start

Helpful: [How does a picture become numbers?](https://arqiv.si/ai/tensors/) (pixels
as numbers) and the [weed-or-crop sprayer lab](https://arqiv.si/farming/#lab) on
the farming page. No math beyond counting.

1 · You have already seen this

## Your camera already _does more than take photos_

Phones and apps pick things out of pictures every day.

Point a phone at a sign and it can copy the words so you can paste them. A plant
app looks at a leaf and says “dandelion.” On a video call, everything behind you
can blur while you stay sharp, and some phones unlock when they see your face.

All of these use **computer vision**, the part of AI that turns a camera’s picture
into a label, boxes around objects, a map of every pixel or the words in the picture.
The camera only records light as numbers, and when the light changes, the numbers
change too. The rest of this page shows how that works, lets you try it, and shows
where it goes wrong.

2 · What happens, step by step

## From light to _a useful answer_

What happens between the camera and the answer, in four steps. The next section 
goes deeper into steps 1 and 2, and the one after it into step 4.

How computer vision works

### Four steps

 1. 1 · Light hits the sensorA camera sensor is a grid of millions of tiny light detectors.
    Each one counts how much light of a color reached it.
 2. 2 · Light becomes numbersEach count becomes a pixel value, often from 0 (dark) 
    to 255 (bright), with three values per pixel for red, green and blue.
 3. 3 · A model looks for patternsThe model has learned from many labeled examples 
    which patterns of numbers usually go with which answers. That is why it can be 
    fooled when the numbers change in a way it never saw during training.
 4. 4 · Out comes an answerDepending on the job, it gives one label for the whole picture,
    a box around each object, a label for every pixel, or the text in the picture.

Going deeper · steps 1 and 2

## Pixels are _measurements_

Before any AI runs, the picture is already a grid of numbers. Those numbers record
how much light reached the camera. They say nothing yet about what the objects are.

A camera sensor is a grid of millions of tiny light detectors. Each one counts how
much light of a color hit it during the photo. That count becomes a pixel value,
often from 0 (dark) to 255 (bright), with three values per pixel for red, green 
and blue. In [Lightprint](https://arqiv.si/games/lightprint/), a spectrum is a picture
only one row tall; a photo is the same idea in a grid.

Nothing in those numbers says “weed” or “stop sign.” The same leaf gives different
numbers at noon, at dusk, in shadow, through a dusty lens or from a different angle.
A vision model has to learn which patterns of numbers usually go with which answers,
from many labeled examples. That is why it can be fooled when the numbers change
in a way it never saw during training.

Going deeper · step 4

## Label, boxes, mask _or text_

Computer-vision systems differ most in what they output. The output decides what
the system can be used for.

Four common computer-vision jobs| Job | What comes out | Everyday example | What the training labels look like | 
| Classification | One label for the whole picture | A plant app says “dandelion” | One word per photo | 
| Object detection | A box and a label for each object it finds | A self-checkout camera draws boxes around items | A box drawn around every object | 
| Segmentation | A label for every pixel (a mask) | A video call blurs everything except you | Every pixel colored by hand or with tools: slow | 
| OCR (optical character recognition) | The text in the picture, as letters | A phone copies words from a photo of a sign | Pictures of text with the correct text typed out |

Predict first

A sprayer must squirt one weed growing between two crop plants, without hitting 
the crops. Which output does it need: a label, boxes or a mask?

Show the answer

A whole-picture label (“weed present”) cannot aim a nozzle. **Boxes** say where 
each plant is, which may be enough when plants are spaced apart. When leaves overlap,
a **mask** is better, because it says exactly which pixels are weed. Finer outputs
are more useful, and their training labels take much longer to make.

3 · Try it

## Find the plants, _then change the light_

A tiny made-up field, a simple brightness rule, and three ways to report what it
found.

Predict first

Before you open the lab: the rule says “a pixel is a plant if its brightness is 
5 or more.” The sun comes out and every pixel gets 4 brighter. What happens?

Show the answer

Soil was 1 to 3; now it is 5 to 7. The rule marks **every** pixel as a plant: all
76 soil pixels are wrongly marked, and the boxes merge into one box around the whole
field. Try it, then look at what the whole-picture label says.

Lab

### Plant finder under changing light

Pick an output type, then change the lighting. The rule never changes, but the numbers
it sees do. In the box view, dashed outlines show where the plants really are. Then
try the relative rule, which compares each pixel with the picture’s average.

 A made-up 12 × 8 field and a hand-written rule, computed in your browser. Real 
systems use trained models.

1 · What should the AI output?

2 · Change the lighting

3 · The rule

The lab needs JavaScript, so here are its numbers. A made-up camera gives each of
the 12 × 8 = 96 squares one brightness number from 0 (dark) to 9 (bright). There
are 20 plant pixels: a crop on the left and a weed on the right. The rule calls 
a pixel “plant” if it is bright enough.

The made-up field in normal light (plants are the 6s, 7s and 8s)| Row | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 
| Row 0 | 2 | 1 | 2 | 3 | 2 | 1 | 2 | 3 | 2 | 1 | 2 | 3 | 
| Row 1 | 3 | 2 | 3 | 2 | 1 | 2 | 1 | 2 | 1 | 2 | 3 | 2 | 
| Row 2 | 2 | 1 | 6 | 7 | 6 | 3 | 2 | 1 | 2 | 1 | 2 | 3 | 
| Row 3 | 3 | 2 | 7 | 8 | 8 | 7 | 3 | 2 | 1 | 2 | 3 | 2 | 
| Row 4 | 2 | 1 | 6 | 8 | 7 | 2 | 2 | 3 | 1 | 7 | 6 | 3 | 
| Row 5 | 3 | 2 | 1 | 7 | 6 | 2 | 1 | 2 | 6 | 8 | 7 | 2 | 
| Row 6 | 2 | 3 | 2 | 1 | 2 | 3 | 2 | 1 | 7 | 8 | 6 | 3 | 
| Row 7 | 1 | 2 | 3 | 2 | 1 | 2 | 3 | 2 | 1 | 2 | 3 | 2 |

What the rule outputs in each lighting| Lighting · rule | Plant pixels found | Soil pixels wrongly marked | Boxes | Whole-picture label | 
| Normal light · Fixed | 20 of 20 | 0 | 2 | plants: yes | 
| Normal light · Relative | 20 of 20 | 0 | 2 | plants: yes | 
| Bright sun (+4) · Fixed | 20 of 20 | 76 | 1 | plants: yes | 
| Bright sun (+4) · Relative | 20 of 20 | 0 | 2 | plants: yes | 
| Dusk (−3) · Fixed | 5 of 20 | 0 | 2 | plants: yes | 
| Dusk (−3) · Relative | 20 of 20 | 0 | 2 | plants: yes | 
| Shadow on the left half (−3) · Fixed | 11 of 20 | 0 | 2 | plants: yes | 
| Shadow on the left half (−3) · Relative | 16 of 20 | 0 | 2 | plants: yes |

Notice the last column: the whole-picture label says “plants: yes” every time, even
in bright sun when every single pixel is marked as a plant.

Worked example

### Counting what the rule gets right

 1. The field12 × 8 = **96** pixels. Soil is 1 to 3; plant pixels are 6, 7 or 8. There
    are **20** plant pixels: 12 in the crop and 8 in the weed. Five of them are 8s.
 2. Normal lightRule: 5 or brighter is a plant. All 20 plant pixels pass and no soil
    does: **20 of 20** found, **0** false alarms, 2 boxes.
 3. Dusk (−3)Plants drop to 3, 4 or 5. Only the five 8s reach 5: **5 of 20** found (
    25%), 15 missed. There are still 2 boxes, but they are tiny.
 4. Bright sun (+4)Soil rises to 5, 6 or 7 (nothing goes above 9). All 96 pixels are
    marked, so **76** are false alarms, and the two plants merge into 1 box around 
    everything.
 5. Relative ruleThreshold = the picture’s average + 2. Average brightness is 291 ÷
    96 ≈ 3.03 in normal light, ≈ 6.64 in bright sun and ≈ 0.81 at dusk, so the threshold
    moves with the light and all 20 plants are found with no false alarms in all three.
 6. ShadowDarken only the left half. Now the fixed rule finds 11 of 20 and the relative
    rule 16 of 20: one average cannot fit two kinds of light at once.

All pixel values are made up and every count is checked. A real camera gives three
color numbers per pixel and real systems use trained models, but they face the same
problem: the numbers depend on the light.

What changes with lighting?

## When the world _stops matching the training photos_

A model learns from the photos it was given. New conditions mean new numbers.

The lab rule failed because the light changed, not because the plants did. Trained
models fail the same way, only less obviously. If every training photo was taken
at noon on dry soil, the model has never seen dusk, rain, mud or a different camera.
When the data at work differs from the training data, that is called **distribution
shift**. In one well-known test, image classifiers lost much of their accuracy on
photos with added fog, snow, blur or brightness changes.

The [farming page](https://arqiv.si/farming/) lists the same worries for real sprayers:
look-alike weeds, dust, shadows and speed. Common defenses: collect training photos
in many conditions, adjust the numbers before the model sees them (like the relative
rule), test in the exact conditions of use, and keep people checking. A good habit
for any vision system: ask “what changes with lighting, weather, camera or place?”

Cameras that look at people

## Privacy, surveillance _and unequal errors_

The same technology that finds weeds can find faces. That raises questions a plant
never asks.

Face recognition compares the measurements of a face in a photo with stored faces.
People in a crowd usually did not agree to be measured, a face cannot be changed
like a password, and stored images can be shared or leaked. That is why many people
ask who is recorded, who keeps the images and for how long, and who checks the matches.

Errors are not shared equally. In 2019 the U.S. National Institute of Standards 
and Technology (NIST) tested 189 face-recognition algorithms from 99 developers.
For most of them, false matches were more common for some demographic groups than
for others, often by a factor of 10 to more than 100, and the differences varied
a lot between algorithms. A 2018 study, Gender Shades, found that commercial systems
guessing gender from faces were wrong up to 34.7% of the time for darker-skinned
women but at most 0.8% for lighter-skinned men.

Questions to ask about any camera system

 1. What exactly does it output: a label, boxes, masks, text or identities?
 2. Was it tested in conditions like ours, and for people like us?
 3. What are the error rates for each group, not only overall?
 4. Who sees the images, how long are they kept, and can people say no?
 5. What happens to a person when it makes a mistake, and who checks?

4 · Common mix-ups

## The camera does not _see the way we do_

Three ideas that trip people up, and what to think instead.

 * **Mix-up:** The camera sees the way we see.
 * **Better:** It records numbers for light at each pixel. Recognizing a weed or
   a face is a separate step done by a model, and that step can be wrong.
 * **Mix-up:** If it worked in testing, it works everywhere.
 * **Better:** New lighting, weather, cameras or places change the numbers. Test
   in the conditions where the system will be used.
 * **Mix-up:** A single accuracy number tells you whether face recognition is fair.
 * **Better:** An overall rate can hide much higher error rates for some groups.
   Ask for results by group, as NIST reported.

5 · Check yourself

## Three _quick questions_

Answer in your head or on paper before you open each one.

 1.  Name the job: (a) an app says “this photo contains a dog,” (b) a store camera 
     draws a box around every item in a basket, (c) a video call blurs everything behind
     you.
 2.  Check your answer
 3.  (a) **Classification**: one label for the whole picture. (b) **Object detection**:
     a box and label per object. (c) **Segmentation**: every pixel is marked as “person”
     or “background,” so the background can be blurred.
 4.  In the lab’s bright sun, the whole-picture label still said “plants: yes.” Why
     is that not proof that the rule works?
 5.  Check your answer
 6.  The label was right by luck. The mask showed that all 96 pixels were marked as
     plant, including 76 soil pixels. A coarse output can hide a broken system, so 
     check finer outputs or test cases where you know the answer.
 7.  A company says its face-recognition system is “99% accurate.” What would you ask
     before trusting it?
 8.  Check your answer
 9.  Accurate on what photos, and for whom? Ask for false-match and missed-match rates
     for each demographic group and in conditions like yours (lighting, cameras). NIST
     found that differences between groups were often large and varied by algorithm.
     Also ask who checks matches and what happens to a person after a mistake.

In the real world

## An application, a limit _and a link_

Application

### Reading old books

Libraries scan old books and newspapers and use OCR to turn the page images into
searchable text. Faded ink, unusual fonts and handwriting still cause mistakes, 
so important texts get checked by people.

Limitation

### Measurements, not understanding

A system that detects “weed” has learned patterns of light, not what a weed is. 
A photo of a weed printed on a T-shirt can produce similar numbers. Outputs need
checking against the real world.

Connection

### The networks behind it

Most vision models are neural networks built for grids, such as convolutional networks.
Slide a filter yourself on [Why are there different kinds of neural networks?](https://arqiv.si/ai/neural-network-families/)

For younger learners

## A _grades 3–5_ version

No numbers needed: the same object, two kinds of light.

20 minutes · small groups

### Same toy, different light

 1. Photograph one toy by a sunny window and again in a dim corner.
 2. Zoom in until you see the squares (pixels). Compare one square from each photo:
    are they the same color?
 3. Ask: if a robot only knew the sunny photo, would it recognize the toy in the dim
    one?

Exit question

### Say it in one sentence

“What does a camera really collect?” A good answer: “Colors of light in tiny squares,
and they change when the light changes.” Then try the [sprayer lab](https://arqiv.si/farming/#lab)
together.

Honest limits

## What this page _leaves out_

What this page leaves out

How vision models are trained and what their layers learn, depth cameras and lidar,
video tracking, image generation (see [creative AI](https://arqiv.si/ai/creative-ai/)),
medical imaging, and the laws about cameras and faces, which differ from place to
place and change over time.

Where to go next

## Keep _looking closer_

Next in this branch

### What else can AI do with language?

Translation, speech recognition and more, beyond chatbots.

Vision at work

### Be the sprayer’s brain

Set how sure the AI must be before it sprays, and see misses versus wasted spray.

Testing fairly

### Good at what, for whom?

Misses, false alarms and why one score is never the whole story.

Sources

## Where this comes from

 1. [Patrick Grother, Mei Ngan and Kayee Hanaoka, “Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects,” NISTIR 8280, National Institute of Standards and Technology, December 2019.](https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf)
 2. [Joy Buolamwini and Timnit Gebru, “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification,” Proceedings of Machine Learning Research 81, 2018.](https://proceedings.mlr.press/v81/buolamwini18a.html)
 3. [Dan Hendrycks and Thomas Dietterich, “Benchmarking Neural Network Robustness to Common Corruptions and Perturbations,” 2019: accuracy drops under fog, snow, blur and brightness changes.](https://arxiv.org/abs/1903.12261)
 4. [Tsung-Yi Lin and colleagues, “Microsoft COCO: Common Objects in Context,” 2014: a dataset with labels, boxes and per-pixel masks for detection and segmentation.](https://arxiv.org/abs/1405.0312)
 5. [Stanford CS231n course notes, “Image Classification”: images as arrays of numbers, and challenges such as illumination and viewpoint.](https://cs231n.github.io/classification/)
 6. [AI4K12, the Five Big Ideas in AI: Big Idea 1, perception (computers perceive the world using sensors).](https://ai4k12.org/)

AI words on this page

[Computer vision](https://arqiv.si/teachers/ai-vocabulary/#computer-vision), [Classification](https://arqiv.si/teachers/ai-vocabulary/#classification),
[Object detection](https://arqiv.si/teachers/ai-vocabulary/#object-detection), [OCR](https://arqiv.si/teachers/ai-vocabulary/#ocr),
[Convolutional neural network](https://arqiv.si/teachers/ai-vocabulary/#convolutional-neural-network),
[Distribution shift](https://arqiv.si/teachers/ai-vocabulary/#distribution-shift),
[Training data](https://arqiv.si/teachers/ai-vocabulary/#training-data), [Label](https://arqiv.si/teachers/ai-vocabulary/#label),
[False positive](https://arqiv.si/teachers/ai-vocabulary/#false-positive), [Bias](https://arqiv.si/teachers/ai-vocabulary/#bias),
[Fairness](https://arqiv.si/teachers/ai-vocabulary/#fairness), [Tensor](https://arqiv.si/teachers/ai-vocabulary/#tensor)

All definitions live in one place: [AI words in plain language →](https://arqiv.si/teachers/ai-vocabulary/)

Review status

Written by ARQIV. Last checked October 2026. Independent subject review and classroom
pilots are still to come, so treat the activities as drafts and [tell us](https://arqiv.si/ai/computer-vision/info@arqiv.fm?subject=AI%20foundations%20correction)
about anything that is wrong or unclear.

[← Previous**All AI topics**](https://arqiv.si/ai/topics/)[Next →**Language & speech**](https://arqiv.si/ai/language-and-speech/)

[arqiv](https://arqiv.si/)

A little understanding changes everything.

[LISTEN: ARQIV.FM, OUR WEEKLY SHOW ↗](https://arqiv.fm/)

Educators

 * [Start here](https://arqiv.si/teachers/)
 * [Lessons & kits](https://arqiv.si/teachers/resources/)
 * [Curriculum](https://arqiv.si/teachers/curriculum/)
 * [Labs](https://arqiv.si/teachers/labs/)
 * [AI vocabulary](https://arqiv.si/teachers/ai-vocabulary/)

For your role

 * [Counselors](https://arqiv.si/counselors/)
 * [School leaders & PD](https://arqiv.si/schools/)
 * [Speaking & workshops](https://arqiv.si/speaking/)
 * [Products](https://arqiv.si/products/)

Learn

 * [Learn AI](https://arqiv.si/ai/topics/)
 * [Games & labs](https://arqiv.si/games/)
 * [Future of work](https://arqiv.si/future-of-work/)
 * [Research](https://arqiv.si/research/)
 * [Headlines](https://arqiv.si/news/)

ARQIV

 * [About & corrections](https://arqiv.si/about/)
 * [Our approach](https://arqiv.si/research/#our-method)
 * [Newsletter](https://arqiv.si/newsletter/)
 * [Privacy](https://arqiv.si/privacy-policy/)
 * [Get in touch](https://arqiv.si/ai/computer-vision/info@arqiv.fm)

Classroom activities collect no student data.[info@arqiv.fm](https://arqiv.si/ai/computer-vision/info@arqiv.fm)
