Go deeper · How machines learn

Why are there different kinds of neural networks?

Because data comes in different shapes. A list of measurements, a photo, a sentence and a molecule are arranged in different ways, and each family of networks builds in a guess about that arrangement. A good guess means the network needs less data and computing to learn. No family is best at everything, and the newest is not automatically the right one.

Grades 9+20 minutesMultiply and addYounger version for grades 5–8

What you will be able to do

Match six neural-network families (MLP, CNN, RNN, transformer, GNN and mixture of experts) to the kind of structure each one suits, compute one step of a convolution by hand, and explain why the choice depends on data, compute and constraints rather than on which family is newest.

Before you start

Before this: How AI works (neurons, weights and layers) and How does a picture become numbers? (pictures as grids). Helpful: What is a transformer?

1 · You have already seen this

Different things, read in different ways

You already read different kinds of things in different ways, without thinking about it.

You read a bus timetable across and down, and a text message one word after another, in order. In a photo you look for edges and shapes wherever they happen to be. A family tree you follow along the lines, from one person to the people linked to them.

Computers meet the same problem. A list of measurements, a photo, a sentence and a molecule are arranged in different ways, so researchers wire neural networks in different patterns, called families, each with a built-in guess about how its data is laid out. This page shows how that works, lets you try it, and shows where it goes wrong.

2 · What happens, step by step

From data to a working network

Choosing and running a network in five steps. The next section goes deeper into steps 1 and 2, and the lab lets you watch step 3 inside a CNN. After the lab, one section goes deeper into the bend in step 3 and another into step 5.

How a network family is used

Five steps

  1. 1 · Look at the dataAsk how it is arranged: a fixed list of measurements, a grid like a photo, a sequence like a sentence, or things linked to other things, like atoms in a molecule.
  2. 2 · Wire to matchEvery family uses the same neuron, which multiplies inputs by weights and adds them up. The family decides how the neurons connect, and that wiring is its built-in guess about the data.
  3. 3 · Multiply, add, bendEach layer multiplies and adds. Then each total passes through a simple function such as ReLU, which keeps positive numbers and turns negative ones into 0.
  4. 4 · Learn the weightsDuring training, the weights are adjusted using examples. In a CNN that includes the numbers inside the filters.
  5. 5 · Compare optionsTry more than one family, or a simple model with no neural network at all, on your own data and on the device that has to run it.

Going deeper · steps 1 and 2

Six families, six kinds of structure

Every family uses the same building block: a neuron that multiplies inputs by weights, adds them up and passes the total on. What changes is how the neurons are wired together.

If every neuron may connect to every input, a network can in principle learn anything, but it needs a huge number of weights and a lot of examples to find the useful ones.

So researchers wire neurons in patterns that match how the data is arranged. That built-in guess is called an inductive bias (a different meaning of “bias” from unfairness). When the guess fits the data, the network learns faster from fewer examples. When it does not fit, it can get in the way.

Fixed-size lists

MLP

A multilayer perceptron: layers in which every neuron connects to every number in the layer before. It suits a fixed list of measurements, such as 12 readings about a plant.

Built-in guess: almost none. The order of the inputs means nothing, so it has to learn every relationship from data.

Grids

CNN

A convolutional neural network slides small filters across a grid, such as a photo or a picture of sound. You can try one filter in the lab below.

Built-in guess: nearby pixels belong together, and an edge means the same thing wherever it appears.

Sequences, step by step

RNN

A recurrent neural network reads one step at a time, such as one sensor reading or one word, and carries a running summary (its “memory”) to the next step. LSTM and GRU are popular kinds.

Built-in guess: order matters, and the past can be squeezed into a summary.

Any part to any part

Transformer

A transformer uses attention: every token gets a score with every other token, so far-apart words can connect in one layer. Some vision models cut a picture into patches and treat them as tokens.

Built-in guess: very little about position; the data decides which parts to connect.

Graphs

GNN

A graph neural network works on things (nodes) and links (edges): atoms and bonds in a molecule, or people and friendships. Each node updates its numbers using messages from its neighbors.

Built-in guess: what matters is who is linked to whom, not a place on a grid.

Routing to specialists

Mixture of experts

Not a data shape but a way to organize a big network, usually a transformer. A small router sends each token to a few of many “expert” sub-networks, for example 2 out of 8, so only part of the network runs each time.

Built-in guess: different inputs are best handled by different parts.

Predict first

Which family would you try first for each job? (a) Predict how long a delivery takes from 15 numbers about the order. (b) Spot cracks in photos of a bridge. (c) Predict whether a new molecule dissolves in water.

Show the answer

(a) An MLP, or a classical model such as a decision tree: the data is a fixed list of features. On tables like this, classical models are often just as good and easier to check. (b) A CNN (or a vision transformer): cracks are local patterns that can appear anywhere in a grid. (c) A GNN: a molecule is atoms linked by bonds, which is a graph. In every case you would test more than one option on your own data.

3 · Try it

One small filter, many positions

A convolution slides a 3 × 3 grid of weights across the picture. At each stop it multiplies and adds, and the result becomes one number in a new grid called a feature map.

Worked example

The vertical-edge filter at one position

  1. InputA made-up 8 × 8 gray picture: 0 is dark, 9 is bright. A bright 4 × 4 square fills rows 2 to 5 and columns 2 to 5 (counting from 0).
  2. FilterEvery row of the filter is −1, 0, 1: it subtracts the left column and adds the right column.
  3. Place itPut the filter’s top-left corner on row 2, column 1. The window covers columns 1, 2 and 3, so each of its three rows reads 0, 9, 9.
  4. Multiply and addOne row: 0 × (−1) + 9 × 0 + 9 × 1 = 9. Three identical rows: 9 + 9 + 9 = 27. That 27 goes into the feature map at row 2, column 1.
  5. Read itA big positive number means dark on the left and bright on the right: the left edge of the square. Move the window to row 2, column 4 and you get −27, the right edge. Over flat areas the result is 0.
  6. Count weightsThis filter has 9 weights, reused at all 36 positions. A fully connected layer from the 64 pixels to 36 outputs would need 64 × 36 = 2,304 weights. That reuse, called weight sharing, is the CNN’s built-in guess at work.

All numbers are made up and checked. The filters here are written by hand so you can see what they do; in a real CNN the filter weights are learned during training, and a layer has many filters at once.

Predict first

Before you open the lab: with the horizontal-edge filter, where in the output map will the biggest positive numbers be, and where the biggest negative ones?

Show the answer

That filter subtracts the top row of the window and adds the bottom row, so it gives big positive numbers where dark sits above bright: the top edge of the square (rows 0 and 1 of the output map, up to 27). It gives big negative numbers at the bottom edge, where bright sits above dark (rows 4 and 5, down to −27). Check it in the lab.

Lab

Convolution explorer

Pick a filter, then move the 3 × 3 window across the picture. At each position you see the nine multiplications, their total and where it lands in the output map. Press “Next position” until the map is full, or fill it at once.

A made-up 8 × 8 picture and hand-written filters, computed in your browser. No trained model runs here.

1 · Pick a filter
2 · Move the window

You can also click the input picture and use the arrow keys.

Input picture · 8 × 8
Filter · 3 × 3
Output map · 6 × 6

The explorer needs JavaScript, so here are the same numbers as tables. The input is an 8 × 8 gray picture (0 = dark, 9 = bright) with a bright square in the middle. A 3 × 3 filter slides over it; at each of the 6 × 6 positions you multiply the nine pixels by the nine filter numbers and add the products.

Input picture (made up)
RowCol 0Col 1Col 2Col 3Col 4Col 5Col 6Col 7
Row 000000000
Row 100000000
Row 200999900
Row 300999900
Row 400999900
Row 500999900
Row 600000000
Row 700000000

Vertical-edge filter: each row is −1, 0, 1. Horizontal-edge filter: rows −1, −1, −1 / 0, 0, 0 / 1, 1, 1. Blur: all nine weights are 1, then divide the total by 9.

Output map: vertical-edge filter
RowCol 0Col 1Col 2Col 3Col 4Col 5
Row 09900−9−9
Row 1181800−18−18
Row 2272700−27−27
Row 3272700−27−27
Row 4181800−18−18
Row 59900−9−9
Output map: horizontal-edge filter
RowCol 0Col 1Col 2Col 3Col 4Col 5
Row 09182727189
Row 19182727189
Row 2000000
Row 3000000
Row 4−9−18−27−27−18−9
Row 5−9−18−27−27−18−9
Output map: blur
RowCol 0Col 1Col 2Col 3Col 4Col 5
Row 0123321
Row 1246642
Row 2369963
Row 3369963
Row 4246642
Row 5123321

Try it on paper: put the vertical-edge filter with its top-left corner on row 1, column 4. The window covers columns 4, 5 and 6: pixels 9, 9, 0 in rows 2 and 3, and 0, 0, 0 in row 1. The result is 0 + (−9) + (−9) = −18, matching the table.

Notice that the output map is 6 × 6, smaller than the 8 × 8 input, because a 3 × 3 window fits in only six places across and six down. Real CNNs often pad the edges with zeros to keep the size. They stack many layers: early filters respond to edges, later ones combine edges into textures and shapes, much like the layers drawn on How AI works.

One more ingredient

Activation functions: the bend

Between layers, each neuron’s total passes through a simple function. Without it, depth would add nothing.

An activation function takes a neuron’s total and changes it in a fixed way. The most common today, ReLU, keeps positive numbers and turns negative ones into 0. Apply it to the vertical-edge map above and the left edge (27) survives while the right edge (−27) becomes 0: that neuron now reports only “dark-to-bright edges.”

Why bother? Multiplying and adding is a straight-line (linear) step, and two straight-line steps in a row are still one: “double it, then triple it” is just “multiply by 6.” Without a bend between layers, a 50-layer network could learn nothing that one layer could not. Another common choice, the S-shaped sigmoid, squashes any total into the range 0 to 1.

ReLU on one row of the map

before: 27, 27, 0, 0, −27, −27

after:  27, 27, 0, 0, 0, 0

ReLU(x) = the larger of x and 0. Row 2 of the vertical-edge output map.

Choosing a family

Not a ladder, a toolbox

Families appeared at different times, but later does not mean better for every job.

It is tempting to tell the story as a ladder: MLPs, then CNNs and RNNs, then transformers, each replacing the last. That is not how people choose. CNNs remain a common choice for images, especially on phones and cameras where memory and battery are limited. RNNs remain reasonable when data arrives one step at a time and the device is small. Transformers often win when there is a lot of data and computing power, but attention compares every token with every other: 1,000 tokens means 1,000 × 1,000 = 1,000,000 scores, and doubling the length to 2,000 quadruples that to 4,000,000.

Real systems also mix families. A speech system might turn sound into a picture-like grid for convolution layers, then pass the result to a transformer. A large language model can be a transformer whose thinking layers are a mixture of experts. In a mixture of experts with 8 experts and 2 chosen per token, only 2 ÷ 8 = 25% of the expert weights run for each token, although all of them must be stored.

Good questions to ask: What shape is the data? How many labeled examples are there? How fast must it answer, and on what device? Sometimes a simple model with no neural network at all is enough.

4 · Common mix-ups

What people get wrong about network families

Three ideas that trip people up, and what to think instead.

  • Mix-up: Each new family replaced the old ones, so the newest is always best.

    Better: Families suit different data and constraints. CNNs and RNNs are still used, and many systems combine families. Test options on your own data and device.

  • Mix-up: People design the filters inside a CNN.

    Better: The lab’s filters are hand-written for teaching. In a real CNN, training sets the filter weights, and they often end up detecting edges anyway, because edges are useful.

  • Mix-up: A mixture of experts is a team of separate chatbots that vote.

    Better: It is one network. Inside some layers, a router picks a few expert sub-networks for each token. The “experts” are not topic specialists you can name; they are blocks of weights that training shaped.

5 · Check yourself

Three quick questions

Answer on paper before you open each one.

  1. Why does a CNN need far fewer weights than an MLP to process a photo?

    Check your answer

    A CNN reuses the same small filter at every position (weight sharing). In the lab, one 3 × 3 filter has 9 weights for all 36 positions, while a fully connected layer from 64 pixels to 36 outputs needs 64 × 36 = 2,304. The CNN builds in the guess that a pattern means the same thing anywhere in the picture.

  2. A classmate says, “RNNs are obsolete because transformers replaced them.” How would you answer?

    Check your answer

    Transformers often do better when there is lots of data and computing power, and they can train faster because all positions are processed at once. But RNNs read one step at a time with a small running memory, which can suit streaming data on small devices. The choice depends on the data, the compute and the constraints.

  3. A mixture-of-experts layer has 16 experts and its router picks 2 for each token. What fraction of the expert weights runs for one token, and do the other experts still need to be stored?

    Check your answer

    2 ÷ 16 = 1/8, or 12.5%. Yes: all 16 experts must be stored, because the next token may be routed to different ones. That is why such models need lots of memory even though each token uses only part of the network.

In the real world

An application, a limit and a link

Application

A camera that spots weeds

A farm sprayer has to classify plants in camera images in a fraction of a second, on a computer bolted to the machine. A small CNN fits that job well. See the farming page and what camera-based AI measures.

Limitation

The family does not fix the data

Every family learns patterns from its training examples. A perfect choice of family cannot rescue blurry photos, missing groups or wrong labels. See What did we teach it by accident?

Connection

Vectors with meaning

Whatever the family, the layers turn inputs into lists of numbers. When those lists capture meaning, they are called embeddings. See How can meaning become a place on a map?

For younger learners

A grades 5–8 version

The convolution idea works on graph paper without any family names.

20 minutes · pairs

Edge finder on paper

  1. Draw a 6 × 6 grid. Shade a 3 × 3 block and write 1 in shaded squares and 0 in the rest.
  2. Cut a 3 × 3 window from card. Write −1, 0, 1 across each of its rows.
  3. Slide the window, multiply and add, and write each total in a new 4 × 4 grid.
  4. Where are the big numbers? (On the left and right sides of the block.)

Exit question

Say it in one sentence

“Why does a photo network use small sliding windows?” A good answer: “Because an edge looks the same anywhere in a picture, so one small pattern can be checked everywhere.” Younger classes can stop here and skip the family table.

Honest limits

What this page leaves out

What this page leaves out

How each family is trained (backpropagation through the layers, covered on Training vs using), pooling and stride in CNNs, how LSTMs keep information, positional encodings in transformers, autoencoders and generative families such as diffusion models, and the many hybrids researchers build. It also skips the mathematics that explains why these built-in guesses help.