Go deeper · How machines learn
Why are there different kinds of neural networks?
Because data comes in different shapes. A list of measurements, a photo, a sentence and a molecule are arranged in different ways, and each family of networks builds in a guess about that arrangement. A good guess means the network needs less data and computing to learn. No family is best at everything, and the newest is not automatically the right one.
What you will be able to do
Match six neural-network families (MLP, CNN, RNN, transformer, GNN and mixture of experts) to the kind of structure each one suits, compute one step of a convolution by hand, and explain why the choice depends on data, compute and constraints rather than on which family is newest.
Before you start
Before this: How AI works (neurons, weights and layers) and How does a picture become numbers? (pictures as grids). Helpful: What is a transformer?
1 · You have already seen this
Different things, read in different ways
You already read different kinds of things in different ways, without thinking about it.
You read a bus timetable across and down, and a text message one word after another, in order. In a photo you look for edges and shapes wherever they happen to be. A family tree you follow along the lines, from one person to the people linked to them.
Computers meet the same problem. A list of measurements, a photo, a sentence and a molecule are arranged in different ways, so researchers wire neural networks in different patterns, called families, each with a built-in guess about how its data is laid out. This page shows how that works, lets you try it, and shows where it goes wrong.
2 · What happens, step by step
From data to a working network
Choosing and running a network in five steps. The next section goes deeper into steps 1 and 2, and the lab lets you watch step 3 inside a CNN. After the lab, one section goes deeper into the bend in step 3 and another into step 5.
How a network family is used
Five steps
- 1 · Look at the dataAsk how it is arranged: a fixed list of measurements, a grid like a photo, a sequence like a sentence, or things linked to other things, like atoms in a molecule.
- 2 · Wire to matchEvery family uses the same neuron, which multiplies inputs by weights and adds them up. The family decides how the neurons connect, and that wiring is its built-in guess about the data.
- 3 · Multiply, add, bendEach layer multiplies and adds. Then each total passes through a simple function such as ReLU, which keeps positive numbers and turns negative ones into 0.
- 4 · Learn the weightsDuring training, the weights are adjusted using examples. In a CNN that includes the numbers inside the filters.
- 5 · Compare optionsTry more than one family, or a simple model with no neural network at all, on your own data and on the device that has to run it.
Going deeper · steps 1 and 2
Six families, six kinds of structure
Every family uses the same building block: a neuron that multiplies inputs by weights, adds them up and passes the total on. What changes is how the neurons are wired together.
If every neuron may connect to every input, a network can in principle learn anything, but it needs a huge number of weights and a lot of examples to find the useful ones.
So researchers wire neurons in patterns that match how the data is arranged. That built-in guess is called an inductive bias (a different meaning of “bias” from unfairness). When the guess fits the data, the network learns faster from fewer examples. When it does not fit, it can get in the way.
Fixed-size lists
MLP
A multilayer perceptron: layers in which every neuron connects to every number in the layer before. It suits a fixed list of measurements, such as 12 readings about a plant.
Built-in guess: almost none. The order of the inputs means nothing, so it has to learn every relationship from data.
Grids
CNN
A convolutional neural network slides small filters across a grid, such as a photo or a picture of sound. You can try one filter in the lab below.
Built-in guess: nearby pixels belong together, and an edge means the same thing wherever it appears.
Sequences, step by step
RNN
A recurrent neural network reads one step at a time, such as one sensor reading or one word, and carries a running summary (its “memory”) to the next step. LSTM and GRU are popular kinds.
Built-in guess: order matters, and the past can be squeezed into a summary.
Any part to any part
Transformer
A transformer uses attention: every token gets a score with every other token, so far-apart words can connect in one layer. Some vision models cut a picture into patches and treat them as tokens.
Built-in guess: very little about position; the data decides which parts to connect.
Graphs
GNN
A graph neural network works on things (nodes) and links (edges): atoms and bonds in a molecule, or people and friendships. Each node updates its numbers using messages from its neighbors.
Built-in guess: what matters is who is linked to whom, not a place on a grid.
Routing to specialists
Mixture of experts
Not a data shape but a way to organize a big network, usually a transformer. A small router sends each token to a few of many “expert” sub-networks, for example 2 out of 8, so only part of the network runs each time.
Built-in guess: different inputs are best handled by different parts.
Predict first
Which family would you try first for each job? (a) Predict how long a delivery takes from 15 numbers about the order. (b) Spot cracks in photos of a bridge. (c) Predict whether a new molecule dissolves in water.
Show the answer
(a) An MLP, or a classical model such as a decision tree: the data is a fixed list of features. On tables like this, classical models are often just as good and easier to check. (b) A CNN (or a vision transformer): cracks are local patterns that can appear anywhere in a grid. (c) A GNN: a molecule is atoms linked by bonds, which is a graph. In every case you would test more than one option on your own data.
3 · Try it
One small filter, many positions
A convolution slides a 3 × 3 grid of weights across the picture. At each stop it multiplies and adds, and the result becomes one number in a new grid called a feature map.
Worked example
The vertical-edge filter at one position
- InputA made-up 8 × 8 gray picture: 0 is dark, 9 is bright. A bright 4 × 4 square fills rows 2 to 5 and columns 2 to 5 (counting from 0).
- FilterEvery row of the filter is −1, 0, 1: it subtracts the left column and adds the right column.
- Place itPut the filter’s top-left corner on row 2, column 1. The window covers columns 1, 2 and 3, so each of its three rows reads 0, 9, 9.
- Multiply and addOne row: 0 × (−1) + 9 × 0 + 9 × 1 = 9. Three identical rows: 9 + 9 + 9 = 27. That 27 goes into the feature map at row 2, column 1.
- Read itA big positive number means dark on the left and bright on the right: the left edge of the square. Move the window to row 2, column 4 and you get −27, the right edge. Over flat areas the result is 0.
- Count weightsThis filter has 9 weights, reused at all 36 positions. A fully connected layer from the 64 pixels to 36 outputs would need 64 × 36 = 2,304 weights. That reuse, called weight sharing, is the CNN’s built-in guess at work.
All numbers are made up and checked. The filters here are written by hand so you can see what they do; in a real CNN the filter weights are learned during training, and a layer has many filters at once.
Predict first
Before you open the lab: with the horizontal-edge filter, where in the output map will the biggest positive numbers be, and where the biggest negative ones?
Show the answer
That filter subtracts the top row of the window and adds the bottom row, so it gives big positive numbers where dark sits above bright: the top edge of the square (rows 0 and 1 of the output map, up to 27). It gives big negative numbers at the bottom edge, where bright sits above dark (rows 4 and 5, down to −27). Check it in the lab.
Lab
Convolution explorer
Pick a filter, then move the 3 × 3 window across the picture. At each position you see the nine multiplications, their total and where it lands in the output map. Press “Next position” until the map is full, or fill it at once.
A made-up 8 × 8 picture and hand-written filters, computed in your browser. No trained model runs here.
You can also click the input picture and use the arrow keys.
The explorer needs JavaScript, so here are the same numbers as tables. The input is an 8 × 8 gray picture (0 = dark, 9 = bright) with a bright square in the middle. A 3 × 3 filter slides over it; at each of the 6 × 6 positions you multiply the nine pixels by the nine filter numbers and add the products.
| Row | Col 0 | Col 1 | Col 2 | Col 3 | Col 4 | Col 5 | Col 6 | Col 7 |
|---|---|---|---|---|---|---|---|---|
| Row 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Row 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Row 2 | 0 | 0 | 9 | 9 | 9 | 9 | 0 | 0 |
| Row 3 | 0 | 0 | 9 | 9 | 9 | 9 | 0 | 0 |
| Row 4 | 0 | 0 | 9 | 9 | 9 | 9 | 0 | 0 |
| Row 5 | 0 | 0 | 9 | 9 | 9 | 9 | 0 | 0 |
| Row 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Row 7 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Vertical-edge filter: each row is −1, 0, 1. Horizontal-edge filter: rows −1, −1, −1 / 0, 0, 0 / 1, 1, 1. Blur: all nine weights are 1, then divide the total by 9.
| Row | Col 0 | Col 1 | Col 2 | Col 3 | Col 4 | Col 5 |
|---|---|---|---|---|---|---|
| Row 0 | 9 | 9 | 0 | 0 | −9 | −9 |
| Row 1 | 18 | 18 | 0 | 0 | −18 | −18 |
| Row 2 | 27 | 27 | 0 | 0 | −27 | −27 |
| Row 3 | 27 | 27 | 0 | 0 | −27 | −27 |
| Row 4 | 18 | 18 | 0 | 0 | −18 | −18 |
| Row 5 | 9 | 9 | 0 | 0 | −9 | −9 |
| Row | Col 0 | Col 1 | Col 2 | Col 3 | Col 4 | Col 5 |
|---|---|---|---|---|---|---|
| Row 0 | 9 | 18 | 27 | 27 | 18 | 9 |
| Row 1 | 9 | 18 | 27 | 27 | 18 | 9 |
| Row 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Row 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| Row 4 | −9 | −18 | −27 | −27 | −18 | −9 |
| Row 5 | −9 | −18 | −27 | −27 | −18 | −9 |
| Row | Col 0 | Col 1 | Col 2 | Col 3 | Col 4 | Col 5 |
|---|---|---|---|---|---|---|
| Row 0 | 1 | 2 | 3 | 3 | 2 | 1 |
| Row 1 | 2 | 4 | 6 | 6 | 4 | 2 |
| Row 2 | 3 | 6 | 9 | 9 | 6 | 3 |
| Row 3 | 3 | 6 | 9 | 9 | 6 | 3 |
| Row 4 | 2 | 4 | 6 | 6 | 4 | 2 |
| Row 5 | 1 | 2 | 3 | 3 | 2 | 1 |
Try it on paper: put the vertical-edge filter with its top-left corner on row 1, column 4. The window covers columns 4, 5 and 6: pixels 9, 9, 0 in rows 2 and 3, and 0, 0, 0 in row 1. The result is 0 + (−9) + (−9) = −18, matching the table.
Notice that the output map is 6 × 6, smaller than the 8 × 8 input, because a 3 × 3 window fits in only six places across and six down. Real CNNs often pad the edges with zeros to keep the size. They stack many layers: early filters respond to edges, later ones combine edges into textures and shapes, much like the layers drawn on How AI works.
One more ingredient
Activation functions: the bend
Between layers, each neuron’s total passes through a simple function. Without it, depth would add nothing.
An activation function takes a neuron’s total and changes it in a fixed way. The most common today, ReLU, keeps positive numbers and turns negative ones into 0. Apply it to the vertical-edge map above and the left edge (27) survives while the right edge (−27) becomes 0: that neuron now reports only “dark-to-bright edges.”
Why bother? Multiplying and adding is a straight-line (linear) step, and two straight-line steps in a row are still one: “double it, then triple it” is just “multiply by 6.” Without a bend between layers, a 50-layer network could learn nothing that one layer could not. Another common choice, the S-shaped sigmoid, squashes any total into the range 0 to 1.
ReLU on one row of the map
before: 27, 27, 0, 0, −27, −27
after: 27, 27, 0, 0, 0, 0
ReLU(x) = the larger of x and 0. Row 2 of the vertical-edge output map.
Choosing a family
Not a ladder, a toolbox
Families appeared at different times, but later does not mean better for every job.
It is tempting to tell the story as a ladder: MLPs, then CNNs and RNNs, then transformers, each replacing the last. That is not how people choose. CNNs remain a common choice for images, especially on phones and cameras where memory and battery are limited. RNNs remain reasonable when data arrives one step at a time and the device is small. Transformers often win when there is a lot of data and computing power, but attention compares every token with every other: 1,000 tokens means 1,000 × 1,000 = 1,000,000 scores, and doubling the length to 2,000 quadruples that to 4,000,000.
Real systems also mix families. A speech system might turn sound into a picture-like grid for convolution layers, then pass the result to a transformer. A large language model can be a transformer whose thinking layers are a mixture of experts. In a mixture of experts with 8 experts and 2 chosen per token, only 2 ÷ 8 = 25% of the expert weights run for each token, although all of them must be stored.
Good questions to ask: What shape is the data? How many labeled examples are there? How fast must it answer, and on what device? Sometimes a simple model with no neural network at all is enough.
4 · Common mix-ups
What people get wrong about network families
Three ideas that trip people up, and what to think instead.
Mix-up: Each new family replaced the old ones, so the newest is always best.
Better: Families suit different data and constraints. CNNs and RNNs are still used, and many systems combine families. Test options on your own data and device.
Mix-up: People design the filters inside a CNN.
Better: The lab’s filters are hand-written for teaching. In a real CNN, training sets the filter weights, and they often end up detecting edges anyway, because edges are useful.
Mix-up: A mixture of experts is a team of separate chatbots that vote.
Better: It is one network. Inside some layers, a router picks a few expert sub-networks for each token. The “experts” are not topic specialists you can name; they are blocks of weights that training shaped.
5 · Check yourself
Three quick questions
Answer on paper before you open each one.
Why does a CNN need far fewer weights than an MLP to process a photo?
Check your answer
A CNN reuses the same small filter at every position (weight sharing). In the lab, one 3 × 3 filter has 9 weights for all 36 positions, while a fully connected layer from 64 pixels to 36 outputs needs 64 × 36 = 2,304. The CNN builds in the guess that a pattern means the same thing anywhere in the picture.
A classmate says, “RNNs are obsolete because transformers replaced them.” How would you answer?
Check your answer
Transformers often do better when there is lots of data and computing power, and they can train faster because all positions are processed at once. But RNNs read one step at a time with a small running memory, which can suit streaming data on small devices. The choice depends on the data, the compute and the constraints.
A mixture-of-experts layer has 16 experts and its router picks 2 for each token. What fraction of the expert weights runs for one token, and do the other experts still need to be stored?
Check your answer
2 ÷ 16 = 1/8, or 12.5%. Yes: all 16 experts must be stored, because the next token may be routed to different ones. That is why such models need lots of memory even though each token uses only part of the network.
In the real world
An application, a limit and a link
Application
A camera that spots weeds
A farm sprayer has to classify plants in camera images in a fraction of a second, on a computer bolted to the machine. A small CNN fits that job well. See the farming page and what camera-based AI measures.
Limitation
The family does not fix the data
Every family learns patterns from its training examples. A perfect choice of family cannot rescue blurry photos, missing groups or wrong labels. See What did we teach it by accident?
Connection
Vectors with meaning
Whatever the family, the layers turn inputs into lists of numbers. When those lists capture meaning, they are called embeddings. See How can meaning become a place on a map?
For younger learners
A grades 5–8 version
The convolution idea works on graph paper without any family names.
20 minutes · pairs
Edge finder on paper
- Draw a 6 × 6 grid. Shade a 3 × 3 block and write 1 in shaded squares and 0 in the rest.
- Cut a 3 × 3 window from card. Write −1, 0, 1 across each of its rows.
- Slide the window, multiply and add, and write each total in a new 4 × 4 grid.
- Where are the big numbers? (On the left and right sides of the block.)
Exit question
Say it in one sentence
“Why does a photo network use small sliding windows?” A good answer: “Because an edge looks the same anywhere in a picture, so one small pattern can be checked everywhere.” Younger classes can stop here and skip the family table.
Honest limits
What this page leaves out
What this page leaves out
How each family is trained (backpropagation through the layers, covered on Training vs using), pooling and stride in CNNs, how LSTMs keep information, positional encodings in transformers, autoencoders and generative families such as diffusion models, and the many hybrids researchers build. It also skips the mathematics that explains why these built-in guesses help.
Where to go next
Follow the numbers
Next in this branch
How sure should an AI be?
Probabilities, base rates and why a confident output is not the same as a correct one.
One family up close
What is a transformer?
Tokens, embeddings and attention, in five steps.
The data underneath
How does a picture become numbers?
Shapes, axes and batches: what every family takes as input.
Newer shapes
Inside a large language model
Mixture of experts, Mamba state-space models, diffusion language models and small language models compared.
Sources
Where this comes from
- Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning (MIT Press, 2016), free online: chapter 6 (feedforward networks and activation functions), chapter 9 (convolutional networks) and chapter 10 (recurrent networks).
- Stanford CS231n course notes, “Convolutional Neural Networks”: filters, weight sharing, padding and how output size is computed.
- Google for Developers, Machine Learning Crash Course: Neural networks, including activation functions such as ReLU and sigmoid.
- Ashish Vaswani and colleagues, “Attention Is All You Need,” 2017: the transformer.
- Benjamin Sanchez-Lengeling and colleagues, “A Gentle Introduction to Graph Neural Networks,” Distill, 2021.
- Noam Shazeer and colleagues, “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” 2017: a router that picks a few experts per input.
AI words on this page
Neural network, Deep learning, Activation function, Convolutional neural network, Recurrent neural network, Transformer, Attention, Graph neural network, Mixture of experts, Parameters, Compute, Tensor
All definitions live in one place: AI words in plain language →
Review status
Written by ARQIV. Last checked October 2026. Independent subject review and classroom pilots are still to come, so treat the activities as drafts and tell us about anything that is wrong or unclear.