AI foundations · 05
Learning now, or using what it learned?
Training is when a model’s internal numbers, its parameters, are adjusted to fit many examples. Inference is when the finished model, with those numbers frozen, turns a new input into an output. When you ask a chatbot a question, you are almost always doing inference: the model answers, and its parameters stay exactly as they were.
What you will be able to do
Say whether a situation is training, inference or retrieval, and describe one training step in four moves: predict, measure the loss, find the gradient, update.
Before you start
Stops 01 and 02 help. You should be comfortable with negative numbers and squaring a number. Tensors (stop 04) are useful but not needed.
1 · You have already seen this
The answer comes fast because the learning is done
Ask a chatbot a question and it starts answering right away.
That answer comes from a model that was trained earlier. When you ask something new, the app only runs the finished model, and the model stays the same however many questions it answers: its internal numbers stay exactly as they were. Photo apps that label your pictures work the same way.
These are two separate jobs. Training adjusts a model’s numbers, its parameters, to fit many examples. Inference uses the finished model, with those numbers frozen, to turn a new input into an output. The rest of this page shows how that works, lets you train a small model yourself, and shows where it goes wrong.
2 · What happens, step by step
Build it once, use it many times
Training and inference are separate jobs, often done on different computers at different times.
Training · learning now
Examples → training → saved model
- ExamplesInputs paired with the right answers: weeks 1 to 4 and the heights measured.
- Training loopPredict, measure the error (the loss), nudge the parameters, repeat many times.
- Saved modelThe final parameters are written to a file. Here that is just two numbers, w and b.
Inference · using what it learned
New input + saved model → output
- New inputA case the model never saw, such as week 5.
- Saved model, frozenThe same w and b, loaded from the file. Nothing changes them.
- OutputA prediction, such as about 11 cm. Run it a million times and w and b stay the same.
A model is like a recipe with adjustable numbers in it, called parameters. During training, a computer tries the recipe on examples where the right answer is known, measures how wrong it was, and adjusts the numbers a little. It repeats this many times, then saves the numbers.
During inference, the saved recipe is used as it is. A new input goes in, an output comes out, and the recipe does not change. A large language model may be trained for weeks or months on thousands of specialized chips; after that it can answer millions of questions a day without its parameters changing at all.
Think of a bakery. The head baker refines a cake recipe over many trial batches (training), then prints the final recipe card that every shift bakes from (inference). Nobody rewrites the card each time a customer orders a cake.
Going deeper · the training loop
Four moves, repeated
Here is the smallest model that still trains the way big neural networks do: a straight line with two parameters.
Imagine a made-up bean plant, measured once a week: 3 cm after week 1, 6 cm after week 2, 7 cm after week 3 and 9 cm after week 4. The model guesses height from the week with a line: height = w × week + b. The parameter w is the slope, the centimeters added per week, and b is where the line starts. Training means finding good values for w and b.
In neural networks, a number like b is called a bias. That is just a learned offset; it has nothing to do with unfairness. Each step of training makes the same four moves:
- PredictRun the model on the training examples with the current parameters.
- Measure the lossTurn “how wrong” into one number. Here: square each error, then average. That is the mean squared error (MSE). Squaring makes every error positive and punishes big misses most.
- Find the gradientFor each parameter, work out how steeply the loss rises if that parameter increases. Those slopes together are the gradient.
- UpdateMove each parameter a small step the opposite way, downhill. The learning rate sets how big the step is.
Worked example
One training step with real numbers
- Startw = 1, b = 0. Predictions for weeks 1 to 4: 1, 2, 3, 4 cm.
- ErrorsPrediction minus real height: 1 − 3 = −2, 2 − 6 = −4, 3 − 7 = −4, 4 − 9 = −5. Every guess is too low.
- Loss (MSE)Squares: 4, 16, 16, 25. Sum 61. Mean: 61 ÷ 4 = 15.25.
- GradientFor w, multiply each error by its week and add: −2 − 8 − 12 − 20 = −42. Then 2 × (−42) ÷ 4 = −21. For b, just add the errors: −15. Then 2 × (−15) ÷ 4 = −7.5. Both are negative, so making w or b bigger would lower the loss. (Why 2 ÷ 4? See the optional math below.)
- UpdateLearning rate 0.05. New w = 1 − 0.05 × (−21) = 2.05. New b = 0 − 0.05 × (−7.5) = 0.375.
- CheckNew predictions 2.425, 4.475, 6.525, 8.575. Errors −0.575, −1.525, −0.475, −0.425. Squares add to 0.330625 + 2.325625 + 0.225625 + 0.180625 = 3.0625, so the loss is 3.0625 ÷ 4 ≈ 0.77. One step cut the loss from 15.25 to 0.77.
Made-up measurements for a made-up plant. The best possible line for these four points is w = 1.9, b = 1.5, with loss 0.175. No straight line goes through all four points, so the loss can never reach 0.
Where the gradient formulas come from (optional, grades 9+)
With n examples, the loss is
L(w, b) = (1/n) · Σ (w·xi + b − yi)²
Take the derivative of each squared error with the chain rule: the outside gives 2 × (error), the inside gives xi for w and 1 for b. So
∂L/∂w = (2/n) · Σ (errori · xi) and ∂L/∂b = (2/n) · Σ errori
Gradient descent then updates both at once, with learning rate η:
w ← w − η · ∂L/∂w and b ← b − η · ∂L/∂b
For a network with millions of parameters nobody writes these by hand: backpropagation applies the same chain rule, layer by layer, automatically.
Backpropagation and the optimizer do different jobs
For a line, you can write the gradient by hand. A neural network can have billions of parameters arranged in layers, and each one affects the loss through every layer after it. Backpropagation is the method that computes all those gradients efficiently, working backward from the loss through the layers with the chain rule from calculus. It only calculates. It changes nothing.
The optimizer then uses the gradients to update the parameters. Plain gradient descent, as in the worked example, is the simplest optimizer; popular variants such as Adam adjust each parameter’s step using recent history. In PyTorch code you can see the split: loss.backward() computes the gradients and optimizer.step() applies the update. On How AI works this is walking downhill in fog: backpropagation feels the slope under your feet, the optimizer takes the step.
Big models take each step after a small batch of examples, not the whole dataset; one pass through all the training data is an epoch.
3 · Try it
Train a line, then freeze it
Fit the line by hand first. Then let gradient descent take steps, try a bigger learning rate, and finally freeze the model and use it.
Predict first
In the worked example the learning rate was 0.05. If you make it 0.2, four times bigger, will one step from w = 1, b = 0 lower the loss more, or less?
Show the answer
Neither: the loss goes up. The step is so big that it jumps past the best line to w = 5.2, b = 1.5, and the loss rises from 15.25 to 81.85. The next step overshoots even further (loss about 447). Try it in the lab, then press Reset. A moderately bigger rate, 0.1, still works and ends up a little faster than 0.05.
Lab
Train, then use
Move w and b to lower the loss, or press “Training step” to let gradient descent make one update and show its arithmetic. Change the learning rate to see small, good and too-big steps. Then press “Freeze and use” to predict new weeks without changing w or b.
Simulated in your browser with four made-up measurements. Plain arithmetic, no AI service.
Challenge: get the loss below 0.2 by hand.
| Week x | Real y | Predicted | Error | Error² |
|---|---|---|---|---|
| Mean of the squared errors (loss) | ||||
Inference: the model is frozen
w and b are locked. Pick a new week and the model predicts a height. Watch whether w or b change.
The lab needs JavaScript. Here are the numbers it would show, starting from w = 1, b = 0 with the four made-up measurements (week 1: 3 cm, week 2: 6 cm, week 3: 7 cm, week 4: 9 cm).
| Situation | w | b | Loss (MSE) | What it shows |
|---|---|---|---|---|
| Start | 1 | 0 | 15.25 | Every prediction is too low. |
| After one step, learning rate 0.05 | 2.05 | 0.375 | 0.766 | One careful step cuts the loss by about 95%. |
| After 10 steps, learning rate 0.05 | 2.213 | 0.580 | 0.316 | Progress slows as the slope flattens. |
| After one step, learning rate 0.2 | 5.2 | 1.5 | 81.85 | Too big a step overshoots: the loss goes up. |
| After two steps, learning rate 0.2 | −4.7 | −1.8 | 446.67 | Each step overshoots further: training blows up. |
| Best possible line | 1.9 | 1.5 | 0.175 | No straight line goes through all four points. |
Inference with the best line, frozen: week 5 → 1.9 × 5 + 1.5 = 11 cm; week 6 → 12.9 cm; week 50 → 96.5 cm. Making predictions never changes w or b. The week-50 answer is silly because the line only ever saw weeks 1 to 4.
Everyday AI
Does a chatbot learn from you while you chat?
Usually not in the training sense. Several different mechanisms can make it look as if it did.
Predict first
You tell a chat app that your favorite color is green. A week later, in a brand-new chat, it mentions green. Was the model retrained on your message?
Show the answer
Probably not. The likeliest explanation is saved memory: the app stored a note about you and quietly added it to the input of the new chat. The model’s parameters did not change. Apps with a memory feature usually let you see and delete what they saved.
| What happens | How it works | Do the model’s parameters change? | How long it lasts |
|---|---|---|---|
| Context | Your earlier messages in the conversation are sent to the model again with each new message, as part of its input (its context window). | No | Until the conversation ends or grows too long to fit |
| Saved memory | Some apps save notes about you (“likes green, learning Spanish”) in a database and add them to the input of later chats. | No | Until you or the app deletes the note |
| Retrieval | The system searches documents or the web and pastes the passages it finds into the input. See Retrieval and RAG. | No | Just for that answer |
| Later training | Depending on the company’s policy and your settings, saved conversations may become data for a future training run, which produces a new model version. | Yes, but in a separate training run, not during your chat | Until that version is replaced |
| Online learning exception | Some systems do update their parameters continually as new data arrives, for example some spam filters and recommendation feeds that adjust as people click. | Yes, in small, frequent updates | Ongoing |
Other learners
Not every model learns with gradients
Gradient descent trains neural networks and many linear models. Other machine learning methods learn differently, but they still have a training phase and a using phase.
Decision tree
Training picks questions
Training builds a series of yes-or-no questions, such as “Is the fruit heavier than 150 g?” At each branch it tries candidate questions and keeps the one that best separates the examples, by counting. No gradients and no learning rate. Inference: follow the questions down to an answer.
k-nearest neighbors
Training mostly stores
Training is mostly saving the examples. Inference does the work: find the k stored examples closest to the new input and let them vote, or average their values. Here using the model is the slow part, and adding a new example changes later answers without any gradient steps.
More on these on Do you always need a neural network?
In the real world
An application, a limit and a link
Application
Weather forecasts
Some newer forecasting models are trained once on decades of past weather, then run (inference) several times a day on the latest observations. Training takes weeks; each forecast takes minutes. See AI in weather.
Limitation
Frozen means stuck in the past
A frozen model knows only what its training data showed. The bean line predicts 96.5 cm at week 50, because a straight line is all it knows; nothing in weeks 1 to 4 said that plants stop growing. Chatbots likewise know nothing after their training data was collected unless it is put into their input.
Connection
Train, then test, with a class
In Train the Machine, students first train a creature classifier, then test it on new creatures: the same two phases. How AI works shows the loop inside a neural network.
4 · Common mix-ups
Learning and using are easy to blur
Four ideas that trip people up, and what to think instead.
Mix-up: Every chat message retrains the model.
Better: Chat messages become part of the input (the context). Parameters change only in a training run, which for big models happens separately and much later, if at all.
Mix-up: Backpropagation is how a network learns.
Better: Backpropagation only calculates gradients. The optimizer uses them to change the parameters. Training needs both, plus data and a loss.
Mix-up: “Inference” means the AI is reasoning or drawing conclusions.
Better: In machine learning, inference just means running a trained model on an input. (In statistics the word means drawing conclusions from data, which is different again.)
Mix-up: A lower training loss always means a better model.
Better: Loss on the training examples can keep falling while the model gets worse on new data (overfitting). Test on examples it never trained on; see Evaluating AI.
5 · Check yourself
Training, inference or retrieval?
Label each scenario before you open the answer. Some have more than one label.
A photo app labels your new picture “dog” half a second after you take it.
Check your answer
Inference. A model trained earlier runs on your new photo. Its parameters do not change, however many photos you label.
A lab runs thousands of chips for weeks, repeatedly adjusting a model’s parameters to lower its loss on billions of text examples.
Check your answer
Training. Parameters change, guided by a loss and its gradients. The result is a saved model that can then be used for inference.
A school help bot searches the student handbook, finds the paragraph on late homework, and answers using it.
Check your answer
Retrieval and inference. The search step retrieves a passage; then the model runs (inference) with that passage in its input. No training happens: when the handbook changes, the bot learns about it through retrieval, not by changing parameters. See Retrieval and RAG.
For younger learners
A grades 3–5 version
No loss or gradients: the difference between working out a rule and using it.
15 minutes · small groups
The sealed rule card
- Show number pairs: 2 → 5, 3 → 7, 4 → 9. Groups guess the rule and test it on the pairs, changing it until it works (training).
- Each group writes its final rule on a card and seals it in an envelope.
- Call out new numbers, such as 10 and 0. Groups may answer only by using the card (inference). The rule “double it and add 1” gives 21 and 1.
- Ask: if your card gives a wrong answer, may you change it during the quiz? (Not without going back to training.)
Exit question
Learning or using?
“When were you learning the rule, and when were you only using it?” Good answers name both moments: “We learned it while we tested the pairs. We used it when we answered from the card.”
Honest limits
What this page leaves out
What this page leaves out
Mini-batches and epochs in detail, optimizers such as Adam, how to decide when to stop training (validation data), the later training stages that turn a language model into an assistant (see how assistants are trained), and how models are deployed, monitored and updated. Real neural networks also have many layers, nonlinear functions and far more data than one line.
Where to go next
Is the trained model any good?
Next stop · 06
Good at what, for whom, compared with what?
How to test a trained model fairly: unseen examples, baselines and errors that matter.
The neural version
How AI works
Neurons, layers and the same training loop inside a neural network.
Classroom lab
Train the Machine
Train a creature classifier, then test it, and find the misleading pattern it learned.
Sources
Where this comes from
- Google for Developers, Machine Learning Crash Course, “Linear regression: Loss” (squared error and MSE).
- Google for Developers, Machine Learning Crash Course, “Linear regression: Gradient descent.”
- Google for Developers, Machine Learning Crash Course, “Linear regression: Hyperparameters” (learning rate, batch size, epochs).
- Google for Developers, Machine Learning Crash Course, “Neural networks: Training using backpropagation.”
- PyTorch, “Optimizing Model Parameters”: loss.backward() computes gradients, optimizer.step() updates parameters.
- scikit-learn user guide: Decision Trees and Nearest Neighbors (methods that do not train with gradient descent).
AI words on this page
Training data, Parameters, Inference, Loss function, Gradient descent, Backpropagation, Learning rate, Batch, Epoch, Context window, RAG, Decision tree, k-nearest neighbors, Overfitting
All definitions live in one place: AI words in plain language →
Review status
Written by ARQIV. Last checked October 2026. Independent subject review and classroom pilots are still to come, so treat the activities as drafts and tell us about anything that is wrong or unclear.