# Advanced learning approaches

- Link: https://arqiv.si/ai/advanced-learning/
- Published: 2026-10-10T18:31:39-04:00
- Author: Steven Ryan

[Skip to content](https://arqiv.si/ai/advanced-learning/#main-content)

[arqiv](https://arqiv.si/)

[For educators](https://arqiv.si/teachers/)

Teach

 * [Start here](https://arqiv.si/teachers/)
 * [Lessons & printable kits](https://arqiv.si/teachers/resources/)
 * [Classroom labs](https://arqiv.si/teachers/labs/)
 * [Free 20-minute lesson](https://arqiv.si/teachers/free-ai-lesson/)

Plan

 * [Curriculum guide](https://arqiv.si/teachers/curriculum/)
 * [AI Learning Center](https://arqiv.si/teachers/ai-literacy/)
 * [Future of work unit](https://arqiv.si/future-of-work/)
 * [What the research says](https://arqiv.si/teachers/what-works/)
 * [AI education worldwide](https://arqiv.si/teachers/around-the-world/)

Understand

 * [Think First method](https://arqiv.si/teachers/think-first/)
 * [Our approach](https://arqiv.si/research/#our-method)
 * [AI vocabulary](https://arqiv.si/teachers/ai-vocabulary/)

By role

 * [Counselors](https://arqiv.si/counselors/)
 * [School leaders & PD](https://arqiv.si/schools/)
 * [Speaking & workshops](https://arqiv.si/speaking/)
 * [Products](https://arqiv.si/products/)

[Learn AI](https://arqiv.si/ai/topics/)

Start

 * [All AI topics](https://arqiv.si/ai/topics/)
 * [The AI journey, from zero](https://arqiv.si/ai/)
 * [Using AI well](https://arqiv.si/ai/talking-to-ai/)
 * [AI builds](https://arqiv.si/ai-builds/)

AI in the world

 * [Space](https://arqiv.si/space/)
 * [Weather](https://arqiv.si/weather/)
 * [Health](https://arqiv.si/health/)
 * [Engineering](https://arqiv.si/engineering/)
 * [Farming](https://arqiv.si/farming/)
 * [Quantum](https://arqiv.si/quantum/)

What it means

 * [Future of work](https://arqiv.si/future-of-work/)
 * [Your AI path](https://arqiv.si/ai/your-path/)
 * [Research, explained](https://arqiv.si/research/)

[Games & labs](https://arqiv.si/games/)[Research](https://arqiv.si/research/)[Blog](https://arqiv.si/blog/)
[About](https://arqiv.si/about/)

[Search or ask](https://arqiv.si/ai/topics/)[Newsletter](https://arqiv.si/newsletter/)
[ARQIV.FM ↗](https://arqiv.fm/)

Menu

For educators

[For educators: start page →](https://arqiv.si/teachers/)

Teach

 * [Start here](https://arqiv.si/teachers/)
 * [Lessons & printable kits](https://arqiv.si/teachers/resources/)
 * [Classroom labs](https://arqiv.si/teachers/labs/)
 * [Free 20-minute lesson](https://arqiv.si/teachers/free-ai-lesson/)

Plan

 * [Curriculum guide](https://arqiv.si/teachers/curriculum/)
 * [AI Learning Center](https://arqiv.si/teachers/ai-literacy/)
 * [Future of work unit](https://arqiv.si/future-of-work/)
 * [What the research says](https://arqiv.si/teachers/what-works/)
 * [AI education worldwide](https://arqiv.si/teachers/around-the-world/)

Understand

 * [Think First method](https://arqiv.si/teachers/think-first/)
 * [Our approach](https://arqiv.si/research/#our-method)
 * [AI vocabulary](https://arqiv.si/teachers/ai-vocabulary/)

By role

 * [Counselors](https://arqiv.si/counselors/)
 * [School leaders & PD](https://arqiv.si/schools/)
 * [Speaking & workshops](https://arqiv.si/speaking/)
 * [Products](https://arqiv.si/products/)

Learn AI

[Learn AI: start page →](https://arqiv.si/ai/topics/)

Start

 * [All AI topics](https://arqiv.si/ai/topics/)
 * [The AI journey, from zero](https://arqiv.si/ai/)
 * [Using AI well](https://arqiv.si/ai/talking-to-ai/)
 * [AI builds](https://arqiv.si/ai-builds/)

AI in the world

 * [Space](https://arqiv.si/space/)
 * [Weather](https://arqiv.si/weather/)
 * [Health](https://arqiv.si/health/)
 * [Engineering](https://arqiv.si/engineering/)
 * [Farming](https://arqiv.si/farming/)
 * [Quantum](https://arqiv.si/quantum/)

What it means

 * [Future of work](https://arqiv.si/future-of-work/)
 * [Your AI path](https://arqiv.si/ai/your-path/)
 * [Research, explained](https://arqiv.si/research/)

 * [Games & labs](https://arqiv.si/games/)
 * [Research](https://arqiv.si/research/)
 * [Blog](https://arqiv.si/blog/)
 * [About](https://arqiv.si/about/)

[Get the newsletter](https://arqiv.si/newsletter/)

 1. [ARQIV / ADVANCED TRACK](https://arqiv.si/ai/topics/)
 2. [**01**Compute & efficiency](https://arqiv.si/ai/compute-and-efficiency/)
 3. [**02**Deploy & maintain](https://arqiv.si/ai/model-lifecycle/)
 4. [**03**Generative families](https://arqiv.si/ai/generative-models/)
 5. [**04**Knowledge & causality](https://arqiv.si/ai/knowledge-and-causality/)
 6. [**05**Robotics](https://arqiv.si/ai/robotics/)
 7. [**06**Security & provenance](https://arqiv.si/ai/security-and-provenance/)
 8. [**07**Advanced learning](https://arqiv.si/ai/advanced-learning/)

Go deeper · Advanced track

# How can models learn with _less data, less sharing_ or without forgetting?

By changing how training is set up, not only how big it is. A model can reuse what
another model already learned, ask people to label just the examples it is least
sure about, learn something new while protecting old skills, or train across many
devices without collecting the raw data in one place. Automated search can even 
help choose the model itself.

Grades 9+25 minutesAverages onlyOptional: noise and privacy

This is an optional advanced page. You do not need it for the main foundations route.

What you will be able to do

Match transfer, active, continual and federated learning, evolutionary search and
AutoML to the problem each one solves; spend a small label budget wisely; compute
a federated average by hand; and explain why “federated” does not automatically 
mean “private.”

Before you start

Read first: [Learning now, or using what it learned?](https://arqiv.si/ai/training-and-inference/),
[Do you always need a neural network?](https://arqiv.si/ai/classical-machine-learning/)
and [Can you build and test a model yourself?](https://arqiv.si/ai/build-a-model/)

1 · You have already seen this

## What you bring _to a new job_

Most people have started something new without starting from nothing.

On a first day at a new job, you bring skills from the last one, so you learn faster
than someone who never did similar work.

Models can be trained in similar ways. This page covers **transfer learning**, **
active learning**, continual learning, federated learning and automated search: 
ways for a model to learn with less data, less sharing or without forgetting. It
shows how they work, lets you try one, and shows where each goes wrong.

2 · What happens, step by step

## Training, _one choice at a time_

Training a model means making five choices, and each method on this page changes
one of them. The next section sets them side by side for one garden app, and the
one after goes deeper into step 1. The lab lets you try step 2, and the three sections
after it go deeper into steps 5, 3 and 4.

How a model gets trained

### Five choices

 1. 1 · Where to startTraining from scratch needs a lot of labeled data. **Transfer
    learning** starts from a model already trained on another task and adjusts it with
    your few examples, which is called **fine-tuning**.
 2. 2 · Which examples to labelEach label costs a person’s time. In **active learning**
    the model asks for labels on the examples it is least sure about.
 3. 3 · Where the data livesUsually all the data is collected in one place. In **federated
    learning** each device or site trains on its own data and sends back only model
    updates, which a server averages.
 4. 4 · Which model and settingsSomeone has to pick the kind of model and settings 
    such as the learning rate. Evolutionary methods and **AutoML** search for them:
    try, score, keep the best, change, repeat.
 5. 5 · What happens next timeNew data keeps arriving after training. **Continual learning**
    teaches the model new things while protecting what it learned before.

Five problems, one app

## One garden app, _five problems_

Before any new words, here is a situation that needs all of them. The garden and
its numbers are made up.

A school garden club wants a phone app that spots ripe tomatoes in photos. They 
have 60 labeled photos, far too few to train an image model from scratch, plus 2,000
unlabeled ones, and volunteers can label perhaps 100. Two other schools want to 
help, but their photos show students in the background and must not leave the school.
Next spring the app should learn peppers without getting worse at tomatoes. And 
nobody is sure which model or settings to use.

Each problem has a family of methods. None is magic: each moves the work somewhere
else, and the table says where.

Five problems, six setups (the garden is made up)| The problem | Training setup | Core idea | Main catch | 
| Too few labeled photos | Transfer learning, then fine-tuning | Start from a model already trained on millions of other images and adjust it with your few photos | Helps most when the old and new tasks look alike | 
| Labeling takes people’s time | Active learning | The model asks for labels on the examples it is least sure about | The chosen examples are not a fair sample, so test on separate random data | 
| New lessons push out old skills | Continual learning | Learn new data while protecting what was learned before | Still an open research problem for large models | 
| The data must stay where it is | Federated learning | Each site trains locally and sends only model updates, which a server averages | Updates can still leak information: not automatically private | 
| Which model and settings? | Evolutionary methods and AutoML | Automated search: try, score, keep the best, change, repeat | People still choose the goal and the score, and check the data |

Going deeper · step 1

## Don’t start _from zero_

A model that has already learned edges, colors and shapes from millions of photos
knows most of what a tomato spotter needs.

**Transfer learning** means reusing what a model learned on one task for another.
Take an image model that was pretrained on a large, general photo collection. Keep
its early layers, which respond to edges, textures and shapes, and replace its last
layer with a new one for your own labels: “ripe” and “not ripe.”

Then comes **fine-tuning**: a short extra round of training on your small dataset,
usually with a small [learning rate](https://arqiv.si/ai/training-and-inference/)
so the useful old knowledge is nudged rather than overwritten. Sometimes only the
new last layer is trained and everything else stays frozen. PyTorch’s official tutorial
builds an ants-versus-bees classifier this way from about 120 training photos of
each.

Chatbots are built this way too ([pretrained, then fine-tuned](https://arqiv.si/ai/transformers/#assistant)).
Transfer helps most when the old and new tasks share structure, and the old model’s
blind spots and biases come along with its skills.

3 · Try it

## Ask for the labels _that teach the most_

Labels cost people’s time. If you can afford only a few, which examples should you
ask about?

In **active learning** the model helps choose its own training data. It looks at
the unlabeled pool, finds the examples it is least sure about, asks a person to 
label those, retrains and asks again. Burr Settles’s 2009 survey describes this 
strategy, uncertainty sampling, as the simplest and most commonly used.

Why would the unsure examples be better? A photo of a deep red tomato teaches the
model little it does not already know. A tomato right on the border between orange
and red pins down where the line goes.

Predict first

Twelve tomatoes have a redness score from 0 to 100. You know the palest one (6) 
is not ripe and the reddest (93) is ripe. You may ask about four more. Should you
spread your four questions evenly, or keep asking about the tomato in the middle
of what you still don’t know?

Show the answer

Keep asking in the middle. Each answer cuts the zone where the hidden boundary could
be, like the number-guessing game “higher or lower.” In the lab, four well-placed
questions shrink the zone from 87 points wide (6 to 93) to 4 points (50 to 54). 
Questions far from the boundary spend the budget confirming what the model already
guessed.

Lab

### Spend a budget of four labels

Each label you ask for costs a volunteer’s time. Choose which tomatoes to ask about
and watch the model’s rule, its unsure zone and how many of the 12 it sorts correctly(
checked against hidden answers).

 Simulated in your browser with 12 made-up tomatoes. Nothing is learned from you
or stored.

Or ask about a tomato yourself (each unlabeled tomato shows the model’s current 
guess):

Questions asked so far:

Three challenges

 1. Press “next photo in order” four times. How wide is the unsure zone, and how many
    tomatoes does the rule get wrong?
 2. Reset and press “where the model is least sure” four times. Compare. Why is each
    question worth more?
 3. Reset and choose four tomatoes yourself. Can you beat the model’s strategy? Can
    you think of a case where asking only near the rule would mislead it?

Twelve made-up tomatoes have redness scores 6, 12, 19, 27, 35, 46, 50, 54, 58, 61,
78 and 93. The hidden rule is “ripe from 52.” The model starts knowing only that
6 is not ripe and 93 is ripe, and may ask about four more. Its rule is “ripe from
the midpoint between the reddest tomato known to be unripe and the palest tomato
known to be ripe.”

Strategy 1: ask in the order the photos were taken (19, 61, 12, 27, …)| Question | Asked about | Answer | Model’s rule | Unsure zone | Sorted correctly | 
| 1 | 19 | not ripe | ripe from 56 | 19 to 93 (74 wide) | 11 of 12 | 
| 2 | 61 | ripe | ripe from 40 | 19 to 61 (42 wide) | 10 of 12 | 
| 3 | 12 | not ripe | ripe from 40 | 19 to 61 (42 wide) | 10 of 12 | 
| 4 | 27 | not ripe | ripe from 44 | 27 to 61 (34 wide) | 10 of 12 |

Strategy 2: always ask about the tomato closest to the current rule| Question | Asked about | Answer | Model’s rule | Unsure zone | Sorted correctly | 
| 1 | 50 | not ripe | ripe from 71.5 | 50 to 93 (43 wide) | 9 of 12 | 
| 2 | 78 | ripe | ripe from 64 | 50 to 78 (28 wide) | 9 of 12 | 
| 3 | 61 | ripe | ripe from 55.5 | 50 to 61 (11 wide) | 11 of 12 | 
| 4 | 54 | ripe | ripe from 52 | 50 to 54 (4 wide) | 12 of 12 |

The same budget of four labels leaves a zone 34 points wide and 2 tomatoes wrong
in photo order, but only 4 points wide and none wrong when the model asks where 
it is least sure. Labeling all ten would give the same rule as the four well-chosen
questions.

**The catch.** The examples active learning picks are deliberately unusual, so they
are not a fair sample of the world. A model that only asks near its current rule
can also miss a second region it never thought to doubt. Always test on a separate,
randomly chosen set, as on [Build a small model](https://arqiv.si/ai/build-a-model/).

Continual learning

## Learn peppers, _keep tomatoes_

Next spring the club adds peppers. The obvious move has a surprising side effect.

Predict first

You fine-tune the tomato model on 200 pepper photos and nothing else. What happens
to how well it spots ripe tomatoes?

Show the answer

It can get much worse, sometimes suddenly. Training adjusts the same parameters 
that held the tomato knowledge, and nothing in pepper-only data tells the model 
to keep them. This is **catastrophic forgetting**: a standard neural network trained
on one task after another often loses the earlier one.

**Continual learning** studies how to learn from a stream of new tasks or data without
losing old skills. Three families of fixes are common. _Replay_ mixes a sample of
old examples into each new round of training. _Protecting important weights_ makes
the parameters that mattered most for the old task harder to change; Kirkpatrick
and colleagues proposed this in 2017 as elastic weight consolidation. _Adding new
parts_ gives each new task some fresh parameters while freezing the old ones. Each
one costs memory, computing or flexibility, and doing this well for very large models
is still an open research question.

It matters whenever the world drifts (see [after a model is trained](https://arqiv.si/ai/model-lifecycle/)).
A chatbot remembering your earlier messages is different: that text sits in its 
context window, and its parameters do not change.

Federated learning

## Train together _without pooling the photos_

Three schools want one better model, but each wants its photos to stay on its own
computers.

In **federated learning**, a server sends the current model to many devices or sites.
Each one trains on its own data for a short while and sends back only its updated
model numbers, which the server combines into a new shared model. McMahan and colleagues
at Google proposed the standard recipe, federated averaging, in 2016, with phones
in mind: their data is often private, large, or both. Sites with more examples count
more, because the average is weighted by how many examples each one trained on.

Worked example

### Federated averaging for three school gardens

 1. The setupEach school trains the tiny tomato model from the lab on its own photos.
    That model has one number: the redness score where “ripe” starts. School A has 
    20 labeled tomatoes and finds 54; School B has 50 and finds 50; School C has 30
    and finds 53. Only these numbers and the counts are sent. No photo leaves a school.
 2. Weight by examples20 × 54 = 1,080. 50 × 50 = 2,500. 30 × 53 = 1,590. Sum: 1,080
    + 2,500 + 1,590 = 5,170.
 3. Divide by all examples20 + 50 + 30 = 100 tomatoes. 5,170 ÷ 100 = 51.7. The shared
    model now says “ripe from 51.7.”
 4. Compare a plain average(54 + 50 + 53) ÷ 3 = 157 ÷ 3 ≈ 52.3. That would let School
    A’s 20 tomatoes count as much as School B’s 50.
 5. Next roundThe server sends 51.7 back, each school trains a little more from there,
    and the averaging repeats. A real model has millions of numbers; each is averaged
    the same way.
 6. Optional: add noiseEach school adds a small random number first (made-up draws:
    +0.4, −0.3, +0.2) and sends 54.4, 49.7 and 53.2. (20 × 54.4 + 50 × 49.7 + 30 × 
    53.2) ÷ 100 = (1,088 + 2,485 + 1,596) ÷ 100 = 51.69. The shared answer barely moves,
    but no school sent its exact result.

All numbers are made up. Real differential privacy also limits how much one example
can change an update and chooses the noise size mathematically; this only shows 
why averaging many noisy updates can still work.

**Federated is not automatically private.** Keeping raw photos at each school helps,
but an update is computed from the data, so it can still carry information about
it, especially from a site with few examples. **Differential privacy** limits how
much any single example can change an update and then adds carefully sized random
noise, so the result reveals little about any one example. It costs some accuracy,
and the noise size is a trade-off someone has to choose and justify.

Evolution and AutoML

## Let a search _pick the model_

Some choices have no formula: which model family, how many layers, which learning
rate. You can search for them.

**Evolutionary methods** borrow a loose idea from biology. Start with a population
of candidates, score each one (its fitness), keep the best, make new candidates 
by small random changes (mutation) or by mixing two parents, and repeat. Nothing
has to be smooth or differentiable, so evolution can search spaces where [gradient descent](https://arqiv.si/ai/training-and-inference/)
does not apply, such as choices between network designs. The table evolves the lab’s
ripeness rule, scoring each candidate by how many of the 12 tomatoes it sorts correctly.

Mutate, select, repeat: evolving “ripe from …” (made-up random changes)| Generation | Candidates (how made): tomatoes sorted correctly | Kept | 
| Generation 1 | 20 (random): **8**, 40 (random): **10**, 70 (random): **9**, 85 (random): **8** | 40 and 70 | 
| Generation 2 | 40 (kept): **10**, 70 (kept): **9**, 48 (40 + 8): **11**, 62 (70 − 8): **9** | 48 and 40 | 
| Generation 3 | 48 (kept): **11**, 40 (kept): **10**, 53 (48 + 5): **12**, 36 (40 − 4): **10** | 53: all 12 right |

For one number this is overkill (the lab’s midpoint rule finds it directly), but
evolution earns its keep when no direct rule exists.

**AutoML** (automated machine learning) packages this kind of search. A tool tries
model families and [hyperparameters](https://arqiv.si/ai/training-and-inference/),
the settings chosen before training such as tree depth or learning rate, scores 
each on validation data and reports the best. scikit-learn’s grid search and random
search are simple versions. What AutoML does not do: decide which problem is worth
solving, choose a fair score, notice wrong labels, or catch data leakage. A search
that tries hundreds of models can also overfit the validation data, so an untouched
test set still matters.

In practice

## An application, a limit _and a connection_

Real application

### Phones and small projects

Federated learning was designed for data that stays on phones, such as typed text.
Fine-tuning a pretrained model is a common start for small image and language projects.

Limitation

### The work moves, it doesn’t vanish

Transfer needs a related model, active learning a person ready to answer, federated
learning online devices and privacy protection, AutoML a clear goal and clean data.

Connection

### Asking is a kind of exploring

Active learning echoes the [explore-or-exploit](https://arqiv.si/ai/reinforcement-learning/)
trade-off: both decide what to try next to learn the most. Evolution is a cousin
of [search and planning](https://arqiv.si/ai/search-and-planning/).

4 · Common mix-ups

## What people _get wrong about these methods_

Four ideas that trip people up, and what to think instead.

 * **Mix-up:** Federated learning is private by design.
 * **Better:** It keeps raw data on the device, which helps. Updates can still carry
   information about the data, so real systems add protections such as differential
   privacy, and someone still has to check them.
 * **Mix-up:** Fine-tuning teaches a model from scratch.
 * **Better:** Fine-tuning starts from what a pretrained model already learned and
   adjusts it. That is why it needs far less data, and why the old model’s gaps 
   and biases can carry over.
 * **Mix-up:** AutoML means no people are needed.
 * **Better:** It automates the search. People still define the goal and the score,
   check the data and labels, and decide whether the result is fit for use.
 * **Mix-up:** A chatbot that remembers our chat is doing continual learning.
 * **Better:** Remembered text sits in the context window or a saved note. Continual
   learning changes the model’s parameters.

5 · Check yourself

## Three _quick questions_

Answer in your head or on paper before you open each one.

 1.  A wildlife group has 5,000 unlabeled trail-camera photos and one volunteer who
     can label 200. Which two methods from this page would you combine, and how?
 2.  Check your answer
 3.  **Transfer learning:** start from a model pretrained on general photos and fine-
     tune it. **Active learning:** let that model pick the photos it is least sure 
     about for the volunteer, retraining as labels come in. Keep a separate random 
     set of photos for testing, because the chosen 200 are not a fair sample.
 4.  Two sites train a shared model with federated averaging. Site X trained on 10 
     examples and sends 4.0; site Y trained on 30 and sends 8.0. What is the shared
     value?
 5.  Check your answer
 6.  (10 × 4.0 + 30 × 8.0) ÷ (10 + 30) = (40 + 240) ÷ 40 = 280 ÷ 40 = **7.0**. A plain
     average, 6.0, would ignore that Y had three times as many examples.
 7.  After fine-tuning only on pepper photos, the tomato spotter gets much worse at
     tomatoes. Name the problem and one fix.
 8.  Check your answer
 9.  **Catastrophic forgetting.** Fixes: replay some tomato photos while training on
     peppers, protect the weights that mattered most for tomatoes (as in elastic weight
     consolidation), or add new parameters for peppers and freeze the old ones.

For younger learners

## A _grades 5–8_ version

This page is for grades 9 and up. Younger classes can still feel the main idea of
active learning.

15 minutes · teacher-led

### Higher or lower?

 1. Write a secret number from 1 to 100 where students can’t see it.
 2. Team 1 must ask in a fixed order: “Is it 10 or more? 20 or more? 30 or more?”
 3. Team 2 may ask anything that has a yes or no answer. Both teams get five questions.
 4. Compare how close each team got.

Talk about it

### Which question teaches the most?

Good answers say Team 2 asked in the middle of what was still unknown, so every 
answer cut the possibilities in half. That is what an active-learning model does
when it asks for a label.

Honest limits

## What this page _leaves out_

What this page leaves out

Semi-supervised and self-supervised learning (learning from unlabeled data directly),
meta-learning, the math of differential privacy, secure aggregation, neural architecture
search in detail, and how large language models are updated over time. Each section
here is a first look; the sources go further.

Where to go next

## Keep _building_

Advanced track

### What happens after a model is trained?

Monitoring, drift and retraining: where continual learning meets real systems.

Hands on

### Can you build and test a model yourself?

Baselines, splits and honest testing, the habits every method on this page still
needs.

Use, question, build

### Who checks that an AI system is fair and safe?

Privacy, evidence and oversight beyond the training recipe.

Sources

## Where this comes from

 1. [H. Brendan McMahan et al. (2016), “Communication-Efficient Learning of Deep Networks from Decentralized Data”: federated averaging, weighted by each device’s number of examples.](https://arxiv.org/abs/1602.05629)
 2. [Martín Abadi et al. (2016), “Deep Learning with Differential Privacy”: limiting each example’s influence and adding noise during training.](https://arxiv.org/abs/1607.00133)
 3. Burr Settles (2009), “Active Learning Literature Survey,” Computer Sciences Technical
    Report 1648, University of Wisconsin–Madison: uncertainty sampling and other query
    strategies.
 4. [James Kirkpatrick et al. (2017), “Overcoming catastrophic forgetting in neural networks” (PNAS): elastic weight consolidation.](https://arxiv.org/abs/1612.00796)
 5. [PyTorch tutorials, “Transfer Learning for Computer Vision”: fine-tuning a pretrained network on a small ants-and-bees dataset.](https://docs.pytorch.org/tutorials/beginner/transfer_learning_tutorial.html)
 6. [scikit-learn user guide, “Tuning the hyper-parameters of an estimator”: grid search and randomized search with cross-validation.](https://scikit-learn.org/stable/modules/grid_search.html)
 7. [Stuart Russell and Peter Norvig, _Artificial Intelligence: A Modern Approach_, 4th edition, contents: evolutionary algorithms within search and optimization.](https://aima.cs.berkeley.edu/contents.html)

AI words on this page

[Transfer learning](https://arqiv.si/teachers/ai-vocabulary/#transfer-learning),
[Pretraining](https://arqiv.si/teachers/ai-vocabulary/#pretraining), [Fine-tuning](https://arqiv.si/teachers/ai-vocabulary/#fine-tuning),
[Active learning](https://arqiv.si/teachers/ai-vocabulary/#active-learning), [Continual learning](https://arqiv.si/teachers/ai-vocabulary/#continual-learning),
[Federated learning](https://arqiv.si/teachers/ai-vocabulary/#federated-learning),
[Differential privacy](https://arqiv.si/teachers/ai-vocabulary/#differential-privacy),
[Evolutionary algorithm](https://arqiv.si/teachers/ai-vocabulary/#evolutionary-algorithm),
[AutoML](https://arqiv.si/teachers/ai-vocabulary/#automl), [Hyperparameter](https://arqiv.si/teachers/ai-vocabulary/#hyperparameter),
[Label](https://arqiv.si/teachers/ai-vocabulary/#label), [Validation set](https://arqiv.si/teachers/ai-vocabulary/#validation-set),
[Distribution shift](https://arqiv.si/teachers/ai-vocabulary/#distribution-shift)

All definitions live in one place: [AI words in plain language →](https://arqiv.si/teachers/ai-vocabulary/)

Review status

Written by ARQIV. Last checked October 2026. Independent subject review and classroom
pilots are still to come, so treat the activities as drafts and [tell us](https://arqiv.si/ai/advanced-learning/info@arqiv.fm?subject=AI%20foundations%20correction)
about anything that is wrong or unclear.

[← Previous**Security & provenance**](https://arqiv.si/ai/security-and-provenance/)
[Next →**All AI topics**](https://arqiv.si/ai/topics/)

[arqiv](https://arqiv.si/)

A little understanding changes everything.

[LISTEN: ARQIV.FM, OUR WEEKLY SHOW ↗](https://arqiv.fm/)

Educators

 * [Start here](https://arqiv.si/teachers/)
 * [Lessons & kits](https://arqiv.si/teachers/resources/)
 * [Curriculum](https://arqiv.si/teachers/curriculum/)
 * [Labs](https://arqiv.si/teachers/labs/)
 * [AI vocabulary](https://arqiv.si/teachers/ai-vocabulary/)

For your role

 * [Counselors](https://arqiv.si/counselors/)
 * [School leaders & PD](https://arqiv.si/schools/)
 * [Speaking & workshops](https://arqiv.si/speaking/)
 * [Products](https://arqiv.si/products/)

Learn

 * [Learn AI](https://arqiv.si/ai/topics/)
 * [Games & labs](https://arqiv.si/games/)
 * [Future of work](https://arqiv.si/future-of-work/)
 * [Research](https://arqiv.si/research/)
 * [Headlines](https://arqiv.si/news/)

ARQIV

 * [About & corrections](https://arqiv.si/about/)
 * [Our approach](https://arqiv.si/research/#our-method)
 * [Newsletter](https://arqiv.si/newsletter/)
 * [Privacy](https://arqiv.si/privacy-policy/)
 * [Get in touch](https://arqiv.si/ai/advanced-learning/info@arqiv.fm)

Classroom activities collect no student data.[info@arqiv.fm](https://arqiv.si/ai/advanced-learning/info@arqiv.fm)
