Go deeper · Advanced track
Why is it hard to put AI in a body?
Because a robot never sees the world directly and never acts on it exactly. Its sensors give noisy measurements, it has to estimate where it is, and its motors slip and lag. Small errors feed into each other, mistakes move real weight near real people, and something that works in a simulator can fail on the floor.
This is an optional page on the advanced track. You do not need it for the main foundations route.
What you will be able to do
Trace one turn of the sense, estimate, plan, act loop; run a proportional controller by hand; and explain with numbers why a controller tuned in simulation can fail on a real robot.
Before you start
Read first: How does a computer plan without learning?, How does a machine learn from rewards? and What does a camera-based AI actually measure?
1 · You have already seen this
Pulling up to the right spot
Stopping a car with its front just at a painted line takes more checking than it seems.
As you roll toward the line, you look, judge how far is left, decide how hard to press the brake, and press. Then you look again, because the car did not slow down exactly as you expected. A wet road or someone stepping out changes what happens next, so you keep checking and correcting until you stop.
A robot has to do the same with sensors and motors in place of eyes and feet. It never sees the world directly and never acts on it exactly, and its mistakes move real weight near real people. This page follows a small cart that has to stop at a line. It shows how that works, lets you try it, and shows where it goes wrong.
2 · What happens, step by step
A cart that has to stop at a line
Start with one everyday task and count the problems hiding inside it.
Picture a small cart carrying lunch trays down a school hallway. Its only job: stop with its front wheels on a painted line by the cafeteria door, 100 cm away. Glare on the shiny floor makes the camera misjudge the line. A wheel spins on a wet patch, so the wheel counter overstates the distance. A low battery weakens the motors. The position report arrives a moment late. Someone steps into the hallway.
Roboticists split the job into a loop that runs many times a second. The robotics chapter of the standard AI textbook (AIMA) is organized around the same parts: perception, planning and control, and learning.
- 1 · Sense“What do my sensors report?”Pixels, wheel counts, distances, all noisy.
- 2 · Estimate“Where am I, probably?”Combine readings into a best guess.
- 3 · Plan“Where should I go next?”Pick a route or target from that guess.
- 4 · Act“How hard do I push?”A controller sends motor commands.
- 5 · World“What actually happened?”Friction, delays and people change the result. Repeat.
Going deeper · steps 1 and 2
Sensors measure; they do not know
A robot never sees its own position. It sees numbers and has to work out what they mean.
A camera gives a grid of pixel values (see What does a camera-based AI actually measure?). A vision model can turn them into “the line is 23 cm ahead,” but glare or a smudged lens shifts that number. A wheel counter means nothing once the wheel slips. And every reading wobbles a little even when nothing moves: sensor noise.
So the robot keeps a best guess of its situation (position, speed, heading), and updating that guess from noisy readings is state estimation. If the wheels say 102 cm and the camera says 98 cm and you trust both equally, your estimate is (102 + 98) ÷ 2 = 100 cm; on a wet floor you would lean toward the camera. Real robots do this with probability, tracking how unsure each estimate is (see How sure should an AI be?). Everything later in the loop runs on the estimate, not on the world.
Going deeper · step 3
Choosing where to go is a search
Planning decides what the robot should try to do next. It often needs no learning at all.
In a warehouse, “get to shelf 14” is a search over squares on a floor map. ARQIV’s Plan a robot’s path lab solves it with breadth-first search: a hand-coded planner, not a trained model, that never learns from examples (more in How does a computer plan without learning?). But a plan is a list of targets, not wheel commands. Something else must decide how hard to push the motors, moment by moment. That is the controller.
Going deeper · step 4
Measure the error, push to correct it
Feedback control is one of the oldest and most dependable ideas in engineering: keep checking, keep correcting.
The error is the gap between where the robot should be and where it thinks it is. Choosing each command from the latest error is feedback control; thermostats and cruise control do it too.
The simplest version is a proportional controller: push = gain × error. The gain is a number the engineer picks. A small gain is gentle but slow. A large gain is fast but can carry the cart past the line, called overshoot. Too large, and every correction is bigger than the mistake it fixes.
Worked example
One controller, two worlds
- The setupA toy cart starts at 0 cm; the line is at 100 cm. Each step: push = gain × (100 − reading). Gain 0.5.
- SimulationPush 0.5 × 100 = 50, position 50. Then 0.5 × 50 = 25, position 75. Then 12.5, position 87.5. Then 6.25, position 93.75 cm. The gap halves each step; the cart never crosses the line.
- Gain 1.5 instead150, then 150 + 1.5 × (100 − 150) = 75, then 112.5, then 93.75: the gap shrinks too, but the cart overshoots and swings back each step.
- Real world, step 1Add made-up friction (each push loses 3 cm; a push of 3 cm or less does nothing) and a sensor that reports the position one step late. Reading 0, push 50, moves 47. Position 47.
- Step 2The reading is still the old 0, so the push is 50 again: moves 47. Position 47 + 47 = 94.
- Step 3Reading 47. Push 0.5 × (100 − 47) = 26.5, moves 23.5. Position 117.5, 17.5 cm past the line.
- Step 4Reading 94. Push 0.5 × 6 = 3, which friction swallows. Position stays 117.5 cm. Only on step 5 does the controller see 117.5 and push back: 0.5 × (100 − 117.5) = −8.75, moves −5.75, to 111.75.
All numbers are made up for this toy. The controller and the gain did not change; only the world did.
3 · Try it
Tune the cart, then leave the simulator
The same controller and the same gain in two worlds. Everything here is made up and runs in your browser.
Predict first
In the simulation, gain 0.8 reaches the line in about four steps with no overshoot (80, 96, 99.2, 99.84 cm). In the real world, with 3 cm of friction and a one-step sensor delay, does it still work?
Show the answer
No. The cart swings to 169.4 cm, back to 56.3 cm, and is still swinging after 12 steps. The controller keeps pushing on old news: by the time it sees the cart at the line, the cart has already had another big push.
Lab
Can one gain work in both worlds?
Pick a gain with the slider or the arrow keys, run it in the simulation, then switch the world. Each run is 12 steps. If the cart leaves the 0 to 200 cm track, a safety stop ends the run.
Simulated in your browser with a made-up cart, made-up friction and a made-up delay. No real robot data.
Each row is one step, top to bottom. Pink line: the target at 100 cm. Blue dots: the cart. Gray dashes: the same gain in the simulation, shown when any effect is on.
Show the numbers, step by step
Three challenges
- In the simulation, find the gain that reaches the line in a single step. What do larger gains do?
- Set the gain to 0.8 and switch to the real world. Then turn on only friction, then only the delay. Which effect makes the cart swing, and which makes it stop short?
- Find a gain that ends within 5 cm of the line in the real world. Is it the gain you would have picked from the simulation?
A toy cart starts at 0 cm and must stop at a line at 100 cm on a 200 cm track. Each step it pushes gain × (100 − reading) centimeters. The made-up real world adds friction (each push loses 3 cm, and a push of 3 cm or less does nothing) and a one-step sensor delay. Twelve steps per run.
| Gain | Simulation | Real world (friction + delay) |
|---|---|---|
| 0.3 | never crosses the line; ends at 98.6 cm; within 5 cm from step 9 | never crosses the line; ends at 91.1 cm; not within 5 cm by step 12 |
| 0.5 | never crosses the line; ends at 100.0 cm; within 5 cm from step 5 | swings to 117.5 cm; ends at 103.1 cm; within 5 cm from step 7 |
| 0.8 | never crosses the line; ends at 100.0 cm; within 5 cm from step 2 | swings to 169.4 cm; ends at 92.0 cm; not within 5 cm by step 12 |
| 1.0 | never crosses the line; ends at 100.0 cm; within 5 cm from step 1 | swings to 194.0 cm; ends at 24.0 cm; not within 5 cm by step 12 |
| 1.5 | swings to 150.0 cm; ends at 100.0 cm; within 5 cm from step 5 | leaves the track at step 2 (294.0 cm) |
Gain 0.5 in the real world, position after each step: 0.0, 47.0, 94.0, 117.5, 117.5, 111.8, 106.0, 103.1, 103.1, 103.1, 103.1, 103.1, 103.1 cm. In simulation the same gain never crosses the line.
Simulation
Why the simulator lies a little
Simulators are essential, and every one of them is wrong in some way.
A simulation predicts how a robot and its surroundings would move. It is fast, cheap and safe: a simulated robot can crash thousands of times overnight. The sim-to-real gap is every way reality differs: friction on carpet versus tile, motors that weaken as the battery drains, lighting that changes what the camera sees, sensor noise, delays.
Teams measure the real robot to make the simulator more realistic, test in stages, and use domain randomization: training across many deliberately varied simulations so the real world looks like one more variation. Tobin and colleagues (2017) trained an object detector only on simulated images with random, unrealistic textures; it located real objects to about 1.5 cm, enough for a robot arm to grasp them.
Optional · why delay makes things swing
Pushing on old news
Without delay, each step multiplies the error by (1 − gain). With a one-step delay, the next error is the current error minus gain × the previous error. Those errors die away only when the gain is below 1, and they swing whenever it is above 0.25. With the delay alone, gain 0.3 overshoots slightly (to 101.2 cm), and gain 1.0, perfect in the simulation, bounces between 0 and 200 cm forever.
Learning in robots
Rewards, demonstrations and predictions
Machine learning helps most with the parts nobody can write down by hand.
Learning is most common in perception: trained vision models find a line, a box or a person in pixels far better than hand-written rules. Reinforcement learning can also learn a controller by trial and error (see How does a machine learn from rewards?), but real trials cost time and wear, so it usually starts in simulation, and the sim-to-real gap comes back.
Imitation learning copies demonstrations, such as a person steering the cart with a joystick. Small mistakes add up: once the copy drifts off course, it meets situations the person never showed. Ross, Gordon and Bagnell (2011) describe this problem and a fix: collect new demonstrations in the situations the learner actually reaches.
A world model here means a robot’s internal prediction of what happens next: “if I push 50 cm from here, I will end up about there.” The lab’s two-line simulation is a tiny hand-written one. Learning such predictions from data is active research; how far it will take robots is an open question.
Safety and testing
Mistakes here have weight
A wrong sentence can be corrected. A cart that hits someone cannot be undone.
So safety sits in the simplest, most predictable layer: speed limits, an emergency stop button, hard limits like the lab’s “left the track” stop. These are usually hand-written rules, because they must work every time. Testing goes in stages (simulation, a test bench with the wheels off the ground, a closed area at low speed, supervised use) and teams log what the robot sensed, estimated and commanded to replay failures. Open-source tools such as the Robot Operating System (ROS) connect the sensing, planning and control programs.
Application · limitation · connection
Application: warehouse robots plan paths around shelves (AI in engineering); farm sprayers act on what a camera sees (Be the sprayer’s brain).
Limitation: every part can pass its own test and the robot can still fail when the world surprises it.
Connection: search plans, machine learning sees, hand-written feedback controls. A robot combines AI approaches, as the map of AI shows.
4 · Common mix-ups
What people get wrong about robots
Four ideas that trip people up, each with what to think instead.
Mix-up: A robot sees the world as it is.
Better: Sensors produce noisy measurements. The robot acts on an estimate built from them, and that estimate can be wrong.
Mix-up: If it works in simulation, it works.
Better: The sim-to-real gap is real: friction, lighting, sensor noise and delays differ. In the lab, a gain that looked perfect swung almost 70 cm past the line.
Mix-up: A bigger gain is always faster and better.
Better: Past a point it overshoots, and with delays it can swing out of control. The best gain depends on the real world, not only on the simulator.
Mix-up: Robots are run by one big AI model.
Better: Most robots combine hand-written planners, controllers and safety rules with learned parts, especially for perception.
5 · Check yourself
Three quick questions
Answer in your head or on paper before you open each one.
A cart is at 60 cm and the line is at 100 cm. With a proportional controller and gain 0.4, where is it after one step in the simulation? After two?
Check your answer
Error 100 − 60 = 40, push 0.4 × 40 = 16, so 76 cm. Then error 24, push 0.4 × 24 = 9.6, so 85.6 cm. It keeps closing 40% of the remaining gap each step.
A robot tuned in a simulator with perfect wheels keeps stopping a few centimeters short on real carpet. What is the likely cause, and name one way to fix it.
Check your answer
Friction the simulator left out: near the line the pushes become small, and the carpet swallows them, as gain 0.3 did in the lab (stuck at 91.1 cm). Fixes: measure the real friction and add it to the simulator, randomize friction during tuning, retune on the real robot, or use a controller that keeps adding push while an error lasts.
Why do robot teams usually run reinforcement learning in simulation first, and what risk does that bring?
Check your answer
Learning by trial and error needs a very large number of attempts, many of them clumsy. In simulation those attempts are fast, cheap and harmless. The risk is the sim-to-real gap: the robot may learn behavior that depends on how the simulator differs from reality, so it must be tested carefully, in stages, on the real machine.
For younger learners
A grades 5–8 version
This page is for grades 9 and up. A younger class can feel the same idea on a desk with a toy car and a ruler.
20 minutes · groups of three
Be the controller
- Tape a ruler to a desk and mark a “stop line” at 20 cm. A toy car starts at 0.
- The sensor reads the car’s position aloud. The controller says “half the distance to the line.” The motor rolls the car that far. Repeat five times and record each position.
- Now play the “real world”: the sensor must report the position from one turn ago. Record again and compare.
Talk about it
What changed?
Good answers notice that the rule stayed the same and only the information got late, yet the car went past the line. Ask: what would a real robot need so that a late reading cannot cause a crash?
Honest limits
What this page leaves out
What this page leaves out
Real robots move in two or three dimensions, turn, balance and grasp. Their controllers usually add terms that react to speed and to errors that last a long time, and their estimators use filters that track uncertainty carefully. This page also skips robot hardware, motion planning for arms, and working alongside people. AIMA’s robotics chapter and Probabilistic Robotics cover these.
Where to go next
Bodies, plans and risks
Next in this track
How can AI systems be attacked?
Data poisoning, adversarial examples and tracing what an AI made.
Planning at work
Plan a robot’s path
ARQIV’s warehouse pathfinder: hand-coded breadth-first search, square by square.
Learning by doing
How does a machine learn from rewards?
Agents, rewards and the trade-off between exploring and exploiting.
Sources
Where this comes from
- Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 4th edition, table of contents: chapter 26, Robotics (robotic perception, planning and control, planning uncertain movements, reinforcement learning in robotics, humans and robots).
- Sebastian Thrun, Wolfram Burgard and Dieter Fox (2005), Probabilistic Robotics, MIT Press (book): state estimation, sensor noise and robot localization with probability.
- Karl Johan Åström and Richard M. Murray, Feedback Systems: An Introduction for Scientists and Engineers, 2nd edition (2021), free online: feedback, proportional control and the effect of delays.
- Josh Tobin et al. (2017), “Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World,” arXiv:1703.06907.
- Stéphane Ross, Geoffrey J. Gordon and J. Andrew Bagnell (2011), “A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning,” arXiv:1011.0686.
- ROS (Robot Operating System) documentation: open-source software libraries and tools for building robot applications.
AI words on this page
Sim-to-real gap, Search and planning, Computer vision, Reinforcement learning, Distribution shift, Latency, Edge AI, Human oversight, Algorithm, Model
All definitions live in one place: AI words in plain language →
Review status
Written by ARQIV. Last checked October 2026. Independent subject review and classroom pilots are still to come, so treat the activities as drafts and tell us about anything that is wrong or unclear.