US What imitation learning is
Imitation learning is a machine learning approach in which an agent learns a skill by copying demonstrations from an expert instead of being given explicit rules or a reward to chase. The expert is usually a person, but it can be another robot, a planning algorithm or an existing AI model. It is also called learning from demonstration.
Each demonstration is recorded as a sequence of pairs: what the robot could sense at that moment, and the action the expert took. A model, usually a neural network, is trained to predict the expert’s action from the sensor data. Once trained, it is called a policy, and it runs on the robot to choose actions in situations it has not seen exactly before.
The attraction is that many useful skills are easy to show and hard to describe. Folding a cloth, inserting a cable or wiping a surface involves dozens of small corrections that nobody could write out as rules.
How demonstrations are collected
The quality of an imitation learning policy depends mostly on the quality and variety of its demonstrations. There are four common ways to collect them.
-
//
Teleoperation. A person drives the robot remotely with a joystick, a VR headset or a matching leader arm, while the robot records its cameras and joint positions.
-
//
Kinesthetic teaching. A person physically guides a robot arm through the task by hand, which suits collaborative robots built for hand guiding.
-
//
Video and motion capture. Recordings of people doing the task, with pose estimation used to recover the hand and body movements.
-
//
Simulation. Demonstrations generated or multiplied in a simulator, for example with NVIDIA Isaac Lab, to add variety without more hours on real hardware.
Types of imitation learning
Several methods have been developed since the 1990s, and the newest ones are behind much of the recent progress in robot manipulation.
| Method | How it learns | Expert needed during training? | Typical use |
|---|---|---|---|
| Behavioral cloning | Supervised learning on recorded state and action pairs | No, only the recorded data | Simple tasks with plenty of demonstrations |
| DAgger (2011) | The expert labels the situations the learner gets itself into | Yes, repeatedly | Correcting drift in driving and navigation |
| Inverse reinforcement learning | Infers the reward the expert seems to follow, then optimizes it | No | Tasks where the goal matters more than the exact motion |
| GAIL (2016) | A discriminator pushes the policy to behave like the expert | No, but needs a simulator | Locomotion and control in simulation |
| ACT (2023) | A transformer predicts short chunks of future actions | No | Fine two-arm manipulation from small datasets |
| Diffusion Policy (2023) | Generates action sequences by gradually removing noise | No | Tasks with several valid ways to do them |
ACT, short for Action Chunking with Transformers, came from the Stanford ALOHA project, where low-cost leader and follower arms were used to teach fine two-handed tasks from roughly 50 demonstrations each. The open-source Hugging Face LeRobot library now packages ACT, Diffusion Policy and related methods with tools for recording datasets.
Imitation learning vs reinforcement learning
The two are the main ways robots learn skills, and they are often combined. Imitation learning needs demonstrations but no reward function, and it reaches reasonable behavior quickly. Reinforcement learning needs a reward and a great deal of trial and error, usually in simulation, but it can discover behavior better than any demonstrator.
The classic weakness of behavioral cloning is compounding error. A small mistake takes the robot into a situation the expert never showed it, so the next action is a guess, and the errors build up. DAgger, action chunking, more varied demonstrations and a final round of reinforcement learning are all ways of dealing with it.
In industry, imitation learning is used where tasks vary too much to program but are easy for a person to show: assembling flexible parts such as cables and hoses, picking and packing mixed goods, harvesting soft fruit, and kitting parts for a production line. Research groups also use it to teach surgical robots steps such as passing a needle, working on research systems rather than clinical ones.
The hardware behind imitation learning
An imitation learning project uses computers at three points. The data collection rig records several cameras in sync with joint states, often at 30 to 60 frames a second, so it needs camera inputs with tight timing and fast storage. GMSL2 cameras suit this because they carry video and trigger signals over one cable. Training then runs on GPU servers or industrial GPU computers. Finally, the trained policy runs on the robot itself, typically on a computer from our NVIDIA Jetson Orin range, which has the GPU to run the policy and the camera pipeline within the robot’s power budget.
Quick answers on imitation learning
Imitation learning: common questions
Is imitation learning supervised learning?
Behavioral cloning is. It treats each expert action as a label. Other methods, such as inverse reinforcement learning and GAIL, mix in ideas from reinforcement learning.
What is the ACT algorithm in robotics?
Action Chunking with Transformers. A transformer policy that predicts a short sequence of actions at once, which smooths motion and reduces compounding error. It was introduced with the ALOHA two-arm system.
How many demonstrations does a robot need?
From tens to thousands. A single fixed task in one setting may work with around 50. A policy that must handle varied objects, lighting and positions needs far more.
What is behavior cloning?
The simplest form of imitation learning. A model is trained to predict the expert’s action from the robot’s observations, with no reward and no further help from the expert.
Can imitation learning beat the expert?
Not on its own. It copies the demonstrations, mistakes included. Fine-tuning with reinforcement learning is the usual way to improve on them.
We build the data collection and on-robot computers for imitation learning on NVIDIA Jetson, with the cameras, storage and JetPack image fitted and validated as standard. Tell us about the robot, its cameras and its tasks, and we will specify the system to match.