US What a VLA model is
A VLA model, short for vision-language-action model, is a single neural network that looks at the scene through one or more cameras, reads an instruction such as “put the red cup on the tray”, and outputs the motor commands to do it. It is the most common design for a robot foundation model: one model meant to handle many tasks, objects and sometimes many robot types.
The approach became widely known in 2023, when Google DeepMind published RT-2. Its researchers took a vision-language model already trained on web images and text, and trained it further on robot recordings, with the robot’s actions written out as short strings of tokens, just like words. The model kept its general knowledge from the web and could apply it to instructions and objects it had never seen in robot data.
How a VLA model works
Most VLA models have three parts, joined so the whole network can be trained together.
-
//
Vision encoder. Turns each camera image into a set of feature tokens. Many VLAs reuse an encoder from a vision-language model rather than training one from scratch.
-
//
Language model backbone. Reads the instruction together with the image tokens and works out what needs to happen. This is where knowledge from web-scale training lives.
-
//
Action decoder. Produces the movement. Early VLAs output discrete action tokens, each one a small step in position or rotation. Newer designs add a separate action head that uses diffusion or flow matching to output smooth, continuous trajectories several steps at a time.
Key VLA models
The field moves quickly, but a handful of models define the main approaches. OpenVLA, released in 2024 by Stanford, UC Berkeley and partners, is an open 7-billion-parameter model trained on robot episodes from the Open X-Embodiment dataset. Physical Intelligence’s π0 pairs a vision-language model with a flow-matching action expert for dexterous tasks such as folding laundry. NVIDIA Isaac GR00T N1, announced in 2025 as an open model for humanoid robots, splits the work into a slower vision-language module that interprets the scene and a faster diffusion module that generates the movement. Google DeepMind’s Gemini Robotics brings the same idea to its Gemini models.
NVIDIA also describes reasoning VLAs, which write out intermediate reasoning steps before acting, so engineers can see why the model chose an action.
What VLA models are used for
VLA models are still mostly in research labs and pilot deployments, but three uses lead.
-
//
Mixed-item manipulation. Picking, packing and sorting where the objects change every day and a fixed program would need constant rework.
-
//
Humanoid and mobile manipulators. Robots that walk or drive to a location and then handle objects there, following spoken or written instructions.
-
//
Fast task changeover. Teaching a new task with a few dozen demonstrations and fine-tuning, rather than writing and validating new robot code.
VLA vs VLM vs LLM
The three names describe one family of models with different inputs and outputs.
| Model type | Inputs | Outputs | Trained on | Typical use |
|---|---|---|---|---|
| LLM | Text | Text | Web-scale text | Chat, search, code |
| VLM | Images or video plus text | Text | Image-text pairs plus text | Describing scenes, answering questions about images |
| VLA | Camera images, instruction, robot state | Robot actions | VLM data plus robot demonstrations | Manipulation, navigation, humanoid control |
The step from VLM to VLA is the action output. That is what makes training data so expensive: web text is free to collect, while every robot demonstration has to be recorded on real hardware or generated in simulation.
Running a VLA model on the robot
A VLA model is large by robot standards. A 7-billion-parameter model needs around 14 GB of memory for its weights at 16-bit precision and about half that at 8-bit, before the camera pipelines and the rest of the software stack. That is why quantisation to INT8 or lower, and optimization with NVIDIA TensorRT, are normal steps before deployment.
Speed matters as much as memory. A large VLA may only produce a new plan a few times a second, while a robot arm needs commands far more often. Designs such as action chunking, where the model outputs a short trajectory at once, and dual-module models such as GR00T N1, bridge that gap. The low-level servo loops and safety stops stay on a separate controller.
NVIDIA positions Jetson Thor as the on-robot computer for this class of model, with up to 128 GB of memory and a Blackwell GPU. Our Jetson Thor computers include robot computers with GMSL camera inputs and CAN FD, and our guide to choosing a Jetson module explains when a project needs Thor rather than Jetson AGX Orin.
Quick answers on VLA models
VLA models: common questions
What does VLA stand for in AI?
Vision-language-action. The model sees (vision), understands an instruction (language) and outputs movement (action).
Is a VLA model a large language model?
It contains one. Most VLAs are built on a language model backbone, but they add a vision encoder and an action output, and they are trained on robot data as well as text.
Is Octo a VLA model?
Not in the strict sense. Octo, from UC Berkeley, is a generalist robot policy that accepts language instructions, but it is not built on a pretrained vision-language model, so it is usually classed separately from VLAs such as OpenVLA.
Can a VLA model run on NVIDIA Jetson?
Yes, with optimization. Smaller and quantised VLAs run on Jetson AGX Orin, and Jetson Thor is designed for larger robot foundation models. Expect to tune precision, batch size and control rate for the robot.
Do VLA models replace robot programming?
Not yet for most production lines. They suit varied, unstructured tasks. Repetitive, fixed tasks are still usually programmed, with a VLA added where flexibility pays.
We build the on-robot computers that run VLA models, on Jetson AGX Orin and Jetson Thor, with the cameras, storage and JetPack image fitted and validated as standard. Share the model, sensors and power budget, and we will specify the system to match.