NVIDIA's New Robot Learns by Watching You — No Code, No Training Manual

sheen6869

1 hour ago

7 min read

By Shakeel Ahmed — AI Growth Lab | November 10, 2026

Originally published on AI Growth Lab: https://aigrowthlabshakeel.blogspot.com/2026/11/claude-fable-mythos-split-access.html

On November 9, 2026, NVIDIA introduced GR00T 2 at the Conference on Robot Learning in Austin, Texas. It is a robot model that learns manual skills by watching first-person video of people doing the task. NVIDIA calls it an open frontier world action model, built on its Cosmos world-model platform.

Watch a person do a task once, and the robot can copy it. That is the promise, and it is the biggest change in robot training in years. This article explains what GR00T 2 is, how learning from human video works, what the test results actually show, and what it means for factory, warehouse, and home robots — plus the honest limits.

What GR00T 2 Actually Is

GR00T 2 is a robot foundation model, the successor to NVIDIA's GR00T N1 series. It is built on Cosmos, NVIDIA's world-model platform, and it uses a 2-billion-parameter transformer built on Cosmos-Predict2.5-2B.

The key idea is in the name NVIDIA gives it: a world action model. Most robot models either predict what they will see next or decide what to do next. GR00T 2 does both at once. It reads recent video from the robot's cameras, the written instruction, and the robot's current state, then generates a short chunk of future video and the motor commands to go with it, all inside a single diffusion transformer. In plain terms, the model imagines how the physical world will unfold and generates the actions that get it there.

One pretrained checkpoint covers three very different bodies: parallel-gripper arms, dexterous hands, and a humanoid robot. That matters because, until now, each robot body usually needed its own training. GR00T 2 also has a feature NVIDIA calls X-Adapt, or cross-adaptation. The model can learn a skill from human video, or from a completely different robot, and then perform it on a body that never practiced the task.

NVIDIA also introduced a faster version, GR00T 2 Turbo, built for speed, with whole-body loco-manipulation — the ability to walk and handle objects at the same time.

How It Learns From Human Video

The old way to teach a robot was to collect demonstrations from that exact robot, doing that exact task, over and over. It was slow, expensive, and it did not transfer. A robot trained to pick up a cup in one factory could not pick up a box in another.

GR00T 2 changes the training data. Its pretraining mix includes 402,730 hours of first-person human video, 10,184 hours of handheld gripper demonstrations across 2,897 tasks, and 23,500 hours of robot teleoperation across arms, dexterous hands, and Unitree's G1 humanoid.

The human video is the breakthrough. There are far more hours of people doing things on camera than there will ever be hours of robots doing things. NVIDIA ran a systematic study of what happens as you add more human video, and the result was clear: adding all of the human video raised real-robot task progress from 50 percent to 70 percent across 72 tasks.

NVIDIA is also releasing 100,000 hours of that human video as an open dataset, called Human-1M-100K, on Hugging Face. It includes video, language annotations, and head and hand motion data, under a CC BY 4.0 license that permits commercial and noncommercial use. The model code itself is listed as available in December, according to NVIDIA's research page.

The training pipeline does not stop at watching. NVIDIA also released an open Video to Data workflow, which converts human video into simulated robot demonstrations, and a new open model called EgoHand, which reconstructs 3D hand motion from video.

The Numbers: What the Tests Actually Show

NVIDIA reported its results on 50 tasks that were absent from the target robot's training data. Here are the numbers, and the honest reading of each one.

First, with no task-specific demonstrations at all, GR00T 2 completed 40 of the 50 tasks at least once, an average success rate of 40 percent. That is the zero-shot number, and it is the most honest measure of general capability. A robot that has never seen a task succeeding 40 percent of the time, purely from watching humans, is a real step forward.

Second, with one demonstration per task, the average success rate rose to 71 percent. Show the robot once, and it gets the job done nearly three times out of four.

Third, with five demonstrations per task, success reached 95 percent. That number needs a footnote. The evaluated tasks for the five-demonstration result were selected among those the model can learn with five episodes at a success rate of at least 80 percent, so the average is high partly by construction. The 40 percent zero-shot figure is the number to remember.

GR00T 2 also ranks among the top three models across all vision-language-action and world action model categories on the RoboCasa365, RoboLab, RoboDojo, and MolmoSpaces benchmarks, with multiple leaderboard wins, according to NVIDIA.

One more reported detail deserves attention. Coverage of the launch noted that agent-monitoring overhead caused robots to idle in 38 percent of episodes in testing — a real-time latency cost that does not appear in NVIDIA's announcement. Speed on paper and speed on a factory floor are not the same thing yet.

Isaac Lab 3.0 and the Training Machine

GR00T 2 does not train alone. NVIDIA also released Isaac Lab 3.0, the latest version of its open-source robot-learning framework, which has been generally available since October 20, 2026.

Isaac Lab 3.0 adds the Newton physics engine and reaches up to 2.5 times the training throughput of Isaac Lab 2.3 on a dexterous manipulation benchmark, according to NVIDIA. It lets researchers use agents to quickly set up training environments with simulation-ready assets, switch between the PhysX and Newton physics solvers, choose renderers for perception-based training, and apply a new reinforcement post-training library to foundation models like GR00T.

The release also includes two new open benchmarks for industrial work: DexBench, with 15 tasks across four industrial scenes, and VABAR, which evaluates robots across six industrial tasks with standardized variations. A new evaluation layer, Isaac Lab-Arena, sits on top.

There is one more announcement from CoRL worth knowing. NVIDIA and UC Berkeley jointly announced an agentic robotics approach that uses AI coding tools to program robots for industrial tasks, treating robot control as a coding problem. The idea is that AI agents compose interpretable code for physical robots, instead of every behavior being hand-programmed.

Who Is Already Testing It

NVIDIA named the first testers, and the list is practical rather than flashy. Nexuni is testing GR00T 2 for robotic laundry handling. Foxconn is applying it to server-manufacturing automation. Siemens is evaluating it across industrial use cases. MicroAGI is using GR00T 2, post-trained through reinforcement learning, to perform Ethernet cable swapping with a YAM robotic arm.

NVIDIA is careful to say this is testing and evaluation. No production rollout has been announced. The disclosed work is pilots, not deployments.

The market context helps explain the rush. North American companies ordered 10,989 robots worth 723.5 million dollars in the third quarter of 2026, up 24.8 percent in units year over year, with orders for collaborative robots surging 56 percent. Factories are buying robots faster than ever, and the bottleneck is no longer the hardware. It is teaching the robots what to do.

That is also why NVIDIA's wider position matters. The company is not just selling a model. It is building the whole stack: the model, the training framework, the simulation, the benchmarks, and the dataset. If you want to understand how NVIDIA plans to stay at the center of the robot economy, read our earlier piece on its move to own the library every AI builder uses: https://aigrowthlabshakeel.blogspot.com/2026/11/nvidia-wants-to-own-library-every-ai.html

The Honest Limits

GR00T 2 is a research announcement, not a product on a shelf. Several limits deserve plain language.

First, the headline figures are NVIDIA's own, from NVIDIA's own tests. They have not been independently replicated. The 95 percent five-demonstration figure was measured on pre-screened tasks, as explained above.

Second, the model code is not public yet. NVIDIA's research page lists it as available in December. Until then, other labs can read the paper and use the dataset, but they cannot run GR00T 2 itself.

Third, the robots in these tests worked in controlled conditions with monitoring overhead that cost real time. A robot that idles in more than a third of its episodes while an agent watches it is not yet ready for a fast-moving production line.

Fourth, dexterity is still narrow. Folding laundry, swapping a cable, or picking a part are real skills, but they are single skills in fixed settings. A home robot that can do fifty household tasks, in fifty different homes, is still a long way off.

None of this makes GR00T 2 unimportant. It makes it a beginning. The honest way to read this launch is as proof that human video can teach robots, at scale, with far less robot-specific training than anyone expected.

What It Means for Factory, Warehouse, and Home Robots

For factories and warehouses, the near-term effect is faster setup. If a robot can learn a new task from one demonstration, or from a video of a worker doing it, the weeks of programming that used to follow every line change start to shrink. Foxconn testing GR00T 2 on server manufacturing is the shape of that future.

For workers, the question is the same one AI keeps raising. The work that gets automated first is the repetitive, physical work — the same pattern we covered in our look at the AI bubble and what it means for jobs and investment: https://aigrowthlabshakeel.blogspot.com/2026/11/the-ai-bubble-just-lost-1-trillion-in.html

For homes, the timeline is longer. A robot that learns by watching you is the missing piece for a useful home robot, but the home is the hardest environment of all: cluttered, unpredictable, and different in every house. The factory will see GR00T 2 first. The home will wait.

And for anyone who has followed AI agents, there is a satisfying symmetry here. We have spent two years watching AI agents learn to use software by watching humans use software. Now the same idea is coming to the physical world. Our earlier piece on thirteen AI agents left alone on a virtual island showed how far agent behavior has already come: https://aigrowthlabshakeel.blogspot.com/2026/10/13-ai-agents-were-left-alone-on-virtual.html

What to watch next: the December code release, the first independent replications of the 50-task results, and the first pilot that moves from testing to a real production line. The robot that learns by watching you has arrived in the lab. The next question is how fast it reaches the floor.

Sources

NVIDIA Blog — Practice Makes Precise: At CoRL, NVIDIA and Partners Reveal How Robots Rehearse Delicate Work in Simulation: https://blogs.nvidia.com/blog/robotics-simulation-advancements-corl/?ncid=so-othe-843781-vt48

The Robot Report — NVIDIA launches GR00T 2 and Isaac Lab 3.0 at CoRL: https://www.therobotreport.com/nvidia-launches-gr00t-2-isaac-lab-3-0-corl/

Unite.AI — NVIDIA Introduces GR00T 2 for Robots Learning From Human Video: https://www.unite.ai/nvidia-introduces-gr00t-2-for-robots-learning-from-human-video/

NVIDIA's New Robot Learns by Watching You — No Code, No Training Manual

Recent Posts

Comments