Skip to content

World Models for Robots: Embodiment Is the Honest Test of the World-Model Bet

A robot that plans by imagining outcomes needs a world model that is actionable, not merely plausible — and physical reality falsifies wrong models on contact. That is what makes embodiment the hardest and most honest test of the world-model bet.

By Mehdi9 min read
Share
On this page

Robotics is where the world-model thesis stops being a research aesthetic and becomes falsifiable. A robot that plans by imagining outcomes has to carry an internal model that is not merely plausible but actionable: it must predict what the arm will do to the object, and it must be right enough that a plan built on the prediction survives contact with real physics. When the model is wrong about friction, mass, or contact dynamics, the plan does not merely score badly on a metric — it fails in the world, immediately, and the failure is unforgiving. That is my claim: embodiment forces a kind of grounding that text and even video cannot fake, which makes it simultaneously the hardest frontier for world models and the one that matters most, because it is the only test the model cannot talk its way out of.

I have argued elsewhere that the most credible version of "the next leap may not be a bigger LLM" is the bet on world models — systems that learn an environment's dynamics so they can simulate, plan, and reason about interventions. That essay defended the bet in general. This one is about where the bet gets adjudicated. If world models are the route to grounded, causal AI, then robotics is the courtroom, and physical reality is the judge who cannot be lobbied.

What "planning by imagination" actually means on a robot

Strip the romance out of the phrase and it is precise. A model-based agent holds an approximation of the environment's transition dynamics: given the current state and a candidate action, it predicts the next state. To plan, the agent rolls that model forward over sequences of candidate actions — it imagines — scores the imagined outcomes, and executes the sequence that looks best. This is the core loop of model-based reinforcement learning, and it long predates the current moment. It is Sutton's Dyna architecture from around 1990, in which an agent improves its policy using experience imagined from a learned model, and it is the operating principle behind DeepMind's MuZero and Danijar Hafner's Dreamer line, which learn a latent model of dynamics and train behaviors by rolling it out "in imagination."

For a manipulation task, the imagined rollout is a physics prediction. Suppose the robot must slide a mug across a table to a target and stop it there. The plan is a sequence of gripper forces. To choose it, the world model must predict how the mug accelerates under an applied push, how quickly it decelerates when the push stops, and whether it will tip. Those predictions depend on the mug's mass, the coefficient of friction between mug and table, and the contact geometry at the gripper. Get the friction coefficient wrong by a factor of two and the imagined mug stops in a different place than the real one — the plan overshoots or stalls. Nothing about the plan looked wrong in imagination. The world simply declined to cooperate.

This is the property that separates a robot's world model from a language model's internal representations. Both are learned. But the robot's model is queried in a mode the language model never is: action-conditioned forward prediction against an environment that returns an answer whether or not the model was ready for it.

Grounding with teeth

I have made the abstract version of this argument before — that a joint distribution over text captures correlation but not causation, and that prediction is not understanding because no volume of observational data uniquely fixes the mechanism underneath it. Embodiment is that argument made physical. It supplies the one thing observational learning structurally lacks: interventions with measured consequences.

A language model's only corrective signal is textual — does this continuation match the training distribution? A wrong belief about the world survives indefinitely if the text it produces looks fluent, because fluency is the only thing being scored. A robot's world model faces a harder examiner. Its notion of "understanding" is cashed out as "predicts the next physical state correctly," and when the prediction is wrong, the environment falsifies it on contact. The mug tips. The grasp slips. The foot lands where the model said there was ground and there was not. Reality is the adversarial evaluator, and it does not grade on plausibility. It grades on whether the predicted next state actually occurred.

This is why I think embodiment is a stronger test of understanding than any text benchmark. A text benchmark can be saturated by a model that has learned the shape of the answers without the substance of the mechanism — the measure and the target come apart, and the model optimizes the measure. Physical prediction is far harder to game, because the thing being predicted is not a label a human assigned but a state the world produces on its own, indifferent to what the model hoped. You cannot pattern-match your way to correct contact dynamics. Either the wrench turns the bolt or it cams out. The distinction between modeling appearances and modeling mechanism, which stays philosophical in the text setting, becomes operational the instant an actuator applies force.

The four hard problems, stated honestly

None of this makes robot world models easy. It makes them honest, which is different. Four obstacles are real, and I want to state them without flinching, because a forecast that hides its own difficulty is worthless.

The reality gap (sim-to-real). The obvious way to get cheap experience is to learn in simulation, where you can roll out millions of trajectories without wearing out hardware. The trouble is that simulators approximate physics, and a policy or model learned in sim can come to depend on the simulator's specific inaccuracies. It looks excellent in simulation and degrades on the real robot because the real contact, friction, deformation, and sensor noise differ from the modeled versions. The standard mitigation, domain randomization, trains across a wide distribution of simulated physics so that the real world reads as one more sample from the distribution — and it genuinely helps. But it buys robustness by sacrificing peak performance, and it cannot randomize over dynamics the simulator never represents at all. A simulator that does not model a particular contact mode cannot be randomized into modeling it.

Data scarcity. This is the asymmetry that makes robotics categorically unlike language. Text and video are scraped at internet scale; a frontier language model trains on a corpus no human could read in a thousand lifetimes. Real-world robot data is generated one interaction at a time, in real time, on hardware that breaks, needs resetting, and cannot be parallelized across a data center the way a web crawl can. An hour of robot experience is an hour, and it costs a robot. The consequence is that the grounding signal — the very thing that makes embodiment powerful — is metered out slowly and expensively, exactly inversely to how abundant text is. Video pretraining and simulation are attempts to buy back some of this asymmetry, and they help, but neither delivers the action-conditioned, consequence-labeled data that corrects a dynamics model's estimate of mass and friction.

Safety. A wrong prediction from a chatbot produces a wrong token that a human can catch before it acts. A wrong prediction from a robot can produce force against the world — a dropped object, a collision, an injury — before anyone is in the loop. This reframes what evaluation must establish. It is not enough to be right on average; the system has to be safe in the worst case, precisely in the long-tail situations where its world model is least confident. And a planner that optimizes against its own imperfect model can game that model the way reinforcement-learning agents are known to exploit misspecified rewards — except here the exploited flaw drives real actuators. "Usually right" is the wrong bar. "Safe when wrong" is the bar.

The long tail of contact-rich physics. Free-space motion — moving through air to a waypoint — is forgiving; the model can be somewhat wrong and the plan still lands. Contact is not. When surfaces touch, the dynamics turn stiff and discontinuous: a hair's difference in geometry or applied force is the difference between a part seating, slipping, or jamming. The governing quantities — local friction, precise contact geometry, material deformation — are hard to sense and hard to simulate, and small per-step errors compound across a multi-step plan. Contact-rich manipulation is hard for exactly the reason it is important: it demands the model be right about the physics that is hardest to capture.

Here is the honest summary of the terrain.

Problem Why it's hard Why embodiment still makes it the right test
Reality gap (sim-to-real) Simulators approximate physics; policies exploit the approximation Real deployment falsifies models that only fit the simulator
Data scarcity Real robot data is slow, serial, and costly, unlike scraped text/video The scarce data is the grounded data — consequence-labeled interventions
Safety A wrong prediction produces force, not just a wrong token Forces worst-case, long-tail evaluation instead of average-case fluency
Contact-rich physics Stiff, discontinuous, sensitive to unobserved variables Demands correct mechanism, which pattern-matching cannot fake

Why this is the leading indicator

Now the part that is interpretation, and I will label it as such rather than smuggle it in as fact. If the world-model bet is right — if the path to grounded, causal AI runs through learned models of dynamics rather than larger models of text — then robotics is where that claim meets a reality that cannot be gamed. Which makes progress in robot world models a leading indicator for the whole thesis. Not because robots are the only application that matters, but because they are the application where the model's understanding is continuously audited by physics. A world model that plans well on a text or video benchmark might be modeling appearances. A world model that lets a robot generalize to a manipulation it was never trained on, in a kitchen it never saw, is doing something appearances cannot fully account for.

This is a forecast, and I hold it with calibrated, not evangelical, confidence. Here is the strongest version of the counterargument, stated fairly. It is possible that embodiment is not the crucible but a detour — that the general reasoning capacity we care about is substantially decoupled from motor grounding, that scaled multimodal models absorb enough physical structure from video and text to make explicit interactive grounding unnecessary for most valuable tasks, and that robotics remains a specialized, capital-intensive niche whose difficulties (hardware, data cost, safety) are incidental engineering rather than the true test of intelligence. On this view, the mug and the friction coefficient are a hard robotics problem, not a deep understanding problem, and treating them as a referendum on world models is a category error. I take that seriously. My honest position is that the two views make divergent, checkable predictions — the decoupling view predicts that generalization gains keep arriving from scaled passive data with grounding as a garnish; the embodiment view predicts that robust real-world generalization tracks interactive, consequence-labeled experience and stalls without it. We get to watch which one holds.

What to actually watch

The disciplined way to track this bet is to distinguish what is happening now from what might happen, and to refuse the benchmark score as the signal. A robot world model can post a strong number on a fixed test set the way a language model saturates a benchmark: by fitting the distribution it was tuned on. The number that carries information is generalization to genuinely novel physical situations — new objects, new surfaces, new contact regimes, new environments — evaluated on real hardware, not in the simulator that trained it. Watch for a system that manipulates an object it was never shown, on a surface with friction it never trained against, and succeeds because its model of contact dynamics is right rather than memorized. That is the observation that would move me. And watch the failure structure as much as the success rate: a model that fails gracefully and safely in the long tail is worth more than one that scores higher on the average case and does something dangerous when it is wrong.

Text lets a model be fluent about a world it does not model. A robot is the standing refutation of that comfort, because the world it must model is the one it is about to touch. If the world-model bet is going to pay off, this is where you will see it happen first — or see it fail to.

Frequently asked questions

Isn't large-scale video pretraining enough to give a robot a world model, without the expensive real-world data?
Video helps and is worth doing, but it has a structural blind spot: it is observational. A video shows you what happened, not what would have happened had the gripper closed half a centimeter earlier or pressed with two more newtons. That counterfactual — the action-conditioned rollout — is exactly what planning needs and exactly what passive footage underdetermines, because many different force and contact histories are consistent with the same pixels. You can pretrain the perceptual and dynamics priors on video and get real value from it. But the model's estimates of mass, friction, and contact response only get corrected by data where an action was actually taken and its consequence measured. That is why the field pairs video and simulation pretraining with real-robot fine-tuning rather than treating video as a substitute for it.
What exactly is the sim-to-real (reality) gap?
It is the performance drop when a policy or model trained in simulation is deployed on real hardware, caused by the simulator not matching real physics. Simulators approximate contact, friction, deformation, sensor noise, and latency; a policy can exploit the simulator's specific inaccuracies and look excellent in sim while failing on the robot. Domain randomization — training across a wide distribution of simulated physics parameters so the real world looks like one more sample — is the standard mitigation and genuinely helps, but it trades peak performance for robustness and cannot randomize over dynamics the simulator does not represent at all, such as fine contact modes it never models. The gap shrinks with better simulators and better system identification; it does not vanish, which is why real-world evaluation stays necessary.
Why is contact-rich manipulation singled out as especially hard?
Because contact makes the dynamics stiff, discontinuous, and extremely sensitive to unobserved variables. When surfaces touch, tiny differences in geometry, friction, or applied force produce large, sometimes discontinuous differences in outcome — a part slips, jams, or tips. The governing quantities (local friction coefficients, precise contact geometry, deformation) are hard to sense and hard to simulate accurately, and small errors compound over a multi-step plan. Free-space motion is comparatively forgiving; the model can be somewhat wrong and the plan still works. In contact-rich tasks the model has to be right about precisely the physics that is hardest to capture, which is why it remains a frontier rather than a solved subproblem.
Does a wrong prediction on a robot actually matter more than a wrong prediction from a chatbot?
In a specific, consequential way, yes. A language model's wrong prediction produces a wrong token, which a human can catch, ignore, or edit before it does anything. A robot's wrong prediction can produce force against the world — a dropped object, a collision, an injury — before any human is in the loop to intervene. This changes what evaluation has to prove: not average-case fluency but worst-case behavior under the long tail of situations the model is least sure about. It is also why the reward-hacking and specification-gaming failures studied in reinforcement learning are not academic here; a planner that games its own imperfect world model can drive real actuators into a bad state. Physical consequence raises the bar from 'usually right' to 'safe when wrong.'

Filed under Applied AI. AI that ships, not AI that demos.

Essays like this, in your inbox.

Thoughtful essays. No spam. Unsubscribe anytime.

Applied AI

The Jagged Frontier: AI Is Superhuman and Subhuman at the Same Time

AI capability isn't one number climbing toward "human level." It's a jagged frontier — superhuman at some tasks, worse than a child at others, with no smooth link between them — and that jaggedness, not the average, is what makes deployment hard and "AGI" a category error.

8 min read