Robots Need More than VLA and World Models
Summary
This position paper argues that advancing robot intelligence requires integrating unstructured behavioral data through specialized interfaces for labeling, embodiment mapping, world modeling, and reward inference, rather than relying solely on scaling Vision-Language-Action (VLA) models and world models.
View Cached Full Text
Cached at: 06/08/26, 11:15 AM
Paper page - Robots Need More than VLA and World Models
Source: https://huggingface.co/papers/2606.06556
Abstract
Robot intelligence advancement requires integrating unstructured behavioral data through specialized interfaces for labeling, embodiment mapping, world modeling, and reward inference rather than relying solely on policy scaling.
Generalist robot intelligence is often framed as a policy-scaling problem: collect morerobot demonstrations, train larger Vision-Language-Action (VLA) models, and expect broader generalisation. In this position paper, we argue that this framing is incomplete. The central bottleneck is not only policy learning, but the absence of mechanisms that convert the world’s abundant unstructured behavioural data into grounded robot supervision. Human motion, internet video, simulation rollouts, and interactive demonstrations contain rich information about tasks, goals, contacts, failures, and physical constraints, yet most of this information is not directly usable by robot policies because it lacks embodiment-specific action labels, task semantics, and reward structure. We identify four missing components for the next generation of robotics: data interfaces for autolabelling unstructured behaviour, embodiment interfaces forretargetinghuman motion to robot actions, world-model interfaces for physics-grounded3D reasoning, and reward interfaces for inferring task progress and success from video and language. We survey recent progress in robot foundation models,cross-embodiment datasets, learning from video,world models, and reward modelling, and propose a research agenda for building robotics systems that can learn not only fromrobot demonstrations, but from the broader physical world.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.06556
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.06556 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.06556 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.06556 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
From Foundation to Application: Improving VLA Models in Practice
This paper presents LingBot-VLA 2.0, which enhances VLA foundation models for robotics by improving generalization across tasks and embodiments, expanding action space to whole-body degrees of freedom, and incorporating predictive dynamics modeling for better temporal reasoning.
World Model for Robot Learning: A Comprehensive Survey
This comprehensive survey reviews the literature on world models for robot learning, covering their roles in policy learning, planning, and simulation. It highlights key paradigms, benchmarks, and future directions for predictive modeling in embodied agents.
@svlevine: If you want a robot to do something well, you need to know how to talk to it. If you don't, you can learn, with Semanti…
This paper presents Semantic Action RL, which uses reinforcement learning over Vision-Language-Action (VLA) prompts to enable robots to learn new tasks quickly in the real world.
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
SmolVLA is a compact vision-language-action model that achieves competitive robotic control performance at reduced computational cost, enabling deployment on consumer-grade hardware. It introduces asynchronous inference and leverages community-collected datasets.
IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation
IntentVLA is a history-conditioned visual-language-action framework that improves robot imitation learning stability by encoding short-horizon intents from visual observations, addressing challenges from partial observability and ambiguous observations. It also introduces AliasBench, an ambiguity-aware benchmark for evaluating such methods.