Robots Need More than VLA and World Models
Summary
This position paper argues that advancing robot intelligence requires integrating unstructured behavioral data through specialized interfaces for labeling, embodiment mapping, world modeling, and reward inference, rather than relying solely on scaling Vision-Language-Action (VLA) models and world models.
View Cached Full Text
Cached at: 06/08/26, 11:15 AM
Paper page - Robots Need More than VLA and World Models
Source: https://huggingface.co/papers/2606.06556
Abstract
Robot intelligence advancement requires integrating unstructured behavioral data through specialized interfaces for labeling, embodiment mapping, world modeling, and reward inference rather than relying solely on policy scaling.
Generalist robot intelligence is often framed as a policy-scaling problem: collect morerobot demonstrations, train larger Vision-Language-Action (VLA) models, and expect broader generalisation. In this position paper, we argue that this framing is incomplete. The central bottleneck is not only policy learning, but the absence of mechanisms that convert the world’s abundant unstructured behavioural data into grounded robot supervision. Human motion, internet video, simulation rollouts, and interactive demonstrations contain rich information about tasks, goals, contacts, failures, and physical constraints, yet most of this information is not directly usable by robot policies because it lacks embodiment-specific action labels, task semantics, and reward structure. We identify four missing components for the next generation of robotics: data interfaces for autolabelling unstructured behaviour, embodiment interfaces forretargetinghuman motion to robot actions, world-model interfaces for physics-grounded3D reasoning, and reward interfaces for inferring task progress and success from video and language. We survey recent progress in robot foundation models,cross-embodiment datasets, learning from video,world models, and reward modelling, and propose a research agenda for building robotics systems that can learn not only fromrobot demonstrations, but from the broader physical world.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.06556
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.06556 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.06556 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.06556 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
World Model for Robot Learning: A Comprehensive Survey
This comprehensive survey reviews the literature on world models for robot learning, covering their roles in policy learning, planning, and simulation. It highlights key paradigms, benchmarks, and future directions for predictive modeling in embodied agents.
IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation
IntentVLA is a history-conditioned visual-language-action framework that improves robot imitation learning stability by encoding short-horizon intents from visual observations, addressing challenges from partial observability and ambiguous observations. It also introduces AliasBench, an ambiguity-aware benchmark for evaluating such methods.
@FeitengLi: Just said this morning: The intelligence of embodied intelligence should copy the homework of LLM + RL + Agentic. Here it is: Agentic VLA crushes the models of leading embodied companies across the board https://x.com/FeitengLi/status/205909864717506193...
Proposes the Agentic-VLA framework, introducing agents into the VLA loop, enabling the vision-language-action model to self-evolve and surpass existing leading embodied models on all metrics.
@drfeifei: https://x.com/drfeifei/status/2062247238143996275
Fei-Fei Li and the World Labs team present a functional taxonomy of world models, distinguishing between renderers, physics engines, and other components within the reinforcement learning loop, and arguing that spatial intelligence is AI's next frontier.
AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding
AffordanceVLA introduces a unified framework using structured affordance forecasting as an intermediate representation to improve perception-action mapping in robotic manipulation, leveraging vision-language models and a Mixture-of-Transformer architecture.