World Pilot: Steering Vision-Language-Action Models with World-Action Priors
Summary
World Pilot enhances Vision-Language-Action models by incorporating dynamic scene evolution and trajectory priors from a World-Action Model, achieving state-of-the-art zero-shot performance on manipulation tasks.
View Cached Full Text
Cached at: 06/11/26, 01:41 PM
Paper page - World Pilot: Steering Vision-Language-Action Models with World-Action Priors
Source: https://huggingface.co/papers/2606.12403
Abstract
World Pilot enhances Vision-Language-Action models by incorporating dynamic scene evolution and trajectory priors from a World-Action Model, achieving superior performance in zero-shot out-of-distribution manipulation tasks.
Vision-Language-Action (VLA) models inherit semantic grounding from large-scale pretraining and perform competently across in-distributionmanipulation tasks. This grounding, however, is built on static image-text pairs, whereas manipulation is a continuous, contact-rich process whose dynamics this pretraining cannot capture. We present World Pilot, a VLA framework that augments the policy with priors from aWorld-Action Model(WAM), routed into the decision chain through two complementary pathways.Latent Steeringconditions the perception layer on ascene-evolution latent, andAction Steeringsupplies ananticipated trajectoryas amotion priorto the action generator. Together the two priors equip the VLA with an anticipated view of the scene and a trajectory-level motion hint alongside its semantic conditioning, and the scene-evolution prior remains effective even when supplied by a video-pretrained world model that has not been action-post-trained. World Pilot attains a state-of-the-art Total success rate of 84.7% on the LIBERO-Pluszero-shot OOD benchmarkand the highest success rate on every real-robot setting across fourmanipulation tasks, with the largest margins under shifts in viewpoint, geometry, deformable state, and pose. Project Website: https://world-pilot.github.io/
View arXiv pageView PDFProject pageGitHub9Add to collection
Get this paper in your agent:
hf papers read 2606\.12403
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper1
#### Chedan86/WorldPilot-LIBERO Robotics• Updatedabout 12 hours ago • 1
Datasets citing this paper1
#### Chedan86/WorldPilot-LIBERO-precompute Updatedabout 12 hours ago • 841 • 1
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.12403 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
World Action Modeling with Progressive Visual Planning
ProWAM is a progressive world action model that jointly predicts robot actions and an ordered sequence of sparse visual sub-goals, enabling efficient long-horizon control without dense video generation. It achieves state-of-the-art results on LIBERO-Plus and RoboTwin, plus 70% zero-shot real-world success.
World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal
The paper presents World Action Agent (WAA), a multi-agent system that uses Vision-Language Models (VLMs) for robot manipulation through a visual action workspace with contact views, action rehearsal, and in-view correction, achieving state-of-the-art results on benchmarks.
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
AtlasVLA introduces a dual-memory architecture with persistent world-ego state modeling to overcome perception and task-progress forgetting in vision-language-action models, achieving state-of-the-art long-horizon manipulation from a single wrist camera.
Vision-Language-Action or World-Action? Just Action Models
论文提出 Just Action Model (JAM)，将视觉-语言-动作模型与世界-动作模型解耦为骨干初始化、未来观测目标与推理机制三个可分离选择，并用仅在训练期使用的 foresight loss 在 LIBERO、RoboTwin 等基准上超越 FastWAM/ImageWAM，同时减少 73% 的训练 FLOPs。
WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning
Introduces WCM, a World Critic Model that jointly predicts future latent states and estimates values to improve temporal modeling for Vision-Language-Action reinforcement learning, achieving state-of-the-art results across robotic manipulation benchmarks.