World Pilot: Steering Vision-Language-Action Models with World-Action Priors
Summary
World Pilot enhances Vision-Language-Action models by incorporating dynamic scene evolution and trajectory priors from a World-Action Model, achieving state-of-the-art zero-shot performance on manipulation tasks.
View Cached Full Text
Cached at: 06/11/26, 01:41 PM
Paper page - World Pilot: Steering Vision-Language-Action Models with World-Action Priors
Source: https://huggingface.co/papers/2606.12403
Abstract
World Pilot enhances Vision-Language-Action models by incorporating dynamic scene evolution and trajectory priors from a World-Action Model, achieving superior performance in zero-shot out-of-distribution manipulation tasks.
Vision-Language-Action (VLA) models inherit semantic grounding from large-scale pretraining and perform competently across in-distributionmanipulation tasks. This grounding, however, is built on static image-text pairs, whereas manipulation is a continuous, contact-rich process whose dynamics this pretraining cannot capture. We present World Pilot, a VLA framework that augments the policy with priors from aWorld-Action Model(WAM), routed into the decision chain through two complementary pathways.Latent Steeringconditions the perception layer on ascene-evolution latent, andAction Steeringsupplies ananticipated trajectoryas amotion priorto the action generator. Together the two priors equip the VLA with an anticipated view of the scene and a trajectory-level motion hint alongside its semantic conditioning, and the scene-evolution prior remains effective even when supplied by a video-pretrained world model that has not been action-post-trained. World Pilot attains a state-of-the-art Total success rate of 84.7% on the LIBERO-Pluszero-shot OOD benchmarkand the highest success rate on every real-robot setting across fourmanipulation tasks, with the largest margins under shifts in viewpoint, geometry, deformable state, and pose. Project Website: https://world-pilot.github.io/
View arXiv pageView PDFProject pageGitHub9Add to collection
Get this paper in your agent:
hf papers read 2606\.12403
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper1
#### Chedan86/WorldPilot-LIBERO Robotics• Updatedabout 12 hours ago • 1
Datasets citing this paper1
#### Chedan86/WorldPilot-LIBERO-precompute Updatedabout 12 hours ago • 841 • 1
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.12403 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
AtlasVLA introduces a dual-memory architecture with persistent world-ego state modeling to overcome perception and task-progress forgetting in vision-language-action models, achieving state-of-the-art long-horizon manipulation from a single wrist camera.
WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning
Introduces WCM, a World Critic Model that jointly predicts future latent states and estimates values to improve temporal modeling for Vision-Language-Action reinforcement learning, achieving state-of-the-art results across robotic manipulation benchmarks.
APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
Researchers propose APT, a two-stage training method that pretrains action experts on vision-action pairs before integrating language conditioning, significantly improving out-of-distribution instruction generalization for Vision-Language-Action policies.
Learning Transferable Dynamics Priors from Action to World Modeling
This paper introduces A2World, a diffusion-based world model pretrained on large-scale robot manipulation data to learn transferable dynamics priors. The model can be adapted into a real-world simulator (A2World-sim) for policy evaluation or a video-action prediction model (A2World-policy) for action prediction, demonstrating benefits for both simulator-centric and policy-centric robot learning.
World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
This paper introduces World-Language-Action (WLA) models, embodied foundation models that jointly predict textual subtasks, subgoal images, and robot actions from text, images, and robot states, achieving state-of-the-art multi-task and long-horizon learning in simulated and real-world environments.