Learning Transferable Dynamics Priors from Action to World Modeling
Summary
This paper introduces A2World, a diffusion-based world model pretrained on large-scale robot manipulation data to learn transferable dynamics priors. The model can be adapted into a real-world simulator (A2World-sim) for policy evaluation or a video-action prediction model (A2World-policy) for action prediction, demonstrating benefits for both simulator-centric and policy-centric robot learning.
View Cached Full Text
Cached at: 06/30/26, 07:34 AM
Paper page - Learning Transferable Dynamics Priors from Action to World Modeling
Source: https://huggingface.co/papers/2606.29501
Abstract
Action-conditioned world modeling enables transferable dynamics priors for robot learning through pretraining on large-scale manipulation data, supporting both simulator-based policy evaluation and video-action prediction.
We studyaction-conditionedworld modelingas a scalable way to learn transferabledynamics priorsfor robot learning. Bypretraininga model to predict how actions drive visual scene evolution, the resulting world model captures reusable interaction dynamics beyond appearance-level video generation. Concretely, we pretrain amulti-view interactivebasediffusion world model, A2World, on large-scalerobot manipulationdata with real action annotations. We validate the learneddynamics priorsfrom two complementary perspectives. First, we adapt A2World into a task- or scene-specialized real-world simulator, A2World-sim, whose long-horizon rollouts support simulator-based policy evaluation and scalable what-if analysis by replacing real-robot rollouts with world model rollouts. Second, starting from the same pretrained weights, we adapt A2World into avideo-action joint predictionmodel, A2World-policy, that predicts actions under visual and instruction conditioning. Experiments across simulation benchmarks and real-robot settings demonstrate thataction-conditionedworld modelpretrainingyields transferabledynamics priorsthat benefit both simulator-centric and policy-centric robot learning.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.29501
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.29501 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.29501 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.29501 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
InternW0-Δ is a unified world action model that integrates visual dynamics, semantics, geometry, and actions using a Mixture-of-Transformers framework, pretrained on over 20K hours of open data for robot manipulation. It demonstrates strong performance in simulations and real robots and will be open-sourced.
@_akhaliq: Native Action-Prior Learning from Videos for World Action Models paper: https://huggingface.co/papers/2610.03391…
A new paper titled "Native Action-Prior Learning from Videos for World Action Models" is shared on Hugging Face Papers, exploring how to learn action priors directly from videos for world action models.
Transfer from simulation to real world through learning deep inverse dynamics model
This paper proposes a method to bridge the simulation-to-real-world gap in robotics by learning a deep inverse dynamics model that maps desired next states (from simulation) to appropriate real-world actions. The approach is evaluated against baselines like output error control and Gaussian dynamics adaptation.
Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models
This paper introduces DROPJ, a human-centred method for safely training and deploying agent policies by learning a world model from real-world trajectories, then eliciting human preferences with justifications to train a reward model for model predictive control. Experiments show that using human-generated simulated trajectories and justifications improves safety and reduces computational cost.
GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
This paper introduces GE-Act 2.0, a world-action model pretrained from scratch to enable scalable zero-shot robotic manipulation with improved success rates across diverse tasks and conditions.