Learning Transferable World-Action Models from Task-Paired Human Videos

Hugging Face Daily Papers Papers
world-models robotics video-pretraining human-demonstrations transfer-learning imitation-learning

Summary

该论文提出WATCH，一种跨视频预训练方法，通过任务配对的人类演示学习可迁移的世界-动作模型，无需人-机器人配对数据；在机器人后训练后将MimicDroid的上下文内成功率从40.9%提升至68.3%（Cosmos Policy）以及从59.8%提升至77.7%（Cosmos 3）。

World-action models inherit broad visual and physical priors from video pre-training. However, standard within-video next-frame prediction allows them to rely on appearance shared between the context and future, rather than learning interactions that transfer across scenes and embodiments. We introduce WATCH, a cross-video pre-training that learns from task-paired human demonstrations collected across diverse environments. Conditioned on the video and actions of one demonstration, the model jointly predicts another demonstration's actions and future frames from its current observation. Predicting one demonstration from another encourages the model to learn task-relevant interactions that transfer across scenes and demonstrators. Our pre-training shows that these capabilities can be learned from readily scalable human-human pairs alone, without requiring matched human-robot demonstrations. Analyses before robot post-training show improved motion alignment across embodiments and better action and video prediction on unseen robots, with representations closer to those learned from robot pairs. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on the unseen levels. Gains persist under context-free post-training across robot embodiments. WATCH outperforms within-video pre-training and co-training baselines, supporting cross-video prediction as a scalable approach to learning transferable world-action representations.
Original Article
View Cached Full Text

Cached at: 10/14/26, 02:52 PM

Paper page - Learning Transferable World-Action Models from Task-Paired Human Videos

Source: https://huggingface.co/papers/2610.14351

Abstract

World-actionmodelsinheritbroadvisualandphysicalpriorsfromvideopre-training.However,standardwithin-videonext-framepredictionallowsthemtorelyonappearancesharedbetweenthecontextandfuture,ratherthanlearninginteractionsthattransferacrossscenesandembodiments.WeintroduceWATCH,across-videopre-trainingthatlearnsfromtask-pairedhumandemonstrationscollectedacrossdiverseenvironments.Conditionedonthevideoandactionsofonedemonstration,themodeljointlypredictsanotherdemonstration’sactionsandfutureframesfromitscurrentobservation.Predictingonedemonstrationfromanotherencouragesthemodeltolearntask-relevantinteractionsthattransferacrossscenesanddemonstrators.Ourpre-trainingshowsthatthesecapabilitiescanbelearnedfromreadilyscalablehuman-humanpairsalone,withoutrequiringmatchedhuman-robotdemonstrations.Analysesbeforerobotpost-trainingshowimprovedmotionalignmentacrossembodimentsandbetteractionandvideopredictiononunseenrobots,withrepresentationsclosertothoselearnedfromrobotpairs.Afterrobotpost-training,WATCHimprovesMimicDroidin-contextsuccessfrom40.9%to68.3%withCosmosPolicyandfrom59.8%to77.7%withCosmos3,withthelargestgainsontheunseenlevels.Gainspersistundercontext-freepost-trainingacrossrobotembodiments.WATCHoutperformswithin-videopre-trainingandco-trainingbaselines,supportingcross-videopredictionasascalableapproachtolearningtransferableworld-actionrepresentations.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.14351

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.14351 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.14351 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.14351 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Native Action-Prior Learning from Videos for World Action Models

Hugging Face Daily Papers

NAVA-WAM 通过从纯观察视频直接预训练动作策略（native action-prior learning），结合流匹配监督与转移结构化联合注意力，避免了间接的表征-控制转移和独立的隐动作模型，仅需少量动作标注即可实现高效的机器人控制与跨域泛化。

Learning Transferable Dynamics Priors from Action to World Modeling

Hugging Face Daily Papers

This paper introduces A2World, a diffusion-based world model pretrained on large-scale robot manipulation data to learn transferable dynamics priors. The model can be adapted into a real-world simulator (A2World-sim) for policy evaluation or a video-action prediction model (A2World-policy) for action prediction, demonstrating benefits for both simulator-centric and policy-centric robot learning.

Long-WAM: Scaling the Context of World-Action Models

Hugging Face Daily Papers

Long-WAM 提出了一种在实时控制约束下扩展因果世界-动作模型上下文的模型-系统框架，发现更长的视觉历史仅在视频基础模型以自回归方式预训练时才显著有效，在 RoboCasa GR-1 上将成功率从 63.3% 提升至 78.7%，并在 Unitree G1 实机上实现 95% 的动态杯子堆叠成功率。