video-pretraining

该论文提出WATCH，一种跨视频预训练方法，通过任务配对的人类演示学习可迁移的世界-动作模型，无需人-机器人配对数据；在机器人后训练后将MimicDroid的上下文内成功率从40.9%提升至68.3%（Cosmos Policy）以及从59.8%提升至77.7%（Cosmos 3）。

NAVA-WAM 通过从纯观察视频直接预训练动作策略（native action-prior learning），结合流匹配监督与转移结构化联合注意力，避免了间接的表征-控制转移和独立的隐动作模型，仅需少量动作标注即可实现高效的机器人控制与跨域泛化。

Runway Research introduces Praxis-1, its first open-weight world action model for robotics, built on large-scale video pretraining to learn generalist policies across any embodiment and environment. The model is being tested with partners like Noble Machines, Standard Bots, and Ultra ahead of a public release in the coming months.

TT-VidT is a new video pretraining method that efficiently models motion by decoupling the temporal axis, achieving state-of-the-art results on motion-centric benchmarks with reduced computational cost.

This study rigorously tests whether scaling web-video pre-training improves robot task performance, finding that larger models and more compute consistently yield better policies on real industrial tasks, with pre-training quality correlating with performance.

VidaForge is an open research infrastructure that models video data recipes as executable workflows to study their impact on video foundation model pretraining, releasing a dataset of 3.14 million scene-level clips for research.

Dyna-2 is a world-action model pre-trained on over a million hours of human video, showing scaling laws on human data and a human-to-robot transfer scaling law for zero-shot robot performance.

Pantograph introduces Pan, a 4B parameter goal-conditioned model trained on internet video, capable of performing diverse tasks in Minecraft such as fighting mobs, building structures, and exploring. The method uses hindsight relabeling to learn goal-directed behavior during pretraining.

LingBot-Video presents a DiT-based video pretraining framework with Mixture-of-Experts architecture, specialized data augmentation, and multi-dimensional reward system for embodied intelligence applications.

This paper introduces Orca, a world foundation model that learns a unified latent space from multimodal data using next-state-prediction, outperforming specialized baselines on downstream tasks like text generation, image prediction, and embodied action generation.

OpenAI introduced Video PreTraining (VPT), a semi-supervised method that trains neural networks to play Minecraft by learning from 70,000 hours of unlabeled human gameplay video combined with a small labeled dataset. The model learns complex sequential tasks using the native human interface (keyboard and mouse) and demonstrates capabilities like crafting diamond tools and pillar jumping, representing progress toward general computer-using agents.