@tom_doerr: 视觉-语言-动作与世界动作模型研究精选列表 https://github.com/DravenALG/awesome-vla-wam…

精选列表：视觉-语言-动作与世界动作模型研究 https://t.co/NTLcFB8WtD https://t.co/nPkwDzFce4 — # DravenALG/awesome-vla-wam 来源：https://github.com/DravenALG/awesome-vla-wam # 🤖 精彩 VLA 与 WAM 📜 视觉-语言-动作(VLA)与世界动作模型(WAM)及更多研究的精选列表 图片来源：Gemini-Nano-Banana🍌 (https://aistudio.google.com/models/gemini-3-pro-image)。 ## 概述 - 🎯 目标 - 📚 VLA 定义 | WAM 定义 - 🔍 综述 视觉-语言-动作 (VLA) 模型 - 🧠 通用 VLA - 🌐 结合 3D/4D 建模的 VLA - 🔥 结合强化学习的 VLA - 🪶 高效 VLA - 🧪 结合潜在动作的 VLA - 🧭 领域特定 VLA（例如人形、灵巧、触觉） - 🧷 VLA 其他主题 世界动作模型 (WAM) - 🗺️ 通用世界模型 - 🎬 基于视频生成的世界动作模型 - 🌍 基于 VLM 的世界动作模型 - ✨ 从零构建的世界动作模型 其他策略 - 🦾 传统策略 - 🦾 代码即策略 资源 - 💾 机器人数据集 - 👨🏻 自我中心人类数据集 - 📊 基准测试/环境 - 🏞️ 物理引擎 - 🖥️ 硬件 ## 目标 这是一个 VLA 与 WAM 研究的精选列表，系统整理以提供机器人基础模型最新进展的全面视图。我们将持续更新和完善，旨在为机器人基础模型领域的研究人员厘清研究脉络。如果您有值得添加的新论文，欢迎提交 pull request 或发起 issue。与我们一同维护这份高质量的 VLA、WAM 及更多内容的列表。 ## VLA 定义 简而言之，VLA 模型是一种机器人策略，它继承了预训练 VLM 丰富的语言理解和视觉理解能力，为通向通用、语言条件化的机器人策略提供了一条可扩展的路径。我们可以将 VLA 的起源和正式定义追溯到 RT-2 这项工作。 - [⭐️] RT-2, RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. arXiv (https://arxiv.org/abs/2307.15818) 网站 (https://robotics-transformer2.github.io) ## WAM 定义 简而言之，WAM 模型是一种机器人策略，它利用世界建模能力（即预测未来状态）来进行动作预测。我们参考了 DreamZero 这项出色工作，它正式提出了“世界动作模型”这一名称，详情参见原文。 - [⭐️] DreamZero, World Action Models are Zero-shot Policies. arXiv (https://arxiv.org/abs/2602.15922) 网站 (https://dreamzero0.github.io) VLA 与 WAM 之间存在交集：基于预训练 VLM 构建的 WAM 同时属于 VLA 和 WAM。 ## 综述 - Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges. arXiv (https://arxiv.org/abs/2505.04769) 网站 (https://github.com/Applied-AI-Research-Lab/Vision-Language-Action-Models-Concepts-Progress-Applications-and-Challenges) - A Survey on Vision-Language-Action Models for Embodied AI. arXiv (https://arxiv.org/abs/2405.14093) 网站 (https://github.com/yueen-ma/Awesome-VLA) ## 通用 VLA 结合世界建模的 VLA - MolmoAct2, MolmoAct2 Action Reasoning Models for Real-World Deployment. arXiv (https://arxiv.org/abs/2605.02881) 网站 (https://allenai.org/blog/molmoact2) - RLDX-1, RLDX-1 Technical Report. arXiv (https://arxiv.org/abs/2605.03269) 网站 (https://www.rlwrld.ai/en/rldx-1) - StarVLA-α, StarVLA-α: Reducing Complexity in Vision-Language-Action Systems. arXiv (https://arxiv.org/abs/2604.11757) 网站 (https://github.com/starVLA/starVLA) - StarVLA, StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing. arXiv (https://arxiv.org/abs/2604.05014) 网站 (https://github.com/starVLA/starVLA) - VLANeXt, VLANeXt: Recipes for Building Strong VLA Models. arXiv (https://arxiv.org/abs/2602.18532) 网站 (https://dravenalg.github.io/VLANeXt/) - CoVer-VLA, Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment. arXiv (https://arxiv.org/abs/2602.12281) 网站 (https://cover-vla.github.io) - [⭐️] Egoscale, EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data. arXiv (https://arxiv.org/abs/2602.16710) 网站 (https://research.nvidia.com/labs/gear/egoscale/) - HoloBrain-0, HoloBrain-0 Technical Report. arXiv (https://arxiv.org/abs/2602.12062) 网站 (https://horizonrobotics.github.io/robot_lab/holobrain/) - ABot-M0, ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning. arXiv (https://arxiv.org/abs/2602.11236) 网站 (https://amap-cvlab.github.io/ABot-Manipulation/) - SimVLA, SimVLA: A Simple VLA Baseline for Robotic Manipulation. arXiv (https://arxiv.org/abs/2602.18224) 网站 (https://frontierrobo.github.io/SimVLA/) - Lingbot-VLA, A Pragmatic VLA Foundation Model. arXiv (https://arxiv.org/abs/2601.18692) 网站 (https://technology.robbyant.com/lingbot-vla/) - ACoT-VLA, ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2601.11404) 网站 (https://github.com/AgibotTech/ACoT-VLA) - [⭐️] Emergence of Human to Robot Transfer in Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2512.22414) 网站 (https://www.pi.website/research/human_to_robot) - FASTer, FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization. arXiv (https://arxiv.org/abs/2512.04952) - GenieReasoner, Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training. arXiv (https://arxiv.org/abs/2512.24125) 网站 (https://geniereasoner.github.io/GenieReasoner/) - [⭐️] π∗0.6, π∗0.6: a VLA That Learns From Experience. arXiv (https://arxiv.org/abs/2511.14759) 网站 (https://www.pi.website/blog/pistar06) - 10 Open Challenges Steering the Future of Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2511.05936) - AVA-VLA, AVA-VLA: Improving Vision-Language-Action models with Active Visual Attention. arXiv (https://arxiv.org/abs/2511.18960) - AsyncVLA, AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2511.14148) 网站 (https://github.com/YuhuaJiang2002/AsyncVLA) - VLA-0, VLA-0: Building State-of-the-Art VLAs with Zero Modification. arXiv (https://arxiv.org/abs/2510.13054) 网站 (https://vla0.github.io) - X-VLA, X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model. arXiv (https://arxiv.org/abs/2510.10274) 网站 (https://thu-air-dream.github.io/X-VLA/) - ATE, Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance. arXiv (https://arxiv.org/abs/2509.02055) 网站 (https://align-then-steer.github.io) - Discrete Diffusion VLA, Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies. arXiv (https://arxiv.org/abs/2508.20072) 网站 (https://github.com/Liang-ZX/DiscreteDiffusionVLA/tree/libero) - MolmoAct, MolmoAct: Action Reasoning Models that can Reason in Space. arXiv (https://arxiv.org/abs/2508.07917) 网站 (https://allenai.org/blog/molmoact) - SmolVLA, SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics. arXiv (https://arxiv.org/abs/2506.01844) 网站 (https://github.com/huggingface/lerobot) - NORA, NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks. arXiv (https://arxiv.org/abs/2504.19854) 网站 (https://declare-lab.github.io/nora) - CronusVLA, CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling. arXiv (https://arxiv.org/abs/2506.19816) 网站 (https://lihaohn.github.io/CronusVLA.github.io/) - [⭐️] π0.5, π0.5: a Vision-Language-Action Model with Open-World Generalization. arXiv (https://arxiv.org/abs/2504.16054) 网站 (https://www.pi.website/blog/pi05) - [⭐️] Gemini Robotics, Gemini Robotics: Bringing AI into the Physical World. arXiv (https://arxiv.org/abs/2503.20020) 网站 (https://deepmind.google/models/gemini-robotics/) - [⭐️] OpenVLA-OFT, Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. arXiv (https://arxiv.org/abs/2502.19645) 网站 (https://openvla-oft.github.io) - ChatVLA, ChatVLA: Unified Multimodal Understanding and Robot Control with Vision-Language-Action Model. arXiv (https://arxiv.org/abs/2502.14420) 网站 (https://chatvla.github.io) - [⭐️] FAST, FAST: Efficient Action Tokenization for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2501.09747) 网站 (https://www.pi.website/research/fast) - CogACT, CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation. arXiv (https://arxiv.org/abs/2411.19650) 网站 (https://cogact.github.io) - RoboVLMs, Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2412.14058) 网站 (https://robovlms.github.io) - [⭐️] π0, π0: A Vision-Language-Action Flow Model for General Robot Control. arXiv (https://arxiv.org/abs/2410.24164) 网站 (https://www.pi.website/blog/pi0) - [⭐️] OpenVLA, OpenVLA: An Open-Source Vision-Language-Action Model. arXiv (https://arxiv.org/abs/2406.09246) 网站 (https://openvla.github.io) - RoboFlamingo, Vision-Language Foundation Models as Effective Robot Imitators. arXiv (https://arxiv.org/abs/2311.01378) 网站 (https://roboflamingo.github.io) - [⭐️] RT-2, RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. arXiv (https://arxiv.org/abs/2307.15818) 网站 (https://robotics-transformer2.github.io) ## 结合 3D/4D 建模的 VLA - 4D-VLA, 4D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibration. arXiv (https://arxiv.org/abs/2506.22242) 网站 (https://github.com/LogosRoboticsGroup/4D-VLA) - 3D CAVLA, 3D CAVLA: Leveraging Depth and 3D Context to Generalize Vision Language Action Models for Unseen Tasks. arXiv (https://arxiv.org/abs/2505.05800) 网站 (https://3d-cavla.github.io) - SpatialVLA, SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model. arXiv (https://arxiv.org/abs/2501.15830) 网站 (https://spatialvla.github.io) - [⭐️] 3D-VLA, 3D-VLA: A 3D Vision-Language-Action Generative World Model. arXiv (https://arxiv.org/abs/2403.09631) 网站 (https://github.com/UMass-Embodied-AGI/3D-VLA) ## 结合强化学习的 VLA - EVOLVE-VLA, EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2512.14666) 网站 (https://showlab.github.io/EVOLVE-VLA/) - WMPO, WMPO: World Model-based Policy Optimization for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2511.09515) 网站 (https://wm-po.github.io) - SRPO, SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2511.15605) - World-Env, World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training. arXiv (https://arxiv.org/abs/2509.24948) 网站 (https://github.com/amap-cvlab/world-env) - SimpleVLA-RL, SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning. arXiv (https://arxiv.org/abs/2509.09674) 网站 (https://github.com/PRIME-RL/SimpleVLA-RL) - VLA-Reasoner, VLA-Reasoner: Empowering Vision-Language-Action Models with Reasoning via Online Monte Carlo Tree Search. (不严格属于RL，属于规划) arXiv (https://arxiv.org/abs/2509.22643) - ThinkAct, ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning. arXiv (https://arxiv.org/abs/2507.16815) 网站 (https://jasper0314-huang.github.io/thinkact-vla/) - TGRPO, TGRPO: Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization. arXiv (https://arxiv.org/abs/2506.08440) - VLA-RL, VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning. arXiv (https://arxiv.org/abs/2505.18719) 网站 (https://github.com/GuanxingLu/vlarl) - RIPT-VLA, Interactive Post-Training for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2505.17016) 网站 (https://ariostgx.github.io/ript_vla/) - GRAPE, GRAPE: Generalizing Robot Policy via Preference Alignment. arXiv (https://arxiv.org/abs/2411.19309) 网站 (https://grape-vla.github.io) ## 高效 VLA - PokeVLA, PokeVLA: Empowering Pocket-Sized Vision-Language-Action Model with Comprehensive World Knowledge Guidance. arXiv (https://arxiv.org/abs/2604.20834) 网站 (https://getterupper.github.io/PokeVLA) - HBVLA, HBVLA: Pushing 1-Bit Post-Training Quantization for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2602.13710) - MergeVLA, MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent. arXiv (https://arxiv.org/abs/2511.18810) 网站 (https://mergevla.github.io) - VLA-Adapter, VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Model. arXiv (https://arxiv.org/abs/2509.09372) 网站 (https://vla-adapter.github.io) - FLOWER, FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policies. arXiv (https://arxiv.org/abs/2509.04996) 网站 (https://intuitive-robots.github.io/flower_vla/) - [⭐️] TinyVLA, TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation. arXiv (https://arxiv.org/abs/2409.12514) 网站 (https://tiny-vla.github.io) ## 结合潜在动作的 VLA - From Pixels to Tokens, From Pixels to Tokens: A Systematic Study of Latent Action Supervision for Vision-Language-Action Models. arXiv (https://arxiv.org/abs/2605.04678) 网站 (https://github.com/RUCKBReasoning/From_Pixels_to_Tokens) - Motus, Motus: A Unified Latent Action World Model. arXiv (https://arxiv.org/abs/2512.13030) 网站 (https://motus-robotics.github.io/motus) - [⭐️] GR00T N1, GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv (https://arxiv.org/abs/2503.14734) 网站 (https://developer.nvidia.com/isaac/gr00t) - [⭐️] LAPA, Latent Action Pretraining from Videos. arXiv (https://arxiv.org/abs/2410.11758) 网站 (https://latentactionpretraining.github.io) ## 领域特定 VLA（例如人形、灵巧、触觉） - METIS, METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model. arXiv (https://arxiv.org/abs/2511.17366) 网站 (https://aureleopku.github.io/METIS/) - Tactile-VLA, Tactile-VLA: Unlocking Vision-Language-Action Model’s Physical Knowledge for Tactile Generalization. arXiv (https://arxiv.org/abs/2507.09160) 网站 (https://jialeihuang.github.io/tactileVLA.github.io/) - CombatVLA, CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Games. arXiv (https://arxiv.org/abs/2503.09527) 网站 (https://combatvla.github.io) - Humanoid-VLA, Humanoid-VLA: Towards Universal Humanoid Control with Visual Integration. arXiv (https://arxiv.org/abs/2502.14795) ## VLA 其他主题 - DynamicVLA, DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulation. arXiv (https://arxiv.org/abs/2601.22153) 网站 (https://github.com/hzxie/DynamicVLA) - TwinVLA, TwinVLA: Data-Efficient Bimanual Manipulation with Twi