OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies
Summary
OpenRUA 论文提出用现成编码代理(Claude Code、Codex)直接通过原生 ROS 2 控制机器人,作为零样本视觉运动策略,在 CaP-Bench、LIBERO-PRO 和 RoboCasa365 等基准上取得显著的 SOTA 级成绩,并展现出自主编写代码提取 3D 坐标、构建控制库等涌现行为。
View Cached Full Text
Cached at: 10/06/26, 08:53 AM
Paper page - OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies
Source: https://huggingface.co/papers/2610.02459 🤖OpenRUA: Off-the-shelf coding agents (Claude Code, Codex) directly control robots via native ROS 2 aszero-shot visuomotor policies, with no VLA models and no hand-crafted primitives.
🚫Zero-Abstraction: Cuts through over-engineered harness slop and rigid workflows. The agent gets only terminal access to the robot’s native ROS 2 software stack, bypassing bespoke abstraction layers.
🛠️Workspace-as-Harness: Treats robot control as a live software project. A bare workspace directory serves as the entire harness, where: • Perception becomes file I/O: sensor observations are saved and inspected directly as workspace files • Manipulation becomes CLI coding: control scripts and ROS 2 commands are written and executed in the workspace
📊Core Experimental Results: •**CaP-Bench:**99.0% success rate (+18.0% over SOTA ASPIRE), with 100% on Nut assembly (vs. 0-9% for prior methods) •**LIBERO-PRO:**87.0% success (beating Harness VLA’s 82.4%) •**RoboCasa365:**28.1% on unseen composite tasks (learned policies collapse to 5.0%)
💡Striking Emergent Behaviors: • 96.8% episodes: spontaneously writes code to extract 3D coordinates from depth • 95.9% episodes: builds reusable ROS 2 motion-control client libraries • 50.1% episodes: writes closed-loop feedback controllers (force/contact/gap) • Autonomously recovers from physical failures (e.g. fallen objects)
Similar Articles
RobotUse: Allocating Computation, Context, and Decisions
RobotUse 引入一种围绕指定与修订物理动作来组织计算、上下文与决策的机器人智能体 harness,使智能体通过视觉选择目标与位姿、后端处理几何与运动规划,并在执行中持续学习,最终在 RoboLab 上取得 45% 的任务成功率,超过 CaP-X 6.7 个百分点。
Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement
本文提出 Code-Only-As-Policy (COAP) 范式,将机器人本体视为图灵机,用显式代码替代 VLM/VLA 进行决策与状态跟踪,并借助编码智能体实现闭环递归自我改进;在 RoboDojo 的 42 项双臂任务上达到 70.24% 成功率,且测试时无需模型在环。
Co-Evolving Robot Orchestrators and Policies through Deployment
提出了RoboCOP框架,让机器人编排器(VLM orchestrator)与视觉-语言-动作(VLA)策略在部署过程中协同进化,通过自采集演示数据在验证通过时微调策略,实现自改进飞轮,在仿真和真实世界任务上显著提升了泛化成功率。
CoDA: Co-Evolution of Data Engines and Action Policies
CoDA 提出一种仿真数据引擎与机器人动作策略协同演进的框架,通过共享失败与学习进展记忆、视觉语言诊断以及 Text2potential 分阶段势能奖励,将机器人操作任务的分布外成功率从 5.3% 提升至 27.6%,并在 LIBERO-Pro 上取得显著改善。
OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models
OGPO 框架通过语义关键状态重标注和目标条件过程奖励,将视觉-语言-动作模型的后训练从轨迹级模仿转向关键状态可达性学习,从而提升机器人操作策略在偏离演示轨迹时的恢复能力。实验在 LIBERO 和 MetaWorld 上超越标准监督微调,并在 LIBERO-Plus 及新提出的 LIBERO-ReAct 基准中展现出更强的分布偏移鲁棒性。
