OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies

Hugging Face Daily Papers Papers

Summary

OpenRUA 论文提出用现成编码代理(Claude Code、Codex)直接通过原生 ROS 2 控制机器人,作为零样本视觉运动策略,在 CaP-Bench、LIBERO-PRO 和 RoboCasa365 等基准上取得显著的 SOTA 级成绩,并展现出自主编写代码提取 3D 坐标、构建控制库等涌现行为。

Coding agents are extending their reach into the physical world by writing and executing robot control programs. One might expect the agents to use the existing mature software stack that engineers have developed over decades to access sensors and control motion. Yet prior work primarily engineers complex custom harnesses to orchestrate agents for robot use, particularly by prescribing specialized workflows and providing bespoke interfaces. This raises the question: "Is such additional harness engineering necessary?" We introduce OpenRUA, a zero-abstraction harness that bypasses bespoke abstraction layers by providing off-the-shelf coding agents with only terminal access to the robot's native software interface ROS 2. OpenRUA employs a minimalist workspace-as-harness design, only offering ROS 2 documentation and basic tools while leaving the coding agent to organize its own work without orchestrating any agentic workflow. Within this workspace, OpenRUA recasts perception as file I/O and manipulation as coding. With Claude Code powered by Claude Opus 5, OpenRUA achieves success rates of 99.0% on CaP-Bench and 87.0% on LIBERO-PRO, demonstrating that an off-the-shelf coding agent can serve as a zero-shot visuomotor policy through the robot's native interface, without bespoke primitives or task-specific training. Under this minimalist design, further analysis reveals striking emergent behaviors of coding agents: (1) For perception, the agent spontaneously writes programs that process raw sensory inputs and derive metric measurements in 96.80% of episodes. (2) For manipulation, the agent spontaneously builds motion-control clients (e.g., gripper control) in 95.87% of episodes and closed-loop control programs (e.g., adjusting motion based on sensor feedback) in 50.13% of episodes. Our code is available at https://github.com/terminalworld/OpenRUA.
Original Article
View Cached Full Text

Cached at: 10/06/26, 08:53 AM

Paper page - OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies

Source: https://huggingface.co/papers/2610.02459 🤖OpenRUA: Off-the-shelf coding agents (Claude Code, Codex) directly control robots via native ROS 2 aszero-shot visuomotor policies, with no VLA models and no hand-crafted primitives.

🚫Zero-Abstraction: Cuts through over-engineered harness slop and rigid workflows. The agent gets only terminal access to the robot’s native ROS 2 software stack, bypassing bespoke abstraction layers.

🛠️Workspace-as-Harness: Treats robot control as a live software project. A bare workspace directory serves as the entire harness, where: • Perception becomes file I/O: sensor observations are saved and inspected directly as workspace files • Manipulation becomes CLI coding: control scripts and ROS 2 commands are written and executed in the workspace

📊Core Experimental Results: •**CaP-Bench:**99.0% success rate (+18.0% over SOTA ASPIRE), with 100% on Nut assembly (vs. 0-9% for prior methods) •**LIBERO-PRO:**87.0% success (beating Harness VLA’s 82.4%) •**RoboCasa365:**28.1% on unseen composite tasks (learned policies collapse to 5.0%)

💡Striking Emergent Behaviors: • 96.8% episodes: spontaneously writes code to extract 3D coordinates from depth • 95.9% episodes: builds reusable ROS 2 motion-control client libraries • 50.1% episodes: writes closed-loop feedback controllers (force/contact/gap) • Autonomously recovers from physical failures (e.g. fallen objects)

🔗Paper·GitHub·Demo Video overview

Similar Articles

RobotUse: Allocating Computation, Context, and Decisions

Hugging Face Daily Papers

RobotUse 引入一种围绕指定与修订物理动作来组织计算、上下文与决策的机器人智能体 harness,使智能体通过视觉选择目标与位姿、后端处理几何与运动规划,并在执行中持续学习,最终在 RoboLab 上取得 45% 的任务成功率,超过 CaP-X 6.7 个百分点。

Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

Hugging Face Daily Papers

本文提出 Code-Only-As-Policy (COAP) 范式,将机器人本体视为图灵机,用显式代码替代 VLM/VLA 进行决策与状态跟踪,并借助编码智能体实现闭环递归自我改进;在 RoboDojo 的 42 项双臂任务上达到 70.24% 成功率,且测试时无需模型在环。

Co-Evolving Robot Orchestrators and Policies through Deployment

Hugging Face Daily Papers

提出了RoboCOP框架,让机器人编排器(VLM orchestrator)与视觉-语言-动作(VLA)策略在部署过程中协同进化,通过自采集演示数据在验证通过时微调策略,实现自改进飞轮,在仿真和真实世界任务上显著提升了泛化成功率。

CoDA: Co-Evolution of Data Engines and Action Policies

Hugging Face Daily Papers

CoDA 提出一种仿真数据引擎与机器人动作策略协同演进的框架,通过共享失败与学习进展记忆、视觉语言诊断以及 Text2potential 分阶段势能奖励,将机器人操作任务的分布外成功率从 5.3% 提升至 27.6%,并在 LIBERO-Pro 上取得显著改善。

OGPO: Offline Goal-conditioned Policy Optimization for Recoverable Vision-Language-Action Models

arXiv cs.AI

OGPO 框架通过语义关键状态重标注和目标条件过程奖励,将视觉-语言-动作模型的后训练从轨迹级模仿转向关键状态可达性学习,从而提升机器人操作策略在偏离演示轨迹时的恢复能力。实验在 LIBERO 和 MetaWorld 上超越标准监督微调,并在 LIBERO-Plus 及新提出的 LIBERO-ReAct 基准中展现出更强的分布偏移鲁棒性。