RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

Hugging Face Daily Papers Papers
robotics benchmark multimodal-agents embodied-ai simulation evaluation

Summary

RobotWorld 提出了一个涵盖 84 项任务的机器人智能体仿真基准，横跨操作、移动操作、运动、驾驶与空中控制，用于评估多模态智能体将数字任务能力迁移到物理世界的表现，并揭示了模型间的能力差异与关键短板。

General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world. To investigate this, we introduce RobotWorld, a challenging simulation testbed for robot use: turning instructions and observations into physical task execution through robot interfaces. Its 84 tasks span manipulation, mobile manipulation, locomotion, driving, and aerial control, with explicit interaction budgets and executable success checks. By analysing task outcomes alongside execution traces, we identify both the capabilities that transfer and the gaps that prevent reliable completion. Furthermore, we find that current agents can construct sophisticated perception and control workflows, including image segmentation, camera calibration, spatial estimation, and dynamics-based computation. These capabilities, however, do not consistently compose into successful behaviour: agents lose task-relevant object states despite reaching commanded poses, fail to correct ineffective actions, recover too late, or mistake unfinished tasks for completion. This uneven transfer also differs across models: Astra succeeds more often on spatial and constrained-contact goals, whereas Opus 5.5 succeeds more often on continuous-balance and timed-interaction goals. By linking these outcomes to execution behaviour, RobotWorld provides both a rigorous proving ground and an empirical account of the remaining capability gaps, thereby establishing concrete targets for training and designing more reliable physical-world agents.
Original Article
View Cached Full Text

Cached at: 10/08/26, 12:20 PM

Paper page - RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

Source: https://huggingface.co/papers/2610.10409 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

General-purposeagentsincreasinglywritecode,usetools,andcompletecomplexdigitaltasks,raisingthequestionofhowfarthesecapabilitiescarryintothephysicalworld.Toinvestigatethis,weintroduceRobotWorld,achallengingsimulationtestbedforrobotuse:turninginstructionsandobservationsintophysicaltaskexecutionthroughrobotinterfaces.Its84tasksspanmanipulation,mobilemanipulation,locomotion,driving,andaerialcontrol,withexplicitinteractionbudgetsandexecutablesuccesschecks.Byanalysingtaskoutcomesalongsideexecutiontraces,weidentifyboththecapabilitiesthattransferandthegapsthatpreventreliablecompletion.Furthermore,wefindthatcurrentagentscanconstructsophisticatedperceptionandcontrolworkflows,includingimagesegmentation,cameracalibration,spatialestimation,anddynamics-basedcomputation.Thesecapabilities,however,donotconsistentlycomposeintosuccessfulbehaviour:agentslosetask-relevantobjectstatesdespitereachingcommandedposes,failtocorrectineffectiveactions,recovertoolate,ormistakeunfinishedtasksforcompletion.Thisuneventransferalsodiffersacrossmodels:Astrasucceedsmoreoftenonspatialandconstrained-contactgoals,whereasOpus5.5succeedsmoreoftenoncontinuous-balanceandtimed-interactiongoals.Bylinkingtheseoutcomestoexecutionbehaviour,RobotWorldprovidesbotharigorousprovinggroundandanempiricalaccountoftheremainingcapabilitygaps,therebyestablishingconcretetargetsfortraininganddesigningmorereliablephysical-worldagents.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.10409

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.10409 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.10409 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Data World: Scaling Data Environments from Scratch

Hugging Face Daily Papers

论文提出 Data World，一个多智能体引擎，从显式机制出发合成数据环境、任务与评分标准，用于训练数据智能体的分析洞察能力；基于其合成任务通过 RL-zero 训练得到 DataWorld-35B-A3B 模型，在 DataWorld-Bench、DAComp、DABstep-Hard 与 JobBench 等基准上显著超越基座模型。

RobotUse: Allocating Computation, Context, and Decisions

Hugging Face Daily Papers

RobotUse 引入一种围绕指定与修订物理动作来组织计算、上下文与决策的机器人智能体 harness，使智能体通过视觉选择目标与位姿、后端处理几何与运动规划，并在执行中持续学习，最终在 RoboLab 上取得 45% 的任务成功率，超过 CaP-X 6.7 个百分点。

RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies

Hugging Face Daily Papers

RoboLab is a high-fidelity simulation benchmarking framework for evaluating task-generalist robotic policies, introducing the RoboLab-120 benchmark with 120 tasks across visual, procedural, and relational competency axes. It enables scalable, realistic task generation and systematic analysis of policy behavior under controlled perturbations to assess true generalization capabilities.