HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution
Summary
HumanoidToolBench 提出一个18项任务的基准,评估人形机器人从工具选择到移动执行的能力,并发布包含3.1k条演示的 ToolBook 数据集(仿真与真实 Unitree G1 采集),揭示了工具选择与任务成功执行之间的显著差距。
View Cached Full Text
Cached at: 10/15/26, 02:55 AM
Paper page - HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution
Source: https://huggingface.co/papers/2610.02089 Can humanoids choose the right tool and then actually use it?
We introduce HumanoidToolBench, an 18-task benchmark evaluating humanoid tool selection and execution across three scenarios, three execution levels, and two tool-set modes. We also introduce ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1.
Evaluating seven policies in simulation and three on the real robot reveals a substantial gap between selecting a suitable tool and successfully completing the task. Additional GR00T N1.7 probes examine generalization to unseen tools and behavior under unrelated instructions.
Explore the benchmark and demonstrations on our project page. We welcome feedback from the humanoid learning and tool-use communities!
Similar Articles
RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments
RobotWorld 提出了一个涵盖 84 项任务的机器人智能体仿真基准,横跨操作、移动操作、运动、驾驶与空中控制,用于评估多模态智能体将数字任务能力迁移到物理世界的表现,并揭示了模型间的能力差异与关键短板。
HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation
HandsOn 是一个跨具身灵巧操作仿真基准与平台,涵盖 5 种机械臂、6 种灵巧手和约 200 个资产,并提供 18,000 条人类演示与 66 项任务。评测显示当前多任务策略与基于基础模型的控制存在巨大差距,最佳多任务策略在 Core-50 上成功率仅 29.1%。
TOBench: A Task-Oriented Omni-Modal Benchmark for Real-World Tool-Using Agents
TOBench is a new benchmark for evaluating AI agents on real-world, task-oriented tool use with multimodal inputs and closed-loop verification. Experiments show top models like Qwen 3.5 Plus achieve only 41% success, far below the 94% human benchmark, highlighting a significant gap.
UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs
UserToolBench is a new benchmark for evaluating personalized decision-making in tool-use LLMs, testing whether models can infer latent user preferences, decide when to clarify, and produce user-aligned tool-call trajectories under incomplete information. Experiments show current models struggle with multi-tool coordination and long-horizon consistency.
ToolBottleneckBench: Measuring Tool-Agent Performance under Controlled Pressures and Fixed Harness Budgets
We present ToolBottleneckBench, a benchmark measuring tool-agent performance under controlled pressures (tool catalog, visible menu, certified workflow, record encoding) and fixed harness budgets, finding that only the visible tool menu heavily penalizes models — largely via truncation — and most failures end at harness limits.