GlobeBench：评估用于可扩展环境模拟的文本世界模型能力

arXiv cs.AI 论文
benchmark world-models text-simulation llm-evaluation reinforcement-learning

摘要

GlobeBench 提出一个包含 1,632 个任务的标准化基准，用于评估文本世界模型模拟 RL 环境状态转移的能力；评测 20 个 LLM 后发现即使是表现最好的 GPT-6 Astra 任务成功率也仅 49.57%，且 631 个任务无任何模型解决，凸显模型作为通用世界模型仍存在显著差距。

arXiv:2610.22708v1 Announce Type: new Abstract: Text-based world models have recently shown potential to simulate complex RL environment state transitions for a diverse set of use cases. However, no standardized benchmark exists to compare, study and improve these models. Through this paper, we attempt to solve this problem by building GlobeBench, a novel benchmark with 1,632 tasks across five separate domains and nineteen simulation skills. The tasks check whether models can predict the environment's response to a specific action given the full agent trajectory by understanding earlier state changes, following environment rules, and predicting the effects of actions on both related and unrelated objects. We compare the predicted environment response with an execution-derived reference, using a hybrid architecture with both deterministic gates and a semantic judge. Among 20 general-purpose LLMs and specialized text world models, GPT-6 Astra achieves the highest task success rate at 49.57%. We find that model accuracy decreases as actions affect a larger portion of the environment. Specifically, when the agent's changes must propagate to dependent objects, task success drops to an average of 5.1% across all 20 models. Although model strengths differ across tasks, 631 tasks remain unsolved by any model, suggesting a significant gap in general LLM capabilities as a world model.
查看原文
查看缓存全文

缓存时间: 2026/10/19 07:04 

# GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations
Source: [https://arxiv.org/abs/2610.22708](https://arxiv.org/abs/2610.22708)
[View PDF](https://arxiv.org/pdf/2610.22708)

> Abstract:Text\-based world models have recently shown potential to simulate complex RL environment state transitions for a diverse set of use cases\. However, no standardized benchmark exists to compare, study and improve these models\. Through this paper, we attempt to solve this problem by building GlobeBench, a novel benchmark with 1,632 tasks across five separate domains and nineteen simulation skills\. The tasks check whether models can predict the environment's response to a specific action given the full agent trajectory by understanding earlier state changes, following environment rules, and predicting the effects of actions on both related and unrelated objects\. We compare the predicted environment response with an execution\-derived reference, using a hybrid architecture with both deterministic gates and a semantic judge\. Among 20 general\-purpose LLMs and specialized text world models, GPT\-6 Astra achieves the highest task success rate at 49\.57%\. We find that model accuracy decreases as actions affect a larger portion of the environment\. Specifically, when the agent's changes must propagate to dependent objects, task success drops to an average of 5\.1% across all 20 models\. Although model strengths differ across tasks, 631 tasks remain unsolved by any model, suggesting a significant gap in general LLM capabilities as a world model\.

## Submission history

From: Nicholas Saban \[[view email](https://arxiv.org/show-email/f0b7fcfa/2610.22708)\] **\[v1\]**Thu, 15 Oct 2026 22:24:56 UTC \(300 KB\)

相似文章

BlenderWorld：面向真实世界参考的 3D 创作视觉智能体基准

Hugging Face Daily Papers

BlenderWorld 是首个专注于真实世界 3D 创作任务的综合基准，包含 223 个任务，通过渲染输出和 2,651 条二元评分标准评估前沿多模态模型，结果显示 GPT-6-Astra 以 79.9 分领先，但所有模型在细粒度 3D 空间一致性方面仍存在显著不足。

MA-Bench：衡量语言模型的操作规划能力

arXiv cs.LG

Scale AI 与 Data Machines 推出 MA-Bench 基准，通过军事任务分析场景评估语言模型在不完整、矛盾信息下的操作规划判断能力；20 个前沿模型中最佳仅答对 62% 的评分项，最弱环节是需要推断的判断。