MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
Summary
The MineExplorer benchmark evaluates multimodal large language model agents' open-world exploration abilities in Minecraft using atomic and multi-hop tasks designed through multi-agent synthesis. Experiments show that open-world exploration remains challenging, with strong models degrading sharply over longer trajectories.
View Cached Full Text
Cached at: 06/02/26, 03:35 PM
Paper page - MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
Source: https://huggingface.co/papers/2605.30931
Abstract
MineExplorer benchmark evaluates multimodal large language models’ open-world exploration capabilities in Minecraft through atomic and multi-hop tasks designed via multi-agent synthesis.
Multimodal large language models(MLLMs) have shown strong capabilities in perception, reasoning, and action generation. However, their ability to sustain exploration in dynamic open worlds remains unclear. Existing embodied and game-based benchmarks often compress interaction into short-horizon tasks or entangle success with domain-specific game mechanics. In this paper, we introduce MineExplorer benchmark for evaluatingopen-world explorationcapabilities of MLLM agents inMinecraft. We first filteratomic taskswhose solutions rely heavily onMinecraft-specific knowledge to better reflect general open-world reasoning. Then we organize the benchmark around aReAct-style capability formulationand composeatomic tasksinto implicitmulti-hop tasks. To further construct reliable instances, MineExplorer uses amulti-agent synthesisworkflow that jointly designstask graphs,sandbox scenes, and rule-basedmilestone evaluators. Human evaluation shows that themulti-agent synthesisworkflow produces significantly more reliable instances than a single-agent baseline. Experiments with advanced MLLM agents show thatopen-world explorationremains challenging, as strong models can handle many single-hop tasks but degrade sharply when hidden prerequisites must be coordinated over longer trajectories. Further analysis finds that task difficulty tracks agent completion, and larger models or thinking modes do not consistently translate into better performance. Code and dataset are available at https://github.com/Jometeorie/MineExplorer.
View arXiv pageView PDFGitHub2Add to collection
Get this paper in your agent:
hf papers read 2605\.30931
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.30931 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.30931 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.30931 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Look Before You Leap: Autonomous Exploration for LLM Agents
This paper identifies autonomous exploration as a critical capability for LLM agents and proposes the Explore-then-Act paradigm, which decouples information gathering from task execution to improve adaptability and real-world performance. It also introduces Exploration Checkpoint Coverage as a verifiable metric for evaluating exploration breadth.
MineCEraft: Evaluating Language Models as Construction Engineers in the World of Minecraft
The paper introduces MineCEraft, an open-source benchmark for evaluating language models in Minecraft construction tasks, revealing limitations in LLMs for physical, spatial reasoning.
Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft
The paper introduces TickingCollabBench, a Minecraft-based multi-agent benchmark for time-sensitive complementary collaboration tasks with dynamic environments, and demonstrates that LLMs frequently fail under such conditions compared to a global-knowledge oracle.
New LLM Coordination Benchmark - Benchmarking Open-Ended Multi-Agent Coordination in Language Agents [R]
Introduces a new benchmark for evaluating multi-agent coordination in LLMs, finding that most models struggle with long-horizon open-ended tasks, but Gemini 3.1 Pro performs comparably to trained MARL agents on the hardest setting.
Multi-Agent LLMs Fail to Explore Each Other
This paper identifies that current LLM agents fail to systematically explore their peers, leading to poor coordination, and introduces MACE, a lightweight framework using contextual bandits for effective peer selection.