Mine Odyssey：真实环境中的空间智能体智能基准评测

# Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild
Source: [https://arxiv.org/html/2610.11328](https://arxiv.org/html/2610.11328)
###### Abstract

Advances in foundation models are driving efforts to introduce agents to assist people in the physical world\. Such agents require agentic spatial intelligence: exploring unfamiliar environments, updating spatial understanding through interaction, and adapting actions based on feedback to sustain progress toward a sequence of goals\. Evaluating these capabilities requires environments that reflect the diversity and structural complexity of real\-world places, yet existing benchmarks typically cover only a limited range of spatial layouts, scales, and traversal requirements\. To address this gap, we introduceMine Odyssey, a benchmark for evaluating agentic spatial intelligence using Minecraft reconstructions of real\-world locations\. It comprises 180 tasks covering 30 such locations across 20 countries and regions on five continents, including 20 outdoor and 10 indoor settings\. These settings are selected to provide broad coverage across spatial scales, layouts, terrains, and connectivity patterns, ranging from the urban streets of Midtown Manhattan to the rural Entrup, and from the steep terrain of Santa Lucía Hill to Buckingham Palace\. Within each reconstruction, we select meaningful locations as waypoints \(*e\.g\.*, historic landmarks, buildings, and rooms\) and manually verify their accessibility\. Each task provides a natural\-language instruction specifying which waypoints to visit and in what order\. Completing these tasks requires agents to find accessible routes and entrances, open doors, and move between levels using stairs and ladders, while monitoring their progress and recovering from navigation errors\. Across eight evaluated state\-of\-the\-art models, GPT\-6 Astra achieves the highest success rate of 85\.6%\. However, the second\-best model, Claude Opus 5\.5, completes 73\.9% of tasks, while the strongest evaluated open\-weight model, DeepSeek\-V4\.1\-Flash, reaches 23\.9%, highlighting substantial room for improvement in the agentic spatial intelligence of current models\. Comprehensive analyses and ablation studies onMine Odysseyreveal current models’ limitations and provide insights for advancing agentic spatial intelligence\.

11footnotetext:Equal contribution\.## 1Introduction

Advances in foundation models are opening new possibilities for agents to move beyond digital workflows and interact with the physical world\([Gemini Robotics Team et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib45)\)\. Recent evaluations of frontier models such as GPT\-6 Astra\([OpenAI, 2026b](https://arxiv.org/html/2610.11328#bib.bib49)\)using robot\-control tools have begun to explore this possibility in embodied settings\([RoboDojo Team, 2026](https://arxiv.org/html/2610.11328#bib.bib46)\)\. Potential applications for such agents range from robotic guide dogs that assist people with visual impairments\([Hayamizu et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib47)\)to autonomous delivery robots that transport goods\([Wang et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib48)\)\. In these settings, agents must adapt their movements and interactions based on the spatial relationships in their surroundings, including how spaces connect and which objects and locations are reachable\. We refer to this capability as*agentic spatial intelligence*: the ability to proactively acquire and continually update spatial information through interaction, use it to guide observation and action, and adapt those actions as new evidence emerges\([Pei et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib56);[Deng et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib58)\)\.

Existing benchmarks often evaluate agents in relatively narrow settings, such as household environments\([Yang et al\., 2025b](https://arxiv.org/html/2610.11328#bib.bib14);[Qiao et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib15)\)or urban navigation\([Wang et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib48);[Dalal et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib41);[Wang et al\., 2026b](https://arxiv.org/html/2610.11328#bib.bib57);[Ju et al\., 2026b](https://arxiv.org/html/2610.11328#bib.bib31)\), leaving their ability to operate across a broader range of spatial structures and traversal requirements insufficiently tested\. However, real\-world tasks often arise in environments with varied layouts and traversal requirements\. An agent may need to follow irregular paths through a rural settlement, find a traversable route across a steep hillside, or navigate entrances, corridors, and multiple levels within a palace or stadium\. These environments differ not only in appearance, but also in scale, connectivity, access constraints, and elevation\. This diversity, together with the need to maintain spatial information and revise plans throughout execution, makes reliable performance across environments particularly challenging\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/task-overview4.png)Figure 1:Overview ofMine Odysseyenvironments and task construction\. \(a\) 30 Minecraft reconstructions and geographic coverage\. \(b\) Task construction with White House and Ueno Park examples: reference locations are mapped to waypoints and ordered into navigation tasks\. S denotes the start; numbered markers and dashed lines indicate visit order\.To address these challenges, we introduceMine Odyssey, a benchmark for evaluating agentic spatial intelligence through diverse, realistic, and long\-horizon navigation tasks in Minecraft reconstructions of real\-world locations\. Here we choose Minecraft as our evaluation platform for its rich ecosystem of community\-created mods and world saves\. Detailed reconstructions of cities, landscapes, and buildings from around the world provide diverse spatial environments\. The platform also allows episodes to be reset to specified initial conditions and task completion to be verified automatically from the agent’s in\-game position\.Mine Odysseyspans 30 real\-world locations recreated in Minecraft, with 180 tasks across 20 countries and regions on five continents, including 20 outdoor and 10 indoor environments\. To capture the diversity and structural complexity of real\-world spatial environments, we curate spots spanning dense urban districts, rural settlements, mountainous terrain, palace complexes, stadiums, and historic passenger ships\. Representative examples include Midtown Manhattan, Cape Town, Entrup, Santa Lucía Hill, Buckingham Palace, SoFi Stadium, and RMS Queen Mary\. These places provide variation in street connectivity, terrain, room arrangement, entrances, and vertical circulation\. Minecraft makes these structures interactive through first\-person movement, collision, and actions such as opening doors and climbing ladders and stairs\.

Since these maps don’t come with annotated destinations for evaluation, we mark named locations as waypoints that serve as destinations or intermediate stops in navigation tasks\. To preserve realism and semantic relevance, we select recognizable places such as historic landmarks, palace rooms, stadium facilities, and building entrances rather than arbitrary points on the map\. Each task is then defined as an ordered sequence of these waypoints, requiring agents to find feasible paths and execute the movements and interactions needed to reach each destination in order\.

We also provide a programmable agent framework that connects visual observations with game APIs and keyboard–mouse control, allowing agents to compose and execute their own action sequences\. Agents can inspect intermediate feedback and interrupt ongoing commands, supporting closed\-loop interaction as the environment evolves\. Isolated execution prevents access to hidden task data and interference between runs, while independent arrival checks support automated, parallel evaluation\.

Our evaluation reveals substantial differences in task success and decision efficiency\. GPT\-6 Astra\([OpenAI, 2026b](https://arxiv.org/html/2610.11328#bib.bib49)\)achieves an 85\.6% task success rate, compared with 73\.9% for Claude Opus 5\.5\([Anthropic, 2026a](https://arxiv.org/html/2610.11328#bib.bib59)\)and 23\.9% for the strongest open\-weight model, DeepSeek\-V4\.1\-Flash\([DeepSeek, 2026](https://arxiv.org/html/2610.11328#bib.bib54)\)\. Performance differences also depend on the environment, with a larger success\-rate gap between Astra and Opus 5\.5 indoors than outdoors\. In more than half of failed attempts, agents do not reach even the first required destination\. We further conduct ablation analysis and case studies to better understand the strengths and limitations of current agents\. Our task suite and evaluation framework will also be publicly released to support future research on agentic spatial intellgence in diverse real\-world environments\.

## 2Tasks Environment and Evaluation Framework

### 2\.1Environments

#### Game Details

Minecraft has been widely adopted as a research platform with its interactive 3D environments and consistent physical mechanics\([Guss et al\., 2019](https://arxiv.org/html/2610.11328#bib.bib5);[Fan et al\., 2022](https://arxiv.org/html/2610.11328#bib.bib6);[Guo et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib21);[Ju et al\., 2026a](https://arxiv.org/html/2610.11328#bib.bib44)\)\. Its open community ecosystem also allows us to integrate diverse, community\-built worlds into a unified testbed\. Diverse mods provide fine\-grained control over agent observation and interaction and enable automatic verification of task outcomes using in\-game agent positions\. Our experiments adopt[Minecraft Java Edition 1\.21\.11](https://www.minecraft.net/en-us/article/minecraft-java-edition-1-21-11)with[NeoForge 21\.11\.44](https://neoforged.net/news/21.11release/)\. We also install these mods:[*Xaero’s World Map*1\.44\.2](https://modrinth.com/mod/xaeros-world-map)for full\-screen map,[*Xaero’s Minimap*26\.4\.2](https://modrinth.com/mod/xaeros-minimap)for waypoint markers,[*BoccHUD*0\.4\.1](https://modrinth.com/mod/bocchud)with[*MaFgLib*0\.4](https://modrinth.com/mod/mafglib)for HUD, and[*LambDynamicLights*4\.9\.1](https://modrinth.com/mod/lambdynamiclights)for dynamic lighting\. By default, agent in each task begins at a specified position and orientation in first\-person view, with a nominal100∘100^\{\\circ\}field of view, frozen noon lighting, clear weather, Adventure mode, Peaceful difficulty, and only the*Xaero’s World Map*and*LambDynamicLights*are enabled\. Additional configurations and restrictions are detailed in Appendix[C\.1](https://arxiv.org/html/2610.11328#A3.SS1)\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/terrain-interactions-en.png)Figure 2:Representative terrain, spatial constraints, and interactions across the diverse collection of maps in MINE ODYSSEY\.
#### Maps

We source our maps from BuildTheEarth,111[https://buildtheearth\.net](https://buildtheearth.net/)a global community project dedicated to reconstructing real\-world locations in Minecraft\. We aim to evaluate agents in environments that reflect the diversity and spatial realism of real\-world places while preserving experimental control\. As shown in Figure[1](https://arxiv.org/html/2610.11328#S1.F1), we use 30 existing Minecraft builds modeled on locations across five continents, spanning 20 different countries and regions\. More detailed statistics and can be found in Appendix[B](https://arxiv.org/html/2610.11328#A2)\. The collection includes 20 outdoor and 10 indoor environments\. Outdoor maps cover residential and commercial districts, parks, hillsides, and waterfronts, varying in street connectivity, landmark density, and relief\. Indoor maps include palaces, religious buildings, hotels, stadiums, and passenger ships, with variation in room arrangement, corridor structure, entrances, and vertical circulation\. These environments present diverse terrain and spatial constraints, including uneven ground, water boundaries, narrow corridors, and multilevel spaces\. Navigating them can involve ascending stairs, climbing ladders, opening fence gates, or activating buttons to pass through doors, as illustrated in Figure[2](https://arxiv.org/html/2610.11328#S2.F2)\. Versioned world saves allow each task to begin from a consistent map state\.

#### Waypoints

We use waypoints to represent the destinations and intermediate stops that an agent may be asked to visit\. Each waypoint pairs a human\-readable place name with a three\-dimensional arrival position in a specific Minecraft world\. Our waypoints include landmarks, shops, public facilities, transport entrances, and indoor rooms, covering destinations with different functions and access conditions\. We construct each waypoint in three stages\. First, an annotator identifies a named place and its approximate real\-world location using geographic or venue\-specific references, including Google Maps, OpenStreetMap, and site maps\. Second, the annotator finds the corresponding place in the Minecraft reconstruction by matching the surrounding street or room layout and visible cues such as building facades, distinctive structures, and signs\. Third, the annotator enters the world, visits the matched place, and creates the waypoint at an observed, reachable standing position on the correct floor\. During task execution, the agent receives the waypoint name and can access its visual map marker via map, whereas the precise three\-dimensional arrival position remains evaluator\-only\. A waypoint marker will appears in the first\-person view when the agent is within 32 blocks of the waypoint, providing local visual guidance toward the destination\. The catalog contains 1,059 waypoints, of which 715 distinct identifiers occur in the task pool\.

#### Tasks

As shown in Figure[1](https://arxiv.org/html/2610.11328#S1.F1)\(b\), each task specifies a map, a starting waypoint and orientation, and an ordered sequence of destinations described in a natural\-language instruction\. The agent must identify and visit these destinations in order, choosing its own route using the permitted controls and ending at the final waypoint\. To construct tasks, we provide large language models with waypoint names, place types, and map descriptions to generate candidate task instructions and stop orders with a specific role and purpose\. We then manually select and refine these candidates based on three criteria:semantic coherence, so that the stops and their order fit the stated purpose;route plausibility, avoiding redundant visits, unnecessary backtracking, and implausibly short segments; anddiversity, covering varied roles, scenarios, and navigation requirements\. To ensure task diversity, we vary starting points, distances between stops, elevation changes, and interactions such as opening doors, pressing buttons, or climbing ladders, while limiting repeated ordered waypoint pairs within each map\. yx To guarantee task feasibility, we first use Baritone \(a Minecraft pathfinding and navigation mod\) for a preliminary reachability check, then manually traverse the task routes in the game to verify the reachability\. The resulting collection contains 180 tasks with 2–12 stops each\.

### 2\.2Verifier\.

To complete a task, the agent must reach the specified destinations in order\. An independent evaluator is used to verify arrivals by comparing the player’s position with the annotated waypoint coordinates\. It samples the client player position at 1 Hz throughout execution, including during model inference\. The agent is considered to have reached the next required waypointgig\_\{i\}when its sampled positionp∈ℝ3p\\in\\mathbb\{R\}^\{3\}satisfies

∥p−gi∥2<3\.5and\|py−giy\|≤1\.5\\lVert p\-g\_\{i\}\\rVert\_\{2\}<3\.5\\quad\\text\{and\}\\quad\|p^\{y\}\-g\_\{i\}^\{y\}\|\\leq 1\.5All distances are measured in Minecraft blocks\. The vertical constraint reduces ambiguity between nearby floors\. Only the next required waypoint is eligible for credit; out\-of\-order visits do not advance task progress\. An episode can terminate early with failure under three conditions: \(1\)*Death*: the player’s health reaches zero after taking damage, for example from a fall; \(2\)*Leaving the game*: the player exits or disconnects from the game world; \(3\)*Out of bounds*: the player falls beyond the map boundary and cannot return to continue the task\. On maps with a configured height boundary, this is detected when the player remains at or below that height for 15 consecutive seconds\. After visiting all required destinations, the agent must submit an acceptedclaim\_doneto complete the task successfully\.

### 2\.3Agent Framework

Figure[3](https://arxiv.org/html/2610.11328#S2.F3)shows how the agent interacts with the Minecraft client through the action interface and receives feedback from the independent verifier\.

#### Action Space

To enable the agent to control a player and interact with the Minecraft client, we develop a programmable control interface that combines API\-based player control with keyboard and mouse input\. Specifically, we implement*AgentBridge*, a client\-side mod that uses native Minecraft APIs to support player actions such as movement, camera rotation, and item interaction\. A command\-line client,*mcapi*, sends commands to AgentBridge and returns execution results to the agent\. To complement these actions with GUI interactions, we provide*xdo*, a command\-line wrapper aroundxdotoolthat enables the agent to operate the client’s graphical interface through keyboard and mouse\. The agent uses these tools through Bash, either executing individual operations or composing multiple operations within a single Bash tool call with shell commands or Python scripts\. For example, the agent may runmcapi look \-\-yaw \-90 && mcapi press MOVE\_FORWARD 2\.0to turn the camera and then move forward for two seconds\. Scripts can include calculations, loops, and conditional logic, allowing the agent to execute complex action sequences\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/agent_loop_overview.png)Figure 3:Agent interaction and independent verification in MINE ODYSSEY\.
#### Observation Space

The agent receives screenshots of the Minecraft client as, showing either the player’s first\-person view or an open interface, such as the map or inventory\. The in\-game map provides a top\-down view of the explored environment and supports panning, zooming or searching to inspect different areas\. In addition to screenshots, the agent receives textual feedback from command execution, including returned data, execution status, and error messages\.

#### Agent Loop

Given these observation and control interfaces, the agent interacts with the environment through an observation–action loop\. The game simulation runs asynchronously with this loop: the environment evolves continuously without waiting for the agent to finish reasoning or issue its next action\. At each decision step, the agent selects one of four action types based on the latest observations, the task instruction, and its interaction history: \(1\)*exec*: execute a Bash command, interrupting any command still running\. It can run game\-control tools such asmcapiandxdo, execute Python scripts, or perform computations and file operations\. Invalid Bash commands return textual errors, whilemcapi statequeries return textual results; both are returned immediately without waiting for the observation interval\. \(2\)*skip*: wait for further observations while allowing the current command to continue\. \(3\)*stop\_execute*: terminate the current command and immediately return a screenshot\. \(4\)*claim\_done*: indicate that the agent believes it has visited all required waypoints\.

Since the environment evolves independently of the agent’s decision loop, conditions may change during command execution, especially when a command takes a long time to complete\. The agent may therefore need to observe these changes and intervene before the command finishes\. So we provide an optionalobserve\_after\_secparameter forexecandskip, which specifies the requested delay in seconds before the next observation\. It allows the agent to assess intermediate progress and decide whether to let the current command continue, stop it, or replace it\. For example, the agent may issueexec\(content="mcapi press MOVE\_FORWARD 5\.0", observe\_after\_sec=2\.0\), requesting five seconds of forward movement and another observation after two seconds\. If the path remains clear, it may selectskipto wait for another observation while the movement continues\. If an obstacle appears, it may selectstop\_executeto terminate the command or issue a newexecaction to interrupt it and adjust its movement\.

#### Preventing Hacking and Sandbox

To preserve the integrity of the evaluation, we restrict access to information and game features that could allow the agent to bypass the intended task requirements\. Specifically, we add safeguards to AgentBridge to prevent coordinate disclosure through waypoint editors, settings pages, and chat sharing\. The visual world map remains available, while waypoint teleportation and Baritone\-based automatic navigation are disabled during evaluation\.

In addition, to prevent direct access to hidden task data through Bash, we execute agent\-issued commands in a Bubblewrap sandbox within a containerized evaluation environment managed by Podman or Docker\. The sandbox uses Linux namespaces and restricted filesystem mounts to expose the required tools and a writable agent workspace while excluding world saves, waypoint files, project files, and evaluator records\. A separate network namespace disables external network access, while a Unix\-domain socket relay permits access to approved AgentBridge APIs\. Such sandbox isolation supports parallel evaluation\. Each task runs in its own sandbox with a separate copy, client directory, and agent workspace, preventing interference from others\. To improve evaluation efficiency, we also support GPU\-accelerated rendering within the containers to offload graphics processing from CPU\.

## 3Experiments

### 3\.1Setup

#### Models and configuration\.

We evaluate leading proprietary models, including GPT\-6 Astra\([OpenAI, 2026b](https://arxiv.org/html/2610.11328#bib.bib49)\), GPT\-5\.6 Sol\([OpenAI, 2026a](https://arxiv.org/html/2610.11328#bib.bib50)\), Claude Opus 5\([Anthropic, 2026b](https://arxiv.org/html/2610.11328#bib.bib51)\), Claude Opus 5\.5\([Anthropic, 2026a](https://arxiv.org/html/2610.11328#bib.bib59)\), Gemini 3\.8 Flash\([Google DeepMind, 2026](https://arxiv.org/html/2610.11328#bib.bib52)\)and Grok 4\.6\([SpaceXAI, 2026](https://arxiv.org/html/2610.11328#bib.bib53)\), alongside open\-weight models DeepSeek\-V4\.1\-Flash\([DeepSeek, 2026](https://arxiv.org/html/2610.11328#bib.bib54)\)and GLM\-5\.3\-Flash\([Z\.ai, 2026](https://arxiv.org/html/2610.11328#bib.bib55)\)\. All models use native tool calls through the same action interface\. Each task is limited to 500 agent steps and six hours of wall\-clock time, with at most threeclaim\_donecalls\. To manage context growth, we automatically trigger summarization when any of the following three conditions is met: the current context window reaches 100 assistant turns, the latest model call reports more than 200K total tokens, or the active context has more than 100 images\. The agent is prompted to summarize its goals, progress, relevant environment state, failed attempts, and next steps\. The summary replaces the previous interaction history, while the system instructions and original task are retained\. The agent then resumes the same episode from the current game state using the summary and newly received observations\.

#### Tasks and metrics\.

We evaluate each model on 180 tasks and reportSuccess Rate\(SR\), the fraction of tasks where the agent visits all required stops in order and submits an acceptedclaim\_done\. We also reportCheckpoint Coverage\(CC\), the average fraction of ordered intermediate stops to measure partial progress\. To measure movement and route efficiency, we reportPath Length\(PL\) andSuccess weighted by Path Length\(SPL\)\. PL averages the full\-episode 3D distance, computed from consecutive position samples at approximately 1 Hz, over all 180 tasks including failures\. SPL measures efficiency by rewarding successful completion along shorter routes\. It averagesSi​Li/max⁡\(Li,Pi\)S\_\{i\}L\_\{i\}/\\max\(L\_\{i\},P\_\{i\}\)over the same tasks, whereSiS\_\{i\}indicates success,PiP\_\{i\}is the recorded path length, andLiL\_\{i\}is the reference path length through all required stops in order\. Details of reference path computation are provided in Appendix[C\.3](https://arxiv.org/html/2610.11328#A3.SS3)\. Average turns is the average number of agent turns following the agent loop in Section[2\.3](https://arxiv.org/html/2610.11328#S2.SS3)and exclude automatic history\-summary responses\.

### 3\.2Main results

Table[1](https://arxiv.org/html/2610.11328#S3.T1)shows that GPT\-6 Astra leads at 85\.6% SR, followed by Claude Opus 5\.5 at 73\.9%\. Opus 5\.5 exceeds Opus 5 max by 23\.3 percentage points, placing it substantially closer to Astra than the remaining models\. Five of the eight models complete fewer than half of the tasks; the strongest evaluated open\-weight model, DeepSeek\-V4\.1\-Flash, reaches 23\.9%\. All configurations have lower SR indoors\. Astra’s lead over Opus 5\.5 widens from 8\.8 percentage points outdoors to 18\.2 points indoors\. Opus 5\.5 nevertheless substantially improves on Opus 5 max in both environments, especially indoors, where SR rises from 32\.7% to 63\.6%\. These are comparisons between task subsets with different layouts and route composition; they do not isolate the effect of being indoors\. Checkpoint coverage and SPL reveal smaller gaps between the two leading models\. Opus 5\.5 reaches 86\.1% CC and 50\.35% SPL, only 2\.9 and 3\.35 percentage points below Astra, respectively, despite an 11\.7\-point SR gap\. High intermediate\-stop coverage therefore need not translate into full task completion\. Their mean travel distances are also similar, but Opus 5\.5 uses about2\.4×2\.4\\timesas many decisions\. These full\-cohort averages include failures and do not compare routes on a shared set of successful tasks\. Gemini provides a further contrast: it has the shortest mean travel distance but the most decisions and only 26\.7% SR\. Short travel alone therefore does not imply efficient completion\. Within Opus 5, increasing effort from low to max raises SR from 35\.0% to 50\.6% and SPL from 7\.77% to 24\.29%, while mean travel falls from 4454\.2 to 2020\.2 blocks and decisions rise from 218\.8 to 242\.8\. Higher effort thus accompanies more completed tasks and less travel, with a modest increase in decisions\.

Table 1:Main results on 180 tasks, with success rates broken down by 125 outdoor and 55 indoor tasks\. SR, CC, and SPL are percentages; PL is the mean sampled 3D distance over all tasks\.

## 4Analysis

### 4\.1Ablations

We evaluate how spatial information and scene appearance affect DeepSeek\-V4\.1\-Flash and GLM\-5\.3\-Flash under six conditions\. As illustrated in Figure[4](https://arxiv.org/html/2610.11328#S4.F4), HUD enable BoccHUD \(player position and orientation in the upper right\) and Xaero’s Minimap \(local map in the upper left\) and remove the default 32\-block display\-distance limit on first\-person waypoint markers\. Multi\-view observations combine six camera directions into one image, while third\-person view fixes the camera behind the player\. Route guidance display a Baritone\-planned path as a green trail with yellow directional arrows on the ground, as shown in Figure[4](https://arxiv.org/html/2610.11328#S4.F4)\(d\), while leaving movement under agent control\. Midnight replaces the default clear\-noon lighting and supplies a torch that illuminates the surroundings when held\. Rain changes the weather only for outdoor tasks; indoor outcomes remain unchanged\.

Table 2:Ablation across six conditions: SR, CC, SPL\.All conditions preserve the required destinations\. From Table[2](https://arxiv.org/html/2610.11328#S4.T2), we can find that visual aids improve SR and CC for both models, with HUD raising SR by 6\.1 points for DeepSeek and 4\.0 for GLM\. DeepSeek’s multi\-view SR gain of 2\.2 points accompanies a 0\.3 point SPL decrease\. Route guidance slightly raises CC but lowers DeepSeek’s SR by 2\.8 points and leaves GLM’s unchanged, indicating a gap between recognizing guidance and effective action\. Finally, midnight reduces SR by 1\.7 points for both models, with torches available to provide local light\. Rain lowers SR by 3\.9 points for DeepSeek and 2\.2 for GLM\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/ablation.png)Figure 4:The default view and six ablation conditions are shown outside Hofburg Palace\.
### 4\.2Prolonged local stays\.

Figure 5:Local stays over at least ten turns within one block\. Blue: episodes per task \(left axis\)\. Orange: task\-mean share of aligned turns \(right axis\)\.To examine agents’ ability to sustain progress, we count local stays of at least ten consecutive turns within one block of a fixed 3D anchor\. From Figure[5](https://arxiv.org/html/2610.11328#S4.F5), we find that Astra rarely remains in the same local area for many turns with only 0\.17 episodes and 2\.7% of turn, whereas such stays are much more frequent among all other models\. Notably, Gemini spends over half of its turns in local stays at 51\.7%, more than twice the share of any other model, and averages 7\.09 episodes per task\. During these stays, Gemini often engages in repeated map inspection, menu interactions, and screenshot processing without resuming movement\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/recovery-route-cases-combined.png)Figure 6:\(a\) White House: Astra takes a detour; DeepSeek revisits rooms\. \(b\) Plaza Hotel: Astra returns downstairs; Grok remains upstairs\. S/E: start/end; M: shared upper landing; stars: targets\. Times cover the S–E segments\.
### 4\.3Case Studies

#### Case Study 1

Figure[6](https://arxiv.org/html/2610.11328#S4.F6)shows two pairs of agents that pass through a shared location but differ in their subsequent progress\. As shown in Figure[6](https://arxiv.org/html/2610.11328#S4.F6)\(a\), both DeepSeek and Astra enter through the same western doorway\. DeepSeek recognizes the blocked passage but spent 142\.7 minutes repeatedly revisiting the nearby rooms until the step limit\. Astra instead inspects its surroundings, returns toward a southern doorway, and follows the hallway around the wall to Queen’s Bedroom, completing the displayed detour in 2\.4 minutes\. In Figure[6](https://arxiv.org/html/2610.11328#S4.F6)\(b\), Astra and Grok both leave the restaurant floor at S and explore the upper patio at M\. Grok remains upstairs for 135\.7 minutes before reaching the step limit\. Astra instead retraces its route to the stairs, descends to the restaurant floor, and reaches the target in 12\.6 minutes\. In both cases, Astra redirects its search and follows an alternative route to the target, while the unsuccessful models continues exploring within the same local region\.

#### Case Study 2

![Refer to caption](https://arxiv.org/html/2610.11328v1/spatial-reasoning.png)Figure 7:Discovering an upstairs route in Hagia Sophia 003\. Astra explores the side branch and ascends, while Opus 5 approaches the end door and leaves without exploring the branch\.As illustrated in Figure[7](https://arxiv.org/html/2610.11328#S4.F7), Astra and Opus 5 receive nearly matching views of the same passage entrance\. After entering the passage, they see the same door ahead \(outlined in blue\)\. Astra turns right in place, revealing the stairs and identifying an upward route toward the target\. While Opus 5 instead approaches the door and returns outside without exploring the ascending branch, even though the stair opening is visible on its right \(outlined in gold\)\. This contrast illustrates how incomplete local exploration can leave an available connection undiscovered, even when its entrance is visible\.

## 5Conclusion

We introduceMine Odyssey, a benchmark for evaluating agentic spatial intelligence on 180 spatial tasks spanning 30 Minecraft reconstructions of diverse real\-world locations\. Our evaluation reveals substantial performance gaps: GPT\-6 Astra achieves 85\.6% success, while the second\-best model, Claude Opus 5\.5, reaches 73\.9% and the strongest evaluated open\-weight model, DeepSeek\-V4\.1\-Flash, reaches 23\.9%, leaving considerable room for improvement\. Through detailed analyses, we identify key challenges in grounding spatial information, discovering traversable connections, and recovering from unsuccessful routes\. We believeMine Odysseywill support the development of more capable and reliable agents for sustained interaction in complex spatial environments\.

## References

- Andon Labs \(2026\)Andon LabsBlueprint\-Bench 2\.Note:[https://andonlabs\.com/evals/blueprint\-bench\-2](https://andonlabs.com/evals/blueprint-bench-2)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Anthropic \(2026a\)AnthropicIntroducing Claude Opus 5\.5\.External Links:[Link](https://www.anthropic.com/claude-opus-5-5)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p6.1),[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Anthropic \(2026b\)AnthropicIntroducing Claude Opus 5\.External Links:[Link](https://www.anthropic.com/news/claude-opus-5)Cited by:[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Batraet al\.\(2020\)D\. Batra, A\. Gokaslan, A\. Kembhavi, O\. Maksymets, R\. Mottaghi, M\. Savva, A\. Toshev, and E\. WijmansObjectNav revisited: on evaluation of embodied agents navigating to objects\.arXiv preprint arXiv:2006\.13171\.External Links:[Link](https://arxiv.org/abs/2006.13171)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Bhatet al\.\(2026\)S\. Bhat, H\. Khandelwal, T\. Kataria, and V\. GuptaMapVerse: a benchmark for geospatial question answering on diverse real\-world maps\.In2026 IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\),pp\. 8168–8178\.External Links:[Link](https://openaccess.thecvf.com/content/WACV2026/papers/Bhat_MapVerse_A_Benchmark_for_Geospatial_Question_Answering_on_Diverse_Real-World_WACV_2026_paper.pdf)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Chenet al\.\(2024\)B\. Chen, Z\. Xu, S\. Kirmani, B\. Ichter, D\. Sadigh, L\. Guibas, and F\. XiaSpatialvlm: endowing vision\-language models with spatial reasoning capabilities\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 14455–14465\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2024/html/Chen_SpatialVLM_Endowing_Vision-Language_Models_with_Spatial_Reasoning_Capabilities_CVPR_2024_paper.html)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Chenet al\.\(2019\)H\. Chen, A\. Suhr, D\. Misra, N\. Snavely, and Y\. ArtziTouchdown: natural language navigation and spatial reasoning in visual street environments\.In2019 IEEE/CVF conference on computer vision and pattern recognition \(CVPR\),pp\. 12538–12547\.External Links:[Link](https://openaccess.thecvf.com/content_CVPR_2019/html/Chen_TOUCHDOWN_Natural_Language_Navigation_and_Spatial_Reasoning_in_Visual_Street_CVPR_2019_paper.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Dalalet al\.\(2026\)D\. Dalal, U\. Mishra, N\. Ahuja, and N\. JojicCan mllms find their way in a city? exploring emergent navigation from web\-scale knowledge\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8279–8303\.External Links:[Link](https://aclanthology.org/2026.eacl-long.387/)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p2.1)\.
- DeepSeek \(2026\)DeepSeekIntroducing DeepSeek\-V4\.1\-Flash: smarter, faster, more efficient\.External Links:[Link](https://www.deepseek.com/en/news/deepseek-v4-1-flash/)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p6.1),[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Denget al\.\(2026\)Y\. Deng, P\. Lai, X\. Li, C\. Bai, X\. Deng, C\. Sun, X\. Li, and H\. YangSpaceVLN: a zero\-shot vision\-and\-language navigation agent with online spatial cognitive memory and reasoning\.arXiv preprint arXiv:2606\.08992\.External Links:[Link](https://arxiv.org/abs/2606.08992)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1)\.
- Duet al\.\(2024\)M\. Du, B\. Wu, Z\. Li, X\. Huang, and Z\. WeiEmbspatial\-bench: benchmarking spatial understanding for embodied tasks with large vision\-language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),pp\. 346–355\.External Links:[Link](https://aclanthology.org/2024.acl-short.33/)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Fanet al\.\(2022\)L\. Fan, G\. Wang, Y\. Jiang, A\. Mandlekar, Y\. Yang, H\. Zhu, A\. Tang, D\. Huang, Y\. Zhu, and A\. AnandkumarMinedojo: building open\-ended embodied agents with internet\-scale knowledge\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 18343–18362\.External Links:[Link](https://papers.nips.cc/paper_files/paper/2022/hash/74a67268c5cc5910f64938cac4526a90-Abstract-Datasets_and_Benchmarks.html)Cited by:[§2\.1](https://arxiv.org/html/2610.11328#S2.SS1.SSS0.Px1.p1.1)\.
- Gaoet al\.\(2026\)H\. Gao, H\. Qu, J\. Tang, J\. Wang, Z\. Huang, H\. Qiao, S\. Huang, J\. Yang, Y\. Li, H\. Yuan,et al\.SpatialWorld: benchmarking interactive spatial reasoning of multimodal agents in real\-world tasks\.arXiv preprint arXiv:2606\.09669\.External Links:[Link](https://arxiv.org/abs/2606.09669)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Gemini Robotics Teamet al\.\(2025\)Gemini Robotics Team, S\. Abeyruwan, J\. Ainslie, J\. Alayrac, M\. G\. Arenas, T\. Armstrong, A\. Balakrishna, R\. Baruch, M\. Bauza, M\. Blokzijl,et al\.Gemini Robotics: bringing AI into the physical world\.arXiv preprint arXiv:2503\.20020\.External Links:[Link](https://arxiv.org/abs/2503.20020)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1)\.
- Google DeepMind \(2026\)Google DeepMindGemini 3\.8 Flash: model card\.External Links:[Link](https://deepmind.google/models/model-cards/gemini-3-8-flash/)Cited by:[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Guoet al\.\(2025\)J\. Guo, Y\. Ye, T\. He, H\. Wu, Y\. Jiang, T\. Pearce, and J\. BianMineWorld: a real\-time and open\-source interactive world model on minecraft\.arXiv preprint arXiv:2504\.08388\.External Links:[Link](https://arxiv.org/abs/2504.08388)Cited by:[§2\.1](https://arxiv.org/html/2610.11328#S2.SS1.SSS0.Px1.p1.1)\.
- Gusset al\.\(2019\)W\. H\. Guss, B\. Houghton, N\. Topin, P\. Wang, C\. Codel, M\. Veloso, and R\. SalakhutdinovMineRL: a large\-scale dataset of minecraft demonstrations\.InProceedings of the 28th International Joint Conference on Artificial Intelligence,pp\. 2442–2448\.External Links:[Link](https://dl.acm.org/doi/abs/10.5555/3367243.3367379)Cited by:[§2\.1](https://arxiv.org/html/2610.11328#S2.SS1.SSS0.Px1.p1.1)\.
- Hartet al\.\(1968\)P\. E\. Hart, N\. J\. Nilsson, and B\. RaphaelA formal basis for the heuristic determination of minimum cost paths\.IEEE transactions on Systems Science and Cybernetics4,pp\. 100–107\.External Links:[Link](https://ieeexplore.ieee.org/abstract/document/4082128)Cited by:[§C\.3](https://arxiv.org/html/2610.11328#A3.SS3.p2.1)\.
- Hayamizuet al\.\(2026\)Y\. Hayamizu, D\. DeFazio, H\. Mehta, Z\. Altaweel, J\. Choe, C\. Lin, J\. Juettner, F\. Xiao, J\. Blackburn, and S\. ZhangFrom woofs to words: towards intelligent robotic guide dogs with verbal communication\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 38560–38569\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/41198)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1)\.
- Honget al\.\(2026\)Y\. Hong, J\. Liu, H\. Yin, M\. Li, L\. Guibas, L\. Fei\-Fei, J\. Wu, and Y\. ChoiESI\-Bench: towards embodied spatial intelligence that closes the perception\-action loop\.arXiv preprint arXiv:2605\.18746\.External Links:[Link](https://arxiv.org/abs/2605.18746)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Huanget al\.\(2026\)X\. Huang, Q\. He, Z\. Huang, B\. Wang, Z\. Li, G\. Cheng, Y\. Dong, and X\. HuangSpatial\-dise: a unified benchmark for evaluating spatial reasoning in vision\-language models\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 135833–135865\.External Links:[Link](https://iclr.cc/virtual/2026/poster/10008617)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Juet al\.\(2026a\)T\. Ju, Y\. Sun, Z\. Wu, W\. Zhang, Y\. Huo, X\. Su, Q\. Gu, X\. Cai, G\. Liu, and Z\. ZhangMineExplorer: evaluating open\-world exploration of mllm agents in minecraft\.arXiv preprint arXiv:2605\.30931\.External Links:[Link](https://arxiv.org/abs/2605.30931)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p2.1),[§2\.1](https://arxiv.org/html/2610.11328#S2.SS1.SSS0.Px1.p1.1)\.
- Juet al\.\(2026b\)T\. Ju, Z\. Wu, Y\. Sun, Y\. Cui, B\. Li, S\. Wu, P\. Cheng, H\. Zhao, Z\. Wu, X\. Ma, D\. Zhang, K\. Li, M\. Lee, W\. Hsu, H\. Fei, Q\. Gu, G\. Liu, and Z\. ZhangUrbanGround: from local perception to spatial agency in a real\-scale city\.arXiv preprint arXiv:2608\.27456\.External Links:[Link](https://arxiv.org/abs/2608.27456)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1),[§1](https://arxiv.org/html/2610.11328#S1.p2.1)\.
- Khannaet al\.\(2024\)M\. Khanna, R\. Ramrakhya, G\. Chhablani, S\. Yenamandra, T\. Gervet, M\. Chang, Z\. Kira, D\. S\. Chaplot, D\. Batra, and R\. MottaghiGoat\-bench: a benchmark for multi\-modal lifelong navigation\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 16373–16383\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2024/html/Khanna_GOAT-Bench_A_Benchmark_for_Multi-Modal_Lifelong_Navigation_CVPR_2024_paper.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Krantzet al\.\(2020\)J\. Krantz, E\. Wijmans, A\. Majumdar, D\. Batra, and S\. LeeBeyond the nav\-graph: vision\-and\-language navigation in continuous environments\.InEuropean Conference on Computer Vision,pp\. 104–120\.External Links:[Link](https://link.springer.com/chapter/10.1007/978-3-030-58604-1_7)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Liet al\.\(2026\)J\. Li, W\. Zhao, J\. Zhao, W\. Zeng, H\. Wu, X\. Wang, R\. Ge, Y\. Cao, Y\. Huang, W\. Liu, J\. Liu, Z\. Su, Y\. Guo, F\. Zhou, L\. Zhang, J\. Michelini, X\. Wang, X\. Yue, S\. Zhou, G\. Neubig, and J\. HeThe tool decathlon: benchmarking language agents for diverse, realistic, and long\-horizon task execution\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=z53s5p0qhf)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Liuet al\.\(2025\)S\. Liu, Y\. Li, K\. Zhang, Z\. Cui, W\. Fang, Y\. Zheng, T\. Zheng, and M\. SongOdyssey: empowering minecraft agents with open\-world skills\.InProceedings of the Thirty\-Fourth International Joint Conference on Artificial Intelligence,pp\. 187–195\.External Links:[Link](https://doi.org/10.24963/ijcai.2025/22)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p2.1)\.
- Liuet al\.\(2026\)T\. Liu, J\. Feng, Z\. Zheng, S\. Wang, Y\. Guo, Y\. Xi, H\. Fan, Y\. Li, and P\. HuiSpatialAct: probing spatial reasoning\-to\-action capabilities of vlm agents in 3d scenes\.arXiv preprint arXiv:2605\.31148\.External Links:[Link](https://arxiv.org/abs/2605.31148)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Liuet al\.\(2024\)X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.Agentbench: evaluating llms as agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 52989–53046\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/e9df36b21ff4ee211a8b71ee8b7e9f57-Abstract-Conference.html)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Mirowskiet al\.\(2019\)P\. Mirowski, A\. Banki\-Horvath, K\. Anderson, D\. Teplyashin, K\. M\. Hermann, M\. Malinowski, M\. K\. Grimes, K\. Simonyan, K\. Kavukcuoglu, A\. Zisserman,et al\.The streetlearn environment and dataset\.arXiv preprint arXiv:1903\.01292\.External Links:[Link](https://arxiv.org/abs/1903.01292)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- OpenAI \(2026a\)OpenAIGPT\-5\.6 Sol model\.External Links:[Link](https://openai.com/index/gpt-5-6/)Cited by:[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- OpenAI \(2026b\)OpenAIGPT\-6 Astra model\.External Links:[Link](https://openai.com/index/gpt-6-astra/)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1),[§1](https://arxiv.org/html/2610.11328#S1.p6.1),[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Ouyanget al\.\(2026\)M\. Ouyang, S\. Hu, K\. Q\. Lin, H\. T\. Ng, and M\. Z\. ShouGameworld: towards standardized and verifiable evaluation of multimodal game agents\.InEuropean Conference on Computer Vision,pp\. 41–59\.External Links:[Link](https://arxiv.org/abs/2604.07429)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Paglieriet al\.\(2025\)D\. Paglieri, B\. Cupiał, S\. Coward, U\. Piterbarg, M\. Wołczyk, A\. Khan, E\. Pignatelli, Ł\. Kuciński, L\. Pinto, R\. Fergus,et al\.Balrog: benchmarking agentic llm and vlm reasoning on games\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 96666–96702\.External Links:[Link](https://openreview.net/forum?id=fp6t3F669F)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Peiet al\.\(2026\)X\. Pei, J\. Liu, V\. Munasinghe, B\. Miao, U\. Tan, W\. Ding, and N\. ZhaoSAP\-Nav: spatial semantic representation meets active perception for hierarchical open\-vocabulary object navigation\.arXiv preprint arXiv:2608\.12707\.External Links:[Link](https://arxiv.org/abs/2608.12707)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1)\.
- Peterssonet al\.\(2025\)L\. Petersson, A\. Backlund, A\. Wennstöm, H\. Petersson, C\. Sharrock, and A\. DabiriBlueprint\-bench: comparing spatial intelligence of llms, agents and image models\.arXiv preprint arXiv:2509\.25229\.External Links:[Link](https://arxiv.org/abs/2509.25229)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Qiaoet al\.\(2026\)Y\. Qiao, H\. Hong, W\. Lyu, D\. An, S\. Zhang, Y\. Xie, X\. Wang, and Q\. WuNavbench: probing multimodal large language models for embodied navigation\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 95365–95385\.External Links:[Link](https://papers.nips.cc/paper_files/paper/2025/hash/8932bbe603280f51d425e2781cd6ea6e-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p2.1)\.
- RoboDojo Team \(2026\)RoboDojo TeamAn unexpected robot policy: early evaluations of GPT\-6 Astra on RoboDojo and beyond\.Note:Online technical reportExternal Links:[Link](https://robodojo-benchmark.com/report/gpt-6-astra-eval)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1)\.
- Savvaet al\.\(2019\)M\. Savva, A\. Kadian, O\. Maksymets, Y\. Zhao, E\. Wijmans, B\. Jain, J\. Straub, J\. Liu, V\. Koltun, J\. Malik,et al\.Habitat: a platform for embodied AI research\.In2019 IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 9338–9346\.External Links:[Link](https://openaccess.thecvf.com/content_ICCV_2019/html/Savva_Habitat_A_Platform_for_Embodied_AI_Research_ICCV_2019_paper.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Sikarwaret al\.\(2026\)A\. Sikarwar, D\. Mishra, S\. Nikhil, P\. Kumaraguru, and A\. AgrawalCommunicating about space: language\-mediated spatial integration across partial views\.arXiv preprint arXiv:2603\.27183\.External Links:[Link](https://arxiv.org/abs/2603.27183)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Songet al\.\(2025\)X\. Song, W\. Chen, Y\. Liu, W\. Chen, G\. Li, and L\. LinTowards long\-horizon vision\-language navigation: platform, benchmark and method\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 12078–12088\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2025/html/Song_Towards_Long-Horizon_Vision-Language_Navigation_Platform_Benchmark_and_Method_CVPR_2025_paper.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- SpaceXAI \(2026\)SpaceXAIIntroducing Grok 4\.6\.External Links:[Link](https://x.ai/news/grok-4-6)Cited by:[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2024a\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=ehfRiF0R3a)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p2.1)\.
- Wanget al\.\(2025\)J\. Wang, D\. Huo, Z\. Xu, Y\. Shi, Y\. Yan, Y\. Wang, C\. Gao, Y\. Qiao, and G\. ZhouOpenBench: a new benchmark and baseline for semantic navigation in smart logistics\.In2025 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 16202–16208\.External Links:[Link](https://ieeexplore.ieee.org/abstract/document/11127476/?casa_token=LtRoOvc2_ZYAAAAA:8kdO9m__dDRTjs8AscJP9f75SYsOMxa12ig7ep_efmLpy7s-uRhyYBxBvBMEYYtEvc-bz5cU)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p1.1),[§1](https://arxiv.org/html/2610.11328#S1.p2.1)\.
- Wanget al\.\(2026a\)S\. Wang, B\. Liu, Z\. Gao, L\. Ma, X\. Wang, Y\. Xie, and X\. TanExplore with long\-term memory: a benchmark and multimodal llm\-based reinforcement learning framework for embodied exploration\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 37098–37108\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2026/html/Wang_Explore_with_Long-term_Memory_A_Benchmark_and_Multimodal_LLM-based_Reinforcement_CVPR_2026_paper.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Wanget al\.\(2026b\)S\. Wang, C\. Liang, Y\. Gao, E\. Yu, S\. Li, J\. Li, and H\. WangCitySeeker: how do vlms explore embodied urban navigation with implicit human needs?\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 32729–32766\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/3767842e2ebee900236f534855aa2c36-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p2.1)\.
- Wanget al\.\(2024b\)Z\. Wang, S\. Cai, A\. Liu, Y\. Jin, J\. Hou, B\. Zhang, H\. Lin, Z\. He, Z\. Zheng, Y\. Yang,et al\.Jarvis\-1: open\-world multi\-task agents with memory\-augmented multimodal language models\.IEEE Transactions on Pattern Analysis and Machine Intelligence47\(3\),pp\. 1894–1907\.External Links:[Link](https://ieeexplore.ieee.org/document/10778628/)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p2.1)\.
- Waniet al\.\(2020\)S\. Wani, S\. Patel, U\. Jain, A\. Chang, and M\. SavvaMultion: benchmarking semantic map memory using multi\-object navigation\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9700–9712\.External Links:[Link](https://papers.nips.cc/paper/2020/hash/6e01383fd96a17ae51cc3e15447e7533-Abstract.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei,et al\.Osworld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 52040–52094\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/5d413e48f84dc61244b6be550f1cd8f5-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Yanget al\.\(2024\)J\. Yang, R\. Ding, E\. Brown, X\. Qi, and S\. XieV\-IRL: grounding virtual intelligence in real life\.InEuropean Conference on Computer Vision,pp\. 36–55\.External Links:[Link](https://www.ecva.net/papers/eccv_2024/papers_ECCV/html/6117_ECCV_2024_paper.php)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.
- Yanget al\.\(2025a\)J\. Yang, S\. Yang, A\. W\. Gupta, R\. Han, L\. Fei\-Fei, and S\. XieThinking in space: how multimodal large language models see, remember, and recall spaces\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 10632–10643\.External Links:[Link](https://openaccess.thecvf.com/content/CVPR2025/html/Yang_Thinking_in_Space_How_Multimodal_Large_Language_Models_See_Remember_CVPR_2025_paper.html)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Yanget al\.\(2025b\)R\. Yang, H\. Chen, J\. Zhang, M\. Zhao, C\. Qian, K\. Wang, Q\. Wang, T\. V\. Koripella, M\. Movahedi, M\. Li, H\. Ji, H\. Zhang, and T\. ZhangEmbodiedBench: comprehensive benchmarking multi\-modal large language models for vision\-driven embodied agents\.InProceedings of the 42nd International Conference on Machine Learning,Vol\.267,pp\. 70576–70631\.External Links:[Link](https://proceedings.mlr.press/v267/yang25f.html)Cited by:[§1](https://arxiv.org/html/2610.11328#S1.p2.1)\.
- Yanget al\.\(2026\)S\. Yang, R\. Xu, Y\. Xie, S\. Yang, M\. Li, J\. Lin, C\. Zhu, X\. Chen, H\. Duan, X\. Yue,et al\.Mmsi\-bench: a benchmark for multi\-image spatial intelligence\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 157051–157088\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/fe5d4bd3e2af823701b5d8f70a3c7602-Abstract-Conference.html)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Yuanet al\.\(2026\)M\. Yuan, Z\. Zhou, X\. Xiong, W\. Wu, J\. Sun, J\. Song, K\. Cui, B\. Wang, H\. Wu, Y\. Li, D\. Lu, H\. Lu, Q\. Zhen, X\. Wang, J\. Deng, Y\. Yang, C\. Chen, B\. Zheng, A\. Su, X\. Yu, H\. Zou, S\. Agashe, X\. H\. Lu, M\. Kaur, Z\. Qi, V\. S\. Chen, F\. Sala, D\. Liu, J\. Lin, Z\. Yu, Y\. Su, S\. Reddy, X\. E\. Wang, P\. Qi, T\. Xie, and T\. YuOSWorld 2\.0: benchmarking computer use agents on long\-horizon real\-world tasks\.arXiv preprint arXiv:2606\.29537\.External Links:[Link](https://arxiv.org/abs/2606.29537)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Z\.ai \(2026\)Z\.aiGLM\-5\.3\-Flash\.External Links:[Link](https://z.ai/blog/glm-5.3-flash)Cited by:[§3\.1](https://arxiv.org/html/2610.11328#S3.SS1.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025\)A\. L\. Zhang, T\. L\. Griffiths, K\. R\. Narasimhan, and O\. PressVideoGameBench: can vision\-language models complete popular video games?\.arXiv preprint arXiv:2505\.18134\.External Links:[Link](https://arxiv.org/abs/2505.18134)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Zhanget al\.\(2026\)Y\. Zhang, R\. Corcodel, C\. Hori, A\. Cherian, and D\. ZhaoSpinbench: perspective and rotation as a lens on spatial reasoning in vlms\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 70072–70141\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2026/hash/724be4472168f31ba1c9ac630f15dec8-Abstract-Conference.html)Cited by:[§A\.1](https://arxiv.org/html/2610.11328#A1.SS1.p1.1)\.
- Zhenget al\.\(2025\)X\. Zheng, H\. Lin, K\. He, Z\. Wang, Q\. Fu, H\. Fu, Z\. Zheng, and Y\. LiangMCU: an evaluation framework for open\-ended game agents\.InProceedings of the 42nd International Conference on Machine Learning,Vol\.267,pp\. 78221–78259\.External Links:[Link](https://proceedings.mlr.press/v267/zheng25j.html)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p2.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried,et al\.Webarena: a realistic web environment for building autonomous agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 15585–15606\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/4410c0711e9154a7a2d26f9b3816d1ef-Abstract-Conference.html)Cited by:[§A\.3](https://arxiv.org/html/2610.11328#A1.SS3.p1.1)\.
- Zhuanget al\.\(2025\)Y\. Zhuang, J\. Ren, X\. Ye, J\. Shen, T\. Yue, M\. Faayez, X\. He, X\. Zhang, Z\. Ma, L\. Qin,et al\.Simworld\-robotics: synthesizing photorealistic and dynamic urban environments for multimodal robot navigation and collaboration\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 51854–51895\.External Links:[Link](https://papers.neurips.cc/paper_files/paper/2025/hash/4a81c1aad3064fb8243b8938a49ecca0-Abstract-Conference.html)Cited by:[§A\.2](https://arxiv.org/html/2610.11328#A1.SS2.p1.1)\.

## Appendix ARelated Work

### A\.1Spatial Intelligence Evaluation

Spatial intelligence evaluation examines how models infer relations, integrate observations, and use spatial representations\. Research has expanded from spatial relations in individual images\([Du et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib10);[Chen et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib11)\)to integration across views and videos\([Yang et al\., 2025a](https://arxiv.org/html/2610.11328#bib.bib32);[Yang et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib33);[Sikarwar et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib26)\), with finer diagnoses of perspective\-taking and rotation\([Huang et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib24);[Zhang et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib25)\)\. Map understanding and reconstruction further test explicit spatial representations\([Bhat et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib27);[Petersson et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib34);[Andon Labs, 2026](https://arxiv.org/html/2610.11328#bib.bib35)\)\. Recent interactive benchmarks extend this line to scene refinement\([Liu et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib28)\), active evidence gathering\([Hong et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib36)\), and spatial task completion across simulators\([Gao et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib37)\)\. Within this line, we study agentic spatial intelligence in goal\-directed navigation, examining how agents align allocentric maps with egocentric observations, select feasible actions under physical constraints, and revise decisions as their actions reveal new evidence\.

### A\.2Embodied Navigation Evaluation

Embodied navigation evaluates whether agents can reach specified goals in 3D environments through perception and action\. Instruction\-following and object\-goal tasks\([Batra et al\., 2020](https://arxiv.org/html/2610.11328#bib.bib2)\)evaluate navigation to individual destinations\. Multi\-goal and long\-horizon benchmarks\([Wani et al\., 2020](https://arxiv.org/html/2610.11328#bib.bib38);[Khanna et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib4);[Song et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib16);[Wang et al\., 2026a](https://arxiv.org/html/2610.11328#bib.bib17)\)further evaluate whether agents can complete successive navigation stages, adjusting their plans as new observations reveal obstacles, wrong turns, or failed movements\. These navigation tasks are studied in both indoor and outdoor environments\. Indoor benchmarks evaluate navigation through reconstructed rooms and buildings, including low\-level movement in continuous environments\([Savva et al\., 2019](https://arxiv.org/html/2610.11328#bib.bib1);[Krantz et al\., 2020](https://arxiv.org/html/2610.11328#bib.bib3)\)\. For outdoor navigation, street\-view environments provide access to real streets but restrict traversal to sampled viewpoints\([Chen et al\., 2019](https://arxiv.org/html/2610.11328#bib.bib39);[Mirowski et al\., 2019](https://arxiv.org/html/2610.11328#bib.bib40);[Yang et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib19)\)\. City simulators enable continuous navigation through generated urban scenes\([Zhuang et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib18)\)and real\-city reconstructions such as UrbanGround’s Hong Kong environment, where agents can use maps to navigate\([Ju et al\., 2026b](https://arxiv.org/html/2610.11328#bib.bib31)\)\.Mine Odysseybroadens environmental coverage to diverse outdoor and indoor reconstructions of real\-world places, while opening navigation execution to agent\-written programs\. A persistent workspace allows agents to compose navigation commands into control routines, process state feedback, and retain task information across decisions\.

### A\.3Interactive Evaluation of Generalist Agents

Interactive benchmarks evaluate generalist agents through task completion in stateful environments\([Liu et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib12)\)\. WebArena and OSWorld study realistic web and desktop workflows\([Zhou et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib42);[Xie et al\., 2024](https://arxiv.org/html/2610.11328#bib.bib13)\), while OSWorld 2\.0 and Toolathlon extend evaluation to sustained execution across longer workflows\([Yuan et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib29);[Li et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib30)\)\. Game benchmarks examine planning, exploration, and control over extended play\([Paglieri et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib22);[Zhang et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib23)\)\. Beyond task design, GameWorld studies how action interfaces affect agent performance using state\-verifiable task outcomes\([Ouyang et al\., 2026](https://arxiv.org/html/2610.11328#bib.bib43)\)\.

Within Minecraft, agent methods investigate reusable skills and memory for extended execution\([Wang et al\., 2024a](https://arxiv.org/html/2610.11328#bib.bib7);[Wang et al\., 2024b](https://arxiv.org/html/2610.11328#bib.bib8);[Liu et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib9)\)\. Benchmarks assess complementary aspects of this execution: MCU evaluates composable tasks\([Zheng et al\., 2025](https://arxiv.org/html/2610.11328#bib.bib20)\), while MineExplorer tests whether agents can discover and satisfy unstated prerequisites through exploration, measuring progress with rule\-based milestones\([Ju et al\., 2026a](https://arxiv.org/html/2610.11328#bib.bib44)\)\. InMine Odyssey, agents operate through a programmable workspace, where they can process spatial evidence, compose control routines, and retain intermediate information\. Together, interaction logs, movement trajectories, and evaluator\-recorded visits support analysis of how agents use map information, organize control, and attempt recovery during the task\.

## Appendix BMap Collection and Environment Diversity

Table[3](https://arxiv.org/html/2610.11328#A2.T3)lists task counts for all 30 maps in the selected 180\-task benchmark\.

Table 3:Formal task counts in the selected benchmark\.MapTasksMapTasksBuckingham Palace5Cape Town / Bo\-Kaap9Chain Bridge1Copacabana Waterfront3Dublin Departments District7Entrup6Hagia Sophia3Hofburg7Innopolis9Memorial Hall Park1Miljacka Riverside4Harbor City Shipyard11Notre\-Dame4NYC 9/11 Memorials6Ohrid5Plaza de Mayo8Plaza Hotel6Reichstag5RMS Queen Mary10RMS Titanic6Santa Lucía Hill5Shun Lee4SoFi Stadium6Sviyazhsk6Torrey Mall5Ueno Park12Versailles3White House7Würzburg10Zürich6Total180The collection represents locations in countries and regions including South Africa, Brazil, Ireland, Japan, Germany, France, Türkiye, the United Kingdom, the United States, and Hong Kong\. Examples range from city districts in Cape Town, Rio de Janeiro, Dublin, Tokyo, and New York City to Buckingham Palace, Versailles, Hagia Sophia, the Reichstag, SoFi Stadium, and historic passenger ships\. Outdoor maps cover residential and commercial districts, parks, hillsides, and waterfronts, with variation in street connectivity, landmark density, and relief\. Indoor maps include palaces, religious buildings, hotels, stadiums, and passenger ships, varying in room arrangement, corridor structure, entrances, and vertical circulation\. Together, these settings support tasks from route planning across city districts to finding building entrances and moving between floors\. Stairs, ladders, doors, and gates provide additional local traversal and interaction contexts, as illustrated in Figure[2](https://arxiv.org/html/2610.11328#S2.F2)\. In Figure[C](https://arxiv.org/html/2610.11328#A3), we present in\-game world\-map views of all 30 benchmark environments\. Each map is labeled with its name and independently scaled to show its full layout while preserving its aspect ratio\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/xaero-maps.png)Figure 8:In\-game world\-map views of all 30 benchmark environments\.
## Appendix CDefault Settings and Game Configuration

### C\.1Default settings\.

Table[4](https://arxiv.org/html/2610.11328#A3.T4)summarizes the settings shared by the main\-comparison runs\. Controlled variants modify the designated settings as described in Section[4\.1](https://arxiv.org/html/2610.11328#S4.SS1)\.

Table 4:Default settings for the main comparison\. Controlled variants override the relevant entries\.#### Control and condition\-specific configurations\.

Our*AgentBridge*mod \(1\.0\.0\) provides player control\. In the midnight condition,*LambDynamicLights*provides illumination from held torches\. The route\-guidance condition additionally loads our*Ground Navigation*mod \(0\.1\.0\) and*Baritone*\(1\.17\.0\) to display a planned route while leaving movement under agent control \(Section[4\.1](https://arxiv.org/html/2610.11328#S4.SS1)\)\.

### C\.2Observation and interaction\.

The default observation is a single first\-person view\. Observations and execution feedback are returned through the agent’s interaction loop; the game and verifier continue to run during model inference\. Agents can request an observation delay throughobserve\_after\_sec, allowing inspection before a command finishes\. State queries and execution errors can also return textual feedback\. Thus, event\-based observation does not mean that the world pauses between decisions\. Adventure mode permits ordinary locomotion and interactions such as opening doors and climbing ladders, while restricting block modification\. Flight is disabled, and Peaceful difficulty removes hostile combat\. Agents may consult the full\-screen world map and nearby waypoint markers, but cannot access exact target coordinates or evaluator state\. The minimap panel, optional HUD, navigation hints, automated navigation, and waypoint teleportation are disabled by default\. Agent\-issued commands run in an isolated workspace without access to world saves, waypoint files, or evaluator records; a restricted local relay provides access to the approved game\-control APIs\.

### C\.3SPL and reference path construction

Success weighted by Path Length \(SPL\) measures navigation efficiency by combining task success with path length relative to a reference:

SPL=1N​∑i=1NSi​Limax⁡\(Li,Pi\),\\mathrm\{SPL\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}S\_\{i\}\\frac\{L\_\{i\}\}\{\\max\(L\_\{i\},P\_\{i\}\)\},\(1\)whereN=180N=180is the number of tasks,SiS\_\{i\}indicates task success,PiP\_\{i\}is the agent’s full\-episode path length, andLiL\_\{i\}is the reference path length through all required stops in order\. Failed tasks contribute zero; successful tasks receive higher scores when their paths are shorter relative to the reference\. Each task’s contribution is capped at one\.

We use A\*, a shortest\-path search algorithm\([Hart et al\., 1968](https://arxiv.org/html/2610.11328#bib.bib60)\), to compute a reference path for each task, shared across all models\. We represent Minecraft movement as a graph, where nodes denote candidate player positions and edges denote movements allowed by our approximate movement model\. Each edge is weighted by the 3D distance between its endpoints\. Starting from the recorded initial position, A\* finds the minimum\-length route through all required arrival regions in order\. The graph uses a 0\.5\-block horizontal lattice and approximate collision geometry\. It supports walking, steps, ladders, and bounded falls with a crouched player body; it accounts for cumulative fall damage on maps where damage is enabled\. Ordinary doors and registered button\-operated gates are modeled as operable\. Gap jumps, swimming, dynamic stance changes, and button\-operated transport are outside the current graph\. Both reference and agent path lengths are approximate\. Reference lengths depend on the graph’s discretization and simplified movement model, while agent path lengths are estimated from position samples collected at approximately 1 Hz, which may miss movement between samples\.

## Appendix DPerformance and Inference Cost

### D\.1Resource Accounting

Figures[9](https://arxiv.org/html/2610.11328#A4.F9)and[10](https://arxiv.org/html/2610.11328#A4.F10)report estimated monetary cost and output\-token usage\. Means include all 180 tasks, including failed runs\. Turns count model decisions and exclude automatic history\-summary responses\.

### D\.2Performance and Resource Use

The cost comparison distinguishes models with similar spending but different completion rates \(Figure[9](https://arxiv.org/html/2610.11328#A4.F9)\)\. At around $1 per task, DeepSeek\-V4\.1\-Flash achieves 23\.9% SR for $1\.11, compared with 15\.6% for GLM\-5\.3\-Flash at $1\.20\. DeepSeek therefore achieves higher completion at slightly lower estimated cost in this comparison\. Moving to Gemini 3\.8 Flash raises estimated spending to $5\.13 per task, about4\.6×4\.6\\timesas much, while improving SR by 2\.8 percentage points\.

GPT\-6 Astra achieves the highest SR and the lowest reported output\-token usage, reaching 85\.6% SR with 10\.3K output tokens per task \(Figure[10](https://arxiv.org/html/2610.11328#A4.F10), left\)\. Among the other models, GPT\-5\.6 Sol achieves 43\.9% SR with 68\.5K output tokens, while DeepSeek V4\.1 Flash generates 618\.0K tokens but reaches 23\.9% SR\. Output volume alone therefore does not explain the completion ranking\. This comparison concerns reported output usage, rather than a direct measurement of inference compute\.

Figure 9:SR versus estimated cost per task, with cost on a log scale\. Connected points show Claude Opus 5 effort settings\. Costs average over all 180 tasks, including failures, using the accounting in Appendix[D](https://arxiv.org/html/2610.11328#A4)\. The horizontal interval indicates the estimated cost range for Opus 5\.5\.
### D\.3Reasoning Effort

Increasing Claude Opus 5’s reasoning effort from low to max raises SR from 35\.0% to 50\.6%, while mean output usage grows from 73\.2K to 249\.9K tokens per task, approximately3\.4×3\.4\\times\(Figure[10](https://arxiv.org/html/2610.11328#A4.F10), right\)\. Estimated monetary cost increases from $10\.47 to $19\.21 per task, yielding a 15\.6\-point SR gain for an additional $8\.74\. Mean turns increase by only 11\.0%, so output usage grows much faster than the number of interaction turns\.

The additional output required also differs across effort transitions: low to medium adds 28\.3K output tokens per task for a 4\.4\-point SR gain, whereas high to max adds 105\.1K for a 6\.1\-point gain \(Figure[10](https://arxiv.org/html/2610.11328#A4.F10), right\)\. Higher effort improves completion in this model, but the latter transition requires more additional output per percentage point gained\. These comparisons average over all tasks; they do not hold successful\-task membership fixed\.

Figure 10:SR versus output\-token usage\. Left: the eight main model configurations\. Right: Claude Opus 5 at low, medium, high, and max reasoning effort; connected points show increasing effort\. Values use all 180 tasks, including failures; K denotes thousands\. Both panels use linear axes, with different SR ranges\.

## Appendix EFailure Stages

Figure[11](https://arxiv.org/html/2610.11328#A5.F11)distinguishes failure before the first required stop, failure after reaching it, and successful completion\. The first stop is the first intermediate stop, or the destination for tasks without inter\- mediate stops\. Reaching a stop uses the evaluator’s ordered\-arrival record; full success additionally requires an accepted completion claim\. Failure after the first stop can therefore include runs that visit all stops but do not complete successfully\. DeepSeek, Grok, and GLM fail before the first stop on 48\.3–55\.6% of all tasks\. Gemini reaches it more often than Sol \(70\.0% versus 67\.8%\), but fails afterward on 43\.3% of all tasks, compared with 23\.9% for Sol\. These outcomes show why initial progress need not translate into full completion\.

Figure 11:Outcomes over 180 tasks per model\. First stop means the first intermediate stop, or the destination if none\. Deaths count as failures even after all stops\. All shares use the full 180\-task denominator\. Model configurations match the main comparison\.
## Appendix FInteraction Statistics

![Refer to caption](https://arxiv.org/html/2610.11328v1/interaction2.png)Figure 12:Information access and command structure across eight models\. Values are percentages of execution requests, computed per task and averaged equally across tasks; categories can overlap\. Map key counts map toggles, Look counts camera\-orientation commands, State query includes queries combined with controls, and Image proc\. counts explicit image\-processing operations\. Shell sequence counts command sequences or pipelines within a request, and Loops counts requests containing shell or Python loop syntax\.#### Counting and aggregation\.

We classify 400,571 exec requests from the eight configurations, each evaluated on the same 180 tasks\. Each recognized category is counted at most once per request, even when a request contains repeated calls\. For each category, we divide its count by the task’s exec\-request count and then average these percentages equally across tasks\. We do not pool requests across tasks\. A request may contribute to multiple categories, and percentages need not sum to 100%\.

#### Observed interaction patterns

As revealed in Figure[12](https://arxiv.org/html/2610.11328#A6.F12), Astra, Opus 5, DeepSeek, and GLM combine commands in over 90% of requests, compared with 25\.6% for Gemini\. Gemini has the highest share of explicit image processing\. It also includes state queries less frequently\. Section[4\.2](https://arxiv.org/html/2610.11328#S4.SS2)also reveals that Gemini often remains in one place for extended periods, repeatedly checking maps, operating menus, and processing screenshots without resuming movement\. Opus 5 and Gemini use loops more often than Astra, whose requests predominantly use command sequences\. Astra and Grok both adjust the viewing direction in about 59% of requests, while Astra toggles the map more frequently\. These patterns distinguish how models request information and organize commands, without directly ranking navigation ability\.

## Appendix GCASE Studies

![Refer to caption](https://arxiv.org/html/2610.11328v1/programmatic-navigation-process-map-large.png)Figure 13:From map pixels to a movement target\.Step 176 captures the map; Step 177 computes the estimated U20 coordinates; Steps 179 and 181 use the estimate for orientation and movement\. S marks the player position and G the estimated goal; dashed lines indicate pixel offsets, not an executed route\. The displayed map provides context because the measured PNG is unavailable\. The adopted scale is uncalibrated, and execution does not establish arrival\.#### Image processing can construct an actionable spatial target\.

Gemini turns map observations into an actionable spatial target through a short Python program\. It converts player\-to\-marker pixel offsets into estimated world coordinates, then uses the result to orient and initiate movement toward U20 \(Figure[13](https://arxiv.org/html/2610.11328#A7.F13)\)\. This case illustrates how programming connects visual map grounding to action\.

#### Stair programming\.

As shown in Figure[14](https://arxiv.org/html/2610.11328#A7.F14), after testing individual stair movements, Astra combines four 1\.3\-second forward movements and turns into a single shell sequence\. At Step 35, this sequence completes one circuit around the stairwell, raising the agent from Y40 to Y48\. At Step 36, Astra uses a loop to repeat the four\-heading movement pattern three times within one tool request, ascending through Y56 and Y64 to Y72\. The loop reuses the established stair\-climbing pattern across successive levels without requiring a new agent turn for each movement\.

Together, these cases show how agents use programming to turn visual map information into explicit coordinates and individual movement commands into reusable action sequences\.

![Refer to caption](https://arxiv.org/html/2610.11328v1/programmatic-navigation-process-stairs-large.png)Figure 14:From composition to reuse in Plaza Hotel, Steps 35–36\. Step 35 executes one circuit from Y40 to Y48; Step 36 executes all three circuits from Y48 through Y56 and Y64 to Y72\. The cutaway uses the retained world’s native textures, with selected walls and room volumes removed\. Command\-window position samples and matched state endpoints are projected at their original heights over the scene for visibility\. Matching colors link each step in the enlarged scene to its height transition and complete shell command\. The original observation screenshots are omitted\. Only whitespace is reformatted in the code\. Later steps are outside the displayed window; the cutaway is a rendering, not an agent screenshot\.