SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes?
Summary
SpatialScout introduces a benchmark evaluating vision-language model agents on active object-level 3D mapping across 80 indoor scenes, revealing that even the strongest model, Claude Opus 5.5, achieves only 41.71% mean overlap, highlighting major gaps in precise metric map construction.
View Cached Full Text
Cached at: 10/22/26, 03:36 PM
Paper page - SpatialScout: Can VLM Agents Actively Explore and Map 3D Scenes?
Source: https://huggingface.co/papers/2610.26659
Abstract
SpatialScout evaluates vision-language agents on active 3D object mapping, revealing major gaps in precise metric map construction.
Spatial intelligence requires agents to actively acquire visual evidence and integrate observations into a coherent representation of the three-dimensional world. Yet spatial question answering offers limited evidence of whether models can construct accuratemetric mapsthrough autonomous exploration. We introduce SpatialScout, a benchmark for evaluatingvision-language model(VLM) agents onactive object-level 3D mapping. SpatialScout comprises 80 diverse indoor scenes containing 3,140 target objects. Given an object inventory, RGB observations, camera calibration, and camera poses, agents control a camera to explore each scene and estimate object centers in a shared coordinate frame. The task requires coordinating viewpoint selection,cross-view association,geometric estimation, and map revision under bounded interaction budgets. Evaluation jointly measures localization accuracy and target coverage through object-scale overlap scores that penalize missing predictions. Experiments with frontier VLMs reveal substantial limitations: the strongest evaluated model, Claude Opus 5.5, achieves a mean overlap score of 41.71%, with 47.23% of targets exceeding anintersection-over-union(IoU) threshold of 0.5 and only 3.25% exceeding 0.9. These results reveal a substantial gap between recovering coarse object layouts and constructing complete, precisemetric maps. SpatialScout provides a quantitative testbed for developing agents that build reliable 3D spatial representations throughactive perception. Code is available at https://github.com/JesseZZZZZ/SpatialScout.
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2610\.26659
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.26659 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.26659 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.26659 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
SpatialAct is a new simulator-grounded benchmark that probes whether VLM agents can perform coherent spatial reasoning and translate it into actions in 3D environments across multi-turn feedback settings. Experiments reveal a significant reasoning-to-action gap, with current VLMs struggling to maintain spatial beliefs and produce reliable actions despite performing well on isolated reasoning tasks.
SceneActBench: Can Agents Act on the 3D Scenes They See?
SceneActBench is a benchmark for evaluating VLM agents on acting in complete multi-object 3D scenes, using task-specific geometric metrics across five tasks.
MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse
MetaSpatial is a reinforcement learning framework that enhances 3D spatial reasoning in vision-language models, enabling coherent and physically plausible 3D scene generation without hard-coded optimizations.
SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them
SpatialCLI presents a framework that trains vision-language models to use specialist spatial tools and then internalize those capabilities, boosting Qwen3-VL-8B-Instruct on the MindCube benchmark from 29.3% to 84.6% with tools.
GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models
The paper introduces GeoNatureAgent Benchmark, the first benchmark for evaluating LLM agents on environmental geospatial analysis tasks via structured tool calls. It evaluates seven models on 93 tasks across 18 categories and finds Claude Sonnet 4 achieves highest accuracy at 60.8%, while open-weight models like DeepSeek V3.2 offer strong cost-performance tradeoffs.