ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
Summary
Introduces ESI-BENCH, a comprehensive benchmark for embodied spatial intelligence built on OmniGibson, covering 10 task categories and 29 subcategories. Experiments show active exploration substantially outperforms passive approaches, with failures mainly due to action blindness rather than perception, revealing a metacognitive gap in models compared to humans.
View Cached Full Text
Cached at: 05/20/26, 06:36 AM
Paper page - ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
Source: https://huggingface.co/papers/2605.18746
Abstract
Embodied spatial intelligence requires active perception-action loops where agents strategically explore environments to uncover hidden spatial structures, with performance limited by action selection rather than perception capabilities.
Spatial intelligenceunfolds through aperception-action loop: agents act to acquire observations, and reason about how observations vary as a function of action. Rather than passively processing what is seen, they actively uncover what is unseen - occluded structure, dynamics, containment, and functionality that cannot be resolved from passive sensing alone. We move beyond prior formulations ofspatial intelligencethat assume oracle observations by recasting the observer as an actor. We introduce ESI-BENCH, a comprehensive benchmark forembodied spatial intelligencespanning 10 task categories and 29 subcategories built on OmniGibson, grounded in Spelke’score knowledge systems. Agents must decide what abilities to deploy - perception, locomotion, and manipulation - and how to sequence them to actively accumulate task-relevant evidence. We conduct extensive experiments on state-of-the-art MLLMs and find thatactive explorationsubstantially outperforms passive counterparts, with agents spontaneously discovering emergent spatial strategies without explicit instructions, while random multi-view often adds noise rather than signal despite consuming far more images. Most failures stem not from weak perception but fromaction blindness: poor action choices lead to poor observations, which in turn drive cascading errors. While explicit3D groundingstabilizes reasoning on depth-sensitive tasks, imperfect 3D representation proves more harmful than 2D baselines by distorting spatial relations. Human studies further reveal that unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality, exposing a metacognitive gap that neither better perception nor more embodied interaction alone can close.
View arXiv pageView PDFProject pageGitHub11Add to collection
Get this paper in your agent:
hf papers read 2605\.18746
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.18746 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.18746 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.18746 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Exploring Spatial Intelligence from a Generative Perspective
Researchers introduce GSI-Bench, the first benchmark to quantify generative spatial intelligence in multimodal models by evaluating 3D spatial constraint compliance during image generation. Fine-tuning on their synthetic dataset boosts both spatial editing fidelity and downstream spatial understanding, showing generative training can strengthen spatial reasoning.
SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence
Introduces SVI-Bench, a large-scale benchmark for strategic video intelligence using team sports, designed to evaluate models on dynamic scene understanding, causal reasoning, strategic simulation, and agentic synthesis. The benchmark reveals a capability cliff where models perform well on perceptual tasks but sharply degrade on higher-level strategic reasoning.
SpatialBench: Is Your Spatial Foundation Model an All-Round Player?
SpatialBench is a comprehensive benchmark for evaluating spatial foundation models across diverse domains and tasks, revealing limitations in current models and introducing DA-Next-5M and DA-Next to advance spatial representation learning.
Social Intelligence Benchmark
This paper introduces a new benchmark for evaluating social intelligence in AI systems, measuring their ability to understand and respond to social cues and interactions.
SkillEvolBench: Benchmarking the Evolution from Episodic Experience to Procedural Skills
SkillEvolBench is a diagnostic benchmark for evaluating whether large language model agents can distill episodic experience into reusable procedural skills. It includes 180 tasks across six environments and finds that current agents often struggle to form robust reusable skills, with raw trajectory reuse often outperforming distilled skills.