Probing Spatial State Representations in Visual Foundation Models
Summary
This paper probes whether visual foundation models maintain persistent spatial states by evaluating four capability levels—spatial structure, object persistence, predictive spatial update, and actionability—using frozen-feature probes on ten representations under streaming egocentric observations, finding that readout accuracy does not guarantee history-based updates or reliable goal-directed action.
View Cached Full Text
Cached at: 11/02/26, 07:27 AM
Paper page - Probing Spatial State Representations in Visual Foundation Models
Source: https://huggingface.co/papers/2610.40636
Abstract
Visual foundation models vary in supporting persistent spatial states across structure, persistence, prediction, and action, with readout accuracy not guaranteeing history-based updates or reliable goal-directed progress.
An observer sees only part of the surrounding world, yet must keep track of what lies beyond the current view. Spatial perception therefore involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action. Dovisual foundation modelsprovide representations that support such a state? We study this question through four levels of spatial capability---spatial structure,object persistence,predictive spatial update, andactionability---usingfrozen-feature probeson ten representations under streaming egocentric observations of static indoor environments. The resulting capability profiles vary across tasks, network layers, and prefix lengths: strong readout of the visible scene does not ensure accurateobject persistenceor future-feature retrieval. Matched controls separate readout accuracy from reliance on visual history and motion. Native future prediction retains recoverablespatial structure, but a spatial reader trained on real features does not transfer to predicted features unchanged. Finally,goal-dependent candidate selectiondoes not establish reliable continuous action utility: predicted actions respond to goal changes, yet supplying the correct goal does not reliably improve progress. Across the four levels,object persistenceandactionabilitybuild oncamera-pose readout, whereasfeature predictionbuilds onvisible-scene readout. Together, these findings motivate evaluating spatial representations not only by what can be decoded from them, but by whether that information is maintained beyond observation, updated predictively, and used for goal-directed action.
View arXiv pageView PDFProject pageAdd to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.40636 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.40636 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.40636 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
This paper empirically studies spatial memory staleness in vision-language-model agents, finding that models often ignore contradictory visual evidence and that trusting stale memory can increase safety risks. The authors propose auditing mechanisms but show that visual grounding under memory-observation conflicts remains a major open challenge.
SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
SpatialAct is a new simulator-grounded benchmark that probes whether VLM agents can perform coherent spatial reasoning and translate it into actions in 3D environments across multi-turn feedback settings. Experiments reveal a significant reasoning-to-action gap, with current VLMs struggling to maintain spatial beliefs and produce reliable actions despite performing well on isolated reasoning tasks.
StateSight: Benchmarking Latent Spatial-State Reconstruction in Vision-Language Models
StateSight is a new benchmark for evaluating spatial-state reconstruction in vision-language models, showing that models like GPT-5.5 and Claude Sonnet 5 struggle with spatial reasoning tasks compared to human performance.
Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory
The paper proposes Spatial Memory Intelligence (SMI), the first framework to use an understanding model (MLLM) to systematically manage spatial memory in long-video world models via four atomic operations: spatial clustering, within-cluster sparsification, action-aware retrieval, and reliability-aware filtering. Experiments show improvements in memory sparsity, generation stability, and spatial consistency across multiple world-model backbones.
AlloSpatial: Agentic Harness Framework for Spatial Reasoning in Foundation Models
AlloSpatial is an agentic framework that enhances spatial reasoning in foundation models by converting egocentric observations into structured allocentric representations, using cognitive mapping and tool-use reasoning. It improves performance by 5-18% on benchmarks and outperforms larger models through cold-start reinforcement learning.