SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models

Hugging Face Daily Papers Papers
vision-language-models spatial-reasoning benchmark evaluation multimodal fine-tuning

Summary

SpaceCast-Bench introduces the first benchmark for predictive spatial reasoning in vision-language models, evaluating 21 models and finding a large gap between the best model (58.0%) and human performance (87.2%), while showing fine-tuning can significantly lift performance.

Existing spatial reasoning benchmarks mainly test spatial perception: reading off relations already visible in the input. Yet real-world spatial intelligence demands predictive spatial reasoning: constructing a scene from observations, anticipating how an intervention changes it, and reasoning about the unseen outcome. We introduce SpaceCast-Bench, the first benchmark to directly and diagnostically evaluate this capability. Built around an observe-transform-infer framework, its 3,862 questions from 182 real-world scenes span 16 task types at three levels: static perception, local prediction, and global prediction, progressively requiring scene understanding, spatial state updating, and relational inference over unobserved outcomes. Evaluating 21 models exposes a stark gap: the strongest model reaches only 58.0% against 87.2% human performance, while spatially specialized models remain near random chance. Controlled analyses further reveal that bridge views are critical for integrating distributed observations, and that explicit 3D evidence benefits models more reliably than generated outcome images or videos. Fine-tuning on our programmatically generated data lifts Qwen3-VL-4B from 34.0% to 65.7% with macro-average gains across six out-of-domain benchmarks.
Original Article
View Cached Full Text

Cached at: 10/09/26, 06:25 AM

Paper page - SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models

Source: https://huggingface.co/papers/2610.12402

Abstract

Existingspatialreasoningbenchmarksmainlytestspatialperception:readingoffrelationsalreadyvisibleintheinput.Yetreal-worldspatialintelligencedemandspredictivespatialreasoning:constructingascenefromobservations,anticipatinghowaninterventionchangesit,andreasoningabouttheunseenoutcome.WeintroduceSpaceCast-Bench,thefirstbenchmarktodirectlyanddiagnosticallyevaluatethiscapability.Builtaroundanobserve-transform-inferframework,its3,862questionsfrom182real-worldscenesspan16tasktypesatthreelevels:staticperception,localprediction,andglobalprediction,progressivelyrequiringsceneunderstanding,spatialstateupdating,andrelationalinferenceoverunobservedoutcomes.Evaluating21modelsexposesastarkgap:thestrongestmodelreachesonly58.0%against87.2%humanperformance,whilespatiallyspecializedmodelsremainnearrandomchance.Controlledanalysesfurtherrevealthatbridgeviewsarecriticalforintegratingdistributedobservations,andthatexplicit3Devidencebenefitsmodelsmorereliablythangeneratedoutcomeimagesorvideos.Fine-tuningonourprogrammaticallygenerateddataliftsQwen3-VL-4Bfrom34.0%to65.7%withmacro-averagegainsacrosssixout-of-domainbenchmarks.

View arXiv pageView PDFGitHub3Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.12402 in a model README.md to link it from this page.

Datasets citing this paper1

#### hongxingli/SpaceCast-Bench Viewer• Updatedabout 3 hours ago • 3.86k

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.12402 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

PlanBench-V: A Spatial Planning Map Benchmark for Vision-Language Models

arXiv cs.CL

This paper introduces PlanBench-V, the first comprehensive benchmark for evaluating Vision-Language Models on spatial planning map interpretation, including an expert-annotated dataset and a four-dimension evaluation framework. Experiments show significant progress but highlight persistent challenges in implementation-oriented tasks.

OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs

Papers with Code Trending

OVO-S-Bench introduces a comprehensive human-annotated benchmark of 1,680 questions across 348 videos to evaluate streaming spatial intelligence in multimodal LLMs, revealing that even the best model (Gemini-3.1-Pro) trails human experts by 27 points. The benchmark exposes key limitations including allocentric mapping as a major bottleneck and chain-of-thought reasoning amplifying spatial errors.

SpatialBench: Is Your Spatial Foundation Model an All-Round Player?

Hugging Face Daily Papers

SpatialBench is a comprehensive benchmark for evaluating spatial foundation models across diverse domains and tasks, revealing limitations in current models and introducing DA-Next-5M and DA-Next to advance spatial representation learning.