Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text

Hugging Face Daily Papers Papers
spatial-reasoning image-generation benchmark evaluation vision-language ai-research

Summary

Proposes ProVisE, a benchmark-agnostic framework to evaluate spatial cognition in image-generation models using pixel-space outputs, and introduces SpatialGen-Bench for unified evaluation across 14 spatial subtasks.

Spatial intelligence is essential for agents to move from static semantic understanding toward interacting with the physical world. Many spatial tasks are grounded in continuous visual scenes, where locations, regions, and paths are more naturally expressed by pointing, marking, or drawing than by reporting precise coordinates or discrete textual symbols. Yet existing spatial reasoning benchmarks usually require coordinates, options, or text, creating an answer-interface mismatch for image-generation models. This makes it difficult to evaluate image-generation models under the same task semantics as text-output VLMs, despite their ability to externalize spatial judgments directly in pixel space. We propose ProVisE (Protocolized Visual Evaluation), a benchmark-agnostic framework that elicits protocol-constrained visual answers from image-generation models and parses them into structured predictions compatible with original metrics. ProVisE also includes an Agentic builder that constructs and validates task-specific protocols for new benchmarks. We further introduce SpatialGen-Bench, a curated diagnostic benchmark of 470 samples across 14 spatial subtasks, four capability levels, and diverse answer forms. We evaluate representative text-output VLMs and image-generation models in a unified setting and validate Agentic protocol construction on six external spatial benchmarks. Results show that image-generation models are competitive when spatial answers can be externalized directly in pixel space, while text-output VLMs retain a clear advantage in compositional spatial reasoning. These findings reveal complementary strengths of pixel-space expression and text-based reasoning and establish a metric-compatible testbed for studying spatial cognition in image-generation models.
Original Article
View Cached Full Text

Cached at: 07/24/26, 05:05 AM

Paper page - Show, Don’t Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text

Source: https://huggingface.co/papers/2607.21072

Abstract

Spatialintelligenceisessentialforagentstomovefromstaticsemanticunderstandingtowardinteractingwiththephysicalworld.Manyspatialtasksaregroundedincontinuousvisualscenes,wherelocations,regions,andpathsaremorenaturallyexpressedbypointing,marking,ordrawingthanbyreportingprecisecoordinatesordiscretetextualsymbols.Yetexistingspatialreasoningbenchmarksusuallyrequirecoordinates,options,ortext,creatingananswer-interfacemismatchforimage-generationmodels.Thismakesitdifficulttoevaluateimage-generationmodelsunderthesametasksemanticsastext-outputVLMs,despitetheirabilitytoexternalizespatialjudgmentsdirectlyinpixelspace.WeproposeProVisE(ProtocolizedVisualEvaluation),abenchmark-agnosticframeworkthatelicitsprotocol-constrainedvisualanswersfromimage-generationmodelsandparsesthemintostructuredpredictionscompatiblewithoriginalmetrics.ProVisEalsoincludesanAgenticbuilderthatconstructsandvalidatestask-specificprotocolsfornewbenchmarks.WefurtherintroduceSpatialGen-Bench,acurateddiagnosticbenchmarkof470samplesacross14spatialsubtasks,fourcapabilitylevels,anddiverseanswerforms.Weevaluaterepresentativetext-outputVLMsandimage-generationmodelsinaunifiedsettingandvalidateAgenticprotocolconstructiononsixexternalspatialbenchmarks.Resultsshowthatimage-generationmodelsarecompetitivewhenspatialanswerscanbeexternalizeddirectlyinpixelspace,whiletext-outputVLMsretainaclearadvantageincompositionalspatialreasoning.Thesefindingsrevealcomplementarystrengthsofpixel-spaceexpressionandtext-basedreasoningandestablishametric-compatibletestbedforstudyingspatialcognitioninimage-generationmodels.

View arXiv pageView PDFProject pageGitHubAdd to collection

Get this paper in your agent:

hf papers read 2607\.21072

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.21072 in a model README.md to link it from this page.

Datasets citing this paper1

#### wx91726/SpatialGen-Bench Updated42 minutes ago • 87

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.21072 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Exploring Spatial Intelligence from a Generative Perspective

Hugging Face Daily Papers

Researchers introduce GSI-Bench, the first benchmark to quantify generative spatial intelligence in multimodal models by evaluating 3D spatial constraint compliance during image generation. Fine-tuning on their synthetic dataset boosts both spatial editing fidelity and downstream spatial understanding, showing generative training can strengthen spatial reasoning.

Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models

Hugging Face Daily Papers

This paper presents a systematic frozen-feature probing study comparing vision-language models (VLMs) and video generation models (VGMs) on spatial intelligence tasks. It finds that VLMs excel at semantic tagging and instance grouping, while VGMs provide better dense geometry and camera motion signals, and a naive fusion of both yields strong performance across all axes.