GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods
Summary
GridVQA-X introduces a diagnostic framework to evaluate cross-modal explainability by distinguishing genuine spatial-relational reasoning from cross-modal shortcuts in multimodal models.
View Cached Full Text
Cached at: 06/26/26, 02:04 AM
Paper page - GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods
Source: https://huggingface.co/papers/2606.14740
Abstract
GridVQA-X introduces a diagnostic framework to evaluate cross-modal explainability by distinguishing genuine spatial-relational reasoning from cross-modal shortcuts in multimodal models.
With the increasing development ofVision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recentMultimodal Explainable AI(MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between truecross-modal reasoning(e.g., spatial composition) and shallowcross-modal shortcuts(e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the firstdiagnostic frameworkspecifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train pairedground-truth modelson identical architectures: M_{pure}, which learns robustspatial-relational reasoningand M_{spur}, which is structurally forced to rely oncross-modal shortcuts. Thisbehavioral divergencecreates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuinespatial-relational reasoningand those exploitingcross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.
View arXiv pageView PDFGitHub1Add to collection
Get this paper in your agent:
hf papers read 2606\.14740
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.14740 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.14740 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.14740 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG
This paper introduces GranuVistaVQA, a multimodal benchmark with element-level annotations, and GranuRAG, a framework that treats visual elements as first-class retrieval units for verifiable multimodal RAG, achieving up to 29.2% improvement over baselines.
Virgil: Navigating Explainability for Transformer-based Language Models
Virgil is an interactive system designed to help users discover and compare explainability tools for transformer-based language models through a curated knowledge base and unified interface.
DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
Introduces DrawingVQA, the first benchmark for evaluating multimodal large language models on real-world construction drawings, with 33 drawings and 92 QA pairs across three reasoning depths, revealing a gap between model and expert performance.
Do VLMs Reason Like Engineers? A Benchmark and a Stage-wise Evaluation
This paper introduces EngVQA, a multimodal benchmark for evaluating engineering reasoning in vision-language models, along with an 8-stage automatic evaluation framework that enables fine-grained analysis of reasoning failures. It reveals substantial limitations in current VLMs' engineering reasoning capabilities.
iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning
Introduces iVGR, a reinforcement learning framework that internalizes visual localization into textual reasoning for multimodal language models, eliminating the need for explicit visual grounding during inference while improving fine-grained perception performance.