GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods

Hugging Face Daily Papers Papers
multimodal explainability evaluation framework vision-language-models spatial-reasoning shortcuts

Summary

GridVQA-X introduces a diagnostic framework to evaluate cross-modal explainability by distinguishing genuine spatial-relational reasoning from cross-modal shortcuts in multimodal models.

With the increasing development of Vision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recent Multimodal Explainable AI (MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between true cross-modal reasoning (e.g., spatial composition) and shallow cross-modal shortcuts (e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the first diagnostic framework specifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train paired ground-truth models on identical architectures: M_{pure}, which learns robust spatial-relational reasoning and M_{spur}, which is structurally forced to rely on cross-modal shortcuts. This behavioral divergence creates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuine spatial-relational reasoning and those exploiting cross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.
Original Article
View Cached Full Text

Cached at: 06/26/26, 02:04 AM

Paper page - GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods

Source: https://huggingface.co/papers/2606.14740

Abstract

GridVQA-X introduces a diagnostic framework to evaluate cross-modal explainability by distinguishing genuine spatial-relational reasoning from cross-modal shortcuts in multimodal models.

With the increasing development ofVision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recentMultimodal Explainable AI(MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between truecross-modal reasoning(e.g., spatial composition) and shallowcross-modal shortcuts(e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the firstdiagnostic frameworkspecifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train pairedground-truth modelson identical architectures: M_{pure}, which learns robustspatial-relational reasoningand M_{spur}, which is structurally forced to rely oncross-modal shortcuts. Thisbehavioral divergencecreates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuinespatial-relational reasoningand those exploitingcross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2606\.14740

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.14740 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.14740 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.14740 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

VIEScore2: Unified Image Evaluation with Spatially Grounded Explanations

Hugging Face Daily Papers

VIEScore2 introduces a unified evaluator for image generation and editing that predicts quality scores and defect locations on an NxN grid in a single model pass, using GRPO post-training with cell-level localization rewards. It outperforms general-purpose VLMs such as Gemini-3-Flash, achieving 0.601 overall SRCC and top-three results on five of six defect-localization benchmarks.