Mechanisms of Prompt-Induced Hallucination in Vision-Language Models
Summary
This paper investigates prompt-induced hallucinations in vision-language models through mechanistic analysis, identifying specific attention heads responsible for the models' tendency to favor textual prompts over visual evidence. The authors demonstrate that ablating these PIH-heads reduces hallucinations by at least 40% without additional training, revealing model-specific mechanisms underlying this failure mode.
View Cached Full Text
Cached at: 04/20/26, 08:33 AM
# Mechanisms of Prompt-Induced Hallucination in Vision-Language Models Source: https://arxiv.org/abs/2601.05201 View PDF (https://arxiv.org/pdf/2601.05201) > Abstract: Large vision-language models (VLMs) are highly capable, yet often hallucinate by favoring textual prompts over visual evidence. We study this failure mode in a controlled object-counting setting, where the prompt overstates the number of objects in the image (e.g., asking a model to describe four waterlilies when only three are present). At low object counts, models often correct the overestimation, but as the number of objects increases, they increasingly conform to the prompt regardless of the discrepancy. Through mechanistic analysis of three VLMs, we identify a small set of attention heads whose ablation substantially reduces prompt-induced hallucinations (PIH) by at least 40% without additional training. Across models, PIH-heads mediate prompt copying in model-specific ways. We characterize these differences and show that PIH ablation increases correction toward visual evidence. Our findings offer insights into the internal mechanisms driving prompt-induced hallucinations, revealing model-specific differences in how these behaviors are implemented. ## Submission history From: William Rudman Jr [view email (https://arxiv.org/show-email/22c8a476/2601.05201)] **[v1]** Thu, 8 Jan 2026 18:23:03 UTC (4,825 KB) **[v2]** Fri, 17 Apr 2026 15:14:39 UTC (4,822 KB)
Similar Articles
Evaluating the Effects of Prompt Perturbation on Bias and Hallucination in Large Language Models
This arXiv paper evaluates how prompt perturbations affect bias and hallucination in LLMs used for decision-making tasks, finding that perturbations can mitigate these issues in some models, with Claude 3 outperforming GPT-3.5 across most datasets.
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
This paper analyzes hallucination in large language models as a structural consequence of three architectural decisions: self-attention's co-occurrence learning, maximum likelihood estimation training objective, and autoregressive decoding's left-to-right commitment. It maps each mechanism to specific hallucination types and argues that dataset pathologies amplify but do not cause these vulnerabilities.
Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors
This paper studies why language models hallucinate, proposing that hallucinations often stem from biased latent inference (inference misalignment) rather than missing knowledge. It introduces TrapQA, a controlled diagnostic testbed to test reasoning against priors, and demonstrates that hallucinations can arise from misleading latent associations.
Hallucination as a Feature, not a Defect: Evaluating a multi-agent architecture to transform speculative language-model outputs into testable scientific hypotheses
This paper proposes a Rust-based multi-agent architecture that uses LLM hallucinations as a feature to generate and evaluate scientific hypotheses, comparing its performance against direct prompting and other methods.
Dismantling Pathological Shortcuts: A Causal Framework for Faithful LVLM Decoding
This paper reveals that hallucination in large vision-language models is caused by a dynamic structural misalignment where certain attention heads act as risky mediators, decoupling from visual evidence to lock onto language priors. The authors propose Fox, a training-free causal intervention framework that diagnoses and physically severs these pathological shortcuts, achieving state-of-the-art performance in faithful decoding.