检测LLMs中的幻觉：追踪受损上下文共享的拓扑特征

# Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharing
Source: [https://arxiv.org/html/2609.21096](https://arxiv.org/html/2609.21096)
Anderson RochaEric WongMarcos Medeiros Raimundo††thanks:Amir Jalilifard is with the Institute of Computing, Universidade Estadual de Campinas \(UNICAMP\), Campinas, SP, Brazil\. E\-mail: jalilifard@ic\.unicamp\.br\.††thanks:Anderson Rocha is with the Institute of Computing, Universidade Estadual de Campinas \(UNICAMP\), Campinas, SP, Brazil\. E\-mail: anderson\.rocha@unicamp\.br\.††thanks:Eric Wong is with the Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA\. E\-mail: exwong@cis\.upenn\.edu\.††thanks:Marcos Medeiros Raimundo is with the Institute of Computing, Universidade Estadual de Campinas \(UNICAMP\), Campinas, SP, Brazil\. E\-mail: mrai@unicamp\.br\.

###### Abstract

In this work, we examine the topology of information flow patterns within attention graphs to effectively distinguish hallucinated from non\-hallucinated responses\. We analyze the Forman–Ricci curvature to identify structural patterns indicating information bottlenecks in attention graphs\. We then introduce a method that captures both semi\-local and global information\-flow characteristics of attention heads associated with hallucinated responses\. We evaluate our approach extensively across several LLMs and established benchmarks\. Empirical results demonstrate that our proposed single\-pass approach provides consistent improvements over existing attention\-based and multi\-response baselines across two hallucination\-detection benchmarks, while achieving competitive performance across diverse LLM architectures\. Further analysis reveals that impaired context sharing among tokens during causal generation is strongly associated with hallucination occurrences in LLMs\. In particular, hallucinated responses are consistently characterized by an over\-reliance on self\-attention, diffused context retrieval from earlier tokens, or information over\-squashing, especially in the final transformer layer\.

## IIntroduction

Amid the advances and extensive use of LLMs in various domains, new challenges regarding the factuality and correctness of LLM responses have emerged\[[1](https://arxiv.org/html/2609.21096#bib.bib20)\]\. While these models are increasingly integrated into broader autonomous workflows and downstream applications\[[2](https://arxiv.org/html/2609.21096#bib.bib21),[3](https://arxiv.org/html/2609.21096#bib.bib22)\], ensuring the factual precision of individual generated outputs remains a fundamental bottleneck for their reliability\. A single unfaithful or factually incorrect response can compromise user trust and limit a model’s practical utility\. Misleading outputs generally fall into two categories: factual errors and faithfulness errors\[[4](https://arxiv.org/html/2609.21096#bib.bib3)\]\. These errors may arise from reasoning failures\[[4](https://arxiv.org/html/2609.21096#bib.bib3)\], defective inference\[[5](https://arxiv.org/html/2609.21096#bib.bib4)\], architectural limitations\[[6](https://arxiv.org/html/2609.21096#bib.bib1)\], training data biases, or inconsistencies in the provided context\[[7](https://arxiv.org/html/2609.21096#bib.bib2)\]\. Consequently, isolating and detecting factual deviations within single\-turn generations is critical for secure deployment\.

Recent research indicates that hallucination, defined as the generation of factually incorrect or unsubstantiated responses, correlates with confusion and instability in the model during response generation\. As a result, verifying the self\-consistency of LLMs has become an important strategy for detecting hallucination\[[8](https://arxiv.org/html/2609.21096#bib.bib5),[9](https://arxiv.org/html/2609.21096#bib.bib6)\]\. Although this methodology has demonstrated promising results, it possesses two significant limitations\. Firstly, it is computationally expensive because it requires generating multiple responses to a single prompt\[[10](https://arxiv.org/html/2609.21096#bib.bib7)\]\. Secondly, this approach offers limited insight into the underlying causes and mechanisms behind these uncertainties\.

Building upon literature that interprets attention weights as a weighted adjacency matrix\[[11](https://arxiv.org/html/2609.21096#bib.bib9)\], we evaluate topological properties of the attention graph that can impede information flow\. Consequently, we propose a set of topological features derived from attention weights that effectively distinguish hallucinated from factual responses\. The main contributions of our work are as follows:

- •We demonstrate that LLM hallucinations are characterized by information bottlenecks in the attention mechanism, which we identify using Ricci curvature on the attention graph\.
- •We propose a novel set of topological features from attention maps that enable a simple linear probe to consistently outperform strong attention\-based and multi\-response baselines\.
- •We provide a comparative analysis across three LLM families, showing that the specific topological signatures of hallucination vary with model architecture, even though the core phenomenon of information bottlenecks is consistent\.

## IIBackground

In this section, we establish the conceptual foundations for our topological analysis of Large Language Models \(LLMs\)\. We first define the formal mechanism of attention as a context\-sharing process\. Subsequently, we discuss how structural bottlenecks within attention graphs lead to information over\-squashing\. Finally, we introduce the Forman\-Ricci curvature as a mathematical framework for quantifying and detecting these bottlenecks using graph topology\.

### II\-AAttention Formalism

In attention\-based language models, the entire attention generation can be seen as a context\-sharing process\. Let\{T1,T2,…,Tn\}\\left\\\{T\_\{1\},T\_\{2\},\\dots,T\_\{n\}\\right\\\}be a sequence of tokens,A=\(a1,a2,…​an\)A=\(a\_\{1\},a\_\{2\},\\dots a\_\{n\}\)denote attention score vector, andV=\{v1,v2,…​vn\}V=\\left\\\{v\_\{1\},v\_\{2\},\\dots v\_\{n\}\\right\\\}the matrix of value vectors carries the token contextual information\. The context vector for the nth token is thereforeCn=∑j=1naj⋅vj\\text\{C\}\_\{n\}=\\sum\_\{j=1\}^\{n\}a\_\{j\}\\cdot v\_\{j\}\. This context vector determines how much information from previous tokens contributes to the representation of tokenn\. Although attention weights reference previous tokens, the overall information flows forward through the sequence\.

### II\-BInformation Bottlenecks and Over\-squashing

![Refer to caption](https://arxiv.org/html/2609.21096v1/img/bottleneck.png)Fig\. 1:Information over\-squashing occurs when information from many tokens is funneled through only a single token that bears the burden of representing all that context\. However, this central token and its predecessors receive insufficient attention from subsequent tokens, preventing effective context sharing\. In this example, all the information passes through tokenT4, but later, due to high self\-attention onT5, a significant amount of information from earlier tokens is lost\. The line thickness and its color show the amount of information flow from previous tokens to the subsequent ones\.A bottleneck happens when a significant part of the information from previous tokens passes through a single token\. Such a narrow information flow leads to two main problems\. First, concentrating the context of multiple tokens into a single token leads to excessive information compression, which can cause loss of contextual details\. This effect is particularly pronounced in long sequences containing dozens and hundreds of tokens\. Second, as illustrated in Figure[1](https://arxiv.org/html/2609.21096#S2.F1), even if a single token carries comprehensive context from previous tokens, this information can subsequently be lost if attention becomes overly focused on a small subset of tokens further along the sequence, or if a token predominantly attends to itself\.

An attention matrix can be seen as a weighted graph, with tokens as vertices and attention weights as edges\. Information flows between tokens and attention layers via a message\-passing mechanism\. Distortion of the information flow due to bottlenecks limits the efficiency of message\-passing\. This phenomenon, called information over\-squashing, is frequently associated with information bottlenecks in graphs and is measured by Forman\-Ricci curvatures\[[12](https://arxiv.org/html/2609.21096#bib.bib13),[13](https://arxiv.org/html/2609.21096#bib.bib11),[14](https://arxiv.org/html/2609.21096#bib.bib12)\]\.

### II\-CForman\-Ricci Curvature

The presence of bottlenecks in attention graphs strongly co\-occurs with context loss, disrupting the coherent semantic construction expected in transformer models\. The Forman\-Ricci Curvature \(FRC\)\[[12](https://arxiv.org/html/2609.21096#bib.bib13),[15](https://arxiv.org/html/2609.21096#bib.bib23)\]provides a measure of information flow passing through an edgee=\(v,u\)e=\(v,u\)\. It consists of two components: the positive part, which measures the direct connections, and the penalty part, which measures the triangular alternative connection between two nodes\. Therefore,positiveexpected value of FRCs of a graph indicates alternative information flux rather than passing through edgeee, andnegativevalues indicate information bottleneck, i\.e\., the information route betweenvvandwwis focused on the edgeee\. Since attention graphs are directed weighted graphs, we denote the attention weight from tokenuuto tokenvvbywu​vw\_\{uv\}\.

LetG=\(V,E\)G=\(V,E\)be the attention graph, with verticesvvandwwconnected by an edgeee\. FRC is defined as:

Forman−Ricci⁡\(v,u\)=wu​v​\(wvwu​v\+wuwu​v−CLOSEOPEN∑y∼N⁡\(v\)∖uz∼N⁡\(u\)∖v\[wvwu​v​wv​y\+wuwu​v​wu​z\]\)\\begin\{split\}\\operatorname\{Forman\-Ricci\}\(v,u\)=w\_\{uv\}\\Biggl\(\\frac\{w\_\{v\}\}\{w\_\{uv\}\}\+\\frac\{w\_\{u\}\}\{w\_\{uv\}\}\\;\-\\;\\\\ \\sum\_\{\\begin\{subarray\}\{c\}y\\sim N\(v\)\\setminus u\\\\ z\\sim N\(u\)\\setminus v\\end\{subarray\}\}\\left\[\\frac\{w\_\{v\}\}\{\\sqrt\{w\_\{uv\}\\,w\_\{vy\}\}\}\+\\frac\{w\_\{u\}\}\{\\sqrt\{w\_\{uv\}\\,w\_\{uz\}\}\}\\right\]\\Biggr\)\\end\{split\}\(1\)
wherewu​vw\_\{uv\}denotes the attention weight associated with the directed edge\(u,v\)\(u,v\), and

wu=∑y∼N⁡\(u\)wu​yw\_\{u\}=\\sum\_\{y\\sim N\(u\)\}w\_\{uy\}
denotes the weighted out\-degree of nodeuu\. The neighborhoodsN⁡\(u\)N\(u\)andN⁡\(v\)N\(v\)denote the sets of vertices connected touuandvv, respectively\.

## IIIMethodology

### III\-ATopological Indexes of Information Flow

To quantify the efficiency of context sharing within attention graphs, we define three specific topological and information\-theoretic indices\.

#### III\-A1Mutual Outgoingness

In its simplified form, when the penalty term is omitted, the Forman–Ricci curvature reduces to the sum of the weighted degrees of the two connected nodes\. Let

d⁡\(u\)=∑v∼N⁡\(u\)wu​vd\(u\)=\\sum\_\{v\\sim N\(u\)\}w\_\{uv\}
denote the weighted out\-degree of tokenuu\. In attention graphs,d⁡\(u\)d\(u\)measures how broadly a token distributes attention across neighboring tokens\.

We therefore define the mutual outgoingness score of an edge\(u,v\)\(u,v\)as

Outgoingness⁡\(u,v\)=d⁡\(u\)\+d⁡\(v\)\.\\operatorname\{Outgoingness\}\(u,v\)=d\(u\)\+d\(v\)\.
Intuitively, low outgoingness values indicate highly self\-focused attention patterns, whereas high outgoingness values indicate broader context sharing across tokens\. The score can be computed efficiently in𝒪⁡\(E\)\\mathcal\{O\}\(E\)\.

#### III\-A2Forman\-Ricci Penalty

In contrast, the full Forman\-Ricci curvature includes a penalty term that reflects edge redundancy and alternative paths, and is directly tied to local information flow\. The penalty part of FRC of a weighted graph from \([1](https://arxiv.org/html/2609.21096#S2.E1)\) can be written in terms of probabilities:

P⁡\(u,v\)=∑y∼N⁡\(v\)∖u1pv​\(u\)​pv​\(y\)\+∑z∼N⁡\(u\)∖v1pu​\(v\)​pu​\(z\)\\begin\{split\}P\(u,v\)=\\sum\_\{y\\sim N\(v\)\\setminus u\}\\frac\{1\}\{\\sqrt\{p\_\{v\}\(u\)p\_\{v\}\(y\)\}\}\\\\ \+\\sum\_\{z\\sim N\(u\)\\setminus v\}\\frac\{1\}\{\\sqrt\{p\_\{u\}\(v\)p\_\{u\}\(z\)\}\}\\end\{split\}\(2\)wherepu​\(v\)=wu​vwup\_\{u\}\(v\)=\\frac\{w\_\{uv\}\}\{w\_\{u\}\},pv​\(u\)=wu​vwvp\_\{v\}\(u\)=\\frac\{w\_\{uv\}\}\{w\_\{v\}\}and so on\.

The function1pu​\(v\)​pu​\(z\)\\frac\{1\}\{\\sqrt\{p\_\{u\}\(v\)p\_\{u\}\(z\)\}\}is convex over the probability simplex with respect topup\_\{u\}\. Because convexity is preserved under summation, the penalty term is minimized when the attention distribution is uniform\[[16](https://arxiv.org/html/2609.21096#bib.bib14)\]\.

#### III\-A3Joint Entropy as an FRC Penalty Proxy

From the chain rule of entropy, we can write:

H⁡\(U,V\)=H⁡\(U\)\+H⁡\(V∣U\)H\(U,V\)=H\(U\)\+H\(V\\mid U\)\(3\)where

H\(V∣U\)=−∑u∈Vp\(u\)∑v∼N⁡\(u\)p\(v∣u\)logp\(v∣u\)H\(V\\mid U\)=\-\\sum\_\{u\\in V\}p\(u\)\\sum\_\{v\\sim N\(u\)\}p\(v\\mid u\)\\log p\(v\\mid u\)\(4\)
assuming thatpu​\(v\)=wu​vwup\_\{u\}\(v\)=\\frac\{w\_\{uv\}\}\{w\_\{u\}\}, andp⁡\(u\|v\)=p⁡\(u,v\)p⁡\(v\)p\(u\|v\)=\\frac\{p\(u,v\)\}\{p\(v\)\}andp⁡\(u,v\)=wu​vWtotalp\(u,v\)=\\frac\{w\_\{uv\}\}\{W\_\{\\text\{total\}\}\}, then

p⁡\(v∣u\)=p⁡\(u,v\)p⁡\(u\)=wu​vWtotalwuWtotal=wu​vwu=pu​\(v\)p\(v\\mid u\)=\\frac\{p\(u,v\)\}\{p\(u\)\}=\\frac\{\\frac\{w\_\{uv\}\}\{W\_\{\\text\{total\}\}\}\}\{\\frac\{w\_\{u\}\}\{W\_\{\\text\{total\}\}\}\}=\\frac\{w\_\{uv\}\}\{w\_\{u\}\}=p\_\{u\}\(v\)\(5\)
Therefore,

H\(V∣U\)=−∑u∈Vp\(u\)∑v∼N⁡\(u\)pu\(v\)logpu\(v\)H\(V\\mid U\)=\-\\sum\_\{u\\in V\}p\(u\)\\sum\_\{v\\sim N\(u\)\}p\_\{u\}\(v\)\\log p\_\{u\}\(v\)\(6\)
From equations \([2](https://arxiv.org/html/2609.21096#S3.E2)\), \([3](https://arxiv.org/html/2609.21096#S3.E3)\), and \([6](https://arxiv.org/html/2609.21096#S3.E6)\), a uniformPu​\(v\)P\_\{u\}\(v\)maximizes the joint entropy while it minimizes the total penalty, indicating a possible reverse correlation betweenH⁡\(u,v\)H\(u,v\)and total penalty in Forman\-Ricci\. Both the entropy formulation and the Forman–Ricci penalty term are optimized under more uniformly distributed attention weights\. This motivates the use of attention entropy as a computationally efficient proxy for curvature\-related information dispersion and bottleneck behavior\.

### III\-BExtracting Features for Hallucination Detection

The exact computation of the full Forman\-Ricci penalty incurs a worst\-case complexity of𝒪⁡\(V⋅d¯2\)\\mathcal\{O\}\(V\\cdot\\bar\{d\}^\{2\}\), whered¯\\bar\{d\}is the average node degree, making it costly for long sequences or densely connected graphs\. Given that attention graphs often span hundreds or thousands of tokens, using raw edge\-level curvatures can be noisy and overly granular\. More importantly, such local curvatures make the final set of features for hallucination detection sentence\-length dependent\.

To address this and create a set of meaningful, fixed\-size features as the probe’s input, we summarize the distribution of mutual outgoingness scores in each attention head by computing the 10th, 50th, and 90th percentiles, capturing the distribution of local connectivity patterns within each attention head\. Additionally, motivated by the inverse relationship between curvature penalties and joint Shannon entropyH⁡\(U,V\)H\(U,V\), and its linear time complexity of𝒪⁡\(V\)\\mathcal\{O\}\(V\), we include the joint entropy of each attention head as a cheap\-to\-calculate feature to characterize global information flow and identify systemic information bottlenecks associated with hallucinated generations\.

## IVExperiments

In this section, we describe the experimental setup, report the analytical results, and present the outcomes of hallucination detection and discuss the effectiveness of our approach\.

### IV\-AExperimental Setup

We carried out all the experiments using three small\-to\-medium\-sized models — Mistral 7B Instruct, LLaMA 3\.1 8B Instruct, and Phi\-4—and one larger model, Qwen3\-32B\. In particular, we included Qwen3\-32B as a larger reasoning\-oriented model to examine whether reasoning\-specific mechanisms \(e\.g\., explicit intermediate reasoning steps\) influence characteristics such as outgoingness and attention entropy\. For each model–benchmark pair, we generated responses using two sampling temperatures \(0\.1 and 1\.0\)\. We assessed the performance of our approach using two benchmarks commonly employed in hallucination studies: TruthfulQA\[[17](https://arxiv.org/html/2609.21096#bib.bib16)\]and NQ\-Open\[[18](https://arxiv.org/html/2609.21096#bib.bib15)\]\. The TruthfulQA comprises 817 questions and ground\-truth answers specifically designed to elicit imitative falsehoods and mimic human misconceptions\. It contains 38 categories, including health, law, conspiracies, and fiction\. The NQ\-Open is a dataset of realistic questions collected from Google Search and consists of 3,610 questions also accompanied by ground\-truth answers\.

Both LLaMA 3\.1 8B and Mistral 7B Instruct contain 32 layers, each with 32 attention heads, whereas Phi\-4 has 40 layers with 40 heads per layer\. Qwen3\-32B with 64 layers and 64 heads per layer, has the largest architectural depth and width among the evaluated models\. The experimental setup, including system prompt templates, attention implementation \(Eager attention\), hallucination probe, and dimensionality reduction settings, follows the exact configuration in\[[19](https://arxiv.org/html/2609.21096#bib.bib8)\], with the exception of label generation, where we used GPT\-4 instead of GPT\-4 mini due to its superior ability to capture nuanced linguistic patterns and provide significantly more accurate labels\.

The use of an LLM\-as\-a\-judge was motivated by the generation of tens of thousands of responses with complex linguistic variations, making exhaustive human evaluation impractical\. To assess the reliability of the judge, we randomly sampled 100 prompts and manually compared the generated labels against the reference answers\. The evaluation showed that GPT\-4 achieved accuracies of 92\.0%±3\.34%, 91\.0%±5\.61%, 88\.0%±6\.4%, and 87\.0%±6\.6% \(95% confidence intervals\) when evaluating responses generated by LLaMA 3\.1 8B Instruct, Phi\-4, Mistral 7B Instruct, and Qwen3\-32B, respectively\. Since generated responses can contain complex and nuanced information, determining whether a response should be classified as a hallucination may involve subjective judgment\. For instance, a model may initially provide a correct answer but later introduce an unsupported or fabricated detail during the continuation of its response\. In such cases, due to our strict evaluation criteria, we consider the entire response to be hallucinated even if it is a minor portion of the overall response, whereas an LLM\-as\-judge may assign a correct label\.

In all experiments, the Mann–Whitney U test was used to assess the statistical significance of differences in per\-head features\. We show the analytics results only for the 10th percentile since the 50th and 90th percentiles follow the same patterns\.

### IV\-BForman\-Ricci curvature patterns

We first evaluate whether there are distinct information\-flow patterns in Forman–Ricci curvature that differentiate hallucinated answers from the rest\. To this end, the LLM responses for each of the NQ\-Open questions are partitioned into hallucinated and non\-hallucinated subsets\. For each subset, we then calculate the mean curvature of each attention head in the last five transformer layers, as the last few layers are known to capture higher\-level semantic representations\[[20](https://arxiv.org/html/2609.21096#bib.bib17)\]\.

![Refer to caption](https://arxiv.org/html/2609.21096v1/img/ricci-diff-llama.png)Fig\. 2:The difference between the mean values of Forman\-Ricci curvatures per head for hallucinated and not hallucinated responses for LlaMA\. For the majority of the attention heads, hallucinated responses have lower curvature values, indicating a less fluid information flow\.Figure[2](https://arxiv.org/html/2609.21096#S4.F2)presents the average per\-head differences in Forman–Ricci curvature between hallucinated and non\-hallucinated responses for LLaMA\. Negative values \(shown in blue\) indicate that hallucinated responses tend to exhibit lower curvature values, suggesting more obstructed information flow due to the higher penalty\. Across all models, the last layer exhibits less fluid information flow in hallucinated responses\. This effect is particularly pronounced in LLaMA and Mistral, where the distinction is more consistent across layers and heads\. For Phi and Qwen, however, only the majority of heads in the last layer show attenuated information flow for hallucinated responses\. The complete results and statistical significance values are presented in the Appendix, Figures[7](https://arxiv.org/html/2609.21096#A1.F7)and[8](https://arxiv.org/html/2609.21096#A1.F8)\.

### IV\-COutgoingness, entropy and self\-focus patterns

To investigate how outgoingness, distribution of attention across preceding tokens, and self\-attention correlate with hallucinated responses, we compared per\-head averages of the 10th, 50th, and 90th percentiles of outgoingness scores, attention entropy, and self\-attention values across all layers\.

As illustrated in Figure[3](https://arxiv.org/html/2609.21096#S4.F3)\(also in Appendix, Figure[9](https://arxiv.org/html/2609.21096#A1.F9)\), LLaMA 3\.1 8B Instruct shows the strongest separation of outgoingness and attention entropy among the other models\. In turn, Phi\-4 shows a statistically significant difference between the means of hallucinated and non\-hallucinated responses for a subset of attention heads, while exhibiting almost no separation of attention entropies for most heads\. Mistral 7B Instruct also exhibits meaningful differences among the majority of heads for both outgoingness and attention entropy, though the effect is less pronounced than in LLaMA 3\.1 8B Instruct\. Finally, Qwen3\-32B has the fewest attention heads and shows statistically significant differences in outgoingness and attention entropy\.

![Refer to caption](https://arxiv.org/html/2609.21096v1/img/10th-percentile-llama.png)\(a\)Outgoingness \- LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/attention-entropy-llama.png)\(b\)Attention entropy \- LLaMA 3\.1 8B Instruct

Fig\. 3:P\-values for the per\-head comparisons of the 10th percentile outgoingness scores and attention entropy across all layers for LLaMA 3\.1 8B Instruct\.We also analyzed how each feature evolves across layers over time by averaging attention heads, and compared these patterns for hallucinated responses\. We compute the mean by taking the arithmetic average of a given feature across all attention heads in a given layer\. As shown in Figure[4](https://arxiv.org/html/2609.21096#S4.F4)for LLaMA 3\.1 8B \(and Appendix Figure[10](https://arxiv.org/html/2609.21096#A1.F10)for Mistral 7B\), both models exhibit higher average outgoingness scores and more diffuse attention distributions in hallucinated responses, suggesting impaired context sharing\. In contrast, both Phi and Qwen show minimal or no variation in outgoingness across layers and no noticeable change in attention entropy \(see Appendix, Figure[10](https://arxiv.org/html/2609.21096#A1.F10)\)\. Also, no clear pattern was observed that differentiates the earlier layers from the later ones in these behaviors\.

![Refer to caption](https://arxiv.org/html/2609.21096v1/img/llama-10th-percentile-layer-wise.png)\(a\)Outgoingness \- LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/llama-attention-entropy-layer-wise.png)\(b\)Attention entropy \- LLaMA 3\.1 8B Instruct

Fig\. 4:Feature evolution across layers and comparison between hallucinated and non\- hallucinated response for LLaMA 3\.1 8B Instruct\.Excessive self\-attention is frequently associated with a loss of contextual information from surrounding tokens, tracking closely with a reduction in the accuracy of transformer outputs\[[21](https://arxiv.org/html/2609.21096#bib.bib18)\]\. Our experiments indicate that both Phi\-4 and Qwen3\-32B do not exhibit diffused attention, even when producing hallucinated content\. To investigate this further, we analyzed the self\-attention patterns across all layers and heads\. We computed the average self\-attention per head, then measured the difference in mean self\-attention between hallucinated and non\-hallucinated outputs\. Figure[5](https://arxiv.org/html/2609.21096#S4.F5)illustrates that for both models, in most attention heads, hallucinated responses correspond to higher self\-attention—an effect that is especially evident in the later layers\. Interestingly, this pattern is reversed and less statistically significant for the majority of heads in LLaMA and Mistral, where the hallucinated responses exhibit more diffuse attention \(see Appendix, Figure[11](https://arxiv.org/html/2609.21096#A1.F11)\)\.

![Refer to caption](https://arxiv.org/html/2609.21096v1/img/self-attention-phi4.png)Fig\. 5:Mean per\-head self\-attention differences between hallucinated and non\-hallucinated responses for Phi\-4\.
### IV\-DResults

We compare the results with Laplacian eigenvalues\[[19](https://arxiv.org/html/2609.21096#bib.bib8)\]as our main baseline due to its state\-of\-the\-art performance, single\-pass approach, and a simple yet effective and fast approach for identifying hallucination\.

We further compared our method with Eigen Score\[[10](https://arxiv.org/html/2609.21096#bib.bib7)\], which detects hallucinations by generating multiple responses to a query and measuring the variance of the resulting information as a proxy for model uncertainty\. In the original formulation, the user prompt is transformed into multiple template\-based variants that preserve intent while inducing response diversity\. However, this setup assumes access to prompt\-engineering mechanisms that are not typically available in real\-world use, where users submit a single prompt\. Although prompt variants could be generated using an auxiliary LLM, doing so introduces additional computational cost and latency\. To better reflect practical deployment constraints, we did not use template\-based prompt reformulation\. Instead, we generated multiple responses to the original prompt using a sampling temperature of 0\.5, thereby inducing controlled stochasticity without modifying the input query\.

We use a linear probe in three main feature configurations to distinguish hallucinated from non\-hallucinated responses: \(a\) using the 10th, 50th, and 90th percentiles of outgoingness scores along with attention entropy; \(b\) using all deciles \(10th through 90th\) of outgoingness scores and attention entropy to assess the effect of including finer\-grained percentiles; and \(c\) extending configuration \(a\) by adding the mean per\-head self\-attention scores to isolate the contribution of self\-attention features to the probe’s performance\. The best results were achieved using setting \(c\) \(see Appendix, Table[III](https://arxiv.org/html/2609.21096#A1.T3)\)\.

The experiments indicate that the semi\-local outgoingness scores and global attention entropy features yield substantial gains in area under the curve \(AUC\) compared to the Laplacian eigenvalues\. When fine\-grained outgoingness features were added, no significant gains were observed\. When the average self\-attention per\-head values were added to the feature set, this increased AUC for all Phi\-4 and Qwen3\-23B models\.

Qwen3\-32B achieved the weakest performance among the evaluated models\. A plausible explanation is its explicit multi\-step reasoning mechanism\. Since attention entropy and outgoingness are proxies for uncertainty and dispersion of information flow, they may be less sensitive to structured reasoning processes\. As a result, these features have limited discriminative power across most attention heads\.

To assess the contribution of each feature type to hallucination detection, we evaluated the model’s performance under a cumulative feature\-addition scheme\. As shown in Figure[6](https://arxiv.org/html/2609.21096#S4.F6)and Table[II](https://arxiv.org/html/2609.21096#S4.T2), progressively adding feature groups generally led to consistent improvements in AUC\. While incorporating the 90th percentile of outgoingness improved performance in most settings, its contribution was occasionally unstable, leading to marginal decreases in certain configurations\. Overall, our method achieves an average improvement of approximately 3\.4 AUC points over the selected baselines\.

TABLE I:AUC Comparison Between Laplacian eigenvalues, EigenScore, and our feature set classification results\. Here we show the results for setting \(a\) which includes our base features\. The best results however, were attained using setting \(c\)\.TempBenchmarkModelAUCLapEigvalsEigenScoreOurs \- setting \(a\)0\.1TruthfulQALLaMA 3\.1 8B Instruct80\.5283\.7985\.340\.1TruthfulQAPhi 478\.4381\.6281\.750\.1TruthfulQAMistral 7B Instruct v0\.378\.8081\.6980\.000\.1TruthfulQAQwen3\-32B77\.3672\.1780\.171\.0TruthfulQALLaMA 3\.1 8B Instruct78\.2680\.2280\.731\.0TruthfulQAPhi 474\.4679\.8284\.001\.0TruthfulQAMistral 7B Instruct v0\.373\.2375\.3878\.001\.0TruthfulQAQwen3\-32B75\.1769\.3573\.530\.1NQOPENLLaMA 3\.1 8B Instruct75\.6771\.4979\.340\.1NQOPENPhi 480\.2681\.6982\.000\.1NQOPENMistral 7B Instruct v0\.377\.1079\.4579\.750\.1NQOPENQwen3\-32B76\.1671\.7378\.021\.0NQOPENLLaMA 3\.1 8B Instruct76\.1971\.1680\.461\.0NQOPENPhi 483\.6184\.1184\.491\.0NQOPENMistral 7B Instruct v0\.376\.3879\.6177\.291\.0NQOPENQwen3\-32B78\.4973\.6381\.73TABLE II:Cumulative Feature Ablation \(AUC %\) across TruthfulQA and NQOPEN datasets\. Features are added sequentially\.BenchmarkTempModelq10\+ q50\+ q90\+ att\_entropyTruthfulQA0\.1LLaMA 3\.1 8B Instruct83\.1084\.0584\.1585\.34Phi 479\.3080\.2080\.1081\.75Mistral 7B Instruct v0\.377\.5078\.2078\.3580\.00Qwen3\-32B78\.0078\.8578\.9080\.171\.0LLaMA 3\.1 8B Instruct78\.5079\.6079\.5080\.73Phi 482\.1082\.7082\.8584\.00Mistral 7B Instruct v0\.376\.1076\.8576\.7078\.00Qwen3\-32B70\.3372\.073\.0273\.53NQOPEN0\.1LLaMA 3\.1 8B Instruct78\.0778\.7878\.9479\.34Phi 479\.5280\.4580\.2882\.00Mistral 7B Instruct v0\.377\.2877\.5078\.1779\.75Qwen3\-32B74\.1874\.9877\.3578\.021\.0LLaMA 3\.1 8B Instruct78\.9879\.9880\.0380\.46Phi 482\.8882\.9983\.0284\.49Mistral 7B Instruct v0\.375\.3376\.1376\.6877\.29Qwen3\-32B79\.2080\.1080\.2581\.73![Refer to caption](https://arxiv.org/html/2609.21096v1/img/cummulative-auc_line_plot.png)Fig\. 6:Cumulative feature set effect for Llama 3\.1 8B Instruct accross NQOPEN dataset\.

## VRelated works

Hallucination detection methods generally fall into two categories\. The first one supposes that hallucination is the result of the model’s sequence\-level uncertainty or semantic inconsistency during token generation, and therefore its main focus is to identify hallucinated responses through measuring the amount of intravariability of multiple generated responses\[[8](https://arxiv.org/html/2609.21096#bib.bib5)\],\[[10](https://arxiv.org/html/2609.21096#bib.bib7)\],\[[22](https://arxiv.org/html/2609.21096#bib.bib19)\]\. Such methods obtain multiple responses by applying top\-p/top\-k sampling strategy and then measure the semantic divergence among them\. Although this approach has shown high detection rates, its performance depends on the number of generated samples and the variability of prompts with the same meaning\. The more diversified the prompt and the higher the number of responses, the higher the detection accuracy\[[10](https://arxiv.org/html/2609.21096#bib.bib7)\]\. This characteristic makes this approach costly, time\-consuming, and therefore inadequate for real\-time detection\.

The second approach, which draws on more recent studies, has explored the use of specific attention\-weight patterns to detect hallucinated responses in large language models \(LLMs\)\[[19](https://arxiv.org/html/2609.21096#bib.bib8)\]\[[23](https://arxiv.org/html/2609.21096#bib.bib10)\]\. While it has demonstrated promising results, the studies that have followed the use of attention weights for hallucination detection often rely on simplified assumptions and lack a more in\-depth analysis of the complex patterns co\-occurring with hallucinations\.

Bazarova et al\.\[[23](https://arxiv.org/html/2609.21096#bib.bib10)\]posit that hallucinated responses introduce a unique and measurable topological dissimilarity in the attention graphs connecting the response to the prompt\. However, this framework may oversimplify the nature of attentional failures\. It does not fully account for scenarios such as information bottlenecks, strong attentional links between response tokens and irrelevant prompt tokens, or the diverse patterns that may emerge across different domains and LLM architectures\.

Similarly, Binkowski et al\.\[[19](https://arxiv.org/html/2609.21096#bib.bib8)\]hypothesize that hallucinations are a direct consequence of information bottlenecks and propose using the eigenvalues of the Laplacian matrix of attention graphs to detect them\. While innovative, Laplacian eigenvalues reflect a range of graph characteristics beyond bottlenecks, including graph bipartiteness and the number of spanning trees, which may or may not be related to hallucination\. Despite their hypothesis, the authors did not conduct empirical experiments to show how Laplacian eigenvalues vary depending on whether a response is hallucinated\. Furthermore, this method provides a global perspective on graph properties, potentially obscuring localized attentional features that could be more directly associated with hallucinated responses\. Moreover, the authors show that the higher the number of eigenvalues, the better the distinguishability of their feature set and report that their best results were achieved when 100 eigenvalues were extracted from each attention head\. However, this puts a hard constraint on the minimum response length, making the method less effective for short responses\.

In the current study, we carried out a series of analyses to examine the relationship between attention graph bottlenecks and hallucinations, mapping out how these patterns manifest across different LLM architectures\. Afterward, we propose a set of features that enable high\-performance hallucination detection, and explain why and how hallucination occurs across different LLMs\.

## VILimitations

##### Theoretical and Operational Confounders\.

Our topological framework cannot distinctly separate pathological context\-sharing impairments from either absolute parametric knowledge gaps or functional attention sinks \(*e\.g\.*, initial tokens or punctuation\)\. Because absolute data omissions and architectural attention anchors both induce localized distortions in attention distributions, the observed signatures risk confounding true semantic bottlenecks with benign structural properties or generic model uncertainty\.

##### Architectural and Evaluation Scope\.

The proposed method exhibits explicit architectural dependencies, resulting in significant performance drops on reasoning\-dense models like Qwen3\-32B\. A recent study\[[24](https://arxiv.org/html/2609.21096#bib.bib24)\]has suggested that reasoning traces can contain signals associated with incorrect, deceptive, or fabricated information in the final response; however, our analysis does not attempt to establish a causal relationship between reasoning mechanisms and the proposed topological features\. Rather, our objective was to examine whether the identified information\-flow patterns remain observable in a reasoning\-oriented model or whether they differ under such generation processes\. Furthermore, our empirical validation is restricted to single\-turn, English short\-form question\-answering which is the predominant setting adopted by existing hallucination detection benchmarks\. The generalizability of these topological indices to non\-English syntax, long\-form multi\-turn generation, or complex agentic reasoning pathways remains unverified\.

## VIIConclusion

In this study, we conducted a comprehensive set of analyses and provided evidence that hallucinations manifest differently across various large language models \(LLMs\)\. We demonstrate that simplified attention\-based properties alone are insufficient to characterize these phenomena\. To address this shortcoming, we introduced a set of response\-length\-independent features to capture a broader range of hallucination patterns\. Through a detailed examination and extensive experimentation, we observed that while attention behaviors during hallucination vary across different LLMs, impaired context sharing is a consistent topological signature correlated with hallucinations across all models examined\.

Subsequently, using the proposed features, a simple linear probe achieved competitive hallucination\-detection performance and consistently outperformed the selected baselines across the evaluated models, benchmarks, and generation settings\. Our selection of diverse LLMs allowed us to analyze both shared and distinct behavioral patterns\. However, the impact of model size, specifically the number of parameters, was not investigated in this work\. For instance, although the Phi\-4 model, which has approximately twice as many parameters as Llama 3\.1 8B Instruct and Mistral 7B Instruct, exhibits notably different behavior, it remains unclear whether these differences are attributable to model size or other architectural factors\. Therefore, the interplay between model size and attention behavior during hallucination can be investigated further in future work\. Also, our results indicate that producing correct responses requires tokens to balance reliance on their own semantic representations with the contextual information from other tokens\. Hallucinations can therefore be interpreted as a failure to correctly integrate contextual information from the prompt\. Still, it remains an open question how internal model dynamics differ when hallucination is caused by genuine knowledge gaps rather than misaligned context sharing\.

## References

- \[1\]S\. Tao, L\. Yao, H\. Ding, Y\. Xie, Q\. Cao, F\. Sun, J\. Gao, H\. Shen, and B\. Ding\(2024\)When to trust llms: aligning confidence with response quality\.arXiv preprint arXiv:2404\.17287\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[2\]N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.Advances in Neural Information Processing Systems36,pp\. 8634–8652\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[3\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)React: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[4\]L\. Berglund, M\. Tong, M\. Kaufmann, M\. Balesni, A\. C\. Stickland, T\. Korbak, and O\. Evans\(2023\)The reversal curse: llms trained on" a is b" fail to learn" b is a"\.arXiv preprint arXiv:2309\.12288\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[5\]X\. Chen, M\. Li, X\. Gao, and X\. Zhang\(2022\)Towards improving faithfulness in abstractive summarization\.Advances in Neural Information Processing Systems35,pp\. 24516–24528\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[6\]L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin,et al\.\(2025\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems43\(2\),pp\. 1–55\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[7\]J\. Maynez, S\. Narayan, B\. Bohnet, and R\. McDonald\(2020\)On faithfulness and factuality in abstractive summarization\.arXiv preprint arXiv:2005\.00661\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p1.1)\.
- \[8\]S\. Farquhar, J\. Kossen, L\. Kuhn, and Y\. Gal\(2024\)Detecting hallucinations in large language models using semantic entropy\.Nature630\(8017\),pp\. 625–630\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p2.1),[§V](https://arxiv.org/html/2609.21096#S5.p1.1)\.
- \[9\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p2.1)\.
- \[10\]C\. Chen, K\. Liu, Z\. Chen, Y\. Gu, Y\. Wu, M\. Tao, Z\. Fu, and J\. Ye\(2024\)INSIDE: llms’ internal states retain the power of hallucination detection\.arXiv preprint arXiv:2402\.03744\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p2.1),[§IV\-D](https://arxiv.org/html/2609.21096#S4.SS4.p2.1),[§V](https://arxiv.org/html/2609.21096#S5.p1.1)\.
- \[11\]F\. Barbero, A\. Banino, S\. Kapturowski, D\. Kumaran, J\. Madeira Araújo, O\. Vitvitskyi, R\. Pascanu, and P\. Veličković\(2024\)Transformers need glasses\! information over\-squashing in language tasks\.Advances in Neural Information Processing Systems37,pp\. 98111–98142\.Cited by:[§I](https://arxiv.org/html/2609.21096#S1.p3.1)\.
- \[12\]Forman\(2003\)Bochner’s method for cell complexes and combinatorial ricci curvature\.Discrete & Computational Geometry29\(3\),pp\. 323–374\.Cited by:[§II\-B](https://arxiv.org/html/2609.21096#S2.SS2.p2.1),[§II\-C](https://arxiv.org/html/2609.21096#S2.SS3.p1.1)\.
- \[13\]U\. Alon and E\. Yahav\(2020\)On the bottleneck of graph neural networks and its practical implications\.arXiv preprint arXiv:2006\.05205\.Cited by:[§II\-B](https://arxiv.org/html/2609.21096#S2.SS2.p2.1)\.
- \[14\]J\. Topping, F\. Di Giovanni, B\. P\. Chamberlain, X\. Dong, and M\. M\. Bronstein\(2021\)Understanding over\-squashing and bottlenecks on graphs via curvature\.arXiv preprint arXiv:2111\.14522\.Cited by:[§II\-B](https://arxiv.org/html/2609.21096#S2.SS2.p2.1)\.
- \[15\]R\. P\. Sreejith, K\. Mohanraj, J\. Jost, E\. Saucan, and A\. Samal\(2016\)Forman curvature for complex networks\.Journal of Statistical Mechanics: Theory and Experiment2016\(6\),pp\. 063206\.Cited by:[§II\-C](https://arxiv.org/html/2609.21096#S2.SS3.p1.1)\.
- \[16\]H\. H\. Bauschke and P\. A\. DiBerardino\(2022\)Minimal angle spread in the probability simplex with respect to the uniform distribution\.Journal of Nonsmooth Analysis and Optimization3\(Original research articles\)\.Cited by:[§III\-A2](https://arxiv.org/html/2609.21096#S3.SS1.SSS2.p3.1)\.
- \[17\]S\. Lin, J\. Hilton, and O\. Evans\(2021\)Truthfulqa: measuring how models mimic human falsehoods\.arXiv preprint arXiv:2109\.07958\.Cited by:[§IV\-A](https://arxiv.org/html/2609.21096#S4.SS1.p1.1)\.
- \[18\]K\. Lee, M\. Chang, and K\. Toutanova\(2019\)Latent retrieval for weakly supervised open domain question answering\.arXiv preprint arXiv:1906\.00300\.Cited by:[§IV\-A](https://arxiv.org/html/2609.21096#S4.SS1.p1.1)\.
- \[19\]J\. Binkowski, D\. Janiak, A\. Sawczyn, B\. Gabrys, and T\. Kajdanowicz\(2025\)Hallucination detection in llms using spectral features of attention maps\.arXiv preprint arXiv:2502\.17598\.Cited by:[§IV\-A](https://arxiv.org/html/2609.21096#S4.SS1.p2.1),[§IV\-D](https://arxiv.org/html/2609.21096#S4.SS4.p1.1),[§V](https://arxiv.org/html/2609.21096#S5.p2.1),[§V](https://arxiv.org/html/2609.21096#S5.p4.1)\.
- \[20\]H\. Zhao, H\. Chen, F\. Yang, N\. Liu, H\. Deng, H\. Cai, S\. Wang, D\. Yin, and M\. Du\(2024\)Explainability for large language models: a survey\.ACM Transactions on Intelligent Systems and Technology15\(2\),pp\. 1–38\.Cited by:[§IV\-B](https://arxiv.org/html/2609.21096#S4.SS2.p1.1)\.
- \[21\]K\. H\. I\. Arif, S\. A\. Dip, K\. Hussain, L\. Zhang, and C\. Thomas\(2025\)PAINT: paying attention to informed tokens to mitigate hallucination in large vision\-language model\.arXiv preprint arXiv:2501\.12206\.Cited by:[§IV\-C](https://arxiv.org/html/2609.21096#S4.SS3.p4.1)\.
- \[22\]J\. Ren, J\. Luo, Y\. Zhao, K\. Krishna, M\. Saleh, B\. Lakshminarayanan, and P\. J\. Liu\(2022\)Out\-of\-distribution detection and selective generation for conditional language models\.arXiv preprint arXiv:2209\.15558\.Cited by:[§V](https://arxiv.org/html/2609.21096#S5.p1.1)\.
- \[23\]A\. Bazarova, A\. Yugay, A\. Shulga, A\. Ermilova, A\. Volodichev, K\. Polev, J\. Belikova, R\. Parchiev, D\. Simakov, M\. Savchenko,et al\.\(2025\)Hallucination detection in llms with topological divergence on attention graphs\.arXiv preprint arXiv:2504\.10063\.Cited by:[§V](https://arxiv.org/html/2609.21096#S5.p2.1),[§V](https://arxiv.org/html/2609.21096#S5.p3.1)\.
- \[24\]T\. Korbak, M\. Balesni, E\. Barnes, Y\. Bengio, J\. Benton, J\. Bloom, M\. Chen, A\. Cooney, A\. Dafoe, A\. Dragan,et al\.\(2025\)Chain of thought monitorability: a new and fragile opportunity for ai safety\.arXiv preprint arXiv:2507\.11473\.Cited by:[§VI](https://arxiv.org/html/2609.21096#S6.SS0.SSS0.Px2.p1.1)\.

## Appendix AAdditional Experiments

### A\-AComplementary results

![Refer to caption](https://arxiv.org/html/2609.21096v1/img/ricci-diff-llama.png)\(a\)LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/ricci-diff-misteral.png)\(b\)Mistral 7B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/ricci-diff-phi4.png)\(c\)Phi\-4
\(d\)Qwen3\-32B

Fig\. 7:The difference between the mean values of Forman\-Ricci curvatures per head for hallucinated and not hallucinated responses\. For the majority of the attention heads, hallucinated responses have a lower curvature values which is an indication of a less fluid information flow\. This pattern is more evident accross all layers in LLaMA and Mistral\. For Phi and Qwen, the last layer presents less fluid context sharing for hallucinated responses\.![Refer to caption](https://arxiv.org/html/2609.21096v1/img/llama-pval-ricci-diff.png)\(a\)LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/misteral-pval-ricci-diff.png)\(b\)Mistral 7B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/phi-pval-ricci-diff.png)\(c\)Phi\-4
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/qwenpval-ricci-diff.png.png)\(d\)Qwen3\-32B

Fig\. 8:The p\-values of the differences between the mean values of Forman\-Ricci curvatures per head for hallucinated and not hallucinated responses\. The heads with statistical signiuficance are highlited with red lines\.![Refer to caption](https://arxiv.org/html/2609.21096v1/img/10th-percentile-llama.png)\(a\)Outgoingness – LLaMA
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/attention-entropy-llama.png)\(b\)Attention entropy – LLaMA
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/10th-percentile-misteral.png)\(c\)Outgoingness – Mistral
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/attention-entropy-misteral.png)\(d\)Attention entropy – Mistral
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/10th-percentile-phi.png)\(e\)Outgoingness – Phi\-4
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/attention-entropy-phi.png)\(f\)Attention entropy – Phi\-4
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/outgoingness-10th-heatmap-QWEN.png)\(g\)Outgoingness – Qwen3\-32B
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/attention-entropy-heatmap-QWEN.png)\(h\)Attention entropy – Qwen3\-32B

Fig\. 9:P\-values for the per\-head comparisons of the 10th\-percentile outgoingness scores and attention entropy across all layers\. LLaMA 3\.1 8B Instruct exhibits the strongest separation across the majority of attention heads\.![Refer to caption](https://arxiv.org/html/2609.21096v1/img/llama-10th-percentile-layer-wise.png)\(a\)Outgoingness \- LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/llama-attention-entropy-layer-wise.png)\(b\)Attention entropy \- LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/misteral-10th-percentile-layer-wise.png)\(c\)Outgoingness \- Mistral 7B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/misteral-attention-entropy-layer-wise.png)\(d\)Attention entropy \- Mistral 7B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/phi-10th-percentile-layer-wise.png)\(e\)Outgoingness \- Phi\-4
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/phi-attention-entropy-layer-wise.png)\(f\)Attention entropy \- Phi\-4
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/outgoingness-10th-QWEN.png)\(g\)Outgoingness \- Qwen3\-32B
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/attention-entropy-QWEN.png)\(h\)Attention entropy \- Qwen3\-32B

Fig\. 10:Feature average evolution across layers and comparison between hallucinated and non\-hallucinated responses\. Both LLaMA 3\.1 Instruct and Mistral 7B Instruct shows diffused attention whereas for Phi\-4 shows little to no noticeable difference in outgoingness or attention distribution between the two response types\. Also, for LLaMA 3\.1 Instruct and Mistral 7B Instruct, the average of 10th percentile of the outgoingness score over attention heads is higher for hallucinated response in the terminal layers\. The same pattern was observed for 50th and 90th percentiles\.![Refer to caption](https://arxiv.org/html/2609.21096v1/img/self-attention-llama.png)\(a\)LLaMA 3\.1 8B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/self-attention-misteral.png)\(b\)Mistral 7B Instruct
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/self-attention-phi4.png)\(c\)Phi\-4
![Refer to caption](https://arxiv.org/html/2609.21096v1/img/self-attention-QWEN.png)\(d\)Qwen3\-32B

Fig\. 11:Mean per\-head self\-attention differences between hallucinated and non\-hallucinated responses across the evaluated models\. Unlike LLaMA 3\.1 8B Instruct and Mistral 7B Instruct, Phi\-4 and Qwen3\-32B exhibit higher self\-attention for hallucinated responses across a large proportion of layers and attention heads\.TempBenchmarkModelAUCLapEigvalsEigenScoreOurs \-setting \(a\)Ours \-setting \(b\)Ours \-setting \(c\)0\.1TruthfulQALLaMA 3\.1 8B Instruct80\.5283\.7985\.3485\.5685\.470\.1TruthfulQAPhi 478\.4381\.6281\.7581\.8082\.410\.1TruthfulQAMistral 7B Instruct v0378\.8081\.6980\.0080\.1780\.110\.1TruthfulQAQwen3\-32B77\.3672\.1780\.1780\.3380\.471\.0TruthfulQALLaMA 3\.1 8B Instruct78\.2680\.2280\.7380\.2581\.001\.0TruthfulQAPhi 474\.4679\.8284\.0083\.9885\.311\.0TruthfulQAMistral 7B Instruct v0373\.2375\.3878\.0078\.6878\.191\.0TruthfulQAQwen3\-32B75\.1769\.3573\.5374\.1174\.650\.1NQOPENLLaMA 3\.1 8B Instruct75\.6771\.4979\.3479\.3879\.330\.1NQOPENPhi 480\.2681\.6982\.0082\.3683\.340\.1NQOPENMistral 7B Instruct v0377\.1079\.4579\.7579\.3479\.770\.1NQOPENQwen3\-32B76\.1671\.7378\.0278\.1678\.371\.0NQOPENLLaMA 3\.1 8B Instruct76\.1971\.1680\.4680\.1580\.211\.0NQOPENPhi 483\.6184\.1184\.4984\.7785\.121\.0NQOPENMistral 7B Instruct v0376\.3879\.6177\.2977\.9277\.131\.0NQOPENQwen3\-32B78\.4973\.6381\.7381\.4381\.79TABLE III:AUC comparison between LapEigvals, EigenScore, and our method across three feature settings\. Setting \(a\) includes the 10th, 50th, and 90th percentile outgoingness per\-head features, along with attention entropy\. Setting \(b\) extends \(a\) by incorporating all deciles \(10th–90th\) to evaluate the impact of finer\-grained local information\. Setting \(c\) augments \(a\) with average per\-head self\-attention values as additional features\.

### A\-BLLM\-as\-judge

To assess the reliability of the LLM\-as\-judge evaluations, we randomly sampled 100 prompts, established the correct answer for each prompt, and manually evaluated the judgments produced by GPT\-4\. Although none of the evaluated model responses exhibited severe label imbalance \(moderate average ration of 15:85 for incorrect and correct judgments, respectively\), we also report Gwet’s AC1 alongside accuracy as a more conservative measure of LLM\-as\-judge performance\.

ModelAccuracyGwet’s AC1LLaMA 3\.1 8B Instruct0\.920\.86Mistral 7B Instruct v0\.30\.880\.76Qwen3\-32B0\.870\.82Phi\-40\.910\.85TABLE IV:The performance of the LLM\-as\-judge across different models’ responses\.