Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs
Summary
This paper investigates whether open-source quantized LLMs encode a linearly separable truthfulness signal in their hidden states. Across three 7B-8B instruction-tuned models, a linear probe on a single mid-network layer achieves 0.904-1.000 AUROC on hallucination detection benchmarks, outperforming sampling-based methods.
arXiv:2606.02628v1 Announce Type: new
Abstract: We investigate whether open-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest. Across three $7$B--$8$B instruction-tuned models (Llama-3.1-8B, Mistral-7B, Qwen2.5-7B) loaded in $4$-bit NF4 quantization, we extract per-layer hidden states on four hallucination benchmarks (TruthfulQA, HaluEval-QA, FEVER, and a controlled synthetic set) and compare four detection approaches: linear and MLP probes, INSIDE EigenScore, self-consistency, and attention entropy. A linear probe on a single mid-network layer achieves $0.904$--$1.000$ AUROC on held-out splits, while sampling-based detectors do not exceed $0.541$ AUROC under the same protocol. The truthfulness signal is approximately linear: MLP probes rarely surpass linear probes by more than $0.01$ AUROC. Peak probing layers fall in a consistent band across model families on natural-language benchmarks -- blocks~$13$--$18$ of~$32$ for Llama and Mistral, and blocks~$19$--$25$ of~$28$ for Qwen. First-block attention entropy provides a complementary signal in knowledge-grounded settings ($0.866$--$0.941$ AUROC on HaluEval-QA) at no additional inference cost. The low discriminability of sampling methods under this protocol reflects a structural mismatch between paired-label evaluation and the information these methods access, rather than an inherent limitation of those methods. Code and data are released for full reproducibility on a single $8$\,GB GPU.
View Cached Full Text
Cached at: 06/03/26, 09:39 AM
# Hallucination Is Linearly Decodable from Mid-Layer Hidden States in Quantized LLMs
Source: [https://arxiv.org/html/2606.02628](https://arxiv.org/html/2606.02628)
11institutetext:University of Macau
11email:mc25101@um\.edu\.mo###### Abstract
We investigate whether open\-source LLMs encode a linearly separable truthfulness signal in their hidden states, and at which network depth this signal is strongest\. Across three77B–88B instruction\-tuned models \(Llama\-3\.1\-8B, Mistral\-7B, Qwen2\.5\-7B\) loaded in44\-bit NF4 quantization, we extract per\-layer hidden states on four hallucination benchmarks \(TruthfulQA, HaluEval\-QA, FEVER, and a controlled synthetic set\) and compare four detection approaches: linear and MLP probes, INSIDE EigenScore, self\-consistency, and attention entropy\. A linear probe on a single mid\-network layer achieves0\.9040\.904–1\.0001\.000AUROC on held\-out splits, while sampling\-based detectors do not exceed0\.5410\.541AUROC under the same protocol\. The truthfulness signal is approximately linear: MLP probes rarely surpass linear probes by more than0\.010\.01AUROC\. Peak probing layers fall in a consistent band across model families on natural\-language benchmarks—blocks1313–1818of3232for Llama and Mistral, and blocks1919–2525of2828for Qwen\. First\-block attention entropy provides a complementary signal in knowledge\-grounded settings \(0\.8660\.866–0\.9410\.941AUROC on HaluEval\-QA\) at no additional inference cost\. The low discriminability of sampling methods under this protocol reflects a structural mismatch between paired\-label evaluation and the information these methods access, rather than an inherent limitation of those methods\. Code and data are released for full reproducibility on a single88GB GPU\.
## 1Introduction
Hallucinations, i\.e\., fluent but factually unsupported generations, remain a prevalent failure mode of large language models in real\-world deployments\[[11](https://arxiv.org/html/2606.02628#bib.bib1),[10](https://arxiv.org/html/2606.02628#bib.bib2)\]\. Detecting them at inference time is a key practical concern\. A growing literature offers two complementary families of detectors\.*Sampling\-based*methods draw several stochastic generations for the same prompt and flag disagreement at the lexical\[[23](https://arxiv.org/html/2606.02628#bib.bib7),[17](https://arxiv.org/html/2606.02628#bib.bib8)\], semantic\[[13](https://arxiv.org/html/2606.02628#bib.bib5),[8](https://arxiv.org/html/2606.02628#bib.bib6)\], or internal\-state level\[[4](https://arxiv.org/html/2606.02628#bib.bib4)\]\.*Representation\-based*methods train a small classifier \(probe\) to decode truthfulness directly from intermediate hidden states\[[2](https://arxiv.org/html/2606.02628#bib.bib3),[3](https://arxiv.org/html/2606.02628#bib.bib17),[15](https://arxiv.org/html/2606.02628#bib.bib18),[1](https://arxiv.org/html/2606.02628#bib.bib22)\]\. These two families differ sharply in compute: sampling requiresK≥5K\\\!\\geq\\\!5extra generations per prompt, while a probe requires only a single forward pass\. Yet they are rarely compared head\-to\-head on identical models and datasets, especially in the small, open, quantized regime that practitioners deploy on consumer hardware\.
We address this gap with a unified evaluation framework that runs end\-to\-end on a single88GB GPU\. We study three77B–88B instruction\-tuned models in44\-bit NF4 quantization\[[7](https://arxiv.org/html/2606.02628#bib.bib15)\]\(Llama\-3\.1\-8B\-Instruct\[[9](https://arxiv.org/html/2606.02628#bib.bib12)\], Mistral\-7B\-Instruct\-v0\.3\[[12](https://arxiv.org/html/2606.02628#bib.bib13)\], and Qwen2\.5\-7B\-Instruct\[[19](https://arxiv.org/html/2606.02628#bib.bib14)\]\) across four hallucination datasets: TruthfulQA\[[16](https://arxiv.org/html/2606.02628#bib.bib9)\], HaluEval\-QA\[[14](https://arxiv.org/html/2606.02628#bib.bib10)\], FEVER\[[21](https://arxiv.org/html/2606.02628#bib.bib11)\], and a controlled synthetic benchmark\. For every \(model, dataset\) combination we evaluate four detection methods: \(i\) SAPLMA\-style linear and MLP probes\[[2](https://arxiv.org/html/2606.02628#bib.bib3)\]on per\-layer hidden states; \(ii\) INSIDE EigenScore\[[4](https://arxiv.org/html/2606.02628#bib.bib4)\]over multiple sampled completions; \(iii\) self\-consistency\[[23](https://arxiv.org/html/2606.02628#bib.bib7),[17](https://arxiv.org/html/2606.02628#bib.bib8)\]scored by exact\-match plurality and sentence\-embedding similarity\[[20](https://arxiv.org/html/2606.02628#bib.bib21)\]; and \(iv\) attention\-entropy summaries at three transformer blocks\. Our main findings and contributions are as follows\.
##### Contributions\.
\(1\) A head\-to\-head, layer\-resolved comparison of four hallucination\-detection methods on three open77B–88B chat models across four datasets, fully reproducible on consumer hardware via44\-bit quantization\. \(2\) Empirical evidence that truthful and hallucinated hidden states are linearly separable in mid\-to\-late transformer blocks, with consistent peak\-probing blocks across model families—blocks1313–1818of3232for Llama and Mistral, blocks1919–2525of2828for Qwen\. \(3\) A negative result: INSIDE and self\-consistency provide no discriminative signal under a paired\-label protocol, which we trace to a structural mismatch between the evaluation setup and the information these methods access \(§[5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px2)\)\.
## 2Related Work
Hallucination taxonomy\.\[[11](https://arxiv.org/html/2606.02628#bib.bib1)\]and\[[10](https://arxiv.org/html/2606.02628#bib.bib2)\]provide comprehensive surveys distinguishing*intrinsic*\(contradicting the input\) from*extrinsic*\(unsupported by the input\) hallucinations\. We focus on the extrinsic / factual\-correctness dimension, since all four of our datasets supply binary truthful/hallucinated labels per \(prompt, answer\) pair\.
Representation\-based detection\.Linear classifier probes\[[1](https://arxiv.org/html/2606.02628#bib.bib22)\]expose information that is present, but not necessarily used, in a network’s internal states\. SAPLMA\[[2](https://arxiv.org/html/2606.02628#bib.bib3)\]applies this idea to true/false classification of LLM\-generated statements; CCS\[[3](https://arxiv.org/html/2606.02628#bib.bib17)\]extracts a truth direction without labels; ITI\[[15](https://arxiv.org/html/2606.02628#bib.bib18)\]edits activations along that direction to improve factuality\. Our framework re\-implements the SAPLMA per\-layer probing protocol end\-to\-end on three modern open chat models and contrasts it with sampling\-based baselines under matched compute\.
Sampling\-based uncertainty\.Self\-consistency\[[23](https://arxiv.org/html/2606.02628#bib.bib7)\]uses generation agreement as a confidence signal; SelfCheckGPT\[[17](https://arxiv.org/html/2606.02628#bib.bib8)\]extends this to black\-box hallucination detection\. Semantic entropy\[[13](https://arxiv.org/html/2606.02628#bib.bib5),[8](https://arxiv.org/html/2606.02628#bib.bib6)\]clusters samples by meaning to obtain a more robust uncertainty estimate\. INSIDE\[[4](https://arxiv.org/html/2606.02628#bib.bib4)\]replaces semantic clustering with the log\-determinant of the regularized covariance of internal states across sampled completions \(the*EigenScore*\); this is a recent and competitive sampling\-and\-internal\-state hybrid, and we include it as our primary sampling baseline\.
Cross\-examination and external supervision\.Other lines of work query a second model to detect errors\[[5](https://arxiv.org/html/2606.02628#bib.bib19)\]or supervise hallucination labels with external knowledge\. We do not consider these here: our goal is white\-box detection inside a single, deployed model\.
Quantized inference\.44\-bit NF4\[[7](https://arxiv.org/html/2606.02628#bib.bib15)\]and88\-bit\[[6](https://arxiv.org/html/2606.02628#bib.bib16)\]quantization make77B–88B models accessible on commodity GPUs\. Our results indicate that*hidden\-state probing remains effective under44\-bit weight quantization*, with peak AUROC reaching0\.9980\.998on HaluEval\-QA and1\.0001\.000on the synthetic benchmark\.
Positioning\.Each of the above methods was typically evaluated in isolation or on a different model family, and we are not aware of a prior head\-to\-head, layer\-resolved comparison of probe\-based and sampling\-based detectors on the same models under the same compute budget\. Our study brings all four methods into a single framework and provides evidence that the evaluation protocol \(paired\-label vs\. generate\-then\-judge\) is a key factor governing their relative performance \(§[3\.1](https://arxiv.org/html/2606.02628#S3.SS1)\)\.
## 3Methodology
Models\.We study three publicly accessible open\-source chat models loaded in44\-bit NF4 quantization\[[7](https://arxiv.org/html/2606.02628#bib.bib15)\]with double quantization andbfloat16compute dtype \(see Appendix[0\.R](https://arxiv.org/html/2606.02628#Pt0.A18)for full configuration details\)\. The three models, summarized in Table[1](https://arxiv.org/html/2606.02628#S3.T1), span three widely deployed open\-source families \(Llama, Mistral, Qwen\) and two architectural sizes \(2828\- and3232\-block transformers\)\.
Table 1:Models used in this study, all loaded in44\-bit NF4 quantization withbfloat16compute dtype\. VRAM denotes the quantized weight footprint, exclusive of activations\.Datasets\.We cap each dataset atN=400N\\\!=\\\!400items\. For every dataset we pair a truthful \(label=1\\text\{label\}\{=\}1\) and a hallucinated \(label=0\\text\{label\}\{=\}0\) answer for the same prompt, yielding near\-perfect50:5050\{:\}50class balance \(positive\-class ratios range from0\.5000\.500to0\.5130\.513\)\.
TruthfulQA\.We use the multiple\-choice configuration of TruthfulQA\[[16](https://arxiv.org/html/2606.02628#bib.bib9)\]and, for each question, pair the canonical correct choice with one randomly sampled incorrect choice\. This produces controlled adversarial pairs of plausible but wrong continuations and is the most difficult of our four benchmarks because the wrong answers are explicitly chosen to mimic plausible human errors\.
HaluEval\-QA\.HaluEval\[[14](https://arxiv.org/html/2606.02628#bib.bib10)\]provides a knowledge\-conditioned QA split with paired correct \(label11\) and hallucinated \(label0\) answers generated by GPT\-style models\. We prepend the supplied knowledge to the question, giving the model an explicit evidence channel that the other datasets lack\.
FEVER\.FEVER\[[21](https://arxiv.org/html/2606.02628#bib.bib11)\]provides Wikipedia\-supported claims labelled as*supported*\(11\) or*refuted*\(0\); claims with insufficient evidence are excluded to preserve binary clarity\.
Synthetic\.A controlled benchmark of paired true/false statements about world capitals, chemical symbols, literary authorship, and planetary orbits, generated from four knowledge banks \(see Appendix[0\.I](https://arxiv.org/html/2606.02628#Pt0.A9)\)\. Both items in a pair share the same prompt, differing only in a single substantive token, giving a clean class signal\.
### 3\.1Paired\-Label Protocol
A design choice central to interpreting our results is the*paired\-label protocol*\. In each dataset, the candidate answer is supplied by the benchmark rather than generated by the model\. We concatenate prompt and candidate answer, run a single forward pass, and extract hidden states following the SAPLMA convention\[[2](https://arxiv.org/html/2606.02628#bib.bib3)\]\. Probe\-based detectors thus observe the model’s internal representation of the supplied answer, whereas sampling\-based detectors \(INSIDE, self\-consistency\) drawKKfree continuations from the prompt alone and measure their diversity*without conditioning on the supplied candidate*\.
This asymmetry has two consequences\. First, probes receive a clean binary signal with near\-perfect50:5050\{:\}50class balance\. Second, theKKstochastic completions are unconstrained by the supplied candidate, so the EigenScore and self\-consistency scores measure*generative diversity*, which is only weakly correlated with the truthfulness of the supplied answer\. The two detector families thus answer*different questions*under this protocol \(§[5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px2)\)\.
### 3\.2Hidden\-state and attention extraction
For every⟨prompt,answer⟩\\langle\\text\{prompt\},\\text\{answer\}\\ranglepair we run a single forward pass and extract allL\+1L\\\!\+\\\!1hidden\-state tensors \(embedding output plusLLtransformer\-block outputs\)\. Each tensor is reduced to a single vector via*last\-token*pooling over the answer span, following the SAPLMA convention\[[2](https://arxiv.org/html/2606.02628#bib.bib3)\], yielding a representation of shape\[N,L\+1,D\]\[N,L\\\!\+\\\!1,D\]per \(model, dataset\) pair \(N=400N\\\!=\\\!400,DD= hidden dimensionality\)\.
For attention analysis we capture attention weight matrices at the first, middle, and last transformer blocks \(indices0,⌊L/2⌋\\lfloor L/2\\rfloor,L−1L\\\!\-\\\!1\)\. From each block we compute the Shannon entropy of the last\-answer\-token attention distribution per head, yielding a compact per\-head entropy vector𝐞∈ℝH\\mathbf\{e\}\\in\\mathbb\{R\}^\{H\}\.
### 3\.3Detection methods
SAPLMA\-style probes\.For each block indexℓ∈\{0,…,L\}\\ell\\in\\\{0,\\dots,L\\\}we train two probes on theDD\-dimensional hidden vector: \(a\) a*Linear*probe – a single affine logistic head; \(b\) an*MLP*probe – two hidden layers \(256→64256\\\!\\to\\\!64, GELU,0\.20\.2dropout\) followed by a softmax head\. Probes are trained with AdamW \(lr=10−3\\text\{lr\}\\\!=\\\!10^\{\-3\}, weight decay10−410^\{\-4\}\) for3030epochs with batch size128128, on a stratified70:10:2070\\\!:\\\!10\\\!:\\\!20train/val/test split selected by class\. We run each probe with33seeds and report the mean and standard deviation of AUROC, AUPRC, accuracy andF1F\_\{1\}\. The peak block and peak AUROC summarize the model’s probing score\.
INSIDE EigenScore\.Following\[[4](https://arxiv.org/html/2606.02628#bib.bib4)\], for each prompt we sampleK=10K\{=\}10stochastic continuations \(temperature0\.70\.7, top\-pp0\.950\.95,6464tokens\) and capture the last\-token hidden state at the final block for each completion\. Stacking these into𝐇∈ℝK×D\\mathbf\{H\}\\in\\mathbb\{R\}^\{K\\times D\}, the EigenScore is
EigenScore\(𝐇\)=logdet\(1D𝐗𝐗⊤\+αIK\),\\mathrm\{EigenScore\}\(\\mathbf\{H\}\)\\;=\\;\\log\\det\\\!\\bigl\(\\tfrac\{1\}\{D\}\\,\\mathbf\{X\}\\mathbf\{X\}^\{\\\!\\top\}\\\!\+\\alpha I\_\{K\}\\bigr\),\(1\)where𝐗=𝐇−𝐇¯\\mathbf\{X\}=\\mathbf\{H\}\-\\bar\{\\mathbf\{H\}\}centers the rows andα=10−3\\alpha\{=\}10^\{\-3\}\. We compute in theK×KK\\\!\\times\\\!KGram space for efficiency\. A higher EigenScore indicates greater dispersion across completions and is interpreted as evidence of hallucination\.
Self\-consistency\.For each prompt we drawK=5K\{=\}5stochastic completions with the same generation parameters and measure their agreement via \(a\) exact\-match plurality \(largest equivalence class of normalized strings divided byKK\) and \(b\) the mean pairwise cosine similarity of their sentence embeddings\[[20](https://arxiv.org/html/2606.02628#bib.bib21)\]\. The hallucination score is1−consistency1\-\\text\{consistency\}\.
Attention entropy\.For each kept attention blockℓ∈\{0,⌊L/2⌋,L−1\}\\ell\\in\\\{0,\\lfloor L/2\\rfloor,L\\\!\-\\\!1\\\}we summarize the last\-answer\-token attention distribution by its head\-averaged Shannon entropy, and treat that scalar as a continuous hallucination signal \(high entropy⇒\\Rightarrowdiffuse attention⇒\\Rightarrowunclear evidence\)\.
Metrics and layer geometry\.Probes are evaluated on the held\-out test split with AUROC, AUPRC, accuracy, andF1F\_\{1\}\(threshold0\.50\.5\)\. The remaining signals are evaluated with AUROC and AUPRC directly, plus accuracy andF1F\_\{1\}at the Youden\-JJoptimal threshold\. We additionally report classifier\-free geometric statistics per block: centroid distance, within\-class spread, and their ratio \(the*separation ratio*\), which tracks whether class separability develops as a geometric property of the representations independently of any probe\.
##### Reproducibility and compute\.
All random seeds and determinism flags are fixed \(see Appendix[0\.R](https://arxiv.org/html/2606.02628#Pt0.A18)for the complete list\)\. The full experiment runs end\-to\-end on a single RTX 5060 \(88GB\) in roughly44–88hours; the bulk of wall\-clock time is consumed by the INSIDE and self\-consistency stages, each of which requiresKKstochastic generations per prompt across all1212model–dataset configurations\.
## 4Experiments
For each of the1212model–dataset combinations we extract hidden states and attention summaries in a single forward pass, then run all four detection methods\. Results are organized into four groups: best\-block detection performance, layer\-wise trajectories, class geometry, and attention entropy\.
### 4\.1Best\-block detection performance
Table[2](https://arxiv.org/html/2606.02628#S4.T2)reports the best\-block AUROC of each method for every model–dataset pair\. For probes, “best block” is the layerargmaxℓAUROC¯ℓ\\mathrm\{argmax\}\_\{\\ell\}\\,\\overline\{\\mathrm\{AUROC\}\}\_\{\\ell\}across three seeds \(ℓ∈\{0,…,L\}\\ell\\in\\\{0,\\dots,L\\\}, where0is the embedding output andLLis the final transformer block\); for INSIDE and self\-consistency, a single score per prompt is produced; for attention entropy we report the best of the three captured blocks \(first, middle, last\)\.
Table 2:Best\-block AUROC per \(model, dataset\) cell\.Linear/MLP: SAPLMA\-style probes; parenthetical numbers are the best block index \(0= embedding,LL= final block\)\.INSIDE: EigenScore at the final block \(K=10K\{=\}10\)\.SC: self\-consistency \(K=5K\{=\}5\), exact\-match or embedding\.Attn: head\-averaged last\-token attention entropy at the best of three blocks\. Bold marks the best probe per row\. Probe values are means over three seeds \(std≤0\.025\\leq\\\!0\.025\)\. Llama = Llama\-3\.1\-8B, Mistral = Mistral\-7B, Qwen = Qwen2\.5\-7B\.Three findings emerge from Table[2](https://arxiv.org/html/2606.02628#S4.T2):
\(F1\) Probes yield higher AUROC than sampling\-based detectors in this setting\.The best probe AUROC exceeds the best sampling\-based AUROC by at least0\.3670\.367in every cell \(median gap0\.4530\.453\)\. On HaluEval\-QA all probes reach≥0\.997\\geq\\\!0\.997; on TruthfulQA, the most challenging of our four benchmarks, probes attain0\.9040\.904–0\.9250\.925\.
\(F2\) MLP≈\\approxLinear\.The MLP probe rarely exceeds the linear probe by more than0\.010\.01AUROC: the absolute MLP–Linear difference is≤0\.005\{\\leq\}\\,0\.005in1010of1212cells and≤0\.01\{\\leq\}\\,0\.01in all1212; the largest gap is\+0\.010\+0\.010\(TruthfulQA, Qwen2\.5\)\. This suggests that the truthfulness signal is approximately*linear*in the hidden\-state space, consistent with\[[2](https://arxiv.org/html/2606.02628#bib.bib3),[3](https://arxiv.org/html/2606.02628#bib.bib17)\]\.
\(F3\) INSIDE and self\-consistency are near chance under this protocol\.The INSIDE EigenScore lies in\[0\.433,0\.529\]\[0\.433,\\,0\.529\]across all1212cells; exact\-match self\-consistency in\[0\.467,0\.505\]\[0\.467,\\,0\.505\]; and embedding self\-consistency in\[0\.425,0\.541\]\[0\.425,\\,0\.541\]\. No sampling method exceeds0\.550\.55AUROC on any cell\. We attribute this to our evaluation protocol \(§[3\.1](https://arxiv.org/html/2606.02628#S3.SS1)\), discussed further in §[4\.5](https://arxiv.org/html/2606.02628#S4.SS5)and §[5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px2)\. Figure[1](https://arxiv.org/html/2606.02628#S4.F1)visualizes the pattern on HaluEval\-QA: probes dominate, attention entropy is second, and sampling variants cluster at chance\.
Figure 1:Method comparison on HaluEval\-QA\. Probes reach≥0\.997\\geq\\\!0\.997AUROC on all three models; attention entropy at the first block yields0\.8660\.866–0\.9410\.941AUROC at no extra inference cost; INSIDE and self\-consistency are at chance\.Tables[3](https://arxiv.org/html/2606.02628#S4.T3)and[4](https://arxiv.org/html/2606.02628#S4.T4)confirm the same pattern with accuracy andF1F\_\{1\}: probes achieve≥0\.800\\geq\\\!0\.800accuracy on every natural\-language benchmark; sampling methods remain near chance; attention entropy is again the exception on HaluEval\-QA \(0\.8030\.803–0\.8630\.863accuracy\)\.
Table 3:Accuracy at the best block\. Probe threshold is0\.50\.5; other methods use the Youden\-JJoptimal threshold\. Bold marks the best probe variant per row\.Table 4:F1F\_\{1\}at the best block\. Bold marks the best probe variant per row\. SCexvalues are unstable because exact\-match self\-consistency often assigns identical scores to all samples, yielding degenerateF1F\_\{1\}\.
### 4\.2Layer\-wise probing trajectories
Probing performance varies smoothly with depth \(Figure[2](https://arxiv.org/html/2606.02628#S4.F2)\)\. For Llama\-3\.1\-8B on TruthfulQA, the MLP\-probe AUROC peaks near block1414\(≈0\.914\\approx\\\!0\.914, std<0\.005<\\\!0\.005\)\. The heatmap \(right panel\) shows the peak band lies in40%40\\%–90%90\\%of network depth across models; all1212cells exhibit qualitatively similar trajectories \(Appendix[0\.K](https://arxiv.org/html/2606.02628#Pt0.A11)\)\.
Figure 2:*Left*: Layer\-wise MLP\-probe AUROC for Llama\-3\.1\-8B on TruthfulQA; the signal peaks near block1414\(shaded band: std over three seeds\)\.*Right*: Per\-block MLP\-probe AUROC heatmap across all three models on TruthfulQA; the peak band is consistently in the second half of the network\.This pattern is consistent across all four datasets \(Appendix[0\.B](https://arxiv.org/html/2606.02628#Pt0.A2)\)\. On natural\-language benchmarks the MLP\-probe peak blocks for Llama, Mistral, and Qwen are\{14,16,19\}\\\{14,16,19\\\}on TruthfulQA,\{15,18,25\}\\\{15,18,25\\\}on HaluEval\-QA, and\{17,15,21\}\\\{17,15,21\\\}on FEVER \(Table[2](https://arxiv.org/html/2606.02628#S4.T2)\)\. On the synthetic benchmark the peak shifts later \(\{28,20,18\}\\\{28,20,18\\\}\) because the AUROC plateau reaches1\.0001\.000across several late blocks, making the argmax arbitrary\.
### 4\.3Class geometry through depth
Probing performance mirrors the underlying hidden\-state geometry\. Figure[3](https://arxiv.org/html/2606.02628#S4.F3)\(left\) shows the separation ratio for Llama\-3\.1\-8B on TruthfulQA: it increases through early blocks and plateaus where probes peak, confirming that class separability is a geometric property of the representations\. The t\-SNE projection at the probing peak \(right\) shows visually distinct clusters\. This behavior is reproduced across all models and datasets \(Appendix[0\.D](https://arxiv.org/html/2606.02628#Pt0.A4),[0\.L](https://arxiv.org/html/2606.02628#Pt0.A12)\)\.
Figure 3:*Left*: Layer\-wise class geometry for Llama\-3\.1\-8B on TruthfulQA showing centroid distance, within\-class spread, and their ratio; the separation ratio plateaus in the second half of the network, mirroring probe AUROC \(Figure[2](https://arxiv.org/html/2606.02628#S4.F2)\)\.*Right*: t\-SNE projection of layer\-1414hidden states; truthful \(blue\) and hallucinated \(red\) samples form distinct clusters, consistent with0\.9140\.914AUROC\[[22](https://arxiv.org/html/2606.02628#bib.bib20)\]\.
### 4\.4Attention entropy as an auxiliary signal
Figure[4](https://arxiv.org/html/2606.02628#S4.F4)\(left\) shows the head\-averaged last\-token attention entropy on HaluEval\-QA for Llama\-3\.1\-8B\. At the first block, hallucinated items produce higher entropy \(AUROC0\.9410\.941\): when the supplied knowledge supports the answer, attention concentrates on the relevant tokens\.
Figure 4:*Left*: Last\-token attention entropy on Llama\-3\.1\-8B / HaluEval\-QA at three transformer blocks; the first block produces the best class separation \(AUROC0\.9410\.941\)\.*Right*: INSIDE EigenScore distribution on Llama\-3\.1\-8B / TruthfulQA; the truthful and hallucinated histograms overlap almost completely \(AUROC0\.4330\.433\)\.This effect is robust across models on HaluEval\-QA \(AUROC0\.8660\.866–0\.9410\.941; Appendix[0\.F](https://arxiv.org/html/2606.02628#Pt0.A6)\) but does not generalize to datasets lacking an explicit evidence passage \(Appendix[0\.N](https://arxiv.org/html/2606.02628#Pt0.A14)\)\. For retrieval\-augmented or knowledge\-conditioned settings, first\-block attention entropy requires no additional forward pass and complements a mid\-network probe at no extra cost\.
### 4\.5Why sampling\-based methods underperform
As shown in Figure[4](https://arxiv.org/html/2606.02628#S4.F4)\(right\), the per\-class INSIDE EigenScore distributions are nearly indistinguishable on TruthfulQA; the same holds across all configurations \(Appendix[0\.E](https://arxiv.org/html/2606.02628#Pt0.A5),[0\.M](https://arxiv.org/html/2606.02628#Pt0.A13)\)\. Exact\-match self\-consistency is effectively0\.500\.50on11/1211/12cells, as stochastic completions rarely produce identical strings\. Embedding self\-consistency never exceeds0\.5410\.541AUROC, consistent with prior findings that simple agreement metrics are insufficient without semantic\-entropy clustering\[[13](https://arxiv.org/html/2606.02628#bib.bib5),[8](https://arxiv.org/html/2606.02628#bib.bib6)\]\. This outcome is inherent to our paired\-label protocol \(§[3\.1](https://arxiv.org/html/2606.02628#S3.SS1), §[5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px2)\)\.
## 5Discussion
##### Why probes are strong\.
Three properties of our experimental setting contribute to the high probe AUROC observed\. First, our evaluation protocol \(§[3\.1](https://arxiv.org/html/2606.02628#S3.SS1)\) passes each candidate answer through the network, giving the probe access to the model’s internal representation of that specific answer\. Second, balanced pairing of truthful and hallucinated answers for every prompt yields clean binary supervision with low variance \(≤0\.025\\leq\\\!0\.025AUROC across seeds\)\. Third, instruction\-tuned chat models\[[18](https://arxiv.org/html/2606.02628#bib.bib23),[9](https://arxiv.org/html/2606.02628#bib.bib12),[12](https://arxiv.org/html/2606.02628#bib.bib13),[19](https://arxiv.org/html/2606.02628#bib.bib14)\]may develop more linearly decodable truthfulness directions than base models, though we leave this comparison to future work\.
##### Why sampling methods are weak here\.
As detailed in §[3\.1](https://arxiv.org/html/2606.02628#S3.SS1)and confirmed empirically in §[4\.5](https://arxiv.org/html/2606.02628#S4.SS5), sampling\-based and probe\-based detectors answer*different questions*: probes measure whether the network internally distinguishes a given answer as truthful, while sampling methods measure the model’s generative uncertainty about the prompt\. Under paired\-label evaluation, probes have a structural advantage; in generate\-then\-judge settings, sampling approaches would be expected to recover more of their reported effectiveness\. This distinction is relevant for future benchmark design\.
##### Probing remains effective under quantization\.
The peak AUROC values \(0\.9980\.998on HaluEval\-QA,1\.0001\.000on the synthetic benchmark\) are obtained entirely under44\-bit quantization and are comparable to full\-precision results reported on similar models\[[2](https://arxiv.org/html/2606.02628#bib.bib3)\]\. While we do not include a direct full\-precision baseline, this comparison suggests that hidden\-state detectors can be deployed at the same inference cost as the chat model itself, without requiring de\-quantization\.
##### Where to attach the probe\.
The MLP\-probe peak falls in a consistent band: blocks1414–1818for Llama and Mistral \(4444–56%56\\%of depth\) and blocks1919–2525for Qwen \(6868–89%89\\%\)\. In practice, a single fixed block \(e\.g\.,≈15\{\\approx\}15for Llama,≈16\{\\approx\}16for Mistral,≈20\{\\approx\}20for Qwen\) would incur at most0\.020\.02AUROC loss relative to the per\-dataset optimum\. On the synthetic benchmark the AUROC plateau is close enough to1\.0001\.000that any block in the second half of the network suffices\.
## 6Limitations
Our study is restricted to77B–88B chat models under44\-bit quantization, evaluated on four English\-language benchmarks of400400items each; results may differ for larger or base models, non\-English languages, and free\-form generations without benchmark\-supplied candidate answers\. INSIDE uses final\-block hidden states withK=10K\{=\}10samples and self\-consistency usesK=5K\{=\}5; a budget sweep, intermediate\-layer EigenScores, or semantic\-entropy clustering might recover some discriminative power\. The fixed sampling temperature \(0\.70\.7\) and top\-pp\(0\.950\.95\) may underestimate the dispersion these methods rely on\. Attention entropy is captured at only three blocks; a denser sweep might reveal stronger intermediate\-block signals\. Finally, all probes are trained and tested on the same dataset; cross\-dataset transfer is an important next step\.
## 7Conclusion
We presented a unified evaluation of four hallucination\-detection methods on three77B–88B quantized chat models, reproducible on a single consumer GPU\. Under the paired\-label protocol, a linear probe on a single mid\-network hidden state achieves0\.9040\.904–1\.0001\.000AUROC across three model families and four datasets, while sampling\-based detectors do not exceed0\.5410\.541\. The peak probing block is stable across architectures on natural\-language benchmarks, and the separation it captures is corroborated by classifier\-free geometric analysis\. First\-block attention entropy provides a complementary signal at no additional inference cost in knowledge\-grounded settings\. These results suggest that white\-box probing is both more effective and less expensive than sampling\-based detection in this regime\. The observed weakness of sampling methods reflects the structural mismatch inherent to paired\-label evaluation rather than a limitation of those methods in general\.
##### Data, code, and supplementary material\.
All source code, trained probe weights, and evaluation scripts are publicly available at[https://github\.com/Ezharjan/HallucinationPatternDetection](https://github.com/Ezharjan/HallucinationPatternDetection)\. The repository includes the complete pipeline, pre\-computed results, and the eighteen supplementary appendices referenced above\.\{credits\}
#### 7\.0\.1\\discintname
The authors have no competing interests to declare that are relevant to the content of this article\.
## References
- \[1\]G\. Alain and Y\. Bengio\(2016\)Understanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p2.1)\.
- \[2\]A\. Azaria and T\. Mitchell\(2023\)The internal state of an llm knows when it’s lying\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 967–976\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§2](https://arxiv.org/html/2606.02628#S2.p2.1),[§3\.1](https://arxiv.org/html/2606.02628#S3.SS1.p1.1),[§3\.2](https://arxiv.org/html/2606.02628#S3.SS2.p1.6),[§4\.1](https://arxiv.org/html/2606.02628#S4.SS1.p4.8),[§5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px3.p1.3)\.
- \[3\]C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt\(2022\)Discovering latent knowledge in language models without supervision\.arXiv preprint arXiv:2212\.03827\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p2.1),[§4\.1](https://arxiv.org/html/2606.02628#S4.SS1.p4.8)\.
- \[4\]C\. Chen, K\. Liu, Z\. Chen, Y\. Gu, Y\. Wu, M\. Tao, Z\. Fu, and J\. Ye\(2024\)INSIDE: llms’ internal states retain the power of hallucination detection\.arXiv preprint arXiv:2402\.03744\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§2](https://arxiv.org/html/2606.02628#S2.p3.1),[§3\.3](https://arxiv.org/html/2606.02628#S3.SS3.p2.6)\.
- \[5\]R\. Cohen, M\. Hamri, M\. Geva, and A\. Globerson\(2023\)Lm vs lm: detecting factual errors via cross examination\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 12621–12640\.Cited by:[§2](https://arxiv.org/html/2606.02628#S2.p4.1)\.
- \[6\]T\. Dettmers, M\. Lewis, Y\. Belkada, and L\. Zettlemoyer\(2022\)Gpt3\. int8 \(\): 8\-bit matrix multiplication for transformers at scale\.Advances in neural information processing systems35,pp\. 30318–30332\.Cited by:[§2](https://arxiv.org/html/2606.02628#S2.p5.7)\.
- \[7\]T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. Zettlemoyer\(2023\)Qlora: efficient finetuning of quantized llms\.Advances in neural information processing systems36,pp\. 10088–10115\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§2](https://arxiv.org/html/2606.02628#S2.p5.7),[§3](https://arxiv.org/html/2606.02628#S3.p1.3)\.
- \[8\]S\. Farquhar, J\. Kossen, L\. Kuhn, and Y\. Gal\(2024\)Detecting hallucinations in large language models using semantic entropy\.Nature630\(8017\),pp\. 625–630\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p3.1),[§4\.5](https://arxiv.org/html/2606.02628#S4.SS5.p1.3)\.
- \[9\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px1.p1.1)\.
- \[10\]L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin,et al\.\(2025\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.ACM Transactions on Information Systems43\(2\),pp\. 1–55\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p1.1)\.
- \[11\]Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, E\. Ishii, Y\. J\. Bang, A\. Madotto, and P\. Fung\(2023\)Survey of hallucination in natural language generation\.ACM computing surveys55\(12\),pp\. 1–38\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p1.1)\.
- \[12\]A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. Le Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. El Sayed\(2023\)Mistral 7B\.arXiv preprint arXiv:2310\.06825\.External Links:[Link](https://arxiv.org/abs/2310.06825)Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px1.p1.1)\.
- \[13\]L\. Kuhn, Y\. Gal, and S\. Farquhar\(2023\)Semantic uncertainty: linguistic invariances for uncertainty estimation in natural language generation\.arXiv preprint arXiv:2302\.09664\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p3.1),[§4\.5](https://arxiv.org/html/2606.02628#S4.SS5.p1.3)\.
- \[14\]J\. Li, X\. Cheng, X\. Zhao, J\. Nie, and J\. Wen\(2023\)Halueval: a large\-scale hallucination evaluation benchmark for large language models\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 6449–6464\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§3](https://arxiv.org/html/2606.02628#S3.p4.2)\.
- \[15\]K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg\(2023\)Inference\-time intervention: eliciting truthful answers from a language model\.Advances in Neural Information Processing Systems36,pp\. 41451–41530\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§2](https://arxiv.org/html/2606.02628#S2.p2.1)\.
- \[16\]S\. Lin, J\. Hilton, and O\. Evans\(2022\)Truthfulqa: measuring how models mimic human falsehoods\.InProceedings of the 60th annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 3214–3252\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§3](https://arxiv.org/html/2606.02628#S3.p3.1)\.
- \[17\]P\. Manakul, A\. Liusie, and M\. Gales\(2023\)Selfcheckgpt: zero\-resource black\-box hallucination detection for generative large language models\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 9004–9017\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§2](https://arxiv.org/html/2606.02628#S2.p3.1)\.
- \[18\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px1.p1.1)\.
- \[19\]Qwen Team\(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.External Links:[Link](https://arxiv.org/abs/2412.15115)Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§5](https://arxiv.org/html/2606.02628#S5.SS0.SSS0.Px1.p1.1)\.
- \[20\]N\. Reimers and I\. Gurevych\(2019\)Sentence\-bert: sentence embeddings using siamese bert\-networks\.InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\),pp\. 3982–3992\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§3\.3](https://arxiv.org/html/2606.02628#S3.SS3.p3.3)\.
- \[21\]J\. Thorne, A\. Vlachos, C\. Christodoulopoulos, and A\. Mittal\(2018\)FEVER: a large\-scale dataset for fact extraction and verification\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),pp\. 809–819\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§3](https://arxiv.org/html/2606.02628#S3.p5.2)\.
- \[22\]L\. Van der Maaten and G\. Hinton\(2008\)Visualizing data using t\-sne\.\.Journal of machine learning research9\(11\)\.Cited by:[Figure 3](https://arxiv.org/html/2606.02628#S4.F3)\.
- \[23\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§1](https://arxiv.org/html/2606.02628#S1.p1.1),[§1](https://arxiv.org/html/2606.02628#S1.p2.4),[§2](https://arxiv.org/html/2606.02628#S2.p3.1)\.
## Appendix Roadmap
The eighteen appendices that follow provide exhaustive supporting evidence for every main\-text claim\. The roadmap below maps each appendix to its primary function and lists the figures it contains, so that every one of the2828appendix figures and the appendix hyperparameter table \(Table[5](https://arxiv.org/html/2606.02628#Pt0.A18.T5)\) can be located by cross\-reference\.
- •Appendix[0\.A](https://arxiv.org/html/2606.02628#Pt0.A1)extends the method\-comparison bar plot \(main\-text Figure[1](https://arxiv.org/html/2606.02628#S4.F1)\) to the remaining three datasets \(Figure[5](https://arxiv.org/html/2606.02628#Pt0.A1.F5): TruthfulQA, FEVER, Synthetic\), supporting Finding F1 \(§[4\.1](https://arxiv.org/html/2606.02628#S4.SS1)\)\.
- •Appendix[0\.B](https://arxiv.org/html/2606.02628#Pt0.A2)provides MLP\-probe per\-block AUROC heatmaps on HaluEval\-QA, FEVER and the synthetic benchmark \(Figure[6](https://arxiv.org/html/2606.02628#Pt0.A2.F6)\), complementing Figure[2](https://arxiv.org/html/2606.02628#S4.F2)and supporting the peak\-band consistency claim of §[4\.2](https://arxiv.org/html/2606.02628#S4.SS2)\.
- •Appendix[0\.C](https://arxiv.org/html/2606.02628#Pt0.A3)shows layer\-AUROC trajectories for Mistral and Qwen on TruthfulQA \(Figure[7](https://arxiv.org/html/2606.02628#Pt0.A3.F7)\) and Llama on HaluEval\-QA \(Figure[8](https://arxiv.org/html/2606.02628#Pt0.A3.F8)\), extending Figure[2](https://arxiv.org/html/2606.02628#S4.F2)\.
- •Appendix[0\.D](https://arxiv.org/html/2606.02628#Pt0.A4)shows class\-separation diagnostics for Mistral and Qwen on TruthfulQA \(Figure[9](https://arxiv.org/html/2606.02628#Pt0.A4.F9)\), extending Figure[3](https://arxiv.org/html/2606.02628#S4.F3)\.
- •Appendix[0\.E](https://arxiv.org/html/2606.02628#Pt0.A5)shows INSIDE EigenScore distributions for Mistral and Qwen on TruthfulQA \(Figure[10](https://arxiv.org/html/2606.02628#Pt0.A5.F10)\), extending Figure[4](https://arxiv.org/html/2606.02628#S4.F4)and supporting Finding F3\.
- •Appendix[0\.F](https://arxiv.org/html/2606.02628#Pt0.A6)shows attention\-entropy distributions for Mistral and Qwen on HaluEval\-QA \(Figure[11](https://arxiv.org/html/2606.02628#Pt0.A6.F11)\), extending Figure[4](https://arxiv.org/html/2606.02628#S4.F4)and supporting the attention\-entropy result of §[4\.4](https://arxiv.org/html/2606.02628#S4.SS4)\.
- •Appendix[0\.G](https://arxiv.org/html/2606.02628#Pt0.A7)provides side\-by\-side PCA projections at the probing peak for Llama and Qwen \(Figure[12](https://arxiv.org/html/2606.02628#Pt0.A7.F12)\), complementing the t\-SNE visualization in Figure[3](https://arxiv.org/html/2606.02628#S4.F3)\.
- •Appendix[0\.H](https://arxiv.org/html/2606.02628#Pt0.A8)provides the Qwen t\-SNE projection at its probing peak \(Figure[13](https://arxiv.org/html/2606.02628#Pt0.A8.F13)\), extending Figure[3](https://arxiv.org/html/2606.02628#S4.F3)\.
- •Appendix[0\.I](https://arxiv.org/html/2606.02628#Pt0.A9)describes the synthetic benchmark construction \(no figures\)\.
- •Appendix[0\.J](https://arxiv.org/html/2606.02628#Pt0.A10)provides per\-block AUROC heatmaps for the*linear*probe across all four datasets \(Figure[14](https://arxiv.org/html/2606.02628#Pt0.A10.F14)\), supporting Finding F2 \(§[4\.1](https://arxiv.org/html/2606.02628#S4.SS1)\)\.
- •Appendix[0\.K](https://arxiv.org/html/2606.02628#Pt0.A11)reports layer\-AUROC trajectories for all1212configurations: Llama \(Figure[15](https://arxiv.org/html/2606.02628#Pt0.A11.F15)\), Mistral \(Figure[16](https://arxiv.org/html/2606.02628#Pt0.A11.F16)\), Qwen \(Figure[17](https://arxiv.org/html/2606.02628#Pt0.A11.F17)\)\.
- •Appendix[0\.L](https://arxiv.org/html/2606.02628#Pt0.A12)reports class\-geometry diagnostics for all1212configurations: Llama \(Figure[18](https://arxiv.org/html/2606.02628#Pt0.A12.F18)\), Mistral \(Figure[19](https://arxiv.org/html/2606.02628#Pt0.A12.F19)\), Qwen \(Figure[20](https://arxiv.org/html/2606.02628#Pt0.A12.F20)\)\.
- •Appendix[0\.M](https://arxiv.org/html/2606.02628#Pt0.A13)reports INSIDE EigenScore histograms for all1212configurations: Llama \(Figure[21](https://arxiv.org/html/2606.02628#Pt0.A13.F21)\), Mistral \(Figure[22](https://arxiv.org/html/2606.02628#Pt0.A13.F22)\), Qwen \(Figure[23](https://arxiv.org/html/2606.02628#Pt0.A13.F23)\)\.
- •Appendix[0\.N](https://arxiv.org/html/2606.02628#Pt0.A14)reports attention\-entropy distributions on TruthfulQA \(Figure[24](https://arxiv.org/html/2606.02628#Pt0.A14.F24)\), FEVER \(Figure[25](https://arxiv.org/html/2606.02628#Pt0.A14.F25)\), and the synthetic benchmark \(Figure[26](https://arxiv.org/html/2606.02628#Pt0.A14.F26)\) for all three models, substantiating the claim that the first\-block signal does not generalize beyond knowledge\-grounded settings\.
- •Appendix[0\.O](https://arxiv.org/html/2606.02628#Pt0.A15)provides 2\-D PCA and t\-SNE projections for all1212configurations: Llama \(Figure[27](https://arxiv.org/html/2606.02628#Pt0.A15.F27)\), Mistral \(Figure[28](https://arxiv.org/html/2606.02628#Pt0.A15.F28)\), Qwen \(Figure[29](https://arxiv.org/html/2606.02628#Pt0.A15.F29)\)\.
- •Appendix[0\.P](https://arxiv.org/html/2606.02628#Pt0.A16)provides thresholding details for the accuracy andF1F\_\{1\}tables \(Tables[3](https://arxiv.org/html/2606.02628#S4.T3)and[4](https://arxiv.org/html/2606.02628#S4.T4)\) presented in the main text \(§[4\.1](https://arxiv.org/html/2606.02628#S4.SS1)\)\.
- •Appendix[0\.Q](https://arxiv.org/html/2606.02628#Pt0.A17)reports per\-block*linear*\-probe AUROC trajectories for all1212configurations: Llama \(Figure[30](https://arxiv.org/html/2606.02628#Pt0.A17.F30)\), Mistral \(Figure[31](https://arxiv.org/html/2606.02628#Pt0.A17.F31)\), Qwen \(Figure[32](https://arxiv.org/html/2606.02628#Pt0.A17.F32)\), supporting Finding F2\.
- •Appendix[0\.R](https://arxiv.org/html/2606.02628#Pt0.A18)consolidates all hyperparameters in Table[5](https://arxiv.org/html/2606.02628#Pt0.A18.T5)\.
## Appendix 0\.APer\-Dataset Method Comparisons
For completeness, Figure[5](https://arxiv.org/html/2606.02628#Pt0.A1.F5)reproduces the method\-comparison bar plot for all four datasets\. The pattern visible on HaluEval\-QA \(Figure[1](https://arxiv.org/html/2606.02628#S4.F1)\) holds qualitatively: probes are uniformly at the top, attention entropy is a competitive second on knowledge\-grounded settings, and the sampling\-based detectors cluster at chance\.
Figure 5:Per\-dataset method\-comparison bar plots for TruthfulQA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\)\. Probes are uniformly at the top, attention entropy is competitive on knowledge\-grounded settings, and sampling\-based detectors cluster at chance\. Cf\. Figure[1](https://arxiv.org/html/2606.02628#S4.F1)for HaluEval\-QA\.
## Appendix 0\.BCross\-Dataset Heatmaps
Figure[6](https://arxiv.org/html/2606.02628#Pt0.A2.F6)shows the MLP\-probe per\-block AUROC heatmaps on the remaining three datasets, complementing Figure[2](https://arxiv.org/html/2606.02628#S4.F2)on TruthfulQA\. The bright peak band is consistently in the second half of the network for every configuration\. The HaluEval\-QA peaks are the sharpest, saturating at AUROC≈0\.998\\approx\\\!0\.998over a wide mid\-to\-late plateau, while the synthetic benchmark saturates at AUROC≈1\.000\\approx\\\!1\.000in flat regions spanning several late blocks\.
Figure 6:MLP\-probe per\-block AUROC heatmaps on HaluEval\-QA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\)\. Rows within each heatmap correspond to the three models; columns are block indices; color encodes AUROC\. The bright peak band is consistently in the second half of the network\. Cf\. Figure[2](https://arxiv.org/html/2606.02628#S4.F2)for TruthfulQA\.
## Appendix 0\.CCross\-Model Layer\-AUROC Trajectories
Figure[7](https://arxiv.org/html/2606.02628#Pt0.A3.F7)reproduces the per\-block MLP\-probe AUROC curves of Mistral\-7B and Qwen2\.5\-7B on TruthfulQA, complementing Figure[2](https://arxiv.org/html/2606.02628#S4.F2)for Llama\-3\.1\-8B\. The three curves share the same qualitative shape \(a monotonic rise, a plateau in the second half of the network, and a small decay near the output\), even though the absolute heights and the location of the plateau differ slightly across model families\.
Figure 7:Per\-block MLP\-probe AUROC on TruthfulQA for Mistral\-7B \(top\) and Qwen2\.5\-7B \(bottom\)\. Shaded bands show standard deviation over three seeds\. Both models exhibit the same qualitative rise\-plateau\-decay pattern as Llama\-3\.1\-8B \(Figure[2](https://arxiv.org/html/2606.02628#S4.F2)\)\.For an even cleaner case, Figure[8](https://arxiv.org/html/2606.02628#Pt0.A3.F8)shows the Llama\-3\.1\-8B per\-block MLP\-probe AUROC on HaluEval\-QA: the signal saturates at≈0\.998\\approx\\\!0\.998over the entire middle of the network, with seed standard deviation below0\.0050\.005\.
Figure 8:Per\-block MLP\-probe AUROC for Llama\-3\.1\-8B on HaluEval\-QA\. The plateau saturates at AUROC≈0\.998\{\\approx\}0\.998across blocks∼12\{\\sim\}12–2424, with seed standard deviation below0\.0050\.005\.
## Appendix 0\.DCross\-Model Class Geometry
Figure[9](https://arxiv.org/html/2606.02628#Pt0.A4.F9)reproduces the class\-geometry diagnostic of Figure[3](https://arxiv.org/html/2606.02628#S4.F3)for Mistral\-7B and Qwen2\.5\-7B on TruthfulQA\. The separation ratio increases monotonically through the early blocks and plateaus in the second half of the network for both models, consistent with the corresponding probe\-AUROC trajectories\.
Figure 9:Layer\-wise class geometry on TruthfulQA for Mistral\-7B \(top\) and Qwen2\.5\-7B \(bottom\), showing centroid distance, within\-class spread, and separation ratio\. The separation ratio rises through the early blocks and saturates in the mid\-to\-late blocks, mirroring the probe\-AUROC trajectories \(Figure[7](https://arxiv.org/html/2606.02628#Pt0.A3.F7)\)\.
## Appendix 0\.ECross\-Model INSIDE Score Distributions
Figure[10](https://arxiv.org/html/2606.02628#Pt0.A5.F10)reproduces the per\-class INSIDE EigenScore distribution on TruthfulQA for Mistral\-7B and Qwen2\.5\-7B, complementing Figure[4](https://arxiv.org/html/2606.02628#S4.F4)\. The near\-complete overlap of the truthful and hallucinated distributions persists across all three model families, consistent with the chance\-level AUROC\.
Figure 10:INSIDE EigenScore distributions on TruthfulQA for Mistral\-7B \(top\) and Qwen2\.5\-7B \(bottom\), separated by ground\-truth label\. The near\-complete overlap of the truthful and hallucinated histograms explains the chance\-level AUROC reported in Table[2](https://arxiv.org/html/2606.02628#S4.T2)\.
## Appendix 0\.FCross\-Model Attention Entropy on HaluEval\-QA
Figure[11](https://arxiv.org/html/2606.02628#Pt0.A6.F11)shows the head\-averaged last\-token attention\-entropy distributions at three transformer blocks for Mistral\-7B and Qwen2\.5\-7B on HaluEval\-QA\. The first\-block separation observed for Llama \(Figure[4](https://arxiv.org/html/2606.02628#S4.F4)\) is preserved in both models \(AUROC0\.9020\.902and0\.8660\.866respectively\), confirming that first\-block attention entropy provides a useful signal at no additional inference cost in the knowledge\-conditioned setting\.
Figure 11:Last\-token attention entropy on HaluEval\-QA for Mistral\-7B \(top\) and Qwen2\.5\-7B \(bottom\) at the first, middle, and last captured transformer blocks\. The first\-block separation \(AUROC0\.9020\.902and0\.8660\.866respectively\) confirms the pattern observed for Llama\-3\.1\-8B in Figure[4](https://arxiv.org/html/2606.02628#S4.F4)\.
## Appendix 0\.G2\-D Projection \(PCA\) at the Probing Peak
Figure[12](https://arxiv.org/html/2606.02628#Pt0.A7.F12)shows side\-by\-side PCA projections of the hidden states at the probing peak for Llama\-3\.1\-8B \(block1414\) and Qwen2\.5\-7B \(block1919\) on TruthfulQA\. Unlike t\-SNE, PCA is a linear projection and therefore mirrors what the linear probe sees\. Both models show clear class separation along the leading principal directions, consistent with the0\.9140\.914/0\.9150\.915linear\-probe AUROC at these blocks\.
Figure 12:2\-D PCA projections on TruthfulQA at each model’s probing peak\.*Left*: Llama\-3\.1\-8B at block1414\.*Right*: Qwen2\.5\-7B at block1919\. Both models show clear class separation along the leading principal components, consistent with the high linear\-probe AUROC\. Cf\. Figure[3](https://arxiv.org/html/2606.02628#S4.F3)for the non\-linear t\-SNE counterpart\.
## Appendix 0\.HCross\-Model t\-SNE at the Probing Peak \(Qwen\)
Figure[13](https://arxiv.org/html/2606.02628#Pt0.A8.F13)shows the t\-SNE projection of the layer\-1919hidden states of Qwen2\.5\-7B on TruthfulQA, the block at which the Qwen MLP probe achieves its peak AUROC of0\.9250\.925\(Table[2](https://arxiv.org/html/2606.02628#S4.T2)\)\. As for Llama\-3\.1 at its probing peak \(Figure[3](https://arxiv.org/html/2606.02628#S4.F3)\), the two classes form distinct clusters with a narrow margin region\. The t\-SNE visualization uses perplexity3030\(see Appendix[0\.R](https://arxiv.org/html/2606.02628#Pt0.A18)\) and confirms that the linear separability observed by the probe corresponds to a genuine geometric separation in hidden\-state space, not an artifact of the linear classifier\.
Figure 13:t\-SNE projection of the layer\-1919hidden states of Qwen2\.5\-7B on TruthfulQA \(Qwen probing peak, perplexity3030\)\. The two classes form distinct clusters with a thin margin region, confirming the geometric separation that underlies the0\.9250\.925MLP\-probe AUROC\.
## Appendix 0\.ISynthetic Benchmark Construction
The synthetic benchmark is generated locally from four small knowledge banks: world capitals \(3030countries\), chemical element symbols \(2020elements\), literary authorship \(2020works\), and planetary orbits \(88planets\)\. For each underlying fact we emit two items, one truthful \(label11\) and one hallucinated \(label0\), sharing the same prompt and differing only in the substantive token of the answer \(e\.g\.The capital of France is→\\toParisvs\.Berlin\)\. Items are shuffled with a fixed seed before the400400\-item cap is applied; the resulting class balance is exactly200:200200\{:\}200\. This yields the simplest binary signal in our suite, corresponding to the≥0\.998\\geq\\\!0\.998probing AUROC plateau in Table[2](https://arxiv.org/html/2606.02628#S4.T2)\.
## Appendix 0\.JLinear\-Probe Cross\-Dataset Heatmaps
Figure[14](https://arxiv.org/html/2606.02628#Pt0.A10.F14)reproduces the per\-block AUROC heatmaps of thelinearSAPLMA probe across all four datasets, complementing the MLP heatmaps in Figure[2](https://arxiv.org/html/2606.02628#S4.F2)and Figure[6](https://arxiv.org/html/2606.02628#Pt0.A2.F6)\. The linear and MLP heatmaps are visually almost indistinguishable on every dataset, reflecting the near\-equality of the two probe variants \(F2 in §[4\.1](https://arxiv.org/html/2606.02628#S4.SS1),\|ΔAUROC\|≤0\.01\|\\Delta\\mathrm\{AUROC\}\|\\\!\\leq\\\!0\.01in all1212cells\)\. This provides further evidence that the truthfulness signal in hidden\-state space is effectively linear\.
Figure 14:Per\-block AUROC heatmaps for the*linear*SAPLMA probe across all four datasets\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Rows within each heatmap are the three models; columns are block indices; color is AUROC\. The pattern is qualitatively identical to the MLP counterparts \(Figures[2](https://arxiv.org/html/2606.02628#S4.F2),[6](https://arxiv.org/html/2606.02628#Pt0.A2.F6)\)\.
## Appendix 0\.KLayer\-Wise AUROC Trajectories Across All Cells
Figures[15](https://arxiv.org/html/2606.02628#Pt0.A11.F15)–[17](https://arxiv.org/html/2606.02628#Pt0.A11.F17)report the per\-block MLP\-probe AUROC curves for all1212model–dataset combinations\. Within each model the qualitative shape is preserved across datasets: a monotonic rise through the early blocks, a high\-AUROC plateau in the middle\-to\-late blocks, and a small decay at the output\. The location and height of the plateau vary modestly across datasets \(TruthfulQA peaks lower at≈0\.91\\approx\\\!0\.91–0\.930\.93, while HaluEval\-QA, FEVER and synthetic peak above0\.950\.95and often saturate\)\. The linear\-probe counterparts are visually indistinguishable from the MLP curves shown here \(see Appendix[0\.Q](https://arxiv.org/html/2606.02628#Pt0.A17)for the complete set of linear\-probe layer\-AUROC trajectories across all1212cells\), and the equality of linear and MLP curves is exactly the F2 finding of §[4\.1](https://arxiv.org/html/2606.02628#S4.SS1)\.
Figure 15:Per\-block MLP\-probe AUROC for Llama\-3\.1\-8B\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Shaded band: standard deviation over three seeds\.
Figure 16:Per\-block MLP\-probe AUROC for Mistral\-7B\-Instruct\-v0\.3\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Shaded band: standard deviation over three seeds\.
Figure 17:Per\-block MLP\-probe AUROC for Qwen2\.5\-7B\-Instruct \(2828transformer blocks\)\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Shaded band: standard deviation over three seeds\.
## Appendix 0\.LClass Geometry Across All Cells
Figures[18](https://arxiv.org/html/2606.02628#Pt0.A12.F18)–[20](https://arxiv.org/html/2606.02628#Pt0.A12.F20)extend the class\-geometry diagnostic of Figure[3](https://arxiv.org/html/2606.02628#S4.F3)to all1212configurations\. The qualitative shape is preserved throughout: the centroid distance grows faster than the within\-class spread, the separation ratio rises through the early blocks and plateaus in the mid\-to\-late blocks, and the depth at which it plateaus aligns with the probe\-AUROC peak\. The synthetic benchmark exhibits the steepest centroid drift, consistent with the≈1\.000\\approx\\\!1\.000AUROC plateau\.
Figure 18:Layer\-wise class geometry for Llama\-3\.1\-8B on HaluEval\-QA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\), showing centroid distance, within\-class spread, and separation ratio\. The separation ratio plateaus in mid\-to\-late blocks across all three datasets\. Cf\. Figure[3](https://arxiv.org/html/2606.02628#S4.F3)for TruthfulQA\.
Figure 19:Layer\-wise class geometry for Mistral\-7B on HaluEval\-QA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\)\. The same rise\-and\-plateau pattern holds across datasets\. Cf\. Figure[9](https://arxiv.org/html/2606.02628#Pt0.A4.F9)\(top\) for TruthfulQA\.
Figure 20:Layer\-wise class geometry for Qwen2\.5\-7B on HaluEval\-QA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\)\. The synthetic benchmark exhibits the steepest centroid drift, consistent with the≈1\.000\{\\approx\}1\.000AUROC plateau\. Cf\. Figure[9](https://arxiv.org/html/2606.02628#Pt0.A4.F9)\(bottom\) for TruthfulQA\.
## Appendix 0\.MINSIDE EigenScore Distributions Across All Cells
Figures[21](https://arxiv.org/html/2606.02628#Pt0.A13.F21)–[23](https://arxiv.org/html/2606.02628#Pt0.A13.F23)reproduce the per\-class INSIDE EigenScore histograms across all1212configurations, complementing Figure[4](https://arxiv.org/html/2606.02628#S4.F4)\(Llama / TruthfulQA\) and Figure[10](https://arxiv.org/html/2606.02628#Pt0.A5.F10)\(Mistral, Qwen on TruthfulQA\)\. In every panel the truthful and hallucinated distributions are nearly indistinguishable\. This is a uniform finding: the INSIDE AUROC remains in the narrow band\[0\.433,0\.529\]\[0\.433,0\.529\]across all1212entries in Table[2](https://arxiv.org/html/2606.02628#S4.T2)\.
Figure 21:INSIDE EigenScore distributions for Llama\-3\.1\-8B on HaluEval\-QA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\), separated by ground\-truth label\. The two class distributions are nearly indistinguishable in every panel\. Cf\. Figure[4](https://arxiv.org/html/2606.02628#S4.F4)for TruthfulQA\.
Figure 22:INSIDE EigenScore distributions for Mistral\-7B on HaluEval\-QA \(top\), FEVER \(middle\), and the synthetic benchmark \(bottom\)\. The near\-complete overlap persists across all datasets\. Cf\. Figure[10](https://arxiv.org/html/2606.02628#Pt0.A5.F10)\(top\) for TruthfulQA\.
Figure 23:INSIDE EigenScore distributions for Qwen2\.5\-7B on HaluEval\-QA \(left\), FEVER \(middle\), and the synthetic benchmark \(right\)\. The uniform overlap across all1212configurations confirms that the chance\-level INSIDE AUROC \(0\.4330\.433–0\.5290\.529\) is inherent to the evaluation protocol\. Cf\. Figure[10](https://arxiv.org/html/2606.02628#Pt0.A5.F10)\(bottom\) for TruthfulQA\.
## Appendix 0\.NAttention Entropy on Non\-Knowledge\-Grounded Datasets
Figures[24](https://arxiv.org/html/2606.02628#Pt0.A14.F24)–[26](https://arxiv.org/html/2606.02628#Pt0.A14.F26)report the head\-averaged last\-token attention entropy on TruthfulQA, FEVER, and the synthetic benchmark for all three models, substantiating that the first\-block attention\-entropy signal does*not*generalize beyond HaluEval\-QA\. In each panel the three captured blocks \(first, middle, last\) are shown side by side\. The truthful and hallucinated entropy histograms overlap substantially in all three non\-HaluEval datasets, with the best attention\-entropy AUROC ranging from0\.4890\.489\(Qwen / TruthfulQA; Table[2](https://arxiv.org/html/2606.02628#S4.T2)\) to0\.7740\.774\(Mistral / synthetic\)\. This contrasts with HaluEval\-QA, where first\-block attention entropy reaches0\.8660\.866–0\.9410\.941AUROC\. The disparity is expected: only HaluEval\-QA prepends an explicit evidence passage, giving the model a concrete span to attend to when the answer is supported\.
Figure 24:Last\-token attention entropy on TruthfulQA for Llama\-3\.1\-8B \(top\), Mistral\-7B \(middle\), and Qwen2\.5\-7B \(bottom\) at three captured transformer blocks\. The truthful and hallucinated histograms overlap substantially in every panel, with the best per\-model AUROC remaining below0\.610\.61\.
Figure 25:Last\-token attention entropy on FEVER for Llama\-3\.1\-8B \(top\), Mistral\-7B \(middle\), and Qwen2\.5\-7B \(bottom\)\. Without an explicit supporting\-knowledge channel, the per\-class entropy histograms are no longer cleanly separable, contrasting with the strong first\-block signal on HaluEval\-QA \(Figure[11](https://arxiv.org/html/2606.02628#Pt0.A6.F11)\)\.
Figure 26:Last\-token attention entropy on the synthetic benchmark for Llama\-3\.1\-8B \(top\), Mistral\-7B \(middle\), and Qwen2\.5\-7B \(bottom\)\. The middle block performs best for Mistral \(AUROC0\.7740\.774\), but the signal remains far weaker than probe\-based detectors across all three models\.
## Appendix 0\.O2\-D Projections Across All Cells
Figures[27](https://arxiv.org/html/2606.02628#Pt0.A15.F27)–[29](https://arxiv.org/html/2606.02628#Pt0.A15.F29)provide PCA and t\-SNE projections of each model’s hidden states at one selected block per configuration\. The block shown is the one that maximizes the*separation ratio*\(centroid distance divided by mean within\-class spread; see §[3\.3](https://arxiv.org/html/2606.02628#S3.SS3)\), chosen independently of any trained probe\. For Llama and Qwen on TruthfulQA, this block coincides with the probing peak \(blocks1414and1919respectively\)\. For several other configurations the separation ratio peaks at very early blocks, where surface\-level token identity dominates the centroid geometry while truthful/hallucinated separation remains weak \(visible as heavily overlapping point clouds\)\. The projections at the probing peak \(main text and Appendix[0\.G](https://arxiv.org/html/2606.02628#Pt0.A7)\) remain the most informative visualization of the linearly decodable signal reported in Table[2](https://arxiv.org/html/2606.02628#S4.T2)\.
Figure 27:PCA \(left\) and t\-SNE \(right\) projections of Llama\-3\.1\-8B hidden states at the block maximizing the classifier\-free separation ratio: HaluEval\-QA \(top, block3232\), FEVER \(middle, block0\), and synthetic \(bottom, block1212\)\. Panels at block0show heavily overlapping clouds because the separation ratio peaks early due to lexical\-identity effects rather than truthfulness geometry\. Cf\. Figures[3](https://arxiv.org/html/2606.02628#S4.F3),[12](https://arxiv.org/html/2606.02628#Pt0.A7.F12)for the TruthfulQA probing\-peak projection\.
Figure 28:PCA \(left\) and t\-SNE \(right\) projections of Mistral\-7B hidden states at the separation\-ratio argmax block\. Top to bottom: TruthfulQA \(block0\), HaluEval\-QA \(block0\), FEVER \(block0\), and synthetic \(block1414\)\. The three block\-0panels show overlapping clusters; the synthetic block\-1414panel shows clearer separation, consistent with its high probe AUROC\.
Figure 29:PCA \(left\) and t\-SNE \(right\) projections of Qwen2\.5\-7B hidden states at the separation\-ratio argmax block\. Top to bottom: HaluEval\-QA \(block0\), FEVER \(block0\), and synthetic \(block1818\)\. The block\-1818synthetic panel shows clear class clusters; the block\-0panels reflect lexical rather than truthfulness separation\. Cf\. Figures[12](https://arxiv.org/html/2606.02628#Pt0.A7.F12),[13](https://arxiv.org/html/2606.02628#Pt0.A8.F13)for the TruthfulQA probing\-peak projection\.
## Appendix 0\.PAccuracy andF1F\_\{1\}at Best Block
Tables[3](https://arxiv.org/html/2606.02628#S4.T3)and[4](https://arxiv.org/html/2606.02628#S4.T4)in the main text \(§[4\.1](https://arxiv.org/html/2606.02628#S4.SS1)\) report accuracy andF1F\_\{1\}at the best block for every model, dataset, and method\. Probes use a fixed threshold of0\.50\.5; the remaining methods use the Youden\-JJoptimal threshold \(maximizing sensitivity plus specificity minus one\)\. The tables confirm the same qualitative pattern as the AUROC results: probes achieve accuracy≥0\.800\\geq\\\!0\.800andF1≥0\.807F\_\{1\}\\\!\\geq\\\!0\.807on all natural\-language benchmarks, while the sampling\-based methods stay near chance \(attention entropy again being the exception on HaluEval\-QA\)\. On the synthetic benchmark, MLP probes reach0\.9750\.975–0\.9920\.992accuracy and comparableF1F\_\{1\}, consistent with the≥0\.998\\geq\\\!0\.998AUROC plateau\. Exact\-match self\-consistencyF1F\_\{1\}values are unstable because stochastic completions rarely collide as exact strings atK=5K\{=\}5, causing the method to assign identical scores to nearly all samples\.
## Appendix 0\.QLinear\-Probe Layer\-Wise AUROC Trajectories
Figures[30](https://arxiv.org/html/2606.02628#Pt0.A17.F30)–[32](https://arxiv.org/html/2606.02628#Pt0.A17.F32)report the per\-blocklinear\-probe AUROC curves for all1212configurations\. As noted in §[4\.1](https://arxiv.org/html/2606.02628#S4.SS1)\(F2\), these trajectories are visually near\-identical to their MLP counterparts \(Figures[15](https://arxiv.org/html/2606.02628#Pt0.A11.F15)–[17](https://arxiv.org/html/2606.02628#Pt0.A11.F17)\), providing direct visual confirmation that the truthfulness signal is approximately linear across all three model families and all four datasets\.
Figure 30:Per\-block*linear*\-probe AUROC for Llama\-3\.1\-8B\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Shaded band: standard deviation over three seeds\. Cf\. Figure[15](https://arxiv.org/html/2606.02628#Pt0.A11.F15)for the MLP counterpart\.
Figure 31:Per\-block*linear*\-probe AUROC for Mistral\-7B\-Instruct\-v0\.3\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Shaded band: standard deviation over three seeds\. Cf\. Figure[16](https://arxiv.org/html/2606.02628#Pt0.A11.F16)for the MLP counterpart\.
Figure 32:Per\-block*linear*\-probe AUROC for Qwen2\.5\-7B\-Instruct \(2828transformer blocks\)\.*Top\-left*: TruthfulQA\.*Top\-right*: HaluEval\-QA\.*Bottom\-left*: FEVER\.*Bottom\-right*: synthetic\. Shaded band: standard deviation over three seeds\. Cf\. Figure[17](https://arxiv.org/html/2606.02628#Pt0.A11.F17)for the MLP counterpart\.
## Appendix 0\.RHyperparameters
Table[5](https://arxiv.org/html/2606.02628#Pt0.A18.T5)consolidates all hyperparameters used in the experiments; these values are also included in the released code for full reproducibility\.
Table 5:Hyperparameters used in all experiments\. All values are provided in the released code for reproducibility\.Similar Articles
Mind the Unseen Mass: Unmasking LLM Hallucinations via Soft-Hybrid Alphabet Estimation
Researchers introduce SHADE, a hybrid estimator that combines Good-Turing coverage with graph-spectral cues to quantify semantic uncertainty and detect LLM hallucinations when only a few black-box samples are available.
Readable but Not Controllable: Neuron-Level Evidence for Medical LLM Hallucination
This paper investigates whether hallucination in medical LLMs can be detected and controlled at the neuron level. The authors find that while hallucination signals are detectable across many neurons (AUROC 0.77-0.86), they are not easily corrected by steering those same neurons.
Hallucination as an Anomaly: Dynamic Intervention via Probabilistic Circuits
This paper presents PCNet, a probabilistic circuit trained as a tractable density estimator on LLM residual streams to detect hallucinations as geometric anomalies. It also introduces PC-LDCD, a dynamic correction method that only intervenes on hallucinated tokens, achieving near-perfect detection and reduced corruption rates.
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
This paper presents a mechanistic analysis of why LLMs hallucinate when reasoning over linearized structured knowledge, finding that hallucinations stem from systematic internal dynamics such as attention on shortcut cues and failures in semantic grounding in feed-forward layers, rather than random noise.
Do LLMs Really Know What They Don't Know? Internal States Mainly Reflect Knowledge Recall Rather Than Truthfulness
This paper challenges the assumption that LLMs can reliably distinguish between hallucinated and factual outputs through internal signals, arguing that internal states primarily reflect knowledge recall rather than truthfulness. The authors propose a taxonomy of hallucinations (associated vs. unassociated) and show that associated hallucinations exhibit hidden-state geometries overlapping with factual outputs, making standard detection methods ineffective.