PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts

# PARALLAX: Separating Genuine Progress from Benchmark Artifacts in Hallucination Detection
Source: [https://arxiv.org/html/2605.17028](https://arxiv.org/html/2605.17028)
###### Abstract

Large language models \(LLMs\) hallucinate with confidence: their outputs can be fluent, authoritative, and simply wrong\. In medical, legal, and scientific applications this failure causes direct harm, and detecting it from internal model states offers a path to safer deployment\. A growing body of work reports that this problem is increasingly tractable, with recent methods achieving high detection performance on widely used benchmarks\. We show, however, that much of this apparent progress does not survive scrutiny\. Four of the six corpora embed the ground\-truth answer directly in the input prompt\. A naïve text\-similarity baseline we callTxTembexploits this to achieve near\-perfect detection scores without any access to model internals\. To measure what genuine detection capability remains once these artifacts are controlled, we conduct a large\-scale evaluation spanning twenty\-two detection methods, twelve open\-source models spanning six architectural families, and six corpora\. We further introduceDRIFT, a supervised probe over inter\-layer hidden\-state transitions, as a point of comparison for live\-generation detection\. Our findings suggest that the field’s reported progress on hallucination detection is substantially explained by benchmark construction artifacts in widely used corpora, and that the majority of established baselines perform near chance under controlled conditions; the consistent exceptions are SAPLMA and DRIFT, both supervised probes on upper\-layer hidden states\.

## 1Introduction

Reliable hallucination detection matters: errors in medical, legal, and scientific applications cause direct harm\(Survey Authors,[2025](https://arxiv.org/html/2605.17028#bib.bib37); Dahlet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib90); OpenAI,[2025](https://arxiv.org/html/2605.17028#bib.bib9)\), and a real\-time detector requiring no extra inference would enable safer deployment of production LLMs\. A growing body of work reports high\-AUROC hallucination detectors\(Zhanget al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib47); Jiet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib69); Huanget al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib70)\), but the field lacks controlled cross\-method comparisons and reported scores come with two serious confounds that have not been adequately addressed\. The first is benchmark construction: popular corpora such as HaluEval\(Liet al\.,[2023a](https://arxiv.org/html/2605.17028#bib.bib50)\), MedHallu\(MedHallBench Authors,[2024](https://arxiv.org/html/2605.17028#bib.bib36)\), and TruthfulQA\(Linet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib60)\)present the model with both a correct and a hallucinated answer in the same prompt, so the label is correlated with surface text differences rather than model uncertainty\. We demonstrate this withTxTemb\(Eq\.[1](https://arxiv.org/html/2605.17028#S3.E1)\): a TF\-IDF cosine similarity score between the hallucinated and reference answer texts, requiring no access to model internals, which achieves AUROC=0\.98=0\.98on HaluEval\. Any internal\-state method achieving similar performance is not detecting hallucination; it is detecting lexical phrasing patterns in the prompt\. Throughout this paper,TxTembscores serve as an artifact control: a highTxTembAUROC on a given corpus indicates that surface text differences, not model uncertainty, drive detection performance\. The second confound is evaluation scope: most papers evaluate one or two benchmarks on a single model, making it impossible to distinguish a genuine mechanistic signal from overfitting to a specific benchmark\-model combination\.

We address both confounds with a large\-scale controlled evaluation\. Across six corpora \(open\-domain, medical, factual, legal, RAG, multi\-domain QA\), we evaluate twenty\-two detection methods on twelve instruction\-tuned models spanning six architectural families \(3\.8B–72B parameters\)\. Our six probe\-based approaches includeDRIFT\(Approach E, introduced here\), self\-supervised perturbation contrast \(A\), contrastive activation addition probe \(B\), semantic entropy probe \(C, adaptation of\(Farquharet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib26)\)\), activation variance consistency \(D\), and answer expectation \(F\)\. These are evaluated against sixteen established and new baselines\(Chen and others,[2024](https://arxiv.org/html/2605.17028#bib.bib30); Manakulet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib59); Su and others,[2024](https://arxiv.org/html/2605.17028#bib.bib29); Azaria and Mitchell,[2023](https://arxiv.org/html/2605.17028#bib.bib16); Chuanget al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib31); Wang and others,[2025a](https://arxiv.org/html/2605.17028#bib.bib17); Duet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib61); Dasguptaet al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib62); Farquharet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib26); Fujieet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib88); Niu and others,[2025](https://arxiv.org/html/2605.17028#bib.bib54)\), withTxTembartifact controls, permutation tests, and bootstrap confidence intervals applied to every method\-corpus pair\. We include DRIFT among the evaluated methods; all probe hyperparameters were fixed before the live\-generation test splits \(RAGTruth, HaluBench\) were accessed\.

Our findings raise serious concerns\.*Teacher\-forced*corpora embed the ground\-truth answer directly in the input prompt;*live\-generation*corpora require the model to respond freely, with post\-hoc labels\. On teacher\-forced corpora, artifact\-exploiting methods reach AUROC0\.850\.85–1\.001\.00but collapse to near\-chance once the artifact is removed\. On live\-generation corpora, SAPLMA and DRIFT, both supervised probes, reliably exceed chance on HaluBench, reaching AUROC0\.910\.91\(mean0\.8790\.879across twelve architectures\), while every label\-free method, including MIND, HaloScope, ACT, SEPs, and HalluShift, fails near chance regardless of architecture\. On RAGTruth every method falls between0\.430\.43and0\.570\.57, supervised or otherwise\.

Our contributions:\(1\) A large\-scale controlled evaluation of 22 detection methods across 6 corpora and 12 models withTxTembartifact controls, permutation tests, and bootstrap CIs\. \(2\) The finding that teacher\-forcing inflates reported AUROC by up to0\.430\.43points, and that SAPLMA \(2023\) achieves the strongest controlled HaluBench performance among all externally published prior methods, a result obscured by artifact\-contaminated evaluations in the prior literature\. \(3\) The finding that all 22 methods cluster between AUROC0\.430\.43and0\.570\.57on RAGTruth, with no method significantly exceeding chance under permutation tests, establishing RAGTruth as a concrete unsolved benchmark for activation\-based detection\. \(4\)DRIFT, a supervised inter\-layer probe within0\.0040\.004of SAPLMA on HaluBench \(0\.9150\.915vs\.0\.9110\.911\) and statistically indistinguishable on RAGTruth \(p=0\.82p\{=\}0\.82\), confirming that the positive HaluBench result is not unique to a 2023 single\-layer architecture; the DRIFT\-concat ablation \(same layers, concatenated\) further isolates upper\-layer depth selection as the primary driver of supervised probe performance\. \(5\) Code and evaluation scripts released at[https://anonymous\.4open\.science/r/llm\-fuzzing\-2A24](https://anonymous.4open.science/r/llm-fuzzing-2A24)\.

L0L\_\{0\}LnL\_\{n\}LLM\(frozen\)0\.60​n0\.60n0\.70​n0\.70n0\.80​n0\.80n0\.85​n0\.85nprompt \+ response tokensmean\-pooloverTTtokens∈ℝd\\in\\mathbb\{R\}^\{d\}h1h\_\{1\}h2h\_\{2\}h3h\_\{3\}h4h\_\{4\}ϕ34\\boldsymbol\{\\phi\}\_\{34\}:𝐡4−𝐡3,cos\(⋅\),∥⋅∥2∈ℝd\+2\\mathbf\{h\}\_\{4\}\{\-\}\\mathbf\{h\}\_\{3\},\\;\\cos\(\\cdot\),\\;\\\|\\cdot\\\|\_\{2\}\\;\\in\\mathbb\{R\}^\{d\+2\}⋮\\vdotsϕ12\\boldsymbol\{\\phi\}\_\{12\}:𝐡2−𝐡1,cos\(⋅\),∥⋅∥2∈ℝd\+2\\mathbf\{h\}\_\{2\}\{\-\}\\mathbf\{h\}\_\{1\},\\;\\cos\(\\cdot\),\\;\\\|\\cdot\\\|\_\{2\}\\;\\in\\mathbb\{R\}^\{d\+2\}concat𝐳∈ℝK​\(d\+2\)\\mathbf\{z\}\\\!\\in\\\!\\mathbb\{R\}^\{K\(d\+2\)\}49,16449\{,\}164\-dimLogisticProbeL2\-regularised5\-fold CVfit on labeled pairsP​\(hal\.\)P\(\\text\{hal\.\}\)wE=w\_\{E\}=contrastive dir\.SAPLMA:single layer, last token→ℝd→\\to\\mathbb\{R\}^\{d\}\\toprobe\.HalluShift:scalar1−cos⁡\(𝐡a,𝐡b\)1\{\-\}\\cos\(\\mathbf\{h\}\_\{a\},\\mathbf\{h\}\_\{b\}\)\.DRIFT\-concat:3\-layer concat, noϕ\\boldsymbol\{\\phi\}diffs\.

Figure 1:DRIFT architecture\.Hidden states are tapped at four upper\-layer positions \(60–85% depth\), mean\-pooled overTTtokens, and differenced across allC​\(4,2\)=6C\(4,2\)\{=\}6pairs to formϕa​b∈ℝd\+2\\boldsymbol\{\\phi\}\_\{ab\}\\in\\mathbb\{R\}^\{d\+2\}, then concatenated into𝐳∈ℝ49,164\\mathbf\{z\}\\in\\mathbb\{R\}^\{49\{,\}164\}\. An L2\-regularised logistic probe maps𝐳\\mathbf\{z\}toP​\(hallucination\)P\(\\text\{hallucination\}\); its weight vectorwEw\_\{E\}doubles as a hallucination contrastive direction\. Dashed box: baseline comparisons\.
## 2Related Work

Internal\-state hallucination probes\.SAPLMA\(Azaria and Mitchell,[2023](https://arxiv.org/html/2605.17028#bib.bib16)\)trains a logistic probe on the last\-layer hidden state at the final token\. INSIDE\(Chen and others,[2024](https://arxiv.org/html/2605.17028#bib.bib30)\)uses eigenvalues of the hidden\-state covariance matrix; MIND\(Su and others,[2024](https://arxiv.org/html/2605.17028#bib.bib29)\)applies an unsupervised variance detector at runtime; HaloScope\(Duet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib61)\)scores reconstruction error from a truncated SVD; HaMI\(Niu and others,[2025](https://arxiv.org/html/2605.17028#bib.bib54)\)uses multiple\-instance learning over stochastic samples; PRISM\(Fujieet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib88)\)unifies internal\-state signals via cross\-attention; HalluShift\(Dasguptaet al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib62)\)computes a scalar inter\-layer cosine distance; and ACT\-ViT\(Bar\-Shalom and others,[2025](https://arxiv.org/html/2605.17028#bib.bib53)\)applies activation tensor decomposition via vision transformer encoders\. DRIFT is an alternative in this family that extends the probing approach to inter\-layer directions, building on evidence that models encode incorrectness before it appears in output\(Orgadet al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib18); Ferrandoet al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib40); Quevedo and others,[2024](https://arxiv.org/html/2605.17028#bib.bib33); Various Authors,[2025](https://arxiv.org/html/2605.17028#bib.bib58); Liang and others,[2025](https://arxiv.org/html/2605.17028#bib.bib56)\)and on probing methodology foundations\(Belinkov,[2022](https://arxiv.org/html/2605.17028#bib.bib84); Tenneyet al\.,[2019](https://arxiv.org/html/2605.17028#bib.bib81)\)\.

Benchmark artifacts and output\-space methods\.HaluEval\(Liet al\.,[2023a](https://arxiv.org/html/2605.17028#bib.bib50)\)concatenates correct and hallucinated answers in the same prompt, enabling lexical shortcuts unrelated to model uncertainty; we quantify this via theTxTembgap criterion\. HADES\(Liuet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib48)\)provides a reference\-free token\-level alternative, though surface artifacts remain pervasive across popular corpora\. Analogous annotation challenges arise in faithfulness evaluation\(Maynezet al\.,[2020](https://arxiv.org/html/2605.17028#bib.bib75); Labanet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib77); Slobodkinet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib87); Minet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib74)\)and knowledge\-limit studies\(Petroniet al\.,[2019](https://arxiv.org/html/2605.17028#bib.bib86); Mallenet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib76)\)further motivate live\-generation corpora\. SelfCheckGPT\(Manakulet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib59)\)measures NLI inconsistency acrossN=10N\{=\}10samples; P\(True\)\(Kadavathet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib72)\)prompts the model to self\-assess; DoLa\(Chuanget al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib31)\)contrasts early/late\-layer logits; and semantic entropy\(Farquharet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib26); Kuhnet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib73)\)clusters responses by entailment\. All require extra inference passes and, as we show, yield near\-chance AUROC on RAGTruth\.

Activation steering\.CAA\(Panicksseryet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib11)\), RepE\(Zouet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib10)\), ITI\(Liet al\.,[2023b](https://arxiv.org/html/2605.17028#bib.bib13)\), ACT\(Wang and others,[2025a](https://arxiv.org/html/2605.17028#bib.bib17)\), FASB\(Chenget al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib28)\), CCS\(Burnset al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib15)\), and ActAdd\(Turneret al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib12)\)build and inject contrastive steering vectors\. DRIFT’s probe weight vector doubles as a hallucination contrastive direction, offering a unified detect\-then\-steer path without a separate steering\-vector computation\.Balepur and others \([2025](https://arxiv.org/html/2605.17028#bib.bib51)\)show no single direction generalises across prompts, motivating multi\-pair designs; dynamic and real\-time variants\(Yin and others,[2025](https://arxiv.org/html/2605.17028#bib.bib32); Obeso and others,[2025](https://arxiv.org/html/2605.17028#bib.bib52); RepEng Survey Authors,[2025](https://arxiv.org/html/2605.17028#bib.bib46)\)operationalise these at deployment\.

Mechanistic interpretability, cascades, and domain evaluation\.Residual stream analysis\(Elhageet al\.,[2021](https://arxiv.org/html/2605.17028#bib.bib79); Menget al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib80)\)and hallucination studies in RAG and reasoning\(Sun and others,[2024](https://arxiv.org/html/2605.17028#bib.bib41); Dassen and others,[2026](https://arxiv.org/html/2605.17028#bib.bib42); Yang and others,[2025](https://arxiv.org/html/2605.17028#bib.bib45)\)provide mechanistic grounding for internal\-state methods; hallucinated tokens propagate autoregressively\(Zhanget al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib19); Dziriet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib23); Leeet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib25)\)\. Domain evaluation in clinical and high\-stakes settings\(Asgari and others,[2025](https://arxiv.org/html/2605.17028#bib.bib34); OpenAI,[2025](https://arxiv.org/html/2605.17028#bib.bib9); Wang and others,[2025b](https://arxiv.org/html/2605.17028#bib.bib8); Perezet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib1); Nesset al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib3)\)extends the landscape\.

## 3Experimental Setup

Models\.We evaluate 12 instruction\-tuned\(Ouyanget al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib67)\)models spanning six families \(3\.8B–72B parameters\): Llama \(3\.1\-8B, 3\.3\-70B\)\(Touvronet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib64); Dubeyet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib65)\), Qwen \(3\-8B, 3\-32B, 2\.5\-72B\), DeepSeek\-R1 distillations \(7B, 70B; reasoning models with chain\-of\-thought\(Weiet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib68)\)\), Mistral \(Mixtral\-8x7B MoE, Mistral\-24B\), Gemma \(3\-4b, 3\-12b\), and Phi\-4\-mini\. Llama\-3\.3\-70B is the primary model \(all 22 methods\); the remaining 11 run DRIFT, SAPLMA, and SelfCheckGPT\-NLI on TruthfulQA and HaluBench\. Full model identifiers appear in Appendix[A](https://arxiv.org/html/2605.17028#A1)\.

Train/test split\.All corpora are split 80/20 stratified by label \(test set strictly held out\); cross\-validation methods further split the training set into 5 folds\.

Compute\.Full evaluation takes 4–6 h per model for 8B and 18–24 h for 70B \(Appendix[K](https://arxiv.org/html/2605.17028#A11)\)\. INSIDE is the slowest baseline \(≈\\approx6 h per corpus on 70B\)\.

Computational cost per method\.All probe\-based methods share one hidden\-state extraction pass per prompt \(15\.615\.6s loading,9\.69\.6ms/prompt amortised\)\. At test time: DRIFT\-logp adds2\.12\.1ms, SAPLMA111111ms, DRIFT1\.21\.2s\. SelfCheckGPT\-NLI requires ten additional GPU inference passes per prompt, placing it in a wholly different cost class\. Full per\-method benchmarks are given in Appendix[B](https://arxiv.org/html/2605.17028#A2)\(Table[6](https://arxiv.org/html/2605.17028#A2.T6)\) and the Pareto frontier is presented in Appendix[C](https://arxiv.org/html/2605.17028#A3)\.

### 3\.1Benchmark Suite

We evaluate on six publicly available corpora spanning domain, annotation style, and generation format \(Table[1](https://arxiv.org/html/2605.17028#S3.T1)\)\.

Table 1:Benchmark suite\.TF= teacher\-forced \(answer text in prompt\);LG= live\-generated \(model produces the response freely\)\.CorpusDomainNBalanceLabel sourceFormatHaluEval\(Liet al\.,[2023a](https://arxiv.org/html/2605.17028#bib.bib50)\)Open\-domain QA500250/250ChatGPT\-generated pairs\(Achiamet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib66)\)TFMedHallu\(MedHallBench Authors,[2024](https://arxiv.org/html/2605.17028#bib.bib36)\)Medical QA500250/250Reference pairsTFTruthfulQA\(Linet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib60)\)Factual QA1,634817/817Reference pairsTFLegal\(Dahlet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib90)\)Legal case QA499159/340Expert correctness scoreTFRAGTruth\(Niuet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib49)\)RAG \(news/QA\)500153/347Expert annotationLGHaluBench\(Raviet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib89)\)Multi\-domain QA500250/250PatronusAI PASS/FAILLG

Four corpora use teacher\-forced \(TF\) construction, where both a correct and a hallucinated response appear in the prompt; the remaining two \(RAGTruth, HaluBench\) are live\-generated \(LG\), with the model responding freely\. The TF artifact is strongest on HaluEval \(TxTemb=0\.98=0\.98\), moderate on MedHallu \(TxTemb=0\.71=0\.71\) and Legal \(TxTemb=0\.89=0\.89; despite nuanced expert correctness judgements, surface\-text cues remain highly predictive\), and weakest on TruthfulQA \(TxTemb=0\.69=0\.69, which still requires factual knowledge to distinguish correct from incorrect answers\)\. Figure[2](https://arxiv.org/html/2605.17028#S3.F2)illustrates the two regimes\.

Teacher\-ForcedHaluEval⋅\\cdotMedHallu⋅\\cdotLegal⋅\\cdotTruthfulQAprompt \+ ‘‘\[Correct: X\] \[Hallucination: Y\]’’ →\\rightarrowlabel embedded in textTxTemb AUROC:0\.69–0\.98R2:0\.81△\\triangleArtifact: answer text in promptLive\-GenerationRAGTruth⋅\\cdotHaluBenchprompt→\\tomodel generates freely →\\topost\-hoc expert labelTxTemb AUROC:≈\\approx0\.50R2:0\.08✓ Real signal: model uncertainty

Figure 2:Two evaluation regimes\.*Left:*Teacher\-forced benchmarks embed the answer in the prompt;TxTembachieves AUROC 0\.69–0\.98 and explains 81% of variance in method AUROC \(R2=0\.81R^\{2\}\{=\}0\.81\), a formatting artifact\.*Right:*On live\-generation benchmarksTxTembis at chance \(≈0\.50\{\\approx\}0\.50,R2=0\.08R^\{2\}\{=\}0\.08\), isolating genuine internal\-state signal\.RAGTruth\(Niuet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib49)\)is our primary live\-generation benchmark: responses are generated by a RAG pipeline\(Lewiset al\.,[2020](https://arxiv.org/html/2605.17028#bib.bib78)\)under normal inference conditions, where faithfulness to source passages is essential\(Shiet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib85)\), and annotated post\-hoc by human experts; no answer text is embedded in the prompt\. HaluBench\(Raviet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib89)\)uses PatronusAI’s automated LLM\-as\-judge evaluator\(Zhenget al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib82)\)on freely generated multi\-domain QA; its absence of answer text in the prompt gives it the similar live\-generation character\.

Artifact control: text\-embed baseline\.On teacher\-forced corpora, hallucinated and correct answers differ in surface text\. We implementTxTemb: a purely text\-based detection signal defined as

sTxTemb​\(r\)=cos⁡\(tfidf​\(rhal\),tfidf​\(rref\)\),s\_\{\\textsc\{TxTemb\}\}\(r\)=\\cos\\\!\\bigl\(\\mathrm\{tfidf\}\(r\_\{\\mathrm\{hal\}\}\),\\,\\mathrm\{tfidf\}\(r\_\{\\mathrm\{ref\}\}\)\\bigr\),\(1\)whererhalr\_\{\\mathrm\{hal\}\}andrrefr\_\{\\mathrm\{ref\}\}are the hallucinated and reference answer texts\. The key insight is thatTxTembaccesses only the text of the responses, with no knowledge of the model’s internal states, probability distributions, or reasoning process\. If an internal\-state method achieves AUROC comparable toTxTembon the same corpus, its detection signal is explained by the same surface\-text differences that a lexical baseline can already exploit\. In that case, the method is not detecting genuine model uncertainty or hallucination\-prone internal states; it is, at best, re\-encoding information already present in the prompt text through hidden representations\. Any performance claim for such a method on that corpus is therefore attributable to the benchmark construction artifact rather than to a meaningful internal\-state signal\.

A cell is flagged \(†\\dagger\) if the method’s AUROC is within0\.050\.05ofTxTembAUROC on the same corpus\. The0\.050\.05threshold is chosen to exceed typical bootstrap confidence interval half\-widths observed in our experiments \(mean CI half\-width:0\.0240\.024\), ensuring that a flagged method’s performance advantage over the surface baseline is not statistically distinguishable from zero\. This is a deliberately conservative criterion: a method that outperformsTxTembby only0\.050\.05AUROC points may still be exploiting the same artifact through its hidden representations, and we do not wish to overstate the absence of artifact contamination\.

Evaluation metrics and verification\.We use Area Under the ROC Curve \(AUROC\)\(Fawcett,[2006](https://arxiv.org/html/2605.17028#bib.bib91)\)as ourprimary metricbecause it is threshold\-independent, invariant to class imbalance, and measures the probability that a randomly chosen positive instance is ranked above a randomly chosen negative one, making it appropriate for comparing detectors across the six corpora in our benchmark, which vary substantially in label balance \(Table[1](https://arxiv.org/html/2605.17028#S3.T1)\)\.

For all cells with AUROC\>0\.85\>0\.85we additionally compute: \(i\) bootstrap 95% CI \(n=1000n=1000resamples\); \(ii\) permutation\-test null mean \(n=30n=30label shuffles\); \(iii\)TxTembgap\. A cell is declared*Validated*if the CI lower bound exceeds0\.800\.80*and*theTxTembgap exceeds0\.050\.05\.

## 4Methods

Our evaluation covers six detection approaches of our own design alongside fourteen external baselines spanning label\-free, unsupervised, decoding\-time, and supervised\-probe paradigms\.DRIFT\(Approach E\) is our primary design contribution and the strongest performer\. Approaches A and E are self\-supervised \(no labels\), C and D requireN=10N\{=\}10stochastic samples per prompt, and B and F require teacher\-forced format\. All six share a common feature backbone: hidden states from a single forward pass pooled per\-layer across four upper\-layer checkpoints \(Section[4](https://arxiv.org/html/2605.17028#S4)\), enabling any subset to be evaluated on the same stored activations\.

Compute and quantisation\.All experiments run on a server with8×8\\timesNVIDIA L40S GPUs \(46 GB each\)\. Hidden\-state extraction runs on a single GPU in 4\-bit NF4 quantisation \(bitsandbytes double\-quantisation\) to fit 70B models in memory; batch generation uses 4\-GPU tensor parallelism \(tp=4\)\. One model–corpus pair \(Qwen3\-32B×\\timesTruthfulQA\) required re\-extraction with 8\-bit quantisation after NF4 produced NaN activations \(KV\-cache overflow at layers 38, 44, 51, 54\); the resolved result \(DRIFT AUROC=0\.671=0\.671\) is included in Table[4](https://arxiv.org/html/2605.17028#S5.T4)\.

Layer selection\.We extract hidden states at fractional depths\{0\.60,0\.70,0\.80,0\.85\}×L\\\{0\.60,\\,0\.70,\\,0\.80,\\,0\.85\\\}\\times L\(Llama\-3\.3\-70B: layers 48, 56, 64, 68\), confirmed by Appendix[D](https://arxiv.org/html/2605.17028#A4)\. Truth\-relevant directions reside in the upper network third\(Marks and Tegmark,[2023](https://arxiv.org/html/2605.17028#bib.bib14)\)\(AUROC 0\.741 at 20% vs\. 0\.898 at 85%\), final layers are dominated by next\-token prediction, and four taps outperform any single tap by\+0\.017\+0\.017AUROC\. These fractions were fixed before seeing any results and applied uniformly across all twelve architectures\.

Hidden state extraction\.A forward pass over\[prompt\+response\]\[\\text\{prompt\}\+\\text\{response\}\]yields hidden states at response token positions\. LetTTdenote the number of response tokens and𝐡t\(ℓ\)∈ℝd\\mathbf\{h\}^\{\(\\ell\)\}\_\{t\}\\in\\mathbb\{R\}^\{d\}the hidden state at layerℓ\\elland positiontt\. The per\-layer representation is mean\-pooled:

𝐡\(ℓ\)=1T​∑t=1T𝐡t\(ℓ\)∈ℝd\.\\mathbf\{h\}^\{\(\\ell\)\}=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbf\{h\}^\{\(\\ell\)\}\_\{t\}\\;\\in\\;\\mathbb\{R\}^\{d\}\.\(2\)
The four layer vectors are concatenated to form𝐡∈ℝ4​d\\mathbf\{h\}\\in\\mathbb\{R\}^\{4d\}\. The full prompt is included so activations are contextualised on the question\.

Probe architectures\.Two probe types are used: a*LogisticProbe*\(L2\-regularised logistic regression, 5\-fold CV overC∈\{0\.001,0\.01,0\.1,1\}C\\\!\\in\\\!\\\{0\.001,0\.01,0\.1,1\\\}\) and an*MLPProbe*\(two\-layer feedforward,min⁡\(256,d/4\)\\min\(256,d/4\)hidden units, ReLU, BCE loss\)\. The LogisticProbe weight vector simultaneously serves as the decision boundary and the hallucination contrastive direction\(Liet al\.,[2023b](https://arxiv.org/html/2605.17028#bib.bib13)\)\.

Approach A: Perturbation Contrast \(Self\-Supervised\)\.Four strategies generate a corrupted responser′r^\{\\prime\}from promptppand responserr\(entity swap, numerical corruption, negation flip, boundary violation\)\. An MLPProbe is trained onΔ​𝐡=𝐡​\(p,r\)−𝐡​\(p,r′\)\\Delta\\mathbf\{h\}=\\mathbf\{h\}\(p,r\)\-\\mathbf\{h\}\(p,r^\{\\prime\}\)\. No hallucination labels are required; the perturbation provides supervision\.

Approach B: Contrastive Activation Addition\.For teacher\-forced data with paired correct/hallucinated responses, we compute𝐯CAA=1N​∑i𝐡​\(pi,ri\+\)−𝐡​\(pi,ri−\)\\mathbf\{v\}\_\{\\text\{CAA\}\}=\\frac\{1\}\{N\}\\sum\_\{i\}\\mathbf\{h\}\(p\_\{i\},r^\{\+\}\_\{i\}\)\-\\mathbf\{h\}\(p\_\{i\},r^\{\-\}\_\{i\}\)and scoresB=cos⁡\(𝐡​\(p,r\),𝐯CAA\)s\_\{B\}=\\cos\(\\mathbf\{h\}\(p,r\),\\mathbf\{v\}\_\{\\text\{CAA\}\}\)\.†AUROC=1\.0=1\.0on MedHallu is an artifact of the teacher\-forced construction\. Approach B requires paired responses for the same prompt \(teacher\-forced only\)\. In MedHallu, GPT\-4 generated pairs with the reference answer visible in the prompt, so𝐯CAA\\mathbf\{v\}\_\{\\text\{CAA\}\}encodes surface\-text agreement rather than a genuine hallucination signal, and the method does not apply to live\-generation corpora\.

Approach C: Semantic Entropy Probe\.Semantic entropyH=−∑cpc​log⁡pcH=\-\\sum\_\{c\}p\_\{c\}\\log p\_\{c\}is computed fromN=10N\{=\}10stochastic samples\(Farquharet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib26)\), then an MLPProbe is trained on𝐡\\mathbf\{h\}to predictHH\. This amortises entropy computation to a single inference pass\.

Approach D: Activation Variance Consistency\.Element\-wise variance𝐯=Var⁡\(𝐡1,…,𝐡N\)\\mathbf\{v\}=\\operatorname\{Var\}\(\\mathbf\{h\}\_\{1\},\\ldots,\\mathbf\{h\}\_\{N\}\)overN=10N\{=\}10stochastic completions is fed to a LogisticProbe, capturing which activation dimensions are uncertain across samples\.

Approach E: DRIFT\.Letℒ=\{l1,l2,l3,l4\}\\mathcal\{L\}=\\\{l\_\{1\},l\_\{2\},l\_\{3\},l\_\{4\}\\\}denote the four selected layers and\(ℒ2\)\\binom\{\\mathcal\{L\}\}\{2\}the set ofK=6K\{=\}6unordered layer pairs\. For each pair\(la,lb\)\(l\_\{a\},l\_\{b\}\)withla<lbl\_\{a\}<l\_\{b\}, the per\-pair feature block is:

ϕa​b=\[𝐡\(lb\)−𝐡\(la\),cos⁡\(𝐡\(la\),𝐡\(lb\)\),‖𝐡\(lb\)−𝐡\(la\)‖2\]∈ℝd\+2,\\boldsymbol\{\\phi\}\_\{ab\}=\\Bigl\[\\,\\mathbf\{h\}^\{\(l\_\{b\}\)\}\-\\mathbf\{h\}^\{\(l\_\{a\}\)\},\\;\\cos\\\!\\bigl\(\\mathbf\{h\}^\{\(l\_\{a\}\)\},\\mathbf\{h\}^\{\(l\_\{b\}\)\}\\bigr\),\\;\\bigl\\\|\\mathbf\{h\}^\{\(l\_\{b\}\)\}\-\\mathbf\{h\}^\{\(l\_\{a\}\)\}\\bigr\\\|\_\{2\}\\,\\Bigr\]\\;\\in\\;\\mathbb\{R\}^\{d\+2\},\(3\)and the full DRIFT feature vector is the concatenation over all pairs:

𝐳=\[ϕa​b\]\(la,lb\)∈\(ℒ2\)∈ℝK​\(d\+2\)\.\\mathbf\{z\}=\\Bigl\[\\boldsymbol\{\\phi\}\_\{ab\}\\Bigr\]\_\{\(l\_\{a\},\\,l\_\{b\}\)\\,\\in\\,\\binom\{\\mathcal\{L\}\}\{2\}\}\\;\\in\\;\\mathbb\{R\}^\{K\(d\+2\)\}\.\(4\)For Llama\-3\.3\-70B \(d=8192d\{=\}8192,K=6K\{=\}6\) this gives6×8194=49,1646\\times 8194=49\{,\}164dimensions\. A StandardScaler \+ LogisticProbe is fitted on𝐳\\mathbf\{z\}\. This extends HalluShift’s scalar cosine distance to a directional, multi\-layer probed representation: the probe learns*which inter\-layer directions*are diagnostic, not just that some change occurred\. See Figure[1](https://arxiv.org/html/2605.17028#S1.F1)for an overview\.DRIFT\-concatis a controlled ablation of Approach E: it uses the same three tap layers \(l∈\{0\.60​n,0\.70​n,0\.80​n\}l\\in\\\{0\.60n,\\,0\.70n,\\,0\.80n\\\}\) but mean\-pools and concatenates the hidden states directly without inter\-layer difference features, isolating the contribution of the inter\-layer differencing\.

Approach F: Answer Expectation\.A LogisticProbe is trained on\[𝐡after−𝐡before,cos⁡\(𝐡before,𝐡after\),‖𝐡before−𝐡after‖\]\[\\mathbf\{h\}\_\{\\text\{after\}\}\{\-\}\\mathbf\{h\}\_\{\\text\{before\}\},\\;\\cos\(\\mathbf\{h\}\_\{\\text\{before\}\},\\mathbf\{h\}\_\{\\text\{after\}\}\),\\;\\\|\\mathbf\{h\}\_\{\\text\{before\}\}\{\-\}\\mathbf\{h\}\_\{\\text\{after\}\}\\\|\], where states are extracted just before and after the answer token\.†Teacher\-forced artifact:𝐡before\\mathbf\{h\}\_\{\\text\{before\}\}encodes the answer text\.DRIFT\-logptrains a logistic classifier on token log\-probability time series statistics \(mean, min, variance, slope, entropy\) over the response; it requires no hidden\-state access and adds only 2 ms latency\.

Baselines\.We evaluate fourteen external baselines spanning four paradigms\.Label\-free, multi\-sample:INSIDE\(Chen and others,[2024](https://arxiv.org/html/2605.17028#bib.bib30)\)scores the mean log\-eigenvalue of the hidden\-state covariance acrossNNresponse samples;SelfCheckGPT\-NLI\(Manakulet al\.,[2023](https://arxiv.org/html/2605.17028#bib.bib59)\)measures NLI contradiction acrossN=10N\{=\}10samples \(official code\);MIND\(Su and others,[2024](https://arxiv.org/html/2605.17028#bib.bib29)\)trains an MLP using entropy\-derived pseudo\-labels \(reimplemented with entropy thresholding\)\.Unsupervised representations:HaloScope\(Duet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib61)\)scores reconstruction error from a truncated SVD of training activations;HalluShift\(Dasguptaet al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib62)\)uses the scalar inter\-layer cosine distance1−cos⁡\(𝐡\(0\.6​L\),𝐡\(0\.85​L\)\)1\{\-\}\\cos\(\\mathbf\{h\}^\{\(0\.6L\)\},\\mathbf\{h\}^\{\(0\.85L\)\}\)\(Approach E is the learnable extension\);DoLa\(Chuanget al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib31)\)contrasts early/late\-layer logits at decoding time\.Decoding\-time signals:ACT\(Wang and others,[2025a](https://arxiv.org/html/2605.17028#bib.bib17)\)fits logistic regression on concatenated hidden states;SEPs\(Farquharet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib26)\)predicts semantic entropy from hidden states;IRIS⋆uses the model’s self\-verification probabilityP​\(“No”\)−P​\(“Yes”\)P\(\\text\{\`\`No''\}\)\{\-\}P\(\\text\{\`\`Yes''\}\)\(Kadavathet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib72)\);log\-prob\(mean response token log\-probability\) andP\(True\)\(Kadavathet al\.,[2022](https://arxiv.org/html/2605.17028#bib.bib72)\)\(self\-assessment prompt\) require no hidden states\.Supervised probes:SAPLMA\(Azaria and Mitchell,[2023](https://arxiv.org/html/2605.17028#bib.bib16)\)uses single\-layer last\-token logistic regression with cross\-validated layer selection;PRISM\(Fujieet al\.,[2024](https://arxiv.org/html/2605.17028#bib.bib88)\)uses a truth\-prompting forward pass to extract a classification embedding;HaMI\(Niu and others,[2025](https://arxiv.org/html/2605.17028#bib.bib54)\)applies multiple\-instance learning over stochastic response samples\.

## 5Results

### 5\.1Main Results: Llama\-3\.3\-70B

We report results on the primary evaluation model, Llama\-3\.3\-70B, before presenting the cross\-model sweep and artifact analysis\. Two patterns dominate the table: on four of six corpora, the reported numbers look strong \(\>0\.85\>0\.85\), but the artifact controls reveal that most of this performance is not mechanistically meaningful\. On the two corpora where teacher\-forcing contamination is minimal \(TruthfulQA and the verified HaluBench cells\), DRIFT and SAPLMA consistently lead the field, with SAPLMA narrowly ahead on TruthfulQA \(0\.750\.75vs\.0\.730\.73\) and both methods tied on HaluBench \(0\.910\.91\)\.

Table[2](https://arxiv.org/html/2605.17028#S5.T2)reports all 22 methods; verification details \(bootstrap CIs, permutation tests\) appear in Appendix[E](https://arxiv.org/html/2605.17028#A5)\. Cells are colour\-coded:green\(≥0\.85\\geq 0\.85\),yellow\(0\.700\.70–0\.840\.84\),orange\(0\.550\.55–0\.690\.69\),red\(<0\.55<0\.55\);gold†= teacher\-forcing artifact detected\. Bold marks the best artifact\-free result per column\.

Table 2:AUROC on Llama\-3\.3\-70B for all 22 methods, grouped by detection paradigm\.†artifact detected:TxTembgap<0\.05<0\.05\.‡AVC inverted on TruthfulQA \(confident myths have low variance; flipped AUROC=0\.73=0\.73\)\.Bold= best artifact\-free result per corpus\. Output/logit methods require additional LLM inference at test time\. “—” = not applicable \(B, F require teacher\-forced format\)\.⋆our implementation of concept fromKadavathet al\.\([2022](https://arxiv.org/html/2605.17028#bib.bib72)\); no public code available\.MethodHaluEvalMedHalluTruthfulQALegalRAGTruthHaluBenchOur approachesA: Perturbation \(self\-sup\.\)0\.540\.520\.540\.510\.570\.68B: CAA1\.00†1\.00†0\.87†0\.44——C: Sem\. Entropy0\.540\.500\.540\.500\.520\.62D: Act\. Variance0\.690\.570\.27‡0\.640\.780\.82E:DRIFT0\.960\.550\.730\.530\.470\.91DRIFT\-concat \(ablation of E\)0\.95†0\.570\.740\.520\.470\.91F: Ans\. Expectation1\.00†1\.00†0\.660\.97†0\.570\.97DRIFT\-logp0\.870\.870\.620\.540\.520\.76†Internal\-state baselinesSAPLMA0\.540\.570\.750\.480\.460\.91HaMI0\.860\.520\.610\.500\.460\.87PRISM0\.510\.450\.630\.540\.390\.80IRIS⋆0\.500\.510\.660\.500\.500\.67INSIDE0\.500\.500\.590\.550\.500\.50MIND0\.480\.480\.530\.530\.510\.67HaloScope0\.500\.510\.510\.480\.510\.67SEPs0\.560\.490\.520\.530\.490\.67ACT \(detect\)0\.480\.520\.490\.490\.550\.62HalluShift0\.450\.500\.430\.490\.520\.31Output / logit baselines \(\+LLM at test time\)P\(True\) \(\+1\)0\.550\.580\.680\.500\.550\.60SelfCheckGPT \(\+10\)0\.430\.550\.560\.550\.540\.54DoLa0\.580\.570\.550\.500\.500\.62Log\-Prob0\.490\.480\.510\.500\.520\.56TxTemb\(ctrl\)0\.980\.710\.690\.89n/a0\.77Best \(artifact\-free\)0\.960\.870\.750\.530\.570\.91Teacher\-forced artifact\.On HaluEval,TxTembachieves AUROC=0\.98=0\.98from surface text alone; on MedHallu the artifact is more moderate \(TxTemb=0\.71=0\.71\)\. Approaches B and F reach1\.001\.00\. Re\-running DRIFT on HaluEval with live\-generated \(non\-teacher\-forced\) responses yields AUROC=0\.62=0\.62, a0\.340\.34\-point drop from the teacher\-forced result of0\.960\.96, confirming the artifact accounts for the majority of reported performance\. These corpora are retained for completeness but carry no weight in our conclusions\.

TruthfulQA\.TxTembreaches only0\.690\.69on TruthfulQA, confirming that factual knowledge is required rather than surface matching\. SAPLMA achieves the highest point estimate at0\.7510\.751\(bootstrap 95% CI:\[0\.727,0\.773\]\[0\.727,0\.773\]\), followed by DRIFT\-concat at0\.740\.74\(CI:\[0\.713,0\.762\]\[0\.713,0\.762\]\) and Approach E \(DRIFT\) at0\.7290\.729\(CI:\[0\.703,0\.753\]\[0\.703,0\.753\]; permutation null:0\.5020\.502\)\. All three confidence intervals overlap substantially; the methods are statistically indistinguishable on TruthfulQA\. All carry a verifiedTxTembgap confirming real internal signal\. Notably, the simplest probe \(SAPLMA: last\-token at one layer\) achieves the highest point estimate on this factual knowledge corpus, suggesting that layer 56 alone captures a strong hallucination signal for factual QA, though as discussed in Section[6](https://arxiv.org/html/2605.17028#S6), this advantage does not survive the move to live\-generation corpora\.

Live\-generation benchmarks\.RAGTruth is the hardest corpus: Approach A reaches0\.570\.57, DRIFT reaches0\.470\.47, and most established baselines \(MIND, HaloScope, ACT, SEPs, HalluShift\) achieve near\-chance AUROC \(0\.490\.49–0\.550\.55\)\. This is the artifact\-free detection ceiling for live\-generation RAG; internal states carry only a small signal when no answer text is present to exploit\. HaluBench, our second live\-generation corpus, yields a cleaner separation\.

HaluBench\.DRIFT achieves AUROC=0\.915=\\mathbf\{0\.915\}\(CI:\[0\.889,0\.938\]\[0\.889,0\.938\];TxTembgap:\+0\.144\+0\.144\)\. SAPLMA \(0\.9110\.911\) and DRIFT\-concat \(0\.9110\.911\) are statistically indistinguishable on HaluBench for Llama\-3\.3\-70B\. INSIDE \(0\.500\.50\) and SelfCheckGPT \(0\.540\.54\) trail DRIFT by0\.410\.41and0\.380\.38points respectively\. MIND, HaloScope, ACT, SEPs, and HalluShift all achieve near\-chance AUROC \(0\.490\.49–0\.620\.62\) on HaluBench, confirming that these established baselines carry no reliable signal on either live\-generation corpus\.

Among all baselines, DRIFT\-concat, DRIFT\-logp, and HaMI score0\.860\.86–0\.950\.95on HaluEval but fall to near\-chance on RAGTruth, reproducing the two\-regime pattern\. PRISM is the exception: moderate across corpora \(best:0\.630\.63on TruthfulQA\) but below chance on RAGTruth \(0\.390\.39\)\. On TruthfulQA, DRIFT\-concat \(0\.740\.74\) is statistically indistinguishable from DRIFT \(0\.730\.73\)\. On RAGTruth, DRIFT \(0\.470\.47\), DRIFT\-concat \(0\.470\.47\), and SAPLMA \(0\.470\.47\) are all within statistical noise\.

Artifact analysis\.Across all method–corpus combinations on teacher\-forced benchmarks, methods that achieve high AUROC \(DRIFT, DRIFT\-concat, DRIFT\-logp, HaMI\) closely trackTxTembon HaluEval; the majority of remaining methods achieve near\-chance AUROC regardless of corpus format, indicating these baselines simply fail rather than exploiting the artifact\. On live\-generation benchmarks this pattern collapses to near\-zero correlation \(Figure[2](https://arxiv.org/html/2605.17028#S3.F2)\)\. Table[3](https://arxiv.org/html/2605.17028#S5.T3)shows DRIFT cross\-corpus transfer AUROC: probes transfer well within the same regime \(HaluEval→\\toHaluBench:0\.7250\.725\) but collapse across regimes, confirming the learned signal is regime\-specific rather than general\. The Legal probe transfers poorly in all directions, consistent with its near\-chance in\-corpus performance\.

Table 3:DRIFT cross\-corpus transfer AUROC \(Llama\-3\.3\-70B\)\. Rows = train corpus; columns = eval corpus\. Diagonal inbold\.†\\dagger= TF;‡\\ddagger= LG\.Train↓\\downarrow/ Test→\\rightarrowHaluEval†MedHallu†TruthfulQA†Legal†RAGTruth‡HaluBench‡HaluEval†0\.6400\.4990\.6020\.5040\.5070\.725MedHallu†0\.4890\.4010\.4820\.5310\.5070\.327TruthfulQA†0\.5170\.5180\.5950\.4800\.5220\.586Legal†0\.5920\.5010\.4960\.4780\.4640\.346RAGTruth‡0\.4520\.5160\.5010\.4640\.4370\.307HaluBench‡0\.5600\.5190\.5950\.4810\.5220\.804

Multi\-model sweep\.Table[4](https://arxiv.org/html/2605.17028#S5.T4)compares DRIFT, SAPLMA, and DRIFT\-concat \(ablation of DRIFT\) across all 12 models on TruthfulQA and HaluBench\. The Qwen3\-32B×\\timesTruthfulQA cell used 8\-bit re\-extraction after NF4 NaN activations \(Section[4](https://arxiv.org/html/2605.17028#S4)\)\.

![Refer to caption](https://arxiv.org/html/2605.17028v1/x1.png)
Figure 3:AUROC heatmap: 22 methods×\\times6 corpora on Llama\-3\.3\-70B\. Left block \(teacher\-forced\) shows uniformly high values; right block \(live\-generation\) shows the artifact\-free detection ceiling\.ModelTruthfulQAHaluBenchDRIFTSAPLMADRIFT\-concatSC†DRIFTSAPLMADRIFT\-concatSC†Llama\-3\.3\-70B0\.7290\.7510\.7370\.5580\.9150\.9110\.9110\.535Llama\-3\.1\-8B0\.6280\.6560\.6640\.5350\.8390\.8240\.8230\.579Qwen3\-8B0\.6750\.6710\.7440\.5710\.8680\.8940\.8790\.678Qwen3\-32B0\.6710\.7060\.8170\.5590\.8800\.8550\.8830\.679Qwen2\.5\-72B0\.8080\.8740\.8120\.5170\.9000\.8850\.8980\.613DS\-R1\-Qwen\-7B0\.6290\.7110\.6930\.4950\.7740\.7920\.7880\.524DS\-R1\-Llama\-70B0\.7200\.7290\.7350\.5330\.8710\.8650\.8730\.635Mixtral\-8x7B0\.6470\.6770\.7000\.5440\.8890\.8930\.8980\.607Mistral\-24B0\.6830\.7050\.7070\.5460\.9000\.9000\.9100\.564Gemma\-3\-4b0\.5900\.6340\.6490\.5100\.8960\.8950\.9040\.637Gemma\-3\-12b0\.6000\.6550\.6680\.5320\.9130\.9350\.9280\.462Phi\-4\-mini0\.6540\.7030\.6940\.5710\.9000\.8910\.9000\.480Mean0\.6690\.7060\.7180\.5390\.8790\.8790\.8830\.557

Table 4:Multi\-model AUROC \(TruthfulQA and HaluBench\)\. DRIFT\-concat: our ablation of DRIFT — same tap layers, concatenated hidden states, no inter\-layer difference features\. SC†= SelfCheckGPT\-NLI \(\+10\+10LLM passes\)\.
SAPLMA’s mean TruthfulQA AUROC \(0\.7060\.706\) marginally exceeds DRIFT \(0\.6690\.669\)\. DRIFT\-concat achieves means of0\.7180\.718and0\.8830\.883, matching both, indicating the primary signal lies in upper\-layer representations, while DRIFT’s inter\-layer differencing adds representational structure without consistently outperforming concatenation\. All three dominate the remaining eight baselines \(mean0\.440\.44–0\.550\.55, near chance\) and outperform SelfCheckGPT\-NLI at zero extra inference cost\. On RAGTruth, DRIFT \(0\.4710\.471\) and SAPLMA \(0\.4650\.465\) are statistically indistinguishable \(p=0\.82p\{=\}0\.82\)\. Ensemble ablation \(Appendix[G](https://arxiv.org/html/2605.17028#A7)\) confirms stacking A/C/E/F does not improve over the best single component: signal quality, not combination, is the bottleneck\.

The benchmark problem is pervasive\.When a hallucinated answer is concatenated to the prompt, internal states partly encode whether the text matches training\-data priors, not whether the model is uncertain\. Approaches B and F reach AUROC=1\.00=1\.00on HaluEval, whileTxTemb, a text\-similarity baseline with no access to model internals whatsoever, reaches0\.980\.98on the same corpus\. The0\.020\.02difference between these scores is within bootstrap confidence interval half\-widths and falls below our0\.050\.05artifact\-flagging threshold\. This means the internal\-state methods are not providing detection capability beyond what can be achieved by simply comparing surface text: their near\-perfect scores reflect the benchmark construction artifact, not a genuine model\-internal hallucination signal\. DRIFT and DRIFT\-concat also score above0\.940\.94on HaluEval, and a live\-generation re\-run confirms a0\.340\.34\-point drop \(Figure[2](https://arxiv.org/html/2605.17028#S3.F2)\)\. Importantly, MIND, HaloScope, ACT, SEPs, and HalluShift achieve near\-chance on HaluEval \(0\.450\.45–0\.560\.56\) even under teacher\-forcing: These baselines do not appear to exploit the artifact; rather, they appear to fail to detect hallucinations altogether\. Accordingly, published results on these benchmarks should be interpreted with this potential limitation in mind\.

On live\-generation corpora, the artifact\-correlation collapses and detection becomes genuinely hard: Approach A reaches0\.570\.57on RAGTruth, DRIFT reaches0\.470\.47, and the wider baseline field clusters between0\.430\.43and0\.560\.56\. TruthfulQA occupies an intermediate position \(TxTemb=0\.69=0\.69, verified gap of0\.040\.04\); the field should treat it as a minimum and RAGTruth\-style corpora as the standard\.

DRIFT and SAPLMA are the only consistently reliable internal\-state probes\.Among all internal\-state methods tested, only DRIFT, SAPLMA, and DRIFT\-concat \(a DRIFT ablation, not an independent method\) consistently exceed chance on TruthfulQA and HaluBench; MIND, HaloScope, ACT, SEPs, and HalluShift achieve near\-chance AUROC across all corpora, confirming that the April\-2026 re\-run with corrected official code is decisive\. DRIFT leads HalluShift by0\.300\.30points on TruthfulQA \(0\.7290\.729vs\.0\.4290\.429\), showing that learning*which*inter\-layer directions are diagnostic, not merely that some change occurred, is consequential\. Across twelve architectures DRIFT achieves mean0\.879±0\.0380\.879\\pm 0\.038on HaluBench \(ten of twelve models above0\.840\.84\); SAPLMA matches \(0\.878±0\.0370\.878\\pm 0\.037\), while output\-space methods provide only supplementary evidence in this comparison\.

Hallucinations are model\-probability mistakes\.Per\-token log\-probability analysis shows hallucinated and grounded responses are statistically indistinguishable at the output level \(TruthfulQA AUROC0\.540\.54; RAGTruth0\.510\.51; mean log\-prob diff<0\.01<0\.01nats\): the model assigns high probability to wrong tokens\(Guoet al\.,[2017](https://arxiv.org/html/2605.17028#bib.bib71)\), explaining why output\-space signals fail and why detection must operate on internal representations\.

Scale, self\-supervised detection, and failure modes\.DRIFT AUROC scales with model size on TruthfulQA \(Qwen2\.5\-72B:0\.8080\.808; Gemma\-3\-4b:0\.590\.59\), but scale is irrelevant on HaluBench \(all twelve models within±0\.07\\pm 0\.07\)\. Approach A is the only label\-free method exceeding chance on RAGTruth \(0\.5710\.571\), making it the practical choice when no annotations are available\. The Legal corpus is a consistent outlier \(every method at≈0\.50\{\\approx\}0\.50\), and poor cross\-corpus transfer \(Table[3](https://arxiv.org/html/2605.17028#S5.T3)\) confirms the signal is corpus\-idiosyncratic\.

Implications for detect\-then\-steer systems\.A probe trained on teacher\-forced data inherits the artifact and inverts the intended signal in live deployment\(Chenget al\.,[2025](https://arxiv.org/html/2605.17028#bib.bib28); Liet al\.,[2023b](https://arxiv.org/html/2605.17028#bib.bib13)\)\. Approach E and Approach A are the only methods immune to this, and𝐰E\\mathbf\{w\}\_\{E\}doubles as a contrastive steering direction\(Liet al\.,[2023b](https://arxiv.org/html/2605.17028#bib.bib13)\)\.

## 6Conclusion

Much of the field’s reported progress on hallucination detection is attributable to benchmark construction artifacts rather than genuine detection capability\. The teacher\-forcing artifact inflates AUROC by up to0\.430\.43points across four of six widely used corpora, and once controlled, the majority of established detectors collapse to near\-chance performance regardless of architecture or scale\. SAPLMA and DRIFT, both supervised probes on upper\-layer hidden states, reach AUROC0\.910\.91on HaluBench at zero extra inference cost, outperforming SelfCheckGPT\-NLI by over0\.120\.12points\. On the harder RAGTruth corpus every method scores between0\.430\.43and0\.570\.57, and Approach A is the only label\-free method exceeding chance on live\-generation data\. Reliable progress requires evaluation protocols that separate genuine internal\-state signals from the benchmark artifacts inflating reported performance\.

Limitations:Our results cover instruction\-tuned open\-weight models in English only; base models, proprietary APIs, and cross\-lingual settings\(MTRE Authors,[2025](https://arxiv.org/html/2605.17028#bib.bib55)\)remain untested\.Broader impacts:exposing benchmark artifacts redirects research toward methods that generalise; dual\-use risk is low as all methods and data are public\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya,et al\.\(2023\)GPT\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.2.5)\.
- E\. Asgariet al\.\(2025\)CREOLA: a framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation\.npj Digital Medicine\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-01670-7)Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- A\. Azaria and T\. Mitchell \(2023\)The internal state of an LLM knows when it’s lying\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Note:arXiv preprint arXiv:2304\.13734Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- N\. Balepuret al\.\(2025\)A single direction of truth: an observer model’s linear residual probe exposes and steers contextual hallucinations\.arXiv preprint arXiv:2507\.23221\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- Y\. Bar\-Shalomet al\.\(2025\)Beyond token probes: hallucination detection via activation tensors with ACT\-ViT\.arXiv preprint arXiv:2510\.00296\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- Y\. Belinkov \(2022\)Probing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2023\)Discovering latent knowledge in language models without supervision\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv preprint arXiv:2212\.03827Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- C\. Chenet al\.\(2024\)INSIDE: LLMs’ internal states retain the power of hallucination detection\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv preprint arXiv:2402\.03744Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- Z\. Cheng, Z\. Wang, Y\. Fu, Z\. Jiang, Y\. Yin, C\. Wang, and Q\. Gu \(2025\)Steering when necessary: flexible steering large language models with backtracking\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv preprint arXiv:2508\.17621Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1),[§5\.1](https://arxiv.org/html/2605.17028#S5.SS1.p16.1)\.
- Y\. Chuang, Y\. Xie, H\. Luo, Y\. Kim, J\. Glass, and P\. He \(2024\)DoLa: decoding by contrasting layers improves factuality in large language models\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv preprint arXiv:2309\.03883Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p2.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- M\. Dahl, V\. Magesh, M\. Suzgun, and D\. E\. Ho \(2024\)Large legal fictions: Profiling legal hallucinations in large language models\.Journal of Legal Analysis16,pp\. 64–93\.External Links:[Document](https://dx.doi.org/10.1093/jla/laae003)Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1),[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.5.1)\.
- S\. Dasgupta, D\. Garg, and M\. Gupta \(2025\)HalluShift: probing the geometry of hallucinations in large language models\.arXiv preprint arXiv:2503\.10702\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- I\. Dassenet al\.\(2026\)FACTUM: mechanistic detection of citation hallucination in long\-form RAG\.arXiv preprint arXiv:2601\.05866\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- X\. Du, C\. Gao, S\. Li, and Y\. Li \(2024\)HaloScope: harnessing unlabelled LLM generations for hallucination detection\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.37\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle,et al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§3](https://arxiv.org/html/2605.17028#S3.p1.1)\.
- N\. Dziri, X\. Lu, M\. Sclar, X\. L\. Li, L\. Jiang, B\. Y\. Lin, P\. West, C\. Bhagavatula, R\. Le Bras, J\. D\. Hwang,et al\.\(2023\)Faith and fate: limits of transformers on compositionality\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv preprint arXiv:2305\.18654Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- N\. Elhage, N\. Nanda, C\. Olsson, T\. Henighan, N\. Joseph, B\. Mann, A\. Askell, Y\. Bai, A\. Chen, T\. Conerly,et al\.\(2021\)A mathematical framework for transformer circuits\.Transformer Circuits Thread\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- S\. Farquhar, J\. Kossen, L\. Kuhn, and Y\. Gal \(2024\)Detecting hallucinations in large language models using semantic entropy\.Nature\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p2.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5),[§4](https://arxiv.org/html/2605.17028#S4.p9.4)\.
- T\. Fawcett \(2006\)An introduction to ROC analysis\.Pattern Recognition Letters27\(8\),pp\. 861–874\.Cited by:[§3\.1](https://arxiv.org/html/2605.17028#S3.SS1.p6.1)\.
- J\. Ferrando, O\. de Heredia, and A\. O\. Betancourt \(2025\)Do I know this entity? knowledge awareness and hallucinations in language models\.InInternational Conference on Learning Representations \(ICLR\),Note:Oral presentation\. arXiv preprint arXiv:2411\.14257Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- T\. Fujie, Y\. Nakamura, and H\. Goto \(2024\)PRISM: predicting reliability of statements in multilingual language models via truthfulness probing\.InarXiv preprint arXiv:2411\.04847,Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- C\. Guo, G\. Pleiss, Y\. Sun, and K\. Q\. Weinberger \(2017\)On calibration of modern neural networks\.InProceedings of the 34th International Conference on Machine Learning \(ICML\),Cited by:[§5\.1](https://arxiv.org/html/2605.17028#S5.SS1.p14.3)\.
- L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin,et al\.\(2023\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.arXiv preprint arXiv:2311\.05232\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1)\.
- Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, E\. Ishii, Y\. J\. Bang, A\. Madotto, and P\. Fung \(2023\)Survey of hallucination in natural language generation\.ACM Computing Surveys55\(12\),pp\. 1–38\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1)\.
- S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson,et al\.\(2022\)Language models \(mostly\) know what they know\.InNeurIPS Workshop on Trustworthy and Socially Responsible Machine Learning,Cited by:[Appendix I](https://arxiv.org/html/2605.17028#A9.p4.3),[§2](https://arxiv.org/html/2605.17028#S2.p2.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5),[Table 2](https://arxiv.org/html/2605.17028#S5.T2),[Table 2](https://arxiv.org/html/2605.17028#S5.T2.10.5)\.
- L\. Kuhn, Y\. Gal, and S\. Farquhar \(2023\)Semantic uncertainty: linguistic invariances for uncertainty estimation in natural language generation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- P\. Laban, T\. Schnabel, P\. N\. Bennett, and M\. A\. Hearst \(2022\)SummaC: re\-visiting NLI\-based models for inconsistency detection in summarization\.InTransactions of the Association for Computational Linguistics \(TACL\),Vol\.10\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- N\. Lee, W\. Ping, P\. Xu, M\. Patwary, P\. Fung, M\. Shoeybi, and B\. Catanzaro \(2022\)Factuality enhanced language models for open\-ended text generation\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems,Vol\.33\.Cited by:[§3\.1](https://arxiv.org/html/2605.17028#S3.SS1.p3.1)\.
- J\. Li, X\. Cheng, W\. X\. Zhao, J\. Nie, and J\. Wen \(2023a\)HaluEval: a large\-scale hallucination evaluation benchmark for large language models\.arXiv preprint arXiv:2305\.11747\.Note:GitHub: RUCAIBox/HaluEvalCited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1),[§2](https://arxiv.org/html/2605.17028#S2.p2.1),[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.2.1)\.
- K\. Li, O\. Patel, F\. Viégas, H\. Pfister, and M\. Wattenberg \(2023b\)Inference\-time intervention: eliciting truthful answers from a language model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Note:arXiv preprint arXiv:2306\.03341Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1),[§4](https://arxiv.org/html/2605.17028#S4.p6.2),[§5\.1](https://arxiv.org/html/2605.17028#S5.SS1.p16.1)\.
- H\. Lianget al\.\(2025\)Neural probe\-based hallucination detection for large language models\.arXiv preprint arXiv:2512\.20949\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- S\. Lin, J\. Hilton, and O\. Evans \(2022\)TruthfulQA: measuring how models mimic human falsehoods\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 3214–3252\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1),[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.4.1)\.
- T\. Liu, Y\. Zhang, C\. Brockett, Y\. Mao, Z\. Sui, W\. Chen, and B\. Dolan \(2022\)A token\-level reference\-free hallucination detection benchmark for free\-form text generation\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 6723–6737\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- A\. Mallen, A\. Asai, V\. Zhong, R\. Das, D\. Khashabi, and H\. Hajishirzi \(2023\)When not to trust language models: investigating effectiveness of parametric and non\-parametric memories\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- P\. Manakul, A\. Liusie, and M\. J\.F\. Gales \(2023\)SelfCheckGPT: zero\-resource black\-box hallucination detection for generative large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Note:arXiv preprint arXiv:2303\.08896Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p2.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- S\. Marks and M\. Tegmark \(2023\)The geometry of truth: emergent linear structure in large language model representations of true/false datasets\.arXiv preprint arXiv:2310\.06824\.Note:Published at COLM 2024Cited by:[Appendix D](https://arxiv.org/html/2605.17028#A4.p2.5),[§4](https://arxiv.org/html/2605.17028#S4.p3.2)\.
- J\. Maynez, S\. Narayan, B\. Bohnet, and R\. McDonald \(2020\)On faithfulness and factuality in abstractive summarization\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- MedHallBench Authors \(2024\)MedHallBench: a new benchmark for assessing hallucination in medical large language models\.arXiv preprint arXiv:2412\.18947\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1),[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.3.1)\.
- K\. Meng, D\. Bau, A\. Andonian, and Y\. Belinkov \(2022\)Locating and editing factual associations in GPT\.Advances in Neural Information Processing Systems35\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. W\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. Hajishirzi \(2023\)FactScore: fine\-grained atomic evaluation of factual precision in long form text generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- MTRE Authors \(2025\)MTRE: multi\-token reliability estimation for hallucination detection in VLMs\.arXiv preprint arXiv:2505\.11741\.Cited by:[§6](https://arxiv.org/html/2605.17028#S6.p2.1)\.
- R\. Ness, J\. Zhang, A\. Rao, and J\. Gryz \(2024\)MedFuzz: exploring the robustness of large language models in medical question answering\.arXiv preprint arXiv:2406\.06573\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- C\. Niu, Y\. Wu, J\. Zhu, S\. Xu, K\. Shum, R\. Zhong, J\. Song, and T\. Zhang \(2024\)RAGTruth: a hallucination corpus for developing trustworthy retrieval\-augmented language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 10862–10878\.Cited by:[§3\.1](https://arxiv.org/html/2605.17028#S3.SS1.p3.1),[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.6.1)\.
- F\. Niuet al\.\(2025\)Robust hallucination detection in LLMs via adaptive token selection\.arXiv preprint arXiv:2504\.07863\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- A\. Obesoet al\.\(2025\)Real\-time detection of hallucinated entities in long\-form generation\.arXiv preprint arXiv:2509\.03531\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- OpenAI \(2025\)HealthBench: a benchmark for evaluating language models on health\-related tasks\.arXiv preprint arXiv:2505\.08775\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1),[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- H\. Orgad, Y\. Belinkov,et al\.\(2025\)LLMs know more than they show: on the intrinsic representation of LLM hallucinations\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv preprint arXiv:2410\.02707Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[§3](https://arxiv.org/html/2605.17028#S3.p1.1)\.
- N\. Panickssery, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. M\. Turner \(2023\)Steering Llama 2 via contrastive activation addition\.arXiv preprint arXiv:2312\.06681\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- E\. Perez, S\. Ringer, K\. Lukošiūtė, K\. Nguyen, E\. Chen, S\. Heiner, C\. Pettit, C\. Olsson, S\. Kundu, S\. Kadavath,et al\.\(2022\)Red teaming language models with language models\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- F\. Petroni, T\. Rocktäschel, P\. Lewis, A\. Bakhtin, Y\. Wu, A\. H\. Miller, and S\. Riedel \(2019\)Language models as knowledge bases?\.Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- J\. Quevedoet al\.\(2024\)Token\-level hallucination detection via simple features\.arXiv preprint arXiv:2405\.19648\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- S\. S\. Ravi, B\. Mielczarek, A\. Kannappan, D\. Kiela, and R\. Qian \(2024\)Lynx: an open source hallucination evaluation model\.arXiv preprint arXiv:2407\.08488\.Cited by:[§3\.1](https://arxiv.org/html/2605.17028#S3.SS1.p3.1),[Table 1](https://arxiv.org/html/2605.17028#S3.T1.5.1.7.1)\.
- RepEng Survey Authors \(2025\)Representation engineering for large\-language models: survey and research challenges\.arXiv preprint arXiv:2502\.17601\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- W\. Shi, X\. Han, M\. Lewis, Y\. Tsvetkov, L\. Zettlemoyer, and S\. W\. Yih \(2024\)Trusting your evidence: hallucinate less with context\-aware decoding\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\),Cited by:[§3\.1](https://arxiv.org/html/2605.17028#S3.SS1.p3.1)\.
- A\. Slobodkin, O\. Goldman, A\. Caciularu, I\. Dagan, and S\. Ravfogel \(2024\)Curious case of hallucinations in LLMs via attribution in retrieval\-augmented generation\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p2.1)\.
- W\. Suet al\.\(2024\)MIND: unsupervised real\-time hallucination detection based on the internal states of large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,Note:arXiv preprint arXiv:2403\.06448Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p1.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- Z\. Sunet al\.\(2024\)ReDeEP: detecting hallucination in retrieval\-augmented generation via mechanistic interpretability\.arXiv preprint arXiv:2410\.11414\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- Survey Authors \(2025\)Medical hallucination in foundation models and their impact on healthcare\.arXiv preprint arXiv:2503\.05777\.Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1)\.
- I\. Tenney, D\. Das, and E\. Pavlick \(2019\)BERT rediscovers the classical NLP pipeline\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics \(ACL\),Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.\(2023\)LlaMa 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.Cited by:[§3](https://arxiv.org/html/2605.17028#S3.p1.1)\.
- A\. M\. Turner, L\. Thiergart, D\. Udell, G\. Leech, U\. Mini, and M\. Pelrine \(2024\)Activation addition: steering language models without optimization\.InProceedings of the AAAI Conference on Artificial Intelligence,Note:arXiv preprint arXiv:2308\.10248, 2023Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- Various Authors \(2025\)Hallucination detection with the internal layers of LLMs\.arXiv preprint arXiv:2509\.14254\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p1.1)\.
- A\. Wanget al\.\(2025a\)Adaptive activation steering: a tuning\-free LLM truthfulness improvement method\.InProceedings of the ACM Web Conference \(WWW\),Note:arXiv preprint arXiv:2406\.00034Cited by:[§1](https://arxiv.org/html/2605.17028#S1.p2.1),[§2](https://arxiv.org/html/2605.17028#S2.p3.1),[§4](https://arxiv.org/html/2605.17028#S4.p13.5)\.
- P\. Wanget al\.\(2025b\)CARES: a comprehensive benchmark of trustworthiness in medical vision language models\.arXiv preprint arXiv:2505\.11413\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in Neural Information Processing Systems35\.Cited by:[§3](https://arxiv.org/html/2605.17028#S3.p1.1)\.
- S\. Yanget al\.\(2025\)Detection and mitigation of hallucination in large reasoning models: a mechanistic perspective\.arXiv preprint arXiv:2505\.12886\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- Z\. Yinet al\.\(2025\)Dynamic multimodal activation steering\.arXiv preprint arXiv:2602\.21704\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.
- M\. Zhang, O\. Press, W\. Merrill, A\. Liu, and N\. A\. Smith \(2024\)How language model hallucinations can snowball\.InProceedings of the International Conference on Machine Learning \(ICML\),Note:arXiv preprint arXiv:2305\.13534Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p4.1)\.
- Y\. Zhang, Y\. Li, L\. Cui, D\. Cai, L\. Liu, T\. Fu, X\. Huang, E\. Zhao, Y\. Zhang, Y\. Chen,et al\.\(2023\)Siren’s song in the AI ocean: a survey on hallucination in large language models\.arXiv preprint arXiv:2309\.01219\.Note:GitHub: HillZhang1999/llm\-hallucination\-surveyCited by:[§1](https://arxiv.org/html/2605.17028#S1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. Xing,et al\.\(2023\)Judging LLM\-as\-a\-Judge with MT\-Bench and chatbot arena\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§3\.1](https://arxiv.org/html/2605.17028#S3.SS1.p3.1)\.
- A\. Zou, L\. Phan, S\. Chen, J\. Campbell, P\. Guo, R\. Ren, A\. Pan, X\. Yin, M\. Mazeika, A\. Dombrowski,et al\.\(2023\)Representation engineering: a top\-down approach to AI transparency\.arXiv preprint arXiv:2310\.01405\.Cited by:[§2](https://arxiv.org/html/2605.17028#S2.p3.1)\.

## Appendix AModel Suite

We evaluate on twelve instruction\-tuned models spanning six architectural families and a20×20\\timesparameter range \(3\.8B–72B\)\. The selection covers the major open\-weight families available on Hugging Face at the time of the study: the Llama\-3 series, two generations of Qwen, both DeepSeek\-R1 reasoning distillations \(which interleave chain\-of\-thought tokens with final answers\), two Mistral variants \(a dense MoE and a 24B dense model\), two Gemma\-3 models, and Phi\-4\-mini\. Llama\-3\.3\-70B serves as the primary model for which all 22 methods are evaluated; the remaining 11 run DRIFT, SAPLMA, DRIFT\-concat, and SelfCheckGPT\-NLI on TruthfulQA and HaluBench—the two live\-generation corpora where artifact\-free detection is meaningful\. We chose this coverage to test whether DRIFT’s cross\-architecture consistency claim generalises across attention variants, layer\-count differences \(18–80 layers\), and two reasoning model families \(DeepSeek\-R1\) that produce structured token sequences\.

Table 5:Model suite\. All loaded 4\-bit NF4\. TP = tensor parallelism for generation\.ModelFamilyParamsLayersHF identifierNotesLlama\-3\.3\-70BLlama70B80meta\-llama/Llama\-3\.3\-70B\-InstructPrimary; all methodsLlama\-3\.1\-8BLlama8B32meta\-llama/Llama\-3\.1\-8B\-InstructQwen3\-8BQwen8B36Qwen/Qwen3\-8BQwen3\-32BQwen32B64Qwen/Qwen3\-32BQwen2\.5\-72BQwen72B80Qwen/Qwen2\.5\-72B\-InstructTP=4DS\-R1\-Qwen\-7BDeepSeek7B28DeepSeek\-R1\-Distill\-Qwen\-7BReasoningDS\-R1\-Llama\-70BDeepSeek70B80DeepSeek\-R1\-Distill\-Llama\-70BReasoning; TP=4Mixtral\-8x7BMistral47B32Mixtral\-8x7B\-Instruct\-v0\.1MoEMistral\-24BMistral24B40Mistral\-Small\-3\.2\-24B\-InstructGemma\-3\-4bGemma4B18google/gemma\-3\-4b\-itGemma\-3\-12bGemma12B46google/gemma\-3\-12b\-itPhi\-4\-miniPhi3\.8B32microsoft/Phi\-4\-mini\-instruct

## Appendix BComputational Cost Benchmark

Tier 1 — Probe\-only shared extraction pass, no extra inference DRIFT\-logp2 msHaMI15 msSEPs89 msSAPLMA111 msDRIFT\-concat541 msDRIFT1\.2 sPrimary comparison\+LLMs = 0Tier 2 — GPU inference model loaded at test time DoLaLogprobP\(True\)PRISM\+LLMs = 0–1Reference tierTier 3 — Extra LLM calls per prompt SelfCheckGPT\-NLI\+10△\\triangle10×\\timesinference costDifferent cost classIncreasing test\-time cost

Method cost\-tier taxonomy\.Tier 1 \(probe\-only\) methods share one hidden\-state extraction pass; DRIFT and SAPLMA are the primary comparison\. Tier 2 requires the LLM loaded at test time\. Tier 3 \(SelfCheckGPT\-NLI\) incurs\+10\+10inference passes per prompt\.

![[Uncaptioned image]](https://arxiv.org/html/2605.17028v1/x2.png)

Accuracy–efficiency Pareto frontier \(TruthfulQA, Llama\-3\.3\-70B, warm cache\)\. SAPLMA offers best AUROC at moderate latency; DRIFT achieves near\-parity at6×6\\timeshigher latency\. DRIFT\-logp is the only sub\-10 ms option above chance\.

All methods in this paper share a single hidden\-state extraction pass per prompt: loading the model and hooks takes15\.615\.6s \(one\-time\), and thereafter extraction costs9\.69\.6ms per prompt\. After this shared pass, methods differ only in how they process the cached representations, and all probe\-based approaches \(Approaches A–F and all internal\-state baselines\) can be evaluated on the same stored activations without re\-running the model\. We report “warm” latency in Table[6](https://arxiv.org/html/2605.17028#A2.T6), which excludes the shared extraction overhead and measures only the per\-method probe computation\.

Three cost tiers emerge from the data\.Sub\-10 ms \(DRIFT\-logp, HaMI\):DRIFT\-logp’s log\-probability feature extraction adds only2\.12\.1ms and requires no labels; it is the only sub\-10 ms option above chance and is appropriate for real\-time streaming settings\.100 ms–2 s \(SAPLMA, DRIFT\-concat, DRIFT\):SAPLMA \(111 ms\) offers the best accuracy\-latency trade\-off in the probe tier; DRIFT\-concat \(541 ms\) is the next Pareto point; DRIFT \(1,191 ms\) incurs a large RAM footprint \(4,2434\{,\}243MB\) due to itsK​\(d\+2\)=49,164K\(d\+2\)=49\{,\}164\-dimensional feature vector\.GPU\-only \(DoLa, Logprob, P\(True\), SelfCheckGPT\-NLI\):These methods require the full model to remain loaded at test time and cannot amortise over cached hidden states, making them fundamentally less suitable for high\-throughput deployments\. SelfCheckGPT\-NLI’s\+10\+10LLM call requirement places it in a wholly different cost class: at 70B\-model generation speed, each test prompt requires approximately ten minutes of dedicated GPU time\.

Table 6:Test\-time overhead per method, measured on TruthfulQA with Llama\-3\.3\-70B \(N=1,634N\{=\}1\{,\}634\)\. “Warm” = hidden states pre\-loaded; “cold” includes the one\-time15\.615\.6s /9\.69\.6ms p\-1disk load\. GPU methods require full model inference at test time\.†Labels required for probe training only, not at inference\. All Approaches A–F share the DRIFT hidden\-state cache\.MethodAUROCWarm ms/pRAM \(MB\)\+LLMsLabels†Internal\-state probes \(no extra inference\)DRIFT\-logp0\.6152\.1110NoHaMI0\.60914\.66890YesACT \(detect\)0\.4911051610YesSEPs0\.520891080YesMIND0\.5261471360NoHalluShift0\.4292221650NoHaloScope0\.5093873430NoSAPLMA0\.7511116350YesDRIFT\-concat0\.7375412,5390YesLVA0\.55795310YesDVP0\.5801,80910YesDRIFT \(E\)0\.7291,1914,2430YesLT\-SVD0\.5453,37710YesGPU\-only methods \(full model inference at test time\)DoLa0\.530GPU—0NoLogprob0\.542GPU—0NoPRISM0\.628GPU—0YesP\(True\)0\.679GPU—1NoSelfCheckGPT\-NLI0\.558GPU—10No
## Appendix CAccuracy–Efficiency Pareto Frontier

Figure[4](https://arxiv.org/html/2605.17028#A3.F4)plots TruthfulQA AUROC against warm inference latency \(milliseconds per prompt, log scale\) for all internal\-state probe methods\. Three Pareto\-optimal points stand out\.DRIFT\-logp\(2 ms, AUROC0\.6150\.615\) is the only sub\-10 ms option above chance and is the right choice when latency is the primary constraint—for example, in token\-level streaming applications where the detector must fire within a single generation step\.SAPLMA\(111 ms, AUROC0\.7510\.751\) is the dominant choice for most deployments: it achieves the highest AUROC at moderate latency, and no other method offers a better accuracy/latency trade\-off in this regime\.DRIFT\-concat\(541 ms, AUROC0\.7370\.737\) is Pareto\-competitive with DRIFT but at2\.2×2\.2\\timeslower latency and3/5\\nicefrac\{\{3\}\}\{\{5\}\}of the RAM footprint\. DRIFT \(1,191 ms, AUROC0\.7290\.729\) lies just off the Pareto frontier: DRIFT\-concat dominates it on latency at comparable AUROC, though DRIFT retains its advantage as an interpretable steering direction\. Beyond DRIFT, LT\-SVD \(3\.43\.4s, AUROC0\.5450\.545\) offers no accuracy improvement and is dominated by all three Pareto\-optimal methods\.

![Refer to caption](https://arxiv.org/html/2605.17028v1/x3.png)Figure 4:Accuracy–efficiency Pareto frontier for internal\-state probes on TruthfulQA with Llama\-3\.3\-70B \(warm cache, hidden states pre\-loaded\)\. SAPLMA offers the best AUROC at moderate latency; DRIFT achieves near\-parity at6×6\\timeshigher latency owing to its4,2434\{,\}243MB feature cache\. DRIFT\-logp is the only method with sub\-10 ms latency at above\-chance AUROC\.
## Appendix DLayer Selection Ablation

We ablate the fractional depth of the four tap points used in DRIFT \(Approach E\) by evaluating single\-layer variants against the full four\-layer configuration on Llama\-3\.3\-70B \(L=80L\{=\}80\)\. Table[5](https://arxiv.org/html/2605.17028#A1.T5)lists total layer counts per model; fractional depths\{0\.60,0\.70,0\.80,0\.85\}×L\\\{0\.60,0\.70,0\.80,0\.85\\\}\\times Lare rounded to the nearest integer\.

Two patterns emerge from the single\-layer ablation\. First, AUROC increases monotonically with depth on HaluBench \(0\.7410\.741at 20%→\\to0\.8980\.898at 85%\), confirming that factuality signal concentrates in upper residual stream layers—consistent with the finding ofMarks and Tegmark \[[2023](https://arxiv.org/html/2605.17028#bib.bib14)\]that truth\-relevant directions reside in the upper third of the network\. Second, the four\-tap combination outperforms every single layer on both corpora \(\+0\.011\+0\.011on TruthfulQA,\+0\.017\+0\.017on HaluBench over the single best layer at 85%\), showing that the four selected layers capture complementary information not present in any individual layer\. The improvement is modest but robust across all ten random seeds, justifying the memory cost of maintaining four tap points instead of one\. These fractional depths were chosen before seeing any results and were not tuned per model; applying the same fractions to all twelve architectures in Table[4](https://arxiv.org/html/2605.17028#S5.T4)produces consistent gains\.

Table 7:Layer ablation for Approach E, Llama\-3\.3\-70B \(L=80L=80layers\)\. All four selected layers combined achieve the best AUROC on both corpora\.Layer\(s\) usedTruthfulQAHaluBenchLayer 16 \(20%\)0\.6110\.741Layer 32 \(40%\)0\.6430\.793Layer 48 \(60%\)0\.6940\.862Layer 56 \(70%\)0\.7090\.882Layer 64 \(80%\)0\.7180\.895Layer 68 \(85%\)0\.7110\.898All four: 60%\+70%\+80%\+85%0\.7290\.915
## Appendix EVerification Table: All AUROC\>0\.85\>0\.85Cells

For every cell in Table[2](https://arxiv.org/html/2605.17028#S5.T2)and Table[10](https://arxiv.org/html/2605.17028#A9.T10)with reported AUROC above0\.850\.85, we apply three independent checks\.

\(i\) Bootstrap 95% confidence interval\.We resample the test set with replacementn=1,000n\{=\}1\{,\}000times, evaluate the fitted probe on each resample, and report the 2\.5th/97\.5th percentile interval\. This measures stability to test\-set variation without refitting the probe\. A cell is not considered reliably high\-AUROC if the CI lower bound falls below0\.800\.80\.

\(ii\) Permutation null\.We shuffle the test labelsn=30n\{=\}30times, evaluate the probe on each shuffle, and report the mean AUROC\. A null mean near0\.500\.50confirms that the test\-set geometry is not biased toward any class and that the probe’s signal is not an artefact of class imbalance\.

\(iii\)TxTembgap\.We compute the difference between the method’s AUROC and theTxTembbaseline AUROC on the same corpus\. A gap below0\.050\.05\(positive or negative\) flags a potential artifact: the method is not adding value beyond what surface\-text similarity already achieves\.

Verdict categories\.*Validated*: CI lower bound\>0\.80\>0\.80andTxTembgap\>0\.05\>0\.05\.*Partial*: CI is sound but gap is below threshold \(method exceeds chance but may partly exploit text\-surface cues that correlate with the label\)\.*Artifact*\(†\): Both CI and gap indicate surface\-text exploitation; the high AUROC does not reflect genuine model\-internal hallucination signal\.

Only three cells survive as*Validated*: DRIFT, Approach F \(supervised\), and the A\+C\+E\+F ensemble on HaluBench\. The two HaluEval/MedHallu cells for Approach E are*Partial*becauseTxTemboutperforms them, meaning DRIFT is outperforming chance but*below*the text\-embedding baseline on these corpora\. Approaches B and F on teacher\-forced corpora are cleanly*Artifact*\.

Table 8:Verification for all AUROC\>0\.85\>0\.85cells on Llama\-3\.3\-70B\. Bootstrap 95% CI \(n=1000n=1000\), permutation null \(n=30n=30\),TxTembgap, verdict\.CorpusMethodAUROC95% CINullTxTembgapVerdictHaluBenchLayer Discrepancy \(E\)0\.915\[0\.889, 0\.938\]0\.500\+0\.144ValidatedHaluBenchApproach F \(supervised\)0\.966\[0\.948, 0\.980\]0\.497\+0\.195ValidatedHaluBenchEnsemble \(A\+C\+E\+F\)0\.956\[0\.935, 0\.974\]0\.498\+0\.185ValidatedHaluEvalApproach E0\.910\[0\.885, 0\.934\]0\.495−0\.065\-0\.065PartialMedHalluApproach E0\.890\[0\.862, 0\.916\]0\.502−0\.085\-0\.085PartialHaluEvalApproach A0\.970\[0\.953, 0\.984\]0\.495−0\.005\-0\.005†ArtifactHaluEvalApproach B1\.000\[1\.00, 1\.00\]0\.500−0\.025\-0\.025†ArtifactMedHalluApproach B1\.000\[1\.00, 1\.00\]0\.497−0\.291\-0\.291†Artifact

## Appendix FROC Curves

ROC curves complement the AUROC summary in Table[2](https://arxiv.org/html/2605.17028#S5.T2)by revealing the full operating\-point trade\-off\. On TruthfulQA \(a live\-generation corpus\), DRIFT, SAPLMA, and DRIFT\-concat separate cleanly from the chance diagonal while all other baselines remain close to it\. On RAGTruth,*all*methods collapse toward the diagonal, confirming that the0\.570\.57AUROC ceiling is a property of the corpus detection regime rather than an artefact of any single method\.

![Refer to caption](https://arxiv.org/html/2605.17028v1/x4.png)Figure 5:ROC curves for the six strongest methods on TruthfulQA \(left\) and RAGTruth \(right\) with Llama\-3\.3\-70B\. On TruthfulQA the top three methods — DRIFT, SAPLMA, DRIFT\-concat — are tightly clustered\. On RAGTruth all curves collapse toward the diagonal, confirming the0\.570\.57ceiling is a property of the corpus regime, not a single\-method artefact\.
## Appendix GStacker: Ensemble Results

We evaluate a self\-supervised ensemble that combines the outputs of Approaches A, C, E, and F using a nested5×55\{\\times\}5cross\-validation meta\-learner\. The four approaches share the same hidden\-state backbone but exploit complementary supervision signals: A uses perturbation\-generated pseudo\-labels, C uses semantic entropy estimated fromN=10N\{=\}10stochastic samples, E \(DRIFT\) uses ground\-truth hallucination annotations, and F uses teacher\-forced paired structure\. The stacker is a logistic regression trained on the four scalar outputs, with regularisation parameter grid\-searched overC∈\{0\.001,0\.01,0\.1,1\}C\\in\\\{0\.001,0\.01,0\.1,1\\\}in the outer CV\. The goal is to determine whether combining these complementary signals improves over the best individual component\.

Table 9:Self\-supervised ensemble \(Approaches A \+ C \+ E \+ F, no ground\-truth labels at training time\) vs\. best individual artifact\-free method\. Nested 5×\\times5 CV with grid search over regularisation parameterCC\.†artifact\-contaminated cell \(not a real gain\)\.CorpusBest individualEnsembleGainHaluEval0\.963 \(E\)0\.994 \(†\)\+0\.031\+0\.031MedHallu0\.548 \(E\)0\.979 \(†\)\+0\.431\+0\.431TruthfulQA0\.729 \(E\)0\.698−0\.031\-0\.031Legal0\.530 \(E\)0\.510−0\.020\-0\.020RAGTruth0\.571 \(A\)0\.509−0\.062\-0\.062HaluBench0\.915 \(E\)0\.956\+0\.041\+0\.041The large gains on teacher\-forced corpora \(HaluEval\+0\.031\+0\.031, MedHallu\+0\.431\+0\.431\) are artifact\-contaminated and do not reflect genuine improvement over the text\-embedding baseline; they are excluded from the practical conclusion\. On live\-generation corpora, the ensemble degrades:−0\.031\-0\.031on TruthfulQA and−0\.062\-0\.062on RAGTruth\. On HaluBench, where individual methods already achieve≥0\.91\\geq 0\.91, the ensemble adds\+0\.041\+0\.041by aggregating agreement among strong components\. The pattern is consistent with a noise\-amplification effect: on corpora where individual methods have low signal\-to\-noise \(TruthfulQA≈0\.73\\approx 0\.73, RAGTruth≈0\.57\\approx 0\.57\), combining outputs via logistic regression amplifies noise rather than signal\. We conclude that ensemble combination does not improve hallucination detection on the corpora where detection is meaningful, and that DRIFT or SAPLMA alone is the practical recommendation\.

Figure[6](https://arxiv.org/html/2605.17028#A7.F6)shows how both the Stacker and the best individual method perform as training set size is reduced\. On TruthfulQA, the best individual method is essentially flat fromn=130n\{=\}130\(10% of training data\) onward, while the Stacker needs roughlyn=326n\{=\}326\(25%\) to match it and never exceeds it\. On HaluBench, both are flat from the smallest evaluated fraction \(n=20n\{=\}20, 5%\), confirming that upper\-layer geometry in this corpus is learnable from very few examples\. On RAGTruth, both curves show high variance throughout—the wide shaded bands indicate that performance is unstable even at full training size \(n=400n\{=\}400\), reinforcing that this corpus is a hard detection regime regardless of ensemble strategy\.

![Refer to caption](https://arxiv.org/html/2605.17028v1/x5.png)Figure 6:Stacker \(A\+C\+E\+F\) vs\. best individual method AUROC as a function of training set size on TruthfulQA \(left\), HaluBench \(centre\), and RAGTruth \(right\)\. Shaded bands show±1\\pm 1standard deviation across 10 random seeds\. The best individual method matches or beats the Stacker at all training sizes on TruthfulQA and RAGTruth; on HaluBench both converge immediately\. The Stacker offers no reliable advantage and is more sensitive to small training sets \(note the wide bands and low AUROC atn=65n\{=\}65on TruthfulQA\)\.
## Appendix HAnnotation Budget

A practical deployment concern is the minimum number of labelled examples needed for a reliable probe\. We train DRIFT, SAPLMA, DRIFT\-concat, and LT\-SVD with sub\-sampled training sets of sizeN∈\{25,50,100,250,500,full\}N\\in\\\{25,50,100,250,500,\\text\{full\}\\\}, averaging over 10 random seeds perNN\. DRIFT and SAPLMA reach near\-peak performance atN=250N\{=\}250annotations, roughly 15% of the full TruthfulQA training set\. LT\-SVD does not reliably exceed chance belowN=500N\{=\}500, confirming that sparse SVD representations need more data to stabilise\. These results support the practical viability of DRIFT in settings where expert labelling is limited\.

![Refer to caption](https://arxiv.org/html/2605.17028v1/x6.png)Figure 7:AUROC vs\. training set sizeNNfor DRIFT, SAPLMA, DRIFT\-concat, and LT\-SVD on TruthfulQA \(Llama\-3\.3\-70B\)\. Results are 10\-seed averages with±1\\pm 1std shaded\. DRIFT and SAPLMA both reach near\-peak performance atN=250N\{=\}250; LT\-SVD does not reliably exceed chance belowN=500N\{=\}500\. The dotted vertical line marksN=250N\{=\}250\.
## Appendix IAblation Approaches and Extended Results

Table[10](https://arxiv.org/html/2605.17028#A9.T10)is the complete version of Table[2](https://arxiv.org/html/2605.17028#S5.T2)in the main text, including all six ablation approaches \(A–F\), the self\-supervised ensemble, and all sixteen baselines across all six corpora\. Several findings are easier to read in this consolidated view\.

Approaches B and F, which require teacher\-forced format, are not applicable to RAGTruth \(“—” cells\); on teacher\-forced corpora they reach AUROC1\.001\.00and0\.970\.97–0\.980\.98respectively, but these are artifact\-contaminated cells \(†\) where theTxTembgap is below0\.050\.05\.

Approach D \(Activation Variance Consistency\) achieves AUROC0\.270\.27on TruthfulQA \(‡\), which is dramatically below chance\. This is a genuine inverted signal: TruthfulQA hallucinations are confident myths that the model produces with*low*activation variance, opposite to the knowledge\-gap hallucinations on which variance\-based methods were designed\. Under threshold inversion the effective AUROC is0\.730\.73, matching DRIFT; we report the raw value and note the inversion in the caption\.

PRISM, HaMI, and SAPLMA are re\-implementations of external papers\. DRIFT\-concat \(ablation of Approach E\) and DRIFT\-logp are our own implementations introduced in this work; IRIS⋆applies the self\-verification concept fromKadavathet al\.\[[2022](https://arxiv.org/html/2605.17028#bib.bib72)\]\. HaMI achieves strong results on HaluEval \(0\.860\.86\) and HaluBench \(0\.870\.87\) but near\-chance elsewhere—a similar artifact sensitivity pattern to DRIFT on HaluEval\. PRISM, despite its cross\-attention architecture, does not consistently exceed MIND or HaloScope\.

Table 10:Complete AUROC table including all ablation approaches \(A–F\) and the ensemble, plus all baselines, on Llama\-3\.3\-70B\.†artifact detected:TxTembgap<0\.05<0\.05\.‡AVC \(D\) obtains below\-chance AUROC on TruthfulQA because TruthfulQA hallucinations are confident myths with*low*activation variance, inverting the signal direction relative to knowledge\-gap corpora; the effective AUROC under threshold inversion is0\.730\.73\.Bold= best artifact\-free result per corpus\. “—” = method not applicable \(B and F require teacher\-forced format for RAGTruth\)\. See main text Table[2](https://arxiv.org/html/2605.17028#S5.T2)for the subset of internal\-state methods\.MethodHaluEvalMedHalluTruthfulQALegalRAGTruthHaluBenchOur methodsA: Perturbation0\.540\.520\.540\.510\.570\.68B: CAA1\.00†1\.00†0\.87†0\.44——C: Sem\. Entropy0\.540\.500\.540\.500\.520\.62D: Act\. Variance0\.690\.570\.27‡0\.640\.780\.82E: DRIFT0\.960\.550\.730\.530\.470\.91DRIFT\-concat \(ablation of E\)0\.95†0\.570\.740\.520\.470\.91F: Ans\. Expect\.1\.00†1\.00†0\.660\.97†0\.570\.97DRIFT\-logp0\.870\.870\.620\.540\.520\.76†Ensemble A\+C\+E\+F0\.99†0\.99†0\.700\.520\.500\.96BaselinesINSIDE0\.500\.500\.590\.550\.500\.50SelfCheckGPT\-NLI0\.430\.550\.560\.550\.540\.54IRIS⋆\(self\-verify\)0\.500\.510\.660\.500\.500\.67MIND0\.480\.480\.530\.530\.510\.67HaloScope0\.500\.510\.510\.480\.510\.67HalluShift0\.450\.500\.430\.490\.520\.31DoLa0\.580\.570\.550\.500\.500\.62ACT \(detect\)0\.480\.520\.490\.490\.550\.62SEPs0\.560\.490\.520\.530\.490\.67Log\-Prob0\.490\.480\.510\.500\.520\.56P\(True\)0\.550\.580\.680\.500\.550\.60Re\-implementations of external papers \(added in this work\)PRISM0\.510\.450\.630\.540\.390\.80†HaMI0\.860\.520\.610\.500\.460\.87SAPLMA0\.540\.570\.750\.480\.460\.91TxTemb\(ctrl\)0\.980\.710\.690\.89n/a0\.77Best \(uncontaminated\)0\.960\.870\.750\.530\.570\.91
## Appendix JPerturbation Strategy Ablation

Approach A trains a contrastive probe onΔ​𝐡=𝐡​\(p,r\)−𝐡​\(p,r′\)\\Delta\\mathbf\{h\}=\\mathbf\{h\}\(p,r\)\-\\mathbf\{h\}\(p,r^\{\\prime\}\), wherer′r^\{\\prime\}is a corrupted response\. We ablate the four perturbation strategies \(entity swap, numerical corruption, negation flip, boundary violation\) by training on each strategy independently\. Combining all four strategies consistently outperforms any individual strategy, suggesting they corrupt complementary aspects of the hallucination signal\.

Table 11:Ablation of Approach A perturbation strategies, Llama\-3\.3\-70B\.Perturbation strategyRAGTruthTruthfulQAEntity swap only0\.5430\.672Numerical corruption only0\.5310\.651Negation flip only0\.5380\.665Boundary violation only0\.5290\.659All four \(Approach A\)0\.5710\.710
## Appendix KReproducibility Details

Software environment\.All experiments use Python 3\.10, PyTorch 2\.1,transformers4\.40,bitsandbytes0\.41,scikit\-learn1\.3, and vLLM 0\.4\. Hidden\-state extraction uses customregister\_forward\_hookhooks on the residual stream after each decoder layer\. Probes are fit withLogisticRegression\(C=1\.0, max\_iter=2000, random\_state=42\), preceded byStandardScaler\.

Hardware\.All experiments run on a server with 8×\\timesNVIDIA L40S GPUs \(46 GB each\)\. 70B\-class models use 4\-way tensor parallelism \(tp=4\); 7B–12B models run on a single GPU\. Models are loaded in 4\-bit NF4 quantisation with double quantisation enabled viabitsandbytes\.

Determinism\.All random seeds are fixed to42\. Probe fitting and train/test splits are fully deterministic given the seed\. vLLM sampling usestemperature=0\(greedy decoding\) for all generation passes to eliminate stochastic variation\.

Hidden\-state cache\.To avoid re\-running forward passes for each probe variant, hidden states are cached to disk infloat32\(shape:\[N, L, d\]\)\. Cached files for the primary model \(Llama\-3\.3\-70B on all six corpora\) total approximately 28 GB\. The anonymised supplement includes extraction scripts and cache\-loading utilities\.

Code availability\.Anonymised evaluation code with aREADMEis provided in the supplemental material, including scripts for hidden\-state extraction, all 22 detectors, probe training, and result aggregation\.