Diversity-Oriented Fine-Tuning for Uncertainty-Based Hallucination Detection

# Diversity-Oriented Fine-Tuning for Uncertainty-Based Hallucination Detection
Source: [https://arxiv.org/html/2607.16643](https://arxiv.org/html/2607.16643)
Qiuyuan Li1,2Hongliang Dai1,2Piji Li1,2 1College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China 2The Key Laboratory of Brain\-Machine Intelligence Technology, Ministry of Education, Nanjing, China \{qiuyli,hongldai,pjli\}@nuaa\.edu\.cn

###### Abstract

Existing hallucination detection methods are typically conducted at the inference stage, without making any modifications to the model itself\. In this paper, we are interested in exploring fine\-tuning strategies that enhance the detectability of hallucinations in the resulting model\. Focusing on semantic\-entropy\-based detection, we observe that many erroneous outputs remain undetected because the model produces nearly identical incorrect answers across multiple runs\. To address this, we propose diversity\-oriented fine\-tuning to encourage more varied generations\. We introduce two specific strategies: one based on Supervised Fine\-Tuning \(SFT\) and the other on Direct Preference Optimization \(DPO\)\. Extensive experiments are conducted to evaluate our approach and analyze the behavior of the models before and after fine\-tuning\. We find that after adopting our fine\-tuning methods, the models become less likely to produce low semantic entropy responses for hallucinated answers, thereby improving the effectiveness of hallucination detection, eventually yielding results better than or comparable with state of the art methods\. The code will be publicly released\.

Diversity\-Oriented Fine\-Tuning for Uncertainty\-Based Hallucination Detection

Qiuyuan Li1,2Hongliang Dai1,2††thanks:Corresponding author\.Piji Li1,21College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China2The Key Laboratory of Brain\-Machine Intelligence Technology, Ministry of Education, Nanjing, China\{qiuyli,hongldai,pjli\}@nuaa\.edu\.cn

![Refer to caption](https://arxiv.org/html/2607.16643v1/x1.png)Figure 1:Motivation of diversity\-oriented fine\-tuning\. A base LLM may repeatedly generate the same incorrect answer, yielding low semantic entropy and missed detection\. Diversity\-oriented fine\-tuning induces semantic variation among incorrect samples, increasing entropy and making such hallucinations easier to detect\.## 1Introduction

Despite significant advancements, Large Language Models \(LLMs\) remain susceptible to hallucinations: the generation of linguistically plausible yet factually erroneous or unsubstantiated outputsJiet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib45)\)\. This persistent challenge undermines their reliability and limits their utility in real\-world applicationsWeidingeret al\.\([2021](https://arxiv.org/html/2607.16643#bib.bib49)\)\. Given the considerable theoretical and practical challenges in fully eliminating hallucinationsXuet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib50)\), the reliable detection of hallucinatory outputs from LLMs has emerged as a vital research priorityManakulet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib15)\); Kuhnet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib26)\); Azaria and Mitchell \([2023](https://arxiv.org/html/2607.16643#bib.bib17)\)\.

While many hallucination detection methods have been proposedJiet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib45)\); Manakulet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib15)\), they are typically applied at the inference phase after model training is finalized\. Although these post\-hoc approaches are easy to deploy, their effectiveness can be limited by the fact that they operate on static models developed without regard for future hallucination detection\. Therefore, this work explores fine\-tuning strategies that can enhance the detectability of hallucinations in LLMs\. Specifically, we focus on improving the detection performance of the semantic entropy\-based approachFarquharet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib27)\)since it is well recognized\. This approach detects hallucinations by identifying semantic inconsistencies across multiple sampled responses\. Our preliminary experiments reveal that when using semantic entropy, a significant amount of hallucinations cannot be detected because the model exhibits high token\-level confidence and generates almost identical responses over independent runs\. This observation suggests that diversity\-oriented fine\-tuning, which encourages the model to generate more varied responses, could potentially trigger semantic inconsistencies for these cases and allow hallucinations to be correctly identified\.

We introduce two diversity\-oriented fine\-tuning strategies for hallucination detection\. In the first approach, Supervised Fine\-Tuning \(SFT\)Ouyanget al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib24)\)is applied using a one\-to\-many mapping, where each input prompt is paired with a set of semantically equivalent target outputs\. The second approach leverages Direct Preference Optimization \(DPO\)Rafailovet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib47)\)\. For a given input, it incentivizes the model to simultaneously produce multiple responses that exhibit high lexical diversity but share the same semantic meaning\. Afterwards, we adopt the framework proposed byFarquharet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib27)\)to identify hallucinations at the inference stage by measuring the semantic entropy across the model’s multiple responses\.

Extensive experiments are conducted to analyze our approach and evaluate its effectiveness\. The results demonstrate that both of our fine\-tuning strategies enable the model to generate more diverse outputs for a given input and enhance hallucination detection\. Our analysis shows that while diversity increases across all instances, the change in semantic consistencyKuhnet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib26)\)among the outputs deviates significantly between hallucinating and non\-hallucinating cases\. Specifically, the fine\-tuning amplifies semantic inconsistency in outputs primarily when the model is hallucinating, which effectively enhances the detection of previously false negative cases through semantic entropy\. Consequently, our approach achieves performance that is superior or comparable to current state\-of\-the\-art methods\.

Our main contributions as summarized follows:

- •To the best of our knowledge, we are the first to improve hallucination detection by operating at the fine\-tuning stage\.
- •We propose two diversity oriented fine\-tuning strategies to improve semantic\-entropy\-based hallucination detection by addressing cases where hallucinations go undetected because the model generates identical responses across runs\.
- •We conduct extensive experiments to analyze the behavior of our approach and demonstrate that it consistently boosts hallucination detection performance across diverse datasets\.

## 2Related Work

### 2\.1Hallucination Detection in Large Language Models

Hallucination remains a persistent fundamental challenge for large language models, especially in open\-domain and knowledge\-intensive generation\. Existing detection methods can be broadly categorized into three groups\. External\-verification approaches assess factuality by checking model outputs against retrieved evidence or external knowledge sourcesLewiset al\.\([2020](https://arxiv.org/html/2607.16643#bib.bib8)\); Minet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib9)\)\. While effective in constrained settings, their performance depends heavily on retrieval quality and external supervision\.

Uncertainty\-based and consistency\-based methods estimate response reliability from token\-level likelihoods, log\-probabilities, confidence estimates, or agreement across multiple sampled generationsKadavathet al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib12)\); Linet al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib13)\); Wanget al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib14)\); Manakulet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib15)\)\. These methods are attractive because they do not require explicit external evidence, but they can fail when models consistently repeat the same incorrect answer\.

Representation\-based methods probe hidden states, attention patterns, or activation statistics to identify hallucination\-related signalsAzaria and Mitchell \([2023](https://arxiv.org/html/2607.16643#bib.bib17)\); Burnset al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib18)\); Suet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib19)\)\. However, these approaches often require auxiliary classifiers or task\-specific supervision, increasing system complexity\.

### 2\.2Generation Diversity in Large Language Models

Generative diversity has long been studied in natural language generation, including dialogue, summarization, and open\-ended text generation\. Early approaches mainly rely on inference\-time strategies such as temperature scaling, top\-kk, and nucleus samplingFanet al\.\([2018](https://arxiv.org/html/2607.16643#bib.bib22)\); Holtzmanet al\.\([2019](https://arxiv.org/html/2607.16643#bib.bib21)\), which increase surface variation but may not produce meaningful semantic diversity in instruction\-following LLMs\.

Recent methods further improve diversity through contrastive decoding, which favors tokens preferred by an expert model while penalizing those favored by an interfering modelLiet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib29)\); Suet al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib30)\)\. Training\-stage objectives, such as SimCTGSuet al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib30)\), Unlikelihood TrainingWellecket al\.\([2019](https://arxiv.org/html/2607.16643#bib.bib31)\), and MMI\-based reweightingLiet al\.\([2016](https://arxiv.org/html/2607.16643#bib.bib32)\), also encourage diverse outputs, but often introduce additional training or inference complexity\.

Data\-centric methods such as Mix\-ReviewHeet al\.\([2021](https://arxiv.org/html/2607.16643#bib.bib33)\)mainly improve diversity through data augmentation or regularization\. However, these methods are not specifically designed to improve uncertainty\-based hallucination detection, which is the focus of our work\.

### 2\.3Semantic Uncertainty and Entropy\-Based Measures

Semantic uncertainty measures variation over meanings rather than surface forms by clustering generated responses into semantic equivalence classes and computing entropy over the resulting distributionKuhnet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib26)\)\. It is robust to paraphrasing and suitable for tasks with multiple valid answersFarquharet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib27)\)\. However, its effectiveness depends on sufficiently diverse samples: instruction\-tuned LLMs may produce limited semantic variation even when uncertain, yielding low or degenerate entropySharmaet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib34)\)\. Repeated sampling or aggressive decoding can increase diversity, but introduces additional cost and instabilityKossenet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib28)\)\.

## 3Preliminaries and Pilot Study

### 3\.1Preliminaries

Following existing studiesTonget al\.\([2025a](https://arxiv.org/html/2607.16643#bib.bib51),[b](https://arxiv.org/html/2607.16643#bib.bib52)\); Wanget al\.\([2025](https://arxiv.org/html/2607.16643#bib.bib35)\); Huet al\.\([2025](https://arxiv.org/html/2607.16643#bib.bib53)\), we focus on hallucination detection for question answering tasks\.

Semantic entropy identifies hallucinations by measuring the semantic consistency across multiple independent model generations\. For instance, when asked “What is the capital of Canada?”, if a model produces semantically divergent outputs, such as “Toronto”, “Montreal”, “It’s Vancouver”, and “The answer is Toronto”, the resulting high semantic entropy often indicates a likely hallucination\. Conversely, if the model consistently generates semantically equivalent responses, such as “Ottawa”, “Ottawa is the capital”, and “It’s Ottawa”, the answer is more likely to be correct\.

Based on this observation, this approach groups semantically equivalent answers together and measures uncertainty over meanings rather than token sequences\. Specifically, an incremental clustering approach based on Natural Language Inference \(NLI\) is adopted\. Given a questionqqand a sampled set of candidate answersA=\{a1,a2,…,aN\}A=\\\{a\_\{1\},a\_\{2\},\\ldots,a\_\{N\}\\\}, it iterates through each candidateaia\_\{i\}to assign it to a semantic cluster\. For each existing clusterCjC\_\{j\}represented byrjr\_\{j\}, it evaluates semantic equivalence using a bi\-directional entailment check via an NLI model\. A candidateaia\_\{i\}is integrated into clusterCjC\_\{j\}if and only if it mutually entails the representative\. If a candidate fails to satisfy this bi\-directional condition with all existing clusters, a new cluster is instantiated to accommodate it\. Following prior work on semantic uncertainty and semantic entropyKuhnet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib26)\); Farquharet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib27)\), we use incremental semantic clustering based on bi\-directional entailment; the complete procedure is summarized in Algorithm[1](https://arxiv.org/html/2607.16643#algorithm1)for completenessFarquharet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib27)\)\.

Finally, the entropy over the distribution of these semantic clusters is calculated\. Letp​\(Cj\)p\(C\_\{j\}\)denote the empirical proportion of generated sequences assigned to clusterCjC\_\{j\}\. The semantic entropy is defined as:

SE​\(q\)=−∑Cjp​\(Cj\)​log2⁡p​\(Cj\)\.\\mathrm\{SE\}\(q\)=\-\\sum\_\{C\_\{j\}\}p\(C\_\{j\}\)\\log\_\{2\}p\(C\_\{j\}\)\.A low SE score reflects strong consensus among the generated outputs, suggesting that most answers convey the same meaning despite surface\-level wording differences\. Conversely, a high SE score reflects substantial semantic divergence, where the model vacillates between factually incompatible answers, and therefore serves as an uncertainty signal for hallucination detection\.

![Refer to caption](https://arxiv.org/html/2607.16643v1/x2.png)Figure 2:Diversity\-oriented fine\-tuning data construction\. The SFT pipeline generates multiple semantically equivalent answers for each question, creating one training instance per diverse answer\. The DPO pipeline constructs preference pairs where high\-diversity answer sets are preferred over repetitive low\-diversity outputs, encouraging diverse yet semantically faithful generations\.
### 3\.2Pilot Study

By analyzing hallucination detection via semantic entropy, we find that many hallucinations go undetected \(false negatives\) because the model produces nearly identical responses across multiple independent runs\. Specifically, while using LLaMA3\-8B\-Instruct on the SQuAD dataset to generate 10 independent responses, the model generates identical outputs throughout these iterations for approximately 12% of hallucinations\. This phenomenon is exemplified in Table[9](https://arxiv.org/html/2607.16643#A3.T9)of Appendix[C\.2](https://arxiv.org/html/2607.16643#A3.SS2)\. Figure[1](https://arxiv.org/html/2607.16643#S0.F1)illustrates this failure mode\. When the base LLM repeatedly generates the same incorrect answer across multiple sampling runs, the sampled responses exhibit high self\-consistency and low semantic entropy\. As a result, the hallucinated answer is assigned a low uncertainty score and may be missed by consistency\-based hallucination detection\. This observation motivates diversity\-oriented fine\-tuning: by encouraging the model to produce more varied responses when it is uncertain, hallucinated cases are more likely to expose semantic inconsistency across samples, leading to higher semantic entropy and improved detectability\.

## 4Methodology

This section introduces our two diversity\-oriented fine\-tuning strategies: an SFT\-based approach which increases the variance of outputs across independent sampling runs; a DPO\-based strategy which optimizes the model to generate an output that consists of a diverse set of responses within a single inference step\. These two strategies can be integrated into a unified pipeline by performing SFT\-based tuning first, followed by DPO\.

### 4\.1Diversity Fine\-tuning with SFT

While supervised fine\-tuning typically optimizes for labeled input\-output pairs, our approach associates each prompt with multiple valid reference responses\. This explicitly encourages response diversity at the supervision level\. Formally, we construct the SFT dataset as

DSFT=\{\(xi,𝒴i\)\},D\_\{\\mathrm\{SFT\}\}=\\\{\(x\_\{i\},\\mathcal\{Y\}\_\{i\}\)\\\},wherexix\_\{i\}denotes an input prompt, and𝒴i=\{yi,1,…,yi,K\}\\mathcal\{Y\}\_\{i\}=\\\{y\_\{i,1\},\\ldots,y\_\{i,K\}\\\}represents a set ofKKsemantically equivalent but lexically diverse reference answers corresponding toxix\_\{i\}\. The SFT objective is defined as the average conditional log\-likelihood over all reference answers for each input:

maxθ​∑i1\|𝒴i\|​∑y∈𝒴ilog⁡πθ​\(y∣xi\),\\max\_\{\\theta\}\\sum\_\{i\}\\frac\{1\}\{\|\\mathcal\{Y\}\_\{i\}\|\}\\sum\_\{y\\in\\mathcal\{Y\}\_\{i\}\}\\log\\pi\_\{\\theta\}\(y\\mid x\_\{i\}\),whereπθ​\(y∣x\)\\pi\_\{\\theta\}\(y\\mid x\)denotes the conditional generation probability of the model parameterized byθ\\theta\.

We construct the diverse SFT dataset using a two\-step pipeline: iterative generation via in\-context learning and a subsequent verification phase, as illustrated in Figure[2](https://arxiv.org/html/2607.16643#S3.F2)\. For each input question, we initialize the process with the original answer and progressively generate multiple variants\. Each new generation is conditioned on the previously generated answers, encouraging lexical and structural diversity while ensuring semantic equivalence\. During the procedure, we increase the generation temperature linearly based on the number of variants produced, further promoting novel expressions\. After generating the candidate responses, we perform automated semantic consistency verification\. Using the LLM as a rigorous judge, we ensure that all variants maintain factual alignment with the original answer\. This procedure produces a set of responses per question that are semantically consistent while exhibiting substantial linguistic diversity, making it well suited for diversity\-oriented SFT\.

### 4\.2Diversity Fine\-tuning with DPO

We also design a DPO\-based approach inspired byLiet al\.\([2025](https://arxiv.org/html/2607.16643#bib.bib48)\)\. This method can be applied following our SFT\-based strategy, creating a comprehensive diversity\-oriented fine\-tuning pipeline\.

DPO optimizes the model directly on preference data\. Given a human preference dataset

D=\{\(xi,yi,1,yi,2\)\},D=\\\{\(x\_\{i\},y\_\{i,1\},y\_\{i,2\}\)\\\},whereyi,1y\_\{i,1\}represents the preferred response andyi,2y\_\{i,2\}represents the non\-preferred response, the DPO objective is defined as

maxπθ∑ilogσ\(β\[\\displaystyle\\max\_\{\\pi\_\{\\theta\}\}\\sum\_\{i\}\\log\\sigma\\biggl\(\\beta\\Bigl\[log⁡πθ​\(yi,1∣xi\)πref​\(yi,1∣xi\)\\displaystyle\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{i,1\}\\mid x\_\{i\}\)\}\{\\pi\_\{\\text\{ref\}\}\(y\_\{i,1\}\\mid x\_\{i\}\)\}−logπθ​\(yi,2∣xi\)πref​\(yi,2∣xi\)\]\)\\displaystyle\-\\log\\frac\{\\pi\_\{\\theta\}\(y\_\{i,2\}\\mid x\_\{i\}\)\}\{\\pi\_\{\\text\{ref\}\}\(y\_\{i,2\}\\mid x\_\{i\}\)\}\\Bigr\]\\biggr\)whereπθ​\(y∣x\)\\pi\_\{\\theta\}\(y\\mid x\)denotes the optimized policy,πref​\(y∣x\)\\pi\_\{\\text\{ref\}\}\(y\\mid x\)is a fixed reference policy,σ​\(⋅\)\\sigma\(\\cdot\)denotes the logistic sigmoid function, andβ\\betais a temperature parameter that controls the strength of the preference margin\. Therefore, DPO fine\-tuning relies on well\-paired datasets of preferred and non\-preferred responses to facilitate contrastive learning in LLMs\.

To promote diversity, we ensure a clear structural contrast: preferred outputs consist of multiple distinct responses, whereas non\-preferred outputs are composed of multiple repetitive ones\.Here, each DPO response is a single concatenated sequence representing a set of answers\. The preference is defined over answer sets rather than individual answers: the preferred sequence contains semantically equivalent but lexically diverse answers, whereas the rejected sequence contains repetitive answers\. Figure[2](https://arxiv.org/html/2607.16643#S3.F2)shows how we construct training instances for DPO\. For preferred outputs, we adopt the same iterative generation procedure used in the SFT\-based approach \(Section[4\.1](https://arxiv.org/html/2607.16643#S4.SS1)\)\. After obtaining the diverse set of candidate answers for each question, we combine them as the full output\. For non\-preferred outputs, we utilize a static reference strategy\. For each question, the model is repeatedly prompted with the ground\-truth answer as a fixed reference and instructed to generate another response that is highly similar in wording and structure\. Without conditioning on previously generated answers, the model is likely to produce highly homogeneous and repetitive responses\. These responses are concatenated to serve as an effective non\-preferred output\.

ModelsFine\-Tuning StrategiesDatasets \(AUROC\)TriviaQASQuADNQ\-OpenMistral\-7B\-InstructBaseline0\.7450\.7080\.657Standard SFT0\.7480\.7490\.660Diverse SFT0\.7700\.7100\.699Diverse DPO0\.7610\.6980\.673Diverse SFT \+ DPO0\.7720\.7550\.689LLaMA3\-8B\-InstructBaseline0\.7780\.7050\.676Standard SFT0\.7750\.7200\.745Diverse SFT0\.8070\.7490\.711Diverse DPO0\.8160\.7590\.746Diverse SFT \+ DPO0\.8100\.7490\.752Qwen2\.5\-7B\-InstructBaseline0\.7320\.6920\.715Standard SFT0\.8030\.7070\.720Diverse SFT0\.8050\.7480\.740Diverse DPO0\.8070\.7150\.738Diverse SFT \+ DPO0\.8390\.7470\.739Table 1:AUROC results of semantic\-entropy\-based hallucination detection across models, fine\-tuning strategies, and datasets\. “Baseline” directly applies the model without any fine\-tuning\. The best score for each dataset is highlighted inbold\.

## 5Experiments

### 5\.1Experimental Setup

#### Datasets\.

We evaluate our method on three question answering benchmarks:TriviaQAJoshiet al\.\([2017](https://arxiv.org/html/2607.16643#bib.bib41)\), which tests general knowledge;SQuADRajpurkaret al\.\([2016](https://arxiv.org/html/2607.16643#bib.bib40)\), a Wikipedia\-based reading comprehension dataset; andNQ\-OpenKwiatkowskiet al\.\([2019](https://arxiv.org/html/2607.16643#bib.bib42)\), an open\-domain subset of Natural Questions consisting of real user queries from Google Search\. Their validation sets contain 17,900, 10,600, and 3,610 examples, respectively\.

#### Models\.

Our experiments are conducted across three representative open\-source LLM models:LLaMA\-3\-8B\-InstructMetaet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib38)\),Mistral\-7B\-InstructJianget al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib44)\)andQwen2\.5\-7B\-instructAhmedet al\.\([2025](https://arxiv.org/html/2607.16643#bib.bib39)\)\.

#### Baselines\.

We compare hallucination detection performance with several baselines\. Consistency\-based methods includeSelfCheck\-BERTScore,SelfCheck\-Ngram, andSelfCheck\-PromptManakulet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib15)\), which estimate hallucination by measuring agreement among multiple sampled responses\. Likelihood\-based and confidence\-based methods includePerplexityandP\(True\)Kadavathet al\.\([2022](https://arxiv.org/html/2607.16643#bib.bib12)\)\. Representation\-based methods includeEmbedding RegressionZhanget al\.\([2025](https://arxiv.org/html/2607.16643#bib.bib37)\),ICR ProbeAzaria and Mitchell \([2023](https://arxiv.org/html/2607.16643#bib.bib17)\),EigenScoreChenet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib36)\), andEnergy Score\. Similarity\-based and entropy\-based methods includeLexical Similarity,Kernel Language EntropyKuhnet al\.\([2023](https://arxiv.org/html/2607.16643#bib.bib26)\), andDiscrete Semantic EntropyFarquharet al\.\([2024](https://arxiv.org/html/2607.16643#bib.bib27)\)\. Details of these baselines are provided in Appendix[A\.2](https://arxiv.org/html/2607.16643#A1.SS2)\.

ModelsMethodsTriviaQASQuADNQ\-OpenMistral\-7BSelfCheck\-BERTScore0\.6750\.6330\.625SelfCheck\-Prompt0\.6850\.5480\.588Embedding Regression0\.6810\.6080\.598EigenScore0\.7580\.7120\.664Energy Score0\.6790\.6670\.603Lexical Similarity0\.6500\.6400\.670P\(True\)0\.7240\.6190\.674Ours: SFT0\.7700\.7100\.699Ours: DPO0\.7610\.6980\.673Ours: SFT\+DPO0\.7720\.7550\.689LLaMA3\-8BPerplexity0\.6220\.6590\.709SelfCheck\-Ngram0\.5360\.6060\.650SelfCheck\-Prompt0\.7580\.6300\.690EigenScore0\.6410\.6040\.643Energy Score0\.6910\.5880\.645Lexical Similarity0\.6700\.5960\.652Embedding Regression0\.7100\.5900\.640Kernel Language Entropy0\.7600\.6700\.710Ours: SFT0\.8070\.7490\.711Ours: DPO0\.8160\.7590\.746Ours: SFT\+DPO0\.8100\.7490\.752Qwen2\.5\-7BSelfCheck\-Prompt0\.7070\.7110\.725EigenScore0\.6400\.6300\.669Energy Score0\.7250\.5910\.572Lexical Similarity0\.6520\.5320\.686ICR Probe0\.7580\.7450\.742Discrete Semantic Entropy0\.7660\.6460\.629Ours: SFT0\.8050\.7480\.740Ours: DPO0\.8070\.7150\.738Ours: SFT\+DPO0\.8390\.7470\.739Table 2:AUROC comparison grouped by different models and datasets\. Within each block, we compare existing hallucination detection methods with our diversity\-oriented variants\. Section 6\.1 offers an analysis of the NQ\-Open dataset, examining why our methods yield suboptimal results\.
#### Evaluation Metrics\.

We evaluate hallucination detection using AUROC\. Each instance is treated as positive if the low\-temperature answer is incorrect, and the detection score is the semantic entropy computed from multiple sampled responses\. The ROC curve is constructed by sweeping the decision threshold over the entropy scores used to classify an answer as hallucinated\.

### 5\.2Implementation Details

#### Answer Generation Setup\.

For each question, we employ a two\-stage generation strategy\. First, a single response is generated at a low temperature of 0\.05 to serve as the correctness reference\. Subsequently, we generateNNdiverse responses at a high temperature of 0\.8\. To do this efficiently, the decoder is configured to output99independent sequences in a single forward pass, avoiding repeated calls to the model\.

#### Fine\-tuning Setup\.

We employ a parameter\-efficient LoRA strategy for fine\-tuning\. For SFT fine\-tuning, the model is trained for 4 epochs on a maximum of 10000 samples, with a batch size of 2, gradient accumulation steps of 4, and a maximum sequence length of 2048 tokens\. Optimization uses the AdamW optimizer with a learning rate of1​e−41e\-4and a cosine learning rate scheduler, under fp16 precision\. For DPO fine\-tuning, we adopt a sigmoid preference loss withβ=0\.1\\beta=0\.1and optimize using AdamW with a learning rate of2​e−62e\-6, also paired with a cosine scheduler\. Training runs for 1 epoch with a per\-device batch size of 1 and gradient accumulation steps of 4\. All DPO experiments are conducted in fp16 precision with a maximum sequence length of 1024 tokens\. The prompts used for answer generation, consistency filtering, and data construction are provided in Appendix[A\.3](https://arxiv.org/html/2607.16643#A1.SS3)\.

#### Label Generation Strategy\.

We generate correctness labels with a two\-stage verification pipeline\. We first apply exact matching between the normalized low\-temperature answer and the gold answer\. If exact matching fails, we use DeBERTa\-large\-MNLI to determine whether the gold answer entails the low\-temperature answer under the question context\. The response is labeled as correct if either criterion is satisfied, and incorrect otherwise\.

## 6Results and Analysis

### 6\.1Main Results

Table[1](https://arxiv.org/html/2607.16643#S4.T1)presents a evaluation of our proposed fine\-tuning strategies for hallucination detection\. We can see that both of our approaches, Diverse SFT and Diverse DPO, as well as their combined variant \(Diverse SFT \+ DPO\), almost always outperforms the un\-fine\-tuned models \(Baseline\) and the models that conducts standard SFT\. This demonstrates the effectiveness of our methods\. Furthermore, we observe that combining Diverse SFT and DPO generally yields intermediate performance levels that fall between those of the two individual methods\.

![Refer to caption](https://arxiv.org/html/2607.16643v1/x3.png)\(a\)NQ\-Open
![Refer to caption](https://arxiv.org/html/2607.16643v1/x4.png)\(b\)SQuAD

Figure 3:Semantic entropy distributions across fine\-tuning stages on NQ\-Open and SQuAD\. Compared with pre\-SFT and standard SFT, diversity\-oriented fine\-tuning reduces false\-negative hallucinations below the 80% TNR threshold, making hallucinated responses easier to detect\.ModelsMethodsTriviaQASQuADNQ\-OpenMistralBaseline0\.6630\.2730\.328Std\.SFT0\.6200\.2480\.330SFT0\.6280\.3000\.338SFT\+DPO0\.6150\.2980\.340DPO0\.6900\.2750\.325LLaMABaseline0\.6930\.2700\.310Std\.SFT0\.5550\.2150\.268SFT0\.6080\.2600\.318SFT\+DPO0\.6100\.2680\.325DPO0\.6950\.2750\.325QwenBaseline0\.5880\.2680\.248Std\.SFT0\.4800\.2380\.240SFT0\.5180\.2630\.238SFT\+DPO0\.5280\.2380\.245DPO0\.5880\.2780\.243Table 3:Answer accuracy of different fine\-tuning strategies across models and datasets\.Table[2](https://arxiv.org/html/2607.16643#S5.T2)compares our approach with existing hallucination detection methods\. Our diversity\-enhanced model outperforms prior methods on most datasets\. However, semantic\-entropy\-based detection remains less effective for Mistral\-7B and LLaMA3\-8B on NQ\-Open\. Since NQ\-Open is an open\-domain benchmark, a question may have multiple semantically valid answers\. In this setting, semantic entropy can split correct but diverse answers into fragmented clusters, producing artificially high uncertainty and degrading detection performance\. Representative examples are provided in Appendix[C\.3](https://arxiv.org/html/2607.16643#A3.SS3)\.

### 6\.2Comparison with Inference\-Time Diversity Baselines

We also compare inference\-time diversity baselines, including higher\-temperature sampling, nucleus sampling with top\-p=0\.95p=0\.95, and diverse beam search\. Since our main baseline already uses high\-temperature sampling \(T=0\.8T=0\.8\) for semantic entropy computation, these experiments test whether stronger decoding\-time diversity can replace fine\-tuning\. Detailed experimental results are provided in Appendix[B\.2](https://arxiv.org/html/2607.16643#A2.SS2), where our fine\-tuned models consistently outperform the corresponding inference\-time diversity baselines\.

### 6\.3Analysis of Fine\-tuning Effects

We further analyze how our fine\-tuning strategy improves the detectability of hallucinations\. Figure[3](https://arxiv.org/html/2607.16643#S6.F3)visualizes the semantic entropy distributions on the NQ\-Open and SQuAD datasets before fine\-tuning, after standard SFT, and after diverse SFT\. We observe that diverse SFT increases semantic entropy overall; however, this increase is more pronounced in hallucinating cases\. Specifically, at a threshold yielding an 80% true negative rate, the proportion of hallucinating cases with semantic entropy below this threshold \(False Negatives\) drops substantially from 33\.7% to 19\.0% on NQ\-Open and from 30\.3% to 22\.4% on SQuAD after diverse SFT\. This disproportionate semantic impact on hallucinating versus non\-hallucinating cases effectively enhances detectability\. The examples in Table[8](https://arxiv.org/html/2607.16643#A3.T8)and[9](https://arxiv.org/html/2607.16643#A3.T9)of Appendix[C\.2](https://arxiv.org/html/2607.16643#A3.SS2)illustrate this phenomenon\.

### 6\.4Generalizability of Hallucination Detection across Models

We further examine whether the hallucination signals induced by diversity fine\-tuning are model\-specific or transferable across architectures\. Specifically, we test whether semantic entropy computed from a fine\-tuned model can predict hallucinations in responses generated by its un\-fine\-tuned counterpart\. This setting evaluates whether diversity\-oriented fine\-tuning learns a general uncertainty signal rather than merely overfitting to the fine\-tuned model’s own generations\. If the induced entropy signal remains predictive for base\-model outputs, it suggests that the method captures transferable patterns of semantic inconsistency associated with hallucinations\. Detailed AUROC results are provided in Appendix[B\.1](https://arxiv.org/html/2607.16643#A2.SS1)\.

### 6\.5Does Diversity\-Oriented Fine\-Tuning Increase Hallucinations?

We evaluate answer accuracy before and after diversity\-oriented fine\-tuning under the same inference setting\. As shown in Table[3](https://arxiv.org/html/2607.16643#S6.T3), accuracy remains generally stable across datasets, with some model\-specific variation\. SFT and SFT\+DPO reduce accuracy on TriviaQA, while DPO largely preserves or slightly improves it\. On SQuAD and NQ\-Open, accuracy is comparable to or slightly higher than the baseline, suggesting that diversity\-oriented fine\-tuning does not substantially harm short\-form QA performance\.

## 7Conclusion

In this work, we propose a diversity\-oriented fine\-tuning framework that integrates SFT and DPO to make semantic entropy a more reliable signal for hallucination detection\. Our method targets cases where a model repeatedly produces the same incorrect answer, causing low uncertainty and false\-negative detection\. Experiments on three datasets show that our approach achieves performance comparable to or better than state\-of\-the\-art methods\. Further analyses indicate that the gains come from making hallucinated cases more distinguishable, rather than merely increasing generation randomness or reducing answer accuracy\. We also show that the learned uncertainty signals transfer effectively to base models, demonstrating their practical utility for LLM hallucination detection\.

## Limitations

The proposed diversity\-oriented fine\-tuning approach improves semantic\-entropy\-based hallucination detection but has several limitations\. First, it relies on high\-temperature single\-pass decoding to generate multiple distinct responses, which is more efficient than iterative sampling but still incurs additional decoding overhead compared with standard greedy decoding\. Second, although our experiments span multiple factual question answering datasets, the evaluation still relies on gold\-answer matching and NLI\-based semantic equivalence rather than retrieval\-based external verification\. Extending evaluations to long\-form, open\-ended, and multi\-hop generation tasks with evidence\-grounded factuality metrics or retrieval\-based verification would provide a more comprehensive assessment of the robustness of our approach\.

## References

- I\. Ahmed, S\. Islam, P\. P\. Datta, I\. Kabir, N\. U\. R\. Chowdhury, and A\. Haque \(2025\)Qwen 2\.5: a comprehensive review of the leading resource\-efficient llm with potentioal to surpass all competitors\.Authorea Preprints\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px2.p1.1)\.
- A\. Azaria and T\. Mitchell \(2023\)The internal state of an llm knows when it’s lying\.arXiv preprint arXiv:2304\.13734\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p3.1),[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2022\)Discovering latent knowledge in language models without supervision\.arXiv preprint arXiv:2212\.03827\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p3.1)\.
- C\. Chen, K\. Liu, Z\. Chen, Y\. Gu, Y\. Wu, M\. Tao, Z\. Fu, and J\. Ye \(2024\)INSIDE: llms’ internal states retain the power of hallucination detection\.arXiv preprint arXiv:2402\.03744\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.
- A\. Fan, M\. Lewis, and Y\. Dauphin \(2018\)Hierarchical neural story generation\.arXiv preprint arXiv:1805\.04833\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p1.1)\.
- S\. Farquhar, J\. Kossen, L\. Kuhn, and Y\. Gal \(2024\)Detecting hallucinations in large language models using semantic entropy\.Nature630\(8017\),pp\. 625–630\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p2.1),[§1](https://arxiv.org/html/2607.16643#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.16643#S2.SS3.p1.1),[§3\.1](https://arxiv.org/html/2607.16643#S3.SS1.p3.7),[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.
- T\. He, J\. Liu, K\. Cho, M\. Ott, B\. Liu, J\. Glass, and F\. Peng \(2021\)Analyzing the forgetting problem in pretrain\-finetuning of open\-domain dialogue response models\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,pp\. 1121–1133\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p3.1)\.
- A\. Holtzman, J\. Buys, L\. Du, M\. Forbes, and Y\. Choi \(2019\)The curious case of neural text degeneration\.arXiv preprint arXiv:1904\.09751\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p1.1)\.
- J\. Hu, G\. Tu, S\. Cheng, J\. Li, J\. Wang, R\. Chen, Z\. Zhou, and D\. Shan \(2025\)HARP: hallucination detection via reasoning subspace projection\.arXiv preprint arXiv:2509\.11536\.Cited by:[§3\.1](https://arxiv.org/html/2607.16643#S3.SS1.p1.1)\.
- Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, E\. Ishii, Y\. J\. Bang, A\. Madotto, and P\. Fung \(2023\)Survey of hallucination in natural language generation\.ACM computing surveys55\(12\),pp\. 1–38\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p1.1),[§1](https://arxiv.org/html/2607.16643#S1.p2.1)\.
- A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de Las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed \(2023\)Mistral 7b\.ArXivabs/2310\.06825\.External Links:[Link](https://api.semanticscholar.org/CorpusID:263830494)Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px2.p1.1)\.
- M\. Joshi, E\. Choi, D\. S\. Weld, and L\. Zettlemoyer \(2017\)Triviaqa: a large scale distantly supervised challenge dataset for reading comprehension\.arXiv preprint arXiv:1705\.03551\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px1.p1.1)\.
- S\. Kadavath, T\. Conerly, A\. Askell, T\. Henighan, D\. Drain, E\. Perez, N\. Schiefer, Z\. Hatfield\-Dodds, N\. DasSarma, E\. Tran\-Johnson,et al\.\(2022\)Language models \(mostly\) know what they know\.arXiv preprint arXiv:2207\.05221\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p2.1),[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.
- J\. Kossen, J\. Han, M\. Razzak, L\. Schut, S\. Malik, and Y\. Gal \(2024\)Semantic entropy probes: robust and cheap hallucination detection in llms\.arXiv preprint arXiv:2406\.15927\.Cited by:[§2\.3](https://arxiv.org/html/2607.16643#S2.SS3.p1.1)\.
- L\. Kuhn, Y\. Gal, and S\. Farquhar \(2023\)Semantic uncertainty: linguistic invariances for uncertainty estimation in natural language generation\.arXiv preprint arXiv:2302\.09664\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p1.1),[§1](https://arxiv.org/html/2607.16643#S1.p4.1),[§2\.3](https://arxiv.org/html/2607.16643#S2.SS3.p1.1),[§3\.1](https://arxiv.org/html/2607.16643#S3.SS1.p3.7),[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.
- T\. Kwiatkowski, J\. Palomaki, O\. Redfield, M\. Collins, A\. Parikh, C\. Alberti, D\. Epstein, I\. Polosukhin, J\. Devlin, K\. Lee,et al\.\(2019\)Natural questions: a benchmark for question answering research\.Transactions of the Association for Computational Linguistics7,pp\. 453–466\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px1.p1.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p1.1)\.
- J\. Li, M\. Galley, C\. Brockett, J\. Gao, and W\. B\. Dolan \(2016\)A diversity\-promoting objective function for neural conversation models\.InProceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies,pp\. 110–119\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p2.1)\.
- X\. L\. Li, A\. Holtzman, D\. Fried, P\. Liang, J\. Eisner, T\. B\. Hashimoto, L\. Zettlemoyer, and M\. Lewis \(2023\)Contrastive decoding: open\-ended text generation as optimization\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: Long papers\),pp\. 12286–12312\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p2.1)\.
- Z\. Li, H\. Dai, and P\. Li \(2025\)Generating diverse training samples for relation extraction with large language models\.arXiv preprint arXiv:2505\.23108\.Cited by:[§4\.2](https://arxiv.org/html/2607.16643#S4.SS2.p1.1)\.
- S\. Lin, J\. Hilton, and O\. Evans \(2022\)Teaching models to express their uncertainty in words\.arXiv preprint arXiv:2205\.14334\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p2.1)\.
- P\. Manakul, A\. Liusie, and M\. Gales \(2023\)Selfcheckgpt: zero\-resource black\-box hallucination detection for generative large language models\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 9004–9017\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p1.1),[§1](https://arxiv.org/html/2607.16643#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p2.1),[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.
- A\. Meta, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.217832\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px2.p1.1)\.
- S\. Min, K\. Krishna, X\. Lyu, M\. Lewis, W\. Yih, P\. Koh, M\. Iyyer, L\. Zettlemoyer, and H\. Hajishirzi \(2023\)Factscore: fine\-grained atomic evaluation of factual precision in long form text generation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 12076–12100\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p3.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p3.1)\.
- P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. Liang \(2016\)Squad: 100,000\+ questions for machine comprehension of text\.arXiv preprint arXiv:1606\.05250\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px1.p1.1)\.
- M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. R\. Bowman, N\. Cheng, E\. Durmus, Z\. Hatfield\-Dodds, S\. R\. Johnston,et al\.\(2023\)Towards understanding sycophancy in language models\.arXiv preprint arXiv:2310\.13548\.Cited by:[§2\.3](https://arxiv.org/html/2607.16643#S2.SS3.p1.1)\.
- W\. Su, C\. Wang, Q\. Ai, Y\. Hu, Z\. Wu, Y\. Zhou, and Y\. Liu \(2024\)Unsupervised real\-time hallucination detection based on the internal states of large language models\.arXiv preprint arXiv:2403\.06448\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p3.1)\.
- Y\. Su, T\. Lan, Y\. Wang, D\. Yogatama, L\. Kong, and N\. Collier \(2022\)A contrastive framework for neural text generation\.Advances in Neural Information Processing Systems35,pp\. 21548–21561\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p2.1)\.
- C\. Tong, Q\. Zhang, J\. Gao, L\. Jiang, Y\. Liu, and N\. Sun \(2025a\)HaluNet: multi\-granular uncertainty modeling for efficient hallucination detection in llm question answering\.arXiv preprint arXiv:2512\.24562\.Cited by:[§3\.1](https://arxiv.org/html/2607.16643#S3.SS1.p1.1)\.
- C\. Tong, Q\. Zhang, L\. Jiang, Y\. Liu, N\. Sun, and W\. Li \(2025b\)Semantic reformulation entropy for robust hallucination detection in qa tasks\.arXiv preprint arXiv:2509\.17445\.Cited by:[§3\.1](https://arxiv.org/html/2607.16643#S3.SS1.p1.1)\.
- R\. Wang, Z\. Wei, G\. Yue, and M\. Sun \(2025\)Revisiting hallucination detection with effective rank\-based uncertainty\.arXiv preprint arXiv:2510\.08389\.Cited by:[§3\.1](https://arxiv.org/html/2607.16643#S3.SS1.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§2\.1](https://arxiv.org/html/2607.16643#S2.SS1.p2.1)\.
- L\. Weidinger, J\. Mellor, M\. Rauh, C\. Griffin, J\. Uesato, P\. Huang, M\. Cheng, M\. Glaese, B\. Balle, A\. Kasirzadeh,et al\.\(2021\)Ethical and social risks of harm from language models\.arXiv preprint arXiv:2112\.04359\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p1.1)\.
- S\. Welleck, I\. Kulikov, S\. Roller, E\. Dinan, K\. Cho, and J\. Weston \(2019\)Neural text generation with unlikelihood training\.arXiv preprint arXiv:1908\.04319\.Cited by:[§2\.2](https://arxiv.org/html/2607.16643#S2.SS2.p2.1)\.
- Z\. Xu, S\. Jain, and M\. Kankanhalli \(2024\)Hallucination is inevitable: an innate limitation of large language models\.arXiv preprint arXiv:2401\.11817\.Cited by:[§1](https://arxiv.org/html/2607.16643#S1.p1.1)\.
- Z\. Zhang, X\. Hu, H\. Zhang, J\. Zhang, and X\. Wan \(2025\)ICR probe: tracking hidden state dynamics for reliable hallucination detection in llms\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 17986–18002\.Cited by:[§5\.1](https://arxiv.org/html/2607.16643#S5.SS1.SSS0.Px3.p1.1)\.

## Appendix AExperimental Details

### A\.1Semantic Clustering Algorithm

We compute semantic entropy by clustering generated responses into semantically equivalent groups using an incremental bi\-directional entailment criterion\. Each answer is sequentially matched to an existing cluster based on mutual entailment with a cluster representative, otherwise, a new cluster is created\. Semantic entropy is then calculated from the resulting cluster size distribution, where higher entropy indicates greater semantic uncertainty\. Algorithm[1](https://arxiv.org/html/2607.16643#algorithm1)details the clustering procedure\.

1Input:Question

qq, List of generated answers

𝒜=\{a1,…,aN\}\\mathcal\{A\}=\\\{a\_\{1\},\\dots,a\_\{N\}\\\}
2Output:Semantic Entropy

HH
3Initialize clusters

𝒞←∅\\mathcal\{C\}\\leftarrow\\emptyset
4for*i←1i\\leftarrow 1toNN*do

5

ai←Preprocess​\(ai\)a\_\{i\}\\leftarrow\\text\{Preprocess\}\(a\_\{i\}\)
6

matched←False\\textit\{matched\}\\leftarrow\\text\{False\}
7for*Ck∈𝒞C\_\{k\}\\in\\mathcal\{C\}*do

8

rk←Ck​\[0\]r\_\{k\}\\leftarrow C\_\{k\}\[0\]
9if*BiDirectionalEntailment​\(q,ai,rk\)\\text\{BiDirectionalEntailment\}\(q,a\_\{i\},r\_\{k\}\)*then

10

Ck←Ck∪\{ai\}C\_\{k\}\\leftarrow C\_\{k\}\\cup\\\{a\_\{i\}\\\}
11

matched←True\\textit\{matched\}\\leftarrow\\text\{True\}
12break

13end if

14

15end for

16if*notmatched*then

17Create new cluster

Cnew←\{ai\}C\_\{\\text\{new\}\}\\leftarrow\\\{a\_\{i\}\\\}
18

𝒞←𝒞∪\{Cnew\}\\mathcal\{C\}\\leftarrow\\mathcal\{C\}\\cup\\\{C\_\{\\text\{new\}\}\\\}
19end if

20

21end for

22Compute probability distribution

P​\(Ck\)=\|Ck\|/NP\(C\_\{k\}\)=\|C\_\{k\}\|/N
23Compute Entropy

H=−∑P​\(Ck\)​log2⁡P​\(Ck\)H=\-\\sum P\(C\_\{k\}\)\\log\_\{2\}P\(C\_\{k\}\)
return

HH

Algorithm 1Incremental Semantic Clustering via Bi\-directional Entailment
### A\.2Baseline Methods

This section briefly describes the hallucination detection baselines used in our experiments\. These methods cover different uncertainty signals, including generation consistency, likelihood\-based confidence, representation geometry, and semantic uncertainty\.

#### SelfCheck Methods\.

SelfCheck\-GPT detects hallucinations by measuring consistency across multiple sampled responses\. We include SelfCheck variants based on n\-gram overlap, BERTScore similarity, and prompt\-based consistency checking\. These methods assume that factual answers are more consistent across generations, whereas hallucinated answers tend to show lower agreement\.

#### Perplexity\.

Perplexity uses token\-level likelihood as a confidence signal\. Responses with lower likelihood or higher perplexity are treated as more uncertain and therefore more likely to be hallucinated\.

#### P\(True\)\.

The P\(True\) estimator asks the model to estimate the probability that its generated answer is correct\. A lower estimated probability indicates higher hallucination risk\.

#### Embedding Regression\.

Embedding regression trains a lightweight predictor over hidden representations to estimate answer correctness\. It assumes that factual and hallucinated responses are separable in the model’s representation space\.

#### ICR Probe\.

ICR Probe detects hallucinations from internal model states\. It uses layer\-wise hidden\-state dynamics to train a probe for distinguishing correct and incorrect generations\.

#### EigenScore\.

EigenScore measures uncertainty through the geometry of hidden representations\. It computes dispersion statistics from hidden\-state embeddings across multiple generations, where larger representational divergence indicates higher uncertainty\.

#### Energy Score\.

Energy Score measures uncertainty from the token\-level predictive distribution\. We compute it from the logits along the decoding trajectory, where a larger energy value indicates lower model confidence and higher hallucination risk\.

#### Lexical Similarity\.

Lexical Similarity measures surface\-form agreement among sampled responses\. We compute the average lexical overlap across generations and use lower similarity as an indicator of higher uncertainty\.

#### Kernel Language Entropy\.

Kernel Language Entropy estimates uncertainty by computing entropy in a kernel\-induced space over multiple generations, capturing distributional variation among generated responses\.

#### Discrete Semantic Entropy\.

Discrete Semantic Entropy clusters generated answers into semantic equivalence classes and computes Shannon entropy over the resulting cluster distribution\. Unlike token\-level entropy, it measures uncertainty over meanings rather than surface forms\.

### A\.3Prompt and Data Generation Details

To improve reproducibility, we describe the prompts and decoding parameters used to construct SFT and DPO data\. Our data construction follows an iterative generation and verification pipeline\. The generation prompt include the question and previously generated answers, instructing the model to produce one new semantically consistent paraphrase at each iteration\. The generated variants are then verified by an LLM\-based consistency judge, and inconsistent answer sets are discarded\.

#### Prompt for answer generation\.

For answer generation with the fine\-tuned model, we use the following prompt\.

> You are a helpful assistant\. Use plain text phrases or sentences to provide answers\.

#### Iterative Diverse Answer Generation for SFT\.

For each question, we start from the original reference answer and iteratively generate additional answer variants\. At each step, the model is conditioned on the question and the previously generated answers, and the model is instructed to produce one new answer that is semantically consistent but phrased differently\.

> Task: Answer the following question based on previous examples\. Question: \{question\} Previous answers: \{existing\_examples\} Instruction: Generate ONE new answer that is semantically correct and consistent with the examples above, but phrased differently\. Try to use different sentence structures, synonyms, or formats compared to the previous answers\. IMPORTANT: Only output the answer itself\.

To encourage lexical diversity, we use a dynamic temperature schedule\. As more variants are generated, the decoding temperature is linearly increased from 0\.7 to 1\.1\. This encourages later generations to explore more diverse surface forms while remaining conditioned on the previous valid examples\.

#### LLM\-based Consistency Filtering\.

After generating the answer variants for a question, we apply an LLM\-based consistency check to verify that the generated answers are semantically equivalent and factually consistent\. The judge evaluates the entire answer set for one question rather than each answer independently\.

> You are a strict logic consistency judge\. Here is a question and a list of generated answers\. Question: \{question\} Generated Answers: \{answers\_block\} Task: Determine if ALL the answers above represent the same semantic meaning and are factually consistent with each other\. Minor phrasing differences \(e\.g\., ‘Steve Jobs’ vs ‘Steven Paul Jobs’\) are acceptable, but factual contradictions or entirely different entities are NOT allowed\. Please respond with exactly ‘YES’ if they are consistent, or ‘NO’ if they are not\. Do not provide explanations\.

The filtering decision is binary\. If the judge outputsNO, the entire generated answer set is discarded\. This filtering step ensures that the SFT data increases answer diversity without introducing factual inconsistency\.

#### DPO Preference Data Construction\.

For DPO training, we construct preference pairs that contrast diverse answer sets with repetitive answer sets\. Given a question and its reference answers, we use the first reference answer as the seed answer\. The preferred response is constructed by iteratively generating diverse paraphrases conditioned on all previously generated variants\. The rejected response is constructed by repeatedly prompting the model with only the seed answer and asking for an almost identical reformulation\. Thus, both responses are intended to preserve the same factual content, but the preferred response contains higher lexical and structural diversity, while the rejected response is more repetitive\.

#### Diverse Prompt for Preferred Responses\.

For constructing preferred responses in DPO training, we use the following prompt to generate semantically consistent but lexically diverse answer variants\.

> Question: \{question\} Previous answers: \{existing\_examples\} Instruction: Generate ONE new answer that conveys the SAME meaning as the answers above, remaining fully consistent with their factual content\. Rephrase it naturally using different wording, structure, or tone to ensure diversity in expression, but DO NOT change or add any information\. Output rules: \- Output ONLY the new answer text \(no ‘Answer:’, ‘Response:’, numbering, quotes, or explanations\)\. \- The answer must stay semantically identical to the given examples\. \- Focus on paraphrasing for variety\-\-\-change phrasing, not meaning\. \- Keep it concise, fluent, and in the same style as the examples\.

#### Repetitive Prompt for Rejected Responses\.

For constructing rejected responses in DPO training, we use the following prompt to generate repetitive answer variants that preserve the same meaning but differ only minimally in wording\.

> Question: \{question\} Existing answers: \{seed\_answer\} Instruction: Generate ONE answer that is semantically correct and consistent with the examples above, but phrased almost identically, with very minor variations\. The answer should be the same in content and meaning, but with small differences in wording or structure\. Only output the answer itself\.

#### Hyperparameters\.

As shown in Table[4](https://arxiv.org/html/2607.16643#A1.T4), we use the following parameters during data generation\.

ParameterValueVariants per question9Max\. DPO samples1000Max\. generation tokens100Decoding strategynucleus samplingSFT top\-pp0\.95DPO top\-pp0\.90SFT temperature0\.7–1\.1DPO temperature0\.7Table 4:Hyperparameters used for SFT and DPO data generation\.

## Appendix BAdditional Results

### B\.1Cross\-Model Transferability Analysis

To examine whether hallucination signals induced by diversity fine\-tuning are model\-specific or transferable, we report cross\-model hallucination detection results in Table[5](https://arxiv.org/html/2607.16643#A2.T5)\. The results show that the method exhibits consistent transferability across multiple datasets and base models, indicating that diversity fine\-tuning induces more generalized uncertainty signals rather than overfitting to a specific model\.

![Refer to caption](https://arxiv.org/html/2607.16643v1/x5.png)\(a\)Before Fine\-Tuning
![Refer to caption](https://arxiv.org/html/2607.16643v1/x6.png)\(b\)After Standard SFT
![Refer to caption](https://arxiv.org/html/2607.16643v1/x7.png)\(c\)After Diversity Fine\-Tuning

Figure 4:Token entropy distributions on the SQuAD dataset before fine\-tuning, after standard supervised fine\-tuning, and after diversity\-oriented fine\-tuning\.ModelsTriviaQASQuADNQ\-OpenMistral\-7B\-Instruct0\.6370\.6160\.710LLaMA3\-8B\-Instruct0\.7030\.6650\.668Qwen2\.5\-7B\-Instruct0\.7440\.7180\.768Table 5:AUROC of transferring hallucination judgments from fine\-tuned models to base models\.MethodQwen2\.5\-7BMistral\-7BMistral\-7BLLaMA3\-8BNQ\-OpenSQuADNQ\-OpenNQ\-OpenBase0\.6990\.6530\.6780\.641Top\-pp0\.7270\.739––DBS\-M0\.6860\.6460\.6640\.679DBS\-H0\.7330\.729––Ours0\.7740\.7650\.7040\.714Table 6:Comparison with inference\-time diversity baselines\. Top\-ppuses nucleus sampling withp=0\.95p=0\.95\. DBS\-M and DBS\-H denote moderate and high\-diversity beam search, respectively\. Ours reports the best diversity\-oriented fine\-tuned model for each setting\.ModelDatasetHigh\-Temperature Baseline \(T=1\.0T=1\.0\)Aggressive Decoding \(T=1\.2T=1\.2,N=12N=12\)BaseOursGainBaseOursGainQwenSQuAD0\.6460\.810SFT\+DPO\{\}\_\{\\text\{SFT\+DPO\}\}\+0\.1640\.6890\.700SFT\{\}\_\{\\text\{SFT\}\}\+0\.011NQ\-Open0\.7230\.774SFT\{\}\_\{\\text\{SFT\}\}\+0\.0510\.6760\.775SFT\+DPO\{\}\_\{\\text\{SFT\+DPO\}\}\+0\.099TriviaQA0\.7670\.769DPO\{\}\_\{\\text\{DPO\}\}\+0\.0020\.7260\.807SFT\+DPO\{\}\_\{\\text\{SFT\+DPO\}\}\+0\.081MistralSQuAD0\.6790\.765SFT\+DPO\{\}\_\{\\text\{SFT\+DPO\}\}\+0\.0860\.7020\.758SFT\{\}\_\{\\text\{SFT\}\}\+0\.056NQ\-Open0\.6900\.706DPO\{\}\_\{\\text\{DPO\}\}\+0\.0160\.6800\.741DPO\{\}\_\{\\text\{DPO\}\}\+0\.061TriviaQA0\.7530\.782DPO\{\}\_\{\\text\{DPO\}\}\+0\.0290\.7230\.802DPO\{\}\_\{\\text\{DPO\}\}\+0\.079LLaMASQuAD0\.7020\.750SFT\{\}\_\{\\text\{SFT\}\}\+0\.0480\.7140\.762DPO\{\}\_\{\\text\{DPO\}\}\+0\.048NQ\-Open0\.6790\.714SFT\{\}\_\{\\text\{SFT\}\}\+0\.0350\.6830\.748DPO\{\}\_\{\\text\{DPO\}\}\+0\.065TriviaQA0\.7530\.800SFT\+DPO\{\}\_\{\\text\{SFT\+DPO\}\}\+0\.0470\.7440\.774SFT\+DPO\{\}\_\{\\text\{SFT\+DPO\}\}\+0\.030Table 7:Comparison with high\-temperature inference\-time baselines\. The left block reports results with un\-fine\-tuned models using sampling temperatureT=1\.0T=1\.0\. The right block reports aggressive high\-diversity decoding withT=1\.2T=1\.2andN=12N=12generations\. “Ours” reports the best diversity\-oriented fine\-tuned variant under the corresponding evaluation setting\. Gain is computed against the corresponding un\-fine\-tuned baseline\.
### B\.2Comparison with Inference\-Time Diversity Baselines

We further compare diversity\-oriented fine\-tuning with simpler inference\-time strategies for increasing generation diversity\. This experiment addresses whether the improvements of our method can be reproduced by directly increasing decoding randomness or by using diversity\-oriented decoding algorithms at inference time\.

We consider three types of inference\-time baselines\. First, we increase the sampling temperature beyond the default setting used in the main experiments\. The main baseline uses high\-temperature sampling withT=0\.8T=0\.8andN=9N=9sampled responses for semantic entropy computation\. We further evaluate more aggressive sampling settings, includingT=1\.0T=1\.0andT=1\.2T=1\.2, while keeping the number of sampled generations fixed when applicable\. Second, we evaluate nucleus sampling with top\-p=0\.95p=0\.95\. Third, we evaluate diverse beam search \(DBS\), using both a moderate\-diversity setting with 10 beams, 5 beam groups, and diversity penalty 0\.5, and a high\-diversity setting with 10 beams, 10 beam groups, and diversity penalty 1\.0\.

The results are summarized in Tables[7](https://arxiv.org/html/2607.16643#A2.T7)and[6](https://arxiv.org/html/2607.16643#A2.T6)\. Increasing inference\-time diversity improves the baseline in some settings, confirming that semantic entropy benefits from more varied generations\. However, in many cases, our fine\-tuned models achieve substantially higher AUROC than the corresponding high\-temperature or diversity\-decoding baselines\. This indicates that the gains of our method cannot be explained solely by increased sampling randomness\. Instead, diversity\-oriented fine\-tuning changes the model’s conditional generation behavior, making hallucinated cases more likely to reveal semantic inconsistency while preserving semantic stability for correct cases\.

## Appendix CCase Studies and Error Analysis

### C\.1Additional Pilot Studies On SQuAD

Figure[4](https://arxiv.org/html/2607.16643#A2.F4)presents a comparative analysis of token entropy distributions on the SQuAD dataset under three strategies: \(a\) Before Fine\-Tuning \(b\) Standard SFT and \(c\) our proposed Diversity\-Oriented Fine\-Tuning\. All results are evaluated at a fixed True Negative Rate of 80%, enabling a direct comparison of uncertainty calibration across training regimes\.

After Standard SFT, the entropy distributions of TP, FP, and FN show substantial overlap with the TN distribution\. Notably, the TP and TN distributions are poorly separated, limiting the effectiveness of token entropy as a standalone signal for distinguishing correct responses from hallucinations\. The resulting uncertainty threshold isτ=1\.28\\tau=1\.28\.

In contrast, Diversity\-Oriented Fine\-Tuning further elevates overall token entropy, with the TN median rising to approximately 0\.35, indicating stronger diversity effects\. Crucially, this strategy preserves and improves distributional separability\. As shown in Figure[4](https://arxiv.org/html/2607.16643#A2.F4)\(c\), the TP distribution is centered at a higher entropy level \(around 0\.45\) than the TN distribution, while FP responses are also shifted toward higher uncertainty regions\. This improved separation is reflected in a higher uncertainty threshold ofτ=1\.68\\tau=1\.68, indicating that the model can accommodate greater intrinsic uncertainty while still correctly identifying 80% of valid responses\. These results demonstrate that our approach enhances expressive diversity without collapsing the uncertainty signal required for effective hallucination detection\.

### C\.2Qualitative Case Studies of Diversity\-Oriented Fine\-Tuning

Table[8](https://arxiv.org/html/2607.16643#A3.T8)and[9](https://arxiv.org/html/2607.16643#A3.T9)provides a qualitative comparison illustrating the impact of fine\-tuning of diversity on model generative behavior\.

Before Fine\-TuningAfter Fine\-TuningQuestion:Electrolysis of what can be used to produce oxygen and hydrogen?Reference Answer:Water\.Low\-Temperature Answer Before Fine\-Tuning:Water\.Low\-Temperature Answer After Fine\-Tuning:Water electrolyte pair\.1\. Water\.1\. Distilled water electrolysis for oxygen and hydrogen production\.2\. Water\.2\. Water solution\.3\. Water\.3\. Water electrolysis yields oxygen and hydrogen\.4\. Water\.4\. Distilled water electrolysis for generating oxygen and hydrogen\.5\. Water\.5\. Water electrolyzed\.6\. Water\.6\. Water electrolysis for generating oxygen and hydrogen\.7\. Water\.7\. Distilled water electrolyzed yields pure oxygen and hydrogen gases\.8\. Water\.8\. Water molecules\.9\. Water\.9\. Water electrolyzed\.Table 8:Correct sample from SQuAD\. Before diversity\-oriented fine\-tuning, the model repeating the exact same answer\. After diversity\-oriented fine\-tuning, the model generates diverse formulations while maintaining semantic correctness relative to the reference\.Before Fine\-TuningAfter Fine\-TuningQuestion:What color were the Broncos’ uniforms in Super Bowl 50?Reference Answer:White\.Low\-Temperature Answer Before Fine\-Tuning:Orange\.Low\-Temperature Answer After Fine\-Tuning:The Broncos’ attire at Super Bowl 50 was crimson\.1\. Orange\.1\. Silver\-and\-blue scheme adopted by the Broncos for Super Bowl 50\.2\. Orange\.2\. Deep blue jerseys and pants with white socks\.3\. Orange\.3\. Black\-and\-gold Broncos attire\.4\. Orange\.4\. Silver\-and\-white ensembles\.5\. Orange\.5\. Deep blue Broncos attire\.6\. Orange\.6\. Crimson\-and\-gold scheme for the event\.7\. Orange\.7\. Silver\-and\-gold ensembles\.8\. Orange\.8\. Silver\-and\-gold combination\.9\. Orange\.9\. Black\-and\-gold Broncos attire\.Table 9:Hallucinated sample from SQuAD\. Before diversity\-oriented fine\-tuning, the model repeatedly gave the same incorrect answers\. After diversity\-oriented fine\-tuning, although the generated answers remain incorrect, they exhibit greater semantic dispersion, breaking the previously high semantic consistency and thereby improving the detectability of hallucinations\.
### C\.3Case Studies of Semantic Entropy on Open\-Domain QA

This appendix presents representative NQ\-Open examples to explain why semantic\-entropy\-based hallucination detection performs worse on open\-domain QA\. Since NQ\-Open questions may have multiple semantically valid answers with different surface forms, high\-temperature generations can remain factually plausible while exhibiting high semantic diversity\. As shown in Tables[10](https://arxiv.org/html/2607.16643#A3.T10)–[13](https://arxiv.org/html/2607.16643#A3.T13), low\-temperature answers are often consistent with the reference, but their sampled high\-temperature variants may be split into multiple semantic clusters\. This fragmentation inflates entropy even for non\-hallucinated outputs, leading to lower AUROC on NQ\-Open, especially for Mistral\-7B and LLaMA3\-8B\.

Question:what is the name of india pakistan border?Reference Answer:International Border\.Low\-Temperature Answer:The name of the India\-Pakistan border is the Line of Control \(LoC\) in Kashmir and the International Border \(IB\) elsewhere\.High\-Temperature Answers:1\. The name of the border between India and Pakistan is the Line of Control \(LoC\)\.2\. The name of the India\-Pakistan border is the Line of Control \(LoC\) in Kashmir and the International Border \(IB\) elsewhere\.3\. It is called the Line of Control \(LoC\) or the Working Boundary \(WB\) in some areas\.4\. The name of the India\-Pakistan border is the Line of Control \(LoC\) in Kashmir and the International Border elsewhere\.5\. The name of the India\-Pakistan border is the Line of Control \(LoC\) in Kashmir and the International Border \(IB\) elsewhere\.6\. India\-Pakistan Border, also known as the Line of Control \(LoC\), is a de facto border that separates the disputed territory of Kashmir between India and Pakistan\.7\. The name of the border between India and Pakistan is the Line of Control \(LoC\)\.8\. The name of the India\-Pakistan border is the Line of Control \(LoC\) in Kashmir, and the International Border elsewhere\.9\. The name of the India\-Pakistan border is the Line of Control \(LoC\) in Kashmir and the International Border \(IB\) elsewhere\.Table 10:Example 1 from NQ\-Open under Mistral\-7B\. Labeled as non\-hallucination, high semantic entropy\.Question:what was the main characteristic of post\-world war ii american society?Reference Answer:Strong economic growth\.Low\-Temperature Answer:Post\-World War II American society was characterized by a period of economic prosperity, known as the “Golden Age of Capitalism\.” This was due to the implementation of Keynesian economic policies\.High\-Temperature Answers:1\. After World War II, American society underwent significant changes, including economic growth, suburbanization, and the rise of consumer culture\.2\. After World War II, American society experienced a period of prosperity and growth, often referred to as the “Golden Age\.”3\. Post\-World War II American society was characterized by economic prosperity, cultural expansion, and political conservatism\.4\. The main characteristic of post\-World War II American society was a period of economic prosperity, known as the “Golden Age of Capitalism,” due to the expansion of consumer goods, the growth of the suburbs, and the rise of the middle class\.5\. Post\-World War II American society was characterized by a period of economic growth, known as the post\-war economic boom\.6\. After World War II, American society experienced a period of prosperity and growth known as the postwar economic boom\.7\. The main characteristic of post\-World War II American society was a period of economic prosperity known as the “Golden Age of Capitalism\.” This was characterized by high economic growth, low unemployment, and rising standards of living for many Americans\.8\. Post\-World War II American society was characterized by a period of economic growth and prosperity known as the post\-war economic boom\.9\. Post\-World War II American society was characterized by a period of economic prosperity and growth, known as the post\-war economic boom, driven by factors such as returning soldiers starting families\.Table 11:Example 2 from NQ\-Open under Mistral\-7B\. Labeled as non\-hallucination, high semantic entropy\.Question:where does hydrogen peroxide come from in the body?Reference Answer:Nearly all living cells\.Low\-Temperature Answer:Hydrogen peroxide is produced within the body’s cells\.High\-Temperature Answers:1\. Hydrogen peroxide is produced in the body through the action of enzymes\.2\. Hydrogen peroxide can be found in the body, specifically in the neutrophils\.3\. Hydrogen peroxide is produced within the body’s neutrophils\.4\. Hydrogen peroxide is produced within the body’s neutrophils\.5\. The phagocytes\.6\. The reaction between oxygen and hydrogen in the body\.7\. The body generates hydrogen peroxide through the process of phagocytosis\.8\. Certain cells\.9\. Catalase\.Table 12:Example 1 from NQ\-Open under LLaMA3\-8B\. Labeled as non\-hallucination, high semantic entropy\.Question:What was one reason south carolina gave for its decision to secede from the union?Reference Answer:To preserve slavery\.Low\-Temperature Answer:Preservation of slavery\.High\-Temperature Answers:1\. Preservation of slavery\.2\. Preservation of slavery\.3\. States’ rights\.4\. The threat to its right to own slaves\.5\. Preservation of slavery\.6\. The issue of slavery\.7\. The preservation of slavery\.8\. States’ rights\.9\. Preservation of states’ rights\.Table 13:Example 2 from NQ\-Open under LLaMA3\-8B\. Labeled as non\-hallucination, high semantic entropy\.