HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

# HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews
Source: [https://arxiv.org/html/2609.03580](https://arxiv.org/html/2609.03580)
Dong\-Ting YaoTeng\-Fang HsiaoWei\-Chih ChenHong\-Han Shuai††thanks:Corresponding author\.Affiliation:National Yang Ming Chiao Tung UniversityEmail:[tzulinglin\.11@nycu\.edu\.tw](mailto:[email protected])

###### Abstract

The growing scale of academic peer review has motivated the use of Large Language Models \(LLMs\) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability\. Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers\. We introduceHalluPeer, a benchmark for detecting hallucinations in scientific peer reviews, providing aligned triples of paper content, human\-written reviews, and hallucination\-injected reviews, annotated for detection, classification, and localization\. Our pipeline induces a peer\-review\-specific hallucination taxonomy, identifies review contexts, and injects hallucinations with automated filtering\. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, while evaluation on authentic reviews demonstrates that HalluPeer\-defined hallucination patterns occur in real peer reviews, highlighting the critical need for source\-aware verification\. Our project page can be found in[https://github\.com/Lin\-TzuLing/HalluPeer\.git](https://github.com/Lin-TzuLing/HalluPeer.git)

## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.03580v1/EMNLP26_hallupeer_intro.png)Figure 1:HalluPeer benchmark Overview\.The benchmark comprises triples of paper content, original human\-written reviews, and hallucinated reviews\. Each triple is enriched with hallucination types, injection instructions, and aspect tags to facilitate systematic benchmarking of hallucination detection in peer reviews\.The rapid growth of AI research increasingly burdens the peer\-review system: submissions to major NLP conferences like ACL and EMNLP have surged from approximately 3\.4K in 2020 to over 8K in 2025, making review quality, consistency, and timeliness difficult to maintain\. To address this pressure, AI\-assisted peer review has emerged as a promising solution\. LLMs are increasingly used to draft comments, structure critiques, and streamline meta\-reviews\([Ou et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib17);[Yu et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib18)\); moving beyond prototypes, conferences like AAAI 2026 have piloted LLM integration for initial reviews and committee summarizations,111[https://aaai\.org/aaai\-launches\-ai\-powered\-peer\-review\-assessment\-system/](https://aaai.org/aaai-launches-ai-powered-peer-review-assessment-system/)aiming to boost efficiency while retaining human oversight\.

However, AI\-assisted reviewing introduces a critical reliability problem:*hallucination in paper reviews*\. Despite careful prompting and system safeguards, LLMs can generate fluent but unsupported claims,e\.g\., falsely asserting a missing baseline, misreporting results, or fabricating assumptions\. These factual errors transcend stylistic flaws; they mislead meta\-reviewers and unfairly impact editorial decisions\. Detecting them is therefore essential for trustworthy AI\-assisted review workflows\.

However, detecting these hallucinations is harder than in settings where the problem has been widely studied, such as QA, summarization, and RAG\([Liu et al\., 2022](https://arxiv.org/html/2609.03580#bib.bib4);[Manakul et al\., 2023](https://arxiv.org/html/2609.03580#bib.bib5);[Maynez et al\., 2020](https://arxiv.org/html/2609.03580#bib.bib7);[Li et al\., 2023](https://arxiv.org/html/2609.03580#bib.bib6);[Niu et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib14)\)\. Verifying a review requires synthesizing evidence across long, technical sections \(e\.g\., methods, tables, appendices\), and because reviews intertwine facts with subjective critiques, a detector must verify paper\-grounded claims while setting aside evaluative opinions\. These demands also explain why the problem remains under\-studied: existing hallucination datasets rarely provide the annotation structure this domain needs—paper\-grounded evidence for checking review claims, and fine\-grained hallucination types reflecting scientific critique\. Without these dimensions, evaluation cannot reveal whether a model fails because it cannot retrieve the right evidence, reason across sections, or recognize a specific error type such as a wrong number, fabricated comparison, or unsupported attribution\.

Thus, we introduceHalluPeer, a taxonomy\-driven benchmark for detecting hallucinations in scientific peer reviews\. We formulate review hallucination detection as a paper\-grounded verification problem:a review claim is hallucinated if it is unsupported by, or incorrect with respect to, the submitted paper\. This strictly isolates factual grounding from tone and subjective judgments\.

As shown in Fig\.[1](https://arxiv.org/html/2609.03580#S1.F1), HalluPeer contains aligned triples of*paper content*,*human\-written reviews*, and*hallucination\-injected reviews*\. Construction is guided by two principles:coverageof scientific failure modes, andcontrolover error types and contexts\. To achieve this, we induce a peer\-review\-specific hierarchical hallucination taxonomy, segment human\-written reviews into sentences, assign review\-aspect tags, and construct hallucination templates by pairing taxonomy instructions with aspect\-compatible contexts\. A constrained LLM editor then introduces hallucinations while preserving style, followed by automated filtering\.

Based on HalluPeer, we evaluate existing verifiers across three tasks:detection, classification, and localization\. Results indicate that current methods struggle with scientific reviews\. General verifiers often confuse unsupported claims with legitimate critique, while LLM judges fail on errors requiring technical grounding\. An analysis on authentic reviews shows that HalluPeer\-defined hallucination patterns occur in real peer reviews, and a detector trained solely on HalluPeer recovers all expert\-annotated hallucinations in authentic reviews, while source attribution remains a challenge\. Overall, these results suggest that auditing scientific reviews requires specialized, source\-aware verification rather than off\-the\-shelf factuality models\.

Our contributions are summarized as follows:

- •We introduceHalluPeer, a benchmark consisting of paper, human\-review, and hallucination\-injected\-review, with annotations for detection, type classification, and localization\.
- •We propose a taxonomy\-driven and aspect\-conditioned construction pipeline that induces peer\-review\-specific hallucination types, identifies compatible review contexts, and injects hallucinations with semantic verification\.
- •We systematically evaluate hallucination detection, classification, and localization, showing that general\-purpose verifiers struggle with scientific reviews while domain\-specific fine\-tuning substantially improves performance\. Additional cross\-generator, cross\-venue, and authentic\-review evaluations demonstrate robustness beyond the original synthetic setting and transfer to naturally occurring reviewer errors\.

## 2Related Work

### 2\.1LLM Hallucination Detection

Hallucination detection identifies generated statements unsupported by evidence, and has been studied in question answering, summarization, and retrieval\-augmented generation\. Some methods operate without gold references, using model output or consistency across sampled responses\([Liu et al\., 2022](https://arxiv.org/html/2609.03580#bib.bib4);[Manakul et al\., 2023](https://arxiv.org/html/2609.03580#bib.bib5)\), while others define hallucination relative to explicit evidence such as source documents\([Maynez et al\., 2020](https://arxiv.org/html/2609.03580#bib.bib7);[Cao et al\., 2022](https://arxiv.org/html/2609.03580#bib.bib8);[Bao et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib9)\)or retrieved passages\([Sriramanan et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib13);[Niu et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib14)\)\. Recent benchmarks add labels, span attribution, and type\-level diagnosis\([Mishra et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib11);[Akbar et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib12);[Bang et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib10)\)\. However, these resources are built on general\-domain text or retrieved snippets, and do not capture the verification structure of scientific peer review, where evidence is a long technical paper and a review claim may require reasoning across various sections\. HalluPeer addresses this gap with paper\-aligned evidence and review\-specific hallucination\.

![Refer to caption](https://arxiv.org/html/2609.03580v1/EMNLP26_method.png)Figure 2:Overview of HalluPeer construction framework\. We build a top\-down hallucination taxonomy for peer\-review scenario and derive fine\-grained instructions\. These instructions are used to construct sentence\-level injection templates and generate hallucinated review sentences via an automated injection and verification pipeline\.
### 2\.2LLMs in Automated Peer Review

The growing reviewing burden has motivated the use of LLMs for review drafting, structured critique, meta\-review assistance, and review\-quality assessment\([Russo et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib1);[Thakkar et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib2);[Zhuang et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib3);[Du et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib16)\)\. However, factual grounding remains a challenge: LLM reviewers may produce detailed feedback for incomplete manuscripts\([Ye et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib15)\)and generate inconsistent or paper\-unsupported claims\([Du et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib16);[Ou et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib17)\)\. These findings establish review hallucination as a realistic risk in scientific peer review\. Nevertheless, prior work lacks a dedicated benchmark for systematically detecting, categorizing, and localizing hallucinated review claims grounded in the submitted paper\. HalluPeer is designed to address this gap\.

## 3Taxonomy of Review Hallucinations

Existing hallucination taxonomies are derived from general\-domain text and rely on high\-level distinctions,e\.g\., factual vs\. faithfulness or intrinsic vs\. extrinsic hallucinations[Ji et al\. \(2023\)](https://arxiv.org/html/2609.03580#bib.bib19);[Li et al\. \(2024a\)](https://arxiv.org/html/2609.03580#bib.bib20)\. Such categories are insufficient for peer reviews, where unsupported claims often involve paper\-specific numbers or attribution errors\. As shown in Fig\.[2](https://arxiv.org/html/2609.03580#S2.F2), we refine these coarse categories into a hierarchical taxonomy of fine\-grained review hallucination types, along with operational instructions for controlled injection and evaluation\. Sec\.[3\.1](https://arxiv.org/html/2609.03580#S3.SS1)and[3\.2](https://arxiv.org/html/2609.03580#S3.SS2)describe the taxonomy structure and decomposition procedure\.

### 3\.1Top\-Down Taxonomy Construction

Our benchmark requires hallucination labels that are bothcontrollablefor data construction anddiagnosticfor evaluation\. To this end, we construct a hierarchical, tree\-structured taxonomy in a top\-down manner, where coarse categories are recursively refined into fine\-grained, review\-specific error types\.

Taxonomy structure\.We represent the taxonomy as a rooted tree over a set of nodes𝒱\\mathcal\{V\}\. Each nodev∈𝒱v\\in\\mathcal\{V\}corresponds to a hallucination concept and is defined asv=⟨hv,δv,pv,𝒞v⟩v=\\langle h\_\{v\},\\delta\_\{v\},p\_\{v\},\\mathcal\{C\}\_\{v\}\\rangle, wherehvh\_\{v\}is the name of the hallucination concept,δv\\delta\_\{v\}is an operational description,pvp\_\{v\}is the parent node, and𝒞v=\{u∈𝒱∣pu=v\}\\mathcal\{C\}\_\{v\}=\\\{u\\in\\mathcal\{V\}\\mid p\_\{u\}=v\\\}is the set of child nodes\. Aggregating along root\-to\-leaf paths supports analysis at different granularity levels\. For each leaf nodev∈𝒱leafv\\in\\mathcal\{V\}\_\{\\text\{leaf\}\}\(where𝒞v=∅\\mathcal\{C\}\_\{v\}=\\emptyset\), the descriptionδv\\delta\_\{v\}gives an explicit specification of how the hallucination should appear in review text \(e\.g\., what to alter, fabricate, or violate\); these leaf specifications serve both as controllable injection instructions and as fine\-grained evaluation labels\.

Initialization with coarse anchors\.We initialize the first level with a set of coarse\-grained semantic anchors adapted from prior hallucination categorization work\([Akbar et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib12)\): nine categories capturing common forms of unsupported generation,i\.e\.,Number,Entity,False Concatenation,Attribution Failure,Overgeneralization,Reasoning Error,Hyperbole,Temporal, andContext\-based Meaning Error\.222Definitions are provided in the Appendix\. We omit theOthercategory due to its incompatibility for injection\.These anchors allow downstream refinement into review\-specific manifestations\.

### 3\.2LLM\-Guided Recursive Decomposition

We expand each coarse anchor into review\-specific subtypes via a depth\-first generation procedure \(Algorithm[1](https://arxiv.org/html/2609.03580#algorithm1)\)\. At each recursive step, an LLM proposer \(ℳ\\mathcal\{M\}\) is queried to generate candidate child nodes\. The recursion designates a node as a terminal leaf under two conditions: reaching a predefined maximum depth, or yielding no further valid subdivisions from the LLM \(empty expansion\)\.

Two proposer functions\.The proposerℳ\\mathcal\{M\}is used through two functions\.Expandℳ\\textsc\{Expand\}\_\{\\mathcal\{M\}\}takes a parent concept and its description and returns a list of child concepts;Describeℳ\\textsc\{Describe\}\_\{\\mathcal\{M\}\}takes a child concept and produces an operational definition of how that hallucination manifests in review text\. Each child is then attached to its parent, added to the global node set, assigned its description, and recursively decomposed at the next depth\.

Constraints for usable subcategories\.To keep the taxonomy usable for fine\-grained labeling and controllable injection, we enforce several constraints through prompting \(templates in Appendix[J\.1](https://arxiv.org/html/2609.03580#A10.SS1)\)\. First, generated children should follow a Mutually Exclusive and Collectively Exhaustive \(MECE\)\-style separation: sibling categories should be minimally overlapping while collectively covering the major manifestations of the parent concept\. Second, children should be grounded in concrete peer\-review scenarios \(e\.g\., claims about baselines or experiments\), avoiding abstract distinctions\. Third, the proposer keeps sibling categories at comparable granularity and returns an empty list when the parent is already atomic at the current depth\.

![Refer to caption](https://arxiv.org/html/2609.03580v1/taxonomy_generation_pipeline.png)Figure 3:Overview of the multi\-model ensemble and expert refinement pipeline\.The workflow consists of \(1\) generating taxonomy trees using multiple LLM proposers, \(2\) identifying globally overlapping concepts through multi\-LLM consensus filtering, and \(3\) performing expert validation to finalize the taxonomy\.
### 3\.3Ensemble and Expert Refinement

Relying on a single LLM proposer raises two risks: generative priors may overproduce idiosyncratic concepts, and independently decomposed branches may introduce global semantic redundancy across the hierarchy\. We address both via a three\-stage workflow \(Fig\.[3](https://arxiv.org/html/2609.03580#S3.F3)\): \(1\) multi\-model ensemble with cross\-tree agreement filtering, \(2\) global overlap identification, and \(3\) human expert validation\. Stage 1 first retains taxonomy nodes with cross\-model consensus, while stage 2 performs pairwise overlap identification among the retained nodes\. During this stage, a multi\-LLM consensus process reduces 86,800 candidate node pairs to 434 high\-overlap pairs\. At stage 3, human experts then inspect to decide whether concepts should be merged or kept distinct, ensuring that the taxonomy remains conceptually coherent and distinguishable\. The final taxonomy contains 265 nodes after refinement\. Full details are provided in Appendix[F](https://arxiv.org/html/2609.03580#A6)\.

## 4HalluPeer Construction Pipeline

### 4\.1Data Collection

We source ICLR \(2019–2024\) and NeurIPS \(2021–2024\) records from OpenReview, as their large\-scale submissions and reviews are crucial for paper\-grounded hallucination construction and validation \(see Appendix[A\.1](https://arxiv.org/html/2609.03580#A1.SS1)for statistics\)\.

Following prior work on review aspect analysis\([Lu et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib21)\), we tag each review sentence with an aspect label \(e\.g\.,*Novelty*,*Evaluation*,*Clarity*\) and use it as an injection constraint: the injected hallucination type must remain compatible with the sentence’s role\. For instance, number\-related perturbations are preferentially injected into*Evaluation*sentences and avoided in*Clarity*sentences, where such edits would seem unnatural\.

Although peer reviews are human\-written, they may still contain unsupported statements, which would make them unreliable sources for hallucination injection\. While meta\-reviews are not factual ground truth, they provide a useful proxy for review points that were salient to the final assessment\. We therefore use an LLM filterFilterℳ\\textsc\{Filter\}\_\{\\mathcal\{M\}\}to compare each review with the corresponding meta\-review and assign an alignment judgment with confidence\. We use this alignment as a conservative selection heuristic: only high\-confidence, meta\-review\-aligned reviews are retained for hallucination injection, reducing the potential risk of selecting unreliable base reviews\.

### 4\.2Injection Template Construction

We construct hallucination injection templates by pairing review sentences with fine\-grained hallucination concepts from the taxonomy\. Each reviewr∈ℛpr\\in\\mathcal\{R\}\_\{p\}is segmented into sentences𝒮r=\{s1,…,s\|𝒮r\|\}\\mathcal\{S\}\_\{r\}=\\\{s\_\{1\},\\ldots,s\_\{\|\\mathcal\{S\}\_\{r\}\|\}\\\}, where eachs∈𝒮rs\\in\\mathcal\{S\}\_\{r\}is annotated with an aspect labelasa\_\{s\}\. \(Sec\.[4\.1](https://arxiv.org/html/2609.03580#S4.SS1)\)\. On the taxonomy side, each leaf nodev∈𝒱leafv\\in\\mathcal\{V\}\_\{\\text\{leaf\}\}corresponds to a fine\-grained concept specified by an injection instructionδv\\delta\_\{v\}and a labelπv\\pi\_\{v\}given by its root\-to\-vvpath\. Pairing a sentencesswith a leaf nodevvthen yields a hallucination injection template

Ts,v=\(s,as,δv,πv\)\.T\_\{s,v\}=\(s,a\_\{s\},\\delta\_\{v\},\\pi\_\{v\}\)\.
Naively instantiating every\(s,v\)\(s,v\)pair is intractable, so we narrow the search space by first applying a coarse\-grained screening: for each sentence we test which first\-level concepts are even applicable \(e\.g\.,*Number*requires explicit numeric values\), and build fine\-grained templates only from the leaf nodes descending from the compatible anchors\. Finally, an LLM\-based compatibility checkCheckℳ\\textsc\{Check\}\_\{\\mathcal\{M\}\}filters out templates whose hallucination type cannot be naturally applied to the target sentence and aspect, yielding the feasible set𝒯feasible\\mathcal\{T\}^\{\\text\{feasible\}\}\. The screening prompt is in Appendix[J\.3](https://arxiv.org/html/2609.03580#A10.SS3), and the pseudocode is Appendix[I](https://arxiv.org/html/2609.03580#A9)\.

### 4\.3Automated Injection Pipeline

Given a set of feasible templates, we generate hallucinated counterparts of human\-written sentences via an automated pipeline\. For eachTs,v∈𝒯feasibleT\_\{s,v\}\\in\\mathcal\{T\}^\{\\text\{feasible\}\}, we prompt an LLM injectorInjectℳ\\textsc\{Inject\}\_\{\\mathcal\{M\}\}with the original sentencessand the taxonomy instructionδv\\delta\_\{v\}, producing a hallucinated sentences~=Injectℳ​\(s,δv\)\\tilde\{s\}=\\textsc\{Inject\}\_\{\\mathcal\{M\}\}\(s,\\delta\_\{v\}\)that follows the specified concept while remaining fluent and coherent\. This gives sentence\-level control over hallucination types and broad coverage across taxonomy categories\.

We then apply a post\-hoc verifier defined asVerifyℳ​\(s,s~\)\\textsc\{Verify\}\_\{\\mathcal\{M\}\}\(s,\\tilde\{s\}\), which queries an LLM to check whethers~\\tilde\{s\}is semantically equivalent toss, following the criteria of\([Liang et al\., 2025](https://arxiv.org/html/2609.03580#bib.bib22)\):

Verifyℳ​\(s,s~\)=\{1,ifs~is semantically equivalent tos0,otherwise,\\text\{\{Verify\}\}\_\{\\mathcal\{M\}\}\(s,\\tilde\{s\}\)=\\begin\{cases\}1,&\\text\{if $\\tilde\{s\}$ is semantically equivalent to $s$\}\\\\ 0,&\\text\{otherwise,\}\\end\{cases\}and discard any template withVerifyℳ​\(s,s~\)=1\\textsc\{Verify\}\_\{\\mathcal\{M\}\}\(s,\\tilde\{s\}\)=1\. Whereas the template\-level check in Sec\.[4\.2](https://arxiv.org/html/2609.03580#S4.SS2)assesses*applicability*before generation, this verifier operates on generated outputs for quality control\. The complete pseudocode and prompt are provided in Appendix[I](https://arxiv.org/html/2609.03580#A9)\(Algorithm[3](https://arxiv.org/html/2609.03580#algorithm3)\) and Appendix[J](https://arxiv.org/html/2609.03580#A10)\.

## 5Experimental Results

### 5\.1Task Formulation

We define three sub\-tasks that progressively evaluate a model’s ability to*detect*,*categorize*, and*localize*hallucinated content in peer\-review text\.

Task 1: Hallucination Detection\.This task aims to determine whether hallucinated content is present, formulated as a binary classification problem at two levels of granularity: \(1\)Review\-level\. Given a reviewrrwith sentences𝒮r\\mathcal\{S\}\_\{r\}, the model predicts whether the review contains hallucinated claims\.\(2\)Sentence\-level\. Given a sentences∈𝒮rs\\in\\mathcal\{S\}\_\{r\}, the model predicts whether the sentence contains hallucinated content\.

Task 2: Hallucination Type Classification\.Given a hallucinated sentences~\\tilde\{s\}, together with supporting evidence from the corresponding source paperpp, the model is required to perform multi\-class classification and predict a single hallucination type from the coarse\-grained label setℋ=\{hv∣v∈𝒱\(1\)\}\\mathcal\{H\}=\\\{h\_\{v\}\\mid v\\in\\mathcal\{V\}^\{\(1\)\}\\\}\.

Task 3: Hallucination Localization\.To enable fine\-grained error analysis, we formulate hallucination localization as a span identification task\. Given a reviewrr, the model is required to identify the spans corresponding to hallucinated content\.

### 5\.2Baselines

We evaluate baselines under three paradigms for hallucination detection: \(1\) specialized verification frameworks, \(2\) prompting\-based general\-purpose LLMs, and \(3\) instruction\-tuned LLMs\. For hallucination type classification and localization, we evaluate prompting\-based and instruction\-tuned LLMs\. Implementation details and prompt templates are provided in Appendix[C](https://arxiv.org/html/2609.03580#A3)and Appendix[J](https://arxiv.org/html/2609.03580#A10)\.

Specialized Verification Frameworks\.We evaluate four hallucination verifiers:HHEM\-2\.1\-Open\([Li et al\., 2024b](https://arxiv.org/html/2609.03580#bib.bib23)\)\(consistency\-based\),True\-NLI\([Laurer et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib24)\)\(entailment\-based\), aseNtLI\-style retriever\-verifier pipeline\([Schuster et al\., 2022](https://arxiv.org/html/2609.03580#bib.bib25)\), andRefChecker\([Hu et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib26)\)\.

General\-Purpose LLMs\.We evaluate seven prompting\-based LLMs, includingQwen3\-32B,Llama\-3\.3\-70B,GPT\-OSS\-20B/120B,Mistral\-Small\-3\.1\-24B,RootSignals\-Judge\-Llama\-70B, andGPT\-5\.2\. Following prior hallucination evaluation work\([Li et al\., 2023](https://arxiv.org/html/2609.03580#bib.bib6)\), we additionally implement a Retrieval\-Augmented LLM\-as\-a\-Judge \(RA\-LLM\) framework with three prompting strategies:*Knowledge Retrieval \(KR\)*,*Chain\-of\-Thought \(CoT\)*, and*Sample Contrast \(Contrast\)*\. For instruction\-tuned baselines, we apply QLoRA\-based 4\-bit fine\-tuning toQwen2\.5\-3B/7B\-InstructandQwen3\-32B\.

Evidence Retrieval\.For all specialized verifiers andRA\-LLM, we apply BM25\-based retrieval over the source paper to obtain a compact evidence context\. For other prompting\-based LLM baselines, we evaluate variants that consume the full paper content, along with LCS variants \(detailed results are provided in Appendix[E\.1\.1](https://arxiv.org/html/2609.03580#A5.SS1.SSS1)\)\.

### 5\.3Evaluation Metrics

We evaluate models on the three tasks using following metrics\. Details are deferred to Appendix[D](https://arxiv.org/html/2609.03580#A4)\.

Hallucination Detection\.We reportAccuracy,Precision,Recall, andF1\. Given the class imbalance in hallucination labels, we additionally report theMatthews Correlation Coefficient \(MCC\)\.

Hallucination Type Classification\.We reportMacro\-F1, which equally weights all hallucination categories, andMicro\-F1, which reflects overall instance\-level performance\.

Hallucination Localization\.We evaluate span identification using complementary token\-level and span\-level metrics\. At the token level, we computeToken\-F1by converting predicted and gold spans into BIO tag sequences\. At the span level, we reportExact Match Span\-F1, which requires exact boundary alignment between predicted and gold spans, andOverlap Span\-F1, which considers partially overlapping spans as correct predictions\.

### 5\.4Hallucination Detection \(Task 1\)

Tab\.[1](https://arxiv.org/html/2609.03580#S5.T1)presents the performance of hallucination detection at both review and sentence levels on HalluPeer \(NeurIPS 2024\)\. Additional results on ICLR 2024 are provided in Appendix[E\.1\.2](https://arxiv.org/html/2609.03580#A5.SS1.SSS2)\. Our observations are summarized below:

Limitations of Specialized Verifications\.Pre\-trained verifiers \(e\.g\.,HHEM\-2\.1\-Open,True\-NLI,seNtLI, andRefChecker\) generalize poorly to peer reviews\. At the review level, all models exhibit near\-random performance with MCC scores≤0\.03\\leq 0\.03\. WhileHHEM\-2\.1\-Openperforms slightly better at the sentence level—likely due to fine\-tuning on RAG hallucination datasets—its overall effectiveness remains limited by the domain gap between general RAG settings and scientific peer reviews\. Overall, these findings suggest that existing hallucination verifiers, whether consistency\- or entailment\-based, fail to transfer to scientific review verification\. Verifying technical review claims against full\-length papers requires complex multi\-hop reasoning that differs substantially from the open\-domain RAG, NLI, and general factual consistency datasets used to train these models\.

Impact of Prompting Strategies\.Among the RA\-LLM variants, prompting strategies exhibit distinct performance\. At the sentence level,RA\-LLM \(KR\), which augments prompts with few\-shot demonstrations, achieves the strongest performance \(MCC 0\.61, Accuracy 0\.82\), outperforming both explicit reasoning \(RA\-LLM \(CoT\)\) and contrastive prompting \(RA\-LLM \(Contrast\)\)\. One explanation is thatCoTandContrastprompting rely more heavily on the completeness of source paper evidence\. Since evidence is first filtered through BM25 retrieval, the retrieved context may not provide sufficient information for multi\-step reasoning or contradiction analysis\. In this setting, concise few\-shot demonstrations offer a more stable supervision signal, allowingRA\-LLM \(KR\)to remain effective\.

Under standard prompting, frontier LLMs such asQwen3\-32B,GPT\-OSS\-120B, andGPT\-5\.2achieve competitive sentence\-level performance, with F1 and MCC scores comparable to RA\-LLM variants\. However, they consistently underperform at the review level, indicating difficulty in maintaining globally consistent verification over long\-form contexts and aggregating evidence coherently\.

The Superiority of Domain\-Specific Fine\-tuning\.We observe substantial gains from domain\-specific instruction tuning\. Fine\-tuned models consistently dominate both review\- and sentence\-level evaluation\. Remarkably, even the relatively compactQwen2\.5\-3Bsignificantly outperforms all prompting\-based zero\-shot methods, including frontier models such asGPT\-5\.2andLlama\-3\.3\-70B\. The scaled\-upQwen3\-32B\(Fine\-tuned\) achieves the strongest overall performance, reaching F1 scores of 0\.90 and 0\.91 at the review and sentence levels, respectively, together with a sentence\-level MCC of 0\.87\. These findings suggest that peer\-review hallucination detection depends heavily on domain\-specific verification patterns that are not sufficiently captured by generic zero\-shot prompting alone\.

Table 1:Task 1 results on HalluPeer \(NeurIPS 2024\)\.Review\-/sentence\-level results are shown before/after the slash\. Results are reported under the default evidence and prompting settings described in Sec\.[5\.2](https://arxiv.org/html/2609.03580#S5.SS2)\.Specialized VerificationAcc\.Prec\.Rec\.F1MCCHHEM\-2\.1\-Open0\.51 /0\.640\.51 /0\.470\.55 /0\.630\.53 /0\.540\.02 /0\.26True\-NLI0\.52/ 0\.540\.52/ 0\.370\.56/ 0\.520\.54/ 0\.430\.03/ 0\.07seNtLI0\.51 / 0\.610\.51 / 0\.440\.54 / 0\.550\.52 / 0\.490\.02 / 0\.18Refchecker0\.51 / 0\.490\.51 / 0\.340\.45 / 0\.510\.48 / 0\.400\.03/ \-0\.01LLM \(Prompting\)Acc\.Prec\.Rec\.F1MCCRA\-LLM \(KR\)0\.61 /0\.820\.68 /0\.740\.41 / 0\.740\.51 /0\.740\.24 /0\.61RA\-LLM \(CoT\)0\.61 / 0\.770\.63 / 0\.630\.52 / 0\.780\.57 / 0\.700\.22 / 0\.53RA\-LLM \(Contrast\)0\.57 / 0\.730\.55 / 0\.570\.77 /0\.840\.64 / 0\.680\.15 / 0\.49Qwen3\-32B0\.63/ 0\.790\.63 / 0\.660\.61 / 0\.740\.62 / 0\.700\.25/ 0\.54Llama\-3\.3\-70B0\.57 / 0\.700\.74/ 0\.550\.22 / 0\.630\.34 / 0\.590\.20 / 0\.36Mistral\-Small\-3\.10\.61 / 0\.730\.63 / 0\.580\.52 / 0\.740\.57 / 0\.650\.22 / 0\.45GPT\-OSS\-20B0\.59 / 0\.780\.56 / 0\.650\.79 / 0\.760\.66 / 0\.700\.19 / 0\.53GPT\-OSS\-120B0\.56 / 0\.780\.53 / 0\.640\.92/ 0\.810\.67 / 0\.720\.17 / 0\.55Judge\-Llama\-70B0\.57 / 0\.700\.74/ 0\.550\.23 / 0\.640\.35 / 0\.590\.20 / 0\.36GPT\-5\.20\.58 / 0\.800\.55 / 0\.700\.89 / 0\.730\.68/ 0\.710\.21 / 0\.56LLM \(Fine\-tuned\)Acc\.Prec\.Rec\.F1MCCQwen2\.5\-3B0\.84 / 0\.850\.86 / 0\.710\.83 /0\.930\.84 / 0\.810\.69 / 0\.70Qwen2\.5\-7B0\.87 / 0\.910\.86 / 0\.820\.89/0\.930\.87 / 0\.870\.74 / 0\.80Qwen3\-32B0\.90/0\.940\.96/0\.940\.85 / 0\.890\.90/0\.910\.81/0\.87
### 5\.5Hallucination Type Classification \(Task 2\)

Tab\.[2](https://arxiv.org/html/2609.03580#S5.T2)presents the overall results of hallucination type classification on HalluPeer \(NeurIPS 2024\), while Fig\.[4](https://arxiv.org/html/2609.03580#S5.F4)visualizes per\-label performance\. Additional results on ICLR 2024 are provided in Appendix[E\.2\.2](https://arxiv.org/html/2609.03580#A5.SS2.SSS2)\. Our key findings include:

Limitations of Zero\-shot Prompting\.All prompting\-based LLMs perform poorly, especially at the review level, where the best Macro\-F1 reaches only 0\.19\. Interestingly, larger models do not consistently yield better performance\. For example, the smallerMistral\-Small\-3\.1outperforms larger models such asLlama\-3\.3\-70BandGPT\-OSS\-120Bon both Macro\-F1 and Micro\-F1\. These findings suggest that hallucination category recognition relies more on robust scientific verification behavior than model scale\.

Review\-level Categorization Remains Challenging\.A clear gap exists between sentence\- and review\-level classification across all prompting\-based LLMs, with review\-level performance consistently weak\. One possible explanation is that hallucination evidence in peer reviews is highly localized, with only a small hallucinated span embedded within largely correct content\. Under review\-level classification, such sparse error signals may be diluted by surrounding context or overlooked due to long\-context reasoning limitations such as lost\-in\-the\-middle effects, making review\-level categorization substantially more difficult\.

Semantic Difficulty Varies Across Hallucination Types\.Under prompting\-only settings, performance varies substantially across hallucination categories\. Categories with explicit lexical cues, such asEntityandNumber, achieve relatively higher F1 scores\. In contrast, semantically complex categories requiring contextual grounding or multi\-hop reasoning, includingContext\-based Meaning Error,Hyperbole, andTemporal, remain highly challenging, with several models collapsing to near\-zero review\-level F1\. These findings suggest that current LLMs are more effective at detecting surface\-level factual inconsistencies than deeper semantic distortions in scientific peer reviews\.

Domain\-specific Fine\-tuning Enables Robust Error Taxonomy Recognition\.Fine\-tuning closes the gaps identified above\. Fine\-tuned models substantially outperform their zero\-shot counterparts, particularly on semantically challenging categories\. For example,Hyperboleimproves from near zero to 0\.72/0\.87,Temporalto 0\.48/0\.87, andContext\-based Meaning Errorto 0\.56/0\.82\. These results suggest that the semantic difficulties observed under zero\-shot prompting can be substantially mitigated through domain\-specific adaptation\.

Table 2:Task 2 overall results on HalluPeer \(NeurIPS 2024\)\.Review\-/sentence\-level results are shown before/after the slash\. Detailed per\-label F1 results are provided in Appendix[E\.2\.1](https://arxiv.org/html/2609.03580#A5.SS2.SSS1)\.LLM \(Prompting\)Macro\-F1Micro\-F1Qwen3\-32B0\.15 / 0\.280\.17 / 0\.30Llama\-3\.3\-70B0\.14 / 0\.240\.17 / 0\.27Mistral\-Small\-3\.10\.19/0\.330\.22/0\.35GPT\-OSS\-20B0\.13 / 0\.250\.15 / 0\.28GPT\-OSS\-120B0\.16 / 0\.250\.18 / 0\.26Judge\-Llama\-70B0\.14 / 0\.230\.17 / 0\.26GPT\-5\.20\.14 / 0\.280\.17 / 0\.31LLM \(Fine\-tuned\)Macro\-F1Micro\-F1Qwen2\.5\-3B0\.49 / 0\.720\.52 / 0\.72Qwen2\.5\-7B0\.53 /0\.860\.55 /0\.86Qwen3\-32B0\.59/ 0\.820\.59/ 0\.84![Refer to caption](https://arxiv.org/html/2609.03580v1/figs/hallupeer_heatmap_sentence.png)Figure 4:Visualization of Task 2 sentence\-level per\-label F1 results\.Columns denote hallucination categories: A \(Attribution Failure\), C \(Context\-based Meaning Error\), E \(Entity\), F \(False Concatenation\), H \(Hyperbole\), N \(Number\), O \(Overgeneralization\), R \(Reasoning Error\), and T \(Temporal\)\.
### 5\.6Hallucination Localization \(Task 3\)

Tab\.[3](https://arxiv.org/html/2609.03580#S5.T3)presents the results for hallucination span localization on HalluPeer \(NeurIPS 2024\)\. Additional results on ICLR 2024 are provided in Appendix[E\.3\.1](https://arxiv.org/html/2609.03580#A5.SS3.SSS1)\. Key findings include:

Limitations of Zero\-shot Prompting\.All prompting\-based models exhibit limited span localization capability, particularly under strict boundary matching metrics\. Among zero\-shot methods,GPT\-5\.2achieves the strongest performance, reaching 0\.58 Token\-F1 and 0\.46 Exact Span\-F1\. However, overall performance remains moderate even for frontier models, suggesting that hallucination localization in peer reviews is inherently challenging\. Similar to review\-level hallucination categorization, hallucinated content is often sparse and embedded within otherwise correct scientific critique, making precise grounding and boundary identification considerably more difficult\.

Challenge of Exact Boundary Detection\.A consistent gap is observed between Overlap Span\-F1 and Exact Span\-F1 across models\. For example,GPT\-5\.2achieves 0\.58 Overlap Span\-F1 but only 0\.46 Exact Span\-F1, indicating that models can often localize hallucinated regions approximately but struggle to determine precise token boundaries\. This issue is particularly pronounced in peer reviews, where hallucinations appear as localized semantic distortions embedded within otherwise coherent scientific arguments\.

Fine\-tuning Dramatically Improves Hallucination Grounding\.Fine\-tuned models substantially outperform prompting\-based approaches across all metrics\. Notably,Qwen3\-32Bachieves 0\.91 Token\-F1 and 0\.86 Exact Span\-F1\. The gains suggest that accurate hallucination localization requires specialized supervision for token\-level grounding and error boundary identification, which cannot be reliably induced through only zero\-shot prompting\.

Table 3:Task 3 results on HalluPeer \(NeurIPS 2024\)\.Review\-level results are reported under default evidence retrieval and prompting settings described in Sec\.[5\.2](https://arxiv.org/html/2609.03580#S5.SS2)\.LLM \(Prompting\)Token\-F1Exact Span\-F1Overlap Span\-F1Qwen3\-32B0\.460\.300\.50Llama\-3\.3\-70B0\.490\.350\.49Mistral\-Small\-3\.10\.400\.270\.40GPT\-OSS\-20B0\.510\.380\.49GPT\-OSS\-120B0\.550\.410\.52Judge\-Llama\-70B0\.500\.360\.49GPT\-5\.20\.580\.460\.58LLM \(Fine\-tuned\)Token\-F1Exact Span\-F1Overlap Span\-F1Qwen2\.5\-3B0\.850\.790\.82Qwen2\.5\-7B0\.840\.820\.84Qwen3\-32B0\.910\.860\.90
### 5\.7Cross\-Venue Transferability

To evaluate the cross\-venue transferability of our fine\-tuned detectors, we fine\-tune each on one venue split and evaluate it directly on the completely held\-out venue, considering both transfer directions\. Results are reported in Tab\.[14](https://arxiv.org/html/2609.03580#A5.T14)–[16](https://arxiv.org/html/2609.03580#A5.T16)\.

Task 1 \(Detection\)\.Domain\-specific fine\-tuning demonstrates robust generalization across venues\. As shown in Tab\.[14](https://arxiv.org/html/2609.03580#A5.T14), Qwen3\-32B achieves review\-/sentence\-level F1 scores of 0\.90/0\.91 when trained on NeurIPS 2024 and tested on ICLR 2024, and 0\.86/0\.93 in the reverse direction\. These results indicate that the learned detection capability transfers across different conference distributions\.

Task 2 \(Type Classification\)\.The performance gap between review\- and sentence\-level results persists across venues\. As shown in Tab\.[15](https://arxiv.org/html/2609.03580#A5.T15), Qwen3\-32B achieves sentence\-level Micro\-F1 scores of 0\.83 and 0\.86 for NeurIPS→\\rightarrowICLR and ICLR→\\rightarrowNeurIPS, respectively, indicating that the injected hallucination types remain recognizable across different conference distributions\.

Task 3 \(Localization\)\.Cross\-venue span localization remains highly accurate\. Tab\.[16](https://arxiv.org/html/2609.03580#A5.T16)shows that the fine\-tuned Qwen3\-32B model preserves excellent boundary grounding on unseen venues, reaching 0\.91 Token\-F1 and 0\.88 Exact Span\-F1 when transferring from NeurIPS to ICLR\. The performance in the ICLR→\\rightarrowNeurIPS direction is comparable \(0\.92 Token\-F1 and 0\.87 Exact Span\-F1\), confirming the general applicability and robustness of our localization training\.

Comparison with In\-Domain Results\.We further compare cross\-venue transfer with the corresponding in\-domain results on the same test venue\. For NeurIPS→\\rightarrowICLR, Qwen3\-32B achieves Task 1 F1 of 0\.90/0\.91 and Task 3 Token\-F1 of 0\.91, compared with 0\.89/0\.94 and 0\.93, respectively, for the ICLR in\-domain results \(Appendix[E\.1\.2](https://arxiv.org/html/2609.03580#A5.SS1.SSS2),[E\.3\.1](https://arxiv.org/html/2609.03580#A5.SS3.SSS1)\)\. For ICLR→\\rightarrowNeurIPS, it achieves Task 1 F1 of 0\.86/0\.93 and Task 3 Token\-F1 of 0\.92, compared with the NeurIPS in\-domain results of 0\.90/0\.91 and 0\.91, respectively \(Tab\.[1](https://arxiv.org/html/2609.03580#S5.T1),[3](https://arxiv.org/html/2609.03580#S5.T3)\)\. The relatively small performance differences across transfer directions suggest that the detectors do not rely strongly on venue\-specific writing styles or formatting, but instead internalize the structural definitions of peer\-review hallucinations\.

### 5\.8Cross\-Generation Ablation

To examine whether our fine\-tuned detectors rely on generator\-specific artifacts, we construct additional test sets using Mistral\-Small\-3\.1 and Llama\-3\.3\-70B as hallucination injectors and verifiers, while keeping the taxonomy and construction protocol unchanged\. Detectors are trained exclusively on Qwen3\-32B\-injected data and evaluated on these unseen\-generator test sets\.Across all three tasks, performance remains largely stable under generator changes\.For Task 1, the F1 shift is only 0\.01–0\.02 on average; Task 2 shows shifts generally within 0\.05, while Task 3 Token\-F1 remains within 0\.02 of the in\-domain results\. Detailed results and per\-task analyses are provided in Appendix[E\.5](https://arxiv.org/html/2609.03580#A5.SS5)\.

### 5\.9Evaluation on Authentic Reviews

To quantitatively assess whether detectors trained on synthetic hallucinations transfer to authentic reviewer errors, we construct a manually annotated set of 1,161 independent NeurIPS 2024 reviews, identifying 20 genuine reviewer hallucinations\. The fine\-tuned detectors are trained exclusively on synthetic HalluPeer ICLR 2024 data and have no access to these authentic annotations\. As reported in Appendix[G](https://arxiv.org/html/2609.03580#A7), Qwen3\-32B recovers all 20 authentic hallucinations \(TPR=100\.0%=100\.0\\%\) at FPR=22\.1%=22\.1\\%, substantially improving recall over zero\-shot baselines\. These results provide quantitative evidence that detectors trained on synthetic hallucinations can transfer to naturally occurring reviewer errors\.

## 6Alignment with Real Review Errors

Hallucination Types in Authentic ReviewsTo evaluate whether the hallucination patterns defined by our taxonomy occur in authentic reviews, we conduct a case study on 13,803 human\-written NeurIPS 2024 reviews\. Our fine\-tuned detector flags potential hallucinations, and we manually inspect 200 flagged instances, checking whether each constitutes a genuine hallucination and whether the predicted type matches\. As shown in Tab\.[21](https://arxiv.org/html/2609.03580#A9.T21)–[23](https://arxiv.org/html/2609.03580#A9.T23), the identified review errors align with the hallucination patterns defined by our taxonomy\. These findings provide evidence that the hallucination patterns defined by HalluPeer correspond to errors occurring in authentic peer reviews\. The annotation protocol details are provided in Appendix[H](https://arxiv.org/html/2609.03580#A8)\.

False Positives from Paper Claim Quotation\.Our manual inspection reveals a source of false positives arising from reviewer quotations of exaggerated or unsupported claims in the submitted paper\. Consequently, review texts may contain hallucination\-like statements that originate from the paper rather than the reviewer\. This highlights a key challenge for peer\-review verification systems: distinguishing reviewer\-generated hallucinations from the propagation of unsupported claims in the submitted paper\. Such source\-aware attribution is important for distinguishing genuine reviewer errors from valid criticism\.

## 7Conclusion

We presentHalluPeer, a taxonomy\-driven benchmark for hallucination detection in peer reviews\. HalluPeer formulates review auditing as a*paper\-grounded verification*problem requiring long\-context reasoning over manuscripts\. To support systematic evaluation, we proposed a hierarchical taxonomy and an aspect\-aware injection pipeline for generating realistic hallucinated reviews\.

Experiments show that existing verifiers struggle to distinguish unsupported claims from legitimate scientific critique, while domain\-specific fine\-tuning substantially improves performance\. Our evaluations on authentic reviews provide evidence that HalluPeer\-defined hallucination patterns occur in real peer reviews, while source\-aware attribution remains a key challenge for trustworthy AI\-assisted peer review\.

## Acknowledgments

This work is partially supported by the National Science and Technology Council, Taiwan, under Grant: NSTC\-115\-2923\-E\-A49 \-010\-MY5\.

## 8Limitations

We acknowledge several limitations in our work\.Coverage of naturally occurring errors\.Naturally occurring review hallucinations are sparse and require domain expertise to identify, making large\-scale real\-world annotation prohibitively labor\-intensive\. Rather than modeling an unobservable real\-world distribution, HalluPeer provides a controlled, practically motivated framework that covers plausible error patterns, supporting future research on automated peer\-review auditing and review quality tracking\.

Synthetic nature of the dataset\.While our injection pipeline is designed to simulate realistic errors via aspect\-conditioning and style preservation, the resulting dataset remains synthetic\. The distribution of injected hallucinations may not perfectly reflect the subtle, drift\-based errors found in reviews in real\-world scenarios\. Naturally occurring hallucinations might involve more complex reasoning failures that are difficult to simulate through localized editing\.

Domain specificity\.Our data source is restricted to computer science conferences hosted on OpenReview, primarily due to the scarcity of publicly available peer\-review datasets in other fields\. Reviewing norms, claim structures, and evidence densities vary significantly across scientific disciplines\. Consequently, the taxonomy and detection models developed on HalluPeer may not generalize zero\-shot to other domains without specific adaptation\.

Scope of hallucination definition\.We restrict our definition of hallucination to factual inconsistency with respect to the submission content\. We do not address subjective aspects of the review process, such as unfair novelty judgments, tonal issues, or the validity of critiques regarding potential future work\. These subjective elements are critical for high\-quality peer review but require different evaluation frameworks beyond factual grounding\.

Model\-induced bias\.Since the taxonomy is derived via LLM\-guided decomposition, the resulting structure may inherit the inductive biases of the proposer models\. These biases could influence how hallucination types are grouped or defined\. Consequently, the constructed hierarchy represents a model\-centric perspective on error categorization rather than a canonical or exhaustive standard\.

Selection bias from meta\-review alignment\.Our use of meta\-review alignment as a selection heuristic may bias HalluPeer toward reviews whose main points were reflected in the final assessment, rather than the full distribution of review quality\. This is a deliberate trade\-off in controlled benchmark construction: a more selective base\-review set improves label reliability and reduces the likelihood of selecting unreliable reviews, while broader sampling would provide greater distributional coverage at the cost of noisier supervision\.

## 9Ethical Considerations

While this work aims to enhance the integrity of the academic peer\-review process, we acknowledge the following ethical implications of using scholarly data\.

Dual\-Use Risks\.Our hallucination injection pipeline poses potential dual\-use risks: although developed for benchmarking detection systems, it could be misused to generate more convincing hallucinated reviews\. To mitigate this concern, we restrict our taxonomy and generation scripts to defensive research purposes\.

Integrity of the Peer\-Review Process\.Our research synthetically corrupts human\-written reviews to construct negative samples\. No hallucinated reviews were submitted to real venues or used in editorial decisions\. The dataset is intended solely for offline training and evaluation\. We advocate human\-in\-the\-loop review systems, where AI functions as a diagnostic aid rather than an autonomous decision\-maker\.

## References

- Akbaret al\.\(2024\)S\. A\. Akbar, M\. M\. Hossain, T\. Wood, S\. Chin, E\. M\. Salinas, V\. Alvarez, and E\. CornejoHalluMeasure: fine\-grained hallucination measurement using chain\-of\-thought reasoning\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Cited by:[§A\.3](https://arxiv.org/html/2609.03580#A1.SS3.p1.1),[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.03580#S3.SS1.p3.1)\.
- Banget al\.\(2025\)Y\. Bang, Z\. Ji, A\. Schelten, A\. Hartshorn, T\. Fowler, C\. Zhang, N\. Cancedda, and P\. FungHallulens: llm hallucination benchmark\.arXiv preprint\.Cited by:[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Baoet al\.\(2025\)F\. S\. Bao, M\. Li, R\. Qu, G\. Luo, E\. Wan, Y\. Tang, W\. Fan, M\. S\. Tamber, S\. Kazi, V\. Sourabh, M\. Qi, R\. Tu, C\. Xu, M\. Gonzales, O\. Mendelevitch, and A\. AhmadFaithBench: a diverse hallucination benchmark for summarization by Modern LLMs\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,Cited by:[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Caoet al\.\(2022\)M\. Cao, Y\. Dong, and J\. CheungHallucinated but factual\! inspecting the factuality of hallucinations in abstractive summarization\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics,Cited by:[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Dettmerset al\.\(2023\)T\. Dettmers, A\. Pagnoni, A\. Holtzman, and L\. ZettlemoyerQlora: efficient finetuning of quantized llms\.Advances in neural information processing systems36,pp\. 10088–10115\.Cited by:[Appendix C](https://arxiv.org/html/2609.03580#A3.p9.1)\.
- Duet al\.\(2024\)J\. Du, Y\. Wang, W\. Zhao, Z\. Deng, S\. Liu, R\. Lou, H\. P\. Zou, P\. Narayanan Venkit, N\. Zhang, M\. Srinath, H\. R\. Zhang, V\. Gupta, Y\. Li, T\. Li, F\. Wang, Q\. Liu, T\. Liu, P\. Gao, C\. Xia, C\. Xing, C\. Jiayang, Z\. Wang, Y\. Su, R\. S\. Shah, R\. Guo, J\. Gu, H\. Li, K\. Wei, Z\. Wang, L\. Cheng, S\. Ranathunga, M\. Fang, J\. Fu, F\. Liu, R\. Huang, E\. Blanco, Y\. Cao, R\. Zhang, P\. S\. Yu, and W\. YinLLMs assist NLP researchers: critique paper \(meta\-\)reviewing\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Cited by:[§2\.2](https://arxiv.org/html/2609.03580#S2.SS2.p1.1)\.
- Huet al\.\(2024\)X\. Hu, D\. Ru, L\. Qiu, Q\. Guo, T\. Zhang, Y\. Xu, Y\. Luo, P\. Liu, Y\. Zhang, and Z\. ZhangRefchecker: reference\-based fine\-grained hallucination checker and benchmark for large language models\.arXiv preprint\.Cited by:[Appendix C](https://arxiv.org/html/2609.03580#A3.p2.1),[Appendix C](https://arxiv.org/html/2609.03580#A3.p4.1),[§5\.2](https://arxiv.org/html/2609.03580#S5.SS2.p2.1)\.
- Jiet al\.\(2023\)Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, E\. Ishii, Y\. J\. Bang, A\. Madotto, and P\. FungSurvey of hallucination in natural language generation\.ACM computing surveys\.Cited by:[§3](https://arxiv.org/html/2609.03580#S3.p1.1)\.
- Laureret al\.\(2024\)M\. Laurer, W\. van Atteveldt, A\. Casas, and K\. WelbersLess annotating, more classifying: addressing the data scarcity issue of supervised machine learning with deep transfer learning and bert\-nli\.Political Analysis\.Cited by:[Appendix C](https://arxiv.org/html/2609.03580#A3.p2.1),[§5\.2](https://arxiv.org/html/2609.03580#S5.SS2.p2.1)\.
- Liet al\.\(2024a\)J\. Li, J\. Chen, R\. Ren, X\. Cheng, X\. Zhao, J\. Nie, and J\. WenThe dawn after the dark: an empirical study on factuality hallucination in large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[§3](https://arxiv.org/html/2609.03580#S3.p1.1)\.
- Liet al\.\(2023\)J\. Li, X\. Cheng, X\. Zhao, J\. Nie, and J\. WenHaluEval: a large\-scale hallucination evaluation benchmark for large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Cited by:[Appendix C](https://arxiv.org/html/2609.03580#A3.p8.1),[§1](https://arxiv.org/html/2609.03580#S1.p3.1),[§5\.2](https://arxiv.org/html/2609.03580#S5.SS2.p3.1)\.
- Liet al\.\(2024b\)M\. Li, R\. Luo, and O\. MendelevitchHHEM\-2\.1\-Open\.External Links:[Link](https://huggingface.co/vectara/hallucination_evaluation_model)Cited by:[Appendix C](https://arxiv.org/html/2609.03580#A3.p2.1),[§5\.2](https://arxiv.org/html/2609.03580#S5.SS2.p2.1)\.
- Lianget al\.\(2025\)B\. Liang, L\. Peng, J\. Luo, D\. Thaker, K\. H\. R\. Chan, and R\. VidalSECA: semantically equivalent and coherent attacks for eliciting LLM hallucinations\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,Cited by:[§4\.3](https://arxiv.org/html/2609.03580#S4.SS3.p2.1)\.
- Liuet al\.\(2022\)T\. Liu, Y\. Zhang, C\. Brockett, Y\. Mao, Z\. Sui, W\. Chen, and W\. B\. DolanA token\-level reference\-free hallucination detection benchmark for free\-form text generation\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics,Cited by:[§1](https://arxiv.org/html/2609.03580#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Luet al\.\(2025\)S\. Lu, I\. Kuznetsov, and I\. GurevychIdentifying aspects in peer reviews\.arXiv preprint\.Cited by:[§4\.1](https://arxiv.org/html/2609.03580#S4.SS1.p2.1)\.
- Manakulet al\.\(2023\)P\. Manakul, A\. Liusie, and M\. GalesSelfCheckGPT: zero\-resource black\-box hallucination detection for generative large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,Cited by:[§1](https://arxiv.org/html/2609.03580#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Maynezet al\.\(2020\)J\. Maynez, S\. Narayan, B\. Bohnet, and R\. McDonaldOn faithfulness and factuality in abstractive summarization\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,Cited by:[§1](https://arxiv.org/html/2609.03580#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Mishraet al\.\(2024\)A\. Mishra, A\. Asai, V\. Balachandran, Y\. Wang, G\. Neubig, Y\. Tsvetkov, and H\. HajishirziFine\-grained hallucination detection and editing for language models\.InFirst Conference on Language Modeling,Cited by:[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Niuet al\.\(2024\)C\. Niu, Y\. Wu, J\. Zhu, S\. Xu, K\. Shum, R\. Zhong, J\. Song, and T\. ZhangRagtruth: a hallucination corpus for developing trustworthy retrieval\-augmented language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,Cited by:[§1](https://arxiv.org/html/2609.03580#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Ouet al\.\(2025\)J\. Ou, W\. G\. Walden, K\. Sanders, Z\. Jiang, K\. Sun, J\. Cheng, W\. Jurayj, M\. Wanner, S\. Liang, C\. Morgan,et al\.CLAIMCHECK: how grounded are llm critiques of scientific papers?\.arXiv preprint\.Cited by:[§1](https://arxiv.org/html/2609.03580#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.03580#S2.SS2.p1.1)\.
- Russoet al\.\(2025\)G\. Russo, M\. Horta Ribeiro, T\. R\. Davidson, V\. Veselovsky, and R\. WestThe ai review lottery: widespread ai\-assisted peer reviews boost paper scores and acceptance rates\.Proceedings of the ACM on Human\-Computer Interaction\.Cited by:[§2\.2](https://arxiv.org/html/2609.03580#S2.SS2.p1.1)\.
- Schusteret al\.\(2022\)T\. Schuster, S\. Chen, S\. Buthpitiya, A\. Fabrikant, and D\. MetzlerStretching sentence\-pair NLI models to reason over long documents and clusters\.InFindings of the Association for Computational Linguistics: EMNLP 2022,Cited by:[Appendix C](https://arxiv.org/html/2609.03580#A3.p2.1),[§5\.2](https://arxiv.org/html/2609.03580#S5.SS2.p2.1)\.
- Sriramananet al\.\(2024\)G\. Sriramanan, S\. Bharti, V\. S\. Sadasivan, S\. Saha, P\. Kattakinda, and S\. FeiziLlm\-check: investigating detection of hallucinations in large language models\.Advances in Neural Information Processing Systems\.Cited by:[§2\.1](https://arxiv.org/html/2609.03580#S2.SS1.p1.1)\.
- Thakkaret al\.\(2025\)N\. Thakkar, M\. Yuksekgonul, J\. Silberg, A\. Garg, N\. Peng, F\. Sha, R\. Yu, C\. Vondrick, and J\. ZouCan llm feedback enhance review quality? a randomized study of 20k reviews at iclr 2025\.arXiv preprint\.Cited by:[§2\.2](https://arxiv.org/html/2609.03580#S2.SS2.p1.1)\.
- Yeet al\.\(2024\)R\. Ye, X\. Pang, J\. Chai, J\. Chen, Z\. Yin, Z\. Xiang, X\. Dong, J\. Shao, and S\. ChenAre we there yet? revealing the risks of utilizing large language models in scholarly peer review\.arXiv preprint\.Cited by:[§2\.2](https://arxiv.org/html/2609.03580#S2.SS2.p1.1)\.
- Yuet al\.\(2024\)S\. Yu, M\. Luo, A\. Madasu, V\. Lal, and P\. HowardIs your paper being reviewed by an LLM? investigating AI text detectability in peer review\.InProceedings of the Neurips Workshop \(SafeGenAi\),Cited by:[§1](https://arxiv.org/html/2609.03580#S1.p1.1)\.
- Zhuanget al\.\(2025\)Z\. Zhuang, J\. Chen, H\. Xu, Y\. Jiang, and J\. LinLarge language models for automated scholarly paper review: a survey\.Information Fusion\.Cited by:[§2\.2](https://arxiv.org/html/2609.03580#S2.SS2.p1.1)\.

## Appendix AStatistics

### A\.1HalluPeer Dataset Statistic

We collect peer\-review data from OpenReview, covering two major machine learning conferences: ICLR \(2019–2024\) and NeurIPS \(2021–2024\)\. Tab\.[4](https://arxiv.org/html/2609.03580#A1.T4)summarizes the statistics of HalluPeer across venues and years\.

Data Scale\.To ensure balanced representation, we uniformly sample 1,200 papers from each venue\-year pair\. The resulting dataset contains 12,000 papers, 38,063 reviews, and over 1\.02M review sentences, providing substantial coverage for hallucination detection and analysis\.

Hallucination Distribution\.Our injection pipeline constructs more than 10\.1M hallucination templates in total\. The dataset maintains a consistent injection density with an average of 9\.86 templates per sentence across different venue\-year pairs\. Furthermore, the number of injected hallucinations can be flexibly adjusted, enabling controllable construction of positive and negative instances for diverse training and evaluation settings\.

Table 4:Statistics of the HalluPeer Dataset\.We report the total number of source papers, reviews, parsed review sentences, and hallucination templates \(Tpls\) for each venue and year\.VenueYear\# Papers\# Reviews\# Rev\. Sents\# Hallu\. TplsAvg Tpls/SentNeurIPS20211,2003,955110,8881,143,02710\.3120221,2003,54099,935965,6459\.6620231,2004,311118,1081,068,9699\.0520241,2003,923105,118865,8058\.24ICLR20191,2003,14371,031800,00411\.2620201,2002,97069,032760,75011\.0220211,2003,935105,2871,115,23410\.5920221,2004,038119,6691,262,11610\.5520231,2004,081118,7941,151,7379\.7020241,2004,167110,9891,015,8139\.15Total / Avg\.–12,00038,0631,028,85110,149,1009\.86
### A\.2HalluPeer Taxonomy Statistics

We analyze the structural properties of our constructed peer\-review hallucination taxonomy\. The taxonomy is organized as a hierarchical tree structure with a maximum depth of 3\. Tab\.[6](https://arxiv.org/html/2609.03580#A1.T6)summarizes the distribution of nodes and branching factors across different depths\.

Tree dimensions\.The complete taxonomy comprises a total of 265 nodes\. Among these, there are 205 distinct leaf nodes which represent the terminal, fine\-grained hallucination types used for injection\. The hierarchy expands from the root to a maximum depth of 3, ensuring a granular decomposition of review\-specific errors\.

Depth distribution\.The node distribution demonstrates a comprehensive refinement process\. The operational definitions are located at the deepest level\. Specifically, 200 out of the 205 leaf nodes \(approximately 98%\) reside at Depth 3\. This bottom\-heavy structure indicates that the recursive decomposition successfully transforms abstract high\-level concepts into specific, atomic instructions suitable for localized injection\.

Branching characteristics\.The branching factor analysis illustrates the expansion rate of the semantic space\. The root node is initialized into 9 coarse categories \(Depth 1\)\. The average branching factor then transitions from 6\.11 at Depth 1 to 4\.00 at Depth 2\. This suggests a consistent expansion strategy where broad categories are broken down into approximately 4 to 6 subtypes at each intermediate step, balancing breadth and depth before reaching the terminal leaf nodes\.

### A\.3HalluPeer Taxonomy Examples

Depth 1: Coarse\-grained Hallucinations\.Following prior work on hallucination categorization\([Akbar et al\., 2024](https://arxiv.org/html/2609.03580#bib.bib12)\), we adopt nine broad categories that capture common forms of unsupported generation:Number,Entity,False Concatenation,Attribution Failure,Overgeneralization,Reasoning Error,Hyperbole,Temporal, andContext\-based Meaning Error\. The definitions for these categories are provided in Tab\.[5](https://arxiv.org/html/2609.03580#A1.T5)\.

Depth 2: Domain\-Specific Hallucinations\.Our goal at this level is to expand each coarse concept into a set of review\-specific subtypes that are \(i\) mutually distinguishable, \(ii\) consistent in granularity across siblings, and \(iii\) sufficiently operational to facilitate both dataset annotation and controllable hallucination injection\. The prompt template used for this expansion is detailed in Fig\.[7](https://arxiv.org/html/2609.03580#A10.F7)\. As demonstrated in Fig\.[5](https://arxiv.org/html/2609.03580#A1.F5), a coarse\-grained concept like “Number” is decomposed into academic\-contextualized categories such as “Year Discrepancy” and “Statistical Value Mismatch\.”

Depth 3: Fine\-grained Hallucinations\.As further illustrated in Fig\.[5](https://arxiv.org/html/2609.03580#A1.F5), we decompose the domain\-specific subtypes into precise, operational leaf nodes\. For instance, “Year Discrepancy” is instantiated into highly specific error instructions such as “Event Year Fabrication” and “Citation Year Substitution\.”

Table 5:Typology of hallucination categories with their descriptions\.CategoryDescriptionNumberA claim has a different number than the original context \(e\.g\. 20% vs\. 0\.7%\)\. Any number, including year, dimensions, ages, etc\.EntityA claim includes swapped, incorrectly specified, or inserted noun phrases \(e\.g\. one named entity used in a context where another word is expected\)\.False ConcatenationA claim incorrectly combines information about multiple entities or events\.Attribution FailureA claim lacks proper attribution, either crediting the wrong source or presenting information as fact without citation\.OvergeneralizationA claim is based on accurate contextual information but is too broad or too general to be supported by the context\.Reasoning ErrorA claim is based on accurate contextual information but contains a reasoning error or makes an unsupported conclusion\.HyperboleA claim is based on accurate information but exaggerated or overstated\.TemporalA claim does not accurately incorporate tense, modality \(e\.g\.*might*vs\.*will*\), or time reference in relation to the context\.Context\-based MeaningA claim includes incorrect interpretation of idiomatic language, homonyms, or words with multiple meanings, therefore failing to capture the intended meaning\.Number \(Depth: 1\)

A claim has a different number than the original context \(e\.g\. 20% vs\. 0\.7%\)\. Any number, including year, dimensions, ages, etc\.

Year Discrepancy \(Depth: 2\)

*The review text cites a publication or event year that differs from the original context\.*

Event Year Fabrication \(Depth: 3\)

*The review text references a scientific event, discovery, or milestone as occurring in a year that differs from the actual year in the original context\.*

Citation Year Substitution \(Depth: 3\)

*The review text cites a source with a year that is different from the year listed in the original context for that source\.*

Grant or Funding Year Mismatch \(Depth: 3\)

*The review text references a grant, funding award, or institutional support as being issued or active in a year that differs from the original context\.*

Statistical Value Mismatch \(Depth: 2\)

*The review text reports a statistical value \(e\.g\., p\-value, mean, standard deviation\) that differs from the original context\.*

P\-Value Mismatch \(Depth: 3\)

Figure 5:Example of the recursive taxonomy decomposition\.This figure illustrates a subset of our peer\-review hallucination taxonomy\. A general hallucination concept \(e\.g\., ‘‘Number’’\) is recursively decomposed into intermediate subcategories \(e\.g\., ‘‘Year Discrepancy’’\) and ultimately into fine\-grained leaf nodes \(e\.g\., ‘‘Citation Year Substitution’’\)\. These operational leaf nodes provide concrete instructions that enable highly controllable LLM\-based hallucination injection\.Table 6:Structural Statistics of the Hallucination Taxonomy\.We report the total nodes, leaf nodes, and average branching factor\. The high count at Depth 3 highlights the fine\-grained nature of our taxonomy\.DepthTotal NodesLeaf NodesAvg\. Branch\. Factor0109\.001906\.1125554\.0032002000\.00Total265205–

## Appendix BImplementation Details

### B\.1Model Configuration

Unless otherwise specified, we employQwen3\-32Bas the backbone LLM for most components in the HalluPeer dataset construction pipeline\. This includes the human\-written review filtering \(Filterℳ\\textsc\{Filter\}\_\{\\mathcal\{M\}\}\), template feasibility checking \(Checkℳ\\textsc\{Check\}\_\{\\mathcal\{M\}\}\), hallucination template injection \(Injectℳ\\textsc\{Inject\}\_\{\\mathcal\{M\}\}\), and post\-hoc semantic verification \(Verifyℳ\\textsc\{Verify\}\_\{\\mathcal\{M\}\}\) modules\.

Taxonomy generation and refinement follow the multi\-model ensemble procedure described in Appendix[F](https://arxiv.org/html/2609.03580#A6)\. In particular, recursive taxonomy decomposition \(Expandℳ\\textsc\{Expand\}\_\{\\mathcal\{M\}\}\) and hallucination concept description generation \(Describeℳ\\textsc\{Describe\}\_\{\\mathcal\{M\}\}\) are independently performed usingQwen3\-32B,Llama\-3\.3\-70B, andMistral\-Small\-3\.1\-24B\.

To minimize variance and ensure reproducibility, we set the temperature to 0 for all generation operations and explicitly disable extended reasoning or “thinking” modes\. The maximum generation length is adjusted according to the requirements of each module\.

### B\.2Review Aspect Tagging

To tag the focus topic within the peer review sentences, we implement a zero\-shot aspect tagging procedure\. We utilizeLlama\-3\.3\-70Bto categorize each review sentence into predefined aspects\. The model is configured with a temperature of 0\.0 to favor deterministic outputs, and the maximum generation length is set to 512 tokens to accommodate the tag generation\.

## Appendix CBaseline Implementation

We categorize the evaluated baselines into two groups: \(1\) specialized verification frameworks and \(2\) general\-purpose LLM baselines\. Tab\.[7](https://arxiv.org/html/2609.03580#A3.T7)lists the specific checkpoints and model versions used in our experiments for reproducibility\. Prompts used for the LLM\-based baselines are provided in Appendix[J](https://arxiv.org/html/2609.03580#A10)\.

Specialized Verification Frameworks\.We evaluate four specialized hallucination verification frameworks:HHEM\-2\.1\-Open[Li et al\. \(2024b\)](https://arxiv.org/html/2609.03580#bib.bib23),True\-NLI[Laurer et al\. \(2024\)](https://arxiv.org/html/2609.03580#bib.bib24), aseNtLI\-style retriever\-verifier pipeline\([Schuster et al\., 2022](https://arxiv.org/html/2609.03580#bib.bib25)\), andRefChecker[Hu et al\. \(2024\)](https://arxiv.org/html/2609.03580#bib.bib26)\.

HHEM\-2\.1\-Openproduces a continuous faithfulness score between a review sentence and the corresponding source paper\.True\-NLIformulates hallucination detection as a natural language inference \(NLI\) task by estimating whether a review sentence is entailed by the paper content\. ForTrue\-NLI, we compute entailment probabilities between each review sentence and all candidate paper chunks, and use the maximum entailment probability as the final verification score\.

For theseNtLI\-style pipeline333As the originalseNtLIframework does not release a trained verifier, we implement this baseline usingRoBERTa\-large\-MNLI\., the system first retrieves relevant supporting or contradicting evidence from the source paper, followed by NLI\-based verification conditioned on the retrieved evidence\. Furthermore, we evaluateRefChecker[Hu et al\. \(2024\)](https://arxiv.org/html/2609.03580#bib.bib26), a claim\-level framework that operates on extracted claim triplets rather than full sentence representations\. Following the original setup, we instantiateRefCheckerusingMistral\-7Bfor claim extraction alongside an AlignScore\-based checker\.

For all specialized verification frameworks, binary prediction thresholds are selected by maximizing F1 score on the training split and then fixed during test set evaluation\. All inference procedures are conducted on a single NVIDIA H100 GPU\.

General\-Purpose LLM Baselines\.We evaluate both prompting\-based and instruction\-tuned LLM baselines for hallucination detection, hallucination type classification, and hallucination localization\.

For prompting\-based evaluation, we testQwen3\-32B,Llama\-3\.3\-70B,GPT\-OSS\-20B,GPT\-OSS\-120B,Mistral\-Small\-3\.1\-24B,RootSignals\-Judge\-Llama\-70B, andGPT\-5\.2in zero\-shot settings\.

We additionally implement a Retrieval\-Augmented LLM\-as\-a\-Judge \(RA\-LLM\) framework based onQwen3\-32B\. For each review sentence, the system retrieves relevant evidence chunks from the source paper and incorporates them into the prompt context for verification\. Following prior hallucination evaluation work\([Li et al\., 2023](https://arxiv.org/html/2609.03580#bib.bib6)\), we implement three prompting strategies: \(1\)*Knowledge Retrieval \(KR\)*, which augments the prompt with demonstrations; \(2\)*Chain\-of\-Thought \(CoT\)*, which encourages intermediate reasoning for consistency verification; and \(3\)*Sample Contrast \(Contrast\)*, which prompts the model to first identify supporting and contradictory evidence before making the final verification decision\.

For the instruction\-tuned baselines, we perform supervised fine\-tuning \(SFT\) using theUnslothframework to optimize memory usage and training speed\. We apply 4\-bit NormalFloat \(NF4\) quantization\([Dettmers et al\., 2023](https://arxiv.org/html/2609.03580#bib.bib27)\)onQwen2\.5\-3B\-Instruct,Qwen2\.5\-7B\-Instruct, andQwen3\-32B\.

We utilize QLoRA with a rank ofr=16r=16, an alpha parameter ofα=32\\alpha=32, and a dropout rate of 0\.05\. To maximize representational capacity, the LoRA adapters are applied to all linear modules within the attention and MLP layers \(q\_proj, k\_proj, v\_proj, o\_proj, gate\_proj, up\_proj, down\_proj\), without adding bias terms\. During training, we utilize the 8\-bit AdamW optimizer with a peak learning rate of2×10−42\\times 10^\{\-4\}, decayed following a cosine schedule after a 5% linear warmup\. The models are trained inbfloat16precision with gradient checkpointing enabled\.

We maintain an effective batch size of 16 through gradient accumulation\. To ensure the model focuses strictly on generation quality, we apply a completion\-only loss masking strategy, computing the cross\-entropy loss exclusively on the assistant’s response tokens\. The maximum sequence length is truncated at 4096 tokens\.

To ensure stable initial convergence, the early stopping mechanism is only activated after the completion of the first training epoch\. Subsequently, we evaluate the models on a validation split \(10% of the training data\) every200200steps and employ early stopping with a patience of33evaluations based on the F1 score \(after first epoch\)\. During inference, all evaluations are conducted deterministically with the temperature set to 0\.

All training and inference procedures are conducted on a single NVIDIA H100 GPU\.

Table 7:Model identifiers and their corresponding checkpoints or versions used in this study\.BaselineCheckpointHHEM\-2\.1\-Open[https://huggingface\.co/vectara/hallucination\_evaluation\_model](https://huggingface.co/vectara/hallucination_evaluation_model)True\-NLI[https://huggingface\.co/MoritzLaurer/DeBERTa\-v3\-large\-mnli\-fever\-anli\-ling\-wanli](https://huggingface.co/MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli)SENTLI\-style evidence retrieval[https://huggingface\.co/FacebookAI/roberta\-large\-mnli](https://huggingface.co/FacebookAI/roberta-large-mnli)Refchecker \(claim extractor\)[https://huggingface\.co/dongyru/Mistral\-7B\-Claim\-Extractor](https://huggingface.co/dongyru/Mistral-7B-Claim-Extractor)Qwen3\-32B[https://huggingface\.co/Qwen/Qwen3\-32B](https://huggingface.co/Qwen/Qwen3-32B)Llama\-3\.3\-70B[https://huggingface\.co/meta\-llama/Llama\-3\.3\-70B\-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)Mistral\-Small\-3\.1\-24B[https://huggingface\.co/mistralai/Mistral\-Small\-3\.1\-24B\-Instruct\-2503](https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503)GPT\-OSS\-20B[https://huggingface\.co/openai/gpt\-oss\-20b](https://huggingface.co/openai/gpt-oss-20b)GPT\-OSS\-120B[https://huggingface\.co/openai/gpt\-oss\-120b](https://huggingface.co/openai/gpt-oss-120b)GPT\-5\.2[gpt\-5\.2\-2025\-12\-11](https://gpt-5.2-2025-12-11/)RootSignals\-Judge\-Llama\-70B[https://huggingface\.co/root\-signals/RootSignals\-Judge\-Llama\-70B](https://huggingface.co/root-signals/RootSignals-Judge-Llama-70B)Qwen2\.5\-3B\-Instruct[https://huggingface\.co/Qwen/Qwen2\.5\-3B\-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)Qwen2\.5\-7B\-Instruct[https://huggingface\.co/Qwen/Qwen2\.5\-7B\-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
## Appendix DEvaluation Metrics

This appendix details the metrics used for the three tasks defined in Sec\.[5](https://arxiv.org/html/2609.03580#S5)\.

### D\.1Task 1: Hallucination Detection

Accuracy, Precision, Recall, and F1 are computed using standard definitions\. Given the class imbalance in hallucination labels, we additionally report the Matthews Correlation Coefficient \(MCC\), which accounts for all four entries of the confusion matrix and provides a more informative summary under skewed label distributions\.

### D\.2Task 2: Hallucination Type Classification

Evaluation is restricted to sentences annotated as hallucinated in the ground truth\. We report two complementary metrics: \(1\)Macro\-F1:Computed by averaging F1 scores across all hallucination types, treating each class equally regardless of frequency\. \(2\)Micro\-F1:Computed globally over all instances, reflecting overall classification accuracy weighted by class prevalence\.

### D\.3Task 3: Hallucination Localization

Gold and predicted spans are first aligned at the token level using the original tokenization of the review text\.

Token\-level Evaluation\.Gold and predicted spans are converted into BIO tag sequences, where bothBandItags are treated as positive labels andOas negative\. Token\-F1 is then computed over these binary labels\.

Span\-level Evaluation\.We report two complementary metrics: \(1\)Exact Match Span\-F1:Counts a predicted span as correct only if both its start and end boundaries exactly match a gold span\. \(2\)Partial \(Overlap\) Span\-F1:Relaxes this criterion and counts a predicted span as correct if it overlaps with any gold span by at least one token\. In cases of multiple predicted and gold spans within a sentence, matching is performed greedily to avoid double\-counting\.

## Appendix EAdditional Experiment Results

### E\.1Hallucination Detection \(Task 1\)

#### E\.1\.1Ablation Study of RA\-LLM

In this section, we conduct an ablation study to isolate the impact of different evidence retrieval strategies on our retrieval\-augmented LLM judge \(RA\-LLM\)\. Specifically, we compare three configurations: \(i\)No\-Retrieval, where the judge directly consumes the full source paper content without any retrieval filtering; \(ii\)BM25, where the judge is provided with paper content retrieved via BM25; and \(iii\)LCS, where the judge receives the top\-1 paper chunk containing the Longest Common Subsequence \(LCS\) with the target review sentence\. We report results under the same three prompting variants used in our main RA\-LLM experiments \(KR, CoT, and Contrast\), in order to characterize how the impact of evidence differs across prompting designs\. Together, these ablations clarify the dominant failure modes in peer\-review hallucination detection and quantify the upper bound achievable when perfect evidence is available\.

Tab\.[8](https://arxiv.org/html/2609.03580#A5.T8)reports the RA\-LLM ablation on NeurIPS 2024 under three prompting variants\.

Impact of Prompting Strategies\.Among the RA\-LLM variants, the choice of prompting strategy heavily influences the model’s sensitivity to evidence retrieval\. At the sentence level,RA\-LLM \(KR\), which augments prompts with few\-shot demonstrations, consistently achieves the strongest performance \(e\.g\., an MCC of 0\.61 and Accuracy of 0\.82 under theBM25setting\), outperforming both explicit reasoning \(RA\-LLM \(CoT\)\) and contrastive prompting \(RA\-LLM \(Contrast\)\)\. This discrepancy can be attributed to the dependency ofCoTandContraston the completeness of the source paper evidence\. As observed in the table, transitioning from full\-paper access \(No\-Retrieval\) to filtered chunked evidence \(BM25orLCS\) often limits the context required for multi\-step reasoning or contradiction analysis\. Consequently, when the retrieved context lacks sufficient global information, the performance ofCoTandContraststrategies degrades \(e\.g\., the review\-level MCC ofContrastdrops sharply from 0\.26 underNo\-Retrievalto 0\.15 underBM25\)\. In this setting, the concise few\-shot demonstrations inRA\-LLM \(KR\)provide a more resilient and stable supervision signal, allowing the model to remain highly effective even with fragmented or localized evidence\.

Table 8:Ablation of RA\-LLM on HalluPeer \(NeurIPS 2024\)\.Review\-/sentence\-level results are shown before/after the slash\. For the third configuration, we use LCS as the chunking strategy at the review level, and Oracle at the sentence level\.Ablation SettingAcc\.Prec\.Rec\.F1MCCRA\-LLM \(KR\)No\-Retrieval0\.60 / 0\.810\.68 / 0\.720\.39 / 0\.750\.50 / 0\.730\.23 / 0\.59BM250\.61/0\.820\.68/0\.740\.41/ 0\.740\.51/0\.740\.24/0\.61LCS0\.60 / 0\.820\.67 / 0\.720\.40 /0\.760\.50 / 0\.740\.22 / 0\.60RA\-LLM \(CoT\)No\-Retrieval0\.62/0\.780\.71/0\.660\.42 / 0\.730\.53 / 0\.700\.27/0\.53BM250\.61 / 0\.770\.63 / 0\.630\.52 / 0\.780\.57 /0\.700\.22 / 0\.53LCS0\.60 / 0\.760\.61 / 0\.610\.56/0\.800\.58/ 0\.700\.20 / 0\.52RA\-LLM \(Contrast\)No\-Retrieval0\.63/0\.750\.64/0\.610\.61 / 0\.740\.62 / 0\.670\.26/ 0\.48BM250\.57 / 0\.730\.55 / 0\.570\.77 / 0\.840\.64 /0\.680\.15 /0\.49LCS0\.58 / 0\.720\.56 / 0\.560\.80/0\.850\.66/ 0\.680\.18 / 0\.48
#### E\.1\.2In\-Domain Results on ICLR 2024

Tab\.[9](https://arxiv.org/html/2609.03580#A5.T9)presents the performance of hallucination detection at both review and sentence levels on HalluPeer \(ICLR 2024\)\. The results are consistent with those on NeurIPS 2024, with the key findings in Sec\.[5\.4](https://arxiv.org/html/2609.03580#S5.SS4)remaining unchanged\.

Specialized verifiers again generalize poorly, with review\-levelM​C​C≤0\.04MCC\\leq 0\.04\(near\-random\)\. RA\-LLM \(KR\) remains the strongest prompting strategy at the sentence level \(MCC 0\.59\), while prompting\-based frontier LLMs continue to underperform at the review level relative to the sentence level\. Domain\-specific fine\-tuning again dominates: even the compact Qwen2\.5\-3B surpasses all zero\-shot baselines, and Qwen3\-32B \(Fine\-tuned\) achieves the best sentence\-level performance \(F1 0\.94\), with review\-level F1 tied at 0\.89\.

Table 9:Task 1 results on HalluPeer \(ICLR 2024\)\.Review\-/sentence\-level results are shown before/after the slash\. Results are reported under the default evidence and prompting settings described in Sec\.[5\.2](https://arxiv.org/html/2609.03580#S5.SS2)\.Specialized VerificationAcc\.Prec\.Rec\.F1MCCHHEM\-2\.1\-Open0\.51 /0\.630\.51 /0\.480\.61 /0\.650\.56 /0\.550\.03 /0\.25True\-NLI0\.50 / 0\.550\.50 / 0\.400\.48 / 0\.570\.49 / 0\.470\.00 / 0\.10seNtLI0\.52/ 0\.600\.51 / 0\.450\.63/ 0\.580\.57/ 0\.510\.04/ 0\.18Refchecker0\.52/ 0\.480\.52/ 0\.350\.53 / 0\.520\.52 / 0\.420\.04/ \-0\.02LLM \(Prompting\)Acc\.Prec\.Rec\.F1MCCRA\-LLM \(KR\)0\.59 /0\.810\.59 /0\.720\.59 / 0\.760\.59 /0\.740\.18 /0\.59RA\-LLM \(CoT\)0\.59 / 0\.770\.60 / 0\.650\.56 / 0\.770\.58 / 0\.710\.18 / 0\.53RA\-LLM \(Contrast\)0\.56 / 0\.710\.54 / 0\.560\.75 /0\.850\.63 / 0\.670\.13 / 0\.46Qwen3\-32B0\.59 / 0\.790\.59 / 0\.690\.62 / 0\.730\.60 / 0\.710\.19 / 0\.54Llama\-3\.3\-70B0\.57 / 0\.720\.70/ 0\.600\.23 / 0\.640\.35 / 0\.620\.18 / 0\.40Mistral\-Small\-3\.10\.61/ 0\.740\.65 / 0\.610\.48 / 0\.720\.55 / 0\.660\.22/ 0\.45GPT\-OSS\-20B0\.57 / 0\.780\.55 / 0\.670\.74 / 0\.740\.63 / 0\.710\.15 / 0\.53GPT\-OSS\-120B0\.55 / 0\.790\.53 / 0\.670\.87/ 0\.790\.66/ 0\.730\.13 / 0\.56Judge\-Llama\-70B0\.56 / 0\.710\.70/ 0\.590\.23 / 0\.650\.34 / 0\.620\.17 / 0\.39LLM \(Fine\-tuned\)Acc\.Prec\.Rec\.F1MCCQwen2\.5\-3B0\.85 / 0\.920\.84 / 0\.850\.86 /0\.920\.85 / 0\.890\.70 / 0\.82Qwen2\.5\-7B0\.90/ 0\.940\.93/ 0\.910\.85 /0\.920\.89/ 0\.920\.79/ 0\.87Qwen3\-32B0\.89 /0\.960\.90 /0\.960\.88/0\.920\.89/0\.940\.78 /0\.90

### E\.2Hallucination Type Classification \(Task 2\)

#### E\.2\.1Per\-label F1 Results

Tab\.[10](https://arxiv.org/html/2609.03580#A5.T10)presents both review\-level and sentence\-level per\-label F1 results for hallucination category classification on HalluPeer \(NeurIPS 2024\)\.

Table 10:Detailed Per\-label F1 results for Task 2 on HalluPeer \(NeurIPS 2024\)\.Review\-/sentence\-level results are shown before/after the slash\. Per\-label F1 abbreviations: A \(Attribution Failure\), C \(Context\-based Meaning Error\), E \(Entity\), F \(False Concatenation\), H \(Hyperbole\), N \(Number\), O \(Overgeneralization\), R \(Reasoning Error\), T \(Temporal\)\.LLM \(Prompting\)ACEFHNORTQwen3\-32B0\.06 / 0\.150\.03 /0\.290\.30/0\.410\.12 / 0\.260\.00 / 0\.050\.40/ 0\.550\.36/ 0\.230\.10 / 0\.320\.00 / 0\.25Llama\-3\.3\-70B0\.13 / 0\.130\.00 / 0\.110\.22 / 0\.360\.17 / 0\.160\.00 / 0\.050\.30 / 0\.630\.27 / 0\.230\.16 / 0\.290\.00 / 0\.17Mistral\-Small\-3\.10\.19 / 0\.180\.00 / 0\.210\.26 / 0\.390\.27/0\.380\.09/ 0\.070\.36 /0\.730\.22 /0\.260\.22/0\.410\.11/ 0\.30GPT\-OSS\-20B0\.04 / 0\.170\.03 / 0\.120\.19 / 0\.340\.07 / 0\.210\.07 / 0\.070\.30 / 0\.610\.27 / 0\.210\.16 / 0\.320\.00 / 0\.15GPT\-OSS\-120B0\.20/0\.230\.16 / 0\.230\.21 / 0\.330\.04 / 0\.130\.07 / 0\.040\.32 / 0\.600\.29 / 0\.190\.14 / 0\.270\.00 / 0\.18Judge\-Llama\-70B0\.14 / 0\.130\.00 / 0\.120\.21 / 0\.350\.19 / 0\.160\.00 / 0\.040\.33 / 0\.600\.31 / 0\.240\.10 / 0\.280\.00 / 0\.14GPT\-5\.20\.00 / 0\.190\.17/ 0\.190\.20 / 0\.340\.11 / 0\.120\.07 /0\.150\.33 / 0\.660\.19 / 0\.230\.18 / 0\.380\.00 /0\.31LLM \(Fine\-tuned\)ACEFHNORTQwen2\.5\-3B0\.42 / 0\.600\.63/ 0\.710\.25 / 0\.750\.50 / 0\.630\.75/ 0\.770\.55 / 0\.850\.58 / 0\.670\.61 / 0\.760\.15 / 0\.72Qwen2\.5\-7B0\.56/0\.770\.63/0\.840\.40 /0\.840\.49 /0\.830\.63 /0\.890\.51 /0\.910\.58 /0\.890\.61 /0\.880\.35 / 0\.85Qwen3\-32B0\.49 / 0\.640\.56 / 0\.820\.47/ 0\.830\.62/ 0\.820\.72 / 0\.870\.61/0\.910\.63/ 0\.790\.67/ 0\.870\.48/0\.87
#### E\.2\.2In\-Domain Results on ICLR 2024

Tabs\.[11](https://arxiv.org/html/2609.03580#A5.T11)and[12](https://arxiv.org/html/2609.03580#A5.T12)present the overall and per\-label F1 results on HalluPeer \(ICLR 2024\), respectively\. The results are consistent with those on NeurIPS 2024, with the key findings in Sec\.[5\.5](https://arxiv.org/html/2609.03580#S5.SS5)remaining unchanged\.

Zero\-shot prompting remains weak, with the best review\-level Macro\-F1 reaching only 0\.15, and larger models do not consistently outperform smaller ones \(e\.g\., Mistral\-Small\-3\.1 exceeds Llama\-3\.3\-70B and GPT\-OSS\-120B\)\. The sentence\-vs\. review\-level gap persists, while fine\-tuning yields substantial gains, particularly on semantically challenging categories such as Hyperbole, Temporal, and Context\-based Meaning Error\.

Table 11:Task 2 overall results on HalluPeer \(ICLR 2024\)\.Review\-/sentence\-level results are shown before/after the slash\.LLM \(Prompting\)Macro\-F1Micro\-F1Qwen3\-32B0\.09 / 0\.290\.15 / 0\.31Llama\-3\.3\-70B0\.12 / 0\.240\.16 / 0\.27Mistral\-Small\-3\.10\.15/0\.330\.22/0\.36GPT\-OSS\-20B0\.13 / 0\.260\.17 / 0\.29GPT\-OSS\-120B0\.12 / 0\.250\.16 / 0\.26Judge\-Llama\-70B0\.13 / 0\.230\.17 / 0\.26LLM \(Fine\-tuned\)Macro\-F1Micro\-F1Qwen2\.5\-3B0\.53 / 0\.650\.60 / 0\.67Qwen2\.5\-7B0\.60/ 0\.820\.60 / 0\.82Qwen3\-32B0\.59 /0\.850\.63/0\.86Table 12:Detailed Per\-label F1 results for Task 2 on HalluPeer \(ICLR 2024\)\.Review\-/sentence\-level results are shown before/after the slash\. Per\-label F1 abbreviations: A \(Attribution Failure\), C \(Context\-based Meaning Error\), E \(Entity\), F \(False Concatenation\), H \(Hyperbole\), N \(Number\), O \(Overgeneralization\), R \(Reasoning Error\), T \(Temporal\)\.LLM \(Prompting\)ACEFHNORTQwen3\-32B0\.06 / 0\.220\.05 /0\.320\.24/0\.410\.16 / 0\.280\.00 / 0\.040\.00 / 0\.530\.11 / 0\.210\.22 / 0\.340\.00 / 0\.24Llama\-3\.3\-70B0\.11 / 0\.170\.03 / 0\.160\.18 / 0\.350\.05 / 0\.190\.08/ 0\.030\.18 / 0\.630\.17 / 0\.210\.29 / 0\.300\.00 / 0\.14Mistral\-Small\-3\.10\.15 / 0\.230\.05 / 0\.270\.20 / 0\.410\.28/0\.390\.00 /0\.060\.26/0\.710\.14 /0\.240\.29/0\.420\.00 /0\.28GPT\-OSS\-20B0\.21 /0\.240\.08/ 0\.190\.17 / 0\.310\.09 / 0\.180\.00 / 0\.050\.18 / 0\.600\.16 / 0\.230\.27 / 0\.360\.00 / 0\.15GPT\-OSS\-120B0\.32/ 0\.230\.05 / 0\.300\.16 / 0\.320\.00 / 0\.130\.00 / 0\.040\.20 / 0\.590\.15 / 0\.170\.24 / 0\.300\.00 / 0\.22Judge\-Llama\-70B0\.15 / 0\.160\.05 / 0\.170\.20 / 0\.340\.05 / 0\.190\.08/ 0\.030\.16 / 0\.610\.22/ 0\.210\.25 / 0\.260\.00 / 0\.14LLM \(Fine\-tuned\)ACEFHNORTQwen2\.5\-3B0\.44 / 0\.500\.62 / 0\.650\.50/ 0\.660\.54 / 0\.640\.54 / 0\.670\.42 / 0\.810\.62 / 0\.510\.70 / 0\.730\.34 / 0\.68Qwen2\.5\-7B0\.44/ 0\.720\.63 / 0\.790\.48 / 0\.830\.57 / 0\.790\.71/ 0\.870\.50 / 0\.920\.79/ 0\.810\.61 / 0\.860\.65/ 0\.84Qwen3\-32B0\.42 /0\.750\.68/0\.820\.42 /0\.870\.58/0\.820\.68 /0\.880\.57/0\.930\.76 /0\.840\.73/0\.890\.53 /0\.87

### E\.3Hallucination Localization \(Task 3\)

#### E\.3\.1In\-Domain Results on ICLR 2024

Tab\.[13](https://arxiv.org/html/2609.03580#A5.T13)presents the results for hallucination span localization on HalluPeer \(ICLR 2024\)\. The results are consistent with those on NeurIPS 2024, with the key findings in Sec\.[5\.6](https://arxiv.org/html/2609.03580#S5.SS6)remaining unchanged\.

Zero\-shot span localization remains limited, with the consistent gap between Overlap and Exact Span\-F1 also observed\. Fine\-tuning substantially improves grounding, with Qwen3\-32B \(Fine\-tuned\) achieving 0\.93 Token\-F1 and 0\.90 Exact Span\-F1\.

Table 13:Task 3 results on HalluPeer \(ICLR 2024\)\.Review\-level results are reported under default evidence retrieval and prompting settings described in Sec\.[5\.2](https://arxiv.org/html/2609.03580#S5.SS2)\.LLM \(Prompting\)Token\-F1Exact Span\-F1Overlap Span\-F1Qwen3\-32B0\.410\.250\.44Llama\-3\.3\-70B0\.400\.250\.40Mistral\-Small\-3\.10\.340\.200\.34GPT\-OSS\-20B0\.480\.330\.49GPT\-OSS\-120B0\.540\.430\.54Judge\-Llama\-70B0\.410\.280\.40LLM \(Fine\-tuned\)Token\-F1Exact Span\-F1Overlap Span\-F1Qwen2\.5\-3B0\.840\.770\.82Qwen2\.5\-7B0\.870\.830\.86Qwen3\-32B0\.930\.900\.92

### E\.4Cross\-Venue Transferability

This section presents the full quantitative results \(Tab\.[14](https://arxiv.org/html/2609.03580#A5.T14)–[16](https://arxiv.org/html/2609.03580#A5.T16)\) corresponding to the analysis in Sec\.[5\.7](https://arxiv.org/html/2609.03580#S5.SS7)\.

Table 14:Task 1 \(Detection\) cross\-venue transfer\.ModelAcc\.Prec\.Rec\.F1MCCTrain:NeurIPS 2024→\\rightarrowTest:ICLR 2024Qwen2\.5\-3B0\.84 / 0\.850\.83 / 0\.730\.85 /0\.950\.84 / 0\.820\.68 / 0\.72Qwen2\.5\-7B0\.88 / 0\.910\.88 / 0\.830\.89/ 0\.930\.88 / 0\.880\.76 / 0\.81Qwen3\-32B0\.91/0\.940\.97/0\.940\.85 / 0\.890\.90/0\.910\.83/0\.87Train:ICLR 2024→\\rightarrowTest:NeurIPS 2024Qwen2\.5\-3B0\.82 / 0\.920\.81 / 0\.850\.83 / 0\.910\.82 / 0\.880\.63 / 0\.82Qwen2\.5\-7B0\.88/ 0\.940\.89/ 0\.890\.87/0\.920\.88/ 0\.910\.76/ 0\.86Qwen3\-32B0\.86 /0\.950\.87 /0\.940\.85 / 0\.910\.86 /0\.930\.72 /0\.89Table 15:Task 2 \(Type Classification\) cross\-venue transfer\.ModelMacro\-F1Micro\-F1Train:NeurIPS 2024→\\rightarrowTest:ICLR 2024Qwen2\.5\-3B0\.52 / 0\.730\.56 / 0\.73Qwen2\.5\-7B0\.63/0\.860\.64 /0\.86Qwen3\-32B0\.62 / 0\.820\.66/ 0\.83Train:ICLR 2024→\\rightarrowTest:NeurIPS 2024Qwen2\.5\-3B0\.51 / 0\.650\.54 / 0\.68Qwen2\.5\-7B0\.53 / 0\.810\.57 / 0\.81Qwen3\-32B0\.55/0\.850\.59/0\.86Table 16:Task 3 \(Localization\) cross\-venue transfer\.ModelToken\-F1Exact Span\-F1Overlap Span\-F1Train:NeurIPS 2024→\\rightarrowTest:ICLR 2024Qwen2\.5\-3B0\.890\.850\.89Qwen2\.5\-7B0\.880\.850\.88Qwen3\-32B0\.910\.880\.91Train:ICLR 2024→\\rightarrowTest:NeurIPS 2024Qwen2\.5\-3B0\.780\.670\.73Qwen2\.5\-7B0\.870\.830\.86Qwen3\-32B0\.920\.870\.91
### E\.5Cross\-Generation Ablation

Task 1 \(Detection\)\.As shown in Tab\.[17](https://arxiv.org/html/2609.03580#A5.T17), the performance shift \(Δ\\Delta\) when testing on cross\-generator data is extremely marginal\. For the Mistral injector, the F1 score decreases by an average of only 0\.01 to 0\.02\. Remarkably, for the Llama\-3\.3\-70B injector, theΔ\\Deltafor both F1 and MCC is entirely non\-negative across all splits, indicating that performance is preserved and even slightly improved on the unseen generator\.

Task 2 \(Type Classification\)\.Tab\.[18](https://arxiv.org/html/2609.03580#A5.T18)illustrates that classification performance remains highly stable, with shifts generally contained within 0\.05\. Notably, the review\-level Micro\-F1 improves under the Llama injector on both splits \(up to \+0\.066\), while the sentence\-level performance exhibits only minor degradations\.

Task 3 \(Localization\)\.As detailed in Tab\.[19](https://arxiv.org/html/2609.03580#A5.T19), the boundary grounding capabilities of our detector transfer robustly to unseen generators\. The Token\-F1 scores remain within 0\.02 of the in\-domain baseline for Mistral and show slight improvements on the ICLR split for Llama\.

These findings indicate that our fine\-tuned Qwen3\-32B detector does not merely exploit spurious lexical artifacts or stylistic tics\. Instead, it captures the semantic inconsistencies that define peer\-review hallucinations, rather than exploiting superficial lexical artifacts\.

VenueAccuracyPrecisionRecallF1MCCInCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowHallucination Injector: Mistral\-Small\-3\.1NeurIPS90\.4/94\.288\.7/93\.5−1\.6\-1\.6/−0\.7\-0\.795\.6/93\.895\.5/93\.1−0\.1\-0\.1/−0\.7\-0\.784\.6/88\.681\.2/87\.4−3\.3\-3\.3/−1\.3\-1\.389\.8/91\.187\.8/90\.1−2\.0\-2\.0/−1\.0\-1\.081\.3/86\.878\.3/85\.4−2\.9\-2\.9/−1\.5\-1\.5ICLR89\.2/95\.588\.4/95\.2−0\.8\-0\.8/−0\.3\-0\.389\.9/95\.592\.2/95\.8\+2\.3\+2\.3/\+0\.3\+0\.388\.3/91\.883\.9/90\.5−4\.4\-4\.4/−1\.3\-1\.389\.1/93\.687\.9/93\.1−1\.2\-1\.2/−0\.5\-0\.578\.4/90\.277\.1/89\.5−1\.3\-1\.3/−0\.7\-0\.7Hallucination Injector: Llama\-3\.3\-70BNeurIPS90\.4/94\.292\.1/94\.9\+1\.7\+1\.7/\+0\.8\+0\.895\.6/93\.896\.1/94\.0\+0\.5\+0\.5/\+0\.2\+0\.284\.6/88\.687\.7/91\.0\+3\.1\+3\.1/\+2\.4\+2\.489\.8/91\.191\.7/92\.5\+1\.9\+1\.9/\+1\.4\+1\.481\.3/86\.884\.5/88\.7\+3\.2\+3\.2/\+1\.8\+1\.8ICLR89\.2/95\.589\.3/95\.9\+0\.1\+0\.1/\+0\.3\+0\.389\.9/95\.590\.0/95\.1\+0\.1\+0\.1/−0\.4\-0\.488\.3/91\.888\.4/93\.2\+0\.1\+0\.1/\+1\.4\+1\.489\.1/93\.689\.2/94\.2\+0\.1\+0\.1/\+0\.6\+0\.678\.4/90\.278\.7/91\.0\+0\.2\+0\.2/\+0\.8\+0\.8

Table 17:Task 1 \(Detection\) cross\-generator ablation — Hallucination Injectors: Mistral\-Small\-3\.1 and Llama\-3\.3\-70B\.The Qwen3\-32B detector is fine\-tuned only on Qwen\-injected data and evaluated on Qwen\-injected \(In\-domain\) vs\. Mistral\- or Llama\-injected \(Cross\) test sets\. Venue refers to the HalluPeer NeurIPS 2024 and ICLR 2024 splits\. Values are presented as review\-level / sentence\-level \(in %\)\.Δ=Cross−In\\Delta=\\text\{Cross\}\-\\text\{In\};↓\\downarrowindicates smaller\|Δ\|\|\\Delta\|is better\.VenueMacro\-F1Micro\-F1InCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowHallucination Injector: Mistral\-Small\-3\.1NeurIPS58\.5/82\.461\.4/79\.2\+2\.9\+2\.9/−3\.2\-3\.259\.3/83\.864\.5/81\.1\+5\.2\+5\.2/−2\.6\-2\.6ICLR59\.5/85\.355\.7/84\.8−3\.8\-3\.8/−0\.5\-0\.562\.8/85\.660\.4/85\.0−2\.4\-2\.4/−0\.6\-0\.6Hallucination Injector: Llama\-3\.3\-70BNeurIPS58\.5/82\.462\.4/76\.5\+3\.9\+3\.9/−5\.9\-5\.959\.3/83\.865\.8/78\.9\+6\.6\+6\.6/−4\.9\-4\.9ICLR59\.5/85\.357\.6/83\.7−1\.8\-1\.8/−1\.6\-1\.662\.8/85\.664\.4/84\.5\+1\.6\+1\.6/−1\.1\-1\.1Table 18:Task 2 \(Type Classification\) cross\-generator ablation — Hallucination Injectors: Mistral\-Small\-3\.1 and Llama\-3\.3\-70B\.The Qwen3\-32B detector is fine\-tuned only on Qwen\-injected data and evaluated on Qwen\-injected \(In\-domain\) vs\. Mistral\- or Llama\-injected \(Cross\) test sets\. Venue refers to the HalluPeer NeurIPS 2024 and ICLR 2024 splits\. Values are presented as review\-level / sentence\-level \(in %\)\.Δ=Cross−In\\Delta=\\text\{Cross\}\-\\text\{In\};↓\\downarrowindicates smaller\|Δ\|\|\\Delta\|is better\.VenueToken\-F1Exact Span\-F1Overlap Span\-F1InCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowInCrossΔ↓\\Delta\\downarrowHallucination Injector: Mistral\-Small\-3\.1NeurIPS90\.688\.8−1\.8\-1\.885\.681\.0−4\.5\-4\.589\.886\.6−3\.2\-3\.2ICLR92\.892\.3−0\.4\-0\.489\.687\.6−2\.0\-2\.092\.390\.4−1\.9\-1\.9Hallucination Injector: Llama\-3\.3\-70BNeurIPS90\.689\.2−1\.4\-1\.485\.684\.6−1\.0\-1\.089\.886\.8−3\.0\-3\.0ICLR92\.894\.8\+2\.0\+2\.089\.692\.2\+2\.7\+2\.792\.393\.5\+1\.2\+1\.2Table 19:Task 3 \(Localization\) cross\-generator ablation — Hallucination Injectors: Mistral\-Small\-3\.1 and Llama\-3\.3\-70B\.The Qwen3\-32B detector is fine\-tuned only on Qwen\-injected data and evaluated on Qwen\-injected \(In\-domain\) vs\. Mistral\- or Llama\-injected \(Cross\) test sets\. Venue refers to the HalluPeer NeurIPS 2024 and ICLR 2024 splits\. Values are presented at the review\-level \(in %\), as Task 3 has no sentence\-level split\.Δ=Cross−In\\Delta=\\text\{Cross\}\-\\text\{In\};↓\\downarrowindicates smaller\|Δ\|\|\\Delta\|is better\.

## Appendix FDetails of the Taxonomy Generation and Refinement Pipeline

To reduce model\-specific biases introduced during recursive taxonomy decomposition, we implement a three\-stage refinement pipeline consisting of multi\-model ensemble generation, automated overlap identification, and human expert validation\. As illustrated in Fig\.[3](https://arxiv.org/html/2609.03580#S3.F3), the pipeline progressively refines the initially generated taxonomy into a globally consistent and human\-validated hierarchy\.

### F\.1Stage 1: Multi\-Model Taxonomy Ensemble

Starting from the predefined coarse\-grained anchors \(e\.g\.,*Number*,*Entity*\), we independently generate taxonomy trees using three LLM proposers:Qwen3\-32B,Llama\-3\.3\-70B, andMistral\-Small\-3\.1\-24B\. The taxonomy generated byQwen3\-32Bis treated as the primary structure\.

To determine whether a concept generated in the primary tree is supported by other model\-generated trees, we concatenate each node’s concept name and operational description into a single textual representation and encode it using theBAAI/bge\-large\-en\-v1\.5embedding model\. For each node in the primary tree, we compute cosine similarity scores against all nodes in the other trees\. A node is retained only if at least one cross\-tree node achieves a cosine similarity score greater than 0\.8\. This cross\-model agreement criterion helps reduce model\-specific artifacts and improves the robustness of the induced taxonomy structure\.

### F\.2Stage 2: Global Overlap Identification

Although local decomposition constraints encourage non\-overlapping sibling categories, semantically redundant concepts may still emerge across different branches of the taxonomy\. To improve global mutual exclusivity, we perform a taxonomy\-wide overlap identification stage\.

We first construct a candidate comparison pool by enumerating all possible node pairs across the taxonomy, excluding direct parent–child node pairs\. Exhaustive human inspection over this candidate space would result in an impractically large number of comparisons \(approximately 86,800 node pairs\)\. Therefore, we first apply a multi\-LLM consensus filtering procedure in which multiple evaluator models independently assess whether two taxonomy nodes exhibit substantial semantic overlap\. Only node pairs identified by consensus are forwarded for manual review\. This process reduces the candidate pool to 434 potentially overlapping node pairs while mitigating blind spots introduced by any individual evaluator model\.

### F\.3Stage 3: Human Expert Validation

In the final stage, human annotators manually review the filtered set of candidate overlap pairs\. Annotators inspect the semantic concepts, operational descriptions, and taxonomy paths associated with each node pair, and determine whether the nodes should be merged, preserved as distinct concepts, or revised for clearer separation\.

The core judgment criteria used during human validation are described in Fig\.[6](https://arxiv.org/html/2609.03580#A10.F6)\. This human\-in\-the\-loop verification step helps eliminate residual redundancy and improve the conceptual consistency of the final taxonomy\.

## Appendix GEvaluation on Authentic Reviews

To quantitatively evaluate whether detectors trained on synthetic hallucinations transfer to naturally occurring reviewer errors, we construct a manually annotated set of authentic NeurIPS 2024 reviews\. The evaluation is designed to avoid model\-dependent annotation and to measure both true\-positive and false\-positive behavior\.

### G\.1Setup

Two expert annotators manually examined 1,161 real NeurIPS 2024 reviews against their corresponding submissions and identified 20 reviewer hallucinations, corresponding to a natural prevalence of approximately 1\.7%\. Unlike the case study in Sec\.[6](https://arxiv.org/html/2609.03580#S6), the annotation was performed independently of any detector output: annotators read each review directly and determined which instances constitute genuine reviewer hallucinations\.

The fine\-tuned detectors were trained exclusively on synthetic HalluPeer ICLR 2024 data and had no access to the manually annotated authentic reviews\. We evaluate the resulting detectors directly on the 1,161 authentic reviews, making this an out\-of\-distribution evaluation from synthetic hallucinations to naturally occurring reviewer errors\.

### G\.2Results and Analysis

True and False Positive Rates\.Table[20](https://arxiv.org/html/2609.03580#A7.T20)reports the true\-positive rate \(TPR\) and false\-positive rate \(FPR\) on the manually annotated authentic reviews\. Given the low natural prevalence of hallucinations, recall is the primary measure of detection capability, while FPR characterizes the amount of reviewer\-level screening required\.

Our fine\-tuned Qwen3\-32B detector recovers all 20 authentic hallucinations, achieving TPR=100\.0%=100\.0\\%at FPR=22\.1%=22\.1\\%\. This substantially improves recall over the zero\-shot Qwen3\-32B baseline \(TPR=95\.0%=95\.0\\%\), while Qwen2\.5\-7B reaches TPR=70\.0%=70\.0\\%\. Among the compared frontier\-model prompting baselines, Claude Opus 4\.7 achieves TPR=85\.0%=85\.0\\%at FPR=0\.3%=0\.3\\%\.

Effect of Fine\-tuning\.To isolate the effect of training on HalluPeer, we compare each fine\-tuned detector against its corresponding zero\-shot model\. Fine\-tuning Qwen3\-32B improves TPR from 95\.0% to 100\.0% while reducing FPR from 29\.5% to 22\.1%\. Similarly, Qwen2\.5\-7B improves TPR from 70\.0% to 80\.0% and reduces FPR from 34\.0% to 23\.1%\. The simultaneous improvement in TPR and reduction in FPR indicates that the gain is not explained by a simple shift in the decision threshold\.

Practical Screening\.At the reported operating point, the fine\-tuned Qwen3\-32B detector reduces the 1,161 authentic reviews to 272 candidates while recovering all 20 annotated hallucinations\. Thus, the detector can serve as a high\-recall screening stage, substantially narrowing the candidate pool while retaining the authentic hallucinations identified by expert annotators\. Final adjudication can then be performed by a human or a stronger frontier model\.

Table 20:Detection performance on manually annotated real NeurIPS 2024 reviews\.The evaluation set contains 1,161 reviews with 20 positive and 1,141 negative instances \(1\.7% prevalence\)\. Fine\-tuned models are trained on synthetic HalluPeer ICLR 2024 data only\. Values are reported in %\.ModelTPRFPRTP/FNFP/TNFrontier LLM \(prompting\)Claude Opus 4\.785\.00\.317/33/1138Open\-source, zero\-shotGPT\-OSS\-120B100\.033\.020/0376/765Qwen3\-32B95\.029\.519/1337/804Qwen2\.5\-7B70\.034\.014/6388/753Open\-source, fine\-tuned on HalluPeerQwen3\-32B100\.022\.120/0252/889Qwen2\.5\-7B80\.023\.116/4264/877

## Appendix HAnnotation Protocol for Authentic Review Analysis

We selected 200 consecutive flagged instances from the 13,803 human\-written NeurIPS 2024 reviews identified by our fine\-tuned detector for manual inspection\. Two expert annotators each reviewed a disjoint subset, checking whether each flagged instance constituted a genuine hallucination and whether the predicted hallucination type matched the manually assigned type\. Candidate hallucinations and uncertain cases were subsequently reviewed jointly with a senior expert, with final labels determined through discussion and consensus\. This process yielded 11 validated cases, which are presented in Tab\.[21](https://arxiv.org/html/2609.03580#A9.T21)–[23](https://arxiv.org/html/2609.03580#A9.T23)\.

## Appendix IAlgorithm

### I\.1Algorithm – Hallucination Taxonomy Construction

Algorithm[1](https://arxiv.org/html/2609.03580#algorithm1)summarizes the recursive procedure used to build the hallucination taxonomy\. Starting from the predefined first\-level concepts𝒱\(1\)\\mathcal\{V\}^\{\(1\)\}, the algorithm performs a depth\-first traversal that progressively refines each coarse concept into finer subtypes\. For a node at depthdd, if the maximum depthDDhas not been reached, the LLM proposerExpandℳ\\textsc\{Expand\}\_\{\\mathcal\{M\}\}is queried to generate a set of child concepts𝒞v\\mathcal\{C\}\_\{v\}conditioned on the node and its descriptionδv\\delta\_\{v\}\. Each generated child is attached to its parent, assigned an operational description viaDescribeℳ\\textsc\{Describe\}\_\{\\mathcal\{M\}\}, added to the global node set𝒱\\mathcal\{V\}, and then recursively decomposed at depthd\+1d\+1\. The recursion terminates under two conditions: when the maximum depthDDis reached, or whenExpandℳ\\textsc\{Expand\}\_\{\\mathcal\{M\}\}returns no children \(𝒞v=∅\\mathcal\{C\}\_\{v\}=\\emptyset\), indicating that the concept is atomic\. The procedure yields the full taxonomy tree𝒱\\mathcal\{V\}, whose leaf nodes serve as the fine\-grained hallucination types used for injection\.

Algorithm 1LLM\-Guided Recursive Taxonomy Decomposition \(DFS\)Input:Root node

v\(0\)v^\{\(0\)\}, Predefined first\-level nodes

𝒱\(1\)\\mathcal\{V\}^\{\(1\)\}, LLM

ℳ\\mathcal\{M\}, Max depth

DD
Output:Full taxonomy tree

𝒱\\mathcal\{V\}
1

𝒱←\{v\(0\)\}∪𝒱\(1\)\\mathcal\{V\}\\leftarrow\\\{v^\{\(0\)\}\\\}\\cup\\mathcal\{V\}^\{\(1\)\};

2foreach*v∈𝒱\(1\)v\\in\\mathcal\{V\}^\{\(1\)\}*do

//Start DFS from each first\-level node

3

Decompose​\(v,1\)\\text\{Decompose\}\(v,1\);

4end foreach

5return

𝒱\\mathcal\{V\};

6Function*Decompose\(*v,dv,d*\)*:

7if*d<Dd<D*then

//LLM generates subcategories

8

𝒞v←Expandℳ​\(v,δv\)\\mathcal\{C\}\_\{v\}\\leftarrow\\textsc\{Expand\}\_\{\\mathcal\{M\}\}\(v,\\delta\_\{v\}\);

9if*𝒞v≠∅\\mathcal\{C\}\_\{v\}\\neq\\emptyset*then

10foreach*u∈𝒞vu\\in\\mathcal\{C\}\_\{v\}*do

11

parent​\(u\)←v\\text\{parent\}\(u\)\\leftarrow v;

12

δu←Describeℳ​\(u\)\\delta\_\{u\}\\leftarrow\\textsc\{Describe\}\_\{\\mathcal\{M\}\}\(u\);

13

𝒱←𝒱∪\{u\}\\mathcal\{V\}\\leftarrow\\mathcal\{V\}\\cup\\\{u\\\};

//Recursive DFS call

14Decompose\(*u,d\+1u,d\+1*\);

15end foreach

16end if

17end if

### I\.2Algorithm – Hallucination Injection Template Construction

Algorithm[2](https://arxiv.org/html/2609.03580#algorithm2)details the procedure for generating and filtering sentence\-level hallucination templates\. The process iterates through each sentencesswithin the review corpus\. To circumvent the intractability of naively instantiating every template, we first apply a coarse\-grained screening by evaluating which first\-level anchor conceptsc∈𝐕\(1\)c\\in\\mathbf\{V\}^\{\(1\)\}are applicable to the context\. For each compatible anchor, fine\-grained templates are constructed exclusively from its descending leaf nodes𝐕leaf\(c\)\\mathbf\{V\}\_\{\\text\{leaf\}\}^\{\(c\)\}, forming the initial candidate set𝒯scandidate\\mathcal\{T\}\_\{s\}^\{\\text\{candidate\}\}\. Subsequently, we leverage an LLM\-based compatibility checkCheckℳ​\(T,s\)\\textsc\{Check\}\_\{\\mathcal\{M\}\}\(T,s\)guided by a dedicated screening prompt \(detailed in Appendix[J\.7](https://arxiv.org/html/2609.03580#A10.SS7)\)\. This semantic validation filters out templates whose hallucination types cannot naturally map onto the target sentence, ultimately yielding the feasible template set𝒯feasible\\mathcal\{T\}^\{\\text\{feasible\}\}\.

### I\.3Algorithm – Automated Hallucination Injection Pipeline with Semantic Verification

Algorithm[3](https://arxiv.org/html/2609.03580#algorithm3)describes the automated injection pipeline that turns feasible templates into verified hallucinated sentences\. For each feasible template\(s,as,δv,πv\)\(s,a\_\{s\},\\delta\_\{v\},\\pi\_\{v\}\), the LLM injectorInjectℳ\\textsc\{Inject\}\_\{\\mathcal\{M\}\}rewrites the original sentencessaccording to the taxonomy instructionδv\\delta\_\{v\}, producing a candidate hallucinated sentences~\\tilde\{s\}\. To ensure that the injection actually altered the meaning, a post\-hoc verifierVerifyℳ\\textsc\{Verify\}\_\{\\mathcal\{M\}\}comparess~\\tilde\{s\}againstssand returns whether the two are semantically equivalent\. A candidate is retained only whenVerifyℳ​\(s,s~\)=0\\textsc\{Verify\}\_\{\\mathcal\{M\}\}\(s,\\tilde\{s\}\)=0, i\.e\., the generated sentence is judged*not*equivalent to the original and therefore carries a genuine hallucination; equivalent rewrites \(output11\) are discarded as failed injections\. The procedure returns the set of verified hallucinated sentences\{s~\}\\\{\\tilde\{s\}\\\}\.

Algorithm 2Sentence\-level Hallucination Injection Template ConstructionInput:Set of papers

𝒫\\mathcal\{P\}, hierarchical taxonomy node set

𝐕\\mathbf\{V\}\(where

𝐕\(1\)⊂𝐕\\mathbf\{V\}^\{\(1\)\}\\subset\\mathbf\{V\}denotes first\-level anchors\), LLM

ℳ\\mathcal\{M\}for template compatibility checking

Output:Sentence\-level feasible templates

\{𝒯feasible\}\\\{\\mathcal\{T\}^\{\\text\{feasible\}\}\\\}
1foreach*p∈𝒫p\\in\\mathcal\{P\}*do

2foreach*r∈ℛpr\\in\\mathcal\{R\}\_\{p\}*do

3foreach*s∈rs\\in r*do

4

𝒯scandidate←∅\\mathcal\{T\}\_\{s\}^\{\\text\{candidate\}\}\\leftarrow\\emptyset;

//Step 1: Coarse\-grained screening via anchor nodes from𝐕\\mathbf\{V\}

5foreach*anchor nodec∈𝐕\(1\)c\\in\\mathbf\{V\}^\{\(1\)\}*do

6if*IsApplicable​\(s,c\)\\textsc\{IsApplicable\}\(s,c\)*then

//Extract leaf nodes descending from anchorcc

7

𝐕leaf\(c\)←\{v∈𝐕∣v​is a leaf descendant of​c\}\\mathbf\{V\}\_\{\\text\{leaf\}\}^\{\(c\)\}\\leftarrow\\\{v\\in\\mathbf\{V\}\\mid v\\text\{ is a leaf descendant of \}c\\\};

8

𝒯ℒ←\{Ts,v∣v∈𝐕leaf\(c\)\}\\mathcal\{T\}\_\{\\mathcal\{L\}\}\\leftarrow\\\{T\_\{s,v\}\\mid v\\in\\mathbf\{V\}\_\{\\text\{leaf\}\}^\{\(c\)\}\\\};

9

𝒯scandidate←𝒯scandidate∪𝒯ℒ\\mathcal\{T\}\_\{s\}^\{\\text\{candidate\}\}\\leftarrow\\mathcal\{T\}\_\{s\}^\{\\text\{candidate\}\}\\cup\\mathcal\{T\}\_\{\\mathcal\{L\}\};

10end if

11end foreach

//Step 2: Compatibility check via LLM screening prompt

12

𝒯feasible←∅\\mathcal\{T\}^\{\\text\{feasible\}\}\\leftarrow\\emptyset;

13foreach*T∈𝒯scandidateT\\in\\mathcal\{T\}\_\{s\}^\{\\text\{candidate\}\}*do

14if*Checkℳ​\(T,s\)=feasible\\textsc\{Check\}\_\{\\mathcal\{M\}\}\(T,s\)=\\text\{feasible\}*then

15

𝒯feasible←𝒯feasible∪\{T\}\\mathcal\{T\}^\{\\text\{feasible\}\}\\leftarrow\\mathcal\{T\}^\{\\text\{feasible\}\}\\cup\\\{T\\\};

16end if

17end foreach

18end foreach

19end foreach

20end foreach

21return

\{𝒯feasible\}\\\{\\mathcal\{T\}^\{\\text\{feasible\}\}\\\};

Algorithm 3Automated Hallucination Injection Pipeline with Semantic VerificationInput:Sentence\-level feasible templates

\{𝒯feasible\}\\\{\\mathcal\{T\}^\{\\text\{feasible\}\}\\\}, LLM injector

Injectℳ\\textsc\{Inject\}\_\{\\mathcal\{M\}\}, LLM

Verifyℳ\\textsc\{Verify\}\_\{\\mathcal\{M\}\}for post\-hoc semantic verification

Output:Hallucinated sentences

\{s~\}\\\{\\tilde\{s\}\\\}
1foreach*\(s,as,δv,πv\)∈𝒯sfeasible\(s,a\_\{s\},\\delta\_\{v\},\\pi\_\{v\}\)\\in\\mathcal\{T\}\_\{s\}^\{\\text\{feasible\}\}*do

//Generate hallucinated sentence

2

s~←Injectℳ​\(s,δv\)\\tilde\{s\}\\leftarrow\\textsc\{Inject\}\_\{\\mathcal\{M\}\}\(s,\\delta\_\{v\}\);

//Post\-hoc semantic verification

3if*Verifyℳ​\(s,s~\)=0\\textsc\{Verify\}\_\{\\mathcal\{M\}\}\(s,\\tilde\{s\}\)=0*then

//Retain hallucinated sentence if semantically distinct

4Store

s~\\tilde\{s\};

5end if

6end foreach

7return

\{s~\}\\\{\\tilde\{s\}\\\};

Table 21:Examples of paper\-review hallucinations categorized by taxonomy depth, review content, and verification reasoning\.Class \(Depth 1→\\rightarrow2→\\rightarrow3\)Review ContentLabor ReasoningPaper:EZ\-HOI: VLM Adaptation via Guided Prompt Learning for Zero\-Shot HOI DetectionEntity
↪\\hookrightarrowClaim Entity Extension
↪\\hookrightarrowAttribute SubstitutionThe paper introduces Intent\-Coupled Contrastive Learning \(ICL\), a groundbreaking method that significantly enhances user embeddings by incorporating intent information\.The paper explicitly names the method “intent\-assisted contrastive learning” \(ICL\)\. The reviewer alters the algorithm’s core attribute name \(assisted→\\rightarrowcoupled\) while retaining its acronym and structure\.Key quote:“Section 3\.4\.3\. Intent\-assisted contrastive learning”Paper:Opponent Modeling based on Subgoal InferenceFalse Concatenation
↪\\hookrightarrowEntity Misattribution
↪\\hookrightarrowMethod Misattribution\- L204 and L206\-207 — both sentences here claim seemingly contradictory statements: “…adopting an optimistic strategy akin to the minimax strategy, which applies to cooperative games” seems to contradict the following statement “…leading to a conservative strategy similar to the minimax strategy, which is commonly used for general\-sum games”\.The reviewer misquotes the paper by attributing the “minimax” strategy \(a method described for general\-sum games\) to cooperative games, substituting it for the actual method \(“maximax”\)\. This methodological misattribution causes the reviewer to incorrectly concatenate the two distinct game contexts and fabricate a false contradiction\.Key quote:“thus adopting an optimistic strategy akin to the maximax strategy \[6\], which applies to cooperative games”Paper:EZ\-HOI: VLM Adaptation via Guided Prompt Learning for Zero\-Shot HOI DetectionNumber
↪\\hookrightarrowDimensional Parameter Error
↪\\hookrightarrowScale Parameter Error“, why is the threshold value chosen to be 0\.5 instead of 0\.7 or other?The paper explicitly states that the confidence threshold for the object detector isθ=0\.2\\theta=0\.2\. The reviewer hallucinates the algorithmic scale parameters \(0\.50\.5and0\.70\.7\) as the premise for their clarification question\.Note:While interrogative sentences are often excluded from strict hallucination judgments, the foundational premise of the question contains a clear numerical parameter distortion\.Key quote:“We use an off\-the\-shelf object detector and add a thresholdθ\\thetato filter out some low\-confident predictions and we setθ=0\.2\\theta=0\.2”Paper:ParallelEdits: Efficient Multi\-Aspect Text\-Driven Image Editing with Attention GroupingAttribution Failure
↪\\hookrightarrowMisattributed Result
↪\\hookrightarrowMischaracterized Paper ResultFigure 1 illustrates the swapping of multiple objects within the same image, but it only demonstrates the addition of a single object\.The reviewer mischaracterizes the scope of the paper’s qualitative results by claiming Figure 1 “only demonstrates the addition of a single object\.” In reality, the evidence explicitly describes Figure 1 as showing multi\-aspect edits, including background changes, object removal, and object swapping\. The reviewer artificially narrows the scope of the presented results\.Key quote:“such as adding a necktie to a cat and changing the background wall to a beach \(Fig\. 1, Left\), or removing a man”Table 22:Examples of paper\-review hallucinations categorized by taxonomy depth, review content, and verification reasoning\.Class \(Depth 1→\\rightarrow2→\\rightarrow3\)Review ContentLabor ReasoningPaper:Diffusion Models are Certifiably Robust ClassifiersContext Meaning Error
↪\\hookrightarrowJargon Misapplication
↪\\hookrightarrowAcronym MisinterpretationThen, it proposes Exact Posterior Noised Diffusion Classifier \(EPNDC\) and Approximated Posterior Noised Diffusion Classifier \(APNDC\) by deriving ELBO upper bounds onlog⁡p⁡\(xτ\)\\log p\(x\_\{\\tau\}\)and thereby enabling classifying noisy images\.ELBO stands for Evidence*Lower*Bound, so calling them “upper bounds” is wrong\. And the paper actually boundslog⁡p⁡\(xτ∣y\)\\log p\(x\_\{\\tau\}\\mid y\), notlog⁡p⁡\(xτ\)\\log p\(x\_\{\\tau\}\)\.Key quote:“we generalize diffusion classifiers to calculatep⁡\(y∣xτ\)p\(y\\mid x\_\{\\tau\}\)by estimatinglog⁡p⁡\(xτ∣y\)\\log p\(x\_\{\\tau\}\\mid y\)using its ELBO”Paper:Q\-VLM: Post\-training Quantization for Large Vision\-Language ModelsContext Meaning Error
↪\\hookrightarrowJargon Misapplication
↪\\hookrightarrowJargon Term SubstitutionThe authors separate several layers in a LVLM into blocks and search for the optimal quantization bitwidth for each block individually\.The reviewer replaces the paper’s actual methodological jargon \(“rounding function”\) with an incorrect term \(“quantization bitwidth”\)\. In the field of model quantization, determining the optimal rounding function is conceptually distinct from determining bitwidth\.Key quote:“Searching the optimal rounding function by considering the output quantization errors for each block achieves better trade\-off between the search cost and the quantization accuracy”Paper:Fantasy: Transformer Meets Transformer in Text\-to\-Image GenerationEntity
↪\\hookrightarrowClaim Entity Ext\.
↪\\hookrightarrowAttribute SubstitutionUnlike commonly used text encoders like CLIP and T5, this study introduces an efficient decoder\-only LLM, phi\-3, achieving better semantic understanding\.The reviewer replaces the specific model entity used in the paper \(“Phi\-2”\) with a different model version \(“phi\-3”\)\. Although the discrepancy surfaces as a digit change, “Phi\-2” and “Phi\-3” represent distinct algorithmic entities in NLP\. The reviewer alters the claim’s core algorithmic attribute by substituting the named entity\.Key quote:“we employ Phi\-2 \[24\], a state\-of\-the\-art, lightweight LLM, as the text encoder”Paper:Generalization Bound and Learning Methods for Data\-Driven Projections in Linear ProgrammingReasoning Error
↪\\hookrightarrowMischaracterized Method\.
↪\\hookrightarrowMischaracterized AlgorithmTo achieve this, it is necessary to choose a goodPPthat minimizes the empirical optimal value\.The reviewer states the methodological goal is to “minimize” the empirical optimal value\. However, the paper explicitly aims to*maximize*the expected/empirical optimal value via gradient ascent\. By completely reversing the direction of optimization, the reviewer fundamentally mischaracterizes the paper’s core computational technique and algorithm\.Key quote:“we can use the gradient ascent method to maximizeu⁡\(P,π\)u\(P,\\pi\)under the regularity condition”Paper:DataStealing: Steal Data from Diffusion Models in Federated Learning with Multiple TrojansOvergeneralization
↪\\hookrightarrowResult\-Based Overgen\.
↪\\hookrightarrowDomain Extrapolation\-Broad Applicability: The proposed methodologies and findings are not limited to a specific application but are broadly applicable to various domains where FL and generative models are used\.The reviewer claims the paper’s findings are broadly applicable to various domains involving federated learning \(FL\) and generative models\. However, the paper specifically targets and tests only diffusion models in FL for image generation \(e\.g\., CIFAR10, CelebA\)\. The reviewer unjustifiably extrapolates a narrow, diffusion\-specific empirical result to all FL and generative\-model domains without any experimental evidence or justification provided in the paper\.Key quote:“We propose an attack method for DataStealing, named AdaSCP, to defeat advanced distance\-based defenses and seamlessly incorporate backdoor gradients into the global diffusion model\.”Table 23:Examples of paper\-review hallucinations categorized by taxonomy depth, review content, and verification reasoning\.Class \(Depth 1→\\rightarrow2→\\rightarrow3\)Review ContentLabor ReasoningPaper:Convolutional Differentiable Logic Gate NetworksHyperbole
↪\\hookrightarrowInflated Comparison Advantage
↪\\hookrightarrowBaseline Performance FabricationLowest latency of all SOTA baseline results, the majority of them being much slower with even worse accuracy\.While the latency claim is well supported, the accuracy comparison is highly exaggerated\. The reviewer asserts that the majority of baselines have “even worse accuracy” than the proposed LogicTreeNet\-B\. However, Table 1 shows this baseline performance claim is inconsistent with the data; many baselines \(e\.g\., BinaryNet, FBNA CNV\) actually achievehigheraccuracy\. The reviewer fabricates an inconsistent baseline trend to inflate the paper’s comparative advantage beyond what the data shows\.Key quote:“LogicTreeNet\-B 80\.17% 24 ns … BinaryNet \[29\] 88\.60% 4 090 M Zhao et al\. \[30\] 88\.54% 4 940 M FBNA CNV \[31\] 88\.61% 5 540 M”Paper:Self\-Guided Masked Autoencoders for Domain\-Agnostic Self\-Supervised LearningHyperbole
↪\\hookrightarrowInflated Comparative Advantage
↪\\hookrightarrowMischaracterized Comparative SettingOn all evaluated benchmarks, SMA not only competes but surpasses the state\-of\-the\-art, indicating its potential as a leading approach in self\-supervised learning\.The review overstates the scope of the paper’s SOTA claim\. The abstract limits the claim tothreebenchmarks—protein biology, chemistry, and particle physics—stating SMA “achieves state\-of\-the\-art performance on these three benchmarks\.” The review instead asserts superiority over the state\-of\-the\-art onallevaluated benchmarks\. This is not supported: other evaluated settings, such as GLUE in Table 4, compare SMA only against internal baselines \(No Pretrain, Random, and Word\-masking\) rather than the state\-of\-the\-art\. By generalizing a SOTA claim scoped to three benchmarks into a universal one, the reviewer mischaracterizes the comparative setting\.Key quote:“achieves state\-of\-the\-art performance on these three benchmarks”

## Appendix JPrompt Example

### J\.1Prompt – Hallucination Taxonomy Generation

Fig\.[7](https://arxiv.org/html/2609.03580#A10.F7)presents the prompt template used for automatic hallucination taxonomy generation in our experiment\.

### J\.2Prompt – Hallucination Taxonomy Overlap Filtering

Fig\.[8](https://arxiv.org/html/2609.03580#A10.F8)presents the prompt template used for automatic hallucination taxonomy overlap filtering in our experiment\.

### J\.3Prompt – Coarse Category Screening

Fig\.[10](https://arxiv.org/html/2609.03580#A10.F10)presents the prompt template used for coarse category screening of taxonomy\.

### J\.4Prompt – Template Construction

Fig\.[11](https://arxiv.org/html/2609.03580#A10.F11)presents the prompt template used for template construction\.

### J\.5Prompt – Review Filtering

Fig\.[9](https://arxiv.org/html/2609.03580#A10.F9)presents the prompt template used for review filtering in our experiment\.

### J\.6Prompt – Review Aspect Tagging

Fig\.[12](https://arxiv.org/html/2609.03580#A10.F12)presents the prompt template used for automatic review aspect tagging in our experiment\.

### J\.7Prompt – Hallucination Template Feasibility Check

Fig\.[13](https://arxiv.org/html/2609.03580#A10.F13)presents the prompt template used for the automatic hallucination template feasibility check in our experiment\.

### J\.8Prompt – Hallucination Template Injection

Fig\.[14](https://arxiv.org/html/2609.03580#A10.F14)presents the prompt template used for the automatic hallucination template injection in our experiment\.

### J\.9Prompt – Post\-hoc Verification

Fig\.[15](https://arxiv.org/html/2609.03580#A10.F15)presents the prompt template used for the automatic post\-hoc verification in our experiment\.

### J\.10Prompt – RA\-LLM \(KR\)

Fig\.[16](https://arxiv.org/html/2609.03580#A10.F16)presents the prompt template used for the RA\-LLM \(KR\) baseline in task 1 evaluation \(Hallucination Detection\)\.

### J\.11Prompt – RA\-LLM \(CoT\)

Fig\.[17](https://arxiv.org/html/2609.03580#A10.F17)presents the prompt template used for the RA\-LLM \(CoT\) baseline in task 1 evaluation \(Hallucination Detection\)\.

### J\.12Prompt – RA\-LLM \(Contrast\)

Fig\.[18](https://arxiv.org/html/2609.03580#A10.F18)presents the prompt template used for the RA\-LLM \(Contrast\) baseline in task 1 evaluation \(Hallucination Detection\)\.

### J\.13Prompt – General\-Purpose LLM for Task 1

Fig\.[19](https://arxiv.org/html/2609.03580#A10.F19)presents the prompt template used for the General\-Purpose LLM in task 1 evaluation \(Hallucination Detection\)\.

### J\.14Prompt – General\-Purpose LLM for Task 2

Fig\.[20](https://arxiv.org/html/2609.03580#A10.F20)presents the prompt template used for the General\-Purpose LLM in task 2 evaluation \(Hallucination Classification\)\.

### J\.15Prompt – General\-Purpose LLM for Task 3

Fig\.[21](https://arxiv.org/html/2609.03580#A10.F21)presents the prompt template used for the General\-Purpose LLM in task 3 evaluation \(Hallucination Span Localization\)\.

Core Judgement Criteria for Hallucination Taxonomy RefinementYour task is to assess candidate concept nodes against the following core judgement criteria to ensure the taxonomy is analytically rigorous, MECE\-compliant, and operationally reliable for human annotation\.\#\# 1\. Global Architecture Uniqueness & Redundancy Minimization Each hallucination type must have a unique and unambiguous position in the taxonomy tree\. \-Path Exclusivity: If a concept is already defined as an independent item at the first level \(Depth 1\), it must not be duplicated under other categories \(e\.g\., Reasoning Error\)\. \-Annotation Conflict Prevention: Avoid forcing annotators to choose between two logical paths when confronting the same error, as this reduces data consistency\. \-Redundancy Minimization: The taxonomic structure must remain concise; structural over\-stacking is strictly prohibited\. \-Synonym Merging: Nodes with highly overlapping definitions or that are merely near\-synonyms must be merged \(e\.g\., Citation Year Substitution and Publication Year Inaccuracy\)\. \-Structural Flattening: If a child node’s definition is fully subsumed by its parent and provides no additional operational value, it must be simplified\.

\#\# 2\. Parent\-Child Alignment and Logical Inheritance Evaluation Standard: Does the node logically constitute a proper subset of its parent node? \-Logical Subordination: Child nodes must precisely inherit the categorical scope of their parent node\. \-Retention Principle: Prioritize options that best fit the parent concept\. \-Elimination Mechanism: If a node’s definition exceeds the parent node’s scope or is misclassified, it must be eliminated or restructured, regardless of definitional quality\.

\#\# 3\. Conceptual Precision and Actionability Evaluation Standard: Is the node’s definition concrete enough for annotators \(or models\) to identify unambiguously when encountering actual data? \-Retention Principle: Prioritize nodes that provide concrete mechanisms, clear boundaries, or specific examples \(e\.g\., particular lexical cues, specific comparison targets\)\. \-Elimination Mechanism: Eliminate overly general, vaguely worded, or redundant nodes that merely restate the parent definition\.

\#\# 4\. Coverage and Mutual Exclusivity \(MECE Principle\) Evaluation Standard: Are two nodes fully overlapping \(synonymous\), partially overlapping, or mutually exclusive? \-Near\-Synonymous Case: Select the node with more rigorous wording and stronger alignment with academic terminology\. \-Containment Relation: Prioritize the node that fills a missing level in the current taxonomy tree; if two nodes share the same depth but differ in scope, retain the one with the more precise definition\. \-Fully Distinct Mechanisms \(Mutually Exclusive\): Retain both nodes\.

\#\# 5\. Nomenclature and Structural Consistency Evaluation Standard: Does the node name conform to the naming conventions of the overall taxonomy \(e\.g\., uniform adoption of the “Phenomenon \+ via \+ Mechanism” format\)? \-Structural Alignment: Select options whose naming style aligns with other nodes in the system, especially those at the same hierarchical level\.

Figure 6:Core judgement criteria used for hallucination taxonomy refinement\.Prompt Template for Hallucination Taxonomy GenerationYou are an expert in Large Language Model \(LLM\) hallucination taxonomy construction for scientific paper reviews\. Your task is to decompose a given parent hallucination concept into operationally distinguishable child concepts at a specified taxonomy depth\. The goal is to construct a taxonomy that is analytically rigorous, MECE\-compliant, and suitable as an annotation label space\. The taxonomy must support reliable human annotation and reviewer agreement\. A critical requirement is that concept boundaries are clear and enforceable based on observable review text, not abstract error theory\.\#\# TASK OVERVIEW Given a parent hallucination concept, generate child concepts that represent concrete hallucination realization patterns at the current taxonomy depth\.

Each child concept must be defined by:
\- a concise and stable concept name \(used as the annotation label\), and
\- an operational concept description that explains how the hallucination manifests in review text\.The focus is strictly on \*textually observable hallucination patterns in reviews\*, not on model intent, generation causes, or abstract error classes\.\#\# INPUT SLOTS \- Ancestor Path \(from root to parent, ordered from most general to most specific\): Use this to calibrate how much narrowing each level represents, and match that pace for the children you generate\. \{ancestor\_path\}

\- Parent Concept \(Dictionary\):
\{"concept": \{parent\_concept\}, "description": \{parent\_concept\_desc\}\}\- Current Taxonomy Depth: \{current\_depth\}\- Existing Sibling Concepts at This Depth \(List of Dictionaries, for MECE checking\):
Each sibling concept is provided as: \[ \{ "concept": "…", "description": "…" \}, … \]
\{sibling\_concepts\}\- Maximum Number of Child Concepts to Generate: \{max\_width\}\#\# GENERATION CONSTRAINTS 1\. MECE Compliance \- Each generated child concept must be mutually exclusive with all provided sibling concepts at this depth\. \- The set of generated child concepts must collectively exhaust the parent concept\. \- Distinctions must be justified at the level of observable textual evidence in review text\.

2\. Operational Specificity
Each child concept description must explicitly specify:
\- What aspect of the paper is hallucinated \(e\.g\., method, result, experiment, motivation, comparison, conclusion\)\.
\- How it is hallucinated \(e\.g\., fabrication, attribute substitution, unsupported extension, semantic distortion\)\.
\- Typical linguistic cues or claim structures in the review text that indicate this hallucination\.3\. Depth Awareness
\- Use current depth AND ancestor path together to determine abstraction level\.
\- Shallow depths \(depth 2\): generate abstract categorical distinctions\.
\- Deeper depths \(depth 3\+\): permit concrete, instance\-level distinctions only when the parent is already an appropriately narrow category\.
\- Do NOT jump to instance\-level specifics at intermediate depths\. If concrete variants exist, first define the abstract category that contains them; defer the instances to the next level down\.
\- Child concepts should be parallel in granularity to existing siblings\.4\. Minimality
\- Generate only the minimum number of child concepts required to fully cover the parent concept\.
\- Avoid stylistic variants or overlapping distinctions that are not operationally enforceable\.\#\# OUTPUT FORMAT The output MUST be a single JSON array\.

If child concepts are generated, return an array of objects, each representing one immediate child concept:\[ \{ "concept": "<canonical label\>", "description": "<operational definition based on observable review text\>" \} \]Field constraints:
\- concept:
\- Short, stable, canonical label
\- Noun phrase only
\- No punctuation, qualifiers, or examples\- description:
\- Single sentence only \(no lists, no commentary, no causal speculation\)
\- Operational definition grounded in textually observable review content
\- Specifies what aspect of the paper is hallucinated and how it manifests in review textIf the parent concept is atomic at the current taxonomy depth, return:\[\]Figure 7:Prompt template used for hallucination taxonomy generation\.Prompt Template for Near\-Synonym FilteringYou are an expert in semantic taxonomy alignment\.Your task is to determine whether two taxonomy nodes are near\-synonyms, considering:
\- concept meaning
\- concept descriptions
\- hierarchical context \(ancestor path\)
\- level of abstraction \(depth\)Two nodes are considered "near\-synonyms" ONLY IF:
\- They refer to the same or almost identical concept in this taxonomy context
\- They would be redundant if both kept in the same tree
\- Differences in wording do NOT imply different scope or abstraction —\#\#\# Node A
\- Concept: \{concept\_A\}
\- Description: \{description\_A\}
\- Ancestor Path \(root→\\rightarrownode\): \{path\_A\}
\- Depth: \{depth\_A\}\#\#\# Node B
\- Concept: \{concept\_B\}
\- Description: \{description\_B\}
\- Ancestor Path \(root→\\rightarrownode\): \{path\_B\}
\- Depth: \{depth\_B\} —\#\#\# Instructions
Step 1: Compare semantic meaning ignoring wording differences\.Step 2: Compare taxonomy context:
\- Are they under similar parent concepts?
\- Do they represent the same level of abstraction?
\- Would merging them break the taxonomy structure?Step 3: Decide:Output in JSON format:\{"is\_synonym": True/False, "keep": "A" \| "B" \| "NA", "explanation": "string or NA" \}Rules:
\- If False:
\- keep = "NA"
\- explanation = "NA"\- If True:
\- Choose the node that is:
\* more precise
\* better aligned with its ancestor path
\* more consistent with taxonomy naming style
\- explanation must include:
\* why they are synonyms
\* why the chosen node is better in contextBe strict: do NOT mark as synonyms if there is any meaningful difference in scope, abstraction level, or taxonomy role\.Figure 8:Prompt template used for near\-synonym hallucination taxonomy filtering\.Prompt Template for Review FilteringYou are an expert in academic peer\-review analysis\. Your task is to determine whether the meta review substantively adopted the core opinion of a specific reviewer\. You will be given information about ONE reviewer only\. Do NOT consider other reviewers except as reflected in the meta review\.—\#\#\# Input\[Reviewer Review\] \{reviewer\_review\}\[Meta Review\] \{meta\_review\}—\#\#\# Instructions1\. Identify the reviewer’s core opinion, i\.e\., the main reasons for acceptance or rejection\.
2\. Examine whether the meta review:
\- explicitly endorses or echoes these points, OR
\- implicitly relies on them in its reasoning, OR
\- explicitly rejects or ignores them\.
3\. If the meta review adopts the same reasoning even without mentioning the reviewer explicitly, this still counts as adoption\.—\#\#\# Output Format \(strictly follow\)Return a valid JSON object ONLY, using EXACTLY the following template\. You must fill in the values but MUST NOT change any key names\.\{"adopted": <true \| false\>, "confidence": "<high \| medium \| low\>", "justification": "<one concise sentence explaining how the meta review does or does not adopt the reviewer’s core opinion\.\>" \}—\#\#\# Decision Criteria\- adopted = true: The meta review clearly or implicitly incorporates the reviewer’s core arguments as part of its justification\.\- adopted = false: The meta review explicitly contradicts the reviewer’s core arguments, ignores their main concerns, or reaches a decision based on different reasoning\.\- If the evidence is indirect or ambiguous, lower the confidence accordingly\.Figure 9:Prompt template used for review filtering\.Prompt Template for Coarse\-grained Feasibility Screening\#\# ROLE
You are a hallucination\-injection feasibility screener\. Given a sentence from a scientific paper review, determine which hallucination categories have therequired raw materialpresent in the sentence for injection to be possible\.\#\# TASK
For each category below, check whether the sentence contains the specific linguistic or semantic features listed as prerequisites\. Select a categoryonly if its prerequisites are clearly satisfied\. When in doubt, exclude\.\#\# CATEGORIES AND PREREQUISITESNumber
Prerequisite: The sentence contains at least one explicit numeric value — including percentages, ratios, measurements, counts, years, dimensions, or quantitative comparisons \(e\.g\., “3 layers”, “92\.4% accuracy”, “2018”, “4x faster”\)\.
Exclude if: the sentence only mentions quantity implicitly or qualitatively \(e\.g\., “several”, “many”, “large”\)\.⋯\\cdots\(Entity, False Concatenation, Attribution Failure, Overgeneralization, Reasoning Error, Hyperbole, Temporal omitted\)⋯\\cdotsContext\-based meaning error
Prerequisite: The sentence contains a term, abbreviation, or phrase that hasmultiple plausible interpretationsin a scientific context — including overloaded acronyms, domain\-specific terms with general\-language counterparts, or phrasing whose meaning depends on surrounding context\.
Exclude if: all terms in the sentence have a single unambiguous meaning in context\.\#\# INPUT
Injection\_Sentence: “\{injection\_sentence\}”\#\# OUTPUT FORMAT \(STRICT\)
Output ONLY a JSON array of the category names whose prerequisites are satisfied\. No explanations\. No extra text\.Valid names: “Number”, “Entity”, “False Concatenation”, “Attribution Failure”, “Overgeneralization”, “Reasoning Error”, “Hyperbole”, “Temporal”, “Context\-based meaning error”Example:
\[“Number”, “Hyperbole”, “Reasoning Error”\]Figure 10:Prompt template used for coarse\-grained feasibility screening before fine\-grained template construction\. Seven of the nine categories are omitted for brevity; all nine follow the same prerequisite/exclude format\.Prompt Template for Injection Template ConstructionYou are a compatibility checker for hallucination injection templates in scientific paper reviews\.\#\# TASK
Given a template with four fields, decide whether the hallucination defined byHallucination\_InstructionandHallucination\_Labelscan be coherently injected intoInjection\_Sentencewhile remaining grounded inInjection\_Aspect\. Outputtrueonly if ALL conditions below hold\. Be strict and conservative\.\#\# COMPATIBILITY CONDITIONS1\.Operational applicability
The operation described inHallucination\_Instructioncan be concretely performed onInjection\_Sentence— there must exist a target in the sentence to modify, replace, exaggerate, or fabricate\.2\.Semantic prerequisites of the leaf type
Hallucination\_Labelsis a taxonomy path \(root \> … \> leaf\)\. The LEAF node defines the specific hallucination type\. Its semantic prerequisites must be present inInjection\_Sentence\.
Examples:
\- Number leaf→\\rightarrownumeric value present
\- Entity leaf→\\rightarrownamed entity that can be swapped
\- Temporal leaf→\\rightarrowtense / modality / time marker
\- Reasoning leaf→\\rightarrowinferential or causal step
If the prerequisite is absent, outputfalse\.3\.Aspect grounding
Every tag inInjection\_Aspectmust be substantively supported byInjection\_Sentence\. An aspect that the sentence does not actually discuss makes the template incompatible\.4\.Realistic edit
Applying the hallucination must yield a plausible review sentence, not a nonsensical or category\-error edit\. If the only way to inject the hallucination is to invent unrelated content, outputfalse\.\#\# INPUT
Hallucination\_Instruction:
\{hallucination\_instruction\}Injection\_Sentence:
\{injection\_sentence\}Hallucination\_Labels:
\{hallucination\_labels\}Injection\_Aspect:
\{injection\_aspect\}\#\# OUTPUT \(STRICT\)
Output exactly one word:trueorfalse\. No explanation, no punctuation, no extra text\.Figure 11:Prompt template used for injection template construction\.Prompt Template for Review Aspect TaggingYou are an expert at aspect\-based text classification\.TASK: Given a review sentence and a list of candidate aspect labels, identify the SINGLE aspect label that best matches the main focus of the review\.

REVIEW SENTENCE: "\{review\_sent\}"

ASPECT LABEL SET \(fixed and closed\): \{aspect\_label\_set\}INSTRUCTIONS: \- Choose exactly ONE aspect from the candidate list\. \- Select the aspect that is most semantically relevant to the review sentence\. \- If multiple aspects seem related, choose the one that is the most central or dominant\. \- Do NOT create new aspects\. \- Do NOT explain your reasoning\.

OUTPUT FORMAT \(STRICT\): Return a valid JSON dictionary exactly in the following structure: \{"Aspects": \["Aspect1", "Aspect2", …\]\} If no aspect applies: \{"Aspects": \["\-"\]\}

FORMAT CONSTRAINTS: \- Output must be valid JSON\. \- The key must be exactly "Aspects"\. \- Aspect names must exactly match the predefined labels \(case\-sensitive\)\. \- Do not include trailing commas, comments, or extra fields\. \- Do not include any text outside the JSON output\.

FEW\-SHOT EXAMPLES:

REVIEW SENTENCE: The results presented in the paper are significant\. OUTPUT: "Aspects": \["Significance", "Result"\]

REVIEW SENTENCE: However, authors do not compare their methods against any of the previous works\. OUTPUT: \{"Aspects": \["Comparison", "Related Work"\]\}

REVIEW SENTENCE: The latter is efficient, but the former enables interpretability\. OUTPUT: \{"Aspects": \["Efficiency", "Interpretation"\]\}

REVIEW SENTENCE: In general, I find Experiments II to be much weaker than Experiment I\. OUTPUT: \{"Aspects": \["Comparison", "Experiment"\]\}

REVIEW SENTENCE: \(2\) The paper does not explain why end\-to\-end training in the entropy\-regularization is necessary\. OUTPUT: \{"Aspects": \["Training", "Explanation"\]\}

REVIEW SENTENCE: Finally, the paper shows that compositional languages generalize better to the held\-out validation set\. OUTPUT: "Aspects": \["Generalization", "Findings"\]

REVIEW SENTENCE: Typo: \- Figure 4 caption: "Our variable\-resolution inputs prevent\*\*s\*\*…" OUTPUT: \{"Aspects": \["Typo"\]\}

REVIEW SENTENCE: In the more practical setting of finetuning the model, the attacks are not effective\. OUTPUT: \{"Aspects": \["Fine\-tuning", "Effectiveness"\]\}

REVIEW SENTENCE: The evaluation metric is problematic \(or at least unclear\)\. OUTPUT: \{"Aspects": \["Evaluation", "Clarity", "Metric"\]\}

REVIEW SENTENCE: Overall, this work contributes an interesting framework for analysis\. OUTPUT: \{"Aspects": \["Analysis", "Framework"\]\}

REVIEW SENTENCE: I’m confused as to why some of the visualizations in Fig 3 show white bands along the diagonal\. OUTPUT: \{"Aspects": \["Figure", "Confusion"\]\}

The system must strictly follow the specified output format under all circumstances\.

Figure 12:Prompt template used for review aspect tagging\.Prompt Template for Hallucination Template Feasibility CheckYou are an automatic compatibility checker for hallucination injection templates\.—\#\# TASK\_OVERVIEW
Your task is to determine whether a given injection template is COMPATIBLE\.
Be strict and conservative\. If the hallucination type cannot be meaningfully applied to the sentence, mark it as incompatible\.A template is COMPATIBLE only if:
\- The hallucination instruction can be operationally applied to the sentence\.
\- The semantic requirements implied by the hallucination labels are present in the sentence\.
\- The injection aspects are supported by the sentence content\.
\- Applying the hallucination would result in a coherent and realistic edit, not a nonsensical or impossible one\.—\#\# DECISION\_GUIDELINES
\- If the hallucination type requires specific properties \(e\.g\., numerical values, concrete entities, temporal information, comparisons, or other semantic constraints\), those properties must be present in the sentence\.
\- Number\-related hallucinations are incompatible with non\-quantitative sentences\.
\- Entity\-related hallucinations are incompatible if no concrete entity is mentioned\.
\- Hallucination\_Labels are provided as a single taxonomy path \(coarse to fine\); the leaf node defines the hallucination type and its semantic requirements\.
\- Aspect tags must be grounded in the sentence; unsupported aspects make the template incompatible\.
\- Hallucination\_Labels and Injection\_Aspect define HARD constraints; if either is not satisfied by the sentence, the template is incompatible\.
\- If applying the hallucination would introduce unsupported information or category errors, the template is incompatible\.—\#\# INPUT\_FORMAT
Hallucination\_Instruction: \{hallucination\_instruction\}Injection\_Sentence: \{injection\_sentence\}Hallucination\_Labels: \{hallucination\_labels\}Injection\_Aspect: \{injection\_aspect\}—\#\#\# OUTPUT\_FORMAT \(STRICT\)
You must output exactly ONE token, either:true or falseNo other output is allowed\. Schema correctness is more important than language quality\.

Figure 13:Prompt template used for Hallucination Template Feasibility Check before template injection\.Prompt Template for Hallucination Template InjectionYou are an expert hallucination generator for review sentences\.—\#\# TASK
Your task is to introduce the specific hallucination described in the instruction into the given sentence\. Follow the instruction precisely and produce a natural, coherent sentence that reflects the hallucination\. Do not alter other parts of the sentence unnecessarily\.—\#\# INPUT\_FORMAT
Hallucination\_Instruction: \{hallucination\_instruction\}Injection\_Sentence: \{injection\_sentence\}—\#\# OUTPUT\_FORMAT
Return ONLY the modified sentence as a single line of text\.
Do not add explanations, comments, or JSON\.Figure 14:Prompt template used for Hallucination Template Injection\.Prompt Template for Post\-hoc Verification\#\# TASK:You are an expert reviewer specializing in fine\-grained semantic comparison of review statements\. Your task is to determine whether two review sentences are strictly semantically equivalent\. Equivalence \(score = 1\) should be rare and assigned only in near\-identical paraphrase cases\. In ambiguous cases, choose 0\.\#\# INPUT:

Review Sentence A: \{REVIEW\_SENTENCE\_1\}Review Sentence B: \{REVIEW\_SENTENCE\_2\}\#\# CRITERIA:

The two sentences are semantically equivalent ONLY IF ALL of the following are true:
\- Each sentence fully entails the other\.
\- No opinions, facts, or sentiments are added or omitted\.
\- No difference in emphasis, scope, certainty, or evaluative strength\.
\- No additional qualifiers or implications are introduced\.
\- The sentences are fully interchangeable in the same review context\.If ANY difference exists, the equivalence score MUST be 0\.\#\# OUTPUT:

Return the equivalence score as a single digit:
\- 1: Strict semantic equivalence
\- 0: Not strictly semantically equivalentReturn only: 0 or 1Figure 15:Prompt template used for Post\-hoc Verification after hallucination template injection\.Prompt Template for RA\-LLM \(KR\)You are a careful hallucination detector\. Answer MUST be Yes or No\.
\#\#\# Instruction
I want you act as an answer judge\. Given a question and an answer, your objective is to determine if the provided answer contains non\-factual or hallucinated information\. You SHOULD give your judgement based on the following hallucination types and the world knowledge\.
\(Few\-shot examples including: misunderstanding context, factual contradiction, specificity level, and logical inference…\)
The answer you give MUST be "Yes" or "No"\.
—
\#\#\# Output Format Requirement
1\) You may provide brief reasoning\.
2\) The last line must be exactly: \#Your Judgement\#: Yes or \#Your Judgement\#: No
3\) Do not output both labels in the final line\.
—
\#\#\# Input
\#Question\#: Use the following evidence as knowledge when judging\.
\{knowledge\}
Review statement: \{statement\}
\#Answer\#: \{statement\}
\#Your Judgement\#:Figure 16:Prompt template used for RA\-LLM \(KR\) in Task 1 Evaluation\.Prompt Template for RA\-LLM \(CoT\)You are a scientific peer\-review hallucination detector\.
\#\# Task
Determine whether the review sentence is hallucinated based strictly on the paper evidence\.
\#\# Definitions & Constraints
\- A review claim is hallucinated if it is factually unsupported by, or incorrect relative to, the paper content\.
\- EXCLUSIONS: Do not evaluate tone \(e\.g\., harshness\)\. Do NOT treat subjective judgments, critiques, or opinions \(e\.g\., whether novelty is high\) as hallucinations\. Focus strictly on factual grounding\.
Reasoning steps:
1\) Briefly verify key facts from \#Paper Evidence\# against the \#Review Sentence\#\.
2\) Check consistency between the paper’s intent/context and the \#Review Sentence\#\.
3\) Conclude with the final JSON judgement\.
—
\#\# Inputs
\#\#\# PAPER EVIDENCE
"""\{evidence\}"""
\#\#\# REVIEW SENTENCE
"""\{sentence\}"""
—
\#\# Output Format
Return a valid JSON object ONLY, using EXACTLY the following template:
\{ "reasoning": "<Step\-by\-step analysis following the three reasoning steps above\.\>", "hallucination": <true or false\> \}
\#\# Instructions
\- "true" = the sentence contains a factual claim that is hallucinated \(unsupported or incorrect relative to the evidence\)\.
\- "false" = the sentence is factually supported by and consistent with the evidence, OR the sentence is purely a subjective judgment/critique\.
\- Output ONLY the JSON object\. Do NOT include markdown tags like‘‘‘json\.Figure 17:Prompt template used for RA\-LLM \(CoT\) in Task 1 Evaluation\.Prompt Template for RA\-LLM \(Contrast\)You are a scientific peer\-review hallucination detector\.
\#\# Task
Determine whether the review sentence is hallucinated based strictly on the paper evidence\.
\#\# Definitions & Constraints
\- A review claim is hallucinated if it is factually unsupported by, or incorrect relative to, the paper content\.
\- EXCLUSIONS: Do not evaluate tone \(e\.g\., harshness\)\. Do NOT treat subjective judgments, critiques, or opinions \(e\.g\., whether novelty is high\) as hallucinations\. Focus strictly on factual grounding\.
Contrastive reasoning:
1\) List the strongest evidence from \#Paper Evidence\# that supports the \#Review Sentence\#\.
2\) List the strongest evidence from \#Paper Evidence\# that contradicts the \#Review Sentence\# or shows missing facts\.
3\) Decide which side is stronger and output the final JSON judgement\.
—
\#\# Inputs
\#\#\# PAPER EVIDENCE
"""\{evidence\}"""
\#\#\# REVIEW SENTENCE
"""\{sentence\}"""
—
\#\# Output Format
Return a valid JSON object ONLY, using EXACTLY the following template:
\{ "supporting\_evidence": "<Strongest evidence from the paper that supports the review sentence\.\>", "contradicting\_evidence": "<Strongest evidence from the paper that contradicts the review sentence or shows missing facts\.\>", "hallucination": <true or false\> \}
\#\# Instructions
\- "true" = the sentence contains a factual claim that is hallucinated \(unsupported or incorrect relative to the evidence\)\.
\- "false" = the sentence is factually supported by and consistent with the evidence, OR the sentence is purely a subjective judgment/critique\.
\- Output ONLY the JSON object\. Do NOT include markdown tags like‘‘‘json\.Figure 18:Prompt template used for RA\-LLM \(Contrast\) in Task 1 Evaluation\.Prompt Template for Hallucination DetectionYou are a scientific peer\-review hallucination detector\.\#\# Task Determine whether the review sentence is hallucinated based strictly on the paper evidence\.

\#\# Definitions & Constraints \- A review claim is hallucinated if it is factually unsupported by, or incorrect relative to, the paper content\. \- EXCLUSIONS: Do not evaluate tone \(e\.g\., harshness\)\. Do NOT treat subjective judgments, critiques, or opinions \(e\.g\., whether novelty is high\) as hallucinations\. Focus strictly on factual grounding\.

\#\# Inputs \#\#\# PAPER EVIDENCE """\{evidence\}"""

\#\#\# REVIEW SENTENCE
"""\{sentence\}"""\#\# Output Format
Return a valid JSON object ONLY, using EXACTLY the following template:\{"hallucination": <true or false\> \}\#\# Instructions \- "true" = the sentence contains a factual claim that is hallucinated \(unsupported or incorrect relative to the evidence\)\. \- "false" = the sentence is factually supported by and consistent with the evidence, OR the sentence is purely a subjective judgment/critique\. \- Output ONLY the JSON object\. Do NOT include markdown tags like‘‘‘json\.

Figure 19:Prompt template used for hallucination detection with general\-purpose LLM in Task 1 evaluation\.Prompt Template for Hallucination Type ClassificationYou are a scientific peer\-review hallucination detector\.\#\# Task Given a review sentence that has ALREADY been judged as hallucinated, classify the hallucination TYPE using the provided taxonomy labels\.

\#\# Definitions & Constraints \- The categories are mutually exclusive \(MECE\)\. \- You must rely strictly on the provided operational definitions in the HALLUCINATION LABELS, not just the label names\.

\#\# Inputs \#\#\# HALLUCINATION LABELS """\{label\_space\}"""

\#\#\# PAPER EVIDENCE
"""\{evidence\}"""\#\#\# REVIEW SENTENCE
"""\{sentence\}"""\#\# Output Format
Return a valid JSON object ONLY, using EXACTLY the following template:\{"label": "<concept\>" \}\#\# Instructions \- The sentence is confirmed to be hallucinated\. You MUST choose exactly ONE label corresponding to the "concept" field in the HALLUCINATION LABELS\. \- Do NOT return "None" or invent new labels\. \- If multiple labels seem plausible, choose the most specific one that aligns with the observable textual errors in the review sentence\. \- Output ONLY the JSON object\. Do NOT include markdown tags like‘‘‘json\.

Figure 20:Prompt template used for hallucination detection with general\-purpose LLM in Task 2 evaluation\.Prompt Template for Hallucination Span LocalizationYou are a scientific peer\-review hallucination detector\.\#\# Task Given a list of review sentences, return exactly one sentence that exhibits the specific hallucination described in the target label\.

\#\# Definitions & Constraints \- A claim is hallucinated if it is factually unsupported by, or incorrect relative to, the paper content\. \- EXCLUSIONS: Do NOT treat subjective judgments, critiques, or opinions as hallucinations\. \- You must strictly match the error in the sentence to the definition of the target label\.

\#\# Inputs \#\#\# TARGET HALLUCINATION LABEL """\{label\_with\_definition\}"""

\#\#\# PAPER EVIDENCE
"""\{evidence\}"""\#\#\# REVIEW SENTENCES
"""\{sentences\}"""\#\# Output Format
Return a valid JSON object ONLY, using EXACTLY the following template:\{"result": \[\] \}\#\# Instructions \- Base your judgment ONLY on the PAPER EVIDENCE and the definition of the TARGET HALLUCINATION LABEL\. \- If NO sentence matches the target label, output an empty array for the result field: \[\] \- If a sentence matches, output an array containing EXACTLY ONE string: \["<The exact hallucinated sentence\>"\] \- If multiple sentences are hallucinated for this label, pick the single most definitive one\. If uncertain, pick your best guess\. \- Output ONLY the JSON object\. Do NOT include markdown tags like‘‘‘json\.

Figure 21:Prompt template used for hallucination span localization with generous\-purpose LLM in Task 3 evaluation\.