MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification

# Building and Benchmarking Snippet-Level Granularityfor Medical Fact Verification
Source: [https://arxiv.org/html/2609.12884](https://arxiv.org/html/2609.12884)
## MedSNIP: Building and Benchmarking Snippet\-Level Granularity for Medical Fact Verification

Sarfraz AhmadAffiliation:Mohamed bin Zayed University of Artificial IntelligenceHyunjae KimAffiliation:Yale UniversitySihyeon ParkAffiliation:Korea UniversityJunjie LiaoAffiliation:Beijing Normal UniversityAffiliation:INSAIT, Sofia University, “St\. Kliment Ohridski”hasan\.iqbal@mbzuai\.ac\.ae,yuxia\.wang@insait\.ai[Project](https://mbzuai-nlp.github.io/MedSNIP/)[MedSNIP](https://huggingface.co/datasets/MBZUAI/MedSNIP)[Code](https://github.com/mbzuai-nlp/MedSNIP)Qingyu ChenAffiliation:Yale UniversityPreslav NakovAffiliation:Mohamed bin Zayed University of Artificial IntelligenceYuxia WangAffiliation:INSAIT, Sofia University, “St\. Kliment Ohridski”hasan\.iqbal@mbzuai\.ac\.ae,yuxia\.wang@insait\.ai[Project](https://mbzuai-nlp.github.io/MedSNIP/)[MedSNIP](https://huggingface.co/datasets/MBZUAI/MedSNIP)[Code](https://github.com/mbzuai-nlp/MedSNIP)

###### Abstract

A medical claim’s correctness often depends not on the claim alone, but on the clinical structure around it\. A claim may require a lab reference range, a causal or conditional link, or patient\-specific details to be judged correctly, and atom\-level decomposition can fragment these dependencies, leaving the verifier with clinically incomplete claims\. We reformulate medical fact\-checking around*snippet\-level*verification, where clause\-grouped units preserve local clinical structure\. We introduceMedSNIP\-Bench, a human\-annotated benchmark for snippet\-level medical fact verification, andMedSNIP, an automatic snippet\-generation pipeline\.MedSNIP\-Benchcovers 276 consumer\-health and clinical\-vignette responses, segmented into 2,524 snippets with dual in\-general and in\-patient\-context labels and six structural pattern codes\.MedSNIPis evaluated against human snippet boundaries onMedSNIP\-Benchand then used to generate snippet\-level units for external corpora\. AcrossMedSNIP\-Bench,HealthFC, andMedHallu, snippet\-level verification preserves or improves false\-class F1, with gains concentrated where answers are long enough to fragment and where the verifier is strong enough to exploit the recovered structure\. The largest merge\-pattern gain is on causal\-conditional clinical chains\. It also reduces verifier calls by 24–73%, though the saving survives end\-to\-end only when decomposition is cheap, which an open\-weight decomposer makes possible at no loss of chunking fidelity\.

## 1Introduction

Large language models \(LLMs\) are increasingly used to answer medical questions from patients, consumers, and clinicians\([Singhal et al\., 2023](https://arxiv.org/html/2609.12884#bib.bib22);[Singhal et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib23);[Manes et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib18);[Liu et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib16);[Wang and Zhang, 2024](https://arxiv.org/html/2609.12884#bib.bib25)\)\. A fluent but unsupported recommendation can influence patients’ decisions about care, mislead downstream systems, and propagate through clinical reasoning\([Zhu et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib32)\)\.

Automated fact\-checking is a safety requirement for medical LLMs, not an evaluation convenience\. Currently, most fact\-checking pipelines descend fromFActScore\([Min et al\., 2023](https://arxiv.org/html/2609.12884#bib.bib19)\)and its successors\. They split a generation into minimal standaloneatomicclaims, verify each claim independently, and aggregate the verdicts\([Chern et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib3);[Wei et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib29);[Dhuliawala et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib4);[Iqbal et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib12)\)\. This atom\-level design is natural when each claim can be judged on its own\. However, medical answers often violate this assumption\. A lab interpretation may require both a measured value and a reference range\. A diagnosis may require a pattern of findings rather than any finding in isolation\. A recommendation may be true in general but wrong for the patient described in the question\. In such cases, atomization can turn a clinically meaningful judgment into fragments that lack the information needed for verification\.

This motivates a coarser verification unit that preserves these dependencies\. We call this unit a*snippet*, a clinically meaningful group of related clauses that is smaller than a full answer but coarser than an atom\. It groups the information needed to verify a clinical judgment, such as a finding and its interpretation, a lab value and its reference range, or a diagnosis and its caveat\. Figure[1](https://arxiv.org/html/2609.12884#S1.F1)illustrates this on a clinical\-vignette answer, where one judgment fragments into three or four atoms\. This motivates treating granularity as a core design choice in medical fact\-checking, not as pre\-processing\.

Figure 1:Motivating example fromMedSNIP\-Benchshowing how atomization can split clinically coherent judgments\. Each column presents one judgment from a clinical\-vignette answer\. Snippet\-level decomposition preserves the linked evidence, interpretation, and caveat as one verifiable unit, while atom\-level decomposition fragments the same judgment into three or four separate claims\.Motivated by these observations, we reformulate medical fact\-checking around*snippet\-level verification*\. Rather than decomposing responses into atomic claims or evaluating entire answers holistically, we group semantically and clinically related clauses into coherent snippets that preserve the context needed to verify clinical judgments\. Each snippet is annotated for both general medical correctness and patient\-specific correctness, capturing errors that may be medically plausible in general but unsafe for the particular patient context\.

Existing medical fact\-checking resources do not evaluate this formulation\.HealthFCprovides expert\-labeled consumer\-health claims andMedHalluprovides aPubMedQA\-derived hallucination benchmark, but both evaluate near\-atomic claims rather than clinically structured units\([Vladika et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib24);[Pandit et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib21)\)\. Domain\-tailored decomposers and verifier diagnostics similarly remain within the atom\-level paradigm\([Huang et al\., 2026](https://arxiv.org/html/2609.12884#bib.bib10);[He et al\., 2026](https://arxiv.org/html/2609.12884#bib.bib9)\)\. To fill this gap, we introduceMedSNIP\-Bench, a human\-annotated benchmark for snippet\-level medical fact\-checking, together withMedSNIP, an automatic snippet\-generation pipeline\. Together, this paper contributes:

- •A reformulation of medical fact\-checking around snippet\-level verification, where related clauses are grouped and checked as clinically coherent units that preserve relevant context\.
- •MedSNIP\-Bench, with 276 consumer\-health and clinical\-vignette responses, 2,524 human\-annotated snippet boundaries, dualin\-generalandin\-patient\-contextlabels, and six structural pattern codes\.
- •MedSNIP, an automatic snippet\-generation pipeline evaluated against human snippets onMedSNIP\-Benchand used to generate snippet\-level units for external corpora such asHealthFCandMedHallu\.
- •A cross\-model, cross\-dataset evaluation with paired bootstrap confidence intervals throughout, plus per\-pattern analysis showing where snippet\-level verification helps\.

## 2Related Work

##### Atomic fact\-checking:

FActScoreestablished a common paradigm for long\-form factuality evaluation, where a generation is decomposed into atomic claims and each claim is checked against evidence\([Min et al\., 2023](https://arxiv.org/html/2609.12884#bib.bib19)\)\. Subsequent systems adopt claim\-level verification pipelines\([Chern et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib3);[Wei et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib29);[Gao et al\., 2023](https://arxiv.org/html/2609.12884#bib.bib5);[Iqbal et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib12);[Zerong et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib31);[Wang et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib26);[Wang et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib27)\)\. Recent work questions whether the smallest claim is always the best verification unit\.[Wanner et al\. \(2024\)](https://arxiv.org/html/2609.12884#bib.bib28)show that verifier accuracy depends on decomposition granularity, while[Lu et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib17)learn a verifier\-preferred atomicity policy that improves over default atomization\.

Motivated by these findings, we take a simpler approach and show that a coarser, clause\-grouped unit is sufficient, cheaper, and better aligned with medical reasoning\.

##### Medical fact\-checking:

Medical factuality evaluation spans consumer\-health and clinical question\-answering settings\.HealthFCprovides expert\-labeled consumer\-health claims with abstracts\([Vladika et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib24)\), whileMedHalluextendsPubMedQAinto a hallucination\-detection benchmark with paired ground\-truth and adversarially perturbed answers\([Pandit et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib21)\)\.MedScoreadaptsFActScore\-style decomposition to free\-form medical answers and shows that a domain\-tailored atomizer produces more valid claims than an atomizer\([Huang et al\., 2026](https://arxiv.org/html/2609.12884#bib.bib10)\)\.MedFactidentifies an over\-criticism failure mode in which stronger reasoning verifiers incorrectly mark correct medical text as false\([He et al\., 2026](https://arxiv.org/html/2609.12884#bib.bib9)\)\.[Kim et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib14)show that retrieval covers only a minority of clinician\-identified must\-have information and that retrieval\-augmented generation \(RAG\) can degrade factuality\.[Gunjal and Durrett \(2024\)](https://arxiv.org/html/2609.12884#bib.bib8)make atomic claims interpretable through decontextualization, which is complementary to grouping rather than an alternative to it\.

Rewriting “*A because B*” as separate claims can miss errors when each part is plausible but their link is not\. Existing methods improve claim\-level medical fact\-checking, but lack snippet structure and dual in\-general and in\-patient\-context labels\. We introduceMedSNIP\-Bench, which, to the best of our knowledge, is the first human\-annotated snippet\-level medical fact\-checking dataset to preserve clinical claims and their context\. Our results support snippet\-level verification rather than relying only on atomized claims\.

## 3MedSNIP\-Bench

MedSNIP\-Benchis built on the benchmark released by[Kim et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib14), which pairs 100 consumer\-health queries fromK\-QA\([Manes et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib18)\)with 100USMLE\-style clinical\-vignette questions fromMedBullets\([Chen et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib2)\)and carries expert true\-or\-false labels on atomic claims\. We use 276 entries containing 5,755 expert\-labeled atomic claims, partitioned into 140 consumer\-health and 136 clinical\-vignette entries using a word\-length threshold on the original query\. The parent corpus is bimodal in query length, so the partition is unambiguous \(Appendix[A\.1](https://arxiv.org/html/2609.12884#A1.SS1)\)\.

The atom\-level binary factuality labels \(i\.e\., true or false\) are obtained from physician annotations inherited from[Kim et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib14), whileMedSNIP\-Benchthen groups these atoms into clinically meaningful snippets and provides the corresponding snippet\-level annotations\. Every snippet boundary, pattern code and correctness label inMedSNIP\-Benchwas assigned manually\.

##### Snippets and pattern taxonomy:

The atomic\-claim layer is often too fine\-grained for medical content, because clinical judgments can depend on related clauses\. To preserve these dependencies, we re\-segment the corpus into*snippets*, clause\-grouped verifiable units designed to retain shared entities, clinical framing, and causal or conditional relationships\. Analysis across the parent benchmark showed that snippet boundaries recur around six structural patterns\. We identified these patterns through error analysis of an atomic\-claim verification baseline, where recurring verification failures clustered into the same structural cases\. Patterns A–C, illustrated in Figure[1](https://arxiv.org/html/2609.12884#S1.F1), are merge patterns, where multiple atoms should be verified together as a single snippet, while Patterns D–F are keep\-atomic patterns, where the atom remains the appropriate verification unit\. The six structural patterns are summarized below:

- •Pattern Afor enumerations of properties of a single subject \(features, causes, or factors that only make sense as a set\)
- •Pattern Bfor causal or conditional chains whose links share evidence \(clinical reasoning, including diagnostic logic, drug\-effect chains, and threshold\-dependent recommendations\)
- •Pattern Cfor conclusions together with the premises that warrant them \(a diagnosis or recommendation whose verifiability depends on the supporting findings just stated\)\.
- •Pattern Dfor self\-contained standalone facts or lab interpretations whose verifiability does not depend on surrounding clauses
- •Pattern Efor genuine topic shifts where merging would conflate distinct subjects
- •Pattern Ffor distinct facts about the same subject that do not jointly support a single judgment, including isolated false atoms surrounded by true ones \(a case where atomization protects the false claim from being absorbed into a true context\)\.

DatasetSubset / SplitEntriesUnits%F in\-general%F in\-contextLabelsPatternsCountTypeABCDEFMedSNIP\-BenchStatisticsMedSNIP\-BenchParent corpus2765,755atom–––––––––Consumer1401,091snippet12\.49\.2dual634101791941073Vignette1361,433snippet14\.611\.4dual37412424242329241Train1801,599snippet13\.610\.5dual64014320938526196Dev51494snippet13\.810\.7dual18744581261069Test45431snippet13\.710\.0dual1813854106349Total2762,524snippet13\.610\.5dual100822532161739314External DatasetsHealthFCAtoms7501,076atom–––––––––Snippets \(Mode 1\)750821snippet––single––––––Snippets \(Mode 2\)750761snippet––single––––––MedHalluAtoms2,0003,798atom–––––––––Snippets \(Mode 1\)2,0002,636snippet––single––––––Snippets \(Mode 2\)2,0002,586snippet––single––––––

Table 1:MedSNIP\-Benchand external dataset statistics\.The parent corpus is shown for reference\.%F in\-generaland%F in\-contextare false\-label rates, and A–F are structural pattern counts\. For external datasets, bothMedSNIPmodes and their shared atom baseline are reported \(Section[4\.1](https://arxiv.org/html/2609.12884#S4.SS1)\)\.HealthFCcounts follow question\-to\-proposition conversion\. Dashes indicate unavailable or inapplicable fields\.
##### Dual correctness labels:

Each snippet receives two correctness judgments:*\(i\) in\-general*indicates whether the snippet is correct as a general medical statement\.*\(ii\) with\-patient\-context*indicates whether it is correct for the specific patient or query\. This dual scheme captures cases that are generally true but contextually wrong, such as recommendations that are reasonable in general medicine but inappropriate for the vignette at hand\.

This distinction appears in practice inMedSNIP\-Bench, where 124 snippets, or 4\.9%, have divergent in\-general and with\-context labels\. These cases are concentrated in clinical\-vignette entries, consistent with contextual error being especially relevant in patient\-specific settings\. Full details and examples are in Appendix[B](https://arxiv.org/html/2609.12884#A2)\.

### 3\.1Annotation

Six annotators with biomedical\-NLP backgrounds segmented and labeled the data using a shared interface and written guidelines \(see Appendix[B](https://arxiv.org/html/2609.12884#A2)\)\. Snippet boundaries and pattern codes were based on linguistic and logical judgments over the answer text, rather than clinical judgments requiring medical expertise\. Annotators also saw the parent corpus’s physician true\-or\-false labels for each atomic claim, anchoring the task in expert annotations\. For each entry, an annotator first identified the shared context \(Appendix[A\.3](https://arxiv.org/html/2609.12884#A1.SS3)\), such as patient demographics or query topic\. They then applied the A–F pattern taxonomy to decide which claims should be merged into a snippet and which should remain separate\.

After grouping, the annotator revised each snippet into a self\-contained statement and assigned two binary correctness labels\. On a stratified inter\-annotator agreement \(IAA\) set covering both source subsets, label outcomes, and merge patterns, agreement is substantial for thein\-generallabel and moderate for the harderwith\-patient\-contextlabel, with Fleiss’κ\\kappaof0\.6620\.662and0\.4720\.472, respectively, following[Landis and Koch \(1977\)](https://arxiv.org/html/2609.12884#bib.bib15)\. Snippet boundary agreement is strong, with a mean pairwise adjusted Rand index \(ARI\) of0\.7230\.723\. \(see Appendix[A\.4](https://arxiv.org/html/2609.12884#A1.SS4)\)\.

Figure 2:MedSNIPsnippet\-generation pipeline on a worked Advil example\. After deterministic sentence splitting, Mode 1 extracts atoms and shared context, then clusters atoms into snippets\. Mode 2 generates snippets directly from sentences\. Both modes return snippets with source pointers, A–F pattern codes, and shared context\.
### 3\.2Dataset Statistics

After annotation, the parent corpus’s 5,755 atomic claims map to 2,524 snippets\. Of these, 1,554 are multi\-atom and 970 are single\-atom, with a median of 2 atoms per snippet\. Because the dataset skews toward true snippets, we useF1F\\text\{F\}\_\{1\}^\{F\}, defined as F1 on false claims, as the primary metric\. The train, dev, and test partitions use entry\-level splits that preserve source\-subset proportions and snippet\-level false rates across label dimensions \(Appendix[A\.2](https://arxiv.org/html/2609.12884#A1.SS2)\)\. All 276 entries are used for the zero\-shot snippet\-vs\-atom comparison\. The retrieval\-augmented verifier is developed on dev and confirmed on test\. Consumer\-health entries favor enumeration, while clinical vignettes contain more reasoning chains, diagnostic conclusions, and standalone interpretations\. We use two external medical fact\-checking datasets,HealthFC\([Vladika et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib24)\)andMedHallu\([Pandit et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib21)\), to test whether the snippet\-vs\-atom comparison generalizes beyondMedSNIP\-Bench\.

FromHealthFC, we use 327 claims with binary support labels and exclude 423 insufficient\-evidence claims from F1 computation\. FromMedHallu, we use 2,000 binary items comprising 1,000 paired ground\-truth and hallucinated answers derived fromPubMedQA\([Jin et al\., 2019](https://arxiv.org/html/2609.12884#bib.bib13)\)\. Table[1](https://arxiv.org/html/2609.12884#S3.T1)summarizes corpus statistics, splits, structural patterns, and external datasets\. Since the external datasets lack snippet boundaries, we applyMedSNIPto generate snippets before evaluation\. Atoms and snippets come from the same decomposition call, so both use identical input and differ only in verification unit\. BecauseHealthFCpresents claims as questions, we convert them to propositions before decomposition\.

## 4Method

We compare medical fact\-checking pipelines along three axes:\(i\)verification unit,\(ii\)verification context, and\(iii\)verifier design\. The main comparison isolates verification granularity by evaluating atoms and snippets\. Given a medical answerAA, a decomposer produces units\{u1,…,un\}\\\{u\_\{1\},\\ldots,u\_\{n\}\\\}, and a verifier labels each astrueorfalse\. The atom condition follows theFActScorelineage and verifies the smallest standalone factual statements, while the snippet condition verifies grouped clauses\. ForMedSNIP\-Bench, snippets are human\-annotated\. ForHealthFCandMedHallu, they are generated using theMedSNIPpipeline\.

### 4\.1MedSNIPPipeline

MedSNIPpipeline converts a long\-form answerAAand optional queryqqinto snippetsSS, grouping the answer’s own clauses under the question rather than any retrieved evidence\. Each snippet includes source pointers, one of the A–F structural pattern codes, and shared contextc​t​xctx\. Source pointers identify the atoms or sentences that support the snippet\. Shared context is a short structured summary of the answer’s topic, entities, and patient or query framing\. The LLM steps use this context to interpret otherwise underspecified units\. As a first step, we apply deterministic sentence splitting after removing citation markers, bullets, numbering, and headers\. Sentence indices give the decomposer fixed source pointers, which preserve provenance, let us check that every sentence is assigned, and provide the shared index space in which automatic and human snippets are compared\. The splitter is rule\-based rather than learned, so it introduces no additional model into the pipeline\.

To compare automatic snippets with human snippets, we align source atoms to sentences using normalized\-token overlap and substring matching, enabling sentence\-set F1 evaluation without an additional model\. The pipeline has two modes\. Mode 1 \(Atomize\-then\-group\) first extracts atoms and then uses an LLM to cluster them into snippets, mirroring the benchmark construction process\. Mode 2 \(Snippet\-direct\) skips the atom intermediate and generates snippets directly from sentences\.

Figure[2](https://arxiv.org/html/2609.12884#S3.F2)illustrates both modes using an Advil example, where listed factors are either extracted as atoms and grouped or grouped directly into snippets\. Atomize\-then\-group uses two LLM calls and mirrors human annotation, while snippet\-direct uses one\. We useGPT\-5\.4\([OpenAI, 2026](https://arxiv.org/html/2609.12884#bib.bib20)\)as the decomposer across modes and datasets to avoid confounding the baselines\. Section[5\.5](https://arxiv.org/html/2609.12884#S5.SS5)varies the decomposer across all six models\. Full prompts appear in Appendix[C](https://arxiv.org/html/2609.12884#A3)\. On the fullMedSNIP\-Bench, atomize\-then\-group better matches human snippets than snippet\-direct in sentence\-level F1 \(0\.7560\.756vs\.0\.7340\.734\) and mean embedding similarity \(0\.8240\.824vs\.0\.7940\.794\)\. However, snippet\-direct achieves higherF1F\\text\{F\}\_\{1\}^\{F\}across all three datasets \(Section[5\.5](https://arxiv.org/html/2609.12884#S5.SS5)\)\. Boundary agreement and verification utility are therefore distinct, so we report both modes\. Appendix[A\.5](https://arxiv.org/html/2609.12884#A1.SS5)provides the full pipeline evaluation\. Pattern codes are retained for analysis, while the main per\-pattern results use human annotations fromMedSNIP\-Bench\.

### 4\.2Verification Modes

For each unituu, we evaluate two verification modes\.*\(i\)*Claim\-Onlyverification gives the verifier onlyuu, testing whether the unit is self\-contained\.*\(ii\)*Full\-Contextverification gives the verifieruuwith the original question and full generated answer\. This context is used only to disambiguate the unit, recover omitted entities, and resolve references; the verifier is instructed not to treat the generated answer as evidence\. This tests whether context lost during decomposition can be recovered at verification time\.

### 4\.3Retrieval\-Augmented Verifier

Single\-call verifiers judge each unit from parametric knowledge alone\. This can be brittle for medical text, where verification may depend on drug indications, doses, reference ranges, guidelines, or evidence\. We therefore also evaluate a retrieval\-augmented verifier\. It searches for evidence over multiple rounds, reasons over the accumulated evidence, and returnstrue,false, orabstainwhen the evidence is insufficient\. The verifier maintains an evidence poolℰt\\mathcal\{E\}\_\{t\}, initialized as:

ℰ0=∅\\mathcal\{E\}\_\{0\}=\\emptysetAt retrieval steptt, the model writes a search query from the unit, original query, and current evidence:

st=LLMq​u​e​r​y​\(u,q,ℰt−1\)s\_\{t\}=\\mathrm\{LLM\}\_\{query\}\(u,q,\\mathcal\{E\}\_\{t\-1\}\)
Retrieval runs in parallel over Google Serper111[https://serper\.dev/](https://serper.dev/)for general web evidence and the PubMed E\-utilities API222[https://www\.ncbi\.nlm\.nih\.gov/books/NBK25501/](https://www.ncbi.nlm.nih.gov/books/NBK25501/)for biomedical abstracts\. Each backend returns up tok=3k=3passages per query, and the evidence pool accumulates passages across rounds:

ℰt=ℰt−1∪Retw​e​b​\(st\)∪Retb​i​o​m​e​d​\(st\)\\mathcal\{E\}\_\{t\}=\\mathcal\{E\}\_\{t\-1\}~\\cup~\\mathrm\{Ret\}\_\{web\}\(s\_\{t\}\)~\\cup~\\mathrm\{Ret\}\_\{biomed\}\(s\_\{t\}\)The verifier reads the accumulated evidence and predicts a label with a confidence scorect∈\[0,1\]c\_\{t\}\\in\[0,1\]emitted with its verdict rather than by a separate calibrator:

\(y^t,ct\)=LLMv​e​r​i​f​y​\(u,q,ℰt\)\.\(\\hat\{y\}\_\{t\},c\_\{t\}\)=\\mathrm\{LLM\}\_\{verify\}\(u,q,\\mathcal\{E\}\_\{t\}\)\.Here,y^t∈\{true,false,abstain\}\\hat\{y\}\_\{t\}\\in\\\{\\textsc\{true\},\\textsc\{false\},\\textsc\{abstain\}\\\}\. Following[Xie et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib30), we use a confidence thresholdτ\\tauand accept atrueorfalselabel only whenct≥τc\_\{t\}\\geq\\tau\. Ify^t\\hat\{y\}\_\{t\}isabstainorct<τc\_\{t\}<\\tau, the verifier treats the evidence as insufficient and performs another retrieval round, up toTmax=5T\_\{\\max\}=5\. If no confident verdict is reached, a final prompt returns

y^=LLMfinal​\(u,q,ℰTmax\),\\hat\{y\}=\\mathrm\{LLM\_\{final\}\}\(u,q,\\mathcal\{E\}\_\{T\_\{\\max\}\}\),with the prompt biased towardabstainrather than an unsupportedtrueorfalse\. In the abstention\-dropped setting, units withabstainlabels or confidence belowτ\\tauare left unscored\. We also evaluate a calibrated\-ensemble variant, where uncertain units fall back to the single\-call baseline\. Section[5\.6](https://arxiv.org/html/2609.12884#S5.SS6)reports the resulting coverage\-F1F\\text\{F\}\_\{1\}^\{F\}trade\-off\. Full Prompts are provided in Appendix[C](https://arxiv.org/html/2609.12884#A3)\.

## 5Experiments and Results

We evaluate whether snippet\-level verification improves factuality across datasets, verifier models, and clinical structures\. Unless otherwise stated, experiments useClaim\-Onlyverification and report false\-class F1, denotedF1F\\text\{F\}\_\{1\}^\{F\}\.

MedSNIP\-BenchHealthFCMedHalluModelAtomSnippetMode 1Mode 2AtomMode 1Mode 2AtomMode 1Mode 2GPT\-5\.4\-high0\.3210\.4310\.4290\.4180\.6720\.6720\.6930\.6940\.7200\.721GPT\-4o0\.3440\.4010\.3010\.3170\.6890\.7100\.7360\.5590\.5680\.579Gemma\-4\-31B\-it0\.3400\.3890\.3500\.3770\.6640\.6820\.6900\.6120\.6390\.648GPT\-OSS\-20B0\.3440\.4010\.3300\.3660\.6800\.6780\.6890\.6230\.6270\.636Llama\-3\.3\-70B0\.3160\.3310\.2350\.2800\.6780\.7000\.6930\.5070\.4820\.487Llama\-3\.1\-8B0\.2380\.2780\.2010\.2120\.6120\.5980\.6100\.4710\.5330\.556Table 2:F1F\\text\{F\}\_\{1\}^\{F\}forClaim\-Onlyverification across six models and three datasets\. Green and red mark gains and losses over atoms, with significant differences in bold \(95% confidence interval\)\.MedSNIP\-Benchuses human snippets, while both pipeline modes use drop\-mixed projection\. External units share a decomposition call\.### 5\.1Experimental Protocol

We evaluate on the three datasets defined in Table[1](https://arxiv.org/html/2609.12884#S3.T1),MedSNIP\-Bench,HealthFC, andMedHallu\. On the external datasets, we use the atomize\-then\-group pipeline to produce snippets before verification\. We evaluate six verifier models\. The closed models areGPT\-5\.4with high reasoning effort andGPT\-4owith temperature zero\([Hurst et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib11)\)\.

The open\-weight models areGemma\-4\-31B\-it,GPT\-OSS\-20B,Llama\-3\.3\-70B, andLlama\-3\.1\-8B\([Google, 2026](https://arxiv.org/html/2609.12884#bib.bib6);[Agarwal et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib1);[Grattafiori et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib7)\)\. All models use the same prompts, with atom and snippet decompositions produced byGPT\-5\.4to isolate verification granularity\. ForHealthFCandMedHallu,Or\-Falsepredicts an answer asfalseif any unit is predictedfalse\.MedSNIP\-Benchrequires no answer\-level aggregation because labels are at the snippet level\. Automatic snippets use drop\-mixed projection from human atom labels\. Significance uses 95% bootstrap intervals over 10,000 paired entry\- or claim\-level resamples \(see Appendix[D](https://arxiv.org/html/2609.12884#A4)\)\.

### 5\.2Snippet Verification ImprovesF1F\\text\{F\}\_\{1\}^\{F\}

Table[2](https://arxiv.org/html/2609.12884#S5.T2)compares atoms and snippets across six verifiers and three datasets\. OnMedSNIP\-Bench, human snippets outperform atoms for all verifiers, significantly for the four strongest\. With automatic snippets, onlyGPT\-5\.4\-high gains significantly, scoring0\.4290\.429versus0\.4310\.431on human snippets\. Four other verifiers fall below their atom baselines, significantly forLlama\-3\.3\-70B\. Thus, the human\-automatic gap is verifier\-dependent\. Snippet\-direct outperforms atomize\-then\-group for all but the strongest verifier and on the external datasets\. It yields two significantHealthFCgains and four of eight significantMedHallugains, while atomize\-then\-group yields none onHealthFC\. Gains grow with verifier strength and source structure \(Section[5\.3](https://arxiv.org/html/2609.12884#S5.SS3)\)\. Snippets also reduce verifier calls\. OnMedSNIP\-Bench, human snippets reduce units by 56%, while atomize\-then\-group and snippet\-direct reduce them by 73% and 63%\. The two modes reduce units by 24% and 29% onHealthFC, and 31% and 32% onMedHallu\. Since snippets are longer,MedSNIP\-Benchverification cost falls by 54%, versus 73% for units\. Appendix[D\.5](https://arxiv.org/html/2609.12884#A4.SS5)reports end\-to\-end costs\.

##### Result analysis:

The cross\-model results support the main granularity claim, with a boundary condition\. Snippet\-level verification reduces calls by grouping units and improvesF1F\\text\{F\}\_\{1\}^\{F\}across verifier families, but the improvement is not uniform\. It is largest where the source answer is long enough for atomization to fragment a clinical judgement and where the verifier can reason over the recovered structure\. Where either condition is absent the effect narrows and can reverse, significantly so forLlama\-3\.3\-70Bon bothMedSNIP\-BenchandMedHallu\. At the same time, the automatic\-snippet columns show that segmentation quality matters\. The strongest verifier is less affected by pipeline\-generated snippets, while weaker verifiers show larger drops from human\-snippet performance\. Thus, snippet\-level verification is a granularity choice whose benefit depends on both the verification unit and the verifier’s ability to reason over it\. The advantage is not unconditional\. UnderFull\-Contextverification with weaker models, atoms can match or beat snippets, see Appendix[D](https://arxiv.org/html/2609.12884#A4)for more details\.

### 5\.3Structure and Snippet Gains

Snippet\-level verification can only help when there is meaningful structure for atomization to fragment\. The three corpora differ substantially in this respect, providing a natural explanation for the variation in gains across datasets\.

CorpusWordsSent\.MergeΔ\\DeltaF1F\\text\{F\}\_\{1\}^\{F\}MedSNIP\-Bench244113\.37\+0\.114\+0\.114MedHallu2911\.44\+0\.026\+0\.026HealthFC1111\.31\+0\.000\+0\.000Table 3:Source length and grouping opportunity per corpus, with theGPT\-5\.4snippet–atom gap\. All three columns use atomize\-then\-group, so the comparison is matched\.Table[3](https://arxiv.org/html/2609.12884#S5.T3)relates source length and merge ratio to the snippet\-to\-atom gap underGPT\-5\.4\.MedSNIP\-Benchhas the most structure, with medians of 244 words, 11 sentences, and 3\.37 atoms per snippet, and the largest gain of\+0\.114\+0\.114F1F\\text\{F\}\_\{1\}^\{F\}\. In contrast, 98% ofHealthFCclaims are single sentences, yielding little structure and no gain\.MedHallulies between both datasets in structure and improvement\. This ordering holds only with a strong verifier\. Weaker open\-weight verifiers can lose accuracy on richerMedSNIP\-Benchsnippets \(Table[2](https://arxiv.org/html/2609.12884#S5.T2)\)\. Snippet\-level verification therefore requires useful source structure and a verifier capable of using it\. With three corpora, we report this ordering without fitting a formal relationship\.

### 5\.4Per\-Pattern Analysis

TheMedSNIP\-Benchannotations show where snippets help\. Table[4](https://arxiv.org/html/2609.12884#S5.T4)reports pattern\-levelF1F\\text\{F\}\_\{1\}^\{F\}usingGPT\-5\.4withClaim\-Only\. Snippets outperform atoms on all patterns except the small topic\-shift category E\. The largest merge\-pattern gain is for Pattern B, causal\-conditional chains, atΔ=\+0\.083\\Delta=\+0\.083, suggesting that atomization harms verification of clause relations\. Pattern E favors atoms, but contains only 39 snippets and its confidence interval crosses zero\. For single\-atom patterns D and F, boundaries often coincide, so gains may reflect cleaner snippet text rather than granularity\. Patterns A, B, and C provide a clearer test\. Results also depend on the setting\. WithFull\-Contextor weaker verifiers, several gaps narrow or reverse, including Pattern B underFull\-ContextwithGPT\-4o\(Appendix[D](https://arxiv.org/html/2609.12884#A4)\)\.

PatternnnSnipAtomΔ\\DeltaA enumeration1,0080\.3530\.293\+0\.059B causal/cond\.2250\.3870\.304\+0\.083C concl\.\+prem\.3210\.4420\.391\+0\.051D standalone6170\.3840\.303\+0\.081E topic\-shift390\.1430\.211−\-0\.068F distinct fact3140\.6420\.586\+0\.056Overall25240\.4270\.359\+0\.068Table 4:Per\-patternF1F\\text\{F\}\_\{1\}^\{F\}onMedSNIP\-BenchusingGPT\-5\.4underClaim\-Onlyverification\. BoldΔ\\Deltamarks significance under 95% bootstrap confidence intervals\.DecomposerSent F1MergeΔ\\DeltaF1F\\text\{F\}\_\{1\}^\{F\}CostGPT\-5\.4\(high\)0\.7563\.37\+0\.099\+0\.099$23\.63Gemma\-4\-31B\-it0\.7562\.08\+0\.061\+0\.061$0\.29Llama\-3\.3\-70B0\.7421\.98\+0\.095\+0\.095$0\.21GPT\-4o0\.7222\.31\+0\.092\+0\.092$5\.79GPT\-OSS\-20B0\.7062\.59\+0\.055\+0\.055$0\.20Llama\-3\.1\-8B0\.6951\.37\+0\.045\+0\.045$0\.03Table 5:Decomposer quality and downstream effect onMedSNIP\-Bench, withGPT\-5\.4verifying\. Every gap is measured against the same expert\-atom baseline and every one excludes zero\.##### Result analysis:

The results clarify why snippets help\. The largest merge\-pattern gain occurs on causal\-conditional chains, where a conclusion depends on premises, conditions, or mechanisms\.

These are the cases where atomization can separate the statement being checked from the information that makes it verifiable\. Enumeration and conclusion\-with\-premises patterns also favor snippets, though with smaller gains\. The positive deltas for standalone and distinct\-fact patterns should be interpreted more cautiously, since those gains may reflect cleaner snippet wording rather than grouping itself\. Overall, the pattern analysis supports the central mechanism that snippets help most when the clinical judgment is distributed across related clauses\.

### 5\.5Robustness of the Decomposer

To test dependence onGPT\-5\.4, we generate snippets with all six models and verify them with all six verifiers in both pipeline modes\. Table[5](https://arxiv.org/html/2609.12884#S5.T5)reports theGPT\-5\.4verifier column, while Appendix[D\.7](https://arxiv.org/html/2609.12884#A4.SS7)gives all 72 combinations\. The table includes boundary fidelity, merge ratio, the snippet\-direct gap against expert atoms, and decomposition cost for 276 entries\. Boundary fidelity does not follow model scale\. On the 275 entries completed by both models,Gemma\-4\-31B\-itmatchesGPT\-5\.4at0\.7560\.756, with higher precision \(0\.7780\.778vs\.0\.7230\.723\), while costing $0\.29 instead of $23\.63\.Llama\-3\.3\-70Balso outranksGPT\-4o\. Rankings instead follow merge ratio\.GPT\-5\.4groups most at 3\.37 atoms per snippet, increasing recall to0\.8980\.898but lowering precision\.Llama\-3\.1\-8Bgroups least at 1\.37 and reverses this balance\. WithGPT\-5\.4verification, every decomposer outperforms expert atoms, with all gaps significant under an entry\-clustered bootstrap\.Llama\-3\.1\-8Bgains\+0\.045\+0\.045versusGPT\-5\.4’s\+0\.099\+0\.099, so decomposer variation is smaller than the gain over atoms\. Boundary fidelity and downstream utility differ\.Gemma\-4\-31B\-itties for the best boundary agreement but has one of the smallest verification gains, echoing Section[4\.1](https://arxiv.org/html/2609.12884#S4.SS1)\.

With only six decomposers, we report both rankings without fitting a correlation\. Across the full matrix, verifier capability matters more than decomposer identity\. Averaged over both modes, verifier column means span0\.1130\.113F1F\\text\{F\}\_\{1\}^\{F\}, while decomposer row means span0\.0190\.019, a sixfold difference\. This favors deployment because decomposition runs once per answer, while verification runs once per unit\.

##### Wording against grouping:

Patterns D and F contain one atom each, so any gain there cannot come from grouping\. We isolate the two effects by rewriting each atom with the pipeline’s decontextualization rules and no grouping, then verifying the rewritten atom\. Decontextualization alone recovers 94% of the gain on the keep\-atomic patterns and 26% on the merge patterns, leaving 74% of the merge\-pattern gain attributable to grouping\. Wording explains least on Pattern A at 21%, then Pattern B, the causal\-conditional chains that motivate the method, at 25%\. The full three\-way comparison is in Appendix[D\.6](https://arxiv.org/html/2609.12884#A4.SS6)\.

##### Cost with decomposition included:

Reporting verifier calls alone understates what snippet\-level verification costs, since building snippets takes an extra call per answer in atomize\-then\-group\. Charging each pipeline for its own decomposition reverses the headline for the configuration used above\. WithGPT\-5\.4decomposing, snippet verification costs 19\.1% more end to end than atom verification, because the second decomposition call outweighs the verifier calls it saves\. WithGPT\-OSS\-20Bdecomposing, the same comparison saves 34\.1%\. The open\-weight result above is therefore not only a robustness check\. It is what makes the cost reduction survive honest accounting\. Appendix[D\.5](https://arxiv.org/html/2609.12884#A4.SS5)gives the full breakdown\.

### 5\.6Retrieval\-Augmented Verification Provides a Coverage Lever

Table[6](https://arxiv.org/html/2609.12884#S5.T6)compares the retrieval\-augmented verifier with the single\-call baseline onMedSNIP\-Benchdev and test\. At near\-full coverage, retrieval matches but does not improve over the baseline\. Higher confidence thresholds trade coverage forF1F\\text\{F\}\_\{1\}^\{F\}\. Atτ=0\.90\\tau=0\.90,F1F\\text\{F\}\_\{1\}^\{F\}reaches0\.6090\.609on dev at 63% coverage and0\.5520\.552on test at 61%\. The robust finding is therefore a coverage\-F1F\\text\{F\}\_\{1\}^\{F\}trade\-off, not a full\-coverage improvement\. Full\-coverage ensembles and subset routers improved dev results but did not replicate on test and are reported as negative results in Appendix[E](https://arxiv.org/html/2609.12884#A5)\.

MetricBaselineVerifier, fullτ≥0\.85\\tau\\geq 0\.85τ≥0\.90\\tau\\geq 0\.90DevF1F\\text\{F\}\_\{1\}^\{F\}0\.5180\.5180\.5620\.609Dev cov\.100%96%83%63%TestF1F\\text\{F\}\_\{1\}^\{F\}0\.4910\.4970\.5040\.552Test cov\.100%97%83%61%Table 6:Coverage\-F1F\\text\{F\}\_\{1\}^\{F\}trade\-off onMedSNIP\-BenchwithGPT\-5\.4\. Thresholded results exclude abstained and low\-confidence units\.##### Results analysis:

Retrieval augmentation serves a different role from snippets\. It does not improve on the single\-call baseline at full coverage, but enables confidence\-based abstention\. Excluding uncertain cases raisesF1F\\text\{F\}\_\{1\}^\{F\}on dev and test while reducing coverage, which may help when evidence and abstention matter\. The fact that the full\-coverage variants did not replicate also argues against treating retrieval augmentation as the source of the paper’s main improvement\.

## 6Conclusion and Future Work

Medical fact\-checking depends not only on the verifier, but also on the unit being verified\. We introducedMedSNIP\-Bench, a human\-annotated benchmark for snippet\-level medical fact\-checking, andMedSNIP, an automatic pipeline for generating snippet\-level verification units\. AcrossMedSNIP\-Bench,HealthFC, andMedHallu, snippet\-level verification improves false\-class F1 under capable verifiers while reducing verifier calls\. The strongest and most interpretable gains occur when atomization separates causal, conditional, or premise\-supported clinical reasoning into isolated claims\. This benefit depends on available structure and the verifier’s ability to use it\. These results suggest that verification granularity should be treated as a core design choice in medical fact\-checking pipelines, not a fixed preprocessing default\.

In future work, we plan to explore adaptive granularity policies that choose between atomic, snippet\-level, and document\-level verification based on clinical reasoning structure and verifier uncertainty, rather than relying on a fixed decomposition strategy\. We further plan to extendMedSNIPbeyond English and integrate richer clinical evidence sources, such as guidelines, electronic health record–style data, and longitudinal patient context, to better evaluate factuality in realistic clinical decision\-support settings\.

## Limitations

##### Scope:

Our experiments focus on English medical fact\-checking datasets\. Other languages and domains may require different granularity choices, especially when claims are more independent or when retrieval coverage differs substantially\. We view snippet\-level verification as a design choice for medically structured claims, not as a universal replacement for atom\-level verification\.

##### Model and aggregation dependence:

The main experiments cover six verifier models, withGPT\-5\.4used for decomposition across datasets\. AbsoluteF1F\\text\{F\}\_\{1\}^\{F\}scores depend on verifier strength, and the relative atom–snippet gap can vary with prompt context and aggregation rule\. In particular, weak verifiers inFull\-Contextmode can reduce or reverse the snippet advantage\. We report aggregation sensitivity and verifier ablations in Appendix[D](https://arxiv.org/html/2609.12884#A4)\.

##### Granularity and text quality:

When snippet and atom boundaries coincide, gains may come from clearer wording, such as resolved references, rather than granularity\. Evidence for granularity is strongest for merge patterns, including enumerations, causal or conditional chains, and conclusions with premises\. Appendix[D\.6](https://arxiv.org/html/2609.12884#A4.SS6)separates these effects by rewriting atoms without grouping\. Wording explains nearly all of the keep\-atomic gain but only about a quarter of the merge gain\. Because repeated verification of identical text changesF1F\\text\{F\}\_\{1\}^\{F\}by0\.0130\.013, smaller differences may reflect variation rather than an effect\.

##### Sample size:

MedSNIP\-Benchcontains 276 entries and 2,524 snippets, which is enough to separate the largest pattern categories but not the smallest\. Pattern E holds 39 snippets, and its negative delta has a confidence interval spanning\[−0\.300,\+0\.095\]\[\-0\.300,\+0\.095\], so we draw no conclusion from it\. The external evidence is similarly bounded, sinceHealthFCscores only the 327 claims that carry binary support labels\. Section[5\.3](https://arxiv.org/html/2609.12884#S5.SS3)reports an ordering across three corpora rather than a fitted relationship for the same reason\.

##### Pipeline and retrieval design:

External\-dataset snippets are generated automatically with a single LLM\-based decomposer\. Section[5\.5](https://arxiv.org/html/2609.12884#S5.SS5)varies the decomposer over all six models, but only onMedSNIP\-Bench, so the decomposer sweep and the external corpora do not overlap\.

Similarly, our retrieval\-augmented verifier is intentionally simple and uses general web and biomedical\-literature retrieval rather than specialized clinical\-evidence sources\. Stronger retrieval may improve coverage and absoluteF1F\\text\{F\}\_\{1\}^\{F\}, but retrieval design is not the main focus of this work\.

##### Development and test use:

We use the held\-out test split to confirm patterns observed during development\. Two development\-time variants, a calibrated ensemble and a subset\-conditional router, did not replicate on test\. We report these negative results in Appendix[E](https://arxiv.org/html/2609.12884#A5)\.

## Ethics and Broader Impact

This work studies automated factuality evaluation for medical text generation\. It is intended to support auditing and research on medical LLMs, not to provide medical advice, diagnosis, treatment recommendations, or autonomous clinical decision support\. The systems evaluated here should be used only with appropriate expert oversight\. A central motivation is harm reduction\. Unsupported medical recommendations can mislead patients, clinicians, and downstream systems, and more reliable fact\-checking may help identify such failures earlier\. At the same time, automated verification is imperfect\. A verifier may accept false claims, reject correct claims, or abstain on clinically important cases\. Our results should therefore be interpreted as progress on evaluation methodology, not as evidence that automated medical fact\-checking is solved\.

MedSNIPis derived from previously released medical QA resources and does not contain real patient records, protected health information, or identifiable clinical documentation\. The clinical\-vignette examples are synthetic educational cases rather than electronic health records\. Nevertheless, the dataset may reflect biases in medical education materials, English\-language evidence sources, and annotation practices\. The annotation task also involves judgment calls about snippet boundaries and contextual correctness\. We provide guidelines and report inter\-annotator agreement, but other annotator groups or healthcare settings may choose different decompositions\. We release the annotation guidelines, data, and code for inspection and reuse\. Efficient verification could also scale low\-quality medical content or overconfident evaluation\. Snippet\-level verification should support auditing, not replace expert review\.

## References

- Agarwal et al\. \(2025\)Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K\. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, Kai Chen, and 106 others\. 2025\.[GPT\-OSS\-120B & GPT\-OSS\-20B model card](https://arxiv.org/abs/2508.10925)\.*Preprint*, arXiv:2508\.10925\.
- Chen et al\. \(2025\)Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze\. 2025\.[Benchmarking large language models on answering and explaining challenging medical questions](https://doi.org/10.18653/v1/2025.naacl-long.182)\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 3563–3599, Albuquerque, New Mexico\. Association for Computational Linguistics\.
- Chern et al\. \(2025\)Ethan Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, and Pengfei Liu\. 2025\.[FacTool: Factuality detection in generative AI – a tool augmented framework for multi\-task and multi\-domain scenarios](https://openreview.net/forum?id=hJkQL9VtWT)\.In*Proceedings of the 2025 Conference on Language Modeling*, Montreal, Canada\.
- Dhuliawala et al\. \(2024\)Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston\. 2024\.[Chain\-of\-verification reduces hallucination in large language models](https://doi.org/10.18653/v1/2024.findings-acl.212)\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 3563–3578, Bangkok, Thailand\. Association for Computational Linguistics\.
- Gao et al\. \(2023\)Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da\-Cheng Juan, and Kelvin Guu\. 2023\.[RARR: Researching and revising what language models say, using language models](https://doi.org/10.18653/v1/2023.acl-long.910)\.In*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 16477–16508, Toronto, Canada\. Association for Computational Linguistics\.
- Google \(2026\)Google\. 2026\.Gemma 4 model card\.[https://ai\.google\.dev/gemma/docs/core/model\_card\_4](https://ai.google.dev/gemma/docs/core/model_card_4)\.Accessed 2026\-05\-23\.
- Grattafiori et al\. \(2024\)Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others\. 2024\.[The Llama 3 herd of models](https://arxiv.org/abs/2407.21783)\.*Preprint*, arXiv:2407\.21783\.
- Gunjal and Durrett \(2024\)Anisha Gunjal and Greg Durrett\. 2024\.[Molecular facts: Desiderata for decontextualization in LLM fact verification](https://doi.org/10.18653/v1/2024.findings-emnlp.215)\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 3751–3768, Miami, Florida, USA\. Association for Computational Linguistics\.
- He et al\. \(2026\)Jiayi He, Yangmin Huang, Qianyun Du, Xiangying Zhou, Zhiyang He, Jiaxue Hu, Xiaodong Tao, and Lixian Lai\. 2026\.[MedFact: Benchmarking the fact\-checking capabilities of large language models on Chinese medical texts](https://doi.org/10.18653/v1/2026.gem-main.59)\.In*Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics \(GEM\)*, pages 604–652, San Diego, California, USA\. Association for Computational Linguistics\.
- Huang et al\. \(2026\)Heyuan Huang, Alexandra DeLucia, Vijay Murari Tiyyala, and Mark Dredze\. 2026\.[MedScore: Generalizable factuality evaluation of open\-ended long\-form medical answers by domain\-adapted claim decomposition and verification](https://doi.org/10.18653/v1/2026.findings-acl.693)\.In*Findings of the Association for Computational Linguistics: ACL 2026*, pages 14149–14180, San Diego, California, United States\. Association for Computational Linguistics\.
- Hurst et al\. \(2024\)Aaron Hurst, Adam Lerer, Adam P\. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker\-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, and 399 others\. 2024\.[GPT\-4o System Card](https://arxiv.org/abs/2410.21276)\.*Preprint*, arXiv:2410\.21276\.
- Iqbal et al\. \(2024\)Hasan Iqbal, Yuxia Wang, Minghan Wang, Georgi Nenkov Georgiev, Jiahui Geng, Iryna Gurevych, and Preslav Nakov\. 2024\.[OpenFactCheck: A unified framework for factuality evaluation of LLMs](https://doi.org/10.18653/v1/2024.emnlp-demo.23)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*, pages 219–229, Miami, Florida, USA\. Association for Computational Linguistics\.
- Jin et al\. \(2019\)Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu\. 2019\.[PubMedQA: A dataset for biomedical research question answering](https://doi.org/10.18653/v1/D19-1259)\.In*Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\)*, pages 2567–2577, Hong Kong, China\. Association for Computational Linguistics\.
- Kim et al\. \(2025\)Hyunjae Kim, Jiwoong Sohn, Aidan Gilson, Nicholas Cochran\-Caggiano, Serina Applebaum, Heeju Jin, Seihee Park, Yujin Park, Jiyeong Park, Seoyoung Choi, Brittany Alexandra Herrera Contreras, Thomas Huang, Jaehoon Yun, Ethan F\. Wei, Roy Jiang, Leah Colucci, Eric Lai, Amisha Dave, Tuo Guo, and 8 others\. 2025\.[Rethinking retrieval\-augmented generation for medicine: A large\-scale, systematic expert evaluation and practical insights](https://arxiv.org/abs/2511.06738)\.*Preprint*, arXiv:2511\.06738\.
- Landis and Koch \(1977\)J\. Richard Landis and Gary G Koch\. 1977\.[The measurement of observer agreement for categorical data](https://www.jstor.org/stable/2529310)\.*Biometrics*, pages 159–174\.
- Liu et al\. \(2025\)Fenglin Liu, Hongjian Zhou, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S\. Chen, Yining Hua, Peilin Zhou, Junling Liu, Chengfeng Mao, Chenyu You, Xian Wu, Yefeng Zheng, Lei Clifton, Zheng Li, Jiebo Luo, and David A\. Clifton\. 2025\.[Application of large language models in medicine](https://doi.org/10.1038/s44222-025-00279-5)\.*Nature Reviews Bioengineering*, 3\(6\):445–464\.
- Lu et al\. \(2025\)Yining Lu, Noah Ziems, Hy Dang, and Meng Jiang\. 2025\.[Optimizing decomposition for optimal claim verification](https://doi.org/10.18653/v1/2025.acl-long.254)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 5095–5114, Vienna, Austria\. Association for Computational Linguistics\.
- Manes et al\. \(2024\)Itay Manes, Naama Ronn, David Cohen, Ran Ilan Ber, Zehavi Horowitz\-Kugler, and Gabriel Stanovsky\. 2024\.[K\-QA: A real\-world medical Q&A benchmark](https://doi.org/10.18653/v1/2024.bionlp-1.22)\.In*Proceedings of the 23rd Workshop on Biomedical Natural Language Processing*, pages 277–294, Bangkok, Thailand\. Association for Computational Linguistics\.
- Min et al\. \(2023\)Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen\-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi\. 2023\.[FActScore: Fine\-grained atomic evaluation of factual precision in long form text generation](https://doi.org/10.18653/v1/2023.emnlp-main.741)\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 12076–12100, Singapore\. Association for Computational Linguistics\.
- OpenAI \(2026\)OpenAI\. 2026\.Introducing GPT\-5\.4\.[https://openai\.com/index/gpt\-5\-4\-thinking\-system\-card/](https://openai.com/index/gpt-5-4-thinking-system-card/)\.Accessed 2026\-05\-23\.
- Pandit et al\. \(2025\)Shrey Pandit, Jiawei Xu, Junyuan Hong, Zhangyang Wang, Tianlong Chen, Kaidi Xu, and Ying Ding\. 2025\.[MedHallu: A comprehensive benchmark for detecting medical hallucinations in large language models](https://doi.org/10.18653/v1/2025.emnlp-main.143)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 2858–2873, Suzhou, China\. Association for Computational Linguistics\.
- Singhal et al\. \(2023\)Karan Singhal, Shekoofeh Azizi, Tao Tu, S\. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole\-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner\-Fushman, and 13 others\. 2023\.[Large language models encode clinical knowledge](https://doi.org/10.1038/s41586-023-06291-2)\.*Nature*, 620\(7972\):172–180\.
- Singhal et al\. \(2025\)Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R\. Pfohl, Heather Cole\-Lewis, Darlene Neal, Qazi Mamunur Rashid, Mike Schaekermann, Amy Wang, Dev Dash, Jonathan H\. Chen, Nigam H\. Shah, Sami Lachgar, Philip Andrew Mansfield, and 16 others\. 2025\.[Toward expert\-level medical question answering with large language models](https://doi.org/10.1038/s41591-024-03423-7)\.*Nature Medicine*, 31\(3\):943–950\.
- Vladika et al\. \(2024\)Juraj Vladika, Phillip Schneider, and Florian Matthes\. 2024\.[HealthFC: Verifying health claims with evidence\-based medical fact\-checking](https://aclanthology.org/2024.lrec-main.709/)\.In*Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\)*, pages 8095–8107, Torino, Italia\. ELRA and ICCL\.
- Wang and Zhang \(2024\)Dandan Wang and Shiqing Zhang\. 2024\.[Large language models in medical and healthcare fields: Applications, advances, and challenges](https://link.springer.com/article/10.1007/s10462-024-10921-0)\.*Artificial intelligence review*, 57\(11\):299\.
- Wang et al\. \(2025\)Yuxia Wang, Minghan Wang, Hasan Iqbal, Georgi N\. Georgiev, Jiahui Geng, Iryna Gurevych, and Preslav Nakov\. 2025\.[OpenFactCheck: Building, benchmarking customized fact\-checking systems and evaluating the factuality of claims and LLMs](https://aclanthology.org/2025.coling-main.755/)\.In*Proceedings of the 31st International Conference on Computational Linguistics*, pages 11399–11421, Abu Dhabi, UAE\. Association for Computational Linguistics\.
- Wang et al\. \(2024\)Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Nenkov Georgiev, Rocktim Jyoti Das, and Preslav Nakov\. 2024\.[Factuality of large language models: A survey](https://doi.org/10.18653/v1/2024.emnlp-main.1088)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 19519–19529, Miami, Florida, USA\. Association for Computational Linguistics\.
- Wanner et al\. \(2024\)Miriam Wanner, Seth Ebner, Zhengping Jiang, Mark Dredze, and Benjamin Van Durme\. 2024\.[A closer look at claim decomposition](https://doi.org/10.18653/v1/2024.starsem-1.13)\.In*Proceedings of the 13th Joint Conference on Lexical and Computational Semantics \(\*SEM 2024\)*, pages 153–175, Mexico City, Mexico\. Association for Computational Linguistics\.
- Wei et al\. \(2024\)Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Jie Huang, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V\. Le\. 2024\.[Long\-form factuality in large language models](https://doi.org/10.52202/079017-2567)\.In*Advances in Neural Information Processing Systems*, volume 37, pages 80756–80827\. Curran Associates, Inc\.
- Xie et al\. \(2025\)Zhuohan Xie, Rui Xing, Yuxia Wang, Jiahui Geng, Hasan Iqbal, Dhruv Sahnan, Iryna Gurevych, and Preslav Nakov\. 2025\.[FIRE: Fact\-checking with iterative retrieval and verification](https://doi.org/10.18653/v1/2025.findings-naacl.158)\.In*Findings of the Association for Computational Linguistics: NAACL 2025*, pages 2901–2914, Albuquerque, New Mexico\. Association for Computational Linguistics\.
- Zerong et al\. \(2025\)Zhaxi Zerong, Chenxi Li, Xinyi Liu, Ju\-hui Chen, and Fei Xia\. 2025\.[A systematic survey of claim verification: Corpora, systems, and case studies](https://doi.org/10.18653/v1/2025.findings-emnlp.1170)\.In*Findings of the Association for Computational Linguistics: EMNLP 2025*, pages 21452–21474, Suzhou, China\. Association for Computational Linguistics\.
- Zhu et al\. \(2025\)Zhihong Zhu, Yunyan Zhang, Xianwei Zhuang, Fan Zhang, Zhongwei Wan, Yuyan Chen, Qingqing Long, Yefeng Zheng, and Xian Wu\. 2025\.[Can we trust AI doctors? A survey of medical hallucination in large language and large vision\-language models](https://doi.org/10.18653/v1/2025.findings-acl.350)\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 6748–6769, Vienna, Austria\. Association for Computational Linguistics\.

## Appendix AData Construction Details

This appendix documents the construction ofMedSNIP\-Bench, including how we partitioned the parent corpus, created the train, dev, and test splits, measured inter\-annotator agreement, and evaluated how faithfully theMedSNIPpipeline reproduces human snippet structure\.

### A\.1Subset Selection

The[Kim et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib14)corpus blends short consumer\-health questions fromK\-QA\([Manes et al\., 2024](https://arxiv.org/html/2609.12884#bib.bib18)\)with longerUSMLE\-style clinical\-vignette questions fromMedBullets\([Chen et al\., 2025](https://arxiv.org/html/2609.12884#bib.bib2)\)\. Query length is bimodal, with consumer\-health queries ranging from 2 to 31 words and clinical vignettes from 112 to 254 words, leaving no queries in the 32–111\-word gap \(Figure[3](https://arxiv.org/html/2609.12884#A1.F3)a\)\. We use a threshold of 80 words to split the 276 entries into 140 consumer\-health and 136 clinical\-vignette entries\. Any threshold between 32 and 111 yields the same partition\.

The two subsets differ in verification demands as well as length\. Consumer\-health queries are short questions about a drug, dose, interaction, or symptom, with answers typically grounded in standard\-of\-care knowledge\. Clinical vignettes provide a structured patient history and ask for a diagnosis, next test, or treatment, so the answer connects premises whose correctness depends on the patient’s specific context rather than general medical facts alone\.

##### Representative queries:

A typical consumer\-health query is a single short sentence:

> “Can I take Nyquil and Benadryl at the same time?”

A typical clinical\-vignette query is a paragraph\-length scenario followed by a question:

> “A 1\-year\-old girl is brought to a neurologist due to increasing seizure frequency over the past 2 months\. She recently underwent a neurology evaluation which revealed hypsarrhythmia on EEG with multifocal spikes… Her medications consist of lamotrigine and valproic acid… What is the most appropriate next step in management?”

The two query styles drive snippet patterns: consumer queries skew toward Pattern A \(enumeration of features or factors\), while vignettes skew toward Patterns B and C \(causal–conditional chains and conclusion\-with\-premises\)\. The pattern distribution in Table[1](https://arxiv.org/html/2609.12884#S3.T1)reflects this asymmetry\.

Vignette responses carry a higher atom\-level false\-rate \(10\.0%\) than consumer responses \(6\.5%\), consistent with the harder clinical\-reasoning content \(Figure[3](https://arxiv.org/html/2609.12884#A1.F3)b\)\. All downstream subset\-conditional analyses preserve this distinction\.

\(a\)Query word\-length distribution\.\(b\)Atomic\-claim composition by subset\.
Figure 3:Subset selection from the parent corpus\.

### A\.2Train / Dev / Test Split

We stratify the 276 entries by source subset and a coarse in\-general false\-rate bucket, then sweep random seeds to minimize snippet\-level false\-rate differences across the three label dimensions while maintaining balanced splits\. Seed 202 provides the best balance, with a worst\-case spread of only1\.11\.1percentage points across splits\. Entry and snippet counts for each split are reported in Table[1](https://arxiv.org/html/2609.12884#S3.T1)\.

The resulting partition preserves the original consumer\-to\-vignette ratio across splits \(Figure[4](https://arxiv.org/html/2609.12884#A1.F4)a\)\. Snippet\-level false\-rates are tightly matched across splits within each subset on all three label dimensions \(Figure[4](https://arxiv.org/html/2609.12884#A1.F4)b\)\.

\(a\)Per\-subset entry counts across splits\.![Refer to caption](https://arxiv.org/html/2609.12884v1/split-reasoning-b.png)\(b\)Snippet\-level false\-rate per \(subset×\\timessplit\)\.

Figure 4:Train / dev / test split\.
### A\.3Shared Context

\(a\)Grouping agreement \(ARI per entry\)\.\(b\)Fleiss’κ\\kappaper labeled field\.![Refer to caption](https://arxiv.org/html/2609.12884v1/iaa-reasoning-c.png)\(c\)Pairwise Cohen’sκ\\kappafor in\-general\.

Figure 5:Inter\-annotator agreement on the 64\-snippet IAA batch annotated independently by all six annotators\.Each entry includes ashared contextblock, a structured key–value summary of the query and response framing that applies to all snippets\. It anchors decontextualized snippets in the clinical or topical scenario, supporting verification against the relevant patient or query\. The schema is subset\-dependent\. Clinical\-vignette entries use keys such asage,sex,chief\_complaint,medical\_history,medications,vitals,treatment\_history, andcorrect\_answer\. Consumer\-health entries use lighter keys such astopicanddrug\_a/drug\_b\.The schema is open, allowing annotators to add keys when downstream snippets refer to additional information\.

MedSNIPpipeline preserves this convention\.Mode 1andMode 2both emit a shared\-context block alongside the snippet list \(Section[4\.1](https://arxiv.org/html/2609.12884#S4.SS1)\), so automatic and human\-annotated entries are interchangeable downstream\. Verification prompts may reference the shared context to recover entities that are absent from the snippet text, which matters most for the with\-patient\-context label where the relevant frame is not in the snippet itself\.

### A\.4Inter\-Annotator Agreement

Inter\-annotator reliability forMedSNIP\-Benchwas computed on a 64\-snippet IAA batch annotated independently by all six annotators\. Figure[5](https://arxiv.org/html/2609.12884#A1.F5)shows chunking agreement per entry, Fleiss’κ\\kappafor each labeled field, and pairwise Cohen’sκ\\kappafor the in\-general label\.

##### Binary in\-general label:

Fleiss’κ=0\.662\\kappa=0\.662across six annotators \(Figure[5b](https://arxiv.org/html/2609.12884#A1.F5.sf2)\)\. Pairwise Cohen’sκ\\kapparanged from0\.3850\.385to0\.9470\.947across 15 annotator pairs, with a mean of0\.6630\.663\(Figure[5c](https://arxiv.org/html/2609.12884#A1.F5.sf3)\)\. All six annotators agreed on 52 of 64 items \(81\.3%81\.3\\%\)\.

##### Snippet chunking:

Adjusted Rand index \(ARI\) on snippet partitions gave a mean of0\.7230\.723, ranging from0\.5240\.524to0\.9570\.957across annotator pairs \(Figure[5a](https://arxiv.org/html/2609.12884#A1.F5.sf1)\)\. The more constrained chunking decision therefore shows substantial agreement\.

##### Pattern label:

Pattern annotation was integrated into the same annotation pass under the written guidelines \(Appendix[B](https://arxiv.org/html/2609.12884#A2)\)\. On a held\-aside subset annotated by multiple annotators, pattern agreement wasκ≈0\.59\\kappa\\approx 0\.59, with the largest residual confusions between A \(enumeration\) and C \(conclusion \+ premises\) for snippets resembling both\. The taxonomy and worked anchor cases were re\-circulated for calibration before the production pass\.

### A\.5Auto\-Snippet Pipeline Quality

We evaluate theMedSNIPpipeline against human\-annotatedMedSNIP\-Benchacross all 276 entries\.Chunking fidelitymeasures agreement with human snippet boundaries, whilelabel robustnessmeasures sensitivity to the label\-projection policy for auto\-snippets\.

##### Chunking fidelity:

Against human snippet boundaries,Mode 1\(atom\-to\-snippet\) achieves a sentence\-set F1 of0\.7560\.756, compared with0\.7340\.734forMode 2\(snippet\-direct\)\. Mean embedding cosine similarity shows the same pattern at0\.8240\.824and0\.7940\.794, respectively\.

Mode 1therefore more closely reproduces human snippet structure, consistent with its two\-pass design that preserves atom\-level boundaries as shown in Table[7](https://arxiv.org/html/2609.12884#A1.T7)\.

Modesent F1sent Psent Rembed cosMode 10\.7560\.7560\.7230\.7230\.8980\.8980\.8240\.824Mode 20\.7340\.7340\.7440\.7440\.7930\.7930\.7940\.794Table 7:Auto\-snippet pipeline chunking fidelity against the human gold structure on all 276 entries\.
##### Snippet count:

The pipeline under\-segments relative to humans\. Auto snippets total1,5781\{,\}578inMode 1and2,1322\{,\}132inMode 2, compared with2,5242\{,\}524human gold snippets, giving ratios of0\.620\.62and0\.840\.84\. Consumer entries are reproduced well, with sentence F1 above0\.850\.85in both modes, while vignettes are harder at≈0\.65\\approx 0\.65in both modes\. This is consistent with the harder clinical\-reasoning content noted in the subset\-selection analysis above\.

##### Label projection sensitivity\.

Auto\-snippets inherit labels projected from their source atoms\. We compare three policies\.ANDrequires all covered atoms to be true, with mixed\-label sentences treated as false\.Majorityuses the majority label, with ties resolved as true\.Drop\-mixedremoves auto\-snippets spanning conflicting human labels and is used throughout the main body\. AND and majority serve as robustness checks and agree in sign with drop\-mixed in eleven of the twelve verifier\-by\-mode cells\. Projection effects are small relative to model\-level differences\. Drop\-mixed retains1,3821\{,\}382snippets inMode 1and1,7531\{,\}753inMode 2, with dropped cases reflecting mixed\-label coverage rather than pipeline failures\.

## Appendix BAnnotation Guidelines

The guidelines below were distributed to annotators with only operational and identifying details removed\. The pattern taxonomy and worked anchor examples are reproduced unchanged\.

### B\.1Background

The dataset contains 5,755 atomic claims from 276 medical QA entries\. Annotators group these atoms into snippets and assign dual labels\. Each entry shows:

- •Query: the medical question\.
- •Full Response: the LLM\-generated answer\.
- •Atomic Claims: statements extracted from the response with expert true or false labels\.

##### Why we merge at all:

Atomic\-claim extraction can be too fine\-grained\. An atom such asOption B is the correct answer”orThe factors include the individual’s age”can be unverifiable without surrounding reasoning\. Medical responses, especially clinical vignettes, often connect claims throughbecause / since / therefore / requires\. Splitting these chains removes context needed for verification\.

CodeExampleDecision and RationalePatterns that trigger MERGEPattern AEnumeration of properties of one subject\.Multiple features, factors, or properties describe the same subject\. The enumeration is the verifiable unit\.A\.1Consumer query on safe Advil doseMERGE\.Atoms 2–4 begin with “The factors include…” and are meaningless without Atom 1\.Atom 1“The safe amount of Advil \(ibuprofen\) to take at one time depends on several factors\.”Atom 2“The factors include the individual’s age\.”Atom 3“The factors include the individual’s weight\.”Atom 4“The factors include the individual’s overall health\.”A\.2Vignette on a 72\-year\-old man with suspected leukemiaMERGE\.Three clinical features that together form the “CLL picture\.”Atom 10“Chronic Lymphocytic Leukemia \(CLL\) typically presents with a high leukocyte count\.”Atom 11“CLL typically presents with anemia\.”Atom 12“CLL typically presents with thrombocytopenia\.”Pattern BCausal or conditional chain\.Atoms are connected bybecause,since,as,due to,requires, ortherefore, or one atom serves as the premise for another\.B\.1Vignette on a 23\-year\-old with treatment\-resistant schizophreniaMERGE\.Atom 7 states the condition; Atoms 8 and 9 specify the two required failures\.Atom 7“Clozapine is generally reserved for patients who have failed to respond to at least two previous adequate trials of antipsychotic medications\.”Atom 8“Patients must have failed at least one conventional antipsychotic\.”Atom 9“Patients must have failed at least one second\-generation antipsychotic\.”B\.2Consumer query on Hepatitis A IgMMERGE\.Atom 8 is the reason for Atom 7\.Atom 7“A non\-reactive result does not necessarily rule out a past infection\.”Atom 8“IgM antibodies may not be detectable after a certain period\.”Pattern CConclusion \+ supporting premises\.A recommendation, diagnosis, or final answer is meaningful only when read with the reasoning or referent that supports it\.C\.1Same schizophrenia vignette, end of responseMERGE\.Atom 20 is unverifiable without Atom 19\.Atom 19“The most appropriate next step in management is to initiate clozapine\.”Atom 20“Option B is the correct answer\.”C\.2Same vignette, earlier in responseMERGE\.Atoms 1–2 give the differential; Atom 3 is the conclusion drawn from it\.Atom 1“The patient’s aggressive behavior is likely secondary to schizophrenia\.”Atom 2“The patient’s aggressive behavior could be due to a psychotic disorder\.”Atom 3“The most appropriate next step in management is to consider an alternative antipsychotic medication\.”Table 8:Structural patterns that trigger merging\.Patterns A–C capture cases in which atomic decomposition removes context required for verification\. The worked examples were used as inter\-annotator anchors\.CodeExampleDecision and RationalePatterns that justify KEEPING ATOMICPattern DComplete standalone fact, definition, or lab interpretation\.The atom is a self\-contained medical statement and requires no surrounding text for verification\.D\.1Hepatitis A IgM definitionATOMIC\.A complete definition; nothing else is needed\.Atom 1“A non\-reactive result for the Hepatitis A IgM \(Immunoglobulin M\) antibody test indicates that the individual does not have a recent or current Hepatitis A infection\.”D\.2Lab value interpretationATOMIC\.A single lab\-value claim verifiable in isolation\.Atom 1“A platelet count of 119,000/mm3is within normal limits\.”This claim is false \(normal range: 150,000–400,000\), but factuality is a labeling question rather than a grouping decision\.Pattern ETopic genuinely shifts\.The response moves from one subject to an unrelated one\.EExampleATOMIC\.Split at the topic boundary\.•The response may shift fromdrug composition to overdose risk, or from one differential diagnosis to a different one\.•The claims concern substantively different subjects and do not depend on one another for verification\.Pattern FSame topic, but each atom carries a different checkable fact\.Nearby atoms concern the same subject but each expresses an independently verifiable fact\.FExampleATOMIC\.Keep independently verifiable facts separate\.•Two claims about the same drug can remain separate if they convey distinct information\.•A standalone fact does not need to be merged into a nearby enumeration merely because it appears next to one\.•The “isolated \[false\] atom” idiom also falls under Pattern F: a false atom is kept separate so that it does not contaminate adjacent true claims\.Table 9:Structural patterns that justify keeping atoms separate\.Patterns D–F preserve claims that remain independently verifiable despite appearing near related content\.We therefore merge atoms when separating them would break a logical dependency\. Every rule below and each consensus judgment follows the same question:

> “If I read only this atom \(or only this snippet\), do I have enough information to check whether it is medically correct?”

Ifyes, keep it atomic\. Ifno, merge it with the atoms that provide the missing context\.

### B\.2Task

For each entry, the annotator defines shared context as key\-value pairs describing patient demographics, medications, or topic, then groups related atoms into snippets using an editable LLM draft\. Each snippet receives two labels,*\(i\)\. with context*, indicating correctness for the patient or query, and*\(ii\)\. in general*, indicating medical correctness\. The annotator assigns a structural pattern \(A–F\), marks ambiguity when correctness cannot be determined, and adds notes when labels differ\.

### B\.3Shared Context

Shared context is defined once per every entry and applies to all snippets, capturing information from the original query that may be needed to interpret and verify individual statements\. For clinical vignettes, typical keys includeage,sex,chief\_complaint,medical\_history,medications,vitals,treatment\_history, andcorrect\_answer\. For consumer\-health queries, where less patient\-specific information is required, typical keys includetopic,drug\_a, anddrug\_b\.

### B\.4Grouping Rules

Tables[8](https://arxiv.org/html/2609.12884#A2.T8)and[9](https://arxiv.org/html/2609.12884#A2.T9)give the six structural patterns used to operationalize this decision, together with the worked examples used as inter\-annotator anchors\. Default to atomic\. It is cheaper and easier to merge during consensus review than to split a bad merge\. If Pattern A, B, or C clearly fires, merge without hesitation\. After the merge / keep\-atomic decision, the annotator records thedominant pattern code\(A–F\) on the snippet\. The code captures the structural reason for the chunking decision and enables per\-pattern downstream analyses\.

##### Hybrid snippets:

A snippet may carry an optional secondary code when two patterns clearly apply\. For example, a snippet describing a drug, its side effect, and the resulting need for monitoring is primarily a causal chain \(B\), while the monitoring atom also functions as a conclusion supported by the side\-effect premise \(C\)\.

This is recorded as primary B and secondary C\. The secondary code is reserved for genuinely hybrid cases rather than routine close calls\. When the choice is borderline, when a secondary code is set, or when none of the six codes fit cleanly, the annotator adds a pattern note \(1–2 sentences\) explaining the rationale\. Genuine non\-fits \(truly outside A–F\) are flagged for lead\-researcher adjudication rather than forced into an ill\-fitting code\.

### B\.5Snippet Text

The snippet text should be a self\-contained, verifiable statement that can be judged without reading the original query\. An LLM\-generated draft is provided for the annotator to review and edit\. The annotator resolves pronouns using the shared context, replaces references such as the patient” with a 23\-year\-old male,” keeps standalone facts unchanged, and does not introduce medical information absent from the source atoms\.

![Refer to caption](https://arxiv.org/html/2609.12884v1/figs/dashboard.png)Figure 6:Annotation dashboard\.The left panel shows atomic claims with expert labels, the query, and the full LLM response\. The right panel shows shared context and per\-snippet cards containing atom selection, merged text, dual labels, ambiguity status, notes, and pattern code\. All annotator\-identifying elements have been removed\.
### B\.6Dual Labels

Each snippet receives two labels\. Here, context” refers to information in the user query and earlier statements in the same response, excluding external medical knowledge and other entries\. Thein\-generallabel indicates whether the snippet is correct as a general medical statement\. For example, Initiation of clozapine requires a 2\-week trial” may be labeledTRUEin general if a short initial trial is medically reasonable\. Thewith\-contextlabel indicates whether the same snippet is correct for the specific patient or query\. In a treatment\-resistant schizophrenia vignette, the same statement may be labeledFALSEwith context if the patient requires a longer trial\. When the two labels differ, the statement is generally correct but contextually wrong, or vice versa, and the annotator adds a note explaining the difference\. Labels areTruefor medically or factually correct statements,Falsefor factual errors, andAmbiguouswhen correctness cannot be determined\. Ambiguous labels also require a note\.

### B\.7Annotation Dashboard

Annotators use a browser\-based dashboard implementing the workflow in Appendix[B\.2](https://arxiv.org/html/2609.12884#A2.SS2)\. As shown in Figure[6](https://arxiv.org/html/2609.12884#A2.F6), the interface presents the query, response, and atomic claims with expert labels, alongside controls for grouping atoms into snippets, editing snippet text, assigning dual labels and pattern codes, and recording ambiguity or notes\.

The dashboard ensures that all atoms are assigned before an entry is completed and exports annotations as JSON\.

## Appendix CDecomposition and Verification Prompts

The prompts are built from reusable guideline, schema, formatting, granularity, and task\-instruction components\. We describe each component once and provide a composition map linking production prompts to their components\.

### C\.1Single\-Call Baseline Verifier

The baseline verifier \(Section[4\.2](https://arxiv.org/html/2609.12884#S4.SS2)\) takes one unit and returns a binary verdict in a single LLM call\. Atom and snippet baselines share these prompts; the unit text field isclaimfor atoms andsnippet\_textfor snippets\. Figures[7](https://arxiv.org/html/2609.12884#A3.F7)and[8](https://arxiv.org/html/2609.12884#A3.F8)show both prompts verbatim\.

### C\.2MedSNIP Pipeline: Prompt Composition

The pipeline \(Section[4\.1](https://arxiv.org/html/2609.12884#S4.SS1)\) uses three system prompts: Mode 1 extract, Mode 1 cluster, and Mode 2 direct\. Each prompt is assembled from the reusable blocks in Table[10](https://arxiv.org/html/2609.12884#A3.T10)\. Consumer and vignette variants use the same prompt skeleton, with subset\-specific schemas, text rules, and granularity rules\. The five reusable blocks are shown in Figures[9](https://arxiv.org/html/2609.12884#A3.F9)–[13](https://arxiv.org/html/2609.12884#A3.F13)\.

Mode 1Mode 2ComponentExtractClusterDirectMerge patterns–✓✓Atom extraction✓––Granularity rules✓––Context schema✓✓✓Snippet text rules–✓✓Output JSONextractclusterdirectTable 10:Composition of reusable prompt components\.Mode 1 separates atom extraction from clustering, whereas Mode 2 constructs snippets directly\. Checkmarks indicate the components included at each stage\.Claim\-OnlybaselineSystem Prompt:
You are a cautious medical fact\-checking assistant\. Given a claim, decide if the claim is true\. Respond with exactly one word:"true"if the claim is fully supported, otherwise"false"\. No other words\.User Message:```
Claim:
{snippet_text}
```

Figure 7:Baseline verifier inClaim\-Onlymode\.Full\-ContextbaselineSystem Prompt:
You are a cautious medical fact\-checking assistant\. You will be given a user question, a full answer text, and a single extracted claim \(decomposed from the full answer text\)\. Use the full answer text ONLY to decontextualize the claim\. Then decide whether the interpreted claim is medically correct in the real world\. Do NOT treat the full answer text as evidence the claim is true; it is context for interpretation\. Respond with exactly one word:"true"or"false"\. No other words\.User Message:```
Question:
{query}

Full answer text (interpretation only):
{full_text}

Extracted claim:
{snippet_text}
```

Figure 8:Baseline verifier inFull\-Contextmode\. The full answer is for disambiguation only\.Block: Merge Patterns A–FYou mirror a medical\-text annotator\. Group an LLM\-generated medical response intosnippets, coherent, self\-contained, verifiable units\. Guiding question for every claim:If this stood alone, could a reader verify it medically?If no→\\tomerge\.Three patterns that trigger MERGE<Pattern A\-C from Section 3\>Three patterns that justify KEEPING ATOMIC<Pattern D\-F from Section 3\>Default\.When torn, keep atomic\.Coverage: every unit appears in exactly one snippet\.Figure 9:Shared annotation guidelines, used by Mode 1 cluster and Mode 2\.Block: Shared\-context schemasConsumer Keys:topic, drug\_a, drug\_b, condition, symptom\. Omit absent keys; values are short and concrete\.Vignette Keys:age, sex, chief\_complaint, medical\_history, medications, vitals, exam\_findings, treatment\_history, correct\_answer\. Omit absent keys\.Figure 10:Per\-subsetshared\_contextdict shapes\. Plugged into every pipeline prompt\.Block: Snippet text rulesConsumer\.Each snippet’soutputis a self\-contained verifiable statement\. A reader should judge correctness without the query\. Do NOT introduce facts absent from the source\.Vignette\.Each snippet’soutputmust be self\-contained\. Inline relevant patient demographics fromshared\_contextso a reviewer can judge it standalone \(e\.g\., “this patient”→\\to“this 55\-year\-old male with right arm weakness”\)\. Do NOT introduce facts absent from the source\.Figure 11:Per\-subset snippet wording rules\. Used by Mode 1 cluster and Mode 2\.Block: Granularity rulesConsumer \(fine\)\.“X can cause drowsiness, dizziness, and confusion”→\\tothree atoms, one per side effect\. “Drug X is an antihistamine containing compound Z”→\\toone atom\. Single\-property sentences→\\toone atom\.Vignette \(coarse\)\.One atom per sentence by default\. Causal chains and multi\-property sentences stay as one atom\. The final\-answer sentence is its own atom\. Each differential\-option sentence becomes one atom\.Figure 12:Per\-subset atom granularity rules for consumer\-health and clinical\-vignette entries\. Used by Mode1 step1 to guide atomic claim extraction\.Block: Atom ExtractionYou are an atomic\-claim extractor\. Given a user query and an LLM response pre\-segmented into numbered sentences, produce ashared\_contextdict plus a list of atomic claims that follow the sentence structure\.Atom definition\.Each atom states ONE medically\-meaningful proposition, light\-normalized from its sentence\(s\)\. Resolve pronouns\.Preserve the response’s stance— do NOT correct claims that look wrong\.Sentence\-atom mapping\.One sentence→\\toone atom by default; one sentence→\\tomultiple atoms only for clear comma\-separated enumerations; two consecutive sentences→\\toone atom only for indivisible claims\.Coverage\.Every sentence index must appear in at least one atom’ssource\_sentences\.Figure 13:Atom\-definition and sentence\-mapping rules for extracting self\-contained atomic claims\. Used by Mode 1 step 1 \(extract\)\.#### Task Heads and Output Schemas

Each prompt ends with a mode\-specific task and JSON schema \(Figures[14](https://arxiv.org/html/2609.12884#A3.F14)–[16](https://arxiv.org/html/2609.12884#A3.F16)\)\.

Task head: Mode 1 step 1 \(extract\)Task\.Use the granularity rule for the subset; emitshared\_contextper the schema; cover every sentence index\.Output \(JSON only\):```
{
    <JSON schema>
}
```

Figure 14:Mode 1 step 1 task and output schema for atomic claim extraction\. Combines with the extraction\-common block, granularity rules, and the per\-subset context schema\.Task head: Mode 1 step 2 \(cluster\)Given the query, the pre\-extractedshared\_context, and a numbered list ofatomic\_claims, group atoms into snippets per Patterns A–F\. Every atom index appears in exactly one snippet\. Write each snippet’soutputper the text rules; tag the dominant pattern \(A–F\) and a shortnotesrationale\.Output \(JSON only\):```
{ <JSON schema> }
```

Figure 15:Mode 1 step 2 task and output\. Combines with the merge\-patterns block, the per\-subset context schema, and snippet text rules\.Task head: Mode 2 \(snippet\-direct\)Given the query and numbered sentences, in one pass: \(1\) extractshared\_context; \(2\) identify atomic claims and group them into snippets per Patterns A–F \(every sentence index appears in at least one snippet\); \(3\) writeoutputper the text rules; \(4\) tag the dominant pattern\.Sentence\-boundary default\.The default is one sentence→\\toone snippet\. Merge only when an A/B/C cue is explicit across sentences; if not, split\. Topical similarity alone is NOT enough\. A common failure mode is collapsing the whole response into one snippet — do not do that\. ForNN\-sentence responses, the output usually has0\.6​N0\.6NtoNNsnippets\.Output \(JSON only\):```
{
    <JSON schema>
}
```

Figure 16:Mode 2 task and output\. Combines with the merge\-patterns block, the per\-subset context schema, and snippet text rules\.

### C\.3Retrieval\-Augmented Verifier

The retrieval verifier \(Section[4\.3](https://arxiv.org/html/2609.12884#S4.SS3)\) follows the prompt sequence in Figures[17](https://arxiv.org/html/2609.12884#A3.F17)–[20](https://arxiv.org/html/2609.12884#A3.F20)during iterative verification\. If no verdict is reached, the force\-final fallback in Figure[20](https://arxiv.org/html/2609.12884#A3.F20)is used\.

Verifier: Answer, Abstain or SearchYou are a rigorous medical fact\-checker\. You receive a snippet, theshared\_context, and any evidence gathered so far\. Your job is tocatch wrong claims— false\-negative detection is the critical metric\.Confidence:0\.90–1\.00 direct specific evidence; 0\.70–0\.89 strong with minor uncertainty; 0\.50–0\.69 leaning but not confident;<<0\.50 search more\. Be calibrated, not optimistic\.Output \(JSON only\):```
{
  <JSON schema for Final Answer,
  Abstain and Search>
}
```

Figure 17:Verifier framing and the per\-iteration JSON options with calibrated confidence bands\.Verifier: Must\-Search, RefutationMust\-search triggers\.Do not short\-circuit on parametric knowledge when the snippet has specific numbers \(doses, ranges, percentages, durations, thresholds\), equivalence claims between drugs/conditions/mechanisms, vignette\-style clinical reasoning, recent guidelines, or any uncertainty on a named drug/dose/condition\.Refutation\-biased phrasing\.Look for evidence the snippet is*wrong*, not for confirmation\.Source\.webfor consumer info, brand\-name OTC, recent guidelines, recency\-sensitive content\.pubmedfor primary clinical literature, RCT efficacy, mechanism, rare conditions; use keyword phrases, not full questions\.Figure 18:Verifier search behavior: when to issue a search, how to phrase it, and which retriever to call\.Verifier: General Truth FramingGeneral\-truth framing\.Judge the snippet as a*general*medical statement, NOT vignette applicability\.Compound\-claim rule\.A multi\-sub\-claim snippet is true only if every*material*sub\-claim is true\. Material errors include the wrong drug class, effect direction, mechanism, or target population\. Ignore acceptable rounding, rare\-exception generalizations, hedging, minor phrasing or mechanism imprecision, overlapping\-category framing, and comparative edge cases when the core assertion holds\.Mixed evidence\.If evidence neither clearly supports nor refutes a specific factual claim with named entities, numbers, or mechanism→\\tofalse\. Generic broadly\-true statements with no specific content→\\totrueif nothing specific is wrong\.Figure 19:Verifier truth\-judgement rules for general medical statements, including compound\-claim materiality and handling of mixed or inconclusive evidence\.Verifier: Force\-final FallbackYou are a cautious medical fact\-checker making a final decision; no more searches are allowed\.Prefer abstain over guessed False\.If the evidence does not directly refute a material claim, returnabstain\. If the main assertion is defensible and only peripheral sub\-claims are doubtful, returntrueunder the compound\-claim rule\.Only emitfinal\_answer: falsewith concrete refuting evidence, such as a wrong entity, out\-of\-range number, inverted mechanism, or misassigned category\.Figure 20:Force\-final fallback prompt used at the iteration cap when no verdict is reached\. Biased toward abstention under residual uncertainty\.

## Appendix DExperimental Details

This appendix documents the experimental setup, statistical methodology, full per\-cell numbers, and sensitivity analyses behind the main\-body results\.

### D\.1Compute and API

##### Closed\-source models:

GPT\-5\.4\(with reasoning effort high\) andGPT\-4oare accessed through the OpenAI API\.GPT\-5\.4is the canonical decomposer across atom and snippet conditions to keep the comparison free of atomizer\-quality confounds\. Per\-experiment dollar costs for both closed\-source models are in Table[11](https://arxiv.org/html/2609.12884#A4.T11)\.

##### Open\-weight models:

Gemma\-4\-31B\-it,GPT\-OSS\-20B,Llama\-3\.3\-70B, andLlama\-3\.1\-8Bare accessed through the HF Inference router with the OpenAI\-compatible chat\-completions endpoint, so the same prompt and parsing pipeline applies across all six models\. End\-to\-end dollar costs per cell are reported in Table[11](https://arxiv.org/html/2609.12884#A4.T11)\.

##### Retrieval backends:

The retrieval\-augmented verifier uses Google Serper API \($0\.001 per query\) and a public PubMed biomedical\-literature API \(free, abstracts only\)\. Both returnk=3k=3snippets per query\. Each verified item issues between one and five retrieval rounds depending on confidence\-gated stopping, for an average of≈1\.9\\approx 1\.9retrievals per item\.

##### Random seeds:

All bootstrap analyses use a single fixed seed of 42\. The train/dev/test partition was selected as the random\-seed sweep winner of 202\. Stochastic LLM calls use the API default sampling temperature\.

### D\.2Bootstrap CI Methodology

Every confidence interval for anF1F\\text\{F\}\_\{1\}^\{F\}difference is a 95% bootstrap interval overB=10,000B=10\{,\}000replicates, with significance declared when the interval excludes zero\. All resampling is paired, so each replicate recomputes both arms on the same draw andΔ\\Deltais a paired difference\. Two interval types are used depending on the resampling unit\. For flat samples, as in the per\-pattern comparisons of Table[4](https://arxiv.org/html/2609.12884#S5.T4), we use bias\-corrected and accelerated \(BCa\) intervals\. The bootstrap distribution ofΔ\\Deltacan be skewed when either F1 approaches zero, as in small pattern cells or under weak verifiers, and BCa corrects for bias and skewness\.

When replicates resample clusters, we instead use percentile intervals\. For Table[2](https://arxiv.org/html/2609.12884#S5.T2)and Table[14](https://arxiv.org/html/2609.12884#A4.T14), we resample whole source entries onMedSNIP\-Benchand whole claims on the external corpora\. This accounts for dependence among snippets from the same answer and for the different numbers of units produced by the two arms\. Because the resulting statistic does not have the flat structure required for the BCa jackknife correction, we report percentile intervals\. The paired clustering also tends to narrow the intervals, since errors shared across the two arms cancel within each draw\. The bold cells of Table[2](https://arxiv.org/html/2609.12884#S5.T2)are the 17 of 42 whose interval excludes zero, 7 of 18 onMedSNIP\-Bench, 2 of 12 onHealthFC, and 8 of 12 onMedHallu\.

### D\.3Full Per\-Cell Results

Table[11](https://arxiv.org/html/2609.12884#A4.T11)provides the per\-splitMedSNIP\-Benchresults underlying the aggregate numbers in Table[2](https://arxiv.org/html/2609.12884#S5.T2)\. It reports atom and snippet verifier\-call counts, false\-class F1, and dollar cost for each evaluated split\. Figure[21](https://arxiv.org/html/2609.12884#A4.F21)visualizes the corresponding cost–F1F\\text\{F\}\_\{1\}^\{F\}trade\-off, showing how snippet\-level verification shifts the baselines toward fewer verifier calls

ModelSplitAtom callsSnippet callsAtomF1F\\text\{F\}\_\{1\}^\{F\}SnippetF1F\\text\{F\}\_\{1\}^\{F\}Δ\\DeltaAtom costSnippet costMedSNIP\-BenchClaim\-OnlyVerificationGPT\-5\.4 \(high\)Train3,7351,5990\.3020\.433\+0\.131$10\.27$7\.13Dev1,0784940\.3560\.407\+0\.051$2\.96$2\.20Test9424310\.3510\.451\+0\.100$2\.59$1\.92GPT\-4oTrain3,7351,5990\.3280\.416\+0\.088$1\.70$0\.80Dev1,0784940\.3610\.395\+0\.034$0\.49$0\.24Test9424310\.3820\.354−\-0\.028$0\.43$0\.22Gemma\-4\-31B\-itTrain3,7351,5990\.3240\.394\+0\.070$0\.05$0\.02Dev1,0784940\.3510\.395\+0\.045$0\.01<$0\.01<\\$0\.01Test9424310\.3830\.358−\-0\.025$0\.01<$0\.01<\\$0\.01GPT\-OSS\-20BTrain3,7351,5990\.3230\.400\+0\.077$0\.10$0\.06Dev1,0784940\.3550\.398\+0\.043$0\.03$0\.02Test9424310\.4060\.408\+0\.002$0\.02$0\.02Llama\-3\.3\-70BTrain3,7351,5990\.2990\.322\+0\.023$0\.05$0\.03Dev1,0784940\.3260\.369\+0\.043$0\.02<$0\.01<\\$0\.01Test9424310\.3680\.314−\-0\.054$0\.01<$0\.01<\\$0\.01Llama\-3\.1\-8BTrain3,7351,5990\.2160\.287\+0\.071<$0\.01<\\$0\.01<$0\.01<\\$0\.01Dev1,0784940\.2390\.269\+0\.029<$0\.01<\\$0\.01<$0\.01<\\$0\.01Test9424310\.3160\.255−\-0\.061<$0\.01<\\$0\.01<$0\.01<\\$0\.01MedSNIP\-BenchFull\-ContextVerificationGPT\-5\.4\-highTrain3,7351,5990\.4750\.475−\-0\.000$17\.67$11\.27Dev1,0784940\.4850\.518\+0\.034$5\.10$3\.48Test9424310\.4910\.491−\-0\.000$4\.46$3\.04GPT\-4oTrain3,7351,5990\.3380\.301−\-0\.037$7\.12$3\.25Dev1,0784940\.3210\.227−\-0\.094$1\.97$0\.96Test9424310\.2220\.135−\-0\.087$1\.79$0\.86Table 11:Per\-split verification results onMedSNIP\-Bench\.Atom and snippet calls give verifier\-call counts, with corresponding false\-classF1F\\text\{F\}\_\{1\}^\{F\}\.Δ\\Deltais the snippet–atomF1F\\text\{F\}\_\{1\}^\{F\}difference, with gains and losses shown in green and red\. Cost estimates one complete verification sweep\.Figure 21:Verifier calls versusF1F\\text\{F\}\_\{1\}^\{F\}for snippet\- and atom\-level baselines across three datasets\. Snippets require fewer calls while preserving or improvingF1F\\text\{F\}\_\{1\}^\{F\}\.
### D\.4Aggregation Rule Sensitivity

For datasets with answer\-level labels, unit\-level verdicts must be combined into a single prediction\. We test whether this choice affects the snippet–atom comparison using five aggregation rules\.Or\-Falsepredictsfalseif any unit is false, following theFActScoredefault\.And\-Falserequires all units to be false,Majorityuses majority vote,Threshold\-kkpredictsfalsewhen at leastk=2k=2units are false, and a fifth rule takes the first unit alone\.

Across 14 dataset–granularity–mode–model settings, we recompute the snippet–atomF1F\\text\{F\}\_\{1\}^\{F\}gap and its 95% paired bootstrap interval under each rule\.

##### Findings:

The magnitude of the snippet–atom gap varies with the aggregation rule, but the advantage of snippets holds in most settings\. UnderThreshold\-kk, snippets outperform atoms in 12 of 14 settings withP⁡\(Δ\>0\)≥0\.95P\(\\Delta\>0\)\\geq 0\.95\. The two exceptions occur onMedSNIP\-BenchwithGPT\-4oinFull\-Contextmode, where predictions are nearly degenerate\. UnderOr\-False, snippets win in 7 of 14 settings\. The main exception isAnd\-False, which reverses the comparison onMedHalluby0\.0150\.015F1F\\text\{F\}\_\{1\}^\{F\}in favor of atoms \(P=0\.036P=0\.036\)\. This rule favors atomization because it requires every unit to befalse, and atomization produces more units\.

##### Implication:

Aggregation is therefore not a neutral implementation choice\. It can change both the magnitude and direction of a granularity comparison, so decomposition\-based evaluations should state the aggregation rule explicitly\. The cost reduction reported in the main paper is unaffected because it depends only on the number of verification units\.

### D\.5End\-to\-End Cost

The cost analysis in Section[5\.2](https://arxiv.org/html/2609.12884#S5.SS2)considers verification alone, but decomposition also adds overhead\. An atom pipeline requires one decomposition call per answer, while atomize\-then\-group requires a second for grouping\. Table[12](https://arxiv.org/html/2609.12884#A4.T12)includes this cost forMedSNIP\-BenchwithGPT\-5\.4as the verifier\. WithGPT\-5\.4\-high, decomposition costs $23\.63 and makes the full pipeline 19\.1% more expensive\. WithGPT\-OSS\-20B, it costs only $0\.20, preserving nearly all of the 35\.3% verification saving\. Snippet\-direct further reduces this overhead by requiring only one decomposition call\.

Call counts show the same pattern without relying on model prices\. Atomize\-then\-group reduces total calls by 64\.7% onMedSNIP\-Benchbut increases them by 14\.5% onMedHalluand 27\.1% onHealthFC\. Snippet\-direct instead yields reductions of 60\.1%, 20\.9%, and 17\.3%, respectively\. Decomposition overhead therefore matters most for short answers, where fewer verification calls can be eliminated, consistent with Section[5\.3](https://arxiv.org/html/2609.12884#S5.SS3)\.

DecomposerDecomp\.costVerificationreductionEnd\-to\-endreductionGPT\-5\.4\-high$23\.63\+54\.3%−\-19\.1%GPT\-4o$5\.79\+33\.3%\+8\.8%Gemma\-4\-31B\-it$0\.29\+22\.6%\+21\.3%Llama\-3\.3\-70B$0\.21\+26\.0%\+24\.8%GPT\-OSS\-20B$0\.20\+35\.3%\+34\.1%Llama\-3\.1\-8B$0\.03\+19\.1%\+18\.9%Table 12:End\-to\-end cost reduction from snippet\-level verification across different decomposers, comparing verification cost savings before and after accounting for decomposition overhead\.
### D\.6Wording Against Grouping

Snippet\-level gains can come from grouping dependencies or making snippets easier to verify\. Patterns D, E, and F provide a control because they pair one atom with one snippet, so gains must come from wording\. To separate these effects, we rewrite each atom using decontextualization rules without grouping, then compare the raw atom, rewritten atom, and snippet\.

PatternDecisionRawRewriteSnippetGainAMerge0\.2670\.2920\.38721%BMerge0\.2450\.2790\.38325%CMerge0\.3020\.3540\.42642%DKeep0\.2500\.3360\.36475%FKeep0\.5430\.6240\.621103%A–CMerge0\.2730\.3060\.39926%D–FKeep0\.3650\.4490\.45494%Table 13:Effect of decontextualization by structural pattern\.Raw, rewritten, and snippet columns reportF1F\\text\{F\}\_\{1\}^\{F\}\. Wording gain is the percentage of the raw\-to\-snippet improvement recovered by rewriting alone\.Rewriting explains 94% of the gain for keep\-atomic patterns but only 26% for merge patterns, leaving the remaining 74% attributable to grouping related atoms\. Pattern B shows the clearest effect, with wording explaining only 25% of its gain, consistent with its causal and conditional structure where claims depend on one another\. Pattern E has only 39 units and negative deltas throughout, so we do not interpret it further\.

Mode 1: Atomize\-then\-GroupDecomposer↓\\downarrowVerifier→\\rightarrowGPT\-5\.4GPT\-4oGemma\-4\-31B\-itGPT\-OSS\-20BLlama\-3\.3\-70BLlama\-3\.1\-8BGPT\-5\.4\-high\+0\.114−\-0\.038\+0\.014−\-0\.009−\-0\.077−\-0\.035GPT\-4o\+0\.084−\-0\.038−\-0\.005\+0\.024−\-0\.045\+0\.015Gemma\-4\-31B\-it\+0\.055−\-0\.002−\-0\.025−\-0\.015−\-0\.044−\-0\.004GPT\-OSS\-20B\+0\.066−\-0\.036−\-0\.026−\-0\.001−\-0\.068−\-0\.044Llama\-3\.3\-70B\+0\.051−\-0\.044−\-0\.021−\-0\.013−\-0\.063−\-0\.032Llama\-3\.1\-8B\+0\.032−\-0\.008\+0\.017−\-0\.018−\-0\.004\+0\.036
Mode 2: Snippet\-DirectDecomposer↓\\downarrowVerifier→\\rightarrowGPT\-5\.4GPT\-4oGemma\-4\-31B\-itGPT\-OSS\-20BLlama\-3\.3\-70BLlama\-3\.1\-8BGPT\-5\.4\-high\+0\.099−\-0\.027\+0\.038\+0\.022−\-0\.037−\-0\.025GPT\-4o\+0\.092−\-0\.032−\-0\.005\+0\.033−\-0\.029\+0\.031Gemma\-4\-31B\-it\+0\.061\+0\.005−\-0\.000−\-0\.002−\-0\.023\+0\.035GPT\-OSS\-20B\+0\.055\+0\.005\+0\.002−\-0\.025−\-0\.033\+0\.030Llama\-3\.3\-70B\+0\.095−\-0\.050\+0\.001\+0\.001−\-0\.059\+0\.037Llama\-3\.1\-8B\+0\.045−\-0\.018\+0\.012\+0\.025−\-0\.018\+0\.011

Table 14:Snippet–atomF1F\\text\{F\}\_\{1\}^\{F\}gap across decomposer–verifier pairs onMedSNIP\-Bench\.Rows denote decomposers and columns denote verifiers\. Green cells indicate gains and red cells indicate losses\. Bold values have 95% entry\-clustered bootstrap confidence intervals excluding zero, based on 10,000 resamples\.There are two caveats\. For 382 of 4,868 merge atoms, the rewriter used information from the shared context to effectively reconstruct the merge\. We exclude these cases because they do not provide a valid wording\-only ablation\. In addition, repeating verification on identical text produces a baseline difference of0\.0130\.013F1F\\text\{F\}\_\{1\}^\{F\}, so smaller differences are not informative\.

### D\.7Full Decomposer\-by\-Verifier Matrix

Section[5\.5](https://arxiv.org/html/2609.12884#S5.SS5)reports a subset of the full decomposer\-by\-verifier experiment\. Table[14](https://arxiv.org/html/2609.12884#A4.T14)shows all six decomposers paired with all six verifiers, under both pipeline modes onMedSNIP\-Bench\. Each cell reports the difference between snippet and expert\-atomF1F\\text\{F\}\_\{1\}^\{F\}for the same verifier\. BecauseMedSNIP\-Benchinherits its atomic claims from[Kim et al\. \(2025\)](https://arxiv.org/html/2609.12884#bib.bib14), all decomposers are compared against the same human\-annotated atom baseline\. The results show that the verifier matters more than the decomposer\. All significant gains occur withGPT\-5\.4, while all significant losses occur withLlama\-3\.3\-70B\. No other verifier produces a significant difference\. WithGPT\-5\.4, the six decomposers yield gains from\+0\.032\+0\.032to\+0\.114\+0\.114, with all but one reaching significance\. Even the 31B and 8B open\-weight decomposers therefore produce snippets that outperform expert atoms\.

Across verifiers, however, this advantage disappears, suggesting that gains depend mainly on the verifier’s ability to use the recovered structure rather than the model used to recover it\. Mode 1 and Mode 2 perform similarly overall, supporting our decision to report both rather than select a default\.

## Appendix ERetrieval\-Augmented Verifier Development Checks

The main paper reports the final retrieval\-augmented verifier and its coverage–F1F\\text\{F\}\_\{1\}^\{F\}trade\-off\. During development, we also tested two variants intended to recover full coverage while preserving the benefits of confidence gating\. The first used the retrieval verifier when its confidence exceeded the threshold and otherwise fell back to the single\-call baseline\. The second routed examples by source subset, using the retrieval verifier for one subset and the single\-call baseline for the other\. Both variants improved performance on the development split but did not provide a stable or consistent improvement on the held\-out test split\. We therefore report the simpler confidence\-thresholded verifier in the main paper and treat these full\-coverage routing and ensembling strategies as future work rather than part of the core result\.