BioDivergence: A Benchmark and Evaluation Framework for Hidden Contextual Contradictions in Biomedical Abstracts

# BioDivergence: A Benchmark and Evaluation Framework for Hidden Contextual Contradictions in Biomedical Abstracts
Source: [https://arxiv.org/html/2606.11208](https://arxiv.org/html/2606.11208)
Elias Hossain1, Sanjeda Sara Jennifer1, Sabera Akter Bushra2, Niloofar Yousefi1 1College of Engineering and Computer Science, University of Central Florida 2Burnett School of Biomedical Sciences, University of Central Florida \{mdelias\.hossain, niloofar\.yousefi\}@ucf\.edu

###### Abstract

Biomedical findings often appear to disagree across studies, yet many of those apparent disagreements are not genuine contradictions: differences in cohort, geography, assay protocol, disease subtype, or clinical setting can leave both claims locally valid\. Existing natural\-language\-inference and scientific claim\-verification benchmarks test whether two statements entail, contradict, or are neutral, but none test whether a model can recover the*contextual structure*that explains such context\-conditioned divergence\. Collapsing this divergence into a flat contradiction label overstates disagreement in the literature and hides a capability that downstream biomedical NLP systems need\. We introduceBioDivergence, an evaluation framework built around a six\-class conflict taxonomy, a 13\-axis divergence ontology, and four primary structured targets per claim pair: conflict type, divergence axes, dominant confounder, and reconciliation explanation\. A scoped fifth output, claim\-level evidence\-span extraction, is included as release\-time calibration under the v1\.0 claim\-verbatim silver rule rather than as a grounding benchmark\. We release*BioDivergence\-Silver\-v1\.0*, an article\-disjoint silver benchmark of 11,865 claim pairs \(10,183 / 840 / 842\) across five biomedical domains, with zero article, claim, or pair overlap across splits; a legacy pair\-level\-deduplicated variant is retained only for backward comparability\. Fine\-tuned and zero\-shot baselines yield qualitatively different rankings on contextual\-contradiction F1 across the two variants: the fine\-tuned reference model drops≈\\approx12 points under the article\-disjoint primary relative to the legacy pair\-level packaging, and the full 842\-example primary\-test evaluation withMistral\-7B\-Instruct\-v0\.3reaches 0\.5523 accuracy and 0\.3894 contextual\-F1 without article\-level leakage\. v1\.0 reliably supports two conflict classes and nine of thirteen axes; the remaining schema slots are retained for ontological stability\. BioDivergence provides an evaluation lens that separates contextual divergence from direct contradiction and isolates article\-level memorisation from task learning\.

## 1Introduction

Biomedical findings often appear to disagree across studies, and some of those apparent disagreements reflect omitted contextual differences rather than genuine contradictionsSosa and Altman \([2022](https://arxiv.org/html/2606.11208#bib.bib8)\)\. Two abstracts can report incompatible conclusions while both claims remain*locally valid under different conditions*: cohort, surveillance period, assay protocol, disease subtype, or clinical setting\. Distinguishing such context\-conditioned divergence from flat incompatibility is not a stylistic concern\. Collapsing the two into a single “contradiction” label overstates disagreement in the literature, misrepresents the structure of scientific evidence, and hides a capability that downstream biomedical NLP systems need\.

#### Where prior benchmarks fall short\.

Modern NLI and claim\-verification benchmarks \(FEVERThorne et al\. \([2018](https://arxiv.org/html/2606.11208#bib.bib10)\), VitaminCSchuster et al\. \([2021](https://arxiv.org/html/2606.11208#bib.bib7)\), SciFactWadden et al\. \([2020](https://arxiv.org/html/2606.11208#bib.bib11)\)\) ask whether two statements entail, contradict, or are neutral, and whether an evidence passage supports a claim\. Recent knowledge\-conflict benchmarks \(ConflictBankSu et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib9)\), WikiContradictHou et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib4)\), HealthContradictZhang et al\. \([2025](https://arxiv.org/html/2606.11208#bib.bib15)\)\) extend this setting to how LLMs behave when sources disagree\. None evaluate whether a model can recover*why*two findings diverge when the disagreement is explained by hidden contextual differences between the underlying studies\.

*Motivating example\.*Claim A reports that a colistin MIC breakpoint of 2 mg/L correctly identifies susceptible*Acinetobacter baumannii*isolates in Greek ICUs; Claim B reports that the same breakpoint misclassifies 30% of colistin\-resistant*A\. baumannii*isolates in a Thai surveillance study\. A standard NLI system labels the pair a contradiction\. A biomedically useful evaluation should instead recover that the disagreement is explained by geography, strain composition, and assay usage, and reward a model that can name those axes rather than merely flip a three\-way label\.

#### What BioDivergence adds\.

We introduceBioDivergence, an evaluation framework for hidden contextual contradiction analysis in biomedical abstracts\. It casts contradiction analysis as a structured multi\-output problem: for each claim pair, a model must recover the conflict type \(six\-class taxonomy\), the divergence axes \(13\-axis ontology\), the dominant confounder, and a reconciliation explanation\. These four outputs are our*primary evaluation targets*\. A scoped fifth output, claim\-level evidence\-span extraction, is included in v1\.0 as release\-time calibration under the claim\-verbatim silver annotation rule rather than as a mature abstract\-grounding benchmark\. The construction pipeline processes 202,180 normalised abstracts, extracts 527,907 claims, and mines 50,000 candidate pairs across five biomedical domains\. The release enforces*article\-disjoint*splits with zero article, claim, or pair overlap across train, dev, and test, which shifts model rankings relative to the pair\-level packaging other contradiction benchmarks implicitly use \(§[7](https://arxiv.org/html/2606.11208#S7)\)\. Empirically, zero\-shot LLMs lead on coarse conflict classification and reconciliation generation, the full primary\-test non\-Qwen run provides an annotator\-family\-independent check on all 842 test examples, and lexical and retrieval baselines dominate v1\.0 evidence\-span extraction by construction of the claim\-verbatim rule\.

#### Contributions\.

Our contributions are fourfold\.First, we formalise*contextual contradiction analysis*, the task of recovering*why*two biomedical claims diverge rather than only whether they do, operationalising a gap left by flat NLI, claim\-verification, and knowledge\-conflict benchmarks\.Second, we introduce a six\-class conflict taxonomy \(two classes reliably populated in v1\.0, four retained for ontological stability\) and a 13\-axis divergence ontology \(nine primary, four provisional\), together with a four\-primary\-plus\-one\-scoped task specification covering conflict type, divergence axes, dominant confounder, and reconciliation explanation; claim\-level evidence\-span extraction is included as v1\.0 release\-time calibration under the claim\-verbatim rule\.Third, we release*BioDivergence\-Silver\-v1\.0*, an article\-disjoint silver benchmark accompanied by an explicit leakage audit and a non\-Qwen annotator\-family sensitivity analysis; a legacy pair\-level variant of the same labelled pool is retained only for backward comparability\.Finally, we show that article\-disjoint evaluation shifts model rankings on contextual contradiction, separating article memorisation from contextual\-reasoning learning in a way conventional pair\-level packaging cannot; the full 842\-example primary\-test Mistral evaluation further confirms that the primary split supports annotator\-family\-independent scoring without relying on a subsample, establishing BioDivergence as an evaluation lens rather than a single dataset\.

#### Why BioDivergence changes evaluation conclusions\.

Flat contradiction labels cannot distinguish direct incompatibility from context\-conditioned divergence, so a model that handles coarse contradiction well can still be wrong about*why*two findings disagree; BioDivergence’s divergence\-axis and dominant\-confounder supervision exposes that distinction\. Because a single biomedical abstract typically generates many claim pairs, pair\-level deduplication admits large article\-level overlap, and the resulting rankings shift under article\-disjoint evaluation\. BioDivergence therefore changes what one can honestly conclude about contextual reasoning capability from a benchmark score\.

## 2Related Work

Claim verification against textual evidence has been studied on FEVERThorne et al\. \([2018](https://arxiv.org/html/2606.11208#bib.bib10)\)and VITAMINCSchuster et al\. \([2021](https://arxiv.org/html/2606.11208#bib.bib7)\)for Wikipedia, and on SciFactWadden et al\. \([2020](https://arxiv.org/html/2606.11208#bib.bib11),[2022](https://arxiv.org/html/2606.11208#bib.bib12)\)and Evidence InferenceDeYoung et al\. \([2020](https://arxiv.org/html/2606.11208#bib.bib3)\)for scientific and clinical text;Sosa and Altman \([2022](https://arxiv.org/html/2606.11208#bib.bib8)\)highlight that missing context and contradictions limit biomedical knowledge\-graph inference\. Recent work on knowledge conflict in LLMs \(ClashEvalWu et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib14)\), ConflictBankSu et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib9)\), WikiContradictHou et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib4)\), HealthContradictZhang et al\. \([2025](https://arxiv.org/html/2606.11208#bib.bib15)\)\) shows that models struggle to represent or reconcile competing evidence\. Parallel lines address interactive clarification \(MEDIQLi et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib5)\)\) and interpretable clinical reasoning \(DiReCTWang et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib13)\)\), andRu et al\. \([2024](https://arxiv.org/html/2606.11208#bib.bib6)\); Bean et al\. \([2025](https://arxiv.org/html/2606.11208#bib.bib2)\); Anonymous \([2024](https://arxiv.org/html/2606.11208#bib.bib1)\)raise construct\-validity and metric concerns for RAG and factuality evaluation\.

#### How BioDivergence differs\.

Existing benchmarks either treat contradiction as a binary or ternary label \(SciFact, VitaminC, FEVER\), evaluate LLM behaviour under knowledge conflict with coarse conflict categories \(ConflictBank, WikiContradict, HealthContradict\), or focus on single\-document clinical reasoning \(Evidence Inference, DiReCT\)\. BioDivergence differs along four axes: \(i\) a six\-class taxonomy that separates direct from contextual contradiction and retains underspecified and controversial cases rather than collapsing them; \(ii\) a 13\-axis ontology that supervises*why*two claims diverge; \(iii\) five structured outputs per example rather than a single label; and \(iv\) a biomedical\-abstract construction pipeline with per\-domain controls, an explicit leakage audit, an article\-disjoint primary release, and a legacy pair\-level variant for backward comparability\. We do not claim v1\.0 matches the annotation quality of expert\-curated benchmarks such as SciFact; rather, we position BioDivergence\-Silver as a structured, large\-scale silver benchmark that exposes multi\-component reasoning tasks prior work does not evaluate\.

## 3Task Formulation

Given a claim pair\(ca,cb\)\(c\_\{a\},c\_\{b\}\)with their source abstracts\(da,db\)\(d\_\{a\},d\_\{b\}\), BioDivergence defines four*primary*evaluation targets and one*scoped*output\.

#### Primary evaluation targets \(Tasks 1–4\)\.

\(1\)*Conflict type*y∈𝒴y\\in\\mathcal\{Y\}from the six\-class taxonomy; \(2\)*divergence axes*A⊆𝒜A\\subseteq\\mathcal\{A\}from the 13\-axis ontology; \(3\)*dominant confounder*d∈Ad\\in Awhen one axis is most explanatory; \(4\)*reconciliation explanation*rrdescribing how the claims coexist or why their tension remains unresolved\. Absolute benchmark\-level claims in this paper are anchored to these four outputs on the article\-disjoint primary split\.

#### Scoped output \(Task 5, v1\.0\)\.

\(5\)*Evidence spans*Ea,EbE\_\{a\},E\_\{b\}justifying the structured prediction\. Because the v1\.0 silver annotator copies spans verbatim from the claim text \(Appendix[I](https://arxiv.org/html/2606.11208#A9)\), Task 5 measures claim\-level substring retrieval rather than abstract\-level grounding, and is reported in Section[7](https://arxiv.org/html/2606.11208#S7)as release\-time calibration on the silver pipeline rather than as a grounding capability benchmark\.

## 4BioDivergence Ontology

BioDivergence combines a*conflict taxonomy*with a*contextual divergence ontology*\. The six\-class taxonomy separates direct contradiction, contextual contradiction, underspecified apparent contradiction, evidence insufficiency, unresolved scientific controversy, and no conflict\. The 13\-axis divergence ontology captures the contextual dimensions along which biomedical findings may appear to disagree: population, geography, time period, assay protocol, study design, intervention, disease subtype, molecular background, clinical setting, sample source, endpoint definition, organism strain, and anunknown\_latent\_factorslot\. Together these supply structured supervision for*why*two claims diverge rather than a single undifferentiated label; full definitions are in Appendix[E\.1](https://arxiv.org/html/2606.11208#A5.SS1)\.

## 5Benchmark Construction

BioDivergence is constructed through a six\-stage pipeline: corpus collection, normalization, claim extraction, candidate\-pair mining, structured silver labelling, and benchmark packaging\. Labels are produced by a schema\-constrained annotation pipeline rather than by full expert adjudication\.

### 5\.1Data Construction

We collect biomedical abstracts from PubMed and Europe PMC across five domains \(AMR, oncology, infectious disease, genomics, and clinical trials/epidemiology\), yielding202,180 unique abstractsafter cleaning, normalization, and hierarchical deduplication \(Appendix[F](https://arxiv.org/html/2606.11208#A6)\)\. A rule\-based claim extractor produces527,907 scientific claimsfrom 139,792 abstracts \(69\.1% abstract\-level yield; 2\.61 claims/abstract; Appendix[H](https://arxiv.org/html/2606.11208#A8)\)\. Candidate pairs are mined within each domain by retrieving semantically similar claims using sentence embeddings and ranking them with disagreement\-sensitive signals \(polarity reversal, negation, entity mismatch, contextual conflict cues\); we cap each domain at 10,000 pairs for a balanced50,000\-paircandidate pool \(Appendix[G](https://arxiv.org/html/2606.11208#A7)\)\. Candidate pairs are then silver\-labelled and assembled into two sibling splits of the same pool: theprimaryBioDivergence\-Silver\-v1\.0\(article\-disjoint; 11,865 pairs, 10,183 / 840 / 842\) and a legacyv1\.0\-pairlevelvariant \(pair\-level de\-duplication only; 12,500 pairs, 8,750 / 1,250 / 2,500\) retained for backward comparability\. In both variants dev and test contain only LLM\-relabelled examples; train combines remaining LLM\-relabelled examples with a heuristic top\-up \(confidence≥0\.45\\geq 0\.45\) to meet per\-domain targets\.

#### Split design: article\-disjoint primary, pair\-level legacy\.

In the primary release, connected components of articles \(linked through any shared claim pair\) are kept intact within a single split, andpair\_idduplicates are resolved in favour of the LLM\-labelled copy, yielding zero article, claim, or pair overlap across splits\. Article disjointness is prioritised because a single abstract can generate many claim pairs, so article overlap lets a model reach high scores via abstract\-level memorisation rather than contextual\-contradiction learning\. The legacy pair\-level variant is retained only for backward comparability with our broader baseline suite and is not recommended for headline evaluation \(Table[1](https://arxiv.org/html/2606.11208#S5.T1)\)\. Accordingly, Table[2](https://arxiv.org/html/2606.11208#S7.T2)is the headline primary\-release result; Tables[4](https://arxiv.org/html/2606.11208#S7.T4)and[5](https://arxiv.org/html/2606.11208#S7.T5)extend baseline coverage on the legacy variant, with absolute release\-level claims anchored to the primary\-release table\. Construction details and three\-seed results are in Appendix[M](https://arxiv.org/html/2606.11208#A13)\.

Table 1:Leakage in the legacy pair\-level variant \(v1\.0\-pairlevel\)\. The primary article\-disjoint release has zero overlap on every row below\. Full audit in Appendix[M](https://arxiv.org/html/2606.11208#A13)\.StatisticCount% of testTest rows with either abstract in train2,34093\.6%Test rows with both abstracts in train1,29952\.0%Distinct test claim\-pairs also in train318 / 2,44513\.0%Overlappingpair\_ids \(train∩\\captest\)2018\.0%

### 5\.2Silver Annotation Pipeline

Each record contains paired claims, source abstracts, and structured fields for conflict type, divergence axes, dominant confounder, reconciliation, evidence spans, and annotator confidence\. A 100\-example pilot compared a deterministic heuristic annotator against an LLM\-based annotator \(Qwen2\.5\-72B\-Instruct\); the LLM produced substantially richer contextual supervision \(37 contextual\-contradictions vs\. 3; mean confidence 0\.916 vs\. 0\.398; Appendix[I](https://arxiv.org/html/2606.11208#A9)\)\. The pilot is a distribution\-shift comparison rather than a validation against human\-adjudicated gold labels; it informed prompt design and schema validation, and the full release was then relabelled withqwen2\.5:7bvia Ollama for throughput\. Validation, repair logic, and a heuristic fallback insrc/conflicttopology/silver/annotator\.pyenforce schema consistency: out\-of\-vocabulary conflict types default tono\_conflict, off\-schema divergence axes are filtered against the 13\-axis ontology, and off\-schema dominant confounders are nulled before a record is admitted to the release pool\. The heuristic annotator is retained as a fallback rather than discarded\.

#### Benchmark validity and scope\.

On the primary article\-disjoint split the release supports reliable evaluation of two conflict classes and nine of thirteen divergence axes \(Appendix[F](https://arxiv.org/html/2606.11208#A6)\); the remaining schema slots are retained for ontological stability, so future releases extend the schema rather than break it\. Four safeguards support v1\.0’s evaluation value despite its silver labels: \(i\) schema validation constrains all axis labels to the 13\-axis ontology and drops out\-of\-vocabulary strings before benchmark assembly \(§[5\.2](https://arxiv.org/html/2606.11208#S5.SS2)\); \(ii\) the explicit leakage audit motivates the article\-disjoint primary; \(iii\) a non\-Qwen annotator\-family sensitivity analysis \(raw agreement 64%, Cohen’sκ=0\.20\\kappa=0\.20; Appendix[M\.4](https://arxiv.org/html/2606.11208#A13.SS4)\) scoped as a sensitivity check rather than a label validation; and \(iv\) a three\-seed reference\-model protocol with per\-seed reporting for all trainable results\.

#### Code and data availability\.

## 6Method

For the five BioDivergence outputs defined in §[3](https://arxiv.org/html/2606.11208#S3), we use a discriminative reference model for the three structured\-prediction tasks \(conflict type, divergence axes, dominant confounder\) and LLM\-based structured generation for the two remaining outputs \(reconciliation explanation, claim\-level evidence\-span extraction\)\.

### 6\.1Reference Discriminative Model for Tasks 1–3

The reference model has three modules\. \(i\) A*claim\-pair encoder*tokenises the pair as\[CLS\] claim\_a \[SEP\] claim\_b \[SEP\]and encodes with BiomedBERT, producing pooled and token\-level representations\. \(ii\) A*context\-fusion*stage applies multi\-head cross\-attention over the encoder outputs and concatenates the result with lightweight comparison features \(entity overlap, semantic similarity, disagreement\-sensitive scores from the candidate\-mining stage, negation/polarity cues, claim\-type indicators\)\. \(iii\)*Multi\-task prediction heads*on the fused representation𝐳\\mathbf\{z\}:y^=softmax⁡\(𝐖c​t​𝐳\)\\hat\{y\}=\\operatorname\{softmax\}\(\\mathbf\{W\}\_\{ct\}\\mathbf\{z\}\)\(conflict type\),A^=σ​\(𝐖a​x​𝐳\)\\hat\{A\}=\\sigma\(\\mathbf\{W\}\_\{ax\}\\mathbf\{z\}\)\(multi\-label divergence axes\), andd^=softmax⁡\(𝐖d​c​\[𝐳;A^\]\)\\hat\{d\}=\\operatorname\{softmax\}\(\\mathbf\{W\}\_\{dc\}\[\\mathbf\{z\};\\hat\{A\}\]\)\(dominant confounder conditioned on the predicted axis profile\)\. Full architecture details are in Appendix[J](https://arxiv.org/html/2606.11208#A10)\.

### 6\.2LLM\-Based Structured Generation for Tasks 4–5

Tasks 4 \(reconciliation explanation\) and 5 \(claim\-level evidence\-span extraction\) require open\-form reasoning rather than fixed\-label prediction\. Both are evaluated with LLM\-based structured generation baselines that receive the paired claims, source abstracts, and ontology schema and return a schema\-constrained JSON output\. Task 5 is evaluated under the v1\.0 scoping described in §[3](https://arxiv.org/html/2606.11208#S3)\.

### 6\.3Training Objective

Tasks 1–3 are jointly optimised asℒ=λ1​ℒc​t\+λ2​ℒa​x\+λ3​ℒd​c\\mathcal\{L\}=\\lambda\_\{1\}\\mathcal\{L\}\_\{ct\}\+\\lambda\_\{2\}\\mathcal\{L\}\_\{ax\}\+\\lambda\_\{3\}\\mathcal\{L\}\_\{dc\}\(cross\-entropy for conflict type and dominant confounder; binary cross\-entropy for multi\-label axes\)\. The released run usesλ1=1\.0,λ2=1\.0,λ3=0\.3\\lambda\_\{1\}\{=\}1\.0,\\lambda\_\{2\}\{=\}1\.0,\\lambda\_\{3\}\{=\}0\.3\(Table[32](https://arxiv.org/html/2606.11208#A10.T32)\)\. Joint optimisation shapes representations sensitive to both conflict type and the explanatory contextual axes, avoiding the representation collapse that single\-task training on a label\-skewed benchmark can produce\.

## 7Experiments

### 7\.1Experimental Setup

We evaluate on both release variants \(§[5](https://arxiv.org/html/2606.11208#S5)\): theprimaryarticle\-disjoint split \(10,183 / 840 / 842\) and the legacy pair\-level variant \(8,750 / 1,250 / 2,500\)\. We treat*no\_conflict*and*contextual\_contradiction*as the primary evaluable classes in v1\.0 \(Appendix[F](https://arxiv.org/html/2606.11208#A6)\) and the other four as exploratory\.

#### Evaluation policy\.

Absolute release\-level claims are anchored to the article\-disjoint primary split \(Table[2](https://arxiv.org/html/2606.11208#S7.T2)\): headline contextual\-contradiction scores, primary\-axis learning, and cross\-model comparisons of absolute performance\. The legacy pair\-level variant admits substantial article\-level overlap \(Table[1](https://arxiv.org/html/2606.11208#S5.T1)\) and is therefore not used for absolute claims; it is retained for backward comparability with our broader baseline suite \(Tables[4](https://arxiv.org/html/2606.11208#S7.T4),[5](https://arxiv.org/html/2606.11208#S7.T5)\), which was computed on the pair\-level packaging prior to the article\-disjoint re\-split\. The two table sets are complementary, not equivalent\.

#### Models and metrics\.

We compare \(i\) floor baselines \(majority, stratified random, most\-frequent\-axis\), \(ii\) shallow lexical baselines, \(iii\) zero\-shot NLI transfer baselines, \(iv\) zero\-shot instruction\-tuned LLMs \(Qwen3\-8B, Mistral\-7B\-v0\.3\), and \(v\) span\-extraction\-oriented lexical and retrieval baselines \(span heuristics, BM25, dense retrieval\); Tasks 1–3 additionally include a BiomedBERT multi\-task reference model\. Conflict type is scored by accuracy, Macro\-F1, weighted\-F1,κ\\kappa, and per\-class F1; axes as multi\-label micro/macro precision, recall, F1, and exact match; dominant confounder by accuracy; reconciliation by BLEU, ROUGE\-L, semantic similarity, and slot overlap; claim\-level evidence\-span extraction by token\-overlap IoU, precision, recall, and F1\. Full implementation, prompts, and metric definitions are in the appendix\.

### 7\.2Structured Contradiction Analysis

Table[2](https://arxiv.org/html/2606.11208#S7.T2)reports primary\-release headline numbers: the reference model across three seeds, and the full 842\-example zero\-shot Mistral\-7B\-Instruct\-v0\.3 evaluation on the primary test set for annotator\-family independence\. Tables[4](https://arxiv.org/html/2606.11208#S7.T4)and[5](https://arxiv.org/html/2606.11208#S7.T5)report the broader baseline suite on the legacy pair\-level variant\. We discuss both jointly\.

Table 2:Reference\-model and non\-Qwen zero\-shot performance on the primary article\-disjoint v1\.0 release and the legacy pair\-level variant\. Metrics are on the corresponding test set\. Because the primary test set has zero article\-level overlap with training, any performance retained here is not attributable to article memorisation\.Model / splitAcc\.Macro\-F1W\-F1κ\\kappaCtx F1Reference model \(primary, article\-disjoint, 3\-seed\)0\.693±\\pm0\.0140\.253±\\pm0\.0020\.695±\\pm0\.0070\.276±\\pm0\.0120\.401±\\pm0\.017Mistral\-7B\-Instruct\-v0\.3 \(zero\-shot, primary test,n=842n=842\)0\.55230\.18770\.58590\.16920\.3894Reference model \(legacy pair\-level, 3\-seed\)0\.58800\.30200\.65900\.26000\.5210

On the primary article\-disjoint test the reference model reachesCtx F1=0\.401±0\.017\\text\{Ctx F1\}=0\.401\\pm 0\.017over three seeds, versus 0\.521 on the legacy pair\-level test: a≈\\approx12\-point drop attributable to article\-level overlap in the legacy variant\. The full 842\-example Mistral\-7B\-v0\.3 primary\-test run reaches 0\.5523 accuracy, 0\.1877 macro\-F1, and 0\.3894 contextual\-contradiction F1, slightly below the reference primary score \(0\.4011\) and clearly below the legacy pair\-level reference score \(0\.5210\)\. These numbers keep the non\-Qwen comparison on the full primary split without article leakage, and show that the article\-disjoint test remains a meaningful annotator\-family\-independent evaluation rather than a subsample artifact\. On the legacy pair\-level tables, the majority\-class baseline reaches 0\.7128 accuracy but zero F1 on contextual and direct contradiction, so the per\-class Ctx F1 column carries the informative signal; zero\-shot NLI transfer fails across the board, indicating that entailment\-style label spaces do not capture context\-dependence\. Qwen3\-8B leads on accuracy and Macro\-F1 while the reference model attains the highest Ctx F1, and divergence\-axis and dominant\-confounder prediction remain difficult for all systems, indicating that recovering*why*claims diverge is harder than deciding*whether*they do\. For dominant confounder, the reference model’s marginal gap over the frequency baseline \(0\.561 vs\. 0\.552\) is driven primarily by upstream axis uncertainty: supplying gold divergence axes at inference time lifts dominant\-confounder accuracy to 0\.6901 \(Table[13](https://arxiv.org/html/2606.11208#A4.T13), Appendix[D](https://arxiv.org/html/2606.11208#A4)\), indicating that confounder prediction is learnable given accurate axis supervision and that the current ceiling is set by axis prediction quality rather than confounder\-head capacity\. Direct\-contradiction F1 is deferred to Appendix[C\.1](https://arxiv.org/html/2606.11208#A3.SS1)\(n=7n\{=\}7in the legacy test; sampling noise dominates\)\.

#### What this evaluation reveals\.

Three findings stand out as properties of BioDivergence\-*the\-benchmark*, not of any particular model\. First, flat contradiction classification is insufficient: divergence\-axis extraction and dominant\-confounder prediction lag conflict\-type accuracy across all systems, so models that are right about*whether*two claims conflict can still be wrong about*why*\. Second, the primary\-vs\-legacy delta is diagnostic\. The≈\\approx12\-point contextual\-F1 drop under article\-disjoint evaluation and the proximity of the full non\-Qwen primary\-test run to the reference primary score are exactly the pattern a leakage\-aware benchmark should produce when article memorisation and contextual reasoning are separated, a pattern invisible under conventional pair\-level reporting\. Third, different model families win different capabilities: zero\-shot LLMs dominate coarse classification and reconciliation generation; the fine\-tuned reference model dominates contextual detection and dominant\-confounder prediction on the legacy split; retrieval\-style baselines dominate claim\-level span extraction by construction\. Practitioners evaluating contextual\-contradiction capability should report on the primary article\-disjoint split and anchor claims to a non\-Qwen zero\-shot baseline, treating the silver labels as an evaluation target rather than as fine\-tuning supervision\. For axis reporting, the primary 9\-axis view is the recommended headline summary because it excludes the four sparse provisional axes while preserving the full 842\-example primary test set\.

Table 3:Primary article\-disjoint axis evaluation for the reference model\. We foreground the primary 9\-axis view in v1\.0 because it excludes the four sparsely populated provisional axes while preserving the full 842\-example test set\.Axis spaceMicro\-F1Macro\-F1Exact matchFull 13 axes0\.195±\\pm0\.0090\.116±\\pm0\.0020\.623±\\pm0\.057Primary 9 axes0\.195±\\pm0\.0090\.168±\\pm0\.0030\.623±\\pm0\.057Table 4:Conflict\-type classification on thelegacy pair\-level test set\(n=2,500n\{=\}2\{,\}500\); reported for backward comparability and broad baseline coverage\. Headline article\-disjoint\-primary numbers \(3\-seed reference model \+ non\-Qwen zero\-shot\) are in Table[2](https://arxiv.org/html/2606.11208#S7.T2)\. Macro\-F1 is over the full six\-class taxonomy\. Ctx F1 is the primary rare\-but\-evaluable class\. Direct\-contradiction F1 is deferred to Appendix[C](https://arxiv.org/html/2606.11208#A3)because the class hasn=7n\{=\}7test examples in this variant\. Reference model: mean±\\pmstd over seeds \(42, 123, 7\); zero\-shot LLMs: greedy decoding, single run\. Qwen3\-8B shares a model family with the Qwen2\.5 silver annotator; Mistral\-7B\-v0\.3 is the primary annotator\-independent zero\-shot reference\. Best annotator\-independent result in bold; best overall if different \(including the same\-family Qwen3\-8B row\)underlined\.ModelAcc\.Macro\-F1W\-F1κ\\kappaCtx F1*Independent baselines \(annotator\-family\-independent\)*Majority\-class baseline0\.71280\.16650\.59330\.00000\.0000Stratified\-random baseline0\.49920\.17910\.53820\.01580\.2087Lexical / overlap baseline0\.58600\.19780\.59210\.05380\.2355General\-domain NLI transfer \(zero\-shot\)0\.04800\.03080\.07920\.00730\.0000Biomedical NLI transfer \(zero\-shot\)0\.00440\.00180\.00000\.00000\.0000Mistral\-7B\-v0\.3 \(zero\-shot\)0\.58840\.30610\.63370\.25070\.5019Reference model \(ours\)0\.588±\\pm0\.0080\.302±\\pm0\.0190\.659±\\pm0\.0080\.260±\\pm0\.0070\.521±\\pm0\.009*Same annotator family \(Qwen; caveat applies\)*Qwen3\-8B \(zero\-shot\)0\.75920\.32680\.70490\.26490\.3364

Table 5:Structured prediction on thelegacy pair\-level test set\(n=2,500n\{=\}2\{,\}500\); retained for broad baseline coverage\. Axes evaluated as multi\-label; dominant confounder on examples with that annotation\. Reference model: mean±\\pmstd over three seeds \(42, 123, 7\); zero\-shot LLMs: greedy decoding, single run\. Qwen3\-8B shares a model family with the Qwen2\.5 silver annotator; Mistral\-7B\-v0\.3 is the primary annotator\-independent zero\-shot reference\. Best annotator\-independent result in bold; best overall if different \(including the same\-family Qwen3\-8B row\)underlined\. Matching article\-disjoint\-primary reference\-model numbers are in Table[38](https://arxiv.org/html/2606.11208#A13.T38)\(appendix\)\.ModelDivergence AxesDominant ConfounderMicro\-F1Macro\-F1Prec\.RecallExactAcc\.Exact*Independent baselines \(annotator\-family\-independent\)*Most\-frequent / frequency baseline0\.26430\.02570\.20080\.38650\.00760\.55240\.5524Independent per\-axis / pipeline baseline0\.26080\.15220\.17540\.50810\.58920\.56890\.5689Mistral\-7B\-v0\.3 \(zero\-shot\)0\.16060\.10950\.10870\.30720\.24600\.18710\.1871Reference model \(ours\)0\.215±\\pm0\.0040\.134±\\pm0\.0030\.131±\\pm0\.0030\.594±\\pm0\.0020\.535±\\pm0\.0010\.561±\\pm0\.0090\.561±\\pm0\.009*Same annotator family \(Qwen; caveat applies\)*Qwen3\-8B \(zero\-shot\)0\.11590\.09750\.15140\.09390\.66920\.08230\.0823

### 7\.3Explanation and Evidence\-Span Extraction

Task 5 in v1\.0 is pipeline calibration, not a grounding capability comparison\.Because the silver annotator copies evidence spans verbatim from the claim text \(§[3](https://arxiv.org/html/2606.11208#S3)\), any system that returns a claim\-text substring \(lexical span heuristic, BM25, dense retrieval\) is expected to dominate generative span selection\. Reconciliation and span\-extraction results are reported in Tables[7](https://arxiv.org/html/2606.11208#A3.T7)and[8](https://arxiv.org/html/2606.11208#A3.T8)\. Qwen3\-8B achieves the best lexical and semantic reconciliation quality, with Mistral\-7B\-v0\.3 competitive; on claim\-level span extraction, lexical and retrieval baselines outperform zero\-shot LLMs, as expected under the verbatim\-from\-claim rule\. Models that produce fluent reconciliations do not reliably select supporting spans, indicating that explanation generation and span extraction are distinct capabilities that should be evaluated separately\.

### 7\.4Extended Analysis

We further study benchmark behaviour under controlled context edits, held\-out domains, and robustness perturbations; full results are in Tables[9](https://arxiv.org/html/2606.11208#A3.T9)–[12](https://arxiv.org/html/2606.11208#A3.T12)\. Mistral\-7B\-v0\.3 attains the strongest counterfactual accuracy while Qwen3\-8B leads on explanation consistency and span support, suggesting that counterfactual revision is a distinct capability from reconciliation generation\. Leave\-one\-domain\-out transfer is uneven across domains, indicating that biomedical contradiction reasoning depends partly on domain\-specific vocabulary and knowledge\. The reference model exhibits low raw flip rates but moderate axis\-update rates under controlled perturbations, that is, stable top\-level predictions with retained axis\-level sensitivity\.

### 7\.5Benchmark Validation and Controlled\-Reference Analyses

Detailed validation results are in Table[13](https://arxiv.org/html/2606.11208#A4.T13)\. Dominant\-confounder accuracy improves when gold divergence axes are supplied, so a meaningful portion of downstream error is inherited from upstream axis uncertainty\. When gold structured fields are provided to explanation generation, slot overlap becomes nearly perfect while semantic quality improves only modestly; this is a construction\-verification observation, not a capability comparison, and we use it to sanity\-check the pipeline\. The released JSONL files do not carry native confidence scores, so the high\-confidence slice uses a provenance\-based confidence proxy\.

#### Silver\-label reliability\.

We also report confidence\-filtered slices to show which conclusions remain stable under stricter filtering\. Using the provenance\-based confidence proxy, the 591\-example high\-confidence slice yields 0\.5736 accuracy, 0\.1905 macro\-F1, 0\.4747 contextual\-F1, 0\.1777 axis micro\-F1, and 0\.2386 exact match\. The slice is a robustness check rather than a gold validation, but it suggests the contextual signal is not confined to the lowest\-confidence examples\.

#### Non\-Qwen annotator\-family sensitivity audit\.

As a sanity check on annotator\-family dependence, a stratified 1,000\-example subset of the primary test\+dev pool was re\-annotated withLlama\-3\.1\-8B\-Instruct\(claims\-only; the released JSONL omits abstracts\)\. Overall raw agreement is 64% and Cohen’sκ\\kappais 0\.20 \(fair\)\. The audit does*not*validate the v1\.0 labels: the audit annotator is strictly weaker than the release annotator and lacks abstract context, so it measures how much of the release label structure is recoverable from claims alone by a non\-Qwen\-family model rather than whether the labels themselves are correct\. It*is*useful as a pressure test on annotator\-family dependence \(agreement on the evaluable\-class majority is moderate, as expected for a claims\-only cross\-family spot check\) and as indirect evidence about rare classes: Llama flags direct\-contradiction and evidence\-insufficiency cases substantially more often than the release annotator on the same pairs, which is compatible with, though does not prove, the interpretation that v1\.0 rare\-class scarcity partly reflects release\-annotator conservatism rather than absence in the candidate pool \(Appendix[M\.4](https://arxiv.org/html/2606.11208#A13.SS4)\)\.

## 8Conclusion

We introduced BioDivergence, an evaluation framework for hidden contextual contradiction analysis in biomedical abstracts, targeting a gap left by flat NLI and claim\-verification benchmarks: recovering*why*two biomedical findings disagree when the disagreement is explained by hidden contextual differences rather than direct logical incompatibility\. BioDivergence operationalises this gap as four primary structured prediction targets over claim pairs plus a scoped fifth output, released on an article\-disjoint silver benchmark of 11,865 pairs\. Under article\-disjoint evaluation, the fine\-tuned reference model loses≈\\approx12 points of contextual\-contradiction F1 relative to the legacy pair\-level packaging, separating article memorisation from contextual reasoning in a way flat contradiction evaluation cannot\. Future releases will extend annotation, grounding, and gold calibration; BioDivergence\-v1\.0 already functions as a leakage\-aware evaluation lens for contextual contradiction reasoning\.

## References

- Anonymous \[2024\]Anonymous\.Do automatic factuality metrics measure factuality? a critical evaluation\.*arXiv preprint arXiv:2411\.16638*, 2024\.
- Bean et al\. \[2025\]Andrew M\. Bean, Ryan O\. Kearns, Angelika Romanou, et al\.Measuring what matters: Construct validity in large language model benchmarks\.In*NeurIPS*, 2025\.
- DeYoung et al\. \[2020\]Jay DeYoung, Eric Lehman, Ben Nye, Iain J\. Marshall, and Byron C\. Wallace\.Evidence inference 2\.0: More data, better models\.In*BioNLP Workshop*, 2020\.
- Hou et al\. \[2024\]Yufang Hou, Alessandra Pascale, Javier Carnerero\-Cano, Tigran Tchrakian, Radu Marinescu, Elizabeth Daly, Inkit Padhi, and Prasanna Sattigeri\.Wikicontradict: A benchmark for evaluating llms on real\-world knowledge conflicts from wikipedia\.In*NeurIPS Datasets and Benchmarks Track*, 2024\.
- Li et al\. \[2024\]Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S\. Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov\.Mediq: Question\-asking llms and a benchmark for reliable interactive clinical reasoning\.In*NeurIPS*, 2024\.
- Ru et al\. \[2024\]Dongyu Ru, Lin Qiu, Xiangkun Hu, Tianhang Zhang, Peng Shi, Shuaichen Chang, Cunxiang Wang, Shichao Sun, Pengfei Liu, and Yue Zhang\.Ragchecker: A fine\-grained framework for diagnosing retrieval\-augmented generation\.In*NeurIPS Datasets and Benchmarks Track*, 2024\.
- Schuster et al\. \[2021\]Tal Schuster, Adam Fisch, and Regina Barzilay\.Get your vitamin c\! robust fact verification with contrastive evidence\.In*NAACL*, 2021\.
- Sosa and Altman \[2022\]Daniel N\. Sosa and Russ B\. Altman\.Contexts and contradictions: A roadmap for computational drug repurposing with knowledge inference\.*Briefings in Bioinformatics*, 2022\.
- Su et al\. \[2024\]Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng\.Conflictbank: A benchmark for evaluating knowledge conflicts in large language models\.In*NeurIPS Datasets and Benchmarks Track*, 2024\.
- Thorne et al\. \[2018\]James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal\.Fever: a large\-scale dataset for fact extraction and verification\.In*NAACL*, 2018\.
- Wadden et al\. \[2020\]David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi\.Fact or fiction: Verifying scientific claims\.In*EMNLP*, 2020\.
- Wadden et al\. \[2022\]David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, and Hannaneh Hajishirzi\.Scifact\-open: Towards open\-domain scientific claim verification\.In*Findings of EMNLP*, 2022\.
- Wang et al\. \[2024\]Bowen Wang, Jiuyang Chang, Yiming Qian, Guoxin Chen, Junhao Chen, Zhouqiang Jiang, Jiahao Zhang, Yuta Nakashima, and Hajime Nagahara\.Direct: Diagnostic reasoning for clinical notes via large language models\.In*NeurIPS Datasets and Benchmarks Track*, 2024\.
- Wu et al\. \[2024\]Kevin Wu, Eric Wu, and James Zou\.Clasheval: Quantifying the tug\-of\-war between an llm’s internal prior and external evidence\.In*NeurIPS Datasets and Benchmarks Track*, 2024\.
- Zhang et al\. \[2025\]Boya Zhang, Alban Bornet, Rui Yang, Nan Liu, and Douglas Teodoro\.Healthcontradict: Evaluating biomedical knowledge conflicts in language models\.*npj Digital Medicine*, 2025\.

###### Contents

1. [1Introduction](https://arxiv.org/html/2606.11208#S1)
2. [2Related Work](https://arxiv.org/html/2606.11208#S2)
3. [3Task Formulation](https://arxiv.org/html/2606.11208#S3)
4. [4BioDivergence Ontology](https://arxiv.org/html/2606.11208#S4)
5. [5Benchmark Construction](https://arxiv.org/html/2606.11208#S5)1. [5\.1Data Construction](https://arxiv.org/html/2606.11208#S5.SS1) 2. [5\.2Silver Annotation Pipeline](https://arxiv.org/html/2606.11208#S5.SS2)
6. [6Method](https://arxiv.org/html/2606.11208#S6)1. [6\.1Reference Discriminative Model for Tasks 1–3](https://arxiv.org/html/2606.11208#S6.SS1) 2. [6\.2LLM\-Based Structured Generation for Tasks 4–5](https://arxiv.org/html/2606.11208#S6.SS2) 3. [6\.3Training Objective](https://arxiv.org/html/2606.11208#S6.SS3)
7. [7Experiments](https://arxiv.org/html/2606.11208#S7)1. [7\.1Experimental Setup](https://arxiv.org/html/2606.11208#S7.SS1) 2. [7\.2Structured Contradiction Analysis](https://arxiv.org/html/2606.11208#S7.SS2) 3. [7\.3Explanation and Evidence\-Span Extraction](https://arxiv.org/html/2606.11208#S7.SS3) 4. [7\.4Extended Analysis](https://arxiv.org/html/2606.11208#S7.SS4) 5. [7\.5Benchmark Validation and Controlled\-Reference Analyses](https://arxiv.org/html/2606.11208#S7.SS5)
8. [8Conclusion](https://arxiv.org/html/2606.11208#S8)
9. [References](https://arxiv.org/html/2606.11208#bib)
10. [AConstruction Process of BioDivergence\-Silver\-v1\.0](https://arxiv.org/html/2606.11208#A1)
11. [BReference Multi\-Task Model](https://arxiv.org/html/2606.11208#A2)
12. [CExtended Experimental Results](https://arxiv.org/html/2606.11208#A3)1. [C\.1Direct\-Contradiction F1 \(sparse class\)](https://arxiv.org/html/2606.11208#A3.SS1) 2. [C\.2Reconciliation Generation](https://arxiv.org/html/2606.11208#A3.SS2) 3. [C\.3Claim\-Level Evidence\-Span Extraction](https://arxiv.org/html/2606.11208#A3.SS3) 4. [C\.4Counterfactual Reconciliation](https://arxiv.org/html/2606.11208#A3.SS4) 5. [C\.5Domain Generalization and Robustness](https://arxiv.org/html/2606.11208#A3.SS5) 6. [C\.6Ablation Analysis](https://arxiv.org/html/2606.11208#A3.SS6)
13. [DOracle and Controlled\-Reference Analyses](https://arxiv.org/html/2606.11208#A4)
14. [EFull Ontology and Label Definitions](https://arxiv.org/html/2606.11208#A5)1. [E\.1Full Divergence\-Axis Ontology](https://arxiv.org/html/2606.11208#A5.SS1) 2. [E\.2Conflict Taxonomy and Label Definitions](https://arxiv.org/html/2606.11208#A5.SS2) 3. [E\.3Divergence\-Axis Usage Guidance](https://arxiv.org/html/2606.11208#A5.SS3)
15. [FBenchmark Construction and Data Statistics](https://arxiv.org/html/2606.11208#A6)1. [F\.1Corpus and Candidate\-Pair Construction](https://arxiv.org/html/2606.11208#A6.SS1) 2. [F\.2Released Benchmark Distribution \(Legacy Pair\-Level Variant\)](https://arxiv.org/html/2606.11208#A6.SS2)
16. [GPipeline Implementation Details](https://arxiv.org/html/2606.11208#A7)1. [G\.1Query Pack Structure](https://arxiv.org/html/2606.11208#A7.SS1) 2. [G\.2Representative Query Motifs](https://arxiv.org/html/2606.11208#A7.SS2) 3. [G\.3Normalization and Deduplication](https://arxiv.org/html/2606.11208#A7.SS3) 4. [G\.4Candidate\-Pair Mining](https://arxiv.org/html/2606.11208#A7.SS4) 5. [G\.5Canonical Workflow](https://arxiv.org/html/2606.11208#A7.SS5) 6. [G\.6End\-to\-End Pipeline Overview](https://arxiv.org/html/2606.11208#A7.SS6)
17. [HClaim Extraction Details and Claim Statistics](https://arxiv.org/html/2606.11208#A8)1. [H\.1Claim Type Distribution](https://arxiv.org/html/2606.11208#A8.SS1) 2. [H\.2Extracted Claims by Domain](https://arxiv.org/html/2606.11208#A8.SS2) 3. [H\.3Illustrative Claim Extraction Decisions](https://arxiv.org/html/2606.11208#A8.SS3) 4. [H\.4Extracted Record Structure](https://arxiv.org/html/2606.11208#A8.SS4)
18. [ISilver Annotation Protocol and Prompt Design](https://arxiv.org/html/2606.11208#A9)1. [I\.1Silver Pilot Comparison](https://arxiv.org/html/2606.11208#A9.SS1) 2. [I\.2Structured Annotation Fields](https://arxiv.org/html/2606.11208#A9.SS2) 3. [I\.3Run Artifacts and Provenance](https://arxiv.org/html/2606.11208#A9.SS3) 4. [I\.4Prompt Design](https://arxiv.org/html/2606.11208#A9.SS4)
19. [JTask Schema and Implementation Details](https://arxiv.org/html/2606.11208#A10)1. [J\.1Benchmark Input and Output Schema](https://arxiv.org/html/2606.11208#A10.SS1) 2. [J\.2Model Families and Baselines](https://arxiv.org/html/2606.11208#A10.SS2) 3. [J\.3Reference\-Model Training Configuration](https://arxiv.org/html/2606.11208#A10.SS3)
20. [KEvaluation Protocol Details](https://arxiv.org/html/2606.11208#A11)1. [K\.1Explanation and Span\-Extraction Metrics](https://arxiv.org/html/2606.11208#A11.SS1) 2. [K\.2Robustness and Counterfactual Templates](https://arxiv.org/html/2606.11208#A11.SS2)
21. [LQualitative Examples and Release Notes](https://arxiv.org/html/2606.11208#A12)
22. [MArticle\-Disjoint Primary Split: Construction and Audit](https://arxiv.org/html/2606.11208#A13)1. [M\.1Pipeline and Deduplication](https://arxiv.org/html/2606.11208#A13.SS1) 2. [M\.2Reference Model and Zero\-Shot Performance](https://arxiv.org/html/2606.11208#A13.SS2) 3. [M\.3Primary\-Axis \(9\-Axis\) Evaluation](https://arxiv.org/html/2606.11208#A13.SS3) 4. [M\.4Non\-Qwen Annotator\-Family Sensitivity Audit](https://arxiv.org/html/2606.11208#A13.SS4) 5. [M\.5Interpretation](https://arxiv.org/html/2606.11208#A13.SS5)

## Appendix AConstruction Process of BioDivergence\-Silver\-v1\.0

Algorithm[1](https://arxiv.org/html/2606.11208#alg1)summarizes the end\-to\-end pipeline used to build*BioDivergence\-Silver\-v1\.0*\. The process begins by collecting biomedical abstracts from multiple sources, including PubMed, Europe PMC, and a legacy AMR pathway\. The retrieved corpus is then normalized through text cleaning, encoding repair, citation removal, and duplicate filtering\.

Next, a rule\-based claim extractor identifies claim\-bearing sentences from each domain\-specific corpus\. These extracted claims are embedded into a semantic vector space, where nearest\-neighbor retrieval is used to generate candidate claim pairs\. A ranking function then prioritizes pairs that are both semantically related and likely to contain disagreement signals\.

The highest\-quality candidate pairs are passed to the structured silver annotation stage, where each pair receives labels such as conflict type, divergence axes, dominant confounder, reconciliation, and evidence spans\. Validation, repair logic, and heuristic fallback ensure schema consistency and high annotation coverage\.

Finally, all labeled examples are merged and partitioned into train, dev, and test splits, together with release metadata\. In mathematical form, the algorithm iterates over each biomedical domain, applies collection, normalization, claim extraction, candidate mining, and annotation, then unions the outputs into a final benchmark setℬ=\(𝒯,𝒱,𝒰\)\\mathcal\{B\}=\(\\mathcal\{T\},\\mathcal\{V\},\\mathcal\{U\}\)\.

Algorithm 1Construction of BioDivergence\-Silver\-v1\.0\.1:Domain set

𝒟\\mathcal\{D\}; query packs

\{Qd\}d∈𝒟\\\{Q\_\{d\}\\\}\_\{d\\in\\mathcal\{D\}\}; raw abstract pool

𝒜\\mathcal\{A\}; claim extractor

EE; ranking score

s​\(⋅,⋅\)s\(\\cdot,\\cdot\); annotator

AA; split builder

PP
2:Finalized benchmark

ℬ=\(𝒯,𝒱,𝒰\)\\mathcal\{B\}=\(\\mathcal\{T\},\\mathcal\{V\},\\mathcal\{U\}\)=*BioDivergence\-Silver\-v1\.0*

3:

𝒜\(0\)←∅,𝒞\(0\)←∅,𝒫\(0\)←∅\\mathcal\{A\}^\{\(0\)\}\\leftarrow\\emptyset,\\ \\mathcal\{C\}^\{\(0\)\}\\leftarrow\\emptyset,\\ \\mathcal\{P\}^\{\(0\)\}\\leftarrow\\emptyset
4:for all

d∈𝒟d\\in\\mathcal\{D\}do

5:

𝒜d←Collect​\(𝒜,Qd\)\\mathcal\{A\}\_\{d\}\\leftarrow\\mathrm\{Collect\}\(\\mathcal\{A\},Q\_\{d\}\)⊳\\trianglerightPubMed \+ Europe PMC \+ legacy AMR import

6:

𝒜~d←Normalize​\(𝒜d\)\\tilde\{\\mathcal\{A\}\}\_\{d\}\\leftarrow\\mathrm\{Normalize\}\(\\mathcal\{A\}\_\{d\}\)⊳\\trianglerightcleaning, Unicode, inline\-tag / citation removal

7:

𝒜^d←Dedup​\(𝒜~d\)\\hat\{\\mathcal\{A\}\}\_\{d\}\\leftarrow\\mathrm\{Dedup\}\(\\tilde\{\\mathcal\{A\}\}\_\{d\}\)⊳\\trianglerightDOI→\\toPMID→\\tocontent hash

8:

𝒞d←E​\(𝒜^d\)\\mathcal\{C\}\_\{d\}\\leftarrow E\(\\hat\{\\mathcal\{A\}\}\_\{d\}\)⊳\\trianglerightrule\-based claim extraction

9:

𝒫d←TopK​\(\{\(ci,cj\):ci,cj∈𝒞d,i<j\},s\)\\mathcal\{P\}\_\{d\}\\leftarrow\\mathrm\{TopK\}\\\!\\left\(\\\{\(c\_\{i\},c\_\{j\}\):c\_\{i\},c\_\{j\}\\in\\mathcal\{C\}\_\{d\},\\ i<j\\\},\\,s\\right\)⊳\\trianglerightembedding retrieval \+ disagreement\-aware ranking

10:

ℒd←A​\(𝒫d\)\\mathcal\{L\}\_\{d\}\\leftarrow A\(\\mathcal\{P\}\_\{d\}\)⊳\\trianglerightLLM\-based silver annotation with schema validation

11:

𝒜\(0\)←𝒜\(0\)∪𝒜^d;𝒞\(0\)←𝒞\(0\)∪𝒞d;𝒫\(0\)←𝒫\(0\)∪ℒd\\mathcal\{A\}^\{\(0\)\}\\leftarrow\\mathcal\{A\}^\{\(0\)\}\\cup\\hat\{\\mathcal\{A\}\}\_\{d\};\\ \\mathcal\{C\}^\{\(0\)\}\\leftarrow\\mathcal\{C\}^\{\(0\)\}\\cup\\mathcal\{C\}\_\{d\};\\ \\mathcal\{P\}^\{\(0\)\}\\leftarrow\\mathcal\{P\}^\{\(0\)\}\\cup\\mathcal\{L\}\_\{d\}
12:endfor

13:

ℬ←P​\(𝒫\(0\)\)\\mathcal\{B\}\\leftarrow P\(\\mathcal\{P\}^\{\(0\)\}\)⊳\\trianglerightpair\-level packaging; the primary release is then produced by an article\-disjoint re\-split \(Appendix[M](https://arxiv.org/html/2606.11208#A13)\)

14:return

ℬ\\mathcal\{B\}

## Appendix BReference Multi\-Task Model

BioDivergence uses a reference discriminative model for Tasks 1–3: conflict\-type prediction, divergence\-axis prediction, and dominant\-confounder prediction\. The shared backbone first encodes the paired claims as𝐮←fθ​\(\[ca;cb\]\)\\mathbf\{u\}\\leftarrow f\_\{\\theta\}\(\[c\_\{a\};c\_\{b\}\]\), then a lightweight interaction block produces a fused representation𝐳←gϕ​\(𝐮\)\\mathbf\{z\}\\leftarrow g\_\{\\phi\}\(\\mathbf\{u\}\)augmented with the same comparison features listed in Section[6](https://arxiv.org/html/2606.11208#S6)\(entity overlap, semantic similarity, disagreement\-sensitive scores from the candidate\-mining stage, negation/polarity cues, and claim\-type indicators\)\. The fused representation is passed to three task\-specific heads exactly as in Section[6](https://arxiv.org/html/2606.11208#S6): a six\-way conflict classifier, a multi\-label axis predictor, and a dominant\-confounder classifier conditioned on the predicted axis profile\. Training minimisesℒ=λc​t​ℒc​t\+λa​x​ℒa​x\+λd​c​ℒd​c\\mathcal\{L\}=\\lambda\_\{ct\}\\mathcal\{L\}\_\{ct\}\+\\lambda\_\{ax\}\\mathcal\{L\}\_\{ax\}\+\\lambda\_\{dc\}\\mathcal\{L\}\_\{dc\}by minibatch AdamW with the hyperparameters and task weights summarised in Table[32](https://arxiv.org/html/2606.11208#A10.T32)\.

## Appendix CExtended Experimental Results

This section reports the extended evaluation results that are summarized only briefly in the main paper\. We group them into five parts: \(i\) direct\-contradiction F1 on then=7n\{=\}7test subset, \(ii\) reconciliation generation, \(iii\) claim\-level evidence\-span extraction, \(iv\) counterfactual reconciliation analysis, and \(v\) transfer and robustness experiments\.

### C\.1Direct\-Contradiction F1 \(sparse class\)

For completeness, we report direct\-contradiction \(Dir\) F1 here, but caution against any model\-vs\-model comparison on this column: the test split contains only 7 direct\-contradiction examples \(Table[18](https://arxiv.org/html/2606.11208#A6.T18)\), so a single correct\-or\-missed prediction moves F1 by a large amount\. We include these numbers only as a record of what was computed\.

Table 6:Direct\-contradiction F1 on the test split \(n=7n\{=\}7\)\. Column omitted from the main\-paper Table[4](https://arxiv.org/html/2606.11208#S7.T4)because sampling variance dominates\. The reference model’s three\-seed range of 0\.00 / 0\.00 / 0\.25 is shown as mean±\\pmstd; the zero\-shot numbers are single\-run deterministic\.ModelDir F1 \(n=7n\{=\}7\)Majority\-class baseline0\.0000Stratified\-random baseline0\.0000Lexical / overlap baseline0\.0000General\-domain NLI transfer \(zero\-shot\)0\.0361Biomedical NLI transfer \(zero\-shot\)0\.0000Qwen3\-8B \(zero\-shot\)0\.2857Mistral\-7B\-v0\.3 \(zero\-shot\)0\.4211Reference model \(ours, 3 seeds\)0\.083±\\pm0\.118
### C\.2Reconciliation Generation

Table[7](https://arxiv.org/html/2606.11208#A3.T7)reports reconciliation\-generation results\. The LLM baselines substantially outperform extractive and template baselines on semantic and text\-quality measures, with Qwen3\-8B achieving the strongest overall performance\. However, the template baseline achieves almost perfect slot overlap, indicating that structured fidelity can be enforced even when the resulting explanation is semantically weak\.

Table 7:Reconciliation generation results on the legacy pair\-level test split \(n=2,500n\{=\}2\{,\}500; retained for backward comparability\)\. We report BLEU, ROUGE\-L, semantic similarity \(SemSim\), and structured slot overlap against the reference reconciliation\. Best values are bolded by column\. The template baseline is filled from the gold structured fields \(conflict\_type,divergence\_axes,dominant\_confounder\); because the slot\-overlap metric checks for those same gold tokens, its value for the template baseline \(0\.9956\) is a construction\-verification floor rather than a capability measurement\. LLM slot\-overlap numbers are measured without access to gold structured fields and should be interpreted as the primary result\.ModelBLEUROUGE\-LSemSimSlot OverlapExtractive reconciliation baseline0\.33480\.19360\.63900\.0713Template\-filled reconciliation baseline0\.41550\.14780\.33200\.9956Qwen3\-8B \(zero\-shot\)0\.48080\.30720\.71790\.3721Mistral\-7B\-v0\.3 \(zero\-shot\)0\.44210\.25740\.68820\.1998
### C\.3Claim\-Level Evidence\-Span Extraction

Table[8](https://arxiv.org/html/2606.11208#A3.T8)reports claim\-level evidence\-span results\. Unlike reconciliation generation, the strongest performance comes from lexical and retrieval\-oriented methods rather than LLMs\. The lexical span\-heuristic baseline returns substrings of the paired claims, which closely match the silver reference spans because the v1\.0 annotator is instructed to copy spans verbatim from the claim text \(Appendix[I](https://arxiv.org/html/2606.11208#A9)\)\. BM25 and dense retrieval obtain nearly perfect precision at somewhat lower recall\. We therefore treat Table[8](https://arxiv.org/html/2606.11208#A3.T8)as a calibration of the v1\.0 span\-extraction task under a claim\-verbatim rule, not as a finding about the general grounding capability of LLMs\. A future release will broaden the task to abstract\-level grounding and re\-run these baselines\.

Table 8:Claim\-level evidence\-span extraction results on the legacy pair\-level test split, over the 2,479 examples with reference evidence spans \(retained for backward comparability\)\. We report intersection\-over\-union \(IoU\), precision, recall, and F1 using a consistent span\-overlap evaluation protocol\. Best values are bolded by column\.*Interpretation:*v1\.0 silver spans are copied verbatim from the claim text \(Appendix[I](https://arxiv.org/html/2606.11208#A9)\), so lexical and retrieval baselines that return claim\-substrings should be expected to outperform generative span selection; we therefore interpret these numbers as calibrating the task rather than as a capability comparison\.ModelIoUPrecisionRecallF1Lexical span heuristic0\.81030\.83690\.93750\.8702BM25 sentence retrieval0\.65290\.99960\.65290\.7825Dense retrieval baseline0\.60880\.99960\.60880\.7458Qwen3\-8B LLM span extraction0\.20340\.34770\.33570\.3304Mistral\-7B\-v0\.3 LLM span extraction0\.21620\.35930\.35840\.3457
### C\.4Counterfactual Reconciliation

Table[9](https://arxiv.org/html/2606.11208#A3.T9)reports performance under controlled contextual edits\. Mistral\-7B\-v0\.3 achieves the highest counterfactual accuracy, whereas Qwen3\-8B performs best on explanation consistency and span support rate\. These results show that counterfactual updating is distinct from both free\-form reconciliation generation and evidence localization\.

Table 9:Counterfactual reconciliation results on 871 example\-edit pairs \(derived from the legacy pair\-level test split; retained for backward comparability\)\. We evaluate whether models correctly adapt their reconciliations under controlled contextual edits using counterfactual accuracy, explanation consistency, and span support rate\. Best values are bolded by column\.ModelCF Acc\.Expl\. Cons\.Span SupportTemplate reconciliation baseline0\.51610\.70000\.0616Qwen3\-8B \(zero\-shot\)0\.67110\.73240\.3542Mistral\-7B\-v0\.3 \(zero\-shot\)0\.74170\.72410\.3198
### C\.5Domain Generalization and Robustness

Table[10](https://arxiv.org/html/2606.11208#A3.T10)reports leave\-one\-domain\-out transfer across the five biomedical domains\. The reference model is strongest on some held\-out domains for structured prediction, whereas the zero\-shot LLM baseline is more competitive on coarse conflict classification under transfer\. Table[11](https://arxiv.org/html/2606.11208#A3.T11)reports robustness under controlled contextual perturbations\. The reference model remains conservative at the top\-level label while still showing moderate axis\-level sensitivity to context edits\.

Table 10:Domain generalization results across five biomedical domains\. For the reference model, each row corresponds to a leave\-one\-domain\-out training regime \(train on all domains except the listed test domain\); values are mean±\\pmstandard deviation over three seeds\. For Mistral\-7B\-v0\.3, the model is not fine\-tuned: the rows report deterministic zero\-shot evaluation on each held\-out domain’s test subset\. The “Train Domains” column therefore reflects the training regime for the reference model and is informational for the zero\-shot baseline\.Train DomainsTest DomainAcc\.Macro\-F1Axis F1Conf\. Acc\.Reference model \(ours\)All except AMRAMR0\.650±0\.0790\.650\\pm 0\.0790\.307±0\.0230\.307\\pm 0\.0230\.122±0\.0170\.122\\pm 0\.0170\.233±0\.0240\.233\\pm 0\.024All except OncologyOncology0\.541±0\.0270\.541\\pm 0\.0270\.220±0\.0060\.220\\pm 0\.0060\.240±0\.0090\.240\\pm 0\.0090\.605±0\.1560\.605\\pm 0\.156All except Infectious diseaseInfectious disease0\.630±0\.0290\.630\\pm 0\.0290\.257±0\.0130\.257\\pm 0\.0130\.211±0\.0050\.211\\pm 0\.0050\.553±0\.0430\.553\\pm 0\.043All except GenomicsGenomics0\.490±0\.0030\.490\\pm 0\.0030\.160±0\.0040\.160\\pm 0\.0040\.153±0\.0120\.153\\pm 0\.0120\.458±0\.0300\.458\\pm 0\.030All except Clinical trials / epiClinical trials / epi0\.541±0\.0010\.541\\pm 0\.0010\.240±0\.0290\.240\\pm 0\.0290\.140±0\.0040\.140\\pm 0\.0040\.595±0\.0170\.595\\pm 0\.017Mistral\-7B\-v0\.3 \(zero\-shot\)All except AMRAMR0\.6800\.2650\.1180\.167All except OncologyOncology0\.5240\.2190\.1920\.396All except Infectious diseaseInfectious disease0\.5840\.4160\.1800\.277All except GenomicsGenomics0\.5980\.3530\.1810\.412All except Clinical trials / epiClinical trials / epi0\.5560\.3950\.1120\.328Table 11:Robustness under contextual perturbation for the reference model on 2,500 examples per perturbation type from the legacy pair\-level test\. We report raw flip rate, corrected flip rate, divergence\-axis update rate, and explanation consistency under controlled contextual edits\. The raw flip rate is identical \(0\.0460 = 115 of 2,500 predictions\) across all five perturbation types because the reference model flips on the same small, prediction\-ambiguous subset of examples regardless of the specific text edit applied; the corrected\-flip and axis\-update columns reveal the edit\-specific signal\.*Explanation consistency*here is a derived proxy from the reference model’s structured predictions \(the discriminative reference model does not itself generate text; seescripts/run\_table14\_robustness\.py\), and its identical value across rows follows from the identical raw\-flip subset\.PerturbationFlip RateCorr\. Flip RateAxis Update RateExpl\. Cons\.Geography swap0\.04600\.03320\.09320\.9561Time shift0\.04600\.03520\.10020\.9561Assay change0\.04600\.03720\.10400\.9561Cohort change0\.04600\.03440\.10600\.9561Endpoint change0\.04600\.03600\.10240\.9561

### C\.6Ablation Analysis

Table[12](https://arxiv.org/html/2606.11208#A3.T12)reports ablations for the reference model\. These experiments isolate the contribution of metadata features, disagreement\-sensitive signals, cross\-attention, and multi\-task learning\. For all trainable variants, we report mean±\\pmstandard deviation over three random seeds \(42, 123, and 7\)\. The Pipeline axes→\\rightarrowconfounder setting is reused from the earlier pipeline experiments and is therefore reported as a single\-run result rather than a three\-seed average\.

Table 12:Ablation study for the reference model\. Results are reported as mean±\\pmstandard deviation over three seeds for all trainable variants\. The pipeline variant is a reused single\-run result from earlier tables\.VariantAccuracyMacro\-F1Axis Micro\-F1Dom\. Conf\. Acc\.Full model0\.588±\\pm0\.0080\.302±\\pm0\.0190\.215±\\pm0\.0040\.561±\\pm0\.009w/o metadata0\.584±\\pm0\.0090\.286±\\pm0\.0080\.215±\\pm0\.0050\.560±\\pm0\.009w/o disagreement signals0\.583±\\pm0\.0090\.286±\\pm0\.0060\.214±\\pm0\.0040\.564±\\pm0\.011w/o cross\-attention0\.581±\\pm0\.0040\.295±\\pm0\.0010\.212±\\pm0\.0030\.533±\\pm0\.012w/o multitask learning0\.589±\\pm0\.0100\.300±\\pm0\.0100\.069±\\pm0\.0170\.133±\\pm0\.160Independent axis predictors0\.343±\\pm0\.2760\.090±\\pm0\.0630\.268±\\pm0\.0060\.173±\\pm0\.112Pipeline axes→\\rightarrowconfounder0\.58320\.29300\.26080\.5689The ablations show that multi\-task learning is the most critical design choice in the reference model\. When the model is trained without multi\-task supervision, Axis Micro\-F1 drops from 0\.215 to 0\.069 and dominant\-confounder accuracy drops from 0\.561 to 0\.133, even though Task\-1 accuracy and Macro\-F1 remain close to the full model\. This pattern indicates that the joint training setup is not merely auxiliary, but a core mechanism for learning a useful shared representation across the structured contradiction tasks\.

By contrast, removing metadata features causes only a modest degradation, reducing Macro\-F1 from 0\.302 to 0\.286 while leaving Axis Micro\-F1 and dominant\-confounder accuracy nearly unchanged\. A similar pattern appears when disagreement\-sensitive signals are removed: Macro\-F1 decreases slightly from 0\.302 to 0\.286, but the remaining metrics stay effectively stable\. This suggests that the encoder already captures much of the information provided by these hand\-crafted comparison features, so they offer incremental rather than transformative gains\.

Cross\-attention has a more targeted effect\. Removing it leads to only small declines in Task\-1 accuracy and Macro\-F1, but dominant\-confounder accuracy falls more clearly from 0\.561 to 0\.533\. This indicates that cross\-attention is especially useful for modeling claim\-pair interactions needed by the confounder prediction task, even if its contribution to the other outputs is more limited\.

Finally, the structured prediction variants help clarify the value of joint modeling\. Independent axis predictors obtain a slightly higher Axis Micro\-F1 than the full model \(0\.268 versus 0\.215\), but they collapse on the main contradiction classification task and show very poor confounder performance, with high variance across seeds\. The pipeline axes→\\rightarrowconfounder setup achieves the strongest dominant\-confounder accuracy in the table \(0\.5689\), but because it is a single\-run reused result from earlier experiments and does not jointly optimize all outputs, it should not be interpreted as a replacement for the full multi\-task model\. Overall, the results support the full architecture as the best balanced design, with multi\-task learning providing the main source of performance gains and the other architectural components contributing smaller, task\-specific benefits\.

## Appendix DOracle and Controlled\-Reference Analyses

Because BioDivergence\-Silver\-v1\.0 is a structured benchmark with multiple dependent outputs, it is useful to estimate upper bounds under partially gold conditions\. Table[13](https://arxiv.org/html/2606.11208#A4.T13)reports four controlled\-reference analyses\.

When gold divergence axes are provided, dominant\-confounder prediction rises to 0\.6901, indicating that a substantial portion of confounder error is inherited from upstream axis uncertainty\. When reconciliation is conditioned on gold evidence spans, semantic similarity improves but structured slot overlap remains low, indicating that evidence access alone does not guarantee strongly structured reconciliation\. Conversely, when gold structured fields are provided, slot overlap becomes nearly perfect while semantic naturalness remains modest\. This follows trivially from the experimental design \(the template baseline is constructed to contain the gold slot tokens, and the slot\-overlap metric checks exactly those tokens\), so the row is best read as a construction\-verification sanity check rather than as a claim about the relative difficulty of structured fidelity versus generation quality\. Retrieval with oracle evidence reaches the expected ceiling\.

Table 13:Oracle and controlled\-reference analyses\. These results estimate upper bounds and controlled\-reference performance when gold structured inputs are provided to downstream components\. The “gold structured fields→\\rightarrowexplanation” row corresponds to the template\-baseline slot\-overlap number in Table[7](https://arxiv.org/html/2606.11208#A3.T7); we include it here as a construction\-verification sanity check, not as an independent capability measurement\.SettingMetric\(s\)ResultGold axes→\\rightarrowconfounderAccuracy / Exact Match0\.6901 / 0\.6901Gold spans→\\rightarrowreconciliationSemSim / Slot Overlap0\.6844 / 0\.1339Gold structured fields→\\rightarrowexplanationSemSim / Slot Overlap0\.3320 / 0\.9956Retrieval with oracle evidenceIoU / F11\.0000 / 1\.0000
## Appendix EFull Ontology and Label Definitions

This appendix documents the full ontology underlying BioDivergence and makes the benchmark labeling scheme explicit for reproducibility and error analysis\. Because BioDivergence is designed for structured contradiction analysis rather than flat sentence\-pair prediction, the benchmark requires both a conflict taxonomy and a divergence\-axis ontology\. The conflict taxonomy defines the type of incompatibility between two claims, while the divergence axes identify the contextual dimensions that may explain why apparently conflicting claims can simultaneously be valid under different conditions\. Together, these definitions provide the conceptual foundation for the benchmark and clarify how annotation decisions are mapped into structured labels\.

### E\.1Full Divergence\-Axis Ontology

Table[14](https://arxiv.org/html/2606.11208#A5.T14)lists the full set of divergence axes used in BioDivergence\. These axes capture contextual factors that frequently drive apparent disagreement in biomedical abstracts, including differences in population, geography, time period, protocol, study design, molecular background, and endpoint definition\.

#### Primary vs\. provisional axes in v1\.0\.

Based on the release statistics in Table[18](https://arxiv.org/html/2606.11208#A6.T18), we designate nine axes as*primary*\(sufficiently populated in the test split for reliable metrics, here≥40\\geq 40test\-set occurrences\):disease\_subtype,organism\_strain\_lineage,population\_cohort,clinical\_setting,year\_time\_period,geography,assay\_measurement\_protocol,endpoint\_definition, andstudy\_design\. The remaining four axes \(dosage\_intervention,gene\_mutation\_molecular\_background,sample\_source,unknown\_latent\_factor\) are retained as*provisional*for schema stability and future\-release growth, but we do not make headline evaluation claims on them in v1\.0\. Axis\-level metrics in the main paper are computed over the full ontology; a matching primary\-axis\-only variant is reported in Table[38](https://arxiv.org/html/2606.11208#A13.T38)and reproducible viascripts/compute\_primary\_axis\_metrics\.pyin the released codebase \([https://github\.com/eliashossain001/biodivergence](https://github.com/eliashossain001/biodivergence)\)\.

Table 14:Divergence axes in BioDivergence\.AxisDescriptionpopulation\_cohortDifferences in cohort composition or patient populationgeographyDifferences in region, country, or healthcare systemyear\_time\_periodDifferences in surveillance year or historical periodassay\_measurement\_protocolDifferences in laboratory protocol or measurement methodstudy\_designDifferences in trial design or observational setupdosage\_interventionDifferences in dosage, regimen, or intervention protocoldisease\_subtypeDifferences in subtype or clinical phenotypeorganism\_strain\_lineageDifferences in strain, lineage, or biological variantgene\_mutation\_molecular\_backgroundDifferences in gene, mutation, or molecular backgroundclinical\_settingDifferences in care setting or treatment environmentsample\_sourceDifferences in specimen type or source materialendpoint\_definitionDifferences in outcome definition or endpointunknown\_latent\_factorLikely contextual disagreement not identifiable from the abstractThe axis inventory in Table[14](https://arxiv.org/html/2606.11208#A5.T14)also reflects a practical annotation principle: annotators should mark only those axes that are supported by the abstract and materially relevant to the disagreement\. This constraint prevents over\-labeling and ensures that the structured annotations remain interpretable, reproducible, and useful for downstream modeling\.

### E\.2Conflict Taxonomy and Label Definitions

Table[15](https://arxiv.org/html/2606.11208#A5.T15)defines the six\-way conflict taxonomy used in BioDivergence\. The taxonomy separates true logical incompatibility from context\-dependent disagreement, missing\-context cases, weak\-evidence cases, and literature\-level controversy\. This distinction is important because not all conflicting biomedical claims should be treated as equally strong forms of contradiction\. In particular, the benchmark distinguishes direct contradictions, where two claims are mutually incompatible under matched conditions, from contextual contradictions, where the disagreement is better explained by changes in study conditions, cohorts, protocols, or other structured factors\.

Table 15:Six\-way conflict taxonomy used in BioDivergence\.ClassCore definitionDiagnostic signalBoundary ruleExample scenarioNo conflictClaims are compatible, complementary, or address different questions\.Agreement under paraphrase, compatible outcomes, or distinct endpoints\.Use when both claims can be true without tension\.One study reports reduced bacterial load; another reports tolerability\.Direct contradictionClaims make mutually incompatible assertions under the same conditions\.Same population, same endpoint, opposite conclusion\.If an explicit contextual factor explains the difference, use*contextual contradiction*\.The same therapy is reported effective versus ineffective in the same cohort\.Contextual contradictionClaims appear inconsistent, but the disagreement is explained by one or more contextual axes\.Different cohort, geography, time, assay, subtype, dosage, strain, or setting\.Use when each claim may be locally valid under different conditions\.A breakpoint performs well in one country but poorly in another\.Underspecified apparent contradictionClaims appear contradictory, but key methodological detail is missing\.Missing dosage, cohort, endpoint, assay, or design information\.If a concrete axis is recoverable from the text, prefer*contextual contradiction*\.Two studies report opposite effects, but one omits dosage and population details\.Evidence insufficiencyEvidence is too sparse or weak to support a confident judgment\.Small pilot studies, inconclusive trends, weak statistical support\.Represents weak evidence rather than merely missing context\.Two small exploratory studies provide unstable findings\.Unresolved scientific controversyClaims reflect an active scientific dispute with evidence on multiple sides\.Repeated contradictory findings across multiple studies\.Reserve for persistent literature\-level disputes\.A contested biomarker remains debated despite multiple studies\.The boundary rules in Table[15](https://arxiv.org/html/2606.11208#A5.T15)are included to reduce label ambiguity and improve annotation consistency\. In particular, the taxonomy prioritizes explanation over surface opposition: if the abstract provides a concrete contextual factor that can account for the disagreement, the preferred label is*contextual contradiction*rather than*direct contradiction*\. Conversely,*direct contradiction*should be reserved for cases where the claims remain incompatible after contextual matching\. This design encourages the benchmark to capture scientifically meaningful disagreement rather than only lexical opposition\.

Annotation preference rule\.When labels are ambiguous, prefercontextual\_contradictionif a concrete divergence axis is explicit\. Preferdirect\_contradictiononly when claims are mutually incompatible under matched conditions\.

### E\.3Divergence\-Axis Usage Guidance

While Table[14](https://arxiv.org/html/2606.11208#A5.T14)enumerates the axis inventory, annotators also need operational guidance for when and how to apply those labels in practice\. Table[16](https://arxiv.org/html/2606.11208#A5.T16)therefore provides usage guidance for each axis, including short definitions, common trigger cues, and representative examples\. This guidance is intended to make the ontology actionable during annotation and more interpretable during model analysis\.

Table 16:Divergence\-axis usage guidance in BioDivergence\.AxisDefinitionTrigger cuesExample scenarioPopulation / cohortDifference in patient group or inclusion criteria\.Age, severity, comorbidity, treatment history\.Pediatric versus adult cases show different effects\.GeographyDifference in region, country, or healthcare system\.Country names, hospitals, surveillance setting\.A breakpoint works in Greek ICUs but not in Thai surveillance isolates\.Year / time periodDifference in study era or surveillance year\.Year ranges, pre/post policy shifts\.A treatment appears effective in older cohorts but not later\.Assay / measurement protocolDifference in assay platform or protocol\.MIC method, PCR platform, calibration language\.Broth microdilution and disk diffusion yield different conclusions\.Study designDifference in experimental or observational setup\.Randomized, retrospective, cohort, case\-control\.A trial and a chart review disagree on efficacy\.Dosage / interventionDifference in dose, regimen, or schedule\.Dose, duration, route, combination therapy\.Low\-dose therapy succeeds while standard\-dose therapy fails\.Disease subtypeDifference in subtype, stage, or phenotype\.Molecular subtype, tumor stage, severity class\.A biomarker is prognostic in one subtype but not another\.Organism strain / lineageDifference in strain, lineage, serotype, or variant background\.Strain names, lineages, clades, serovars\.A drug is active against one lineage but not another\.Gene / mutation / molecular backgroundDifference in gene, mutation, or pathway background\.Mutation names, alleles, expression signatures\.A target works in wild\-type tumors but not mutant tumors\.Clinical settingDifference in care setting or treatment environment\.ICU, outpatient, inpatient, screening\.A rule works in ICU patients but not outpatient screening\.Sample sourceDifference in specimen type or material\.Blood, tissue, swab, stool, plasma\.A biomarker is measured in tissue in one study and plasma in another\.Endpoint definitionDifference in outcome definition or evaluation window\.Survival window, relapse definition, response criterion\.One study uses overall survival and another progression\-free survival\.Unknown latent factorA contextual driver is likely present but not recoverable from the abstract\.Omitted protocol detail or unexplained disagreement\.Two studies disagree, but the missing contextual variable is not explicit\.The examples and trigger cues in Table[16](https://arxiv.org/html/2606.11208#A5.T16)are intended to reduce inconsistency in axis assignment, especially in cases where multiple contextual factors are mentioned in the same abstract pair\. In such cases, annotators should prefer the smallest sufficient set of axes needed to explain the disagreement, rather than exhaustively marking every possible contextual difference\. This principle keeps the structured labels focused, interpretable, and aligned with the benchmark’s goal of identifying the most plausible drivers of scientific divergence\.

Axes should be assigned only when the abstract text provides evidence that the difference is material to the disagreement\. When multiple axes apply, the smallest sufficient set should be preferred\.

## Appendix FBenchmark Construction and Data Statistics

This appendix summarizes how BioDivergence was constructed at scale and documents the composition of the released benchmark\. In addition to the final released pairs, we report upstream corpus totals, claim extraction counts, and candidate\-pair mining statistics in order to make the benchmark construction process transparent\. We also provide release\-level label and axis distributions, together with summary statistics that help characterize the difficulty and skew of the final benchmark\.

### F\.1Corpus and Candidate\-Pair Construction

Table[17](https://arxiv.org/html/2606.11208#A6.T17)reports benchmark construction statistics by scientific domain\. For each domain, the table lists the number of abstracts collected from PubMed and Europe PMC, the merged abstract total, the number of extracted claims, the number of mined candidate claim pairs, and the number of final released pairs\. These numbers make clear that BioDivergence was not built from a small manually curated seed set, but from a large\-scale literature mining pipeline that first collected corpora, then extracted claims, then selected a fixed candidate\-pair pool for filtering and annotation\.

Table 17:Construction and release statistics by domain\.DomainPubMedEurope PMCAbstractsClaimsCandidate pairsFinal pairsAMR22,918106,485129,403226,56910,0002,500Oncology11,2906,51417,80486,77210,0002,500Infectious disease11,0387,40918,44774,05510,0002,500Genomics10,6396,59817,23759,43210,0002,500Clinical trials / epi11,2838,00619,28981,07910,0002,500Total67,168135,012202,180527,90750,00012,500As shown in Table[17](https://arxiv.org/html/2606.11208#A6.T17), the benchmark is built from 202,180 abstracts and 527,907 extracted claims across five domains, yielding 50,000 mined candidate pairs before downselection to 12,500 silver\-labelled pairs*in the initial pair\-level packaging*\. These 12,500 pairs are the starting pool for both release variants\. The primary article\-disjoint release \(11,865 pairs: 10,183 / 840 / 842\) is produced by re\-splitting this pool to enforce zero article / claim / pair overlap across splits; the legacy pair\-level variant \(8,750 / 1,250 / 2,500\) preserves the original pair\-level\-deduplicated packaging for backward comparability\. Upstream corpus sizes remain naturally uneven across domains, with AMR contributing the largest raw literature pool\.

### F\.2Released Benchmark Distribution \(Legacy Pair\-Level Variant\)

Table[18](https://arxiv.org/html/2606.11208#A6.T18)summarizes the distribution for thelegacy pair\-level variant\(ConflictTopology\-Silver\-v1\.0\-pairlevel/; 8,750 / 1,250 / 2,500\)\. It is retained here for backward comparability because several tables in the main paper and appendix are computed on this variant\. The*primary*article\-disjoint release \(ConflictTopology\-Silver\-v1\.0/; 10,183 / 840 / 842\) shares the same silver\-labelled pool with this variant but differs in its split construction; the primary release is summarized in Appendix[M](https://arxiv.org/html/2606.11208#A13)\. The three panels below show, for the legacy variant, conflict\-type counts by split, divergence\-axis counts by split, and overall descriptive statistics\.

Table 18:Distributions and summary statistics for thelegacy pair\-level variantof v1\.0 \(v1\.0\-pairlevel/; 8,750 / 1,250 / 2,500\)\. The primary article\-disjoint release \(10,183 / 840 / 842\) is summarized in Appendix[M](https://arxiv.org/html/2606.11208#A13)\.Conflict typeTrainDevTestNo conflict5,5798881,782Direct contradiction1457Contextual contradiction1,267337676Underspecified apparent contradiction1,839611Evidence insufficiency010Unresolved scientific controversy511324
AxisTrainDevTestPopulation / cohort312100168Geography1122868Year / time period1503271Assay / protocol1323262Study design812347Dosage / intervention513Disease subtype967254502Organism strain / lineage33393202Gene / mutation / molecular505Clinical setting24164121Sample source311Endpoint definition861946Unknown latent factor633
StatisticValueNotesMean claim length29\.49 tokensMedian 25Mean abstract length166\.07 tokensMedian 193Mean axes per example0\.351Computed over all 12,500 examples \(legacy variant\)Train / dev / test8,750 / 1,250 / 2,500Legacy pair\-level variant; primary is 10,183 / 840 / 842Most common dominant confounderDisease subtype1,221 labeled examplesNext most common confoundersPopulation / cohort, year / time period, geographyStrong contextual skew

The statistics in Table[18](https://arxiv.org/html/2606.11208#A6.T18)show that the released benchmark is strongly imbalanced by conflict label\. Most examples fall into*no conflict*, while*contextual contradiction*and*underspecified apparent contradiction*account for most of the remaining challenging cases\. By contrast, classes such as*direct contradiction*,*evidence insufficiency*, and*unresolved scientific controversy*are rare\. We do not interpret these frequencies as direct estimates of how often these phenomena occur in the biomedical literature; they reflect the interaction of our mining function, the pilot prompt, and the release\-time annotator on abstract\-level claim pairs\.

#### Evaluable classes in v1\.0\.

Using a conservative threshold ofntest≥100n\_\{\\text\{test\}\}\\geq 100examples for reliable F1 estimation, only*no\_conflict*\(n=1,782n\{=\}1\{,\}782\) and*contextual\_contradiction*\(n=676n\{=\}676\) are directly evaluable as headline classes\.*Unresolved\_scientific\_controversy*\(n=24n\{=\}24\) and*underspecified\_apparent\_contradiction*\(n=11n\{=\}11\) are reported for completeness but should be interpreted as exploratory\.*Direct\_contradiction*\(n=7n\{=\}7\) and*evidence\_insufficiency*\(n=0n\{=\}0\) are too sparse for reliable F1 in v1\.0 and are explicitly flagged in the benchmark card and headline table captions\. We further note that the training split contains 1,839 underspecified\-apparent examples while dev and test contain only 6 and 11 respectively: this asymmetry is a direct consequence of the heuristic top\-up contributing the bulk of that class to train, and it should not be interpreted as a conventional 70/10/20 split\.

The axis distribution in Table[18](https://arxiv.org/html/2606.11208#A6.T18)further shows that the most common structured drivers of disagreement are disease subtype, organism strain or lineage, population or cohort, clinical setting, and year or time period \(in order of total occurrences across train, dev, and test\)\. This contextual skew is scientifically plausible: many apparent contradictions in the biomedical literature arise because findings differ across subpopulations, molecular subtypes, strains, or historical study conditions rather than because one claim simply negates another\. In this sense, the benchmark is designed to test whether models can recover structured explanations for disagreement, not merely detect surface opposition\.

The summary statistics in the lower part of Table[18](https://arxiv.org/html/2606.11208#A6.T18)also help characterize the input space\. Claims are relatively short on average, with a mean length of 29\.49 tokens, while the associated abstracts are substantially longer, with a mean length of 166\.07 tokens\. The mean of 0\.351 axes per example indicates that most released examples do not carry many simultaneous contextual labels, which is consistent with the annotation policy of preferring the smallest sufficient set of explanatory axes\. Finally, the split sizes confirm that the release is balanced by domain across train, dev, and test, even though it remains deliberately skewed by conflict type and dominant confounder\.

Overall, Tables[17](https://arxiv.org/html/2606.11208#A6.T17)and[18](https://arxiv.org/html/2606.11208#A6.T18)show that BioDivergence combines large\-scale literature mining with a carefully controlled final release\. The construction process is broad enough to ensure domain diversity, while the released benchmark preserves the label imbalance and contextual structure that make real\-world scientific contradiction analysis difficult\.

## Appendix GPipeline Implementation Details

This appendix documents the end\-to\-end construction pipeline used to build BioDivergence\. We summarize the query\-pack design, normalization and deduplication rules, candidate\-pair mining procedure, canonical workflow, and the full benchmark\-construction path from raw abstracts to the released silver benchmark\. The query inventory is summarized in Table[19](https://arxiv.org/html/2606.11208#A7.T19), domain query motifs are illustrated in Table[20](https://arxiv.org/html/2606.11208#A7.T20), normalization operations are listed in Table[21](https://arxiv.org/html/2606.11208#A7.T21), and the overall pipeline stages are summarized in Table[22](https://arxiv.org/html/2606.11208#A7.T22)\.

### G\.1Query Pack Structure

The four newly collected domains use YAML query packs designed to balance broad biomedical recall with contradiction\-sensitive retrieval\. Each pack contains four query groups:broad\_recall,contradiction,outcome\_comparison, andcontext\_sensitive\. As shown in Table[19](https://arxiv.org/html/2606.11208#A7.T19), the query packs contain 168 total queries across the four YAML\-backed domains, with relatively even coverage across retrieval purposes\. This design ensures that the collected corpus contains both high\-volume domain material and sufficient contradiction\-rich evidence for downstream pair mining\.

Table 19:Query pack structure for the four newly collected domains\.DomainBroadContradictionComparisonContextTotalOncology121081040Infectious disease141081042Genomics131081041Clinical trials / epi1212101145Total51423441168
### G\.2Representative Query Motifs

Table[20](https://arxiv.org/html/2606.11208#A7.T20)shows representative query motifs used during collection\. These motifs illustrate how each domain was targeted with domain\-adapted retrieval language while preserving the same overall design logic\. For example, oncology queries emphasize biomarkers, survival, and therapy response, whereas genomics queries emphasize variant interpretation, replication, and CRISPR\-related evidence\. The goal is not only topical coverage, but also retrieval of studies likely to yield contradiction\-bearing claim pairs\.

Table 20:Representative query motifs used for collection\.DomainRepresentative query motifAMRAnti\-bacterial agents with microbial resistance, susceptibility, breakpoint, or colistin\-toxicity language\.OncologyNeoplasms with antineoplastic agents, immunotherapy, biomarkers, survival, or resistance language\.Infectious diseaseCommunicable diseases with anti\-infective agents, treatment outcome, failure, or vaccine language\.GenomicsGenomics, GWAS, or variant interpretation with pathogenicity, replication, CRISPR, or expression language\.Clinical trials / epiClinical trials or epidemiology with negative/null results, reproducibility, heterogeneity, or endpoint debates\.
### G\.3Normalization and Deduplication

Before claim extraction, all retrieved abstracts undergo a deterministic normalization and deduplication stage\. As summarized in Table[21](https://arxiv.org/html/2606.11208#A7.T21), this stage includes encoding repair, Unicode normalization, inline\-tag and citation cleanup, whitespace normalization, minimum\-length filtering, and hierarchical deduplication by DOI, PMID, and content hash\. These operations reduce formatting noise, remove cross\-source duplicates, and produce a stable canonical text representation for downstream sentence splitting, claim extraction, and pair mining\.

Table 21:Normalization and deduplication operations\.OperationImplementationEffectEncoding repair and Unicode normalizationTextCleaner/ NFKC normalizationCorrects common text artifacts while preserving biomedical content\.Inline tag removal and citation cleanupRegex\-based cleaningRemoves legacy entity markers and bracketed references\.Whitespace normalizationCollapse repeated spaces and newlinesProduces a stable canonical text form\.Minimum\-length filteringMinimum abstract length of 50 charactersDrops very short abstracts\.Hierarchical deduplicationDOI→\\rightarrowPMID→\\rightarrowcontent hashRemoves cross\-source duplicates and near\-duplicates\.
### G\.4Candidate\-Pair Mining

Candidate\-pair mining begins from the extracted claim inventory and retrieves semantic neighbours within each domain usingall\-MiniLM\-L6\-v2embeddings\. Pairs are then scored by a disagreement\-aware combination of semantic similarity, disagreement cues, entity overlap, claim\-type compatibility, and a publication\-year penalty:

scombined=0\.35​ssim\+0\.35​sdis\+0\.15​sent\+0\.10​stype−pyear\.s\_\{\\text\{combined\}\}=0\.35\\,s\_\{\\text\{sim\}\}\+0\.35\\,s\_\{\\text\{dis\}\}\+0\.15\\,s\_\{\\text\{ent\}\}\+0\.10\\,s\_\{\\text\{type\}\}\-p\_\{\\text\{year\}\}\.The score prioritises pairs that are both semantically related and plausibly contradictory\. Pairs are retained when semantic similarity is at least 0\.40 and the combined score is at least 0\.25, with a cap of 10,000 pairs per domain\. The aggregate effect of this stage is reflected in the candidate\-pair counts reported in Table[22](https://arxiv.org/html/2606.11208#A7.T22)\.

### G\.5Canonical Workflow

The construction and evaluation process is implemented as a sequence of standalone scripts, shown below, that mirror the benchmark pipeline from corpus preparation to final evaluation\. This workflow provides a reproducible path for rebuilding the benchmark artifacts, regenerating intermediate outputs, and rerunning the released experiments\.

```
scripts/prepare_corpus.py
scripts/build_claims.py
scripts/mine_pairs.py
scripts/build_annotation_queue.py
scripts/build_dataset.py
scripts/run_experiment.py
scripts/run_full_evaluation.py
```

### G\.6End\-to\-End Pipeline Overview

Table[22](https://arxiv.org/html/2606.11208#A7.T22)summarizes the full benchmark\-construction pipeline\. The process begins with corpus collection from PubMed and Europe PMC, followed by deterministic cleaning and deduplication, rule\-based claim extraction, disagreement\-aware candidate\-pair mining, LLM\-assisted silver annotation, and deterministic split packaging\. The table reports the verified output of each stage: the pipeline scales from 202,180 unique abstracts to 527,907 claims, 50,000 candidate pairs, and 12,500 silver\-labelled pairs at the initial pair\-level packaging stage\. The final released primary variant \(BioDivergence\-Silver\-v1\.0\) is produced from this pool by an article\-disjoint re\-split step \(see Appendix[M](https://arxiv.org/html/2606.11208#A13)\); the pair\-level packaging output is retained as the legacy v1\.0\-pairlevel variant for backward comparability\.

With the exception of silver relabeling, the pipeline is deterministic once the input sources, query packs, and seed are fixed\. This makes the construction process largely reproducible and isolates the LLM\-based annotation stage as the main non\-deterministic component\.

Table 22:End\-to\-end benchmark construction stages in BioDivergence\.StageImplementationMechanismVerified outputCorpus collectionscripts/prepare\_corpus\.pyAPI\-driven collection from PubMed and Europe PMC plus legacy AMR import\.202,180 unique abstracts after deduplication\.Normalization and deduplicationingestion/cleaning\.py,ingestion/dedup\.pyDeterministic text cleaning and DOI/PMID/content\-hash deduplication\.202,180 normalized abstracts\.Claim extractionscripts/build\_claims\.py,claims/pipeline\.pyHeuristic rule\-based sentence classification with slot extraction\.527,907 claims from 139,792 abstracts\.Candidate\-pair miningscripts/mine\_pairs\.py,pairs/scoring\.pyDeterministic embedding retrieval and disagreement\-sensitive ranking\.50,000 candidate pairs\.Silver annotation / relabelingscripts/run\_silver\_annotation\.py,silver/\*LLM\-assisted structured annotation with heuristic fallback\.12,500 labeled pairs\.Legacy pair\-level packagingSplit\-building and release packagingDeterministic pair\-level\-deduplicated split construction\.8,750 / 1,250 / 2,500 \(v1\.0\-pairlevel; retained for backward comparability\)\.Article\-disjoint re\-split \(primary\)scripts/build\_strict\_split\.pyConnected\-component assignment of articles to splits; duplicatepair\_idresolution with LLM\-labelled preferred\.10,183 / 840 / 842 \(v1\.0 primary; 11,865 unique pairs, zero article/claim/pair overlap across splits\)\.

## Appendix HClaim Extraction Details and Claim Statistics

This appendix summarizes the rule\-based claim extraction stage used to produce the claim inventory for candidate\-pair mining\. The section reports the global claim\-type distribution, domain\-level claim counts, illustrative extraction decisions, and the structure of the extracted claim records\. Table[23](https://arxiv.org/html/2606.11208#A8.T23)summarizes the semantic distribution of extracted claims, Table[24](https://arxiv.org/html/2606.11208#A8.T24)reports the domain\-level claim counts, Table[25](https://arxiv.org/html/2606.11208#A8.T25)provides representative accepted and filtered sentences, and Table[26](https://arxiv.org/html/2606.11208#A8.T26)describes the fields stored for each extracted claim\.

### H\.1Claim Type Distribution

Table[23](https://arxiv.org/html/2606.11208#A8.T23)reports the global claim\-type distribution across all five domains\. The extracted corpus is dominated by*factual*claims, which account for 121,618 instances \(23\.0%\), followed by*efficacy*,*prognostic*, and*correlational*claims, each contributing roughly 14% of the total\.*Prevalence*and*causal*claims form a secondary tier, while*mechanistic*and*recommendation*claims are less frequent\. This distribution indicates that the benchmark is centered on descriptive and outcome\-oriented biomedical statements rather than only one narrow claim category\.

Table 23:Global claim type distribution across all five domains\.Claim typeCount \(%\)Factual121,618 \(23\.0%\)Efficacy74,701 \(14\.2%\)Prognostic74,915 \(14\.2%\)Correlational74,294 \(14\.1%\)Prevalence66,400 \(12\.6%\)Causal51,862 \(9\.8%\)Mechanistic38,047 \(7\.2%\)Recommendation26,070 \(4\.9%\)
### H\.2Extracted Claims by Domain

Table[24](https://arxiv.org/html/2606.11208#A8.T24)reports the number of extracted claims by domain\. AMR contributes the largest share, with 226,569 claims, followed by oncology, clinical trials / epidemiology, infectious disease, and genomics\. In total, the extraction pipeline yields 527,907 claims across the five target domains, providing broad biomedical coverage while preserving strong representation of the AMR literature that originally motivated the benchmark\.

Table 24:Extracted claims by domain\.DomainClaimsAMR226,569Oncology86,772Infectious disease74,055Genomics59,432Clinical trials / epi81,079Total527,907
### H\.3Illustrative Claim Extraction Decisions

Claim extraction is implemented as a deterministic rule\-based stage over the normalized abstract text\. Candidate sentences are scored using lexical and structural cues associated with biomedical claims, and only sufficiently strong claim\-bearing sentences are retained\. Table[25](https://arxiv.org/html/2606.11208#A8.T25)shows representative examples of accepted sentences and filtered sentences\. In general, the extractor retains substantive biomedical findings, such as outcome, resistance, or prevalence statements, while filtering background summaries, methods descriptions, and speculative follow\-up remarks\.

Table 25:Illustrative claim extraction decisions\.Accepted claim sentenceFiltered sentenceThe intervention significantly improved survival in the treatment arm\.Background: We summarize prior work on the pathway\.The isolate was resistant to colistin in the surveillance cohort\.Methods: We retrospectively reviewed 120 records\.No significant difference in mortality was observed between groups\.Further studies are needed to confirm the mechanism\.The prevalence of resistance was 18% in the exposed group\.This paper presents the study design and setting\.
### H\.4Extracted Record Structure

Each extracted claim is stored with both provenance and structured metadata so that the later pair\-mining and annotation stages can operate on a consistent record format\. Table[26](https://arxiv.org/html/2606.11208#A8.T26)summarizes the main field groups, including identity and provenance fields, text and label fields, sentence offsets, structured content such as entities and slots, and quality\-related metadata\. This representation supports both deterministic claim tracking and downstream structured evaluation\.

Table 26:Extracted claim record structure\.Field groupContentsIdentity and provenanceclaim\_id,article\_uid,domain,sourceText and labelsentence,claim\_text,claim\_typeOffsetssentence\_start,sentence\_endStructured contententities,slotsQuality and metadataconfidence,pub\_year,journalClaim extraction is deterministic once the normalized corpus is fixed\. Overall, the released corpus yields 527,907 claims from 139,792 abstracts, corresponding to a 69\.1% abstract\-level yield and an average of 2\.61 claims per abstract\. Together, Tables[23](https://arxiv.org/html/2606.11208#A8.T23)–[26](https://arxiv.org/html/2606.11208#A8.T26)show that the extracted claim set spans multiple biomedical statement types and all five target domains while preserving enough structure for reproducible pair mining and annotation\.

## Appendix ISilver Annotation Protocol and Prompt Design

This appendix describes the structured silver\-labeling protocol used to convert candidate claim pairs into ontology\-constrained benchmark records\. We summarize the pilot comparison between heuristic and LLM\-based annotation, the fields stored in the annotation artifacts, the prompting and validation logic, and the run\-level provenance files retained for reproducibility\. Table[27](https://arxiv.org/html/2606.11208#A9.T27)reports the pilot comparison, while Table[29](https://arxiv.org/html/2606.11208#A9.T29)summarizes the main annotation and benchmark\-record artifacts\.

### I\.1Silver Pilot Comparison

Table[27](https://arxiv.org/html/2606.11208#A9.T27)compares a deterministic heuristic annotator with the LLM\-based structured annotator on a 100\-example pilot set\. The heuristic system is heavily concentrated inno\_conflictand uses almost none of the available contextual structure, whereas the LLM annotator produces a much broader distribution over the label space, substantially higher coverage of divergence axes and dominant confounders, and far higher mean confidence\. These results motivated the use of structured LLM\-based silver annotation in the full release, while preserving the heuristic annotator as a fallback and comparison point\.

Table 27:Silver pilot annotation comparison: heuristic vs\. LLM annotator \(n=100n\{=\}100\)\. Pilot annotator:Qwen/Qwen2\.5\-72B\-Instruct\.MetricHeuristicLLM \(Qwen2\.5\-72B\)no\_conflict8562contextual\_contradiction337direct\_contradiction01underspecified\_apparent120Unique divergence axes used1 of 1312 of 13Dominant confounder coverage3 / 10035 / 100Mean annotator confidence0\.3980\.916#### Release\-time annotator model\.

The full\-release silver relabeling was executed withqwen2\.5:7bserved locally via Ollama, not with the 72B model used in the pilot\. The choice was made for throughput and cost reasons; the release manifest \(split\_manifest\.json\) and the relabeling directories \(relabeling\_medium\_v1\_ollama\_qwen25\_7b,relabeling\_booster\_v1\_ollama\_qwen25\_7b\) record the provider, model, and prompt version used\. We document this model swap explicitly so that users of the benchmark can reason about annotator capacity separately from annotator family\. The same prompt template \(Prompts[I\.4](https://arxiv.org/html/2606.11208#A9.SS4)and[I\.4](https://arxiv.org/html/2606.11208#A9.SS4)\) was used in both the pilot and the release\.

#### Pilot\-equivalent re\-run of the 7B annotator\.

To quantify the quality gap between the pilot 72B annotator and the release\-time 7B annotator, we re\-runQwen/Qwen2\.5\-7B\-Instruct\(the HuggingFace checkpoint backingqwen2\.5:7bon Ollama\) on a fresh 100\-example stratified subset drawn from the primary test\+dev pool with the identical prompt template\. The 72B pilot’s per\-example outputs were not retained in the release repository, so a direct per\-example Cohen’sκ\\kappaagainst the 72B pilot is not available; the comparison is therefore reported at the aggregate\-distribution level \(Table[28](https://arxiv.org/html/2606.11208#A9.T28)\), with an auxiliary agreement check between the 7B audit labels and the release\-time 7B labels as a reproducibility proxy for the annotator itself \(κ=0\.456\\kappa=0\.456at temperature 0\.3; this is*not*a capability measurement of 7B, only a sanity check on the release annotator\)\. The aggregate comparison indicates that the 7B annotator tracks the 72B pilot reasonably well on the two populated classes \(55 vs\. 62*no\_conflict*; 43 vs\. 37*contextual\_contradiction*\) and on mean confidence \(0\.905 vs\. 0\.916\), but departs on structural fields: it assigns a dominant confounder on 100% of pairs \(vs\. 35% for the 72B pilot\) and returns 20 unique divergence\-axis strings in the 100\-example subset, some of which are off\-schema \(the ontology has 13 axes\)\. These off\-schema axis strings were not filtered in this audit because the audit was run directly on raw model outputs for diagnostic purposes; in the full release pipeline, the schema\-validation stage insrc/conflicttopology/silver/annotator\.pyconstrains all axis values to the 13\-axis ontology \(\_VALID\_AXES\) and drops any out\-of\-vocabulary string before benchmark assembly, so the 20 off\-schema strings observed here are an upper bound on annotator vocabulary drift rather than a count of corrupted labels in the released benchmark\. We interpret the 7B annotator as usable for release\-scale silver labelling on the two primary classes, but note that its confounder coverage and axis\-vocabulary fidelity are materially noisier than the 72B pilot’s\. This is an additional motivation for the v1\.1 multi\-annotator pass recommended in the conclusion\.

Table 28:Pilot\-equivalent annotator quality gap\. Columns report the Qwen2\.5\-7B audit distribution \(100\-example stratified subset of the primary test\+dev pool, re\-run for this submission\), the published Qwen2\.5\-72B pilot distribution reproduced from Appendix[I](https://arxiv.org/html/2606.11208#A9), and the deterministic heuristic baseline\. We do not report a per\-example Cohen’sκ\\kappabetween 7B and 72B because the 72B pilot output files were not retained in the release repository; the final row instead reports agreement between the 7B audit labels and the release\-time labels produced by the same 7B annotator, serving as a reproducibility check on the released annotator\.MetricHeuristicLLM \(7B audit\)LLM \(72B pilot\)no\_conflict855562contextual\_contradiction34337direct\_contradiction011underspecified\_apparent1200unresolved\_scientific\_controversy010evidence\_insufficiency000Unique divergence axes used \(of 13\)12012Dominant confounder coverage \(of 100\)310035Mean annotator confidence0\.3980\.9050\.916*7B audit vs\. release labels \(both Qwen2\.5\-7B\):*raw agreement 0\.700, Cohen’sκ\\kappa= 0\.456
#### Same\-family interpretation of Qwen3 baselines\.

Because the release is labelled by a Qwen2\.5 annotator \(7B at release scale, 72B in the pilot\), our zero\-shot Qwen3\-8B numbers in Tables[4](https://arxiv.org/html/2606.11208#S7.T4),[5](https://arxiv.org/html/2606.11208#S7.T5),[7](https://arxiv.org/html/2606.11208#A3.T7),[8](https://arxiv.org/html/2606.11208#A3.T8),[9](https://arxiv.org/html/2606.11208#A3.T9), and[10](https://arxiv.org/html/2606.11208#A3.T10)should be read as an upper bound on agreement between a same\-family evaluator and the labels produced by its release annotator, rather than as a purely external capability measurement\. Mistral\-7B\-v0\.3 is reported alongside as a non\-Qwen reference\.

### I\.2Structured Annotation Fields

Each annotation stage writes structured artifacts rather than only free\-form text outputs\. Table[29](https://arxiv.org/html/2606.11208#A9.T29)summarizes the main artifacts used during queue construction, annotation, adjudication, and benchmark assembly\. Together, these records preserve both the benchmark labels themselves and the provenance needed to reproduce or audit the annotation pipeline\. In particular, the separation betweenAnnotationTask,AnnotatorResponse, and release\-facing benchmark records makes it possible to trace how a mined claim pair becomes a finalized structured example\.

Table 29:Structured annotation and benchmark\-record fields\.ArtifactFieldsAnnotationTasktask\_id,pair\_id,domain,claim\_a\_text,claim\_b\_text,claim\_a\_article\_uid,claim\_b\_article\_uid,claim\_a\_type,claim\_b\_type,abstract\_a,abstract\_b,pub\_year\_a,pub\_year\_b,journal\_a,journal\_bAnnotatorResponsetask\_id,pair\_id,annotator\_id,comparable,conflict\_type,divergence\_axes,dominant\_confounder,reconciliation,evidence\_spans\_a,evidence\_spans\_b,confidence,notes,timestampAdjudicatedExampleExample ID, pair metadata, claims, article UIDs, entities, gold labels, reconciliation, evidence spans, annotator IDs, adjudication notes, publication metadata, and mining scoresannotation\_manifest\.jsonQueue identifier, provider, model name, prompt version, sample count, and run\-level countsprovenance\.jsonLightweight run provenance and backend settingsraw\_responses\.jsonlRaw model outputs and per\-attempt tracesfailures\.jsonlFailed annotation tasks and retry historysplit\_manifest\.jsonLeakage mode, heuristic confidence threshold, source files, and per\-domain counts
### I\.3Run Artifacts and Provenance

For reproducibility, the annotation pipeline records run\-level artifacts in a structured form\. These includeannotation\_manifest\.json, which stores queue and model metadata;provenance\.json, which captures run\-level backend settings;raw\_responses\.jsonl, which preserves model outputs and per\-attempt traces;failures\.jsonl, which records failed or retried examples; andsplit\_manifest\.json, which documents the settings used for final split construction\. Together, these artifacts make the silver\-labeling process auditable and easier to reproduce\. All run artifacts, annotation manifests, and provenance files are included in the released codebase at[https://github\.com/eliashossain001/biodivergence](https://github.com/eliashossain001/biodivergence)\.

### I\.4Prompt Design

The silver annotator uses a single prompt version \(v1\) across pilot, development, and full annotation queues; only the sampled tasks change by tier\. Because the codebase does not include a separate curated example set, the appendix reproduces the exact prompt templates rather than synthetic examples\. The system prompt and user prompt are shown in Prompt[I\.4](https://arxiv.org/html/2606.11208#A9.SS4)and Prompt[I\.4](https://arxiv.org/html/2606.11208#A9.SS4)\.

`Prompt 1: System prompt template``Prompt 2: User prompt template`

## Appendix JTask Schema and Implementation Details

This appendix summarizes the benchmark record format, the evaluation baselines, and the training configuration of the reference model\. We first describe the input and output schema used across tasks, then outline the model families used for benchmarking, and finally report the optimization settings for the shared multi\-task reference system\. Table[30](https://arxiv.org/html/2606.11208#A10.T30)summarizes the record schema, Table[31](https://arxiv.org/html/2606.11208#A10.T31)lists the evaluated baselines, and Table[32](https://arxiv.org/html/2606.11208#A10.T32)reports the reference\-model training configuration\.

### J\.1Benchmark Input and Output Schema

BioDivergence is designed as a structured benchmark rather than a single\-label classification dataset\. Each example contains paired biomedical claims together with provenance, metadata, and multiple target outputs\. As shown in Table[30](https://arxiv.org/html/2606.11208#A10.T30), the input record stores the paired claims, their source identifiers, claim types, entities, disagreement signals, and publication metadata\. The gold outputs then supervise five complementary tasks: conflict\-type prediction, divergence\-axis extraction, dominant\-confounder selection, reconciliation generation, and claim\-level evidence\-span extraction\. This schema allows the same benchmark instance to support both discriminative and generation\-oriented evaluation\.

Table 30:Benchmark input and output schema\.Record typeFields and constraintsInput exampleexample\_id,pair\_id,domain,claim\_a\_text,claim\_b\_text, claim IDs, article UIDs, claim types, entities, disagreement score, publication metadataGold outputconflict\_type,divergence\_axes,dominant\_confounder,reconciliation,evidence\_spansEvidenceSpanarticle\_uid,text, zero\-basedstart, exclusiveend, androleAnnotation responsecomparable,conflict\_type,divergence\_axes,dominant\_confounder,reconciliation, evidence spans, confidence, notes
### J\.2Model Families and Baselines

To evaluate BioDivergence across multiple reasoning styles, we include simple frequency baselines, lexical and feature\-based systems, transfer models, retrieval methods, zero\-shot instruction\-tuned LLMs, and a dedicated reference model\. Table[31](https://arxiv.org/html/2606.11208#A10.T31)groups these systems by task family\. The benchmark therefore tests not only raw predictive accuracy, but also how different modeling paradigms behave under structured biomedical disagreement\. For example, zero\-shot LLMs are evaluated on both classification and generation tasks, while retrieval\-oriented methods are emphasized for claim\-level evidence\-span extraction\.

Table 31:Model families and evaluation baselines used in BioDivergence\.Task familyBaselines / model namesImplementation noteTask 1 conflict typeMajority\-class, stratified\-random, SlotMismatch GBT; DeBERTa NLI; BioBERT NLI; zero\-shot Qwen3\-8B and Mistral\-7B\-v0\.3Deterministic feature baseline, cross\-encoder, or zero\-shot JSON prompt\.Task 2 axesMost\-frequent\-axis heuristic, independent per\-axis predictors, zero\-shot LLM reuseMulti\-label extraction with thresholded logits\.Task 3 confounderFrequency baseline, pipeline axes→\\rightarrowconfounder, zero\-shot LLM reuseConstrained single\-axis prediction\.Task 4 reconciliationExtractive baseline, template baseline, zero\-shot Qwen3\-8B and Mistral\-7B\-v0\.3BLEU/ROUGE/SemSim/slot\-overlap evaluation\.Task 5 claim\-level evidence\-span extractionLexical span heuristic, BM25, dense retrieval, zero\-shot Qwen3\-8B and Mistral\-7B\-v0\.3Token\-overlap span metrics\.Reference modelMultiTaskConflictModel with BiomedBERT encoderShared backbone for Tasks 1–3\.
### J\.3Reference\-Model Training Configuration

The reference model is a shared multi\-task architecture built on a biomedical transformer encoder\. It jointly learns conflict\-type prediction, divergence\-axis extraction, and dominant\-confounder prediction using a common representation backbone with task\-specific heads\. Table[32](https://arxiv.org/html/2606.11208#A10.T32)reports the main optimization settings\. We use BiomedBERT as the encoder, sequence length 256, AdamW optimization, early stopping on macro\-F1, and weighted task losses to balance the three supervised objectives\. Optional constraint modes allow confounder prediction to use either predicted axes or gold axes during controlled analyses\.

Table 32:Reference\-model training configuration\.SettingValueEncodermicrosoft/BiomedNLP\-BiomedBERT\-base\-uncased\-abstract\(encoder hidden size 768\)Input format\[CLS\] claim\_a \[SEP\] claim\_b \[SEP\]Max sequence length256Post\-encoder projection / type\-embed / attention heads256 / 32 / 8 \(projection dimension on top of the 768\-dim encoder output\)Batch size / eval batch size16 / 32Optimizer / LR / weight decayAdamW / 2e\-5 / 0\.01Epochs / warmup / grad norm8 / 0\.1 / 1\.0Early stoppingPatience 3 on macro\-F1Seeds42, 123, 7 \(three\-seed mean / std in main tables\)Task loss weights \(released full Task 1\+2\+3 run\)λc​t=1\.0\\lambda\_\{ct\}=1\.0,λa​x=1\.0\\lambda\_\{ax\}=1\.0,λd​c=0\.3\\lambda\_\{dc\}=0\.3\(configs/experiments/task123\_full\.yaml\)Task loss weights \(default / Task 1\+2 sweep baseline\)λc​t=1\.0\\lambda\_\{ct\}=1\.0,λa​x=0\.5\\lambda\_\{ax\}=0\.5,λd​c=0\.3\\lambda\_\{dc\}=0\.3\(configs/base\.yaml\)Imbalance handlingTrain\-split class weights and axis positive weights enabled in the released runConstraint modenone,predicted\_axes, orgold\_axes\(released run usespredicted\_axes\)

## Appendix KEvaluation Protocol Details

This appendix summarizes the auxiliary evaluation protocols used for reconciliation generation, claim\-level evidence\-span extraction, robustness analysis, and counterfactual evaluation\. Table[33](https://arxiv.org/html/2606.11208#A11.T33)lists the main metrics used for explanation and span\-extraction tasks, while Tables[34](https://arxiv.org/html/2606.11208#A11.T34)and[35](https://arxiv.org/html/2606.11208#A11.T35)summarize the controlled edit templates used in robustness and counterfactual experiments\.

### K\.1Explanation and Span\-Extraction Metrics

BioDivergence evaluates explanation quality and claim\-level span extraction separately because these two abilities capture different aspects of biomedical reasoning\. As shown in Table[33](https://arxiv.org/html/2606.11208#A11.T33), reconciliation generation is evaluated using lexical overlap \(BLEU, ROUGE\-L\), semantic similarity \(SemSim, computed withsentence\-transformers/all\-MiniLM\-L6\-v2at release time\), and slot overlap with the structured benchmark fields\. Span extraction is evaluated using token\-level IoU, precision, recall, and F1 over the predicted versus silver \(claim\-verbatim\) evidence spans\. The separation is intentional: a model may produce a fluent reconciliation without accurately picking the supporting span, and conversely may select the span well without producing a strong explanation\.

Table 33:Evaluation signals for explanation generation and claim\-level evidence\-span extraction\.TaskMetricWhat it checksReconciliationBLEU, ROUGE\-LLexical overlap with the reference explanationReconciliationSemSimSentence\-level semantic similarityReconciliationSlot overlapWhether the explanation names the structured factorsClaim\-span extractionIoU, precision, recall, F1Token overlap between predicted and silver \(claim\-verbatim\) evidence spans
### K\.2Robustness and Counterfactual Templates

The robustness and counterfactual analyses both rely on controlled contextual edits, but they test different properties\. Robustness perturbations modify a single contextual factor to evaluate whether model predictions remain stable yet appropriately sensitive\. Counterfactual alignments instead neutralize a suspected source of disagreement and test whether the generated reconciliation changes in the expected way\. Table[34](https://arxiv.org/html/2606.11208#A11.T34)summarizes the perturbation templates used in the reference\-model robustness analysis, and Table[35](https://arxiv.org/html/2606.11208#A11.T35)summarizes the alignment templates used for counterfactual reconciliation evaluation\.

Table 34:Robustness perturbation templates used in the reference\-model analysis\.Edit typeTarget axisTemplate wordingExpected effectGeography swapGeographyReplace region cues with a different region or setting\.Test geographic sensitivityTime shiftYear / time periodShift dates, era, or surveillance period\.Test temporal sensitivityAssay changeAssay / measurement protocolReplace the assay or measurement method\.Test protocol sensitivityCohort changePopulation / cohortReplace the patient group or demographic profile\.Test cohort sensitivityEndpoint changeEndpoint definitionReplace the endpoint or outcome definition\.Test outcome sensitivityTable 35:Counterfactual alignment templates used for reconciliation evaluation\.Edit typeTarget axisInstructionExpected effectGeography alignmentGeographyAssume both claims were conducted in the same region\.Geography should no longer explain the tensionTime\-period alignmentYear / time periodAssume both claims refer to the same study period\.Temporal difference should be removedAssay alignmentAssay / measurement protocolAssume both claims used the same assay\.Assay difference should be removedPopulation alignmentPopulation / cohortAssume both claims studied the same cohort\.Cohort difference should be removedDisease\-subtype alignmentDisease subtypeAssume both claims refer to the same subtype\.Subtype difference should be removed

## Appendix LQualitative Examples and Release Notes

This appendix provides representative qualitative examples and summarizes key limitations of the current release\. Table[36](https://arxiv.org/html/2606.11208#A12.T36)shows illustrative cases that align with the BioDivergence ontology and demonstrate how contextual contradiction differs from simple logical opposition\. These examples highlight typical disagreement patterns involving geography, subtype, protocol, and non\-conflict cases, and help clarify how the benchmark should be interpreted qualitatively in addition to quantitatively\.

Table 36:Worked qualitative examples\.ExampleConflict type / axesDominantReconciliationWhy not simple contradictionColistin breakpoint in Greek ICUs versus Thai surveillance isolatesContextual contradiction; geography \+ clinical setting \+ organism strain / lineageGeographyDifferent geography and surveillance setting explain the breakpoint mismatch\.The claims are locally compatible under different contexts\.Subtype\-sensitive biomarker resultContextual contradiction; disease subtypeDisease subtypeThe biomarker behaves differently across subtypes\.The disagreement is subtype\-specific, not global\.Method\-variation susceptibility resultContextual contradiction; assay / measurement protocolAssay / measurement protocolDifferent protocols yield different susceptibility conclusions\.The contradiction disappears when the assay changes\.No\-conflict comparison caseNo conflict; noneNullThe claims are complementary and address different outcomes\.They answer different biomedical questions\.The current release remains a silver benchmark rather than a fully expert\-adjudicated resource\. Future versions would benefit from a larger human\-validated subset, stronger representation of rare classes such as direct contradiction and evidence insufficiency, and richer document\-level evidence sources beyond abstracts\. These limitations should be considered when interpreting benchmark scores and when designing future extensions of the resource\.

## Appendix MArticle\-Disjoint Primary Split: Construction and Audit

The primary BioDivergence\-Silver\-v1\.0 release is*article\-disjoint*: it guarantees zero article, zero claim, and zeropair\_idoverlap across train, dev, and test \(Section[5](https://arxiv.org/html/2606.11208#S5)\)\. A legacy pair\-level variant \(ConflictTopology\-Silver\-v1\.0\-pairlevel/\) is retained for backward comparability only \(Table[1](https://arxiv.org/html/2606.11208#S5.T1)\); it is not recommended for headline evaluation\. The primary release is built by resolvingpair\_idduplicates and label\-conflicting duplicates \(LLM\-labelled preferred over heuristic\-labelled\) and by keeping connected components of articles \(linked through any shared claim pair\) intact within a single split, placing components per\-domain at a 70/15/15 target ratio by example count\.

### M\.1Pipeline and Deduplication

The primary split is produced byscripts/build\_strict\_split\.py\. It operates on the union of the pair\-level legacy train, dev, and test JSONL files, resolves duplicatepair\_ids \(635 observed, of which 591 had conflicting labels across the duplicated copies\), builds an undirected graph of articles connected through shared claim pairs, and places entire components into one of the three splits\. The resulting primary release contains 11,865 unique pairs after deduplication\. Table[37](https://arxiv.org/html/2606.11208#A13.T37)reports leakage statistics for both the primary \(article\-disjoint\) and legacy \(pair\-level\) variants\.

Table 37:Leakage comparison between the primary article\-disjoint release \(v1\.0\) and the legacy pair\-level variant \(v1\.0\-pairlevel\) retained for backward comparability\. “Article overlap” is the number of test rows whose A\-side or B\-side source abstract also appears in the training split\. The primary release enforces zero overlap on every row below\.Statisticv1\.0 \(primary, article\-disjoint\)v1\.0\-pairlevel \(legacy\)Total examples1186512500Train / dev / test10183 / 840 / 8428750 / 1250 / 2500Test rows with either article in train0\.0%93\.6%Test rows with both articles in train0\.0%52\.0%Distinct test claim\-pairs also in train0 / 842318 / 2445Overlappingpair\_ids \(train∩\\captest\)0201
### M\.2Reference Model and Zero\-Shot Performance

We train the full\-feature reference model on the primary \(article\-disjoint\) training set using the released configuration \(configs/experiments/task123\_strict\_split\.yaml\) across three seeds \(42, 123, 7\) and evaluate on the primary test set\. For a non\-Qwen zero\-shot comparison independent of the silver\-annotator family, we also runmistralai/Mistral\-7B\-Instruct\-v0\.3on the full 842\-example primary test set with greedy decoding\. The resulting headline comparison table is hoisted into the main body as Table[2](https://arxiv.org/html/2606.11208#S7.T2)\(Section[7](https://arxiv.org/html/2606.11208#S7)\); we do not duplicate it here\. The full run completed in 5,499\.13 seconds \(≈\\approx91\.7 minutes; 6\.53 seconds/example\)\.

### M\.3Primary\-Axis \(9\-Axis\) Evaluation

Because four of the thirteen divergence axes are sparsely populated in v1\.0 and therefore not evaluable individually \(Appendix[E\.1](https://arxiv.org/html/2606.11208#A5.SS1)\), we report multi\-label axis metrics under two label spaces: the full 13\-axis ontology and the primary 9\-axis subset \(population\_cohort,geography,year\_time\_period,assay\_measurement\_protocol,study\_design,disease\_subtype,organism\_strain\_lineage,clinical\_setting,endpoint\_definition\)\. The primary subset is chosen by a fixed threshold ofntest≥40n\_\{\\text\{test\}\}\\geq 40on the v1\.0 release and is therefore not selected post hoc from strict\-split performance\. Table[38](https://arxiv.org/html/2606.11208#A13.T38)reports both\.

Table 38:Multi\-label divergence\-axis metrics for the reference model on the primary article\-disjoint test split, reported on the full 13\-axis label space and on the 9 primary axes \(test\-count≥40\\geq 40in the legacy pair\-level test\)\. Four provisional axes \(dosage\_intervention,gene\_mutation\_molecular\_background,sample\_source,unknown\_latent\_factor\) are excluded from the primary subset\.Axis spaceMicro\-F1Macro\-F1Exact set matchnFull 13 axes0\.195±\\pm0\.0090\.116±\\pm0\.0020\.623±\\pm0\.057842Primary 9 axes0\.195±\\pm0\.0090\.168±\\pm0\.0030\.623±\\pm0\.057842
### M\.4Non\-Qwen Annotator\-Family Sensitivity Audit

As a sensitivity audit on annotator\-family dependence, we re\-annotated a stratified 1000\-example subset of the primary test\+dev pool withmeta\-llama/Llama\-3\.1\-8B\-Instructusing the identical silver prompt template \(system prompt, user prompt, six\-class output schema\)\. This is a*claims\-only*audit: the release JSONL records do not carry the source abstracts, so the prompt renders missing abstracts as “\(not available\)”\. Llama\-3\.1\-8B is strictly weaker than the release annotator \(qwen2\.5:7bvia Ollama\) in parameter count and was run without the abstract context the release annotator had; the audit therefore measures*how much*of the release label structure is recoverable from claims alone by a non\-Qwen\-family model, not whether the release labels themselves are correct\. Table[39](https://arxiv.org/html/2606.11208#A13.T39)reports raw agreement, Cohen’sκ\\kappa, and per\-class agreement \(one\-vs\-rest\)\.

Table 39:Annotator\-family sensitivity audit: label agreement between the release silver annotator \(qwen2\.5:7bvia Ollama, abstracts present\) and a non\-Qwen audit annotator \(meta\-llama/Llama\-3\.1\-8B\-Instruct, claims\-only\) on 998 parseable records from a stratified 1000\-example subset of the primary v1\.0 test\+dev pool\. This is a sensitivity check, not a validation of v1\.0 labels: the audit annotator is strictly weaker and operates without the abstract context\. Overall raw agreement: 0\.644; Cohen’sκ\\kappa: 0\.196\.ClassnQwenn\_\{\\text\{Qwen\}\}nLlaman\_\{\\text\{Llama\}\}nagreen\_\{\\text\{agree\}\}Recall Llama∣\\midQwenPrec\. Llama∣\\midQwenJaccardno\_conflict6977605840\.8380\.7680\.669contextual178109500\.2810\.4590\.211underspec\.1195070\.0590\.1400\.043unresolved2000\.0000\.0000\.000direct16611\.0000\.0150\.015evidence\_insuff\.11311\.0000\.0770\.077Overall998998643raw agree\. 0\.644;κ\\kappa0\.196–The overall Cohen’sκ\\kappais 0\.20 \(fair; raw agreement 64%\)\. We emphasise that the audit is*not*validation of the release labels and we do not interpret the numbers as construct validity of v1\.0\. Three observations, all read narrowly: \(i\) majority\-class agreement drives the 64% raw number: the two annotators agree on 584 / 697 Qwen*no\_conflict*positives \(Jaccard 0\.67\); \(ii\) on*contextual\_contradiction*, Jaccard is 0\.21 and Llama under\-predicts the class \(109 Llama positives vs\. 178 Qwen\), consistent with a claims\-only audit having strictly less context than the abstracts\-present labelling pass; \(iii\) Llama is more liberal thanqwen2\.5:7bon the rare classes; it flags 66*direct\_contradiction*s and 13*evidence\_insufficiency*s in the 1000\-example subset, versus 1 each under the Qwen release labels, which is consistent with the release annotator being conservative on these classes and is compatible with, though not proof of, the interpretation that rare\-class sparsity in v1\.0 partly reflects that conservatism rather than absence from the candidate pool\. No v1\.0 labels are modified by this audit\. A multi\-annotator pass with abstracts present \(needed for either validation or rare\-class lift\) is planned for v1\.1\.

### M\.5Interpretation

Because the primary split closes the article\-level leakage vector entirely, the delta against the legacy pair\-level variant cleanly measures how much of the legacy pair\-level score came from article\-level overlap rather than task learning\. Three observations stand out: \(i\) the reference model’s contextual\-contradiction F1 drops by≈\\approx12 points under article\-disjoint training and evaluation \(0\.521→0\.401±0\.0170\.521\\rightarrow 0\.401\\pm 0\.017over three seeds\), identifying that share of the pair\-level number as article\-supported; \(ii\) the zero\-shot Mistral\-7B baseline, which cannot memorise training articles because it is never fine\-tuned, is evaluated on the full 842\-example primary test set and reaches 0\.3894 contextual\-F1, which is slightly below the reference model’s 0\.4011 on the same split and well below the legacy pair\-level reference score of 0\.5210\. That keeps the non\-Qwen comparison on the full primary split without article leakage while showing the signal is still recoverable by an annotator\-family\-independent baseline; \(iii\) overall accuracy andκ\\kappafor the reference model actually*improve*under the article\-disjoint primary, because the three catastrophically sparse classes \(ntest≤7n\_\{\\text\{test\}\}\{\\leq\}7in the pair\-level variant:*direct\_contradiction*,*evidence\_insufficiency*,*unresolved\_scientific\_controversy*\) haventest=0n\_\{\\text\{test\}\}=0in the primary test set, leaving the two populated classes as a cleaner evaluation target\. The primary\-axis \(9\-axis\) view provides an analogous axis\-level summary that is less dominated by the four provisional axes and is the recommended headline axis reporting space for v1\.0\.