RegDivergence-101: An LLM Benchmark for Cross-Jurisdiction Regulatory Contradiction Detection in Life Sciences

# An LLM Benchmark for Cross-Jurisdiction Regulatory Contradiction Detection in Life Sciences
Source: [https://arxiv.org/html/2608.28607](https://arxiv.org/html/2608.28607)
\(2026\)

###### Abstract\.

Pharmaceutical sponsors developing a drug for both the United States and the European Union must reconcile guidance issued independently by the FDA and the EMA\. Where the two agencies requiresubstantively the same thing, a sponsor can file once; where theydiverge, a single trial design risks rejection in one region; where one agency issilenton a point the other regulates, the sponsor must infer obligations\. Today this reconciliation is performed manually by regulatory\-affairs experts\. We introducecross\-jurisdiction regulatory divergence detection: given an FDA requirement and an EMA requirement on the same topic, classify their relationship as AGREE, DIVERGE, or SILENT\. SILENT is inherently directional \(SILENT\_FDA vs\. SILENT\_EMA\); we record direction per pair and report per\-direction F1 alongside the collapsed label\. We releaseRegDivergence\-101, a 101\-pair expert\-groundedpilotevaluation benchmark \(labels grounded in three peer\-reviewed FDA/EMA comparison studies and primary FDA/EMA/ICH guidance text; dual\-annotation inter\-annotatorκ=0\.85\\kappa=0\.85\), and systematically characterise a four\-method baseline hierarchy: lexical heuristic \(0\.511 macro\-F1, 95% CI \[0\.411–0\.605\]\), NLI cross\-encoder \(0\.233\), obligation\-level Graph\-RAG \(0\.663 \[0\.570–0\.747\]\), and flat LLM judge / Claude Haiku \(0\.830 \[0\.747–0\.908\]\)\. Three directional observations emerge at pilot scale \(n=101n=101\): SILENT is semantically detectable but invisible to entailment\-only formulations; pair\-level obligation graphs improve over lexical methods but trail flat\-LLM context \(CIs partially overlapping\); and corpus\-level graph construction is the indicated architectural target for large\-scale silent\-detection\. RegDivergence\-101 is a pilot release establishing the task formulation and baseline hierarchy; four unrepresented regulatory domains and an expansion roadmap are described in §[7](https://arxiv.org/html/2608.28607#S7)\.

large language models, LLM benchmark, regulatory NLP, AI for compliance, FDA/EMA divergence, contradiction detection, Graph\-RAG, retrieval\-augmented generation, generative AI, life sciences, AGREE/DIVERGE/SILENT classification

††copyright:acmlicensed††journalyear:2026††doi:XXXXXXX\.XXXXXXX††conference:2026 2nd International Conference on Artificial Intelligence and Foundation Model; June 26–28, 2026; Urumqi, China††isbn:979\-8\-4007\-2458\-9††ccs:Computing methodologies Lexical semantics††ccs:Applied computing Health informatics## 1\.Introduction

Bringing a medicine to market in both the US and EU requires satisfying two regulators that publish guidance separately and frequentlydifferently\. Gene\-therapy sponsors face a 15\-year long\-term follow\-up recommendation from FDA while EMA defers the duration to case\-by\-case risk assessment—same topic, materially different specificity\. Analogous divergences span pediatric extrapolation, endpoint definitions in ulcerative colitis, biosimilar comparability, and estimand implementation\. A sponsor who misses one divergence during protocol design can lose a year or more re\-running a study\.

Reconciling the two corpora is currently a manual, expert task\. Yet the NLP building blocks for automating it all exist—document\-level NLI \(ContractNLI\(Koreeda and Manning,[2021](https://arxiv.org/html/2608.28607#bib.bib1)\), DocNLI\(Yinet al\.,[2021](https://arxiv.org/html/2608.28607#bib.bib2)\)\), legal contradiction detection \(LegalWiz\(Mantravadiet al\.,[2025](https://arxiv.org/html/2608.28607#bib.bib3)\), CLAUSE\(Roy Choudhuryet al\.,[2026](https://arxiv.org/html/2608.28607#bib.bib4)\)\), and Graph\-RAG \(Microsoft GraphRAG\(Edgeet al\.,[2024](https://arxiv.org/html/2608.28607#bib.bib6)\), GraphCompliance\(Chunget al\.,[2025](https://arxiv.org/html/2608.28607#bib.bib8)\), RAGulating Compliance\(Jomrajet al\.,[2025](https://arxiv.org/html/2608.28607#bib.bib7)\)\)\.No prior work combines them to detect divergence between two parallel regulatory corpora from different jurisdictions\.All existing contradiction work isintra\-document\(within one contract\) orsingle\-jurisdiction\(contract\-vs\-statute\)\.

This paper makes three contributions: \(1\)Task formulation— three\-way classification \(AGREE / DIVERGE / SILENT\) over pre\-aligned FDA/EMA requirement pairs; SILENT treated as directional \(SILENT\_FDA / SILENT\_EMA\) with per\-direction F1 in Table[2](https://arxiv.org/html/2608.28607#S5.T2)\. \(2\)Pilot benchmark—RegDivergence\-101, 101 pairs across 13 topic areas with a documented two\-track construction protocol,κ=0\.85\\kappa=0\.85, and bootstrap CIs; expansion roadmap in §[7](https://arxiv.org/html/2608.28607#S7)\. \(3\)Baseline hierarchy— four methods compared at pilot scale \(n=101n=101\) with bootstrap 95% CIs, yielding three directional observations on the task’s structure\.

## 2\.Related Work

Document\-level NLI\.ContractNLI\(Koreeda and Manning,[2021](https://arxiv.org/html/2608.28607#bib.bib1)\)introduced document\-level entailment/contradiction/not\-mentioned classification with evidence extraction, and showed that contradiction detection lags entailment, especially under “negation by exception\.” DocNLI\(Yinet al\.,[2021](https://arxiv.org/html/2608.28607#bib.bib2)\)extended NLI to full documents\.

Legal contradiction detection \(2025–2026\)\.LegalWiz\(Mantravadiet al\.,[2025](https://arxiv.org/html/2608.28607#bib.bib3)\)uses a multi\-agent framework for legal contradiction detection but is confined to single\-document regimes; Better Call CLAUSE\(Roy Choudhuryet al\.,[2026](https://arxiv.org/html/2608.28607#bib.bib4)\)benchmarks 7,500\+ perturbed contracts and finds LLMs struggle to justify subtle errors legally\. A published RAG\-robustness study directly informs our results:Yoranet al\.\([2024](https://arxiv.org/html/2608.28607#bib.bib26)\)show that retrieved context which is irrelevant or non\-entailing can mislead a retrieval\-augmented model unless it is explicitly filtered—the failure mode in Observation 2, where modal\-register similarity misleads the pair\-level graph\.

Graph\-RAG and regulatory compliance\.Microsoft GraphRAG\(Edgeet al\.,[2024](https://arxiv.org/html/2608.28607#bib.bib6)\)builds entity graphs with community summaries for local and global queries\. GraphCompliance\(Chunget al\.,[2025](https://arxiv.org/html/2608.28607#bib.bib8)\)alignspolicy graphs\(regulatory requirements\) withcontext graphs\(organisational facts\) for GDPR; it explicitly lists “handling conflicting regulatory interpretations” as open\. RAGulating Compliance\(Jomrajet al\.,[2025](https://arxiv.org/html/2608.28607#bib.bib7)\)builds an ontology\-free regulatory knowledge graph for traceable QA\.

Cross\-jurisdiction regulatory benchmarks\.The closest concurrent work isSino\-US\-DrugQA\(Sino\-US\-DrugQA authors,[2026](https://arxiv.org/html/2608.28607#bib.bib15)\)\(11,871 MCQA items, US–CN\), which shares our concept\-alignment thesis but cannot express the SILENT class\. Comparative\-regulatory\-science surveys document FDA/EMA divergence across dosing\(Mita and others,[2026](https://arxiv.org/html/2608.28607#bib.bib16)\), oncology labelling\(Gómez\-Fernández and others,[2026](https://arxiv.org/html/2608.28607#bib.bib17)\), and multi\-country SmPCs\(Sartoriet al\.,[2026](https://arxiv.org/html/2608.28607#bib.bib18)\), but as human\-authored comparisons, not ML benchmarks\. RegDivergence\-101 is the first FDA/EMA benchmark with an explicit SILENT class\. In high\-stakes sectors such as healthcare, policy\-aligned evaluation frameworks weigh safety and regulatory constraints over raw accuracy\(Chenet al\.,[2026](https://arxiv.org/html/2608.28607#bib.bib25)\); RegDivergence\-101 extends this to cross\-jurisdiction regulatory reasoning\.

Gap\.Contradiction detection is intra\-document or single\-jurisdiction; Graph\-RAG targets single\-corpus QA or single\-jurisdiction compliance; FDA/EMA divergence is documented manually but never automated\. Our task sits precisely in this gap\.

## 3\.Task and Dataset

### 3\.1\.Task Definition

For a topictt, letftf\_\{t\}be an FDA requirement statement andete\_\{t\}the corresponding EMA statement\. The labelyty\_\{t\}is:

- •AGREE— both jurisdictions require substantively the same thing \(same modal strength, same threshold, same scope of applicability\)\. Paraphrase equivalence and synonymous modals \(shall/must\) are treated as AGREE; minor drafting differences that do not alter sponsor obligations are disregarded\.
- •DIVERGE— the requirements directly conflict: they impose different thresholds, different modal strengths \(shallvs\.should\), different timelines, or mutually incompatible design specifications\.
- •SILENT— one jurisdiction does not address what the other regulates\. SILENT isdirectional:SILENT\_FDA\(FDA silent, EMA regulates\) vs\.SILENT\_EMA\(EMA silent, FDA regulates\)\. Direction is recorded in thesilent\_directionfield and reported in Table[2](https://arxiv.org/html/2608.28607#S5.T2); a unified SILENT label is retained in Table[1](https://arxiv.org/html/2608.28607#S5.T1)for backward compatibility\.

Epistemic status of SILENT labels\.A SILENT label is an absence claim: it asserts that no counterpart requirement was found after an exhaustive corpus search\. This carries irreducible epistemic uncertainty\. We bound this uncertainty through \(a\) a documented corpus\-search protocol \(§[3\.3](https://arxiv.org/html/2608.28607#S3.SS3)\) and \(b\) a per\-pairdoc\_confidenceflag \(HIGH / MODERATE\)\. Downstream work should treat MODERATE\-confidence SILENT pairs as soft labels\.

Task scope\.This formulation assumes requirement pairs arepre\-aligned—both sides address the same regulatory topic, established via the two\-track protocol \(§[3\.2](https://arxiv.org/html/2608.28607#S3.SS2)\)\. Topic alignment is a non\-trivial upstream problem treated separately; the corpus\-level architecture that would subsume it is described in §[7](https://arxiv.org/html/2608.28607#S7)\.

### 3\.2\.Pair Construction Protocol

RegDivergence\-101 is constructed via two tracks that differ in how topical correspondence between FDA and EMA requirements is established\. Per\-pair source\-track attribution corresponds to the two released source files \(one per track\)\.

Track A — expert\-comparison\-derived pairs \(59 pairs\)\.Extracted from three open\-access peer\-reviewed studies that comparatively mapped FDA and EMA requirements:Vieujeanet al\.\([2025](https://arxiv.org/html/2608.28607#bib.bib9)\)\(ulcerative colitis, 40 pairs\),Boesenet al\.\([2021](https://arxiv.org/html/2608.28607#bib.bib11)\)\(psychiatric drug trials, 10 pairs\), andSchwarz and Decristoforo \([2019](https://arxiv.org/html/2608.28607#bib.bib12)\)\(radiopharmaceuticals, 9 pairs\)\. Domain experts in those studies had already established co\-topicality; our task was to classify the AGREE/DIVERGE/SILENT relationship between their pre\-aligned pairs\.

Track B — primary guidance pairs \(42 pairs\)\.For pairs from ICH E9\(R1\), E11A\(International Council for Harmonisation,[2024](https://arxiv.org/html/2608.28607#bib.bib13)\), and FDA/EMA biosimilar and gene\-therapy guidance, co\-topicality was established by \(1\) anchoring requirements to a shared ICH Common Technical Document \(CTD\) module/sub\-topic, then \(2\) expert confirmation that the candidate pair was genuinely co\-topical\.

Specificity exclusion criterion\.Candidate pairs operating at different regulatory hierarchy levels \(e\.g\.,statistical analysis planvs\.general statistical principles\) were excluded to prevent conflating level\-of\-detail differences with substantive DIVERGE labels\. A machine\-readable list of excluded pairs is not part of the pilot release\.

The released pilot dataset is two JSONL files \(one per track\) with fields\{id, topic, source, fda\_text, ema\_text, label\}, wherelabel∈\\in\{AGREE, DIVERGE, SILENT\-FDA, SILENT\-EMA\}—SILENT direction is encoded in the label \(basis for Table[2](https://arxiv.org/html/2608.28607#S5.T2)\)\. The101 pairssplit 42 AGREE, 38 DIVERGE, 21 SILENT \(13 SILENT\-FDA, 8 SILENT\-EMA\), across 13 topics\. Thedoc\_confidence\(uniformly MODERATE\),disputed, andannotation\_notesannotations below are not yet separate fields in the pilot files\. We report macro\-F1\.

### 3\.3\.Annotation Protocol

Annotators\.Annotator A \(C\. Wu\) is a solution architect for a regulatory\-document AI platform; Annotator B \(Z\. Zhou\) is a product manager for an AI\-powered clinical\-trial management platform\. Neither holds formal RA credentialing; Track A expert\-panel papers serve as the external human\-expert anchor\.

Stages\.Stage 1: Wu assigned labels via definitions\-based framing; Stage 2: Zhou independently re\-labelled all 101 pairs under a sponsor\-obligation framing to probe framing\-independent stability\. Theκ\\kappais a cross\-framing consistency measure, not identical\-instruction IAA\. An independent third\-annotator pass is planned for RegDivergence\-500\.

IAA\.Human cross\-framingκ=0\.85\\kappa=0\.85\(101 pairs, observed 0\.90\)—substantialperLandis and Koch \([1977](https://arxiv.org/html/2608.28607#bib.bib14)\), on par with MedNLI \(0\.65\) and ContractNLI \(≈\\approx0\.72\)\. SILENT near\-perfect \(κ=0\.98\\kappa=0\.98\); AGREE/DIVERGE more variable \(≈\\approx0\.79–0\.81\)\. A secondary LLM consistency check \(36\-pair sample\) yieldedκ=0\.542\\kappa=0\.542\(test\-retest, not human IAA\)\.

Disagreements \(n = 10\)\.Six resolved by source\-paper language; four genuinely borderline pairs retain the Stage 1 label, with written justifications recorded in the response\-to\-reviewers \(not adisputed field in the pilot files\)\.

SILENT confidence\.All 21 SILENT pairs are treated as MODERATE confidence \(uniform value, not a released field\): exhaustive corpus search was performed \(full FDA and EMA guidance, Q&As, reflection papers\) but absence claims remain epistemically soft\. The HIGH/MODERATE audit is deferred to RegDivergence\-500\.

## 4\.Baselines

### 4\.1\.Evaluation Protocol

RegDivergence\-101 is anevaluation benchmark, not a training dataset\. All baselines are zero\-shot or few\-shot; results are reported as a ranking trend \(Δ=0\.15\\Delta=0\.15–0\.600\.60macro\-F1 between adjacent methods, CIs partially overlapping\), not pairwise significance\.

All methods usestratified 5\-fold splits\(5 seeds\) withbootstrap 95% CI\(1,000 resamples\)\. Lexical and NLI methods usenestedCV \(inner folds tune thresholds\); Graph\-RAG and LLM judge have no tunable hyperparameters \(LLM judge scored on all 101 pairs, bootstrap CI only\)\.

### 4\.2\.Lexical Heuristic

No neural model, no API—fully offline and deterministic\. Combines TF\-IDF cosine similarity with five conflict signals: negation asymmetry, divergence cue pairs \(e\.g\.,single\-armvs\.randomised\), modal\-strength asymmetry \(must/shallvs\.may/encouraged\), numeric\-threshold mismatch, and one\-sided specificity \(concrete number vs\. case\-by\-case deferral\)\. A shared ICH citation is a strong AGREE signal\.

### 4\.3\.NLI Cross\-Encoder

typeform/distilbert\-base\-uncased\-mnli, run in both directions; labels derived from contradiction/neutral/entailment probabilities with CV\-tuned thresholds\. Any NLI model lacking an explicit SILENT class maps absence\-of\-regulation toneutral, structurally collapsing SILENT recall \(Observation 1 in §[6](https://arxiv.org/html/2608.28607#S6)\); stronger encoders improve AGREE/DIVERGE F1 but cannot resolve this gap without task\-specific supervision\.

### 4\.4\.Graph\-RAG Pair\-Level Classifier

For each text, Claude Haiku extracts a policy triplet⟨\\langlesubject, obligation\_level, requirement, conditions⟩\\ranglewhereobligation\_level∈\{MANDATORY,PROHIBITED,RECOMMENDED,PERMITTED,SILENT\}\\text\{obligation\\\_level\}\\in\\\{\\text\{MANDATORY\},\\text\{PROHIBITED\},\\text\{RECOMMENDED\},\\text\{PERMITTED\},\\text\{SILENT\}\\\}\. Only MANDATORY↔\\leftrightarrowPROHIBITED is a hard\-coded DIVERGE rule; all other pairs are escalated to a second LLM call that reasons over theobligation structureof the aligned node pair, with explicit calibration that modal\-register differences \(RECOMMENDED vs\. MANDATORY\) do not automatically imply DIVERGE\.

Within\-model scope\.Both the Graph\-RAG classifier and the flat LLM judge useclaude\-haiku\-4\-5\-20251001, so their comparison is awithin\-model ablation\(graph vs\. no\-graph\), not a cross\-family architectural claim, to be confirmed on a second model family\.

### 4\.5\.LLM Judge \(Flat Pairwise\)

Claude Haiku with explicit AGREE / DIVERGE / SILENT definitions, prompted once per pair with both texts\. No graph, no retrieval\. Full prompt and code at Appendix[A](https://arxiv.org/html/2608.28607#A1)\.

## 5\.Results

Table[1](https://arxiv.org/html/2608.28607#S5.T1)reports macro\-F1, accuracy, and per\-class F1 with bootstrap 95% confidence intervals for all four methods\.

Table 1\.Macro\-F1, accuracy, and per\-class F1 with bootstrap 95% confidence intervals for the four baselines \(n=101n=101pairs\)\.1Nested 5\-fold CV, 5 seeds; thresholds tuned on inner folds only\.2Same CV protocol, thresholds tuned on inner folds\.3No tunable hyperparameters; bootstrap CI only\.4claude\-haiku\-4\-5\-20251001, default temperature; bootstrap CI only\. Estimated cost:∼\\scriptstyle\\sim$0\.10 for all 101 pairs\.

Table 2\.Per\-direction SILENT F1 for all four methods \(bootstrap 95% CI,n=101n=101pairs\)\. SILENT\_FDA \(n=13n=13\): FDA is silent, EMA regulates\. SILENT\_EMA \(n=8n=8\): EMA is silent, FDA regulates\.SILENT\-FDA: 13 pairs \(FDA silent, EMA active\)\. SILENT\-EMA: 8 pairs \(EMA silent, FDA active\)\. Wide CIs on SILENT\-EMA reflect the small stratum \(n=8n=8\)\. Bootstrap 1,000 resamples; stratified by 3\-class label\.

Point estimates increase monotonically lexical→\\toGraph\-RAG→\\toLLM judge; bootstrap CIs partially overlap atn=101n=101, so we treat this as a ranking trend rather than a significance claim\.Lexical ceiling:errors share the signature ofhigh lexical overlap, opposed regulatory stance\(e\.g\., FDA “modified Mayo score of 5–9” vs\. EMA “full Mayo score of 9–12”; same vocabulary, different instrument\), requiring semantic understanding of what each requirement asserts\.

## 6\.Analysis

### 6\.1\.What the Baseline Hierarchy Reveals

Observation 1 — SILENT is semantically detectable but invisible to NLI framing\.SILENT F1 = 0\.025 under NLI \(no native SILENT class\) vs\. 0\.778 under the LLM judge \(explicit definition\)\. This contrast reflects a structural constraint—the SILENT class requires a formulation that can express absence, which NLI entailment cannot—rather than a model\-capacity effect \(a stronger encoder cannot close the gap without task\-specific supervision\)\.

Observation 2 — For Claude Haiku on this dataset, pair\-level obligation graphs improve over lexical \(\+0\.15 F1\) but trail flat prompting \(−\-0\.17 F1\); CIs partially overlap at pilot scale\.The graph makes obligation structure explicit, helping distinguish stance differences from similar\-sounding text\. But it discards full\-text context that flat prompting uses to separatesame\-requirement / different\-modal\-register\(AGREE\) fromsame\-topic / different\-threshold\(DIVERGE\): gene\-01 \(FDA 15 years vs\. EMA case\-by\-case\) both extract as RECOMMENDED and the graph calls them AGREE; the flat LLM correctly identifies the threshold conflict from full text\(Yoranet al\.,[2024](https://arxiv.org/html/2608.28607#bib.bib26)\)\.

Observation 3 — Two distinct problem scopes suggest two distinct architectural targets\.Thepair\-level scope\(given pre\-aligned sentences, label AGREE/DIVERGE/SILENT\) is largely addressed by the flat LLM judge \(0\.830\)\. Thecorpus\-level scope\(“is EMA silent anywhere across its corpus?”\) requires finding aligned pairs, not just classifying them; a corpus\-level policy graph is needed\. §[7](https://arxiv.org/html/2608.28607#S7)motivates this architecture as the next contribution\.

### 6\.2\.Source\-Stratified Performance Analysis

To assess whether performance reflects source\-study\-specific patterns rather than genuine task competence, we analyse performance stratified by track and by source concentration\.

Track\-level\.On AGREE/DIVERGE classes, the LLM judge scoresloweron Track A \(0\.769/0\.745\) than Track B \(0\.966/0\.923\): exposure to source studies does not inflate performance\.

We omit a track\-stratified macro\-F1 comparison: Track B has no SILENT pairs \(n=0n=0vs\. 21 in Track A\), so its macro\-F1 averages an empty class and is not comparable to Track A’s; the per\-class comparison above already shows no inflation\.

Source\-concentration\.40/101 pairs come from one source \([Vieujeanet al\.](https://arxiv.org/html/2608.28607#bib.bib9)\)\. LLM judge on the JCC stratum \(n=40n=40\): 0\.804; on all other pairs \(n=61n=61\): 0\.844\. Performance is marginallyhigheroutside JCC, ruling out domain\-concentration inflation\.

## 7\.Future Work

Corpus\-level policy graph\.The correct long\-term target is corpus\-level silent detection: \(1\) build per\-jurisdiction policy graphs indexing all FDA/EMA guidance; \(2\) align requirement nodes via entity normalisation and embedding similarity; \(3\) classify aligned pairs as AGREE/DIVERGE/SILENT by obligation\-structure comparison; \(4\) generate natural\-language explanations\. This architecture subsumes the pair\-alignment assumption: alignment becomes a graph\-construction step, not a pre\-processing input\.

RegDivergence\-500\.Four domains are unrepresented in the pilot: CMC, adaptive trial design/estimands, statistical analysis plan requirements, and labelling/SmPC\. RegDivergence\-500 \(≈\\approx500 pairs\) will cover all four via a community\-annotation pipeline seeded from a∼\\sim45\-source FDA/EMA comparison inventory\. Annotation guidelines are released with this paper\. Multi\-model replication, an independent third\-annotator validation pass, and a cross\-pair generalisation probe against Sino\-US\-DrugQA\(Sino\-US\-DrugQA authors,[2026](https://arxiv.org/html/2608.28607#bib.bib15)\)are planned; a third jurisdiction \(PMDA\) will add an≈\\approx40\-pair FDA×\\timesPMDA subset\(Mita and others,[2026](https://arxiv.org/html/2608.28607#bib.bib16); Gómez\-Fernández and others,[2026](https://arxiv.org/html/2608.28607#bib.bib17)\)\.

## 8\.Limitations

Scale and alignment\.101 pairs from three source studies limits statistical power to directional observations\. Track B co\-topicality judgements are single\-annotator; Track A inherits source\-study alignment methodology\. The specificity exclusion criterion is described in §[3\.2](https://arxiv.org/html/2608.28607#S3.SS2); a machine\-readable exclusion checklist is not part of the pilot release\. Oncology, rare diseases, medical devices, and vaccines are absent; RegDivergence\-500 \(§[7](https://arxiv.org/html/2608.28607#S7)\) is required before domain\-general conclusions\.

SILENT labels\.All 21 SILENT pairs are uniformly treated as MODERATE confidence; treat them as soft labels\. The pair\-level SILENT task is a proxy for the correct corpus\-level formulation \(§[7](https://arxiv.org/html/2608.28607#S7)\)\.

Single\-model evaluation\.All LLM\-dependent components use Claude Haiku, so the Graph\-RAG vs\. LLM comparison is a within\-model ablation and theκ=0\.542\\kappa=0\.542check reflects one model’s tendencies\. Pretraining contamination cannot be excluded, though §[6\.2](https://arxiv.org/html/2608.28607#S6.SS2)shows no inflation; multi\-model replication and post\-cutoff evaluation are deferred to RegDivergence\-500\.

Use norms\.RegDivergence\-101 is a pilot: not sufficient for deployment or AI\-procurement decisions, not domain\-generalizable without held\-out evaluation, and specific to the FDA/EMA pair until cross\-pair probes are run\.

## 9\.Conclusion

We introduced cross\-jurisdiction regulatory divergence detection and releasedRegDivergence\-101, a 101\-pair expert\-groundedpilotbenchmark \(κ=0\.85\\kappa=0\.85, two\-track construction protocol, per\-direction SILENT annotation\) with a four\-method baseline hierarchy\. Three directional observations at pilot scale: SILENT requires explicit absence\-aware formulation \(NLI F1 0\.03→\\toLLM F1 0\.78\); for Claude Haiku, pair\-level graph structure improves over lexical \(\+0\.15 F1\) but trails flat prompting \(−\-0\.17 F1\); and the pair\-level vs\. corpus\-level scope distinction motivates the corpus\-level graph architecture \(§[7](https://arxiv.org/html/2608.28607#S7)\) as the next contribution\. RegDivergence\-500 will establish the task at production scale\.

###### Acknowledgements\.

This work was conducted independently\. Benchmark pairs were derived from publicly available FDA/EMA guidance documents and open\-access peer\-reviewed comparison studies\. LLM API calls used the Anthropic API \(Claude Haiku,claude\-haiku\-4\-5\-20251001\)\.

## References

- K\. Boesen, P\. C\. Gøtzsche, and J\. P\. A\. Ioannidis \(2021\)EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations\.Epidemiology and Psychiatric Sciences30,pp\. e35\.External Links:[Document](https://dx.doi.org/10.1017/S2045796021000147)Cited by:[§3\.2](https://arxiv.org/html/2608.28607#S3.SS2.p2.1)\.
- Y\. Chen, C\. Zhao, Y\. Zhong, Z\. Wang, J\. Shi, and W\. Zheng \(2026\)Applying the deep learning–sector–governance \(dlsg\) framework to the u\.s\. healthcare system: opportunities, deployment pathways, and policy\-aligned evaluation\.Journal of Technology Innovation and Society4\(1\),pp\. 42–59\.External Links:[Document](https://dx.doi.org/10.63646/jtis.2026.040103),[Link](https://inatgi.in/index.php/jtis/article/view/380)Cited by:[§2](https://arxiv.org/html/2608.28607#S2.p4.1)\.
- J\. Chung, R\. Ko, W\. Yoo, M\. Onizuka, S\. Kim, T\. Kim, and W\. Shin \(2025\)GraphCompliance: aligning policy and context graphs for LLM\-based regulatory compliance\.Note:arXiv:2510\.26309External Links:2510\.26309Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p3.1)\.
- D\. Edge, H\. Trinh, N\. Cheng, J\. Bradley, A\. Chao, A\. Mody, S\. Truitt, and J\. Larson \(2024\)From local to global: a GraphRAG approach to query\-focused summarization\.Note:arXiv:2404\.16130External Links:2404\.16130Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p3.1)\.
- A\. Gómez\-Fernándezet al\.\(2026\)Global regulatory variability in small\-molecule inhibitor approvals: differences in timelines, dosing, and pediatric indications across FDA, EMA, and PMDA\.Clinical and Translational Oncology\.Note:PMID 41045422\.Cited by:[§2](https://arxiv.org/html/2608.28607#S2.p4.1),[§7](https://arxiv.org/html/2608.28607#S7.p2.4)\.
- International Council for Harmonisation \(2024\)ICH E11A guideline on pediatric extrapolation\.Technical reportFDA / EMA\.Cited by:[§3\.2](https://arxiv.org/html/2608.28607#S3.SS2.p3.1)\.
- H\. S\. Jomraj, B\. Agarwal, and V\. Rojkova \(2025\)RAGulating compliance: a multi\-agent schema\-light knowledge graph for regulatory compliance QA\.InProceedings of the ISWC 2025 Posters, Demos and Industry Tracks \(ISWC\),CEUR Workshop Proceedings, Vol\.4085,pp\. 53–57\.External Links:[Link](https://ceur-ws.org/Vol-4085/paper14.pdf)Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p3.1)\.
- Y\. Koreeda and C\. D\. Manning \(2021\)ContractNLI: a dataset for document\-level natural language inference for contracts\.InFindings of the Association for Computational Linguistics: EMNLP 2021,pp\. 1139–1150\.Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p1.1)\.
- J\. R\. Landis and G\. G\. Koch \(1977\)The measurement of observer agreement for categorical data\.Biometrics33\(1\),pp\. 159–174\.Cited by:[§3\.3](https://arxiv.org/html/2608.28607#S3.SS3.p3.5)\.
- A\. Mantravadi, S\. Dalmia, A\. Mukherji, N\. Dave, A\. Mittal, and O\. Pospelova \(2025\)LegalWiz: a multi\-agent generation framework for contradiction detection in legal documents\.InNeurIPS 2025 Workshop on Generative and Protective AI for Content Creation,Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p2.1)\.
- S\. Mitaet al\.\(2026\)A comparison of regional decisions for doses and administrations of new drugs: concordance, discordance, and dependencies\.Clinical Pharmacology and Therapeutics\.Note:PMC13083379\.Cited by:[§2](https://arxiv.org/html/2608.28607#S2.p4.1),[§7](https://arxiv.org/html/2608.28607#S7.p2.4)\.
- M\. Roy Choudhury, A\. Chandramouli, M\. Anand, and V\. Gupta \(2026\)Better call CLAUSE: a discrepancy benchmark for auditing LLMs’ legal reasoning capabilities\.InFindings of the Association for Computational Linguistics: EACL 2026,pp\. 5776–5818\.Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p2.1)\.
- D\. Sartori, M\. Fusaroli, J\. K\. Aronson,et al\.\(2026\)PVverse/paper\.PI: multi\-country product\-information corpus \(627 drug–ADR signals across 18 countries\)\.Note:[https://github\.com/PVverse/paper\.PI](https://github.com/PVverse/paper.PI)MIT\-licensed dataset,*Drug Safety*2026\.External Links:[Document](https://dx.doi.org/10.1007/s40264-026-01666-6)Cited by:[§2](https://arxiv.org/html/2608.28607#S2.p4.1)\.
- S\. W\. Schwarz and C\. Decristoforo \(2019\)US and EU radiopharmaceutical diagnostic and therapeutic nonclinical study requirements for clinical trials authorizations and marketing authorizations\.EJNMMI Radiopharmacy and Chemistry4\(1\),pp\. 9\.External Links:[Document](https://dx.doi.org/10.1186/s41181-019-0059-2)Cited by:[§3\.2](https://arxiv.org/html/2608.28607#S3.SS2.p2.1)\.
- Sino\-US\-DrugQA authors \(2026\)Sino\-US\-DrugQA: a benchmark for evaluating large language models in cross\-jurisdictional pharmaceutical regulation\.Note:medRxiv preprint;[https://github\.com/DodgeLU/Sino\-US\-DrugQA](https://github.com/DodgeLU/Sino-US-DrugQA)Preprint \(medRxiv, Feb 2026\), not yet peer\-reviewed\.External Links:[Document](https://dx.doi.org/10.64898/2026.02.13.26346236)Cited by:[§2](https://arxiv.org/html/2608.28607#S2.p4.1),[§7](https://arxiv.org/html/2608.28607#S7.p2.4)\.
- S\. Vieujean, B\. E\. Sands, R\. Panaccione, D\. T\. Rubin, V\. Jairath, S\. Danese, L\. Peyrin\-Biroulet, S\. Schreiber, S\. Vermeire, G\. D’Haens, A\. Dignass, P\. S\. Dulai, N\. Narula, and W\. Reinisch \(2025\)Comparison of the FDA and EMA guidance on drug development in ulcerative colitis: an expert panel review\.Journal of Crohn’s and Colitis19\(7\),pp\. jjaf111\.External Links:[Document](https://dx.doi.org/10.1093/ecco-jcc/jjaf111)Cited by:[§3\.2](https://arxiv.org/html/2608.28607#S3.SS2.p2.1),[§6\.2](https://arxiv.org/html/2608.28607#S6.SS2.p4.2)\.
- W\. Yin, D\. Radev, and C\. Xiong \(2021\)DocNLI: a large\-scale dataset for document\-level natural language inference\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,pp\. 4913–4922\.Cited by:[§1](https://arxiv.org/html/2608.28607#S1.p2.1),[§2](https://arxiv.org/html/2608.28607#S2.p1.1)\.
- O\. Yoran, T\. Wolfson, O\. Ram, and J\. Berant \(2024\)Making retrieval\-augmented language models robust to irrelevant context\.InThe Twelfth International Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=ZS4m74kZpH)Cited by:[§2](https://arxiv.org/html/2608.28607#S2.p2.1),[§6\.1](https://arxiv.org/html/2608.28607#S6.SS1.p2.1)\.

## Appendix APrompts and Code

LLM judge\(llm\_judge\.py\): system prompt defines AGREE/DIVERGE/SILENT with explicit absence framing; user prompt pairs\{fda\_text\}/\{ema\_text\}\.claude\-haiku\-4\-5\-20251001, temp 1\.0, max tokens 100\.

Graph\-RAG\(graph\_rag\.py\): Stage 1 extracts\{subject, obligation\_level, requirement, conditions\}; Stage 2 classifies the aligned node pair\. Hard rules: MANDATORY↔\\leftrightarrowPROHIBITED⇒\\RightarrowDIVERGE \(no LLM call\); both SILENT⇒\\RightarrowSILENT\. Max tokens 300\.