Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts

# Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts
Source: [https://arxiv.org/html/2607.20462](https://arxiv.org/html/2607.20462)
Melanie Rieff1,∗\\astRobin Staab1Thibaud Gloaguen1 Stefan Hegselmann2,3Martin Vechev1 1Department of Computer Science, ETH Zurich, Switzerland2Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Center of Digital Health, Berlin, Germany3Deutsches Herzzentrum der Charité – Medical Heart Center of Charité and German Heart Institute Berlin, Berlin, Germany∗\\astCorresponding author:mrieff@student\.ethz\.ch

###### Abstract

Large language models \(LLMs\) are increasingly integrated into clinical workflows, stressing the need for reliable traceability of model\-generated output with watermarking\. Yet, most watermarks are evaluated on general\-purpose benchmarks, leaving domains like medicine, where small token\-level perturbations can result in significant semantic changes, under\-explored\. In this work, we present the first rigorous study of how LLM watermarks affect medical performance, benchmarking 5 watermarking schemes across 11 LLMs and 7 VLMs on various tasks spanning unimodal and multimodal clinical reasoning\. Importantly, we complement existing evaluations by introducing a human\-expert\-validated pipeline for systematically auditing medical reasoning quality, terminological precision, and induced hallucinations\. Our results reveal that watermarking can induce substantial degradation across multiple failure modes, including lexical corruption, hallucinated terminology, and amplified misattribution or omission of image findings\. Notably, we find that the absence of domain\-specific analyses, combined with aggregate metrics that miss failures inherent to clinical text, can systematically obscure practical watermark\-induced degradations\. Our findings establish domain\-specific evaluation as a prerequisite for the safe deployment of watermarked models in medicine, where current benchmarks can otherwise mask clinically consequential failures\.

## 1Introduction

Large language models \(LLMs\) are increasingly integrated into clinical workflows: 81% of U\.S\. physicians now report using AI professionally\[[1](https://arxiv.org/html/2607.20462#bib.bib1)\], and LLM\-based tools are actively explored for report generation, clinical decision support, and patient communication\[[2](https://arxiv.org/html/2607.20462#bib.bib2),[3](https://arxiv.org/html/2607.20462#bib.bib3)\]\. In particular, LLM outputs inform downstream judgement, from summarising patient histories and drafting reports to suggesting differential diagnoses\. As model capabilities improve, the scope of clinical reliance is likely to grow, making the quality of generated text a matter of patient safety\. In parallel, emerging\[[4](https://arxiv.org/html/2607.20462#bib.bib4)\]regulation is pushing toward provenance marking of AI\-generated content, with watermarking becoming the leading compliance mechanism for tracing LLM outputs\[[5](https://arxiv.org/html/2607.20462#bib.bib5)\]\. Crucially, this implies that LLMs used in clinical settings are increasingly likely to be using a watermark\.

##### LLM Watermarking

Watermarking embeds a statistically detectable signal by modifying the LLM sampling procedure at every generation step, which enables traceability but, by design, directly influences the model’s output distribution\. Importantly, prior works evaluate the impact of watermarks on output quality with general\-domain benchmarks, and coarse quality proxies such as perplexity\. In medicine, where correctness depends jointly on precise reasoning, terminology, qualifiers, and numerical values, even small token\-level perturbations can alter clinical meaning significantly, suggesting that results from those simple evaluations might not transfer\[[6](https://arxiv.org/html/2607.20462#bib.bib6),[7](https://arxiv.org/html/2607.20462#bib.bib7),[8](https://arxiv.org/html/2607.20462#bib.bib8)\]\. This makes understanding how watermarks alter LLMs’ clinical performance a critical, yet under\-explored issue\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x1.png)Figure 1:Overview of our evaluation pipeline\.Given a multimodal benchmark instance, we prompt VLMs \(using chain\-of\-thought prompting\) with and without watermarks for reference\. We first evaluate the detectability of watermarks in the answers and their accuracy\. We then examine the reasoning traces in more depth using three specific LLM\-as\-judges, validated by clinical experts\.
##### This work

In this work, we propose the first systematic study of how text watermarking affects medical LLM performance, illustrated in[Figure˜1](https://arxiv.org/html/2607.20462#S1.F1)\. In particular, we evaluate55prominent watermarks on two medical benchmarks, mixing clinical knowledge and visual understanding \([Figure˜1](https://arxiv.org/html/2607.20462#S1.F1)left\)\. To measure the impact of watermarking on reasoning, we introduce \(in[Section˜3\.2](https://arxiv.org/html/2607.20462#S3.SS2)\) three specific LLM\-as\-judges validated against clinical expert annotations \([Figure˜1](https://arxiv.org/html/2607.20462#S1.F1)right\)\.

Our results reveal that watermarking can leave benchmark accuracy largely intact while substantially degrading the underlying reasoning: the rate of correct answers backed by flawed reasoning more than doubles on several models \(e\.g\. 11\.4%→\\to26\.3% onPhi\-4\-14Bwith distortionary SynthID\), and fabricated medical entities increase by up to \+39\.2 pp\. Among questions where both watermarked and unwatermarked outputs select the correct answer, up to 40% rely on mutually exclusive diagnostic statements\. On multimodal inputs, watermarks increasingly corrupt visual interpretation, and for reasoning models, watermarking the reasoning trace inflates output length by up to69%69\\%while degrading reasoning quality, whereas restricting the watermark to the final answer preserves both\.

##### Key Contributions

Our main contributions are:

- •A systematic evaluation of five watermarking schemes across 11 LLMs and 7 VLMs on medical benchmarks \(MedQA, MedXpertQA\-MM\), with*physician\-validated*LLM\-as\-a\-judge audits that assess reasoning quality, terminological precision, and hallucinations beyond letter accuracy\.
- •The first analysis of how text\-decoding watermarks interact with multimodal medical input, revealing that watermarking during text generation degrades visual grounding, e\.g\., suppressing correct image\-based claims and amplifying contradicted ones, despite accuracy remaining stable\.
- •A study of reasoning model watermarking, showing that restricting watermarks to the user\-visible final answer preserves reasoning quality, while watermarking the thinking degrades terminological precision and induces circular deliberation\.

## 2Related Work

We review current evaluation practices for LLMs in medicine, the main families of text watermarking, and existing work on watermark quality assessment, highlighting the gap that motivates our study\.

##### LLMs in medicine\.

General\-purpose and domain\-specialized LLMs are commonly evaluated on USMLE\-style benchmarks such as MedQA\[[9](https://arxiv.org/html/2607.20462#bib.bib9),[10](https://arxiv.org/html/2607.20462#bib.bib10)\], while multimodal benchmarks\[[11](https://arxiv.org/html/2607.20462#bib.bib11),[12](https://arxiv.org/html/2607.20462#bib.bib12),[13](https://arxiv.org/html/2607.20462#bib.bib13)\]extend evaluation to joint interpretation of images, labs, and patient records\. Crucially, benchmark accuracy alone is an insufficient measure of clinical quality: models may arrive at the correct answer through flawed reasoning\[[14](https://arxiv.org/html/2607.20462#bib.bib14)\], produce confident hallucinations\[[15](https://arxiv.org/html/2607.20462#bib.bib15)\], or misuse medical terminology while maintaining a fluent narrative\. These observations have motivated richer evaluation frameworks, from structured reasoning rubrics\[[16](https://arxiv.org/html/2607.20462#bib.bib16)\]to physician\-graded benchmarks\[[17](https://arxiv.org/html/2607.20462#bib.bib17)\], and underscore that even small prompt variations can substantially change medical output quality\[[8](https://arxiv.org/html/2607.20462#bib.bib8)\]\. Our work builds on this line by introducing 3 LLM\-as\-judges, validated against two board\-certified physicians, specifically designed to audit watermark\-induced degradation along the quality axes that matter in clinical practice\.

##### Watermarking\.

Generation\-time text watermarking modifies the sampling procedure to embed statistically detectable signals tied to a secret key, enabling traceability of AI\-generated content\[[18](https://arxiv.org/html/2607.20462#bib.bib18)\]\. At each token position, a context\-dependent hash seeds a pseudorandom score assignment over the vocabulary, and the sampling distribution is modified accordingly\.*Distortionary*methods such as KGW\[[19](https://arxiv.org/html/2607.20462#bib.bib19)\]and PPL\[[20](https://arxiv.org/html/2607.20462#bib.bib20)\]bias next\-token distributions toward high\-scoring tokens, introducing a systematic shift away from the model’s original distribution\.*Distortion\-free*methods such as DipMark\[[21](https://arxiv.org/html/2607.20462#bib.bib21)\], AAR at zero strength\[[22](https://arxiv.org/html/2607.20462#bib.bib22)\], and SynthID\-Text with two leaves per tournament\[[23](https://arxiv.org/html/2607.20462#bib.bib23)\]instead resample tokens in a way that preserves the original model distribution in expectation over the random watermark key\. However, this guarantee is an*expectation*over keys, and under the assumption of no hash collisions\. We find that, in practice, even distortion\-free schemes can induce some small reasoning degradation \([Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)\)\.

##### Evaluating watermark quality\.

Watermarking studies typically report detection power alongside coarse quality proxies such as perplexity or brief LLM ratings\[[19](https://arxiv.org/html/2607.20462#bib.bib19),[23](https://arxiv.org/html/2607.20462#bib.bib23),[24](https://arxiv.org/html/2607.20462#bib.bib24)\]\. Yet, these proxies are poorly aligned with human preference\[[6](https://arxiv.org/html/2607.20462#bib.bib6)\]and mask substantial, task\-dependent degradation: instruction\-following quality drops considerably under matched watermark strength\[[6](https://arxiv.org/html/2607.20462#bib.bib6)\], smaller models suffer disproportionately\[[25](https://arxiv.org/html/2607.20462#bib.bib25)\], and reasoning\-intensive tasks degrade more severely than open\-ended generation\[[26](https://arxiv.org/html/2607.20462#bib.bib26),[27](https://arxiv.org/html/2607.20462#bib.bib27)\]\. These findings establish that the impact of watermarking is model\- and task\-specific, yet most evaluations are confined to general\-domain benchmarks, leaving a twofold gap: \(i\) the tasks studied do not capture the demands of clinical text, where a single\-token substitution can alter a diagnosis, and \(ii\) the quality proxies employed cannot detect failure modes such as terminology corruption, hallucinated findings, or inconsistency between reasoning and conclusion\. Separately, recent work has shown that watermarks can be applied to reasoning traces\[[28](https://arxiv.org/html/2607.20462#bib.bib28),[27](https://arxiv.org/html/2607.20462#bib.bib27)\], yet no study has assessed whether such application preserves clinical reasoning quality\. One prior study attempts to evaluate watermarking on medical text\[[29](https://arxiv.org/html/2607.20462#bib.bib29)\]\. Yet, we find that it suffers from fundamental methodological flaws, which makes its findings unreliable\. Namely, it uses instruction\-tuned models without chat templates, caps generation at 25 tokens, too short for reliable watermark detection, and fails to establish a paired unwatermarked baseline from which to measure hallucinations\.

Our work introduces, for the first time, a rigorous and systematic evaluation of watermarks in medicine, spanning five watermarking schemes, multiple model scales and domain specializations, and text\-decoding watermarks applied to models that process multimodal medical inputs\. Importantly, we go beyond accuracy and introduce 3 LLM\-as\-judges, validated by clinical experts, to probe reasoning quality, lexical precision, and diagnostic consistency\.

## 3Our Evaluation Pipeline

In this section, we introduce our evaluation pipeline illustrated in[Figure˜1](https://arxiv.org/html/2607.20462#S1.F1)\. We describe the models, watermarks, and data we evaluate \([Section˜3\.1](https://arxiv.org/html/2607.20462#S3.SS1)\), and the metrics we use \([Section˜3\.2](https://arxiv.org/html/2607.20462#S3.SS2)\)\. We defer additional details on the clinician validation of our LLM judges to[Section˜D\.2](https://arxiv.org/html/2607.20462#A4.SS2)\.

### 3\.1Setup

Our evaluation spans two medical benchmarks probing complementary aspects of clinical knowledge, 18 language and vision–language models covering general\-purpose and medical\-specialised architectures at 4B–70B scale, and five prominent watermarking schemes\. Together these define the \(model, watermark, strength\) configurations we evaluate throughout the next sections\.

##### Benchmarks\.

We anchor our evaluation with two medical benchmarks, each of them chosen to probe distinct aspects of medical knowledge and reasoning\.

*MedQA*\[[9](https://arxiv.org/html/2607.20462#bib.bib9)\]contains 2,000 four\-option, multiple\-choice questions drawn from the United States medical licensing examinations\. A meta\-analysis byKim and Yoon \[[10](https://arxiv.org/html/2607.20462#bib.bib10)\]identifies MedQA as the benchmark most predictive of real\-world clinical performance among existing medical QA datasets\.

*MedXpertQA\-MM*\[[13](https://arxiv.org/html/2607.20462#bib.bib13)\]comprises 2,000 expert\-curated, five\-option questions spanning 17 medical specialties\. Each question is paired with one to six images \(on average 1\.43, across 10 modalities including radiology, pathology, and schematic diagrams\), requiring joint interpretation alongside clinical vignettes and structured patient data such as laboratory results\.

##### Models\.

We evaluate three groups of models: On*MedQA*, we include six instruction\-tuned LLMs: 4 general\-purpose models \(Llama\-3\.1\-8B,Llama\-3\.1\-70B\[[30](https://arxiv.org/html/2607.20462#bib.bib30)\],Gemma\-3\-12B\[[31](https://arxiv.org/html/2607.20462#bib.bib31)\], andPhi\-4\-14B\[[32](https://arxiv.org/html/2607.20462#bib.bib32)\]\), and 2 medical\-specialised models \(OpenBioLLM\-70B\[[33](https://arxiv.org/html/2607.20462#bib.bib33)\], andUltraMedical\-70B\[[34](https://arxiv.org/html/2607.20462#bib.bib34)\]\), as well as 4 reasoning models \(DeepSeek\-R1\-Distill\-Llama\-8B,DeepSeek\-R1\-Distill\-Llama\-70B,DeepSeek\-R1\-Distill\-Qwen\-32B\[[35](https://arxiv.org/html/2607.20462#bib.bib35)\], andQwen\-3\.5\-27B\[[36](https://arxiv.org/html/2607.20462#bib.bib36)\]\)\. On*MedXpertQA\-MM*, we evaluate 7 VLMs spanning 4B–32B parameters:Qwen\-3\-VL\-8BandQwen\-3\-VL\-32B\[[37](https://arxiv.org/html/2607.20462#bib.bib37)\],Gemma\-4\-31B\[[38](https://arxiv.org/html/2607.20462#bib.bib38)\],Lingshu\-7BandLingshu\-32B\[[39](https://arxiv.org/html/2607.20462#bib.bib39)\], andMedGemma\-4BandMedGemma\-27B\[[40](https://arxiv.org/html/2607.20462#bib.bib40)\]\. In particular,Lingshu\-7B,Lingshu\-32B,MedGemma\-4B, andMedGemma\-27Bare medical\-specialised VLMs and were not trained on MedXpertQA\.

##### Watermarking schemes\.

Five prominent generation\-time watermarking schemes are evaluated, spanning the major algorithmic families: KGW\[[19](https://arxiv.org/html/2607.20462#bib.bib19)\]\(soft logit bias\), DipMark\[[21](https://arxiv.org/html/2607.20462#bib.bib21)\]\(CDF reweighting\), AAR\[[22](https://arxiv.org/html/2607.20462#bib.bib22)\]\(Gumbel\-max sampling\), PPL\[[20](https://arxiv.org/html/2607.20462#bib.bib20)\]\(perplexity\-budgeted argmax\), and SynthID\[[23](https://arxiv.org/html/2607.20462#bib.bib23)\]\(tournament sampling\)\. All five share a common structure: at each token position, a context\-dependent hash assigns pseudorandom scores to vocabulary entries, and the sampling distribution is modified accordingly\. The schemes fall into two categories\.*Distortion\-based*methods \(KGW, PPL at high strength\) bias next\-token distributions toward high\-scoring tokens, introducing a systematic shift away from the model’s original distribution\.*Distortion\-free*methods \(DipMark, AAR at zero strength, SynthID with binary tournaments\) preserve the original distribution in expectation over the random watermark key\. For each scheme, we sweep multiple strength levels to evaluate their detectability–quality trade\-off\. We describe each scheme in\-depth in[Section˜A\.1](https://arxiv.org/html/2607.20462#A1.SS1)\.

### 3\.2Evaluation Dimensions

Each \(model, watermark, strength\) configuration is scored along two complementary fronts: \(1\) baseline metrics that capture the watermark strength and benchmark utility, and \(2\) structured audits that probe whether watermarking corrupts underlying medical reasoning\. The audits are necessary as benchmark accuracy fails to detect fabricated entities, misapplied terminology, or distorted reasoning\.

##### Baseline metrics\.

We sample each \(model, watermark, strength\) configuration on the full 2,000\-question benchmark using chain\-of\-thought prompting \(with prompts in[Section˜F\.1](https://arxiv.org/html/2607.20462#A6.SS1)\)\. For each completion, we measure both the*accuracy*and the watermark*detectability*\. For watermark detectability, we measure the average true positive rate \(TPR\) at a1%1\\%false positive rate \(FPR\) over the 2,000 answers\.

##### Reasoning\-quality LLM\-Judge\.

A model may select the correct answer while relying on fabricated entities or misapplied terminology, a failure mode documented in prior work on medical LLMs\[[14](https://arxiv.org/html/2607.20462#bib.bib14),[15](https://arxiv.org/html/2607.20462#bib.bib15)\]and one that standard accuracy metrics cannot detect\. Letter accuracy alone therefore cannot show whether watermarking degrades medical reasoning\. We complement accuracy with three structured LLM\-as\-judges, usingGemini\-3\-Flashas the judge for its balance of cost and reasoning capability\.

1. 1\.*Single\-response audit*evaluates each completion in isolation: the judge sees one response at a time, blind to both watermark condition and gold answer, and flags eight axes \(fabricated entities, corrupted spellings, and misapplied real terms \(terminology\), confident error, vignette fabrication, clue misread, within\-response contradiction, linguistic corruption, and format/termination failure\) plus an abstention indicator\. For MedXpertQA\-MM, the judge additionally classifies each perceptual image\-grounding claim assupported,contradicted, orunverifiable\.
2. 2\.*Pairwise divergence audit*presents the watermarked and unwatermarked completions for the same question side by side to the judge, with response order randomised and condition labels stripped\. The judge records the correctness configuration of the pair, the level of diagnostic interpretation divergence \(none,minor,major\), and any scenario fabrication\.
3. 3\.*Reasoning\-trace audit*targets thinking models, whose hidden chain\-of\-thought may be independently distorted by the watermark\. We use a two\-block judging protocol\. First, without seeing the gold answer, the judge rates four aspects of reasoning quality: reasoning efficiency, clinical coherence, alignment between the reasoning and the final answer, and engagement with distractors\. Second, with the gold answer, the judge rates two factual aspects: terminological precision and diagnostic accuracy\. The judge also provides two overall clinical\-deployment ratings,*clinical safety*and*supervisory burden*, as well as the primary degradation pattern, defined in[Section˜F\.2](https://arxiv.org/html/2607.20462#A6.SS2)\. To separate degradation in the reasoning trace from degradation in the final answer, we run two paired comparisons: \(i\)*full watermark vs\. final\-answer\-only watermark*and \(ii\)*final\-answer\-only watermark vs\. no watermark*\.

Importantly, we validate our judges with physician agreement in[Sections˜4\.3](https://arxiv.org/html/2607.20462#S4.SS3)and[D](https://arxiv.org/html/2607.20462#A4)\. For each configuration, we conduct judge audits at the watermark hyperparameters that yield the lowest strength while still achieving≥99%\\geq 99\\%TPR at1%1\\%FPR, which prior work has considered the operating point required for regulatory deployment\. We defer the remaining judge parameters \(e\.g\., full prompts, axis definitions, calibration examples, and decoding configurations\) to[Section˜F\.2](https://arxiv.org/html/2607.20462#A6.SS2)\.

## 4Results

In this section, we evaluate how prominent watermarks affect the clinical performance of LLMs and validate our judges\. We show that even though pure benchmark accuracy may remain high \([Section˜4\.1](https://arxiv.org/html/2607.20462#S4.SS1)\), watermarks can still degrade reasoning quality \([Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)\)\. We then validate our judges in[Section˜4\.3](https://arxiv.org/html/2607.20462#S4.SS3)\. Lastly, in[Section˜4\.4](https://arxiv.org/html/2607.20462#S4.SS4), we evaluate watermarks in the context of reasoning models\.

### 4\.1Accuracy Looks Mostly Stable, But the Picture is Misleading

![Refer to caption](https://arxiv.org/html/2607.20462v1/x2.png)Figure 2:MedQA accuracy vs\. detectability \(TPR@1%1\\%FPR\), non\-reasoning LLMs across five schemes\. Dashed line and grey band: unwatermarked mean and95%95\\%CI; dotted vertical: TPR=99%=99\\%\.Llama\-3\.1\-70B,Gemma\-3\-12B, andPhi\-4\-14Bstay within the band across all schemes; the two specialised 70Bs collapse under complementary schemes;Llama\-3\.1\-8Bdegrades broadly\.![Refer to caption](https://arxiv.org/html/2607.20462v1/x3.png)Figure 3:MedXpertQA\-MM accuracy vs\. detectability, seven VLMs; conventions as in[Figure˜2](https://arxiv.org/html/2607.20462#S4.F2)\. Models consistently stay near the baseline band despite reasoning varying noticeably \([Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)\)\.[Figures˜2](https://arxiv.org/html/2607.20462#S4.F2)and[3](https://arxiv.org/html/2607.20462#S4.F3)show that, for most schemes and models, accuracy remains within the 95% confidence interval \(CI\) of the unwatermarked baseline\. On MedQA \([Figure˜2](https://arxiv.org/html/2607.20462#S4.F2)\), three configurations show a visible drop: the medical\-specialised 70Bs \(UltraMedical\-70BandOpenBioLLM\-70B\) at high TPR@1%FPR, and*Llama\-3\.1\-8B*more broadly\. That two equally large medical\-specialised models break at high TPR@1%FPR, while the equally large general\-purposeLlama\-3\.1\-70Bremains consistently stable, rules out scale or domain specialisation as sufficient explanations\.111*BioMistral\-7B*also collapses \(KGW−37\.5\-37\.5, SynthID−14\.2\-14\.2pp\), but the signal is confounded by poor instruction tuning on the CoT\-MCQ template \(median∼6\\sim 6words per response vs\.250250–450450for other models\); we exclude it from the headline claims and report it only in the appendix grids\.On MedXpertQA\-MM \([Figure˜3](https://arxiv.org/html/2607.20462#S4.F3)\), all VLMs stay within or near the CI\. We find in[Section˜B\.1](https://arxiv.org/html/2607.20462#A2.SS1)that although perplexity rises monotonically with watermark strength on every model, it does not correlate with drops in accuracy or reasoning degradation\. Hence, a standard accuracy benchmark would conclude that watermarking is safe for medical use\. Our audits in[Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)show that this conclusion is incorrect: even where accuracy holds, watermarking corrupts the reasoning that clinicians would actually read\.

### 4\.2Reasoning Quality Degrades Even When Accuracy Holds

We score every \(model, scheme\) configuration at the≥99%\\geq 99\\%TPR operating point using the judge audits of[Section˜3\.2](https://arxiv.org/html/2607.20462#S3.SS2)\. We bootstrap all our results, and our p\-values quantify the significance between watermarked and unwatermarked results\. We show the two most\-affected models per benchmark in[Table˜2](https://arxiv.org/html/2607.20462#S4.T2), and defer full results to[Section˜C\.1](https://arxiv.org/html/2607.20462#A3.SS1)\.

##### Correct answers hide broken reasoning\.

Watermarking increases the rate at which models reach the right MCQ letter through flawed reasoning\. The*faulty\-but\-correct*rate \(F&C\) \(i\.e\., correct letter with at least one flagged defect\) climbs from11\.4%11\.4\\%to26\.3%26\.3\\%onPhi\-4\-14Bunder SynthID while accuracy drops only−3\.2\-3\.2pp, and similarly onOpenBioLLM\-70BF&C more than doubles \(12\.9%→27\.5%12\.9\\%\\to 27\.5\\%,∗\)\. However, F&C only points out the existence of inconsistencies within the reasoning, but does not explain exactly what the inconsistencies are\.

##### Watermarking silently corrupts medical language\.

Even when accuracy and F&C barely move, watermarks inject three categories of medical\-language errors that standard metrics miss:*fabricated entities*,*misapplied real terms*, and*corrupted spellings*\([Table˜1](https://arxiv.org/html/2607.20462#S4.T1)\)\. OnMedGemma\-4Bunder SynthID,Δ\\DeltaF&C is only\+0\.4\+0\.4, yet the model fabricates\+12\.6\{\+12\.6\}extra entities per100100questions and misapplies\+24\.3\{\+24\.3\}real terms; onLingshu\-7Bunder SynthID, fabrications reach\+39\.2\{\+39\.2\}per100100questions\. Fabricated and misapplied terms are especially dangerous: they produce fluent, confident text that requires domain expertise to catch and that no spell\-checker or final\-letter check would flag\.

Table 1:Illustrative watermark\-induced terminology errorsdrawn from our judge audits and paired qualitative reviews of watermarked outputs\. Matched unwatermarked completions on the same questions use the correct terms\.Error typeWatermarked TermCorrect Term / ExplanationFabricated entity*Streptococcus epidermidis**Staphylococcus epidermidis*PentaprazolePantoprazoleNaftifloxacinNo such antibiotic\.plutonium channelspotassium channels; plutonium is a radioactive element\.Misapplied real termEosinophilic granuloma applied to the brown tumors of hyperparathyroidismEosinophilic granuloma denotes Langerhans cell histiocytosis—unrelated to the lytic lesions of hyperparathyroidism\.“β\\beta\-blockers preferred over CCBs in COPD”Reversed:β\\beta\-blockers risk bronchospasm; non\-dihydropyridine CCBs are safe in COPD\.Benzodiazepines “increase the*duration*of GABA\-channel opening”Benzodiazepines increase the channel\-opening*frequency*;*duration*is the mechanism of barbiturates\.Corrupted spelling*Clostridioides diffidile*Misspelling of*C\. difficile*\.meningritisMisspelling of meningitis\.bradiceardiaMisspelling of bradycardia\.Table 2:Reasoning damage at99%99\\%TPR, for two most\-affected models per benchmark with full results in[Section˜C\.1](https://arxiv.org/html/2607.20462#A3.SS1)\. We report the accuracy \(Δ\\DeltaAcc\), the percentage of fabricated facts \(Fab/100\), the percentage of misapplied medical terms \(Misapp/100\), the percentage of corrupted spellings \(Spell/100\), the vignette fabrication ratio \(VigFab\), the faulty\-but\-correct rate \(F&C\), and the percentage of paired responses with major diagnostic divergence \(Major%\)\. For the unwatermarked row we report the actual value, whereas for watermarked rows we report the difference/ratio with the unwatermarked baseline\. Weboldentries that are statistically significant \(p\-value below0\.010\.01\)\.ModelSchemeΔ\\DeltaAccFab/100Misapp/100Spell/100VigFabF&CMajor%MedQAPhi\-4\-14BUnwatermarked81\.40\.34\.10\.11\.0×\\times11\.4–AAR\-2\.5\+1\.8\+4\.0\+1\.514\.0×\\times\+6\.51\.9DipMark\-0\.9\+0\.1\+2\.6\+0\.30\.8×\\times\+3\.62\.8KGW\-0\.7\+1\.5\+3\.6\+2\.23\.0×\\times\+9\.53\.6PPL\-8\.6\+3\.8\+4\.3\+69\.618\.0×\\times\+9\.62\.3SynthID\-3\.2\+6\.6\+8\.0\+10\.67\.6×\\times\+14\.93\.3OpenBioLLM\-70BUnwatermarked73\.60\.04\.50\.61\.0×\\times12\.9–AAR\-0\.2\+0\.9\+1\.5\+0\.90\.9×\\times\+4\.44\.4DipMark\-1\.1\+0\.4\+0\.8\+0\.31\.1×\\times\+1\.53\.2KGW\-17\.4\+1\.9\+4\.4\+5\.23\.5×\\times\+4\.25\.6PPL\-7\.7\+2\.1\+1\.8\+51\.51\.9×\\times\+6\.76\.6SynthID\-11\.4\+13\.6\+6\.9\+25\.96\.6×\\times\+14\.68\.7MedXpertQA\-MMMedGemma\-4BUnwatermarked24\.11\.911\.11\.01\.0×\\times16\.5–AAR\-1\.0\+6\.5\+8\.1\+3\.43\.1×\\times\+1\.238\.3DipMark\+0\.2\+0\.7\+2\.0\+0\.21\.2×\\times\+1\.635\.0KGW\-2\.6\+8\.9\+16\.2\+15\.43\.1×\\times\+0\.631\.7PPL\-1\.7\+6\.0\+5\.0\+43\.62\.1×\\times0\.034\.7SynthID\-3\.1\+12\.6\+24\.3\+18\.04\.2×\\times\+0\.436\.0Lingshu\-7BUnwatermarked23\.91\.512\.10\.41\.0×\\times13\.7–AAR\-0\.5\+12\.9\+18\.1\+5\.42\.8×\\times\+3\.733\.0DipMark\+1\.0\+0\.3\+2\.7\+0\.11\.0×\\times\+1\.422\.6KGW\-0\.8\+13\.2\+2\.4\+16\.23\.6×\\times\+6\.042\.9PPL\-0\.6\+3\.9\+11\.5\+37\.92\.4×\\times\+2\.320\.0SynthID\-2\.1\+39\.2\+20\.2\+18\.97\.0×\\times\+5\.530\.5
##### Each scheme damages a different axis\.

We find in[Table˜7](https://arxiv.org/html/2607.20462#A3.T7)that watermarking is not generic noise: each scheme damages a different axis\. PPL drives surface corruption\. SynthID and KGW instead inflate fabrications and term misuse, with vignette\-fabrication ratios up to18×18\\times\(Phi\-4\-14B/PPL\) and7\.6×7\.6\\times\(Phi\-4\-14B/SynthID\)\. AAR produces broad small\-to\-moderate effects\. DipMark causes the least damage, consistent with its distortion\-free guarantee\[[21](https://arxiv.org/html/2607.20462#bib.bib21)\]\. Yet distortion\-free does not mean benign: the guarantee holds in expectation over the random key and without hash collisions\. We find that, at a fixed key,Phi\-4\-14Bunder DipMark still shows a statistically significant increase in F&C\.

##### Watermarks corrupt visual reporting\.

Here, we find that watermark\-induced degradation may change what models claim to see\. On MedXpertQA\-MM, we track supported and contradicted visual claims per question against the provided image \([Figure˜4](https://arxiv.org/html/2607.20462#S4.F4)\)\.supportedclaims fall andcontradictedclaims rise simultaneously in most configurations with medical VLMs\. For the larger general\-purpose VLMs, we find in[App\.˜C](https://arxiv.org/html/2607.20462#A3)no significant change\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x4.png)Figure 4:Impact of watermarking on visual understanding\.We show the difference between watermarked and unwatermarked reasoning in the average number of supported \(top\) and contradicted \(bottom\) visual claims on MedXpertQA\.
##### Domain\-specialised and smaller models are most vulnerable\.

At matched scale and TPR@1%FPR, general\-purpose models are less degraded by watermarking than medical\-specialised or small models\.Llama\-3\.1\-70Bis stable under all five schemes while the two same\-size medical 70Bs each degrade on at least one scheme\. We find in[Section˜C\.1](https://arxiv.org/html/2607.20462#A3.SS1)a similar split among3030B VLMs \(Gemma\-4\-31Bvs\.Lingshu\-32B\)\. Yet, at higher watermark distortion, evenLlama\-3\.1\-70B’s F&C nearly doubles andGemma\-4\-31B’s vignette fabrication reaches6×6\\times\.

##### Harm grows with strength, but the trajectory is scheme\-specific\.

We find in[App\.˜C](https://arxiv.org/html/2607.20462#A3)and[7](https://arxiv.org/html/2607.20462#A3.F7)that the quality\-detectability trajectory \(i\.e\., how medical degradation varies with each scheme’s detectability\) is scheme\-specific\. For instance, KGW degrades smoothly with detectability whereas SynthID’s quality matches the unwatermarked baseline up to9090% TPR@1%FPR and then degrades\.

##### Understanding why watermarks degrade reasoning

We observed that the failures above share a common underlying cause: when the watermark perturbs the sampling distribution at a token where the model commits to a clinical interpretation, it can redirect the entire downstream reasoning chain\. We isolate this with a controlled prefix\-seeding experiment on three MedXpertQA\-MM questions \([App\.˜E](https://arxiv.org/html/2607.20462#A5)\) and two schemes \(KGW and PPL\): watermarking amplifies distractor prefixes the clean model dismisses \(accuracy drops of 14–25 pp\) and suppresses rescue cues it would otherwise exploit\.

### 4\.3Judge Validation

A board\-certified physician annotated650650completions blinded to model and watermark configuration, drawn from four \(model, scheme\) configurations spanning the full error\-density range:Phi\-4\-14B/SynthID andLlama\-3\.1\-70B/DipMark on MedQA,Lingshu\-7B/SynthID andGemma\-4\-31B/DipMark on MedXpertQA\-MM \(250250MedQA\+\+250250MedXpertQA\-MM single\-response completions, plus150150pairwise pairs\)\. As a second automated baseline,Claude\-Sonnet\-4\.6scores the same650650samples independently\.

On the four axes that carry our headline claims, judge–physician agreement is substantial: fabricated\-entity counts reachκ=0\.75\\kappa\{=\}0\.75/ρ=0\.75\\rho\{=\}0\.75\(n=500n\{=\}500\), misapplied\-term countsκ=0\.42\\kappa\{=\}0\.42/ρ=0\.47\\rho\{=\}0\.47, image\-contradicted claimsρ=0\.74\\rho\{=\}0\.74\(n=250n\{=\}250, MM only\), and pairwise diagnostic divergenceκw=0\.65\\kappa\_\{w\}\{=\}0\.65/ binaryκ=0\.67\\kappa\{=\}0\.67\(n=150n\{=\}150\)\. Agreement betweenGemini\-3\-FlashandClaude\-Sonnet\-4\.6on the medical\-content axes \(fabrications, misapplied terms, pairwise divergence\) matches or exceeds the corresponding judge–physician values \(0\.800\.80/0\.810\.81,0\.500\.50/0\.520\.52,κw=0\.66\\kappa\_\{w\}\{=\}0\.66/κ=0\.72\\kappa\{=\}0\.72, respectively\), confirming that two independently trained LLM\-judges recover the same signal\. The full per\-axis agreement grid \([Table˜10](https://arxiv.org/html/2607.20462#A4.T10)\) shows that lowerκ\\kappavalues concentrate on low\-prevalence stylistic flags \(format/termination, linguistic corruption, spelling\), where judge–physician and judge–judge agreement drop jointly—consistent with base\-rate sensitivity ofκ\\kapparather than systematic bias\. We therefore anchor claims on these axes to pairedΔ\\Deltas between watermarked and unwatermarked conditions rather than to absolute error levels\.

### 4\.4Reasoning Models: Full vs\. Answer\-Only Watermarking

The two highest\-scoring models on MedQA in our setup are reasoning models \(Qwen\-3\.5\-27B\(94\.6%94\.6\\%\) andDeepSeek\-R1\-Distill\-Llama\-70B\(92\.3%92\.3\\%\)\), narrowly above the strongest instruction\-tuned baseline \(Llama\-3\-70B\(91\.0%91\.0\\%\)\)\. Unlike instruction\-tuned models, they generate a reasoning trace \(usually hidden\) inside<thinking\>tags before producing the final answer; a watermark can therefore be applied in two distinct ways:*answer\-only*\(FA\-only\), which watermarks only the post\-<thinking\>answer tokens, and*full watermark*\(Full WM\), which watermarks both the reasoning trace and the final answer\. We compare both approaches on four reasoning models–DeepSeek\-R1\-Distill\-Llama\-8BandDeepSeek\-R1\-Distill\-Llama\-70B,DeepSeek\-R1\-Distill\-Qwen\-32B, andQwen\-3\.5\-27Bwith KGW at 3 strength levels \(δ∈\{1,3,5\}\\delta\\in\\\{1,3,5\\\}\)\.

##### Full WM degrades reasoning quality on every axis except the final answer\.

Across the six judge dimensions, Full WM drives substantial negative shifts on*reasoning efficiency*and*terminological precision*, with smaller drops on*clinical coherence*and*distractor engagement*for each of the four models \([Figure˜11](https://arxiv.org/html/2607.20462#A3.F11)\)\. Two dimensions change negligibly:*answer–reasoning alignment*and*diagnostic accuracy*\. The model thus still selects the correct letter through reasoning that is internally consistent with its conclusion, but its output becomes substantially harder to use as a clinical reference due to imprecise or fabricated terminology, redundant deliberation, and distractor engagement that the unwatermarked response dismisses\. We also rate how harmful the reasoning is \(no\_harm\_risk/low\_risk/harm\_risk\_present\)\. Even though both small and large models similarly degrade the reasoning, only small models’ reasoning is judged to be significantly harmful\.

##### Watermarking only the final answer is essentially free\.

Scoring Full WM against FA\-only as the reference reproduces the same per\-dimension pattern within bootstrap intervals \([Figure˜11](https://arxiv.org/html/2607.20462#A3.F11), orange vs\. blue\): FA\-only is equivalent to Base on every axis\.harm\_risk\_presentlikewise stays at the No WM baseline under FA\-only on every model \([Table˜9](https://arxiv.org/html/2607.20462#A3.T9)\): the watermark, applied only to post\-<thinking\>answer tokens, leaves the model’s chain of thought unperturbed\.

Baseline \(No WM\)Full WM \(KGWδ=3\\delta=3\)*Linear:*“The patient has low Vitamin D but normal Ca\+\+, suggesting secondary hyperparathyroidism\.”*Circular:*“Wait, let me rethink\. The Ca\+\+is normal\. Actually, the Ca\+\+is not low\. Hmm\. Let me check the Ca\+\+again\.”Table 3:Excerpt ofDeepSeek\-R1\-Distill\-Qwen\-32Breasoning traces, baseline vs\. Full WM\.
##### Trace watermarking inflates verbosity and induces recursive self\-correction loops\.

Inspection of Full WM traces reveals a consistent pathology across the four models: revisiting already established \(and corrected\) premises\. A representative excerpt is shown in[Table˜3](https://arxiv.org/html/2607.20462#S4.T3)\. Quantitatively, Full WM atδ=5\\delta=5inflates mean total output length \(reasoning trace plus final answer\) by\+36\+36% to\+69\+69% across the four models, while FA\-only does not \([Table˜6](https://arxiv.org/html/2607.20462#A2.T6)\)\. This means that Full WM’s increase in detectability \(at a given distortion level\) is partly a length artefact rather than a stronger per\-token signal\.

## 5Conclusion and Limitations

We introduce a rigorous and systematic evaluation of watermark\-induced degradation in medicine, with three specific LLM\-as\-judges validated by clinicians\. We find that, with most watermarks and models tested, the accuracy on medical benchmarks barely drops even at high watermark strength\. Nonetheless, our analysis of reasoning traces tells a different story\. Watermarks may severely degrade the reasoning of the models \(e\.g\., hallucinate facts, misinterpret data\) and may even harm their visual understanding\. For reasoning models in particular, we suggest watermarking only the final answer, which we show prevents such degradation\.

##### Limitations

Both benchmarks are multiple\-choice and single\-turn, so they do not capture open\-ended or interactive clinical workflows such as report writing, patient communication, multi\-turn dialogue, or retrieval\-augmented and tool\-augmented reasoning\. In such settings, each watermarked response becomes context for subsequent generation, and it remains unclear whether watermarking\-induced degradation compounds or attenuates across turns\. Future work should extend the evaluation to these settings\. Additionally, our evaluation is restricted to open\-weight models \(watermarking closed\-source models is, in most cases, not possible\)\. Because proprietary models are often substantially larger, they may behave differently under watermarking\. Finally, our judge pipeline, while validated against two board\-certified physicians on a subset of the data, must reliably detect even subtle failure modes \(e\.g\., hallucinated image findings\), which requires a judge \(LLM or VLM\) substantially more capable than the models being evaluated\.

##### Future work: improving watermark design\.

The watermarking methods we test are model\- and domain\-agnostic and therefore do not account for the clinical importance of specific tokens\. It remains unclear whether domain\-adapted schemes would outperform generic ones or how such schemes should be designed in practice\. Given the societal impacts, we believe future work should explore watermarking methods that better preserve clinically critical content\.

## Acknowledgments and Disclosure of Funding

Robin Staab has been supported by the SERI grant SAFEAI \(Certified Safe, Fair and Robust Artificial Intelligence, contract no\. MB22\.00088\)\. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or European Commission\. Neither the European Union nor the European Commission can be held responsible for them\. The work has received funding from the Swiss State Secretariat for Education, Research and Innovation \(SERI\) \(SERI\-funded ERC Consolidator Grant\)\. We thank Dr\. Andrea Radebold \(Internal Medicine, Quinnipiac Medical of Branford/Yale School of Medicine\) for her contribution to data annotation\.

## References

- American Medical Association \[2026\]American Medical Association\.2026 physician survey on augmented intelligence\.[https://www\.ama\-assn\.org/system/files/physician\-ai\-sentiment\-report\.pdf](https://www.ama-assn.org/system/files/physician-ai-sentiment-report.pdf), 2026\.Accessed: 2026\-04\-30\.
- Gowda et al\. \[2026\]Anirudh S\. Gowda, Meer C\. Chhabria, and Ryan C\. Lam\.Use of large language models on radiology reports: A scoping review\.*Journal of the American College of Radiology*, 23\(3\):437–454, 2026\.doi:10\.1016/j\.jacr\.2025\.10\.005\.Published online 2025\.
- Weissman et al\. \[2025\]Gary E\. Weissman, Toni Mankowitz, and Genevieve P\. Kanter\.Unregulated large language models produce medical device\-like output\.*npj Digital Medicine*, 8\(1\):148, 2025\.doi:10\.1038/s41746\-025\-01544\-y\.
- European Union \[2024\]European Union\.Regulation \(EU\) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence \(AI Act\)\.Official Journal of the European Union, L 2024/1689, 2024\.
- European Commission \[2026\]European Commission\.Second draft code of practice on marking and labelling of AI\-generated content\.[https://digital\-strategy\.ec\.europa\.eu/en/library/commission\-publishes\-second\-draft\-code\-practice\-marking\-and\-labelling\-ai\-generated\-content](https://digital-strategy.ec.europa.eu/en/library/commission-publishes-second-draft-code-practice-marking-and-labelling-ai-generated-content), 2026\.Published: 2026\-03\-05\. Accessed: 2026\-04\-30\.
- Tu et al\. \[2024\]Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, and Juanzi Li\.WaterBench: Towards holistic evaluation of watermarks for large language models\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, Bangkok, Thailand, August 2024\. Association for Computational Linguistics\.URL[https://aclanthology\.org/2024\.acl\-long\.83/](https://aclanthology.org/2024.acl-long.83/)\.
- Moll et al\. \[2025\]Johannes Moll, Markus Graf, Tristan Lemke, Nicolas Lenhart, Daniel Truhn, Jean\-Benoit Delbrouck, Jiazhen Pan, Daniel Rueckert, Lisa C\. Adams, and Keno K\. Bressem\.Evaluating reasoning faithfulness in medical vision\-language models using multimodal perturbations\.In*Proceedings of Machine Learning for Health \(ML4H\)*, volume 297 of*Proceedings of Machine Learning Research*, 2025\.arXiv:2510\.11196v2\.
- Hager et al\. \[2024\]Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, Jakob Vielhauer, Marcus Makowski, Rickmer Braren, Georgios Kaissis, and Daniel Rueckert\.Evaluation and mitigation of the limitations of large language models in clinical decision\-making\.*Nature Medicine*, 30:2613–2622, 2024\.doi:10\.1038/s41591\-024\-03097\-1\.
- Jin et al\. \[2021\]Di Jin, Eileen Pan, Nassim Oufattole, Wei\-Hung Weng, Hanyi Fang, and Peter Szolovits\.What disease does this patient have? a large\-scale open domain question answering dataset from medical exams\.*Applied Sciences*, 11\(14\):6421, 2021\.
- Kim and Yoon \[2025\]Siun Kim and Hyung\-Jin Yoon\.Questioning our questions: How well do medical QA benchmarks evaluate clinical capabilities of language models?In*Proceedings of the 24th Workshop on Biomedical Language Processing \(BioNLP\)*, pages 274–296, Vienna, Austria, 2025\. Association for Computational Linguistics\.doi:10\.18653/v1/2025\.bionlp\-1\.24\.
- Hu et al\. \[2024\]Y\. Hu et al\.Omnimedvqa: A new large\-scale comprehensive evaluation benchmark for medical lvlm\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2024\.
- Chen et al\. \[2024a\]Pengcheng Chen, Jin Ye, Guoan Wang, Yanjun Li, Zhongying Deng, et al\.GMAI\-MMBench: A comprehensive multimodal evaluation benchmark towards general medical AI\.*arXiv preprint arXiv:2408\.03361*, 2024a\.
- Zuo et al\. \[2025\]Yuxin Zuo, Shang Qu, Yifei Li, Zhang\-Ren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou\.MedXpertQA: Benchmarking expert\-level medical reasoning and understanding\.In*Proceedings of the 42nd International Conference on Machine Learning \(ICML\)*, volume 267 of*Proceedings of Machine Learning Research*, pages 80961–80990\. PMLR, 2025\.arXiv:2501\.18362\.
- Maharana et al\. \[2025\]Umakanta Maharana, Sarthak Verma, Avarna Agarwal, Prakashini Mruthyunjaya, Dwarikanath Mahapatra, Sakir Ahmed, and Murari Mandal\.Right prediction, wrong reasoning: Uncovering LLM misalignment in RA disease diagnosis\.*arXiv preprint arXiv:2504\.06581*, 2025\.
- Pal et al\. \[2023\]Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu\.Med\-HALT: Medical domain hallucination test for large language models\.In*Proceedings of the 27th Conference on Computational Natural Language Learning \(CoNLL\)*, pages 314–334, Singapore, 2023\. Association for Computational Linguistics\.doi:10\.18653/v1/2023\.conll\-1\.21\.
- Kocaman et al\. \[2025\]Veysel Kocaman, Mustafa Aytuğ Kaya, Andrei Marian Feier, and David Talby\.Clinical large language model evaluation by expert review \(CLEVER\): Framework development and validation\.*JMIR AI*, 4\(1\):e72153, 2025\.doi:10\.2196/72153\.
- Arora et al\. \[2025\]Rahul K\. Arora, Jason Wei, Rebecca S\. Hicks, et al\.HealthBench: Evaluating large language models towards improved human health\.*arXiv preprint arXiv:2505\.08775*, 2025\.
- Liu et al\. \[2024\]Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Lijie Wen, Irwin King, and Philip S\. Yu\.A survey of text watermarking in the era of large language models\.*ACM Computing Surveys*, 57:47:1–47:36, 2024\.doi:10\.1145/3691626\.URL[https://arxiv\.org/abs/2312\.07913](https://arxiv.org/abs/2312.07913)\.
- Kirchenbauer et al\. \[2023\]John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein\.A watermark for large language models\.In*International Conference on Machine Learning*, pages 17061–17084\. PMLR, 2023\.
- Gloaguen et al\. \[2026\]Thibaud Gloaguen, Robin Staab, Nikola Jovanović, and Martin Vechev\.A unified framework for llm watermarks, 2026\.URL[https://arxiv\.org/abs/2602\.06754](https://arxiv.org/abs/2602.06754)\.
- Wu et al\. \[2023\]Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang\.Dipmark: A stealthy, efficient and resilient watermark for large language models\.2023\.
- Aaronson \[2023\]Scott Aaronson\.Watermarking of large language models\.In*Workshop on Large Language Models and Transformers, Simons Institute, UC Berkeley*, 2023\.
- Dathathri et al\. \[2024\]Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po\-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al\.Scalable watermarking for identifying large language model outputs\.*Nature*, 634\(8035\):818–823, 2024\.
- Kuditipudi et al\. \[2024\]Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang\.Robust distortion\-free watermarks for language models\.2024\.
- Yang et al\. \[2025\]Peiru Yang, Xintian Li, Wanchun Ni, Jinhua Yin, Huili Wang, Guoshun Nan, Shangguang Wang, Yongfeng Huang, and Tao Qi\.Enhancing watermarking quality for LLMs via contextual generation states awareness\.*arXiv preprint arXiv:2506\.07403*, 2025\.
- Lee et al\. \[2024\]Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim\.Who wrote this code? watermarking for code generation\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\), Volume 1: Long Papers*, 2024\.
- Chen et al\. \[2024b\]Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, and Kam\-Fai Wong\.WatME: Towards lossless watermarking through lexical redundancy\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 9166–9180, Bangkok, Thailand, August 2024b\. Association for Computational Linguistics\.doi:10\.18653/v1/2024\.acl\-long\.496\.URL[https://aclanthology\.org/2024\.acl\-long\.496/](https://aclanthology.org/2024.acl-long.496/)\.
- Liu et al\. \[2026\]Shuliang Liu, Xingyu Li, Hongyi Liu, Dong Fang, Duan Bingchen, Zheng Qi, Lingfeng Su, and Xuming Hu\.Distilling the thought, watermarking the answer: A principle semantic guided watermark for reasoning large language models\.*OpenReview*, 2026\.URL[https://openreview\.net/forum?id=T6NVogsXCZ](https://openreview.net/forum?id=T6NVogsXCZ)\.ICLR 2026 Poster\.
- Hastuti et al\. \[2025\]Rochana Prih Hastuti, Rian Adam Rajagede, Mansour Al Ghanim, Mengxin Zheng, and Qian Lou\.Factuality beyond coherence: Evaluating LLM watermarking methods for medical texts\.In*Findings of the Association for Computational Linguistics: EMNLP 2025*, pages 15129–15147, Suzhou, China, 2025\. Association for Computational Linguistics\.doi:10\.18653/v1/2025\.findings\-emnlp\.818\.arXiv:2509\.07755\.
- Grattafiori et al\. \[2024\]Aaron Grattafiori et al\.The Llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*, 2024\.
- Gemma Team \[2025\]Gemma Team\.Gemma 3 technical report\.*arXiv preprint arXiv:2503\.19786*, 2025\.
- Abdin et al\. \[2024\]Marah Abdin et al\.Phi\-4 technical report\.*arXiv preprint arXiv:2412\.08905*, 2024\.
- Pal and Sankarasubbu \[2024\]Ankit Pal and Malaikannan Sankarasubbu\.OpenBioLLMs: Advancing open\-source large language models for healthcare and life sciences\.[https://huggingface\.co/aaditya/Llama3\-OpenBioLLM\-70B](https://huggingface.co/aaditya/Llama3-OpenBioLLM-70B), 2024\.
- Zhang et al\. \[2024\]Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang\-Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, Xingtai Lv, Jin\-Fang Hu, Zhiyuan Liu, and Bowen Zhou\.UltraMedical: Building specialized generalists in biomedicine\.In*Advances in Neural Information Processing Systems 37: Datasets and Benchmarks Track \(NeurIPS\)*, 2024\.Spotlight; arXiv:2406\.03949\.
- Guo et al\. \[2025\]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al\.DeepSeek\-R1: Incentivizing reasoning capability in LLMs via reinforcement learning\.*Nature*, 645:633–638, 2025\.doi:10\.1038/s41586\-025\-09422\-z\.
- Qwen Team \[2026\]Qwen Team\.Qwen3\.5: Towards native multimodal agents, February 2026\.URL[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5)\.Blog post\.
- Qwen Team \[2025\]Qwen Team\.Qwen3 technical report\.*arXiv preprint*, 2025\.
- Google Open Source \[2026\]Google Open Source\.Gemma 4: Expanding the gemmaverse with Apache 2\.0, April 2026\.URL[https://opensource\.googleblog\.com/2026/03/gemma\-4\-expanding\-the\-gemmaverse\-with\-apache\-20\.html](https://opensource.googleblog.com/2026/03/gemma-4-expanding-the-gemmaverse-with-apache-20.html)\.Google Open Source Blog\.
- Xu et al\. \[2025\]Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, et al\.Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning\.*arXiv preprint arXiv:2506\.07044*, 2025\.
- Sellergren et al\. \[2025\]Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al\.MedGemma technical report\.*arXiv preprint arXiv:2507\.05201*, 2025\.

## Appendix AAdditional Experimental Details

### A\.1Watermarking Scheme Details

All five schemes share a common structure: at each token position, a hash of the preceding context seeds a pseudorandom number generator that assigns a scoregug\_\{u\}to every vocabulary entryuu\. The*sampling mechanism*then maps the original next\-token distributionppand the scoresggto a watermarked next\-token distributionq​\(g\)q\(g\), from which the next token is drawn\. The schemes differ in the score distribution, in this mapping, and consequently in their detectability–quality trade\-offs\. We refer toGloaguen et al\. \[[20](https://arxiv.org/html/2607.20462#bib.bib20)\]for a unified optimization\-theoretic treatment of several of the schemes below\. In all cases the score assignment and the reweighting are restricted to the top\-kkofppas configured in the vLLM sampling parameters\.

##### KGW\[[19](https://arxiv.org/html/2607.20462#bib.bib19)\]\.

The context\-dependent hash partitions the top\-kkvocabulary into a*green list*and a*red list*via i\.i\.d\. Bernoulli\(12\\tfrac\{1\}\{2\}\) draws \(gu∈\{0,1\}g\_\{u\}\\in\\\{0,1\\\}\), so that in expectation half of the tokens are green\. A constant logit biasδ\\deltais added to the green\-list tokens before sampling, yielding the soft logit\-bias watermark

q​\(g\)∝p​exp⁡\(δ​g\)\.q\(g\)\\;\\propto\\;p\\,\\exp\(\\delta\\,g\)\.\(1\)This is the soft Kirchenbauer scheme; we do not use the hard variant that excludes red\-list tokens entirely\. Detection tests whether the proportion of green\-list tokens among the generated text exceeds the12\\tfrac\{1\}\{2\}chance level via a one\-sided binomial test \(and azz\-test for cross\-checking\)\. Watermark strength is controlled byδ\\delta\.

##### DipMark\[[21](https://arxiv.org/html/2607.20462#bib.bib21)\]\.

A distribution\-preserving scheme parameterized byα∈\(0,12\]\\alpha\\in\(0,\\tfrac\{1\}\{2\}\]\. The context hash seeds a uniform\-random permutationπ\\piof the top\-kkvocabulary; tokens are then reweighted according to the symmetricα\\alpha\-shift of the permuted CDF\. Concretely, in the permuted ordering the scheme averages two indicator\-style reweights at the boundariesα\\alphaand1−α1\-\\alphaof the cumulative mass, which atα=12\\alpha=\\tfrac\{1\}\{2\}yields exact distortion\-freeness in expectation,𝔼G​\[q​\(G\)\]=p\\mathbb\{E\}\_\{G\}\[q\(G\)\]=p, and asα→0\\alpha\\\!\\to\\\!0progressively biases mass toward the upper half of the permutation\. Detection compares each generated token’s rank under the recomputed permutation against the thresholdγ\\gamma: tokens above the threshold count as “green”, and we apply a one\-sided binomial test \(and azz\-test\) against1−γ1\-\\gamma\. We evaluate only the distortion\-free configurationα=γ=0\.5\\alpha=\\gamma=0\.5, as is standard in the literature\.

##### AAR\[[22](https://arxiv.org/html/2607.20462#bib.bib22)\]\.

For each token position, i\.i\.d\. Gumbel\-distributed scoresgug\_\{u\}\(location 0, scaleβ=1\\beta=1\) are drawn and the next token is selected deterministically as

u^=arg⁡maxu⁡\(gu\+τ​log⁡pu\),τ=β1\+β​δ,\\hat\{u\}\\;=\\;\\arg\\max\_\{u\}\\Bigl\(g\_\{u\}\+\\tau\\,\\log p\_\{u\}\\Bigr\),\\qquad\\tau\\;=\\;\\frac\{\\beta\}\{1\+\\beta\\,\\delta\},\(2\)and represented as a Dirac distribution atu^\\hat\{u\}\. We fixβ=1\\beta=1throughout and sweepδ≥0\\delta\\geq 0as the watermark\-strength knob\. Atδ=0\\delta=0we recover the canonical AAR samplerarg⁡maxu⁡\(gu\+log⁡pu\)\\arg\\max\_\{u\}\(g\_\{u\}\+\\log p\_\{u\}\): by the Gumbel\-max trick the marginal distribution ofu^\\hat\{u\}over the score draw is exactlypp, so the scheme is distortion\-free, and AAR is the optimal sampling mechanism among all distortion\-free watermarks that use a sum\-based detector\[[20](https://arxiv.org/html/2607.20462#bib.bib20)\], corresponding to the soft KL constraintKL​\(𝔼G​\[q​\(G\)\]∥p\)=0\\mathrm\{KL\}\(\\mathbb\{E\}\_\{G\}\[q\(G\)\]\\,\\\|\\,p\)=0\. Forδ\>0\\delta\>0,τ<1\\tau<1down\-weightslog⁡p\\log prelative to the Gumbel score, biasing the argmax toward the score sequence and yielding a more easily detectable but no\-longer\-distortion\-free signal\.

##### PPL\[[20](https://arxiv.org/html/2607.20462#bib.bib20)\]\.

For each token position, the scheme draws i\.i\.d\. Bernoulli\-sum scoresgu∼Binomial​\(N,12\)g\_\{u\}\\sim\\mathrm\{Binomial\}\(N,\\tfrac\{1\}\{2\}\)and emits a Dirac distribution at the*tilted argmax*

u^​\(β\)=arg⁡maxu⁡\(gu\+β​log⁡pu\),\\hat\{u\}\(\\beta\)\\;=\\;\\arg\\max\_\{u\}\\bigl\(g\_\{u\}\+\\beta\\,\\log p\_\{u\}\\bigr\),\(3\)where the tiltβ≥0\\beta\\geq 0is solved per token by 1D bisection so that the expected log\-likelihood ofu^\\hat\{u\}exactly saturates the per\-token log\-perplexity budget,

𝔼G​\[log⁡pu^​\(β\)\]=∑vpv​log⁡pv−ε\.\\mathbb\{E\}\_\{G\}\\bigl\[\\log p\_\{\\hat\{u\}\(\\beta\)\}\\bigr\]\\;=\\;\\sum\_\{v\}p\_\{v\}\\log p\_\{v\}\\;\-\\;\\varepsilon\.\(4\)The constraint is the same per\-token log\-perplexity budget\(p−q​\(g\)\)⋅log⁡p≤ε\(p\-q\(g\)\)\\cdot\\log p\\leq\\varepsilonas the*hard*variant from the unified framework, but enforced*in expectation over the scores*via the scalar tiltβ\\betarather than by a per\-realization linear program; in practice this matches the constraint while admitting the simple Gumbel\-style implementation\. The expectation in the bisection is approximated by 128 Monte Carlo score draws, with a fixed schedule of 60 bisection iterations and the search bracketβ∈\[0,100\]\\beta\\in\[0,100\]\. Watermark strength is controlled byε\\varepsilon\(largerε\\varepsilonpermits more deviation from the model’s distribution and thus a stronger watermark\)\.

##### SynthID\[[23](https://arxiv.org/html/2607.20462#bib.bib23)\]\.

For each token,mmindependent layers of i\.i\.d\. Bernoulli scoresg\(i\)∈\{0,1\}\|V\|g^\{\(i\)\}\\in\\\{0,1\\\}^\{\|V\|\}are drawn \(in our configurationm=30m=30\)\. Starting fromq\(0\):=pq^\{\(0\)\}:=p, the algorithm appliesmmtournament reweighting layers parameterized by the number of leaves per matchℓ\\ell, controlling the strength of the watermark\. Withℓ=2\\ell=2\(*non\-distortionary*tournament\), each layer applies the multiplicative update

qu\(i\)=qu\(i−1\)​\(1\+gu\(i\)−q\(i−1\)⋅g\(i\)\),q^\{\(i\)\}\_\{u\}\\;=\\;q^\{\(i\-1\)\}\_\{u\}\\bigl\(1\+g^\{\(i\)\}\_\{u\}\-q^\{\(i\-1\)\}\\\!\\cdot g^\{\(i\)\}\\bigr\),\(5\)which preserves the marginal distribution𝔼g\(i\)​\[q\(i\)\]=q\(i−1\)\\mathbb\{E\}\_\{g^\{\(i\)\}\}\[q^\{\(i\)\}\]=q^\{\(i\-1\)\}\. Withℓ\>2\\ell\>2leaves per match \(*distortionary*tournament\), the layer instead applies the closed\-form tournament\-winner reweight

qu\(i\)=qu\(i−1\)⋅\{1−\(1−mi\)ℓmiif​gu\(i\)=1,\(1−mi\)ℓ−1if​gu\(i\)=0,mi:=q\(i−1\)⋅g\(i\),q^\{\(i\)\}\_\{u\}\\;=\\;q^\{\(i\-1\)\}\_\{u\}\\cdot\\begin\{cases\}\\dfrac\{1\-\(1\-m\_\{i\}\)^\{\\ell\}\}\{m\_\{i\}\}&\\text\{if \}g^\{\(i\)\}\_\{u\}=1,\\\\\[6\.0pt\] \(1\-m\_\{i\}\)^\{\\ell\-1\}&\\text\{if \}g^\{\(i\)\}\_\{u\}=0,\\end\{cases\}\\qquad m\_\{i\}:=q^\{\(i\-1\)\}\\\!\\cdot g^\{\(i\)\},\(6\)which biases each layer toward tokens withgu\(i\)=1g^\{\(i\)\}\_\{u\}=1and is no longer distortion\-free forℓ\>2\\ell\>2\. Stacking layers progressively strengthens the watermark signal\. The original SynthID\-Text scheme ofDathathri et al\. \[[23](https://arxiv.org/html/2607.20462#bib.bib23)\]is typically configured withℓ=2\\ell=2leaves \(the non\-distortionary tournament\) and a depth in the rangem∈\{20,30\}m\\in\\\{20,30\\\}, with detectability controlled by varyingmm\.

### A\.2Generation and Reproducibility

All completions are generated with vLLM \(v0\.19\.1\), with the watermark realised as a customLogitsProcessorso that watermarked and unwatermarked runs differ only at the logit\-modification step\. All experiments use the model\-native tokenizer without post\-processing\. Decoding uses temperature0\.70\.7and unrestricted top\-kkthroughout\. Each \(model, watermark, strength\) configuration is run with a fixed samplingseed\(default0\); paired watermarked/unwatermarked completions for a given question therefore share both prompt and seed, isolating the watermark as the only source of divergence\. The watermark uses output\-only context hashing, so the first44generated tokens \(one context window\) are not watermarked; this affects<1%<\\\!1\\%of tokens and is consistent between generation and detection\.

For non\-reasoning models we cap output at2,5002\{,\}500tokens, while for reasoning models we allocate a generation budget of10,00010\{,\}000tokens, and post\-hoc split out the answer portion \(after</think\>\) with a maximum of2,5002\{,\}500tokens for comparability with the non\-reasoning pipeline; the watermark is applied to both reasoning trace and answer tokens by default\. In practice, the2,5002\{,\}500\-token cap never truncated a well\-formed response; the only completions reaching the limit were degenerate outputs \(e\.g\. repetitive token loops\), which are captured by the format/termination failure axis of the single\-response audit\. Answer letters are extracted by a two\-tier regex parser\. A primary pass scans for structured formats observed across models —ANSWER: X,FINAL ANSWER: X,ANSWER: \(X\),ANSWER: <X\>,\\boxed\{X\},FINAL ANSWER is X, andFINAL ANSWER:<free text\>\(X\)where exactly one letter appears parenthesised\. If, and only if, the primary pass finds nothing, a fallback pass admits conservative natural\-language patterns \(e\.g\. “the answer is X”, “I would choose X”, or a candidate name followed by a parenthesised letter at end of line\)\. The parser only commits to a letter when exactly one option \(A\\mathrm\{A\}–E\\mathrm\{E\}\) is unambiguously indicated; otherwise the response is recorded as unanswered\.

All experiments run on a single node with up to88NVIDIA RTX Blackwell60006000Pro GPUs \(9696GB VRAM each\)\. Watermark implementations, generation and analysis pipelines are released alongside the paper to enable reproduction of the reported numbers\.

### A\.3Existing Asset Licenses

We use existing datasets and model checkpoints only for evaluation, and cite the original creators in[Section˜3\.1](https://arxiv.org/html/2607.20462#S3.SS1)\. The licenses below are the license names stated by the corresponding official repositories or model cards\.

##### Datasets\.

- •*MedQA*: MIT License\.
- •*MedXpertQA\-MM*: MIT License\.

##### Models\.

- •*Llama\-3\.1\-8B*and*Llama\-3\.1\-70B*: Meta Llama 3 Community License Agreement\.
- •*BioMistral\-7B*: Apache License 2\.0\.
- •*Gemma\-3\-12B*: Gemma Terms of Use\.
- •*Phi\-4\-14B*: MIT License\.
- •*OpenBioLLM\-70B*and*UltraMedical\-70B*: Meta Llama 3 Community License Agreement\.
- •*DeepSeek\-R1\-Distill\-Llama\-8B*,*DeepSeek\-R1\-Distill\-Llama\-70B*, and*DeepSeek\-R1\-Distill\-Qwen\-32B*: MIT License; the Llama\-derived variants also inherit the applicable Meta Llama Community License terms, and the Qwen\-derived variant inherits the applicable Apache License 2\.0 upstream terms\.
- •*Qwen\-3\.5\-27B*,*Qwen\-3\-VL\-8B*, and*Qwen\-3\-VL\-32B*: Apache License 2\.0\.
- •*Gemma\-4\-31B*: Apache License 2\.0\.
- •*Lingshu\-7B*and*Lingshu\-32B*: MIT License\.
- •*MedGemma\-4B*and*MedGemma\-27B*: Health AI Developer Foundations Terms of Use\.

## Appendix BPer\-Configuration Accuracy and Perplexity

[Tables˜4](https://arxiv.org/html/2607.20462#A2.T4)and[5](https://arxiv.org/html/2607.20462#A2.T5)report detectability \(TPR at1%1\\%FPR\) and benchmark accuracy for every \(model, scheme, strength\) triple evaluated in[Section˜4\.1](https://arxiv.org/html/2607.20462#S4.SS1), one benchmark per table\. Rows are grouped by scheme and sorted by increasing watermark strength; each model contributes a \(TPR, Acc\) sub\-column pair, with the unwatermarked baseline shown beneath the model header\. Configurations strictly stronger than the first to reach TPR≥99%\\geq 99\\%are omitted \(—\), as they yield no additional detectability\.

Table 4:MedQA: watermark detectability \(TPR @ 1%FPR, in %\) and benchmark accuracy \(Acc, in %\) for every \(model, scheme, strength\) configuration evaluated in[Section˜4\.1](https://arxiv.org/html/2607.20462#S4.SS1), computed overN=2000N=2000questions \(full benchmark\)\. Per\-model unwatermarked baseline accuracy is shown beneath each model header\. For each \(model, scheme\) we report all configurations up to and including the first one with TPR≥\\geq99%; stronger configurations beyond that saturation point are omitted \(—\), as they yield no additional detectability\.Gemma\-3\-12B\(68\.2\)Llama\-3\.1\-70B\(91\.0\)Llama\-3\.1\-8B\(72\.8\)OpenBioLLM\-70B\(73\.8\)Phi\-4\-14B\(81\.5\)UltraMedical\-70B\(86\.6\)SchemeStrengthTPRAccTPRAccTPRAccTPRAccTPRAccTPRAccAARδ=0\\delta=096\.369\.237\.389\.591\.468\.536\.676\.070\.082\.567\.887\.0AARδ=0\.1\\delta=0\.198\.268\.547\.190\.195\.768\.945\.374\.5——76\.586\.9AARδ=0\.2\\delta=0\.299\.369\.255\.689\.698\.165\.851\.773\.786\.081\.384\.486\.8AARδ=0\.3\\delta=0\.3——64\.790\.899\.462\.059\.174\.992\.081\.690\.586\.2AARδ=0\.4\\delta=0\.4——73\.189\.5——66\.772\.796\.280\.893\.586\.5AARδ=0\.5\\delta=0\.5——80\.789\.8——76\.172\.098\.179\.795\.685\.4DipMarkα=0\.5\\alpha=0\.590\.168\.530\.289\.883\.867\.225\.472\.059\.881\.157\.787\.5KGWδ=0\.5\\delta=0\.516\.969\.26\.490\.118\.670\.53\.174\.65\.981\.811\.787\.6KGWδ=1\\delta=165\.169\.722\.290\.063\.069\.611\.474\.927\.582\.241\.286\.7KGWδ=2\\delta=299\.468\.870\.789\.699\.264\.551\.473\.487\.380\.791\.485\.9KGWδ=3\\delta=3——95\.290\.0——85\.569\.099\.280\.399\.185\.2KGWδ=4\\delta=4——99\.190\.4——95\.963\.2————KGWδ=5\\delta=5——————98\.854\.9————PPLε=0\\varepsilon=099\.568\.481\.789\.699\.168\.357\.273\.490\.878\.894\.480\.2PPLε=0\.1\\varepsilon=0\.1——92\.888\.2——73\.973\.596\.778\.897\.778\.5PPLε=0\.2\\varepsilon=0\.2——97\.889\.3——86\.671\.9——99\.370\.0PPLε=0\.3\\varepsilon=0\.3——99\.387\.9——92\.769\.899\.877\.5——PPLε=0\.4\\varepsilon=0\.4——————95\.469\.2————PPLε=0\.5\\varepsilon=0\.5——————97\.567\.5————SynthIDℓ=2\\ell=291\.169\.538\.389\.885\.971\.722\.975\.948\.481\.267\.487\.0SynthIDℓ=3\\ell=398\.969\.563\.989\.698\.365\.048\.173\.984\.580\.889\.284\.0SynthIDℓ=4\\ell=4100\.068\.879\.489\.6100\.048\.470\.571\.297\.280\.096\.381\.2SynthIDℓ=5\\ell=5——87\.990\.4——83\.567\.299\.179\.398\.876\.8SynthIDℓ=6\\ell=6——93\.490\.3——93\.359\.2——99\.263\.1
Table 5:MedXpertQA\-MM: watermark detectability \(TPR @ 1%FPR, in %\) and benchmark accuracy \(Acc, in %\) for every \(model, scheme, strength\) configuration evaluated in[Section˜4\.1](https://arxiv.org/html/2607.20462#S4.SS1), computed overN=2000N=2000questions \(full benchmark\)\. Per\-model unwatermarked baseline accuracy is shown beneath each model header\. For each \(model, scheme\) we report all configurations up to and including the first one with TPR≥\\geq99%; stronger configurations beyond that saturation point are omitted \(—\), as they yield no additional detectability\.Gemma\-4\-31B\(54\.9\)Lingshu\-32B\(29\.9\)Lingshu\-7B\(24\.3\)MedGemma\-27B\(32\.8\)MedGemma\-4B\(24\.4\)Qwen\-3\-VL\-32B\(38\.9\)Qwen\-3\-VL\-8B\(29\.8\)SchemeStrengthTPRAccTPRAccTPRAccTPRAccTPRAccTPRAccTPRAccAARδ=0\\delta=065\.255\.472\.631\.473\.723\.890\.033\.684\.224\.293\.539\.197\.926\.5AARδ=0\.1\\delta=0\.171\.255\.082\.529\.682\.923\.594\.234\.291\.224\.795\.539\.398\.627\.0AARδ=0\.2\\delta=0\.279\.554\.689\.528\.891\.423\.496\.633\.094\.024\.897\.940\.499\.127\.3AARδ=0\.3\\delta=0\.384\.854\.793\.830\.194\.524\.298\.033\.596\.824\.098\.939\.0——AARδ=0\.4\\delta=0\.490\.554\.196\.230\.497\.523\.898\.932\.198\.223\.999\.538\.7——AARδ=0\.5\\delta=0\.592\.853\.698\.330\.298\.523\.599\.631\.999\.022\.2————DipMarkα=0\.5\\alpha=0\.545\.654\.457\.031\.257\.525\.782\.032\.472\.523\.687\.139\.094\.529\.0KGWδ=0\.5\\delta=0\.57\.154\.39\.230\.610\.124\.914\.834\.114\.122\.117\.138\.830\.327\.8KGWδ=1\\delta=126\.954\.034\.131\.838\.923\.459\.232\.554\.023\.262\.340\.181\.229\.1KGWδ=2\\delta=281\.054\.491\.029\.591\.022\.497\.033\.691\.823\.898\.039\.799\.229\.2KGWδ=3\\delta=396\.954\.499\.230\.499\.025\.2100\.032\.596\.521\.599\.938\.5——KGWδ=4\\delta=499\.954\.0————————————KGWδ=5\\delta=5————————97\.221\.8————PPLε=0\\varepsilon=088\.953\.392\.830\.694\.723\.998\.633\.191\.122\.499\.137\.499\.626\.9PPLε=0\.1\\varepsilon=0\.196\.152\.697\.829\.697\.423\.299\.732\.5——————PPLε=0\.2\\varepsilon=0\.298\.853\.899\.429\.299\.223\.9——94\.021\.9————PPLε=0\.3\\varepsilon=0\.399\.653\.4————————————PPLε=0\.4\\varepsilon=0\.4————————96\.421\.1————PPLε=0\.5\\varepsilon=0\.5————————96\.721\.1————SynthIDℓ=2\\ell=261\.253\.863\.330\.469\.023\.892\.931\.988\.624\.589\.040\.094\.826\.7SynthIDℓ=3\\ell=381\.253\.889\.630\.392\.524\.198\.732\.995\.022\.698\.138\.699\.426\.4SynthIDℓ=4\\ell=490\.753\.898\.027\.998\.024\.399\.932\.696\.521\.699\.539\.0——SynthIDℓ=5\\ell=595\.452\.999\.426\.699\.823\.0——96\.620\.1————
ModelConditionAcc\. \(%\)TPR \(%\)Mean output length \(chars\)DeepSeek\-R1\-Distill\-Llama\-8BNo WM59\.2–9,023FA\-only,δ=1\\delta\{=\}161\.014\.38,890FA\-only,δ=3\\delta\{=\}359\.287\.18,856FA\-only,δ=5\\delta\{=\}558\.996\.88,870Full WM,δ=1\\delta\{=\}161\.215\.49,165Full WM,δ=3\\delta\{=\}359\.193\.011,621Full WM,δ=5\\delta\{=\}557\.298\.915,257DeepSeek\-R1\-Distill\-Qwen\-32BNo WM83\.6–6,447FA\-only,δ=1\\delta\{=\}183\.68\.16,362FA\-only,δ=3\\delta\{=\}382\.974\.36,444FA\-only,δ=5\\delta\{=\}581\.597\.76,482Full WM,δ=1\\delta\{=\}184\.110\.36,737Full WM,δ=3\\delta\{=\}383\.784\.48,166Full WM,δ=5\\delta\{=\}582\.199\.610,027DeepSeek\-R1\-Distill\-Llama\-70BNo WM92\.3–6,025FA\-only,δ=1\\delta\{=\}192\.47\.05,995FA\-only,δ=3\\delta\{=\}391\.959\.26,072FA\-only,δ=5\\delta\{=\}590\.892\.46,088Full WM,δ=1\\delta\{=\}191\.97\.16,239Full WM,δ=3\\delta\{=\}391\.482\.37,227Full WM,δ=5\\delta\{=\}589\.899\.78,759Qwen\-3\.5\-27BNo WM94\.6–13,258FA\-only,δ=1\\delta\{=\}194\.123\.513,125FA\-only,δ=3\\delta\{=\}394\.597\.513,285FA\-only,δ=5\\delta\{=\}594\.198\.813,743Full WM,δ=1\\delta\{=\}194\.826\.113,285Full WM,δ=3\\delta\{=\}394\.099\.014,976Full WM,δ=5\\delta\{=\}577\.199\.518,019Table 6:MedQA accuracy, TPR at1%1\\%FPR, and mean output length \(reasoning trace plus final answer, characters\) for four reasoning models under No WM, FA\-only \(post\-<thinking\>tokens only\), and Full WM \(entire output\), KGWδ∈\{1,3,5\}\\delta\\in\\\{1,3,5\\\}\. TPR is computed on the final\-answer tokens only\.### B\.1Perplexity Under Watermarking

[Figures˜5](https://arxiv.org/html/2607.20462#A2.F5)and[6](https://arxiv.org/html/2607.20462#A2.F6)plot generation perplexity against watermark detectability\. Perplexity increases monotonically with watermark strength on every model and scheme, but its magnitude is a poor predictor of downstream harm:Gemma\-3\-12Bshows clear perplexity shifts under KGW with no accuracy effect, whileUltraMedical\-70Bsuffers a3131pp accuracy collapse under PPL despite only moderate perplexity shifts\. Exact per\-configuration values accompany the code release\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x5.png)Figure 5:MedQA: generation\-perplexity shift vs\. watermark detectability, six non\-reasoning LLMs\.Δ\\DeltaPPL==PPL−wm\{\}\_\{\\mathrm\{wm\}\}\-PPLbase\. Monotonic in strength on every model×\\timesscheme cell; absolute magnitude spans an order of magnitude \(note differingyy\-axes\)\.Llama\-3\.1\-8Bhas the steepest increases;Gemma\-3\-12BandUltraMedical\-70Bshow contained shifts despite their sharply different accuracy robustness\.![Refer to caption](https://arxiv.org/html/2607.20462v1/x6.png)Figure 6:MedXpertQA\-MM: generation\-perplexity shift vs\. watermark detectability, seven VLMs\. Same overall shape as MedQA; KGW and SynthID drive the steepest high\-TPR increases\.Lingshu\-32BandLingshu\-7Bshow pronounced jumps at the highest strengths despite minimal accuracy changes \(cf\.[Figure˜3](https://arxiv.org/html/2607.20462#S4.F3)\)\.

## Appendix CAudit Outcomes Across Watermark Detectability

[Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)fixes a single deployment operating point per cell \(≥99%\\geq 99\\%TPR\), leaving the*shape*of the strength–harm relationship unresolved – whether harm grows smoothly with detectability, or sits near baseline up to a threshold and then jumps\. We sweep watermark strength on two contrasting cells:*Phi\-4\-14B*on MedQA \(an accuracy\-stable text\-only LLM\) and*MedGemma\-4B*on MedXpertQA\-MM \(a smaller medical VLM\), under the two schemes whose≥99%\\geq 99\\%\-TPR signatures differ most in[Table˜2](https://arxiv.org/html/2607.20462#S4.T2)–*KGW*\(broad multi\-axis effect\) and*SynthID*\(the largest silent\-error cell on both models\)\.

[Figure˜7](https://arxiv.org/html/2607.20462#A3.F7)plots one harm metric per axis\-of\-interest against detectability\. ForPhi\-4\-14Bwe report the per\-response confident\-error rate \(a major component of F&C\) and the pairwise vignette\-fabrication rate \(the silent\-error signature picked up by VigFab\)\. ForMedGemma\-4Bwe report the net visual signal used in[Figure˜4](https://arxiv.org/html/2607.20462#S4.F4)and again vignette\-fabrication, so the silent\-error axis is comparable across the two cells\.

##### KGW vs\. SynthID: monotonic slider vs\. threshold cliff\.

KGW degrades smoothly with detectability\. OnPhi\-4\-14B, watermarked confident\-error climbs from14\.2%14\.2\\%at TPR5\.9%5\.9\\%to21\.6%21\.6\\%at TPR99\.9%99\.9\\%, against a stable≈12%\\approx 12\\%unwatermarked baseline; onMedGemma\-4B, the net visual signal slides from\+0\.09\+0\.09at TPR14%14\\%through zero to−0\.21\-0\.21at TPR97%97\\%\(baseline≈\+0\.10\\approx\+0\.10\)\. SynthID is threshold\-y: it tracks baseline up to TPR≈90%\\approx 90\\%and then jumps in the final detectability decade\. OnPhi\-4\-14B, confident\-error is flat at≈13\.7%\\approx 13\.7\\%across TPRs4848–84%84\\%, then rises to19\.1%19\.1\\%\(TPR97%97\\%\) and25\.0%25\.0\\%\(TPR99%99\\%\); pairwise vignette\-fabrication stays≤0\.8%\\leq 0\.8\\%up to TPR97%97\\%and jumps to3\.8%3\.8\\%at TPR99%99\\%\. OnMedGemma\-4B, vignette\-fabrication tracks5\.8%→7\.9%→18\.7%5\.8\\%\\to 7\.9\\%\\to 18\.7\\%at TPRs89,95,97%89,95,97\\%\.

These shapes have different operational consequences: a KGW deployment can move a slider to trade detectability against quality, whereas a SynthID deployment at the detectability a regulator would require sits past the elbow of a cliff\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x7.png)Figure 7:Audit outcomes vs\. watermark detectabilityonPhi\-4\-14B\(MedQA, top\) andMedGemma\-4B\(MedXpertQA\-MM, bottom\) under KGW \(blue\) and SynthID \(purple\)\. Each point is one \(scheme, strength\); dashed line and grey band give the unwatermarked baseline mean and95%95\\%\-bootstrap CI; dotted vertical marks TPR=99%=99\\%\. Top: per\-response confident\-error rate restricted to right\-letter responses \(left\); pairwise vignette\-fabrication rate \(right\)\. Bottom: net visual signal,supported−contradicted\\textsc\{supported\}\-\\textsc\{contradicted\}visual claims per question \(left\); pairwise vignette\-fabrication rate \(right\)\.
### C\.1Full Reasoning\-Audit Grids

[Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)reports only the two most\-affected models per benchmark\. Below: the full77\-model grids per benchmark, per\-scheme FabRatio breakdowns, and the all\-VLM companion to[Figure˜4](https://arxiv.org/html/2607.20462#S4.F4)\.

Table 7:Full reasoning audit at99%99\\%TPR on MedQA, for all77LLMs\. We report the accuracy \(Δ\\DeltaAcc\), the confident\-error rate \(Δ\\DeltaConfErr\), the faulty\-but\-correct rate \(Δ\\DeltaF&C\), the percentage of corrupted spellings \(Δ\\DeltaSpell/100\), the percentage of fabricated facts \(Δ\\DeltaFab/100\), the within\-response contradiction rate \(Δ\\DeltaContr\), vignette fabrication \(Δ\\DeltaVigFab\), clue misreads \(Δ\\DeltaMisread\), formatting failures \(Δ\\DeltaFmt\), linguistic corruption \(Δ\\DeltaLing\), the percentage of misapplied medical terms \(Δ\\DeltaMisapp/100\), the percentage of paired responses with major diagnostic divergence \(Major%\), and the vignette fabrication ratio \(FabRatio\)\. For unwatermarked rows we report the actual value, whereas for watermarked rows we report the difference/ratio with the unwatermarked baseline\. Weboldentries that are statistically significant \(p\-value below0\.010\.01\)\.ModelSchemeΔ\\DeltaAccΔ\\DeltaConfErrΔ\\DeltaF&CΔ\\DeltaSpell/100Δ\\DeltaFab/100Δ\\DeltaContrΔ\\DeltaVigFabΔ\\DeltaMisreadΔ\\DeltaFmtΔ\\DeltaLingΔ\\DeltaMisapp/100Major%FabRatioLlama\-3\.1\-8BUnwatermarked73\.335\.329\.60\.81\.422\.65\.86\.60\.90\.114\.7–1\.0×\\timesAAR\-0\.6\+8\.1\+7\.4\+5\.8\+4\.1\+6\.1\+5\.5\+3\.2\+1\.5\+6\.1\+0\.231\.92\.1×\\timesDipMark\-1\.6\-0\.7\+1\.0\+0\.9\+1\.0\+3\.1\+1\.3\+3\.4\-0\.3\+0\.5\-1\.529\.21\.6×\\timesKGW\-2\.3\+3\.0\+2\.4\+1\.8\+2\.2\+5\.1\+1\.7\+1\.6\+0\.5\+0\.3\-0\.329\.01\.0×\\timesPPL\-0\.7\+5\.4\+4\.3\+24\.6\+1\.0\+4\.2\+0\.5\+3\.8\+0\.3\+3\.4\+0\.729\.81\.6×\\timesSynthID\-2\.0\+3\.0\+2\.2\+2\.1\+1\.5\+4\.0\+3\.7\+2\.6\+1\.5\+0\.7\+0\.233\.51\.2×\\timesLlama\-3\.1\-70BUnwatermarked91\.86\.98\.10\.40\.15\.80\.41\.00\.00\.01\.5–1\.0×\\timesAAR\-1\.2\-0\.5\+0\.1\+0\.30\.0\+0\.1\+0\.1\+0\.50\.0\+0\.1\-0\.34\.1–DipMark\-1\.2\+1\.4\+1\.5\-0\.1\-0\.1\+1\.4\-0\.2\+0\.2\+0\.20\.00\.04\.0–KGW\-1\.3\+1\.8\+1\.9\+1\.00\.0\+1\.6\+0\.4\+0\.3\+0\.1\+0\.1\+0\.65\.11\.0×\\timesPPL\-1\.8\+3\.3\+7\.5\+89\.4\+1\.9\+2\.1\+0\.3\+3\.5\+0\.6\+30\.9\+0\.35\.0–SynthID\-0\.9\+2\.0\+1\.2\+6\.5\+0\.2\+0\.9\+0\.4\-0\.1\+0\.1\+3\.40\.05\.90\.0×\\timesGemma\-3\-12BUnwatermarked67\.029\.222\.50\.71\.320\.43\.44\.50\.00\.09\.0–1\.0×\\timesAAR\+1\.3\+2\.5\+3\.4\+0\.2\+0\.4\+3\.2\-0\.7\+0\.3\+0\.20\.0\+3\.621\.00\.9×\\timesDipMark\+1\.8\-1\.0\+1\.1\-0\.3\+0\.2\+1\.7\-0\.4\-0\.7\+0\.4\+0\.1\+1\.520\.90\.4×\\timesKGW\+1\.5\+0\.7\+1\.7\+0\.10\.0\+1\.9\+1\.0\+0\.1\+0\.10\.0\+0\.822\.41\.3×\\timesPPL\+1\.5\+2\.5\+3\.2\+20\.8\+1\.2\+2\.1\+0\.1\+3\.9\+0\.6\+3\.7\+3\.423\.11\.1×\\timesSynthID\+1\.9\+3\.0\+4\.5\+2\.6\+0\.3\+3\.0\+1\.1\+0\.5\+0\.2\+0\.1\+1\.522\.91\.1×\\timesOpenBioLLM\-70BUnwatermarked73\.613\.912\.90\.60\.08\.62\.22\.214\.80\.14\.5–1\.0×\\timesAAR\-0\.2\+6\.6\+4\.4\+0\.9\+0\.9\+3\.8\+1\.7\+0\.6\+4\.0\+0\.9\+1\.520\.60\.9×\\timesDipMark\-1\.1\+2\.3\+1\.5\+0\.3\+0\.4\+1\.4\+0\.8\+1\.2\+5\.3\+0\.1\+0\.818\.61\.1×\\timesKGW\-17\.4\+11\.4\+4\.2\+5\.2\+1\.9\+9\.3\+7\.0\+6\.6\+20\.2\+0\.3\+4\.424\.63\.5×\\timesPPL\-7\.7\+7\.1\+6\.7\+51\.5\+2\.1\+4\.1\+3\.9\+4\.6\+13\.0\+8\.5\+1\.823\.01\.9×\\timesSynthID\-11\.4\+24\.0\+14\.6\+25\.9\+13\.6\+12\.3\+13\.0\+7\.9\+15\.5\+13\.8\+6\.927\.86\.6×\\timesUltramedical 70BUnwatermarked86\.611\.011\.81\.00\.08\.00\.51\.523\.22\.22\.4–1\.0×\\timesAAR\-1\.5\+2\.3\+2\.5\+0\.5\+0\.4\+4\.5\+0\.9\+1\.8\-13\.1\-0\.9\+0\.610\.71\.0×\\timesDipMark\+1\.2\+2\.0\+1\.70\.00\.0\+0\.9\+0\.9\+0\.2\-1\.7\+0\.7\+0\.110\.20\.5×\\timesKGW\-1\.5\+2\.0\+1\.8\+0\.4\+0\.2\+0\.9\+1\.0\+0\.4\+10\.3\+1\.4\+0\.610\.33\.0×\\timesPPL\-39\.5\+6\.5\+0\.2\+97\.8\+1\.3\+2\.7\+2\.4\+7\.4\+27\.6\+22\.3\+2\.611\.422\.0×\\timesSynthID\-9\.2\+3\.9\+3\.3\+6\.1\+0\.5\+3\.5\+1\.3\+2\.7\+4\.1\+4\.1\+0\.810\.11\.2×\\timesPhi\-4\-14BUnwatermarked81\.412\.311\.40\.10\.37\.81\.62\.00\.40\.04\.1–1\.0×\\timesAAR\-2\.5\+6\.2\+6\.5\+1\.5\+1\.8\+4\.4\+2\.9\+2\.2\+0\.6\+0\.2\+4\.011\.314\.0×\\timesDipMark\-0\.9\+3\.9\+3\.6\+0\.3\+0\.1\+2\.0\+0\.6\-0\.1\+0\.20\.0\+2\.610\.80\.8×\\timesKGW\-0\.7\+9\.2\+9\.5\+2\.2\+1\.5\+5\.5\+3\.4\+3\.6\+0\.2\+0\.1\+3\.614\.23\.0×\\timesPPL\-8\.6\+10\.0\+9\.6\+69\.6\+3\.8\+5\.2\+3\.2\+4\.0\+5\.4\+10\.4\+4\.313\.418\.0×\\timesSynthID\-3\.2\+13\.7\+14\.9\+10\.6\+6\.6\+9\.4\+6\.8\+5\.5\+0\.7\+3\.9\+8\.015\.47\.6×\\timesBioMistral\-7BUnwatermarked40\.81\.11\.30\.30\.41\.81\.90\.77\.40\.00\.4–1\.0×\\timesAAR\-1\.1\+21\.2\+8\.4\+5\.2\+12\.3\+8\.6\+13\.3\+8\.2\+7\.3\+3\.4\+9\.125\.114\.0×\\timesDipMark\-1\.3\+7\.9\+3\.4\+0\.9\+1\.4\+5\.8\+3\.8\+2\.6\-6\.5\+0\.2\+0\.911\.54\.9×\\timesKGW\-38\.7\+27\.8\-0\.6\+19\.6\+9\.1\+16\.9\+13\.8\+11\.7\+28\.3\+3\.9\+16\.748\.918\.1×\\timesPPL\-5\.0\+13\.2\+5\.0\+18\.5\+1\.9\+8\.6\+5\.2\+5\.3\+11\.1\+2\.0\+5\.611\.89\.5×\\timesSynthID\-30\.4\+10\.5\+3\.4\+82\.3\+153\.4\+1\.8\+30\.4\+8\.2\+66\.7\+69\.0\+1\.363\.093\.0×\\times
- •Single\-response axes \(per100100questions where suffixed\): ConfErr \(confident error, on BOTH\-CORRECT pairs\), F&C \(faulty\-but\-correct: correct letter\+≥1\+\\geq 1defect\), Spell/100, Fab/100 \(fabricated entities\), Contr \(within\-response contradiction\), VigFab \(vignette fabrication\), Misread \(clue misread\), Fmt \(format/termination failure\), Ling \(linguistic corruption\), Misapp/100 \(misapplied real terms\)\. Pairwise: Major% \(MAJOR divergence on BOTH\-CORRECT pairs\), FabRatio \(WM/Base vignette\-fabrication rate\)\.

Table 8:Full reasoning audit at99%99\\%TPR on MedXpertQA\-MM, for all77VLMs\. We report the accuracy \(Δ\\DeltaAcc\), the confident\-error rate \(Δ\\DeltaConfErr\), the faulty\-but\-correct rate \(Δ\\DeltaF&C\), the percentage of corrupted spellings \(Δ\\DeltaSpell/100\), the percentage of fabricated facts \(Δ\\DeltaFab/100\), the image\-grounding net change \(Δ\\DeltaNet%\), the within\-response contradiction rate \(Δ\\DeltaContr\), vignette fabrication \(Δ\\DeltaVigFab\), clue misreads \(Δ\\DeltaMisread\), formatting failures \(Δ\\DeltaFmt\), linguistic corruption \(Δ\\DeltaLing\), the percentage of misapplied medical terms \(Δ\\DeltaMisapp/100\), supported visual claims \(Δ\\DeltaSup\), contradicted visual claims \(Δ\\DeltaCon\), the percentage of paired responses with major diagnostic divergence \(Major%\), and the vignette fabrication ratio \(FabRatio\)\. For unwatermarked rows we report the actual value, whereas for watermarked rows we report the difference/ratio with the unwatermarked baseline\. Weboldentries that are statistically significant \(p\-value below0\.010\.01\)\.ModelSchemeΔ\\DeltaAccΔ\\DeltaConfErrΔ\\DeltaF&CΔ\\DeltaSpell/100Δ\\DeltaFab/100Δ\\DeltaNet%Δ\\DeltaContrΔ\\DeltaVigFabΔ\\DeltaMisreadΔ\\DeltaFmtΔ\\DeltaLingΔ\\DeltaMisapp/100Δ\\DeltaSupΔ\\DeltaConMajor%FabRatioGemma\-4\-31BUnwatermarked56\.212\.29\.20\.10\.325\.00\.81\.90\.10\.01\.72\.550\.53–1\.0×\\timesAAR\-0\.6\-0\.6\-0\.3\+0\.9\+0\.3\+1\+0\.7\+0\.1\-0\.1\+0\.1\+0\.8\-1\.1\-0\.01\-0\.036\.03\.0×\\timesDipMark\-0\.1\-1\.0\-0\.3\+0\.5\+0\.2\+2\+0\.5\-0\.50\.0\+0\.10\.0\-0\.2\+0\.02\-0\.033\.62\.0×\\timesKGW\+0\.2\-1\.7\-0\.3\+1\.8\+0\.2\+1\+1\.7\-0\.2\+0\.3\+0\.2\+1\.5\+0\.2\+0\.04\+0\.016\.80\.5×\\timesPPL\-0\.9\+1\.6\+3\.1\+34\.5\+1\.5\-2\+1\.8\-0\.3\+3\.4\+2\.1\+22\.50\.0\-0\.04\-0\.015\.06\.0×\\timesSynthID\-1\.0\+0\.4\-0\.3\+7\.6\+0\.8\+3\+1\.0\-0\.1\-0\.3\+0\.1\+11\.0\-0\.9\+0\.02\-0\.046\.84\.0×\\timesLingshu\-7BUnwatermarked23\.955\.313\.70\.41\.5017\.03\.57\.60\.80\.512\.11\.150\.77–1\.0×\\timesAAR\-0\.5\+13\.8\+3\.7\+5\.4\+12\.9\-67\+3\.5\+7\.9\+3\.8\+2\.0\+3\.5\+18\.1\-0\.15\+0\.0933\.02\.8×\\timesDipMark\+1\.0\+5\.4\+1\.4\+0\.1\+0\.3\-6\+2\.7\+0\.10\.0\-0\.1\+0\.1\+2\.7\-0\.02\-0\.0022\.61\.0×\\timesKGW\-0\.8\+28\.6\+6\.0\+16\.2\+13\.2\-103\+20\.2\+10\.4\+11\.0\+5\.4\+4\.6\+2\.4\-0\.12\+0\.3742\.93\.6×\\timesPPL\-0\.6\+7\.3\+2\.3\+37\.9\+3\.9\-44\+3\.9\+3\.0\+7\.9\+5\.1\+8\.4\+11\.5\-0\.13\+0\.0320\.02\.4×\\timesSynthID\-2\.1\+26\.2\+5\.5\+18\.9\+39\.2\-105\+13\.2\+16\.8\+8\.5\+7\.7\+13\.5\+20\.2\-0\.14\+0\.2830\.57\.0×\\timesLingshu\-32BUnwatermarked31\.140\.815\.10\.80\.119\.62\.03\.70\.40\.15\.91\.380\.68–1\.0×\\timesAAR\-1\.8\+9\.2\+1\.3\+2\.9\+5\.4\-46\+4\.8\+5\.5\+3\.4\+2\.5\+3\.5\+2\.0\-0\.15\+0\.1718\.03\.8×\\timesDipMark\+0\.4\-1\.2\-1\.0\-0\.1\+0\.4\-16\+1\.0\+0\.1\+0\.6\-0\.20\.00\.0\-0\.05\+0\.0715\.30\.8×\\timesKGW\-4\.0\+10\.1\-0\.1\+1\.9\+1\.8\-39\+8\.9\+2\.4\+4\.8\+1\.1\+0\.8\+3\.6\-0\.05\+0\.2314\.41\.8×\\timesPPL\-2\.4\-3\.2\-2\.0\+27\.4\+0\.8\-26\+2\.3\+1\.2\+7\.1\+6\.7\+5\.3\-0\.6\-0\.06\+0\.1211\.22\.2×\\timesSynthID\-3\.8\+6\.7\+2\.0\+23\.9\+13\.7\-56\+16\.2\+9\.8\+9\.5\+8\.8\+13\.2\+5\.2\-0\.12\+0\.2821\.74\.9×\\timesMedGemma\-4BUnwatermarked24\.156\.916\.51\.01\.9029\.73\.59\.07\.70\.011\.11\.241\.11–1\.0×\\timesAAR\-1\.0\+13\.0\+1\.2\+3\.4\+6\.5\-108\+2\.1\+7\.4\+3\.3\+11\.2\+4\.6\+8\.1\+0\.15\+0\.2938\.33\.1×\\timesDipMark\+0\.2\+11\.6\+1\.6\+0\.2\+0\.7\-103\-0\.9\+0\.1\-0\.2\-0\.2\+0\.3\+2\.0\-0\.08\+0\.0635\.01\.2×\\timesKGW\-2\.6\+19\.2\+0\.6\+15\.4\+8\.9\-266\+11\.2\+6\.5\+7\.4\+13\.1\+3\.5\+16\.2\-0\.01\+0\.3231\.73\.1×\\timesPPL\-1\.7\+8\.30\.0\+43\.6\+6\.0\-46\+4\.2\+3\.1\+8\.2\+8\.0\+9\.7\+5\.0\-0\.01\+0\.0534\.72\.1×\\timesSynthID\-3\.1\+24\.3\+0\.4\+18\.0\+12\.6\-168\+8\.5\+11\.1\+8\.1\+22\.1\+8\.7\+24\.3\-0\.05\+0\.2036\.04\.2×\\timesMedGemma\-27BUnwatermarked33\.741\.516\.90\.62\.3012\.41\.65\.82\.80\.09\.11\.901\.59–1\.0×\\timesAAR\-0\.9\-3\.0\-0\.8\+1\.5\+1\.9\-43\+2\.6\+2\.9\-0\.7\+2\.6\+0\.1\-2\.0\-0\.08\+0\.0619\.51\.6×\\timesDipMark\+1\.2\-5\.8\-0\.3\+0\.3\-0\.2\-16\+0\.2\+0\.1\-1\.1\+0\.7\+0\.1\-1\.0\-0\.06\+0\.0018\.41\.0×\\timesKGW\-1\.0\+4\.7\+1\.9\+0\.7\+0\.3\-41\+3\.3\+1\.6\+0\.8\+1\.6\+0\.2\+3\.1\-0\.01\+0\.1318\.51\.8×\\timesPPL\-0\.2\+7\.4\+3\.4\+34\.2\+2\.7\-23\+5\.5\+1\.5\+4\.5\+1\.8\+6\.7\-3\.0\-0\.04\+0\.0423\.01\.4×\\timesSynthID\-0\.3\-1\.9\-0\.8\+1\.5\+1\.3\+3\+2\.7\+0\.9\-0\.3\+0\.3\+0\.2\-1\.9\+0\.01\+0\.0022\.80\.9×\\timesQwen\-3\-VL\-8BUnwatermarked31\.347\.019\.90\.61\.8021\.73\.04\.97\.60\.611\.01\.951\.57–1\.0×\\timesAAR\-4\.30\.0\-5\.20\.0\+2\.5\-3\+1\.3\+0\.6\+0\.7\+7\.3\+0\.6\+7\.3\+0\.06\+0\.0722\.21\.8×\\timesDipMark\-2\.5\+2\.9\-1\.5\-0\.1\+1\.8\+17\+1\.5\+1\.1\+2\.6\+1\.2\-0\.2\+2\.3\+0\.06\-0\.0125\.01\.1×\\timesKGW\-1\.0\+1\.1\+0\.7\+0\.7\+1\.9\+8\+4\.5\+2\.2\+1\.7\+2\.9\+0\.1\+3\.4\+0\.07\+0\.0424\.61\.7×\\timesPPL\-5\.1\+2\.0\-2\.5\+22\.5\+1\.1\-22\-1\.1\+0\.6\+4\.7\+7\.3\+7\.2\+4\.7\-0\.10\-0\.0224\.91\.5×\\timesSynthID\-4\.5\+3\.1\-2\.1\+0\.1\+1\.4\-9\+2\.0\+1\.9\+0\.6\+6\.2\-0\.4\+5\.0\-0\.04\-0\.0024\.51\.0×\\timesQwen\-3\-VL\-32BUnwatermarked40\.827\.915\.70\.20\.7115\.31\.33\.83\.80\.23\.32\.691\.29–1\.0×\\timesAAR\-3\.0\+3\.1\-0\.7\+1\.5\+1\.9\-2\-1\.1\+0\.3\-0\.1\+0\.5\+3\.0\+2\.7\-0\.01\+0\.0215\.40\.6×\\timesDipMark\-1\.4\+5\.1\+1\.0\+0\.5\+0\.9\-3\-1\.0\+0\.10\.0\+0\.6\+0\.8\+4\.0\-0\.00\+0\.0414\.10\.8×\\timesKGW\-0\.9\+3\.9\-0\.1\+1\.1\+0\.2\+3\+1\.4\+0\.8\-0\.2\+0\.6\+1\.2\+2\.1\+0\.08\+0\.0415\.41\.0×\\timesPPL\-2\.6\+4\.1\-0\.6\+35\.6\+2\.0\-3\+1\.2\-0\.5\+2\.6\+2\.2\+11\.6\+1\.6\+0\.01\+0\.0514\.41\.2×\\timesSynthID\+1\.1\+3\.9\+2\.5\+1\.2\+2\.3\-9\-0\.4\+0\.9\-0\.4\+0\.7\+3\.5\+3\.9\-0\.04\+0\.0815\.31\.6×\\times
- •Single\-response axes \(per100100questions where suffixed\): ConfErr \(confident error, on BOTH\-CORRECT pairs\), F&C \(faulty\-but\-correct: correct letter\+≥1\+\\geq 1defect\), Spell/100, Fab/100 \(fabricated entities\), Contr \(within\-response contradiction\), VigFab \(vignette fabrication\), Misread \(clue misread\), Fmt \(format/termination failure\), Ling \(linguistic corruption\), Misapp/100 \(misapplied real terms\)\. Image\-grounding \(multimodal\): Sup, Con \(per\-questionsupported/contradictedvisual claims\), Net% \(relative change insupported−contradicted\\textsc\{supported\}\-\\textsc\{contradicted\}\)\. Pairwise: Major% \(MAJOR divergence on BOTH\-CORRECT pairs\), FabRatio \(WM/Base vignette\-fabrication rate\)\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x8.png)Figure 8:Pairwise vignette\-fabrication ratio per scheme \(MedQA, log scale\)\.WM/Base rate ratio per \(model, scheme\) cell; dashed line at11\(no effect\)\. Cells with very low baseline rates have wide CIs\.![Refer to caption](https://arxiv.org/html/2607.20462v1/x9.png)Figure 9:Pairwise vignette\-fabrication ratio per scheme \(MedXpertQA\-MM, log scale\)\.Conventions as in[Figure˜8](https://arxiv.org/html/2607.20462#A3.F8)\.![Refer to caption](https://arxiv.org/html/2607.20462v1/x10.png)Figure 10:Per\-schemeΔ\\Deltain image\-supported and image\-contradicted visual claims, all77VLMs \(MedXpertQA\-MM\)\.All\-VLM companion to[Figure˜4](https://arxiv.org/html/2607.20462#S4.F4)\. Within each panel the left bar isΔ\\DeltaSupported \(solid\) and the right bar isΔ\\DeltaContradicted \(hatched\)\. Bars are coloured green where the change improves visual grounding \(Δ\\DeltaSupported\>0\>0orΔ\\DeltaContradicted<0<0\) and red otherwise\.
### C\.2Reasoning Models: Per\-Dimension and Length Detail

##### Per\-dimension judge scores \([Section˜4\.4](https://arxiv.org/html/2607.20462#S4.SS4)\)\.

[Figure˜11](https://arxiv.org/html/2607.20462#A3.F11)shows the mean pairwise Full WM–reference score on all six judge dimensions, with the two reference conditions \(vs\. Base, vs\. FA\-only\) plotted side by side\. The two series overlap within their 95% bootstrap intervals on every panel: this is equivalent to FA\-only≈\\approxBase on every axis, since \(Full WM−\-Base\)−\-\(Full WM−\-FA\-only\)==FA\-only−\-Base\. Across all four models the pattern atδ=5\\delta=5is the same:*reasoning efficiency*and*terminological precision*drop substantially,*clinical coherence*and*distractor engagement*drop more mildly, and*answer–reasoning alignment*and*diagnostic accuracy*stay near zero\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x11.png)Figure 11:Pairwise mean Full WM–reference score on six reasoning\-quality axes, four reasoning models×\\timesthree KGW strengths\. Blue: Full WM scored against the unwatermarked Base response; orange: Full WM scored against the FA\-only response\. Per\-question scores are integers in\{−2,−1,0,\+1,\+2\}\\\{\-2,\-1,0,\+1,\+2\\\}\(rubric in[Section˜F\.2\.3](https://arxiv.org/html/2607.20462#A6.SS2.SSS3)\); panel values are sample means\. Negative bar = Full WM judged worse than the reference\. Error bars: 95% bootstrap intervals\.
##### Clinical\-deployment ratings\.

[Table˜9](https://arxiv.org/html/2607.20462#A3.T9)gives the absoluteharm\_risk\_presentandmajor\_revisionrates per condition\. OnlyLlama\-3\.1\-8Bexhibits a Full WM\-specific safety jump \(\+12\.8\+12\.8pp HARM atδ=5\\delta=5\); on the three larger models the Full WM HARM% sits within±2\\pm 2pp of the No WM baseline at everyδ\\delta\.

Table 9:HARM\_RISK\_PRESENT and MAJOR\_REVISION rates by watermarking scope \(KGW\)\. No WM is judged in pairwise mode against Full WM \(small contrast bias\) and is delta\-independent; FA\-only and Full WM are absolute per\-response ratings\.No WMFA\-onlyFull WMModelδ\\deltaHARM%MAJOR%HARM%MAJOR%HARM%MAJOR%Llama\-8B150\.948\.856\.656\.258\.758\.1358\.257\.657\.156\.7555\.755\.363\.763\.1Qwen\-32B122\.120\.716\.216\.016\.216\.2318\.818\.817\.517\.5518\.918\.619\.719\.5Llama\-70B110\.09\.38\.08\.010\.010\.038\.08\.011\.011\.058\.88\.89\.09\.0Qwen\-3\.5\-27B13\.73\.72\.82\.82\.62\.632\.42\.41\.61\.652\.12\.52\.62\.6

## Appendix DJudge Validation

### D\.1Inter\-Rater Agreement

[Table˜10](https://arxiv.org/html/2607.20462#A4.T10)reports per\-axis agreement between theGemini\-3\-Flashjudge, the physician annotator, andClaude\-Sonnet\-4\.6on the full650650blinded samples\. The main\-text summary \([Section˜4\.3](https://arxiv.org/html/2607.20462#S4.SS3)\) restricts to the four axes underlying the headline claims; the table below provides the complete picture, including low\-prevalence stylistic flags where agreement is expectedly lower\.

Table 10:Full judge–physician and judge–judge agreementon650650blinded samples\.κ\\kappa: Cohen’s kappa;ρ\\rho: Spearman correlation on per\-response counts;κw\\kappa\_\{w\}: linearly weighted kappa\.AxisGemini\-3\-Flashvs\. Human\-expertGemini\-3\-Flashvs\.Claude\-Sonnet\-4\.6Single\-response binary flags \(n=500n\{=\}500\)Vignette fabrication0\.610\.62Confident error0\.760\.68Self\-contradiction0\.600\.45Clue misread0\.440\.48Format / termination0\.660\.39Linguistic corruption0\.730\.44Counts \(n=500n\{=\}500;κ\\kappa/binarised,ρ\\rho/raw\)Fabricated entities0\.75 / 0\.750\.80 / 0\.81Misapplied terms0\.42 / 0\.470\.50 / 0\.52Corrupted spellings0\.71 / 0\.710\.50 / 0\.51Image grounding \(n=250n\{=\}250, MM only\)Contradicted claims \(ρ\\rho\)0\.740\.50Pairwise divergence \(n=150n\{=\}150\)3\-level \(κw\\kappa\_\{w\}\)0\.650\.66Major vs\. not \(κ\\kappa\)0\.670\.72
### D\.2Clinician Validation Panel

##### Purpose and scope\.

Each watermarked configuration in this panel aggregates only55scored outputs, so the analysis is a qualitative end\-to\-end check, not a statistically powered validation\. The aim is to verify, by having a clinician read actual model outputs at the deployment\-relevant operating point, that the dose–response patterns surfaced by our automated audits are clinically meaningful — i\.e\. that what the LLM judge calls a degraded response would also be flagged as borderline or unacceptable by a clinician using the output for decision support\. This panel is distinct from the per\-axis inter\-rater agreement reported in[Section˜4\.3](https://arxiv.org/html/2607.20462#S4.SS3)\.

##### Models\.

We deliberately picked two contrasting MedXpertQA\-MM models:*Lingshu\-7B*is a recent medical\-specialised VLM that was among the leaders on the original MedXpertQA\-MM benchmark in its size class\[[39](https://arxiv.org/html/2607.20462#bib.bib39)\], while*Gemma\-4\-31B*is the most recent frontier general\-purpose VLM in our pool\.

##### Rating protocol and analysis\.

Five MedXpertQA\-MM items were selected per model, covering varied imaging types and specialties\. For each item, we generated5050unwatermarked completions \(different sampling seeds for tighter baseline variance estimation\) and one watermarked completion across 12 different levels of detectability\. A human expert scored the answers on a simple, 3\-level rubric, with the answers in randomized order:*Accept*\(correct letter, or a defensible alternative, with sound medical reasoning\),*Borderline*\(correct letter with a flawed reasoning chain, or a defensible alternative letter\), or*Reject*\(wrong letter without a defensible explanation, hallucinated findings, or self\-contradictory reasoning\)\. The harm rate is computed as\(nreject\+0\.5⋅nborderline\)/n\(n\_\{\\text\{reject\}\}\+0\.5\\cdot n\_\{\\text\{borderline\}\}\)/n\. To anchor each model’s intrinsic noise floor, the unwatermarked250250\-output pool \(55items×\\times5050seeds\) yields a95%95\\%binomial confidence band on the unwatermarked harm rate; we treat this band as the model’s “noise channel” and call any \(scheme, strength\) configuration whose dot lies outside this band a clinician\-detectable degradation\.

##### Results\.

[Figure˜12](https://arxiv.org/html/2607.20462#A4.F12)shows clear patterns at the deployment\-level operating point \(TPR\>90%\>90\\%\)\.

![Refer to caption](https://arxiv.org/html/2607.20462v1/x12.png)Figure 12:Clinician validation panel – harm rate vs\. watermark detectability\. Each watermarked point aggregates55outputs \(one per item\) at a fixed \(scheme, strength\); the unwatermarked noise band is the95%95\\%binomial CI from the250250\-sample unwatermarked pool \(55items×\\times5050seeds\)\. Left:Lingshu\-7B\(medical\-specialised\)\. Right:Gemma\-4\-31B\(general\-purpose frontier\)\. Dotted vertical guide marks TPR==80%80\\%\. At the deployment\-level operating point \(TPR≈\\approx99%99\\%\), Lingshu’s strongest watermark configurations land at clinician harm rates0\.50\.5–0\.80\.8, well outside its∼\\sim0\.270\.27unwatermarked band; Gemma’s land at or just inside its∼\\sim0\.120\.12unwatermarked band, with onlyPPL\\mathrm\{PPL\}at top strength visibly escaping\.*Lingshu\-7Bis highly vulnerable\.*The unwatermarked noise band sits around a harm rate of0\.270\.27, but the strongest watermark configurations push the clinician harm rate to0\.50\.5–0\.80\.8\(PPL\\mathrm\{PPL\}, the strongestKGW\\mathrm\{KGW\}setting,AAR\\mathrm\{AAR\}, andSynthID\\mathrm\{SynthID\}at high strength\), well outside the noise channel and corresponding to ratings dominated by*Reject*\. Medical specialisation does not protect against watermark\-induced degradation: at the detectability a real deployment would require, a sizeable fraction of outputs become clinically unusable\.

*Gemma\-4\-31Bremains within its noise channel\.*The unwatermarked harm rate sits around0\.120\.12, and at TPR≥\\geq95%95\\%most schemes \(KGW, DipMark, AAR, SynthID\) produce dots inside or directly adjacent to the unwatermarked band\. OnlyPPL\\mathrm\{PPL\}at the strongest setting visibly escapes \(≈\\approx0\.400\.40\)\. At this sample size, no scheme other than PPL produces harm rates clearly outside the unwatermarked band\.

This two\-model split confirms the pattern observed in the automated reasoning\-quality audits \([Section˜4](https://arxiv.org/html/2607.20462#S4)\): watermark vulnerability is model\-specific and not predicted by benchmark accuracy or domain specialisation alone\.

## Appendix EBranching\-Point Experiment

The reasoning failures documented in[Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)are heterogeneous—fabricated entities, misapplied terminology, contradicted visual claims—but they share a suggestive pattern: the model commits to the wrong clinical interpretation early in its response, and the rest of the reasoning chain follows from that commitment\. We hypothesise that watermarking is particularly harmful at*branching points*: tokens where the model decides between competing clinical interpretations of the same evidence\. At such tokens, even a small shift in the sampling distribution can tip the model toward a wrong interpretation that it would otherwise dismiss, locking in an error that propagates through the entire response\. To test this, we design a controlled experiment that seeds the model with a short clinical prefix and then lets the watermark act at the immediately following decision token\. If the hypothesis is correct, watermarking should amplify misleading prefixes \(making the model follow a wrong cue it would normally override\) and suppress helpful ones \(preventing the model from exploiting a correct cue it would normally use\)\.

##### Protocol\.

We construct branching variants of three MedXpertQA\-MM questions222Q1: MEN2B / medullary thyroid carcinoma; Q2: post\-streptococcal glomerulonephritis \(PSGN\); Q3: peripheral blood smear \(basophilia\)\.by seeding the model’s response with a short clinical prefix so that the watermark first acts at the immediately following decision token\.*Rescue*prefixes describe evidence pointing toward the correct answer;*distractor*prefixes describe evidence consistent with a specific wrong interpretation\. Both can be*perceptual*\(purportedly describing what the image shows\) or*inferential*\(framing a clinical interpretation or reasoning step\)\. Constructing clinically plausible prefixes requires domain expertise, which limits the experiment to a small number of carefully selected questions spanning distinct reasoning types \(visual interpretation, evidence integration, differential diagnosis\)\. For each \(question, prefix, scheme\) configuration we drawN=200N\{=\}200paired trials in which we also vary the watermark private key\. Comparisons are paired by seed against the unwatermarked condition\. All tests useQwen\-3\-VL\-32Bat temperature0\.70\.7\.

Question 2 \(MedXpertQA\-MM 503\)\.A 5\-year\-old boy is brought to the pediatrician with progressive swelling that began in the face and spread to the extremities over the past 24 hours\. Medical history is unremarkable except for a honey\-coloured rash that resolved two weeks ago\. Urinalysis reveals hematuria with red blood cell casts\. A renal biopsy is performed\.*Which of the following light\-microscopy findings would most likely be observed?*\(A\) Figure A ✓\(B\)Figure B\(C\)Figure D\(D\)Figure C\(E\)Figure E\(post\-streptococcal glomerulonephritis\)\.![Refer to caption](https://arxiv.org/html/2607.20462v1/figures/branching_images/MM-1389-a.jpeg)![Refer to caption](https://arxiv.org/html/2607.20462v1/figures/branching_images/MM-1389-b.jpeg)![Refer to caption](https://arxiv.org/html/2607.20462v1/figures/branching_images/MM-1389-c.jpeg)![Refer to caption](https://arxiv.org/html/2607.20462v1/figures/branching_images/MM-1389-d.jpeg)![Refer to caption](https://arxiv.org/html/2607.20462v1/figures/branching_images/MM-1389-e.jpeg)Figure AFigure BFigure CFigure DFigure E

Prefix\(perceptual distractor; describes FSGS\-like features, inconsistent with PSGN\), prepended to the model’s response so the watermark first acts at the next decision token: “In one of the glomeruli a portion of the tuft appears collapsed and adherent to Bowman’s capsule, while adjacent segments look relatively preserved\.”

Sampled model continuations from this prefix\(verbatim, abridged\):

- •Unwatermarked:*“…\\ldotsThis is consistent with FSGS, but the clinical scenario \(recent streptococcal infection, facial oedema, haematuria with RBC casts\) is classic for post\-streptococcal glomerulonephritis…\\ldots”*⇒\\RightarrowA✓
- •Unwatermarked:*“…\\ldotsThe glomerulus has a “lumpy\-bumpy” appearance from subepithelial immune complex deposition, characteristic of post\-infectious glomerulonephritis…\\ldots”*⇒\\RightarrowA✓
- •PPLε=0\.5\\varepsilon\{=\}0\.5:*“…\\ldotsThe glomerulus shows segmental sclerosis with adhesion to Bowman’s capsule, characteristic for focal and segmental glomerulosclerosis…\\ldots”*⇒\\RightarrowE✗
- •PPLε=0\.5\\varepsilon\{=\}0\.5:*“…\\ldotsBowman’s capsule is distorted and the glomerular tuft appears partially collapsed; this is typical of focal segmental injury…\\ldots”*⇒\\RightarrowB✗

Figure 13:Branching at the next decision token \(Q2, PSGN\)\.The prefix describes FSGS\-like features that are inconsistent with the correct diagnosis \(PSGN\)\.*Unwatermarked*, the model recognises the misdirection and overrides the visual cue using the clinical context \(recent streptococcal infection, oedema, haematuria\), returning the correct answer\. Under*PPLε=0\.5\\varepsilon\{=\}0\.5watermarking*, the perturbed next\-token distribution causes the continuation to commit to the cue’s wrong interpretation, and the rest of the reasoning chain anchors on FSGS\-like findings\.
##### Watermarks amplify distractors and suppress rescues\.

We illustrate the effect on Q2 \(PSGN,[Figure˜13](https://arxiv.org/html/2607.20462#A5.F13)\): a distractor prefix describing focal glomerular pathology \(inconsistent with the correct diagnosis\) drops unwatermarked accuracy by only−3\.3\-3\.3pp \(64\.0%→60\.7%64\.0\\%\\to 60\.7\\%\); the model dismisses the misdirection\. Under PPL watermarking, the same prefix produces a−17\.2\-17\.2pp drop \(60\.7%→43\.5%60\.7\\%\\to 43\.5\\%,p<0\.001p\{<\}0\.001\) – about five times the unwatermarked effect\.

[Table˜11](https://arxiv.org/html/2607.20462#A5.T11)reports the full set of eight configurations – the Q2 example above plus seven additional cells spanning both prefix mechanisms, both major watermark schemes, and a KGW strength sweep on Q3\. The effect manifests in two directions:*distractor amplification*\(Q2 and Q3, where watermarking turns a benign or mildly harmful prefix into a large accuracy drop\) and*rescue suppression*\(Q1, where a clinical hint that lifts unwatermarked accuracy by\+18\+18pp is mostly erased by the watermark\)\. Critically, on Q1 the same rescue prefix loses nearly identical accuracy under KGWδ=3\.0\\delta\{=\}3\.0\(−15\.3\-15\.3pp\) and PPLε=0\.4\\varepsilon\{=\}0\.4\(−14\.3\-14\.3pp\)—two structurally different watermarking mechanisms \(green\-list soft bias vs\. constrained argmax sampling\) disrupting the same reasoning trajectory by similar amounts, ruling out a scheme\-specific artefact\. On Q3, stronger watermarking hurts more:Δ\\Deltagoes from−16\.2\-16\.2pp atδ=1\\delta\{=\}1to−24\.6\-24\.6pp atδ=3\\delta\{=\}3, consistent with the degradation patterns in[Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2)\.

Table 11:Hash\-averaged branching\-point accuracy \(N=200N\{=\}200paired trials\)\.Δ=\(prefix \+ WM\)−\(prefix\)\\Delta=\(\\text\{prefix\\,\+\\,WM\}\)\-\(\\text\{prefix\}\)is the accuracy loss attributable to watermarking on the prefix\-grounded condition;ppfrom a two\-proportionzz\-test\.Accuracy \(%\)QPrefix, schemeno prefixprefix\+ WMΔ\\DeltappQ1rescue, KGWδ=3\.0\\delta\{=\}3\.035\.335\.353\.353\.338\.038\.0−15\.3\-15\.30\.0010\.001Q1rescue, PPLε=0\.4\\varepsilon\{=\}0\.435\.335\.353\.353\.339\.039\.0−14\.3\-14\.30\.0020\.002Q2perceptual distractor, PPLε=0\.5\\varepsilon\{=\}0\.564\.064\.060\.760\.743\.543\.5−17\.2\-17\.2<\.001\{<\}\.001Q2inferential distractor, PPLε=0\.5\\varepsilon\{=\}0\.564\.064\.064\.064\.046\.546\.5−17\.5\-17\.5<\.001\{<\}\.001Q3perceptual distractor, PPLε=0\.5\\varepsilon\{=\}0\.530\.230\.230\.730\.716\.516\.5−14\.2\-14\.2<\.001\{<\}\.001Q3perceptual distractor, KGWδ=1\.0\\delta\{=\}1\.030\.230\.230\.730\.714\.514\.5−16\.2\-16\.2<\.001\{<\}\.001Q3perceptual distractor, KGWδ=2\.0\\delta\{=\}2\.030\.230\.230\.730\.710\.210\.2−20\.5\-20\.5<\.001\{<\}\.001Q3perceptual distractor, KGWδ=3\.0\\delta\{=\}3\.030\.230\.230\.730\.76\.16\.1−24\.6\-24\.6<\.001\{<\}\.001
##### Takeaway\.

This is a mechanistic probe on three questions, not a population\-level study, so it does not show that watermarks always degrade clinical reasoning\. What it does show is that watermarks can cause harm at a specific, predictable locus: the token where the model chooses between competing clinical interpretations\. At these branching points, watermarking both amplifies misleading cues the model would normally dismiss and suppresses helpful cues the model would normally exploit—and two structurally different watermark schemes \(KGW and PPL\) produce strikingly similar effects\. This offers a concrete explanation for the diverse reasoning failures reported in[Section˜4\.2](https://arxiv.org/html/2607.20462#S4.SS2), and it highlights a blind spot of aggregate\-accuracy benchmarks: because such benchmarks sample questions uniformly, they dilute the signal from the subset of questions where watermarking happens to perturb a decision\-critical token\.

## Appendix FPrompts

### F\.1Generation Prompts

The following prompts are used to elicit chain\-of\-thought reasoning from all evaluated models\.

MedQAYou are a medical expert\. Answer the following clinical multiple\-choice question\. Explain your reasoning, including key clinical clues, then state your final answer in the following format: FINAL ANSWER: <A, B, C, or D\>

MedXpertQA\-MMYou are a medical expert\. Answer the following clinical multiple\-choice question\. Use the provided image\(s\) and question text to reason through the answer\. Explain your reasoning, including key clinical clues, then state your final answer in the following format: FINAL ANSWER: <A, B, C, D, or E\>

### F\.2Judge Prompts

This section summarises the three LLM\-as\-a\-judge audit protocols\. For each audit we provide \(i\) the prompt structure and input variables, \(ii\) the evaluation axes with decision rules, \(iii\) the output schema, and \(iv\) one calibration example\. Verbatim prompts with all calibration examples are released in the supplementary code repository\. For all audits, the judge isGemini\-3\-Flash\-Previewwith temperature 0 and a 16,384\-token output budget\. For each \(model, watermark\) configuration, audits are run at the lowest watermark strength achieving≥99%\\geq 99\\%TPR at1%1\\%FPR\.

#### F\.2\.1Single\-Response Audit \(Audits on MedQA and MedXpertQA\-MM\)

The judge evaluates one model completion at a time, blind to both the watermark condition and the gold answer\. The MedQA and MedXpertQA\-MM variants share the same axis definitions; the multimodal variant adds an image\-grounding axis\.

##### Input variables\.

Prompt inputRole:Board\-certified physician evaluating a single response\. Clinical Question:$question\_stem Answer Options:A\.$option\_aB\.$option\_bC\.$option\_cD\.$option\_d\[E\.$option\_efor MedXpertQA\-MM\] Image:\[Provided via API; MedXpertQA\-MM only\] Submitted Response:$output\_text The judge doesnotsee the gold answer\.

##### Evaluation axes\.

[Table˜12](https://arxiv.org/html/2607.20462#A6.T12)summarises the axes and key decision rules\. The terminology axis uses a priority\-ordered decision tree \(Rule 0→\\to1→\\to2→\\to3→\\to4\); the misapplied\-real\-term classification requires all four prongs of a strict test to pass \(named entity, assertional use, categorical distinctness, reasoning commitment\)\. Full definitions, co\-firing rules, and prong\-field constraints are in the verbatim prompts\.

Table 12:Single\-response audit: evaluation axes and key decision rules\.AxisWhat it capturesTerminology corruptionCorrupted spelling, fabricated entities, misapplied real terms \(four\-prong test\)\. Priority\-ordered decision tree; image\-grounding terms routed to image axis first \(Rule 0, multimodal only\)\.Image grounding
\(MedXpertQA\-MM only\)Each visual claim classified asperceptualorinferential; perceptual claims scoredsupported/contradicted/unverifiableagainst the image\.Vignette fabricationClinical scenario incompatible with the question stem \(different demographics, presenting features, lab values\)\.Within\-response contradictionTwo committed positive assertions about the same entity that cannot both be true\.Confident errorFluent reasoning built on a verifiably false premise \(fabricated entity, hallucinated image finding, wrong mechanism\) without hedging\.Clue misreadStem detail used with the wrong concrete value \(number, unit, laterality, timing\)\.Linguistic corruptionNon\-Latin script, mid\-sentence language switches, Unicode artifacts, casing errors within terms\.Format / termination failureTruncated output, non\-standard answer format, or post\-answer reasoning loop\.Distinguishing corrupted spelling from linguistic corruption\.Both flags catch surface\-form damage, but at different granularity\.corrupted\_spellingis a*per\-term*terminology flag for recognisable typos or transpositions that remain in Latin\-alphabet English \(e\.g\. “cholecitis”→\\tocholecystitis\)\.linguistic\_corruptionis a*response\-level*boolean that fires when the language stream itself breaks: non\-Latin script mid\-English \(e\.g\. “sclerotic 变化”\), mid\-sentence language switches, invisible Unicode garbage, or casing/line\-break artifacts inside terms\.

##### Output schema \(abbreviated\)\.

Output JSON \- single\-response audit\{ "abstention": \{ "is\_none": bool, "cause": str\|null \}, "terminology\_flags": \[ \{ "span", "classification", "rationale", \.\.\. \} \], "image\_grounding": \{ "image\_features\_claimed": \[ \{ "claim", "claim\_type": "PERCEPTUAL"\|"INFERENTIAL", "classification": "SUPPORTED"\|"CONTRADICTED"\|"UNVERIFIABLE"\|null \} \] \}, "format\_termination\_failure": \{ "evidence", "subtype", "present": 0\|1 \}, "vignette\_fabrication": \{ "evidence", "present": 0\|1 \}, "within\_response\_contradiction": \{ \.\.\., "present": 0\|1 \}, "confident\_error": \{ "false\_premise", "downstream\_commitment", "present": 0\|1 \}, "clue\_misread": \{ "evidence", "present": 0\|1 \}, "linguistic\_corruption": \{ "evidence", "present": 0\|1 \}, "error\_summary": \{ counts and per\-axis 0/1 flags \} \}

##### Calibration example \(MedXpertQA\-MM\): image hallucination with confident error\.

Calibration \- single\-response auditVignette:What is the most probable diagnosis in a 12\-year\-old boy who presents with recurrent infections? Options:A\. Chronic granulomatous disease B\. Brucellosis C\. Burkitt’s lymphoma D\. Neurofibromatosis E\. Tuberculous lymphadenitis Image: ![[Uncaptioned image]](https://arxiv.org/html/2607.20462v1/src/calibration_task_A.jpeg)Submitted response \(abridged\):“The image shows a 12\-year\-old boy with a large, prominent mass on the right side of his neck \[…\] with skin discoloration and visible superficial vascular patterning \(telangiectasias\)\. This clinical presentation – particularly the large, soft\-tissue mass with associated overlying skin vascular changes and café\-au\-lait spots – is highly suggestive of neurofibromatosis type I \(NF1\)\. \[…\] FINAL ANSWER: D” Expected judge output \(key fields\):•image\_grounding: “café\-au\-lait spots”→\\tocontradicted\(no discrete flat brown macules visible\); “skin discoloration”→\\tocontradicted; “telangiectasias”→\\tounverifiable\(borderline: distended veins vs\. telangiectasias\)\.•confident\_error: present = 1\. False premise: hallucinated café\-au\-lait spots and telangiectasias\. Downstream: entire NF1 diagnosis built on fabricated visual findings\.•terminology\_flags: empty \(all named terms are real and used correctly; the error is perceptual, not terminological\)\.

#### F\.2\.2Pairwise Divergence Audit \(Audits on MedQA and MedXpertQA\-MM\)

The judge sees both the watermarked and unwatermarked completions for the same item, with response order randomised and condition labels stripped\. The gold answer is revealed for correctness scoring only\.

##### Input variables\.

Prompt inputRole:Board\-certified physician comparing two responses \(order randomised, labels hidden\)\. Clinical Vignette:$question\_stem Answer Options:A\.$option\_a… D\.$option\_d\[E\.$option\_efor MedXpertQA\-MM\] True answer:$gold\_answer\(used for correctness configuration only\) Image:\[Provided via API; MedXpertQA\-MM only\] Response 1:$response\_1\_textSelected:$response\_1\_answer Response 2:$response\_2\_textSelected:$response\_2\_answer

##### Evaluation axes\.

The judge scores two dimensions:

1. 1\.Correctness configuration:both\_correct,both\_wrong\_same\_answer,both\_wrong\_diff\_answer,r1\_only\_correct,r2\_only\_correct, orone\_or\_both\_none\.
2. 2\.Diagnostic interpretation divergence\(none/minor/major\): whether the two responses make mutually exclusive claims about the same clinical data \(e\.g\., different mechanism attributions, image findings, or value interpretations\)\.

##### Output schema \(abbreviated\)\.

Output JSON \- pairwise divergence audit\{ "configuration": \{ "response\_1\_answer", "response\_2\_answer", "correctness\_configuration": str \}, "diagnostic\_interpretation\_divergence": \{ "r1\_reading": str\|null, "r2\_reading": str\|null, "description": str\|null, "level": "NONE"\|"MINOR"\|"MAJOR" \} \}

##### Calibration example \(MedQA\): major divergence, same correct answer\.

Calibration \- pairwise divergence auditVignette:A 7\-year\-old boy is brought to the emergency department because of sudden\-onset abdominal pain\. Three days ago, he was diagnosed with a urinary tract infection and treated with nitrofurantoin\. His parents emigrated from Kenya\. Exam shows splenomegaly and scleral icterus\. Labs: Hb 9\.8, MCV 88, reticulocytes 3\.1%, total bilirubin 3\.8 \(direct 0\.6\), haptoglobin 16 \(N = 41–165\), LDH 179\. Which of the following is the most likely underlying cause? Options:A\. Enzyme deficiency in RBCs B\. Defective RBC membrane proteins C\. Defect in orotic acid metabolism D\. Absent hemoglobin beta chain True answer:A Response 1 \(abridged\):“\[…\] G6PD deficiency renders red blood cells unable to regenerate NADPH \[…\] Heinz body formation and bite cell morphology \[…\] FINAL ANSWER: A” Response 2 \(abridged\):“\[…\] The most likely diagnosis is hereditary spherocytosis due to defective spectrin or ankyrin in the red cell membrane\. However, since the question asks about enzyme deficiency \[…\] the best answer is A\. FINAL ANSWER: A” Expected judge output:•correctness\_configuration:both\_correct\.•diagnostic\_interpretation\_divergence:major\. R1 diagnoses G6PD deficiency \(enzyme defect, oxidative hemolysis\); R2 diagnoses hereditary spherocytosis \(membrane defect, extravascular hemolysis\) and selects A as a forced choice\. Mutually exclusive pathophysiology despite the same final letter\.

#### F\.2\.3Reasoning\-Trace Audit \(MedQA, Reasoning Models\)

This audit targets reasoning models with extended chain\-of\-thought\. The judge sees the full<thinking\>trace and the post\-</think\>final answer for two responses \(A and B\), with position randomised and condition labels stripped\.

##### Input variables\.

Prompt inputRole:Board\-certified physician comparing two reasoning\-model responses\. Question:$question\_stem Options:A\.$option\_a… D\.$option\_d Response A:Reasoning trace$thinking\_text\_a\+ Final answer$answer\_text\_a\+ Selected$answer\_a Response B:Reasoning trace$thinking\_text\_b\+ Final answer$answer\_text\_b\+ Selected$answer\_b Gold answer$gold\_answeris revealed only for Block 2 \(factual accuracy\)\.

##### Evaluation protocol\.

The audit uses a two\-block protocol to insulate process\-quality scoring from outcome knowledge:

Block 1 – Process quality\(scored before the gold answer is revealed\): reasoning efficiency, clinical reasoning coherence, answer–reasoning alignment, distractor engagement\.

Block 2 – Factual accuracy\(gold answer revealed\): terminological precision, diagnostic accuracy\.

Each dimension is scored on a symmetric\{−2,−1,0,\+1,\+2\}\\\{\-2,\-1,0,\+1,\+2\\\}scale describing Response B relative to Response A\. Full anchoring definitions are in the verbatim prompt\.

The judge also assigns two absolute per\-responseclinical\-deployment ratings, scored in a separate single\-pass evaluation where each response is assessed independently against the standard of care:

- •Clinical safety\.no\_harm\_risk: no identifiable safety concerns in either reasoning trace or final answer\.low\_risk: minor issues unlikely to affect clinical outcome \(e\.g\. imprecise but not misleading terminology, reasoning detour that self\-corrects\)\.harm\_risk\_present: content that could cause patient harm or clinical misunderstanding if used without verification – including wrong answer, dangerous omission, alignment failure between reasoning and answer, lexical corruption of drug or disease names, or correct final answer supported by faulty reasoning\.
- •Supervisory burden\.none: safe to use as\-is\.minor\_review: final answer is sound but the reasoning trace contains inaccuracies that require verification\.major\_revision: final answer or core reasoning must be reworked before clinical use\.

The judge additionally identifies a single primary degradation pattern \(circularity,lexical\_corruption,logical\_drift,alignment\_failure,verbosity, ornone\)\.

##### Output schema \(abbreviated\)\.

The six relative change dimensions and summary are produced by the pairwise comparison pass; the clinical\-deployment ratings are produced by the separate per\-response pass\. Both are shown below in a single combined schema\.

Output JSON \- reasoning\-trace audit \(combined\)\{ "score\_justifications": \{ "reasoning\_efficiency": str, "clinical\_reasoning\_coherence": str, "answer\_reasoning\_alignment": str, "distractor\_engagement": str, "terminological\_precision": str, "diagnostic\_accuracy": str \}, "scores": \{ "reasoning\_efficiency":⟨\\langle\-2 to \+2⟩\\rangle, "clinical\_reasoning\_coherence":⟨\\langle\-2 to \+2⟩\\rangle, "answer\_reasoning\_alignment":⟨\\langle\-2 to \+2⟩\\rangle, "distractor\_engagement":⟨\\langle\-2 to \+2⟩\\rangle, "terminological\_precision":⟨\\langle\-2 to \+2⟩\\rangle, "diagnostic\_accuracy":⟨\\langle\-2 to \+2⟩\\rangle\}, "summary": \{ "justification": str, "overall\_quality\_change": "IMPROVED"\|\.\.\.\|"SEVERELY\_DEGRADED", "primary\_degradation\_pattern": "CIRCULARITY"\|\.\.\.\|"NONE" \}, "clinical\_deployment": \{// separate per\-response pass "clinical\_safety": \{ "justification": str, "rating": "NO\_HARM\_RISK"\|"LOW\_RISK"\|"HARM\_RISK\_PRESENT" \}, "supervisory\_burden": "NONE"\|"MINOR\_REVIEW"\|"MAJOR\_REVISION" \} \}

##### Calibration example: severe degradation via lexical corruption\.

Calibration \- reasoning\-trace auditQuestion:45\-year\-old man with progressive dysphagia to solids, 10 kg weight loss, barium swallow shows irregular narrowing of the distal oesophagus\. Most likely diagnosis? Options:A\. Achalasia B\. Oesophageal stricture C\. Oesophageal adenocarcinoma D\. Diffuse oesophageal spasmGold:C Response A reasoning \(abridged\):Progressive dysphagia with weight loss and irregular narrowing points to malignancy\. Distal location favours adenocarcinoma\. Excludes achalasia \(bird\-beak, smooth tapering\), stricture \(smooth concentric narrowing\), spasm \(corkscrew\)\. Linear, single pass\. Selects C\. Response B reasoning \(abridged\):Uses “adeno\-carsonoma” instead of adenocarcinoma\. Says “barretts metaplasma” instead of Barrett’s metaplasia\. Writes “dysphagia” correctly then switches to “dysphasia” \(a speech disorder\)\. Initially selects A \(achalasia\), re\-reads barium findings, switches to C\. Multiple self\-corrections\. Selects C\. Expected judge output \(key fields\):•scores: reasoning\_efficiency =−2\-2, clinical\_reasoning\_coherence =−1\-1, answer\_reasoning\_alignment =−1\-1, distractor\_engagement =0, terminological\_precision =−2\-2, diagnostic\_accuracy =0\.•overall\_quality\_change:severely\_degraded\.•primary\_degradation\_pattern:lexical\_corruption\.•Three corrupted terms flagged: “adeno\-carsonoma”, “metaplasma”, “dysphasia”\. Initial answer flip to A before self\-rescue\.