Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs

# Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs
Source: [https://arxiv.org/html/2605.30501](https://arxiv.org/html/2605.30501)
###### Abstract

Watermarking embeds statistical signatures in AI\-generated text for detection and attribution\. We reveal a fundamental vulnerability: when users access multiple models \(today’s reality\), watermarks trivially fail\. Watermarks perturb output distributions away from the original, and in competitive markets, these perturbations are typically independent across providers\. We theoretically prove that averaging output probability distributions recovers the unwatermarked distribution with up to a second\-order error term\. Empirically, simply averaging 3\-5 models cancels out these perturbations\. We introduce WASH \(Watermark Attenuation via Statistical Hybridisation\), which solves practical challenges in ensemble generation: vocabulary misalignment and tokenisation differences across heterogeneous models\. Experiments across six watermarking schemes and three LLMs show that averaging across 3 models suppresses detection z\-scores from 5\-300 tobelow 2\(below the detection threshold of 4\) and reduces TPR@5%FPR tobelow 50%, while improving quality by27\.5%and running6×\\timesfaster than the best baseline on the long sequence generation\. Our results suggest that robust AI\-text detection via watermarking requires either accepting this fundamental vulnerability or unprecedented coordination among model providers\.

Machine Learning, ICML

## 1Introduction

The rapid deployment of large language models \(LLMs\) across critical applications, from educational assessment to content creation, has made reliable attribution mechanisms an urgent necessity\(Wanget al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib27); Khasentinoet al\.,[2025](https://arxiv.org/html/2605.30501#bib.bib31); Wanget al\.,[2026](https://arxiv.org/html/2605.30501#bib.bib32)\)\. Can we determine whether a given text was generated by an AI system? In an era where synthetic text has become increasingly indistinguishable from human writing, these questions have profound implications for academic integrity, content authenticity, and intellectual property protection\(Weiet al\.,[2023](https://arxiv.org/html/2605.30501#bib.bib29); Yaoet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib30)\)\. Watermarking has emerged as the technical solution to this attribution problem\(Kirchenbaueret al\.,[2023a](https://arxiv.org/html/2605.30501#bib.bib2); Liuet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib28)\)\. By embedding statistical signatures during text generation, unbiased watermarking promises to make AI\-generated content detectable while minimising quality degradation and avoiding architectural changes\(Huet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib3); Maoet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib4)\)\.

However, current watermarking research relies on a critical simplifying assumption: adversaries have access to only a single watermarked model\. In reality, users today can easily and freely access 10\+ frontier LLMs through unified platforms \(e\.g\., Hugging Face\), such as GPT, LLaMA, Qwen, Mistral, and dozens of other capable models\. The modern LLM landscape is not a monopoly but a hyper\-competitive marketplace with multiple providers, and this competitive structure is the Achilles’ heel of watermarking\.

![Refer to caption](https://arxiv.org/html/2605.30501v1/figures/visual_token_ensemble.png)Figure 1:Effect of Linear Ensembles\.The ensemble averagep¯N\\bar\{p\}\_\{N\}\(Right\)neutralises the independent watermark perturbationspip\_\{i\}\(Middle\), effectively recovering the original consensusp∗p^\{\*\}\(Left\), which is calculated by averaging the unwatermarked models\.Our key insight is deceptively simple: watermarking works by perturbing a model’s output distribution, and these perturbations are independent across providers due to different secret keys and architectural design\. By querying multiple models in parallel and averaging their output distributions, these independent perturbations cancel out, recovering the original unwatermarked distribution \(as illustrated in Figure[1](https://arxiv.org/html/2605.30501#S1.F1)\)\. We formalise this by proving that for any unbiased watermarking scheme with independent per\-model perturbations, linear ensembling recovers the consensus distribution up to a second\-order error with convergence rateO​\(1/N\)O\(1/\\sqrt\{N\}\)\. This establishes a fundamental limit rooted in market structure: competitive providers must use secret keys for provenance verification \(guaranteeing independence\) and maintain quality to retain users \(bounding perturbation magnitude\)\. Under these constraints, watermark signals are mathematically guaranteed to vanish under averaging\.

While theoretical result guarantees asymptotic removal, practical deployment faces critical obstacles in efficiency and coherence\. Frameworks that seek to approximate watermark parameters via “random selection probing”\(Chenet al\.,[2025a](https://arxiv.org/html/2605.30501#bib.bib12)\)require extensive iterative querying, causing prohibitive inference latency\. Meanwhile, attacks that concatenate tokens from heterogeneous models require predicting additional tokens at every time step and re\-encoding the entire context at every switch\(Huanget al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib8)\), incurring significant computational inefficiency\. To bridge these gaps, we introduceWASH\(Watermark Attenuation via Statistical Hybridisation\)\.WASHemploys fluency\-aware routing to overcome vocabulary mismatches, enabling effective probability aggregation across distinct tokenisers\. Furthermore, by leveraging parallel inference with response caching,WASHeliminates the need for iterative probing or context re\-encoding, achieving*constant\-time per\-token complexity*regardless of the ensemble sizeNN\. Extensive experiments demonstrate thatWASHimproves generation quality by27\.5%27\.5\\%while running6×6\\timesfaster than state\-of\-the\-art removal baselines\.

Our analysis reveals that watermarking faces a fundamental choice: True robustness that distinguishes AI from human text regardless of model mixing requirescoordinated watermarking, namely some common signal shared across all models\. This paper rigorously characterises this fundamental limitation through the following contributions:

1. 1\.We theoretically prove that linear ensembling asymptotically recovers the original unwatermarked distribution\. We demonstrate the convergence rate isO​\(1/N\)O\(1/\\sqrt\{N\}\)withNNindependent models, revealing fundamental limits of unbiased watermarking in multi\-provider settings\.
2. 2\.We introduceWASH, a novel algorithm that overcomes the vocabulary mismatch problem in heterogeneous model ensembles\. By employing fluency\-aware routing and context re\-synchronisation,WASHpreserves semantic integrity while neutralising watermark signals\.
3. 3\.We conduct a systematic evaluation across six representative watermarking schemes and three LLMs under two complementary detection settings\. Experiments demonstrate near\-complete detection failure: \(a\) for generation\-time detection,WASHsuppresses z\-scores from 5\-300 \(strongly detectable\) to<2<2\(near random choice\) with just 3 models; \(b\) for final\-text detection,WASHlowers the TPR@5%FPR on native sequence detectors to below 50%, rendering the watermarks statistically undetectable\.

## 2Methodology

Watermarking exhibits a structural statistical vulnerability when outputs from multiple independently watermarked models are aggregated\. Formally, we conceptualise watermarking as a stochastic perturbation applied to a shared underlying distributionp∗p^\{\*\}\. While these perturbations are necessary for detection, they act as uncoordinated noise across different providers\. Consequently, when outputs from multiple models are combined, the watermark signals interfere destructively, allowing the underlying consensus distribution to be asymptotically recovered\.

We now formalise this perspective by characterising the relationship between the consensus distribution, the perturbed watermarked distributions, and their aggregation\. Let𝒱\\mathcal\{V\}be the vocabulary and𝒳\\mathcal\{X\}be the space of possible contexts \(i\.e\., sequences of tokens\)\. We denote the probability distributions over the vocabulary asΔ​\(𝒱\)=\{𝐩∈ℝ\|𝒱\|\|pv≥0,∑v∈𝒱pv=1\}\\Delta\(\\mathcal\{V\}\)=\\left\\\{\\mathbf\{p\}\\in\\mathbb\{R\}^\{\|\\mathcal\{V\}\|\}\\;\\middle\|\\;p\_\{v\}\\geq 0,\\sum\_\{v\\in\\mathcal\{V\}\}p\_\{v\}=1\\right\\\}\.

### 2\.1Problem Formulation

Conceptually, the probability distribution of any modeliideviates from the ideal semantic distributionpG​Tp\_\{GT\}\(align with human expert\) due to shared systematic errors and provider\-specific variations\. We model this as:

pi​\(v\|x\)∝pG​T​\(v\|x\)⋅exp⁡\(δs​y​s​\(v,x\)⏟Shared Model Bias\+δi​\(v,x\)⏟Watermark\)\.p\_\{i\}\(v\|x\)\\propto p\_\{GT\}\(v\|x\)\\cdot\\exp\\left\(\\underbrace\{\\delta\_\{sys\}\(v,x\)\}\_\{\\text\{Shared Model Bias\}\}\+\\underbrace\{\\delta\_\{i\}\(v,x\)\}\_\{\\text\{Watermark\}\}\\right\)\.Here,δs​y​s\\delta\_\{sys\}represents common biases inherent to current LLM architectures, whileδi\\delta\_\{i\}encapsulates the provider\-specific signal, primarily the watermarking signal, but also including model\-specific characteristics\.

###### Definition 2\.1\(Consensus Distribution and Watermarked Perturbation\)\.

We define the consensus distributionp∗\(⋅\|x\)∈Δ\(𝒱\)p^\{\*\}\(\\cdot\|x\)\\in\\Delta\(\\mathcal\{V\}\)as the effective baseline of current models, absorbing the shared systematic bias:p∗​\(v\|x\)∝pG​T​\(v\|x\)⋅exp⁡\(δs​y​s​\(v,x\)\)p^\{\*\}\(v\|x\)\\propto p\_\{GT\}\(v\|x\)\\cdot\\exp\(\\delta\_\{sys\}\(v,x\)\)\. Consequently, the output distribution of a specific watermarked modeliidefines a perturbed distributionpi\(⋅\|x\)p\_\{i\}\(\\cdot\|x\):

pi​\(v\|x\)=p∗​\(v\|x\)⋅exp⁡\(δi​\(v,x\)\)Zi​\(x\),p\_\{i\}\(v\|x\)=\\frac\{p^\{\*\}\(v\|x\)\\cdot\\exp\(\\delta\_\{i\}\(v,x\)\)\}\{Z\_\{i\}\(x\)\},\(1\)whereδi:𝒱×𝒳→ℝ\\delta\_\{i\}:\\mathcal\{V\}\\times\\mathcal\{X\}\\to\\mathbb\{R\}is the provider\-specific perturbation function \(the watermark signal\), andZi​\(x\)Z\_\{i\}\(x\)is the normalisation term\.

The key insight enabling watermark removal is that perturbations across independent providers are statistically unbiased\. We formalise this assumption below\.

###### Assumption 2\.2\(Unbiased Perturbations\)\.

Consider a discrete set of providers indexed byii, where each provider is associated with a random perturbation vectorδi​\(⋅,x\)\\delta\_\{i\}\(\\cdot,x\)that modulates the output distribution\. We assume the following properties hold for\{δi\}i=1N\\\{\\delta\_\{i\}\\\}\_\{i=1\}^\{N\}:

\(a\) Bounded Magnitude:The perturbation magnitude is uniformly bounded by a constantξ≤1\\xi\\leq 1\. Specifically,‖δi​\(⋅,x\)‖∞≤ξ\\\|\\delta\_\{i\}\(\\cdot,x\)\\\|\_\{\\infty\}\\leq\\xifor allx∈𝒳x\\in\\mathcal\{X\}and providersii\.

\(b\) Independence:The perturbation vectorsδi\\delta\_\{i\}are independent across providers\.

\(c\) Zero Mean:The watermarking signals are centered around the consensus distribution in log\-space, such that𝔼​\[δi​\(v,x\)\]=0\\mathbb\{E\}\[\\delta\_\{i\}\(v,x\)\]=0for allv∈𝒱v\\in\\mathcal\{V\},x∈𝒳x\\in\\mathcal\{X\}, and providersii\.

\(d\) Bounded Expected Variance:The expected value of the weighted variation of the perturbations across the vocabulary \(weighted by the consensus probability\) is bounded by a constantη2\\eta^\{2\}\. That is, for allx∈𝒳x\\in\\mathcal\{X\}:

𝔼​\[Varu∼p∗​\(δi​\(u,x\)\)\]≤η2,\\mathbb\{E\}\\bigl\[\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u,x\)\)\\bigr\]\\leq\\eta^\{2\},\(2\)where

Varu∼p∗​\(δi​\(u,x\)\):=∑up∗​\(u\)​\(δi​\(u\)−∑vp∗​\(v\)​δi​\(v\)\)2\.\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u,x\)\):=\\sum\_\{u\}p^\{\*\}\(u\)\\bigl\(\\delta\_\{i\}\(u\)\-\\sum\_\{v\}p^\{\*\}\(v\)\\delta\_\{i\}\(v\)\\bigr\)^\{2\}\.\(3\)

This assumption is natural in the multi\-provider setting: \(a\) Market forces impose a strict upper bound on perturbation magnitude \(ξ\\xi\), as any watermark strong enough to violate the linear approximation \(largeδ\\delta\) would result in perceptible quality degradation\. \(b\) Each provider employs watermark configurations unknown to each other, making the perturbations behave as independent random variables\. This can be relaxed to a grouped setting where providers share common latent factors: under between\-group independence and within\-group conditional independence, the same convergence guarantee holds; see Appendix[B](https://arxiv.org/html/2605.30501#A2)\. \(c\) Providers independently optimise for quality, with no reason to systematically favor or disfavor specific tokens beyond whatp∗p^\{\*\}suggests\. We further stress\-test this zero\-mean assumption under deliberately biased watermark perturbations in Appendix[C](https://arxiv.org/html/2605.30501#A3)\. \(d\) Finally, providers maximise utility by prioritising stability on high\-probability tokens; thep∗p^\{\*\}\-weighting ensures that significant variance is restricted to rare tokens where it least impacts the overall text quality\.

### 2\.2Watermark Removal via Linear Ensembles

GivenNNindependent models, we propose to neutralise the watermarking signals and recover the consensus distribution via a uniform mixture, a method we termWASH\. Since the perturbationsδi\\delta\_\{i\}are uncorrelated across providers, they act as noise that can be averaged out\. Given a fixed contentx∈𝒳x\\in\\mathcal\{X\}, the aggregated probability distribution is defined as

p¯N\(⋅\|x\):=1N∑i=1Npi\(⋅\|x\)\.\\bar\{p\}\_\{N\}\(\\cdot\|x\):=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}p\_\{i\}\(\\cdot\|x\)\.\(4\)We implement this removal process as an autoregressive ensemble\. The generation process can be formalised as a recursive sequence\. Letx<t=\(x1,…,xt−1\)x\_\{<t\}=\(x\_\{1\},\\dots,x\_\{t\-1\}\)denote the context at steptt\. The next tokenxtx\_\{t\}is sampled from the aggregated distribution:

xt∼p¯N\(⋅\|x<t\)\.x\_\{t\}\\sim\\bar\{p\}\_\{N\}\(\\cdot\|x\_\{<t\}\)\.The process repeats until a termination token is generated\. Having formalised the generation process, we now provide theoretical guarantees for its effectiveness\.

###### Theorem 2\.3\(Convergence to Consensus Distribution\)\.

Under Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2), for any fixed contextxx, letp¯N\(⋅\|x\)=1N∑i=1Npi\(⋅\|x\)\\bar\{p\}\_\{N\}\(\\cdot\|x\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}p\_\{i\}\(\\cdot\|x\)be the aggregated distribution\. For anyδ\>0\\delta\>0, with probability at least1−δ1\-\\delta, theℓ∞\\ell\_\{\\infty\}distance between the aggregated distribution and the consensus distributionp∗\(⋅\|x\)p^\{\*\}\(\\cdot\|x\)satisfies:

∥p¯N\(⋅\|x\)−p∗\(⋅\|x\)∥∞≲log⁡\(\|𝒱\|/δ\)N\+η2,\\bigl\\\|\\bar\{p\}\_\{N\}\(\\cdot\|x\)\-p^\{\*\}\(\\cdot\|x\)\\bigr\\\|\_\{\\mathrm\{\\infty\}\}\\lesssim\\sqrt\{\\frac\{\\log\(\|\\mathcal\{V\}\|/\\delta\)\}\{N\}\}\+\\eta^\{2\},\(5\)where\|𝒱\|\|\\mathcal\{V\}\|denotes the vocabulary size, andη2\\eta^\{2\}is the upper bound on the expected weighted variance of the perturbation as in \([2](https://arxiv.org/html/2605.30501#S2.E2)\)\.

![Refer to caption](https://arxiv.org/html/2605.30501v1/x1.png)Figure 2:WASH Framework Overview\.The method ensemblesNNindependent models to neutralise watermarks via probability averaging\. To resolve vocabulary mismatches,Fluency\-Aware Routingcommits to the specialist models \(K∗K^\{\*\}\) whenever a token falls outside the shared vocabulary intersection \(𝒱∩\\mathcal\{V\}\_\{\\cap\}\), employingre\-tokenisationto synchronise context and ensure semantic alignment across heterogeneous tokenisers\.###### Proof Sketch\.

The proof relies on decomposing the approximation error into astochastic deviation termand asystematic bias term\. By the triangle inequality, the distance between the aggregated distributionp¯N\\bar\{p\}\_\{N\}and the consensus distributionp∗p^\{\*\}can be bounded as:

‖p¯N−p∗‖∞≤‖p¯N−𝔼​\[p¯N\]‖∞⏟Stochastic Deviation\+‖𝔼​\[p¯N\]−p∗‖∞⏟Systematic Bias\.\\\|\\bar\{p\}\_\{N\}\-p^\{\*\}\\\|\_\{\\infty\}\\leq\\underbrace\{\\\|\\bar\{p\}\_\{N\}\-\\mathbb\{E\}\[\\bar\{p\}\_\{N\}\]\\\|\_\{\\infty\}\}\_\{\\text\{Stochastic Deviation\}\}\+\\underbrace\{\\\|\\mathbb\{E\}\[\\bar\{p\}\_\{N\}\]\-p^\{\*\}\\\|\_\{\\infty\}\}\_\{\\text\{Systematic Bias\}\}\.
Bounding the Stochastic Deviation\.Since the providers are independent \(Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(b\)\), the aggregated probabilityp¯N​\(v\)\\bar\{p\}\_\{N\}\(v\)is the average of independent bounded random variables\. A direct application of Hoeffding’s inequality shows thatp¯N\\bar\{p\}\_\{N\}concentrates around its expectation𝔼​\[pi\]\\mathbb\{E\}\[p\_\{i\}\]at a rate ofO​\(1/N\)O\(1/\\sqrt\{N\}\)\.

Bounding the Systematic Bias\.The term‖𝔼​\[pi\]−p∗‖∞\\\|\\mathbb\{E\}\[p\_\{i\}\]\-p^\{\*\}\\\|\_\{\\infty\}arises from the nonlinearity of the softmax function\. Although the perturbationsδi\\delta\_\{i\}are zero\-mean \(Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(c\)\), the expected output probability is biased \(i\.e\.,𝔼​\[pi\]≠p∗\\mathbb\{E\}\[p\_\{i\}\]\\neq p^\{\*\}\) due to the convexity of the exponential function\. We utilise the shift\-invariance of the softmax to centre the perturbations and apply a second\-order Taylor expansion topi​\(v\)∝p∗​\(v\)​exp⁡\(δi​\(v\)\)p\_\{i\}\(v\)\\propto p^\{\*\}\(v\)\\exp\(\\delta\_\{i\}\(v\)\)\. The first\-order \(linear\) terms vanish due to the zero\-mean assumption\. The remaining error is dominated by the second\-order terms, which are controlled by the variance boundη2\\eta^\{2\}defined in Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(d\)\. We provide the detailed proof in Appendix[A](https://arxiv.org/html/2605.30501#A1)\. ∎

### 2\.3Preserving Semantic Integrity in Heterogeneous Ensembles

While linear ensemble effectively neutralises watermark perturbations in the limit ofNN, strictly enforcing this over heterogeneous topologies introduces a vocabulary mismatch problem\(Yuet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib18); Chenet al\.,[2025b](https://arxiv.org/html/2605.30501#bib.bib17)\)\. Let𝒱i\\mathcal\{V\}\_\{i\}denote the vocabulary of modelii\. A standard baseline restricts sampling to the vocabulary intersection𝒱∩=⋂i=1N𝒱i\\mathcal\{V\}\_\{\\cap\}=\\bigcap\_\{i=1\}^\{N\}\\mathcal\{V\}\_\{i\}, thereby ensuring distributional consensus across all models\. However, this conservative approach inducesexpressivity bottleneck\. Restricting generation to the vocabulary intersection𝒱∩\\mathcal\{V\}\_\{\\cap\}often excludes semantic ground truth tokens, such as specific entities or technical terms that are absent in at least one model’s vocabulary \(i\.e\.,𝒱Δ=\(⋃𝒱i\)∖𝒱∩\\mathcal\{V\}\_\{\\Delta\}=\(\\bigcup\\mathcal\{V\}\_\{i\}\)\\setminus\\mathcal\{V\}\_\{\\cap\}\)\.

Conversely, projecting onto the vocabulary union𝒱∪=⋃i=1N𝒱i\\mathcal\{V\}\_\{\\cup\}=\\bigcup\_\{i=1\}^\{N\}\\mathcal\{V\}\_\{i\}introducesgranularity mismatch: as illustrated in Figure[2](https://arxiv.org/html/2605.30501#S2.F2), heterogeneous tokenisers may represent the same semantic unit at different granularities\. For instance, one model represents “Gracious” as “\[Gr\], \[acious\]” while another decomposes it into “\[Gra\], \[cious\]”\. Aggregating over the union forces the imputation of undefined probabilities for different disjoint tokens \(e\.g\., the token “\[Gra\]” does not exist in the first model’s vocabulary\), thereby diluting the semantic fidelity\. To reconcile vocabulary heterogeneity with watermark removal, we propose a dynamic switching process that routes generation between the ensembled distribution and a set of local “specialist” models based on support constraints\. The generation at stepttis determined by the policy outlined below:

#### Generation via Union\-based Ensemble

To avoid semantic truncation, we construct the ensemble distributionp¯N\\bar\{p\}\_\{N\}over the vocabulary union𝒱∪\\mathcal\{V\}\_\{\\cup\}\. For any modeliiwith tokenv∉𝒱iv\\notin\\mathcal\{V\}\_\{i\}, we assignpi​\(v\|x<t\)=0p\_\{i\}\(v\|x\_\{<t\}\)=0, so that the ensemble average reduces top¯N​\(v\|x<t\)=1N​∑i:v∈𝒱ipi​\(v\|x<t\)\\bar\{p\}\_\{N\}\(v\|x\_\{<t\}\)=\\frac\{1\}\{N\}\\sum\_\{i:v\\in\\mathcal\{V\}\_\{i\}\}p\_\{i\}\(v\|x\_\{<t\}\)\. For any token supported by only one model, this capsp¯N​\(v\|x<t\)\\bar\{p\}\_\{N\}\(v\|x\_\{<t\}\)at approximately1/N1/Nof that model’s own probability\. However, this scaling does not indicate a loss of confidence: models lackingvvin their vocabulary still allocate comparable probability mass to the same semantic content, but route it through alternative tokenisation paths \(e\.g\., one model emits “Gracious” as an atomic token while others emit the same word as “\[Gr\]\[acious\]”\)\. The semantic signal is therefore dispersed across divergent tokenisation paths rather than discarded\. Crucially, unlike the intersection approach, every token in𝒱∪\\mathcal\{V\}\_\{\\cup\}remains accessible\. The next token is then sampled from this union\-aggregated distribution:xt∼p¯N\(⋅\|x<t\)\.x\_\{t\}\\sim\\bar\{p\}\_\{N\}\(\\cdot\|x\_\{<t\}\)\.

#### Fluency\-Aware Routing

Ifxt∈𝒱∩x\_\{t\}\\in\\mathcal\{V\}\_\{\\cap\}\(i\.e\., the token is valid across all models\), we directly outputxtx\_\{t\}\. However, a critical challenge arises when the selected tokenxt∈𝒱Δx\_\{t\}\\in\\mathcal\{V\}\_\{\\Delta\}, it is undefined to the subset of models that lack this token in their vocabulary\. Accepting such a token would break the autoregressive chain, as the incompatible models cannot processxtx\_\{t\}as valid input context for the subsequent generation step \(t\+1t\+1\)\. To preserve fluency, we employ a randomised routing mechanism that restricts subsequent sampling to “specialist” models whose vocabularies admit the committed tokens\. LetKKdenote the set of all available models and we initialiseKt=KK\_\{t\}=K\. For eacht′∈\[t\+1,τ​\(t\)\]t^\{\\prime\}\\in\[t\+1,\\tau\(t\)\], whereτ​\(t\)\\tau\(t\)denotes word completion, we recurrently update the specialist set as

Kt′=\{i:xt′−1∈𝒱i\}∩Kt′−1\.K\_\{t^\{\\prime\}\}=\\\{i:x\_\{t^\{\\prime\}\-1\}\\in\\mathcal\{V\}\_\{i\}\\\}\\cap K\_\{t^\{\\prime\}\-1\}\.We then uniformly sample a model from the specialist set before generating the token att′t^\{\\prime\}:

kt′∼Uniform\(Kt′\),xt′∼pkt′\(⋅\|x<t′\)\.k\_\{t^\{\\prime\}\}\\sim\\text\{Uniform\}\(K\_\{t^\{\\prime\}\}\),\\quad x\_\{t^\{\\prime\}\}\\sim p\_\{k\_\{t^\{\\prime\}\}\}\(\\cdot\|x\_\{<t^\{\\prime\}\}\)\.Crucially, we avoid selecting the model using maximum likelihood principles, as likelihood disparities may themselves be watermark\-induced artifacts\. Random selection ensures the routing decision is orthogonal to watermark signals, preventing detectors from exploiting systematic biases in model selection\.

While routing temporarily re\-admits the watermark signalδwm\(kt′\)\\delta\_\{\\text\{wm\}\}^\{\(k\_\{t^\{\\prime\}\}\)\}, such events are sparse and confined to vocabulary boundaries\. Moreover, our strategy consistently engages as many specialist models as possible, ensuring that ensembling remains active throughout the generation\. Critically, even if some watermark signal exhibits, watermark artefacts are temporally fragmented: they alternate stochastically across different specialists and are interspersed with non\-routing tokens\. This prevents the accumulation of sustained statistical regularities required for reliable watermark identification\. We provide a complete generation example in Appendix[E](https://arxiv.org/html/2605.30501#A5)\.

#### Context Re\-synchronisation

Routing to specialist models createscontext de\-synchronisation: non\-selected models do not observe the generated tokens in their native tokenisation\. To maintain coherence, we apply a decode\-encode cycle\. We decode the generated string with the last selected specialist modelSt=Decodekτ​\(t\)​\(x^t:τ​\(t\)\)S\_\{t\}=\\text\{Decode\}\_\{k\_\{\\tau\(t\)\}\}\(\\hat\{x\}\_\{t:\\tau\(t\)\}\), and update each model’s context via:

x<τ​\(t\)\+1\(i\)←x<t\(i\)⊕𝒯i​\(St\),∀i∈\[N\],x\_\{<\\tau\(t\)\+1\}^\{\(i\)\}\\leftarrow x\_\{<t\}^\{\(i\)\}\\oplus\\mathcal\{T\}\_\{i\}\(S\_\{t\}\),\\quad\\forall i\\in\[N\],\(6\)where𝒯i\\mathcal\{T\}\_\{i\}is modelii’s tokeniser\. This ensures all models observe semantically equivalent contexts despite tokenisation differences, preserving valid probability aggregation in subsequent ensemble steps\.

## 3Experiment

![Refer to caption](https://arxiv.org/html/2605.30501v1/figures/main_plot_v3.png)Figure 3:Watermark signal decay under different ensemble configurations\.We measure detection strength \(z\-score\) as ensemble sizeNNincreases\.\(a\)Fixed base model withNNindependent watermark keys\.\(b\)NNindependent models, each with independent watermarks\.\(c\)3 independent base models sharing the same watermark, with the signals coordinated across models: signal persists, demonstrating that coordination defeats averaging attacks\.### 3\.1Experimental Setup

Models and Watermarks\.We conduct experiments on three widely used pre\-trained LLMs:Qwen3\-8B\(Qwen,[2025](https://arxiv.org/html/2605.30501#bib.bib22)\),Llama\-3\.1\-8B\(Meta AI,[2024](https://arxiv.org/html/2605.30501#bib.bib23)\), andMinistral3\-8B\(Mistral,[2026](https://arxiv.org/html/2605.30501#bib.bib24)\)\. To ensure broad coverage, we evaluate six representative watermarking schemes spanning different design paradigms:AAR, which relies on uniform distribution\(Aaronson and Kirchner,[2022](https://arxiv.org/html/2605.30501#bib.bib16)\);DIPMark, which applies logit reweighting\(Wuet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib14)\);ITSEdit, based on inverse transform sampling with exponential minimum sampling\(Kuditipudiet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib1)\);KGW, which encourages sampling to a green list of tokens\(Kirchenbaueret al\.,[2023a](https://arxiv.org/html/2605.30501#bib.bib2)\);Exp\-Edit, which uses key\-based transformations\(Kuditipudiet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib1)\); andWater\-Bag, which combines a set of watermark keys and their mathematical inversions during generation to mask statistical biases\(Liuet al\.,[2025](https://arxiv.org/html/2605.30501#bib.bib15)\)\.

Baselines\.We compareWASHwith two classes of watermark removal attacks\.

Generation\-time attacksproduce the output directly under the removal procedure, matching theWASHsetting: \(1\)De\-mark\(Chenet al\.,[2025a](https://arxiv.org/html/2605.30501#bib.bib12)\): A watermark removal method designed against red\-green\-list watermarking\. It first identifies the green\-list tokens whose logits are biased by the watermark using crafted prompts, then removes the bias to restore the original generation distribution\. \(2\)ToBlend\(Huanget al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib8)\): A model generation mixture method against AI text detections\. It blends multiple model generations by predicting a fixed number of tokens with a single selected model each time\.

Final\-text rewrite attacksinstead operate on an already generated watermarked sequence: \(1\)RandomWalk\(Liuet al\.,[2025](https://arxiv.org/html/2605.30501#bib.bib15)\): A strategy that repeatedly rewrites spans of the generated text using a weaker unwatermarked model and accepts only quality\-preserving variants based on a quality\-check oracle\.

Benchmarks and Metrics\.We use two evaluation suites: \(1\)Detection\. Forgeneration\-time attacks, we follow the z\-score protocol ofLiuet al\.\([2025](https://arxiv.org/html/2605.30501#bib.bib15)\)to quantify watermark signal strength\. Forfinal\-text rewrite attacks, perturbation detection on the small set of generated tokens is no longer compatible, so we additionally use native sequence detectors\(Panet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib37)\)\. Detection thresholds and confidence categories are reported in Appendix[D\.1](https://arxiv.org/html/2605.30501#A4.SS1)\. \(2\)Quality and Efficiency\. To assess generation quality and inference speed, we use four benchmarks covering representative reasoning and language tasks: GSM8K \(math reasoning\)\(Cobbeet al\.,[2021](https://arxiv.org/html/2605.30501#bib.bib19)\), MMLU \(knowledge\)\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.30501#bib.bib20)\), SQuAD \(reading comprehension\)\(Rajpurkaret al\.,[2016](https://arxiv.org/html/2605.30501#bib.bib21)\), WritingBench \(open\-ended writing\)\(Wuet al\.,[2025](https://arxiv.org/html/2605.30501#bib.bib36)\)\.

Table 1:Comparison of Watermark Removal Effectiveness\. The table evaluates z\-score after various removal attacks\.indicates high\-confidence watermark identification, andindicates low\-confidence watermark identification, while no colour indicates no watermark identified\.Base ModelRemoval MethodTarget Watermark \(Individual\)AARDIPMarkITS\-EditKGWExp\-EditWater\-BagLlama3\.1\-8BWatermarked Baseline176\.43858\.127304\.02250\.11361\.98110\.237De\-mark—99\.281—1\.786—\-10\.687ToBlend3\.6361\.3440\.4890\.8190\.482\-0\.003WASH\(N=3\)3\.6991\.4770\.3840\.634\-0\.245\-0\.109WASH\(N=4\)3\.0820\.5630\.4080\.5500\.3720\.093WASH\(N=5\)2\.2320\.1820\.1190\.372\-0\.108\-0\.111Qwen3\-8BWatermarked Baseline15\.53414\.21015\.41834\.54726\.9045\.245De\-mark—4\.560—3\.770—\-8\.452ToBlend1\.9410\.5230\.4640\.4890\.175\-0\.169WASH\(N=3\)1\.6290\.2360\.6020\.0580\.517\-0\.205WASH\(N=4\)1\.2860\.0940\.0680\.1300\.2000\.000WASH\(N=5\)0\.6540\.1890\.3370\.2220\.079\-2\.570Ministral3\-8BWatermarked Baseline9\.17523\.13527\.08227\.40759\.5079\.568De\-mark—91\.174—\-2\.814—0\.729ToBlend1\.2041\.5560\.7111\.1680\.6230\.437WASH\(N=3\)1\.0700\.0590\.5581\.7320\.7870\.775WASH\(N=4\)1\.9011\.3501\.3631\.6020\.630\-1\.809WASH\(N=5\)0\.4210\.1410\.4560\.4610\.435\-0\.382

### 3\.2Watermark Removal Effectiveness

The experimental results in Figure[3](https://arxiv.org/html/2605.30501#S3.F3)demonstrate the inherent fragility of distributional watermarking when subjected to linear ensemble averaging\. Our analysis proceeds in three stages: validating the scaling law of signal decay, isolating the removal mechanism through control experiments, and comparing comprehensive performance against baselines\.

Signal Decay: From Theory to Reality\.We first validate the scaling law of signal decay in an idealised setting where models utilise independent watermarks on the same base model \(Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(a\)\)\. The detection signal diminishes rapidly withNN, mirroring our theoretical prediction ofO​\(1/N\)O\(1/\\sqrt\{N\}\)\. The pivotal result lies in Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(b\), which simulates the realistic user scenario: accessing distinct heterogeneous models \(e\.g\., Llama, Qwen, Ministral\) with independent watermarks\. Despite the challenge of vocabulary mismatches, our fluency\-aware routing successfully neutralises the watermark signal\. Notably, the detection strength drops even faster than in the homogeneous setting \(reaching nearz=0z=0atN=5N=5\), suggesting that the diversity of base model distributions acts as additional noise that further obscures the watermark trace\. We provide the experiment details in Appendix[D\.2](https://arxiv.org/html/2605.30501#A4.SS2)\.

A Control Experiment: Coordinated Watermark\.To verify the mechanism of removal, we evaluate a “Coordinated Watermark” scenario where all ensemble models share the same watermark scheme, and signals are coordinated across distinct vocabularies during each token generation\. Specifically, we calculate the relative perturbations of the common tokens from one watermarked model and map them to other models using the same relative scale\. As illustrated in Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(c\), when the ensemble models are synchronised, the washing effect fails\. The ensemble average retains a statistically significant z\-score well above the detection threshold \(z\>4z\>4\) across all schemes\. Strong schemes like AAR and Exp\-Edit maintain high z\-scores of 25\.11 and 17\.44, respectively\. Crucially, thepersistenceof these signals demonstrates that the ensemble process itself does not inherently obliterate watermark information\. Instead, the removal efficacy in the independent setting \(Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(a,b\)\) exploits the lack of correlation between providers\. This finding highlights a critical defensive insight: robust AI\-text detection against ensemble attacks requirescoordinated watermarking strategiesacross model providers\.

Comparison with Generation\-time Attacks\.Table[1](https://arxiv.org/html/2605.30501#S3.T1)comparesWASHagainstgeneration\-time attacksacross various LLMs using the same z\-score detection protocol\. Individual watermarked models exhibit strong detection signals, with z\-scores ranging from 5\.2 to 304\.0\. Prior removal methods show limited efficacy:De\-markperforms well only on pure red\-green\-list watermarking methods with exact logits bias value like KGW and Water\-Bag, while collapsing on other distribution modifications \(4\.6 to 99\.3 z\-score on DIPMark\)\. It also cannot apply to watermarks that operate on probabilities\. In contrast,WASHconsistently neutralises detection signals\. An ensemble ofN=3N=3proves practically sufficient to suppress z\-scores below the threshold \(z<4z<4\) across all cases\. Extending toN=5N=5offers a safety margin for aggressive schemes, ensuring near\-zero distinguishability\.ToBlend, as a model mixture method like ours, could achieve similar removal efficacy toWASHin the detection evaluation, while significantly harming generation quality and efficiency, as we discuss in the next section\.

Table 2:Native\-detector watermark removal results with final\-text rewrite attacks\. The results are reported in TPR@5% FPR calibrated on unwatermarked results, where lower is better\.MethodDIPMarkKGWAARITS\-EditExp\-EditWatermarked83\.792\.895\.983\.585\.9De\-mark76\.558\.8———ToBlend28\.238\.445\.421\.411\.2RandomWalk49\.056\.724\.253\.624\.0WASH\(N=5\)27\.638\.942\.816\.211\.3

Comparison with Final\-text Rewrite Attacks\.Final\-text rewrite attacksrequire a separate evaluation as they perturb the completed sequence, making token perturbation detectors inapplicable\. We thus report the TPR@5% FPR from native sequence\-level detectors\. Lower values indicate stronger removal under thresholds calibrated on unwatermarked samples\. As shown in Table[2](https://arxiv.org/html/2605.30501#S3.T2),RandomWalkleaves several schemes near or within the low\-confidence range\.WASHkeeps all schemes below 43%, matchingToBlend’s removal ability and outperformingRandomWalkon 4 out of 5 schemes, while retaining quality and efficiency advantages analysed below\.

Table 3:Performance and Efficiency Comparison\.We evaluateBluegeneration quality \(GSM8K, MMLU, SQuAD\) andRedrelative computational cost \(Time\) across the watermarked baseline, removal attacks, and our proposed method with varying ensemble sizes \(nn\)\. Time is normalised to the baseline cost \(1\.0×1\.0\\times\)\.Base ModelMethodGSM8KMMLUSQuADAcc\.↑\\uparrowTime↓\\downarrowAcc\.↑\\uparrowTime↓\\downarrowF1↑\\uparrowTime↓\\downarrowLlama3\.1\-8BUnwatermarked0\.567±\\pm0\.0121\.00×\\times0\.583±\\pm0\.0001\.00×\\times0\.795±\\pm0\.0131\.00×\\timesWatermarked Baseline0\.511±\\pm0\.0321\.00×\\times0\.560±\\pm0\.0231\.00×\\times0\.749±\\pm0\.0381\.00×\\timesDe\-mark0\.550±\\pm0\.04038\.80×\\times0\.565±\\pm0\.01510\.01×\\times0\.642±\\pm0\.07132\.50×\\timesToBlend0\.568±\\pm0\.08714\.60×14\.60\\times0\.631±\\pm0\.0381\.06×\\times0\.633±\\pm0\.04517\.55×17\.55\\timesWASH\(N=3\)0\.695±\\pm0\.0672\.19×\\times0\.631±\\pm0\.0381\.06×\\times0\.764±\\pm0\.0311\.49×\\timesWASH\(N=4\)0\.701±\\pm0\.0742\.37×\\times0\.644±\\pm0\.0321\.06×\\times0\.765±\\pm0\.0281\.53×\\timesWASH\(N=5\)0\.698±\\pm0\.0572\.28×\\times0\.646±\\pm0\.0251\.06×\\times0\.764±\\pm0\.0311\.54×\\timesQwen3\-8BUnwatermarked0\.713±\\pm0\.0211\.00×\\times0\.726±\\pm0\.0001\.00×\\times0\.421±\\pm0\.0041\.00×\\timesWatermarked Baseline0\.693±\\pm0\.0511\.00×\\times0\.723±\\pm0\.0021\.00×\\times0\.396±\\pm0\.0141\.00×\\timesDe\-mark0\.755±\\pm0\.03530\.97×\\times0\.724±\\pm0\.0029\.47×\\times0\.434±\\pm0\.01230\.98×\\timesToBlend0\.701±\\pm0\.05011\.40×11\.40\\times0\.722±\\pm0\.0111\.01×\\times0\.474±\\pm0\.07119\.18×19\.18\\timesWASH\(N=3\)0\.804±\\pm0\.0501\.53×\\times0\.722±\\pm0\.0111\.01×\\times0\.610±\\pm0\.0841\.69×\\timesWASH\(N=4\)0\.809±\\pm0\.0471\.58×\\times0\.723±\\pm0\.0091\.01×\\times0\.629±\\pm0\.0671\.82×\\timesWASH\(N=5\)0\.808±\\pm0\.0351\.60×\\times0\.723±\\pm0\.0101\.01×\\times0\.612±\\pm0\.0711\.96×\\timesMinistral3\-8BUnwatermarked0\.837±\\pm0\.0251\.00×\\times0\.731±\\pm0\.0001\.00×\\times0\.839±\\pm0\.0001\.00×\\timesWatermarked Baseline0\.809±\\pm0\.0411\.00×\\times0\.716±\\pm0\.0271\.00×\\times0\.795±\\pm0\.0321\.00×\\timesDe\-mark0\.805±\\pm0\.00536\.39×\\times0\.717±\\pm0\.01510\.02×\\times0\.832±\\pm0\.00436\.42×\\timesToBlend0\.715±\\pm0\.03612\.17×12\.17\\times0\.716±\\pm0\.0151\.04×\\times0\.775±\\pm0\.01615\.92×15\.92\\timesWASH\(N=3\)0\.815±\\pm0\.0301\.25×\\times0\.716±\\pm0\.0151\.04×\\times0\.805±\\pm0\.0261\.19×\\timesWASH\(N=4\)0\.821±\\pm0\.0261\.30×\\times0\.718±\\pm0\.0071\.03×\\times0\.807±\\pm0\.0201\.30×\\timesWASH\(N=5\)0\.818±\\pm0\.0121\.28×\\times0\.718±\\pm0\.0051\.03×\\times0\.821±\\pm0\.0291\.30×\\times

### 3\.3Quality, Semantic Integrity, and Efficiency

Comparison with Generation\-time Attacks\.As shown in Table[3](https://arxiv.org/html/2605.30501#S3.T3), watermarking inevitably distorts the optimal output distribution, leading to up to 10% performance degradation relative to the unwatermarked baseline\. While removal attacks aim to recover this lost utility by restoring the original distribution, prior methods face two bottlenecks: \(1\) excessive perturbation may further harm generation quality, and \(2\) the computational cost of recovery can be prohibitive\.

Consistent with our theoretical analysis,WASHresolves these tensions by recovering the unwatermarked distribution through ensembling\. It achieves generation quality comparable to or superior to the strongest baseline \(De\-mark\) in 7 out of 9 benchmark settings, while outperforming the other mixture method \(ToBlend\) across all settings\.

The efficiency advantage is decisive for generation\-intensive tasks\. While prior overhead may be tolerable for short discriminative tasks such as MMLU, it becomes prohibitive for long\-form generation, where watermarking is most relevant for content provenance, copyright protection, and misuse attribution\. On GSM8K and SQuAD,De\-marksuffers severe latency \(\>30×\>30\\times\) due to iterative distribution probing, andToBlendincurs similarly high overhead \(∼12×\\sim 12\\times\) because it repeatedly reprocesses context and predicts redundant future tokens\. In contrast,WASHoperates at only1\.0×1\.0\\times–2\.4×2\.4\\timesthe baseline cost through deterministic sampling and synchronised KV caching, giving roughly a 6×\\timesspeedup over competing methods while maintaining stronger quality\.

Comparison with Final\-text Rewrite Attacks\.We evaluateRandomWalkon GSM8K and WritingBench rather than MMLU and SQuAD because final\-text rewriting requires sufficiently long generations to meaningfully change surface form while preserving content\. MMLU multiple\-choice outputs and SQuAD short answers are often too short to expose either rewriting effectiveness or its utility cost\.

Table 4:Comparison with final\-text rewriting on GSM8K and WritingBench\. Runtime is normalised to the watermarked baseline\.MethodGSM8KWritingBenchAcc\.↑\\uparrowTime↓\\downarrowScore↑\\uparrowTime↓\\downarrowWatermarked0\.5111\.00×\\times4\.101\.00×\\timesDe\-mark0\.55038\.46×\\times4\.0443\.97×\\timesToBlend0\.56813\.28×\\times2\.327\.89×\\timesRandomWalk0\.4674\.39×\\times3\.6110\.03×\\timesWASH\(N=5\)0\.6982\.28×\\times4\.261\.85×\\times

As shown in Table[4](https://arxiv.org/html/2605.30501#S3.T4),RandomWalkunderperforms the watermarked baseline on both reasoning and open\-ended writing, likely because in\-place rewriting with another model introduces instability\. Its additional rewriting phase also incurs substantial overhead, reaching4×4\\timesruntime on GSM8K and10×10\\timeson WritingBench\. In contrast,WASHimproves quality while requiring only about2×2\\timesruntime\. Thus, final\-text rewriting may weaken detection, but it pays for this through lower utility and significantly higher latency\.

Resource and Serving Trade\-offs\.Table[5](https://arxiv.org/html/2605.30501#S3.T5)further evaluates the system’s cost ofWASH\. The parallel implementation keeps all specialists resident and synchronises their KV caches, prioritising latency\. The sequential implementation loads specialists on demand, reducing peak memory at the cost of higher token latency\. This givesWASHa practical deployment trade\-off:WASH\-Par\. is substantially faster thanDe\-markandToBlend, whileWASH\-Seq\. reduces peak memory close to the single\-model baseline\. Thus,WASHremains practical under both latency\-constrained and memory\-constrained serving regimes\.

Table 5:Resource usage and decoding latency\. Peak memory is measured during generation, and token latency is averaged over generated tokens\. WASH\-Par\. keeps all specialists resident, whereas WASH\-Seq\. offloads inactive specialists dynamically\.MethodPeak MemRel\.↓\\downarrowToken LatencyRel\.↓\\downarrowUnwatermarked15\.09 GB1\.00×\\times29\.5 ms0\.98×\\timesWatermarked15\.10 GB1\.00×\\times30\.0 ms1\.00×\\timesDe\-mark16\.58 GB1\.10×\\times190\.6 ms6\.35×\\timesToBlend71\.46 GB4\.73×\\times172\.3 ms5\.74×\\timesWASH\-Par\.40\.87 GB2\.71×\\times56\.9 ms1\.90×\\timesWASH\-Seq\.15\.76 GB1\.04×\\times165\.5 ms5\.52×\\times

### 3\.4Ablation and Analysis of Fluency\-Aware Routing

Routing Ablation\.Fluency\-aware routing is the main difference betweenWASHand naive distribution averaging\. Table[6](https://arxiv.org/html/2605.30501#S3.T6)evaluates this design choice\.Naive Avgremoves routing and directly averages distributions\. TheRewritevariants then attempt to repair theNaive Avgoutput using an additional watermarked or unwatermarked copy of the same base model, testing whether post\-hoc rewriting can replace generation\-time routing\.

Table 6:Ablation of fluency\-aware routing\.MethodGSM8K↑\\uparrowWritingBench↑\\uparrowDetectTPR@5% FPR↓\\downarrowNaive Avg0\.3393\.9541\.3\+ Rewrite \(WM\)0\.1974\.0276\.8\+ Rewrite \(Un\-WM\)0\.2054\.0510\.9WASH\(N=5\)0\.6984\.2633\.3

The results show that post\-hoc repair is not an effective substitute for routing\.Naive Avgsuppresses detection, but substantially degrades GSM8K accuracy\.Rewriteimproves surface\-level writing quality, yet it further hurts reasoning accuracy\.WASHinstead performs sparse local routing during generation, preserving both low detection and substantially stronger utility\.

Routing Robustness\.We further test whether fluency\-aware routing could reintroduce watermark signal when specialised vocabulary is frequent\. This concern is most relevant in domains such as medicine and law, where rare terminology might trigger long single\-specialist spans, weakening the averaging effect ofWASH\. Table[7](https://arxiv.org/html/2605.30501#S3.T7)shows that this does not occur\. The routed tokens account for less than 3\.2% of generation; routed spans are short; and final detection scores remain far below the watermark threshold\. Manual inspection suggests that complex technical terms are often decomposed into shared subword units, while routing is more frequently triggered by ordinary lexical units with tokeniser mismatches\. Thus, fluency\-aware routing acts as a sparse local repair mechanism rather than sustained single\-model generation\.

Table 7:Routing statistics under specialised vocabulary\. Medical and legal MMLU subsets are converted into free\-form reasoning prompts\. We report routing frequency, routed\-span length, and final detection score\.DomainRouted Token Frac\.Avg\. Routed LenDetectionzz↓\\downarrowMedical3\.2%3\.70\.84Legal2\.7%3\.61\.08

## 4Related Works

Providing and Detecting Watermarks\. Recent research has established LLM watermarking through green/red lists\.\(Kirchenbaueret al\.,[2023a](https://arxiv.org/html/2605.30501#bib.bib2)\)\. While effective, this approach struggles with low\-entropy text and is prone to Type II errors\. To mitigate these limitations, subsequent research has focused on distribution\-preserving and resilient schemes\. For instance,Kuditipudiet al\.\([2024](https://arxiv.org/html/2605.30501#bib.bib1)\)andWuet al\.\([2024](https://arxiv.org/html/2605.30501#bib.bib14)\)propose distortion\-free methods to maintain generation quality\. Similarly,Huet al\.\([2024](https://arxiv.org/html/2605.30501#bib.bib3)\)introduce unbiased watermarking via reweighting mechanisms and log\-likelihood tests; however, this method assumes knowledge of the generation process and often lacks adversarial robustness\. Further advancements explore alternative mechanisms on modifying generation bias through a semantic\-aware watermarking\(Guoet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib7)\)or maximal coupling strategies\(Xieet al\.,[2025](https://arxiv.org/html/2605.30501#bib.bib6)\), while others aim to tackle detection challenges in low\-entropy scenarios\(Maoet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib4)\)or improve scalability\(Dathathriet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib5)\)\.

Watermark Removal Attacks\. Early attack approaches use paraphrasing to preserve semantics while resampling expressions\(Krishnaet al\.,[2023](https://arxiv.org/html/2605.30501#bib.bib34)\), or sampling from multiple keys to perform majority voting\(Panget al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib35)\)\.RandomWalkfurther rewrites local spans and accepts quality\-preserving candidates through an external quality check\(Liuet al\.,[2025](https://arxiv.org/html/2605.30501#bib.bib15)\)\. However, these methods typically rely on detector\-available scenarios, assuming the adversary has access to an oracle verifier\. Recent research attempts to eliminate this dependence\. Many theoretical attacks necessitate query\-intensive strategies to reverse\-engineer watermarking rules\. Methods such asWatermark Stealing\(Jovanovićet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib9)\),SCTS\(Wu and Chandrasekaran,[2024](https://arxiv.org/html/2605.30501#bib.bib11)\), andDe\-Mark\(Chenet al\.,[2025a](https://arxiv.org/html/2605.30501#bib.bib12)\)require a high volume of specific prompt queries or iterative token\-level probing to infer red\-green lists\. Similarly, optimisation\-based attacks employ computationally expensive random walks\(Zhanget al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib10)\)\. While theoretically effective, these strategies are impractical for real\-time or long\-form generation tasks due to prohibitive latency and computational costs\. Additionally,ToBlendattempts to bypass watermarks by alternating between models\(Huanget al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib8)\), though often at the cost of coherence or inference efficiency due to frequent context re\-encoding\.

## 5Conclusion

We demonstrate that current distributional watermarking schemes are structurally vulnerable to linear ensembling\. Theoretically, we prove that averaging outputs from independent models cancels watermark perturbations\. To address vocabulary mismatches in heterogeneous ensembles, we introduce WASH, which utilises fluency\-aware routing to enable effective probability aggregation\. Empirically, WASH renders watermarks statistically undetectable \(z<2z<2\) with as few as three models, while preserving generation quality with practical inference efficiency\. Our findings suggest that reliable detection in a competitive marketplace is unattainable without industry\-wide standardisation of watermark keys among model providers\.

## Impact Statement

This paper presents work whose goal is to advance the field of AI\-generated content detection\. This is increasingly important for maintaining societal trust in digital information and content authenticity, and for protecting intellectual property\. We demonstrate that various contemporary LLM watermarking schemes are fragile under a general linear ensembling attack\. While this attack exposes current limitations, it also serves as an alert to model providers and researchers to strengthen watermarking against adaptive and cross\-model threats\.

Moreover, we have conducted a coordination experiment, revealing that even simple signal coordination across heterogeneous models can partially mitigate the effectiveness of the ensemble attack\. These findings suggest that true watermarking robustness depends not only on isolated model\-level defences, but also on cooperative ways that may be more effective\. In general, we underscore the value of cross\-provider collaboration in developing robust watermarking provenance mechanisms for AI\-generated content\.

## Acknowledgments

This work was supported in part by the UK Engineering and Physical Sciences Research Council \(EPSRC\) through a Turing AI Fellowship \(grant no\. EP/V020579/1, EP/V020579/2\)\. We thank Xiaojia Rao for participating in the initial discussions that helped shape the idea of this work\.

## References

- S\. Aaronson and H\. Kirchner \(2022\)Watermarking gpt outputs\.Note:[https://www\.scottaaronson\.com/talks/watermark\.ppt](https://www.scottaaronson.com/talks/watermark.ppt)Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1)\.
- R\. Chen, Y\. Wu, J\. Guo, and H\. Huang \(2025a\)De\-Mark: watermark removal in large language models\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p4.3),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p3.1),[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- Z\. Chen, X\. Lu, J\. Li, P\. Chen, Z\. Li, K\. Sun, Y\. Luo, Q\. Mao, M\. Li, L\. Xiao, D\. Yang, X\. Huang, Y\. Ban, H\. Sun, and P\. S\. Yu \(2025b\)Harnessing multiple large language models: a survey on llm ensemble\.arXiv preprint arXiv:2502\.18036\.Cited by:[§2\.3](https://arxiv.org/html/2605.30501#S2.SS3.p1.6)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman \(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p5.1)\.
- S\. Dathathri, A\. See, S\. Ghaisas, P\. Huang, R\. McAdam, J\. Welbl, V\. Bachani, A\. Kaskasoli, R\. Stanforth, T\. Matejovicova,et al\.\(2024\)Scalable watermarking for identifying large language model outputs\.Nature634\(8035\),pp\. 818–823\.Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- Y\. Guo, Z\. Tian, Y\. Song, T\. Liu, L\. Ding, and D\. Li \(2024\)Context\-aware watermark with semantic balanced green\-red lists for large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 22633–22646\.Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.Proceedings of the International Conference on Learning Representations \(ICLR\)\.Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p5.1)\.
- W\. Hoeffding \(1963\)Probability inequalities for sums of bounded random variables\.Journal of the American Statistical Association58\(301\),pp\. 13–30\.External Links:[Document](https://dx.doi.org/10.1080/01621459.1963.10500830),[MathReview Entry](https://www.ams.org/mathscinet-getitem?mr=0144363)Cited by:[Appendix A](https://arxiv.org/html/2605.30501#A1.SS0.SSS0.Px2.p1.4),[Appendix B](https://arxiv.org/html/2605.30501#A2.SS0.SSS0.Px1.p1.12)\.
- Z\. Hu, L\. Chen, X\. Wu, Y\. Wu, H\. Zhang, and H\. Huang \(2024\)Unbiased watermark for large language models\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1),[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- F\. Huang, H\. Kwak, and J\. An \(2024\)ToBlend: token\-level blending with an ensemble of llms to attack ai\-generated text detection\.arXiv preprint arXiv:2402\.11167\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p4.3),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p3.1),[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- N\. Jovanović, R\. Staab, and M\. Vechev \(2024\)Watermark stealing in large language models\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),pp\. 22570 – 22593\.Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- J\. Khasentino, A\. Belyaeva, X\. Liu, Z\. Yang, N\. A\. Furlotte, C\. Lee, E\. Schenck, Y\. Patel, J\. Cui, L\. D\. Schneider, R\. Bryant, R\. G\. Gomes, A\. Jiang, R\. Lee, Y\. Liu, J\. Perez, J\. K\. Rogers, C\. Speed, S\. Tailor, M\. Walker, J\. Yu, T\. Althoff, C\. Heneghan, J\. Hernandez, M\. Malhotra, L\. Stern, Y\. Matias, G\. S\. Corrado, S\. Patel, S\. Shetty, J\. Zhan, S\. Prabhakara, D\. McDuff, and C\. Y\. McLean \(2025\)A personal health large language model for sleep and fitness coaching\.Nature Medicine31\(10\),pp\. 3394–3403\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1)\.
- J\. Kirchenbauer, J\. Geiping, Y\. Wen, J\. Katz, I\. Miers, and T\. Goldstein \(2023a\)A watermark for large language models\.InProceedings of the 40th International Conference on Machine Learning \(ICML\),pp\. 17061–17084\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1),[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- J\. Kirchenbauer, J\. Geiping, Y\. Wen, M\. Shu, K\. Saifullah, K\. Kong, K\. Fernando, A\. Saha, M\. Goldblum, and T\. Goldstein \(2023b\)On the reliability of watermarks for large language models\.arXiv preprint arXiv:2306\.04634\.Cited by:[§D\.1](https://arxiv.org/html/2605.30501#A4.SS1.SSS0.Px1.p2.3)\.
- K\. Krishna, Y\. Song, M\. Karpinska, J\. Wieting, and M\. Iyyer \(2023\)Paraphrasing evades detectors of ai\-generated text, but retrieval is an effective defense\.InProceedings of the 37th Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- R\. Kuditipudi, J\. Thickstun, T\. Hashimoto, and P\. Liang \(2024\)Robust distortion\-free watermarks for language models\.Transactions on Machine Learning Research\.Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1),[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- A\. Liu, S\. Guan, Y\. Liu, L\. Pan, Y\. Zhang, L\. Fang, L\. Wen, P\. S\. Yu, and X\. Hu \(2025\)Can watermarked LLMs be identified by users via crafted prompts?\.InProceedings of the International Conference on Learning Representations \(ICLR\),Cited by:[§D\.1](https://arxiv.org/html/2605.30501#A4.SS1.SSS0.Px1.p1.3),[§D\.1](https://arxiv.org/html/2605.30501#A4.SS1.SSS0.Px1.p2.3),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p4.1),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p5.1),[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- A\. Liu, L\. Pan, Y\. Lu, J\. Li, X\. Hu, X\. Zhang, L\. Wen, I\. King, H\. Xiong, and P\. Yu \(2024\)A survey of text watermarking in the era of large language models\.ACM Computing Surveys57\(2\),pp\. 1–36\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1)\.
- M\. Mao, D\. Wei, Z\. Chen, X\. Fang, and M\. Chau \(2024\)Watermarking low\-entropy generation for large language models: an unbiased and low\-risk method\.arXiv preprint arXiv:2405\.14604\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1),[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- Meta AI \(2024\)Introducing Meta Llama 3: The most capable openly available LLM to date\.Note:[https://ai\.meta\.com/blog/meta\-llama\-3/](https://ai.meta.com/blog/meta-llama-3/)Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1)\.
- Mistral \(2026\)Ministral 3\.arXiv preprint arXiv:2601\.08584\.External Links:[Link](https://arxiv.org/abs/2601.08584)Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1)\.
- L\. Pan, A\. Liu, Z\. He, Z\. Gao, X\. Zhao, Y\. Lu, B\. Zhou, S\. Liu, X\. Hu, L\. Wen,et al\.\(2024\)Markllm: an open\-source toolkit for llm watermarking\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,pp\. 61–71\.Cited by:[§D\.1](https://arxiv.org/html/2605.30501#A4.SS1.SSS0.Px1.p2.3),[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p5.1)\.
- Q\. Pang, S\. Hu, W\. Zheng, and V\. Smith \(2024\)No free lunch in llm watermarking: trade\-offs in watermarking design choices\.arXiv preprint arXiv:2402\.16187\.Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- Qwen \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1)\.
- C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. Liu \(2020\)Exploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of machine learning research21\(140\),pp\. 1–67\.Cited by:[§D\.1](https://arxiv.org/html/2605.30501#A4.SS1.SSS0.Px1.p2.3)\.
- P\. Rajpurkar, J\. Zhang, K\. Lopyrev, and P\. Liang \(2016\)SQuAD: 100,000\+ questions for machine comprehension of text\.InProceedings of the 2016 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 2383–2392\.External Links:1606\.05250Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p5.1)\.
- L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin, W\. X\. Zhao, Z\. Wei, and J\. Wen \(2024\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1)\.
- S\. Wang, T\. Xu, H\. Li, C\. Zhang, J\. Liang, J\. Tang, P\. S\. Yu, and Q\. Wen \(2026\)Large language models for education: a survey and outlook\.IEEE Signal Processing Magazine42\(6\),pp\. 51–63\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1)\.
- A\. Wei, N\. Haghtalab, and J\. Steinhardt \(2023\)Jailbroken: how does llm safety training fail?\.InProceedings of the 37th Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1)\.
- Q\. Wu and V\. Chandrasekaran \(2024\)Bypassing LLM watermarks with color\-aware substitutions\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 8549–8581\.Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.
- Y\. Wu, Z\. Hu, J\. Guo, H\. Zhang, and H\. Huang \(2024\)A resilient and accessible distribution\-preserving watermark for large language models\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p1.1),[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- Y\. Wu, J\. Mei, M\. Yan, C\. Li, S\. Lai, Y\. Ren, Z\. Wang, J\. Zhang, M\. Wu, Q\. Jin, and F\. Huang \(2025\)WritingBench: a comprehensive benchmark for generative writing\.InProceedings of the 39th Conference on Neural Information Processing Systems \(NeurIPS\),Cited by:[§3\.1](https://arxiv.org/html/2605.30501#S3.SS1.p5.1)\.
- Y\. Xie, X\. Li, T\. Mallick, W\. Su, and R\. Zhang \(2025\)Debiasing watermarks for large language models via maximal coupling\.Journal of the American Statistical Association,pp\. 1–21\.Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p1.1)\.
- Y\. Yao, J\. Duan, K\. Xu, Y\. Cai, Z\. Sun, and Y\. Zhang \(2024\)A survey on large language model \(llm\) security and privacy: the good, the bad, and the ugly\.High\-Confidence Computing4\(2\)\.Cited by:[§1](https://arxiv.org/html/2605.30501#S1.p1.1)\.
- Y\. Yu, C\. C\. Kuo, Z\. Ye, Y\. Chang, and Y\. Li \(2024\)Breaking the ceiling of the LLM community by treating token generation as a classification for ensembling\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 1826–1839\.Cited by:[§2\.3](https://arxiv.org/html/2605.30501#S2.SS3.p1.6)\.
- H\. Zhang, B\. L\. Edelman, D\. Francati, D\. Venturi, G\. Ateniese, and B\. Barak \(2024\)Watermarks in the sand: impossibility of strong watermarking for generative models\.InProceedings of the 41st International Conference on Machine Learning \(ICML\),Cited by:[§4](https://arxiv.org/html/2605.30501#S4.p2.1)\.

## Appendix AProof of the Main Theorem

###### Theorem A\.1\(Convergence to Consensus Distribution\)\.

Under Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2), for any fixed contextxx, letp¯N\(⋅\|x\)=1N∑i=1Npi\(⋅\|x\)\\bar\{p\}\_\{N\}\(\\cdot\|x\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}p\_\{i\}\(\\cdot\|x\)be the aggregated distribution\. For anyδ\>0\\delta\>0, with probability at least1−δ1\-\\delta, theℓ∞\\ell\_\{\\infty\}distance between the aggregated distribution and the consensus distributionp∗\(⋅\|x\)p^\{\*\}\(\\cdot\|x\)satisfies:

∥p¯N\(⋅\|x\)−p∗\(⋅\|x\)∥∞≲log⁡\(\|𝒱\|/δ\)N\+η2,\\bigl\\\|\\bar\{p\}\_\{N\}\(\\cdot\|x\)\-p^\{\*\}\(\\cdot\|x\)\\bigr\\\|\_\{\\mathrm\{\\infty\}\}\\lesssim\\sqrt\{\\frac\{\\log\(\|\\mathcal\{V\}\|/\\delta\)\}\{N\}\}\+\\eta^\{2\},where\|𝒱\|\|\\mathcal\{V\}\|denotes the vocabulary size, andη2\\eta^\{2\}is the upper bound on the expected weighted variance of the perturbation as in \([2](https://arxiv.org/html/2605.30501#S2.E2)\)\.

###### Proof\.

For brevity, we omit the dependency onxxin the notation throughout this proof \(e\.g\.,p∗​\(v\)p^\{\*\}\(v\)instead ofp∗​\(v\|x\)p^\{\*\}\(v\|x\)\)\. The perturbed distribution for modeliiis given by:

pi​\(v\)=p∗​\(v\)​exp⁡\(δi​\(v\)\)Zi,where​Zi=∑u∈𝒱p∗​\(u\)​exp⁡\(δi​\(u\)\)\.p\_\{i\}\(v\)=\\frac\{p^\{\*\}\(v\)\\exp\(\\delta\_\{i\}\(v\)\)\}\{Z\_\{i\}\},\\quad\\text\{where \}\\,Z\_\{i\}=\\sum\_\{u\\in\\mathcal\{V\}\}p^\{\*\}\(u\)\\exp\(\\delta\_\{i\}\(u\)\)\.

#### Step 1: Shift invariance and centring\.

We perform a centring operation onδi\\delta\_\{i\}\. Due to the shift invariance property of the softmax function, replacingδi​\(v\)\\delta\_\{i\}\(v\)withδi′​\(v\)=δi​\(v\)−Ci\\delta^\{\\prime\}\_\{i\}\(v\)=\\delta\_\{i\}\(v\)\-C\_\{i\}does not changepi​\(v\)p\_\{i\}\(v\)\. In particular, by choosingCi=∑up∗​\(u\)​δi​\(u\)C\_\{i\}=\\sum\_\{u\}p^\{\*\}\(u\)\\delta\_\{i\}\(u\), we have∑up∗​\(u\)​δi′​\(u\)=0\\sum\_\{u\}p^\{\*\}\(u\)\\delta^\{\\prime\}\_\{i\}\(u\)=0\. It remains to check that the shifted versionδi′​\(v\)\\delta^\{\\prime\}\_\{i\}\(v\)still satisfies the assumptions \(up to some constants\)\.

- •Bounded magnitude:\|δi′​\(v\)\|≤\|δi​\(v\)\|\+∑up∗​\(u\)​\|δi​\(v\)\|≤2​ξ\|\\delta^\{\\prime\}\_\{i\}\(v\)\|\\leq\|\\delta\_\{i\}\(v\)\|\+\\sum\_\{u\}p^\{\*\}\(u\)\|\\delta\_\{i\}\(v\)\|\\leq 2\\xi;
- •Zero mean:𝔼​\[δi′​\(v\)\]=𝔼​\[δi​\(v\)\]−∑up∗​\(u\)​𝔼​\[δi​\(u\)\]=0\\mathbb\{E\}\[\\delta^\{\\prime\}\_\{i\}\(v\)\]=\\mathbb\{E\}\[\\delta\_\{i\}\(v\)\]\-\\sum\_\{u\}p^\{\*\}\(u\)\\mathbb\{E\}\[\\delta\_\{i\}\(u\)\]=0;
- •Variance over vocabulary remains unchanged: Varu∼p∗​\(δi′​\(u\)\)=∑up∗​\(u\)​\(δi′​\(u\)\)2=∑up∗​\(u\)​\(δi​\(u\)−∑vp∗​\(v\)​δi​\(v\)\)2=Varu∼p∗​\(δi​\(u\)\)\.\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta^\{\\prime\}\_\{i\}\(u\)\)=\\sum\_\{u\}p^\{\*\}\(u\)\(\\delta^\{\\prime\}\_\{i\}\(u\)\)^\{2\}=\\sum\_\{u\}p^\{\*\}\(u\)\\Bigl\(\\delta\_\{i\}\(u\)\-\\sum\_\{v\}p^\{\*\}\(v\)\\delta\_\{i\}\(v\)\\Bigr\)^\{2\}=\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\.

Therefore, for the remainder of the proof, we assume, without loss of generality, thatδi\\delta\_\{i\}satisfies the centring property:

𝔼u∼p∗​\[δi​\(u\)\]=∑u∈𝒱p∗​\(u\)​δi​\(u\)=0\.\\mathbb\{E\}\_\{u\\sim p^\{\*\}\}\[\\delta\_\{i\}\(u\)\]=\\sum\_\{u\\in\\mathcal\{V\}\}p^\{\*\}\(u\)\\delta\_\{i\}\(u\)=0\.\(7\)Fix av∈𝒱v\\in\\mathcal\{V\}, using triangle inequality, we can write

\|p¯N​\(v\)−p∗​\(v\)\|≤\|p¯N​\(v\)−1N​∑i=1N𝔼​\[pi​\(v\)\]\|\+\|1N​∑i=1N𝔼​\[pi​\(v\)\]−p∗​\(v\)\|\.\\left\|\\bar\{p\}\_\{N\}\(v\)\-p^\{\*\}\(v\)\\right\|\\leq\\left\|\\bar\{p\}\_\{N\}\(v\)\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{E\}\[p\_\{i\}\(v\)\]\\right\|\+\\left\|\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{E\}\[p\_\{i\}\(v\)\]\-p^\{\*\}\(v\)\\right\|\.\(8\)

#### Step 2: Concentration around the mean\.

We first bound the first term in \([8](https://arxiv.org/html/2605.30501#A1.E8)\)\. Note thatpi​\(v\)∈\[0,1\]p\_\{i\}\(v\)\\in\[0,1\]is bounded fori∈ℕi\\in\\mathbb\{N\}\. Asδ1,δ2,…\\delta\_\{1\},\\delta\_\{2\},\\ldotsare independent random vectors according to Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(b\), thenp1​\(v\),p2​\(v\),…p\_\{1\}\(v\),p\_\{2\}\(v\),\\ldotsare also independent and bounded\. By Hoeffding’s inequality\(Hoeffding,[1963](https://arxiv.org/html/2605.30501#bib.bib25)\), we have

P​\(\|p¯N​\(v\)−1N​∑i=1N𝔼​\[pi​\(v\)\]\|≥t\)≤2​exp⁡\(−2​N​t2\)\.P\\left\(\\left\|\\bar\{p\}\_\{N\}\(v\)\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{E\}\[p\_\{i\}\(v\)\]\\right\|\\geq t\\right\)\\leq 2\\exp\\left\(\-2Nt^\{2\}\\right\)\.\(9\)We study the bias term in \([8](https://arxiv.org/html/2605.30501#A1.E8)\) in the next steps\.

#### Step 3: Upper and lower bounds onpi​\(v\)p\_\{i\}\(v\)\.

Given Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(a\),\|δi​\(v\)\|≤ξ≪1\|\\delta\_\{i\}\(v\)\|\\leq\\xi\\ll 1\. We use standard inequalities1\+y≤ey≤1\+y\+y21\+y\\leq e^\{y\}\\leq 1\+y\+y^\{2\}, valid for\|y\|≤1\|y\|\\leq 1\. First, we bound the normalisation quantityZiZ\_\{i\}:

Zi\\displaystyle Z\_\{i\}≥∑up∗​\(u\)​\(1\+δi​\(u\)\)=1\+∑up∗​\(u\)​δi​\(u\)⏟0 by \([7](https://arxiv.org/html/2605.30501#A1.E7)\)=1,and\\displaystyle\\geq\\sum\_\{u\}p^\{\*\}\(u\)\(1\+\\delta\_\{i\}\(u\)\)=1\+\\underbrace\{\\sum\_\{u\}p^\{\*\}\(u\)\\delta\_\{i\}\(u\)\}\_\{\\text\{0 by~\\eqref\{eq:centering\_prop\}\}\}=1,\\quad\\text\{and\}Zi\\displaystyle Z\_\{i\}≤∑up∗​\(u\)​\(1\+δi​\(u\)\+δi2​\(u\)\)=1\+0\+∑up∗​\(u\)​δi2​\(u\)⏟Varu∼p∗​\(δi​\(u\)\)=1\+σi2\.\\displaystyle\\leq\\sum\_\{u\}p^\{\*\}\(u\)\(1\+\\delta\_\{i\}\(u\)\+\\delta\_\{i\}^\{2\}\(u\)\)=1\+0\+\\underbrace\{\\sum\_\{u\}p^\{\*\}\(u\)\\delta\_\{i\}^\{2\}\(u\)\}\_\{\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\}=1\+\\sigma\_\{i\}^\{2\}\.Next, we derive bounds forpi​\(v\)p\_\{i\}\(v\)\. First, we upper bound it by

pi​\(v\)=p∗​\(v\)​eδi​\(v\)Zi≤p∗​\(v\)​\(1\+δi​\(v\)\+δi2​\(v\)\)1=p∗​\(v\)​\(1\+δi​\(v\)\+δi2​\(v\)\)\.p\_\{i\}\(v\)=\\frac\{p^\{\*\}\(v\)e^\{\\delta\_\{i\}\(v\)\}\}\{Z\_\{i\}\}\\leq\\frac\{p^\{\*\}\(v\)\(1\+\\delta\_\{i\}\(v\)\+\\delta\_\{i\}^\{2\}\(v\)\)\}\{1\}=p^\{\*\}\(v\)\\left\(1\+\\delta\_\{i\}\(v\)\+\\delta\_\{i\}^\{2\}\(v\)\\right\)\.Using the inequality11\+x≥1−x\\frac\{1\}\{1\+x\}\\geq 1\-xforx∈ℝx\\in\\mathbb\{R\}, we lower boundpi​\(v\)p\_\{i\}\(v\)by

pi​\(v\)\\displaystyle p\_\{i\}\(v\)≥p∗​\(v\)​\(1\+δi​\(v\)\)1\+Varu∼p∗​\(δi​\(u\)\)≥p∗​\(v\)​\(1\+δi​\(v\)\)​\(1−Varu∼p∗​\(δi​\(u\)\)\)\\displaystyle\\geq\\frac\{p^\{\*\}\(v\)\(1\+\\delta\_\{i\}\(v\)\)\}\{1\+\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\}\\geq p^\{\*\}\(v\)\(1\+\\delta\_\{i\}\(v\)\)\(1\-\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\)=p∗​\(v\)​\(1\+δi​\(v\)−Varu∼p∗​\(δi​\(u\)\)−δi​\(v\)​Varu∼p∗​\(δi​\(u\)\)\)≥p∗​\(v\)​\(1\+δi​\(v\)−2​V​a​ru∼p∗​\(δi​\(u\)\)\)\.\\displaystyle=p^\{\*\}\(v\)\\bigl\(1\+\\delta\_\{i\}\(v\)\-\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\-\\delta\_\{i\}\(v\)\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\\bigr\)\\geq p^\{\*\}\(v\)\\bigl\(1\+\\delta\_\{i\}\(v\)\-2\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\\bigr\)\.Putting things together, we have,

p∗​\(v\)​\(δi​\(v\)−2​V​a​ru∼p∗​\(δi​\(u\)\)\)≤pi​\(v\)−p∗​\(v\)≤p∗​\(v\)​\(δi​\(v\)\+δi2​\(v\)\)\.p^\{\*\}\(v\)\\bigl\(\\delta\_\{i\}\(v\)\-2\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\\bigr\)\\leq p\_\{i\}\(v\)\-p^\{\*\}\(v\)\\leq p^\{\*\}\(v\)\\bigl\(\\delta\_\{i\}\(v\)\+\\delta\_\{i\}^\{2\}\(v\)\\bigr\)\.\(10\)

#### Step 4: Bounds on the bias term\.

Taking the expectation on all sides of \([10](https://arxiv.org/html/2605.30501#A1.E10)\), we have, by Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(c\), that

p∗​\(v\)​𝔼​\[δi​\(v\)\]⏟0−2​p∗​\(v\)​𝔼​\[Varu∼p∗​\(δi​\(u\)\)\]≤𝔼​\[pi​\(v\)\]−p∗​\(v\)≤p∗​\(v\)​𝔼​\[δi​\(v\)\]⏟0\+p∗​\(v\)​𝔼​\[δi2​\(v\)\]\.p^\{\*\}\(v\)\\underbrace\{\\mathbb\{E\}\[\\delta\_\{i\}\(v\)\]\}\_\{0\}\-2p^\{\*\}\(v\)\\mathbb\{E\}\\bigl\[\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\\bigr\]\\leq\\mathbb\{E\}\[p\_\{i\}\(v\)\]\-p^\{\*\}\(v\)\\leq p^\{\*\}\(v\)\\underbrace\{\\mathbb\{E\}\[\\delta\_\{i\}\(v\)\]\}\_\{0\}\+p^\{\*\}\(v\)\\mathbb\{E\}\[\\delta^\{2\}\_\{i\}\(v\)\]\.Using Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(d\), we have

supv∈𝒱p∗​\(v\)​𝔼​\[Varu∼p∗​\(δi​\(u\)\)\]≤η2andsupv∈𝒱p∗​\(v\)​𝔼​\[δi2​\(v\)\]≤𝔼​\[∑up∗​\(u\)​δi2​\(u\)\]=η2\.\\sup\_\{v\\in\\mathcal\{V\}\}\\ p^\{\*\}\(v\)\\mathbb\{E\}\\bigl\[\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\delta\_\{i\}\(u\)\)\\bigr\]\\leq\\eta^\{2\}\\quad\\text\{and\}\\quad\\sup\_\{v\\in\\mathcal\{V\}\}\\ p^\{\*\}\(v\)\\mathbb\{E\}\[\\delta^\{2\}\_\{i\}\(v\)\]\\leq\\mathbb\{E\}\\biggl\[\\sum\_\{u\}p^\{\*\}\(u\)\\delta^\{2\}\_\{i\}\(u\)\\biggr\]=\\eta^\{2\}\.Thus

supv∈𝒱\|𝔼​\[pi​\(v\)\]−p∗​\(v\)\|≤2​η2\.\\sup\_\{v\\in\\mathcal\{V\}\}\\bigl\|\\mathbb\{E\}\[p\_\{i\}\(v\)\]\-p^\{\*\}\(v\)\\bigr\|\\leq 2\\eta^\{2\}\.\(11\)

#### Step 5: Final derivation\.

Combining \([9](https://arxiv.org/html/2605.30501#A1.E9)\) and \([11](https://arxiv.org/html/2605.30501#A1.E11)\), we conclude that for anyδ\>0\\delta\>0, with probability at least1−δ1\-\\delta,

\|p¯N​\(v\)−p∗​\(v\)\|≤2​log⁡\(2/δ\)N\+2​η2\.\\bigl\|\\bar\{p\}\_\{N\}\(v\)\-p^\{\*\}\(v\)\\bigr\|\\leq\\sqrt\{\\frac\{2\\log\(2/\\delta\)\}\{N\}\}\+2\\eta^\{2\}\.Note that this holds for any fixedv∈𝒱v\\in\\mathcal\{V\}\. For theℓ∞\\ell\_\{\\infty\}distance, by a standard union bound argument, with probability at least1−δ1\-\\delta,

∥p¯N\(⋅\|x\)−p∗\(⋅\|x\)∥∞\\displaystyle\\bigl\\\|\\bar\{p\}\_\{N\}\(\\cdot\|x\)\-p^\{\*\}\(\\cdot\|x\)\\bigr\\\|\_\{\\infty\}=supv∈𝒱\|p¯N​\(v\)−p∗​\(v\)\|\\displaystyle=\\sup\_\{v\\in\\mathcal\{V\}\}\\bigl\|\\bar\{p\}\_\{N\}\(v\)\-p^\{\*\}\(v\)\\bigr\|≤supv∈𝒱\|p¯N​\(v\)−1N​∑i=1N𝔼​\[pi​\(v\)\]\|\+supv∈𝒱\|1N​∑i=1N𝔼​\[pi​\(v\)\]−p∗​\(v\)\|\\displaystyle\\leq\\sup\_\{v\\in\\mathcal\{V\}\}\\left\|\\bar\{p\}\_\{N\}\(v\)\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{E\}\[p\_\{i\}\(v\)\]\\right\|\+\\sup\_\{v\\in\\mathcal\{V\}\}\\left\|\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{E\}\[p\_\{i\}\(v\)\]\-p^\{\*\}\(v\)\\right\|≤2​log⁡\(2​\|𝒱\|/δ\)N\+2​η2\.\\displaystyle\\leq\\sqrt\{\\frac\{2\\log\(2\|\\mathcal\{V\}\|/\\delta\)\}\{N\}\}\+2\\eta^\{2\}\.∎

## Appendix BExtension of the Theoretical Result to Grouped Watermarking Settings

The theoretical analysis in Section[2](https://arxiv.org/html/2605.30501#S2)assumes full independence across allNNproviders\. In practice, however, certain providers may share common watermarking toolkits, licensing agreements, or underlying model families, inducing statistical dependence among their perturbation vectors\.

We show here that the convergence guarantee extends naturally to a*grouped*setting in which providers are partitioned into independent clusters, with only conditional independence required within each cluster\. We also relax the unbiasedness assumption by allowing models within a group to share a common perturbation component\.

###### Assumption B\.1\(Grouped Perturbations\)\.

Consider a set ofNNproviders\. Suppose they can be partitioned intoMMgroups:

G1,…,GM,⋃g=1MGg=\{1,…,N\},Gg∩Gg′=∅​for​g≠g′,G\_\{1\},\\ldots,G\_\{M\},\\quad\\bigcup\_\{g=1\}^\{M\}G\_\{g\}=\\\{1,\\ldots,N\\\},\\quad G\_\{g\}\\cap G\_\{g^\{\\prime\}\}=\\emptyset\\text\{ for \}g\\neq g^\{\\prime\},with∑g=1M\|Gg\|=N\\sum\_\{g=1\}^\{M\}\|G\_\{g\}\|=N\. For each groupgg, letWgW\_\{g\}be a group\-level latent variable\. Each provider is associated with a random perturbation vectorδi​\(⋅,x\)\\delta\_\{i\}\(\\cdot,x\)that modulates the output distribution\. For every contextx∈𝒳x\\in\\mathcal\{X\}, we assume the following properties hold for\{δi​\(⋅,x\)\}i=1N\\\{\\delta\_\{i\}\(\\cdot,x\)\\\}\_\{i=1\}^\{N\}:

\(a\) Bounded Magnitude:The perturbation magnitude is uniformly bounded by a constantξ≤1\\xi\\leq 1\. Specifically,‖δi​\(⋅,x\)‖∞≤ξ\\\|\\delta\_\{i\}\(\\cdot,x\)\\\|\_\{\\infty\}\\leq\\xifor all providersii\.

\(b\) Group independence structure:*Across groups*, the collections\(\{δi​\(⋅,x\)\}i∈Gg,Wg\)g=1,…,M\\bigl\(\\\{\\delta\_\{i\}\(\\cdot,x\)\\\}\_\{i\\in G\_\{g\}\},W\_\{g\}\\bigr\)\_\{g=1,\\ldots,M\}are mutually independent;*within each group*, conditional onWgW\_\{g\}, the perturbations\{δi​\(⋅,x\)\}i∈Gg\\\{\\delta\_\{i\}\(\\cdot,x\)\\\}\_\{i\\in G\_\{g\}\}are mutually independent\. Specifically, these two assumptions imply

ℙ​\(⋂i=1N\{δi​\(⋅,x\)∈Ai\}\|W1,…,WM\)=∏g=1M∏i∈Ggℙ​\(δi​\(⋅,x\)∈Ai∣Wg\)\\mathbb\{P\}\\biggl\(\\bigcap\_\{i=1\}^\{N\}\\\{\\delta\_\{i\}\(\\cdot,x\)\\in A\_\{i\}\\\}\\biggm\|W\_\{1\},\\ldots,W\_\{M\}\\biggr\)=\\prod\_\{g=1\}^\{M\}\\prod\_\{i\\in G\_\{g\}\}\\mathbb\{P\}\\bigl\(\\delta\_\{i\}\(\\cdot,x\)\\in A\_\{i\}\\mid W\_\{g\}\\bigr\)for any measurable setsA1,…,ANA\_\{1\},\\ldots,A\_\{N\}\.

\(c\) Group\-specific Bias:For eachg∈\{1,…,M\}g\\in\\\{1,\\ldots,M\\\}andi∈Ggi\\in G\_\{g\},

𝔼​\[δi​\(⋅,x\)∣Wg\]=bg​\(⋅,x\),\\mathbb\{E\}\[\\delta\_\{i\}\(\\cdot,x\)\\mid W\_\{g\}\]=b\_\{g\}\(\\cdot,x\),wherebg​\(⋅,x\)b\_\{g\}\(\\cdot,x\)is a group\-specific bias function\.

\(d\) Conditionally Bounded Expected Variance of the Idiosyncratic Perturbation:For eachg∈\{1,…,M\}g\\in\\\{1,\\ldots,M\\\}andi∈Ggi\\in G\_\{g\}, we decompose the perturbation into a group\-specific component and an idiosyncratic component as

δi​\(⋅,x\)=bg​\(⋅,x\)\+εi​\(⋅,x\),\\delta\_\{i\}\(\\cdot,x\)=b\_\{g\}\(\\cdot,x\)\+\\varepsilon\_\{i\}\(\\cdot,x\),wherebg​\(⋅,x\)b\_\{g\}\(\\cdot,x\)is defined above and shared by all providers within groupGgG\_\{g\}, andεi​\(⋅,x\)\\varepsilon\_\{i\}\(\\cdot,x\)denotes the idiosyncratic perturbation\. The conditional expectation of the weighted variation of the idiosyncratic perturbations across the vocabulary \(weighted by the consensus probability\) is bounded by a constantη2\\eta^\{2\}\. That is,

𝔼​\[Varu∼p∗​\(εi​\(u,x\)\)∣Wg\]≤η2\\mathbb\{E\}\\bigl\[\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\\bigl\(\\varepsilon\_\{i\}\(u,x\)\\bigr\)\\mid W\_\{g\}\\bigr\]\\leq\\eta^\{2\}where

Varu∼p∗​\(εi​\(u,x\)\):=∑up∗​\(u\|x\)​\(εi​\(u,x\)−∑vp∗​\(v\|x\)​εi​\(v,x\)\)2\.\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\varepsilon\_\{i\}\(u,x\)\):=\\sum\_\{u\}p^\{\*\}\(u\|x\)\\Bigl\(\\varepsilon\_\{i\}\(u,x\)\-\\sum\_\{v\}p^\{\*\}\(v\|x\)\\varepsilon\_\{i\}\(v,x\)\\Bigr\)^\{2\}\.

Define the group consensus distribution for groupggas

pg†​\(v\|x\):=p∗​\(v\|x\)​exp⁡\(bg​\(v,x\)\)∑u∈𝒱p∗​\(u\|x\)​exp⁡\(bg​\(u,x\)\),p\_\{g\}^\{\\dagger\}\(v\|x\):=\\frac\{p^\{\*\}\(v\|x\)\\exp\(b\_\{g\}\(v,x\)\)\}\{\\sum\_\{u\\in\\mathcal\{V\}\}p^\{\*\}\(u\|x\)\\exp\(b\_\{g\}\(u,x\)\)\},\(12\)and the group\-size\-weighted averagep¯†\(⋅\|x\):=1N∑g=1Mngpg†\(⋅\|x\)\\bar\{p\}^\{\\dagger\}\(\\cdot\|x\):=\\frac\{1\}\{N\}\\sum\_\{g=1\}^\{M\}n\_\{g\}p\_\{g\}^\{\\dagger\}\(\\cdot\|x\)\. The irreducible group bias is defined as

B\(x\):=∥p¯†\(⋅\|x\)−p∗\(⋅\|x\)∥∞=supv∈𝒱\|p¯†\(v\|x\)−p∗\(v\|x\)\|\.B\(x\):=\\left\\\|\\bar\{p\}^\{\\dagger\}\(\\cdot\|x\)\-p^\{\*\}\(\\cdot\|x\)\\right\\\|\_\{\\infty\}=\\sup\_\{v\\in\\mathcal\{V\}\}\\left\|\\bar\{p\}^\{\\dagger\}\(v\|x\)\-p^\{\*\}\(v\|x\)\\right\|\.\(13\)
###### Theorem B\.2\.

Under Assumption[B\.1](https://arxiv.org/html/2605.30501#A2.Thmtheorem1), for any fixed contextxx, letp¯N\(⋅\|x\)=1N∑i=1Npi\(⋅\|x\)\\bar\{p\}\_\{N\}\(\\cdot\|x\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}p\_\{i\}\(\\cdot\|x\)be the aggregated distribution\. Then, for anyδ\>0\\delta\>0, with probability at least1−δ1\-\\delta,

∥p¯N\(⋅∣x\)−p∗\(⋅∣x\)∥∞≲log⁡\(\|𝒱\|/δ\)N\+η2\+B\(x\)\.\\bigl\\\|\\bar\{p\}\_\{N\}\(\\cdot\\mid x\)\-p^\{\*\}\(\\cdot\\mid x\)\\bigr\\\|\_\{\\infty\}\\lesssim\\sqrt\{\\frac\{\\log\(\|\\mathcal\{V\}\|/\\delta\)\}\{N\}\}\+\\eta^\{2\}\+B\(x\)\.

###### Proof\.

For brevity, we again omit the dependency onxxin the notation throughout this proof\. Recall that the perturbed distribution for modeliiis given by

pi​\(v\)=p∗​\(v\)​exp⁡\(δi​\(v\)\)∑u∈𝒱p∗​\(u\)​exp⁡\(δi​\(u\)\)=p∗​\(v\)​exp⁡\(bg​\(v\)\+εi​\(v\)\)∑u∈𝒱p∗​\(u\)​exp⁡\(bg​\(u\)\+εi​\(u\)\)=pg†​\(v\)​exp⁡\(εi​\(v\)\)∑upg†​\(u\)​exp⁡\(εi​\(u\)\),p\_\{i\}\(v\)=\\frac\{p^\{\*\}\(v\)\\exp\(\\delta\_\{i\}\(v\)\)\}\{\\sum\_\{u\\in\\mathcal\{V\}\}p^\{\*\}\(u\)\\exp\(\\delta\_\{i\}\(u\)\)\}=\\frac\{p^\{\*\}\(v\)\\exp\(b\_\{g\}\(v\)\+\\varepsilon\_\{i\}\(v\)\)\}\{\\sum\_\{u\\in\\mathcal\{V\}\}p^\{\*\}\(u\)\\exp\(b\_\{g\}\(u\)\+\\varepsilon\_\{i\}\(u\)\)\}=\\frac\{p\_\{g\}^\{\\dagger\}\(v\)\\exp\(\\varepsilon\_\{i\}\(v\)\)\}\{\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)\\exp\(\\varepsilon\_\{i\}\(u\)\)\},\(14\)wherepg†​\(⋅\)p\_\{g\}^\{\\dagger\}\(\\cdot\)is defined in \([12](https://arxiv.org/html/2605.30501#A2.E12)\)\. Denote

μW​\(v\):=1N​∑g=1M∑i∈Gg𝔼​\[pi​\(v\)∣Wg\]\.\\mu\_\{W\}\(v\):=\\frac\{1\}\{N\}\\sum\_\{g=1\}^\{M\}\\sum\_\{i\\in G\_\{g\}\}\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\.
By the triangle inequality,

‖p¯N−p∗‖∞≤‖p¯N−μW‖∞⏟Concentration\+‖μW−p¯†‖∞⏟Second\-order idiosyncratic effect\+‖p¯†−p∗‖∞⏟group biasB​\(x\)\.\\\|\\bar\{p\}\_\{N\}\-p^\{\*\}\\\|\_\{\\infty\}\\leq\\underbrace\{\\\|\\bar\{p\}\_\{N\}\-\\mu\_\{W\}\\\|\_\{\\infty\}\}\_\{\\text\{Concentration\}\}\+\\underbrace\{\\\|\\mu\_\{W\}\-\\bar\{p\}^\{\\dagger\}\\\|\_\{\\infty\}\}\_\{\\text\{Second\-order idiosyncratic effect\}\}\+\\;\\underbrace\{\\\|\\bar\{p\}^\{\\dagger\}\-p^\{\*\}\\\|\_\{\\infty\}\}\_\{\\text\{group bias $B\(x\)$\}\}\.\(15\)The third term equalsB​\(x\)B\(x\)by definition \([13](https://arxiv.org/html/2605.30501#A2.E13)\)\.

#### Step 1: Concentration around the mean\.

We now bound the first term\. By Assumption[B\.1](https://arxiv.org/html/2605.30501#A2.Thmtheorem1)\(b\), fori∈Ggi\\in G\_\{g\}, the conditional law ofδi\\delta\_\{i\}givenWWdepends only onWgW\_\{g\}\. Thus, we have

𝔼​\[pi​\(v\)∣W\]=𝔼​\[pi​\(v\)∣Wg\]andμW​\(v\)=1N​∑i=1N𝔼​\[pi​\(v\)∣W\]\.\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\]=\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\\qquad\\text\{and\}\\qquad\\mu\_\{W\}\(v\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\]\.In addition, conditional onW=\(W1,…,WM\)W=\(W\_\{1\},\\ldots,W\_\{M\}\), the perturbationsδ1,…,δN\\delta\_\{1\},\\ldots,\\delta\_\{N\}are mutually independent\. Sincepi​\(v\)p\_\{i\}\(v\)is a measurable function ofδi\\delta\_\{i\}, the random variablesp1​\(v\),…,pN​\(v\)p\_\{1\}\(v\),\\ldots,p\_\{N\}\(v\)are also mutually independent conditional onWW\. Aspi​\(v\)∈\[0,1\]p\_\{i\}\(v\)\\in\[0,1\]is bounded fori∈ℕi\\in\\mathbb\{N\}\. By Hoeffding’s inequality\(Hoeffding,[1963](https://arxiv.org/html/2605.30501#bib.bib25)\), we have

ℙ​\(\|p¯N​\(v\)−μW​\(v\)\|≥t∣W\)≤2​exp⁡\(−2​N​t2\)\.\\mathbb\{P\}\\bigl\(\|\\bar\{p\}\_\{N\}\(v\)\-\\mu\_\{W\}\(v\)\|\\geq t\\mid W\\bigr\)\\leq 2\\exp\(\-2Nt^\{2\}\)\.Taking expectation overWW, the same inequality holds unconditionally\. A union bound overv∈𝒱v\\in\\mathcal\{V\}then yields: with probability at least1−δ1\-\\delta,

‖p¯N−μW‖∞≤log⁡\(\|𝒱\|/δ\)2​N\.\\\|\\bar\{p\}\_\{N\}\-\\mu\_\{W\}\\\|\_\{\\infty\}\\leq\\sqrt\{\\frac\{\\log\(\|\\mathcal\{V\}\|/\\delta\)\}\{2N\}\}\.\(16\)

#### Step 2: Shift invariance and centering\.

Denoteε~i​\(v\):=εi​\(v\)−∑upg†​\(u\)​εi​\(u\)\\widetilde\{\\varepsilon\}\_\{i\}\(v\):=\\varepsilon\_\{i\}\(v\)\-\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)\\varepsilon\_\{i\}\(u\), which ensures

∑upg†​\(u\)​ε~i​\(u\)=0\.\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)\\widetilde\{\\varepsilon\}\_\{i\}\(u\)=0\.\(17\)By shift invariance of the softmax function, we have

pi​\(v\)=pg†​\(v\)​exp⁡\(ε~i​\(v\)\)∑upg†​\(u\)​exp⁡\(ε~i​\(u\)\)\.p\_\{i\}\(v\)=\\frac\{p\_\{g\}^\{\\dagger\}\(v\)\\exp\(\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\)\}\{\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)\\exp\(\\widetilde\{\\varepsilon\}\_\{i\}\(u\)\)\}\.By Assumption[B\.1](https://arxiv.org/html/2605.30501#A2.Thmtheorem1)\(c\) and the definition of the group consensus \([12](https://arxiv.org/html/2605.30501#A2.E12)\), we have thatpg†p\_\{g\}^\{\\dagger\}isWgW\_\{g\}\-measurable and thus, for everyv∈𝒱v\\in\\mathcal\{V\}

𝔼​\[ε~i​\(v\)∣Wg\]\\displaystyle\\mathbb\{E\}\[\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\\mid W\_\{g\}\]=𝔼​\[εi​\(v\)∣Wg\]−∑u∈𝒱𝔼​\[pg†​\(u\)​εi​\(u\)∣Wg\]\\displaystyle=\\mathbb\{E\}\[\{\\varepsilon\}\_\{i\}\(v\)\\mid W\_\{g\}\]\-\\sum\_\{u\\in\\mathcal\{V\}\}\\mathbb\{E\}\[p\_\{g\}^\{\\dagger\}\(u\)\\varepsilon\_\{i\}\(u\)\\mid W\_\{g\}\]=𝔼​\[εi​\(v\)∣Wg\]⏟0−∑u∈𝒱pg†​\(u\)​𝔼​\[εi​\(u\)∣Wg\]⏟0=0\.\\displaystyle=\\underbrace\{\\mathbb\{E\}\[\{\\varepsilon\}\_\{i\}\(v\)\\mid W\_\{g\}\]\}\_\{0\}\-\\sum\_\{u\\in\\mathcal\{V\}\}p\_\{g\}^\{\\dagger\}\(u\)\\underbrace\{\\mathbb\{E\}\[\\varepsilon\_\{i\}\(u\)\\mid W\_\{g\}\]\}\_\{0\}=0\.

#### Step 3: Bounding the second\-order idiosyncratic effect term\.

We write

∥μW−p¯†∥∞=supv∈𝒱\|1N∑g=1M∑i∈Gg\(𝔼\[pi\(v\)∣Wg\]−pg†\(v\)\)\|≤1N∑g=1M∑i∈Ggsupv∈𝒱\|𝔼\[pi\(v\)∣Wg\]−pg†\(v\)\|\\\|\\mu\_\{W\}\-\\bar\{p\}^\{\\dagger\}\\\|\_\{\\infty\}=\\sup\_\{v\\in\\mathcal\{V\}\}\\left\|\\frac\{1\}\{N\}\\sum\_\{g=1\}^\{M\}\\sum\_\{i\\in G\_\{g\}\}\\left\(\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\-p\_\{g\}^\{\\dagger\}\(v\)\\right\)\\right\|\\leq\\frac\{1\}\{N\}\\sum\_\{g=1\}^\{M\}\\sum\_\{i\\in G\_\{g\}\}\\sup\_\{v\\in\\mathcal\{V\}\}\\bigl\|\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\-p\_\{g\}^\{\\dagger\}\(v\)\\bigr\|
LetRi​\(v\):=eε~i​\(v\)−1−ε~i​\(v\)R\_\{i\}\(v\):=e^\{\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\-1\-\\widetilde\{\\varepsilon\}\_\{i\}\(v\)andAi:=∑upg†​\(u\)​Ri​\(u\)A\_\{i\}:=\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)R\_\{i\}\(u\)\. Then, using \([17](https://arxiv.org/html/2605.30501#A2.E17)\), we have

pi​\(v\)−pg†​\(v\)=pg†​\(v\)​\(1\+ε~i​\(v\)\+Ri​\(v\)\)1\+Ai−pg†​\(v\)=pg†​\(v\)​ε~i​\(v\)\+pg†​\(v\)​Ri​\(v\)−Ai−Ai​ε~i​\(v\)1\+Ai\.p\_\{i\}\(v\)\-p\_\{g\}^\{\\dagger\}\(v\)=\\frac\{p\_\{g\}^\{\\dagger\}\(v\)\(1\+\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\+R\_\{i\}\(v\)\)\}\{1\+A\_\{i\}\}\-p\_\{g\}^\{\\dagger\}\(v\)=p\_\{g\}^\{\\dagger\}\(v\)\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\+p\_\{g\}^\{\\dagger\}\(v\)\\frac\{R\_\{i\}\(v\)\-A\_\{i\}\-A\_\{i\}\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\{1\+A\_\{i\}\}\.By the result of Step 2,

𝔼​\[pi​\(v\)∣Wg\]−pg†​\(v\)=pg†​\(v\)​𝔼​\[ε~i​\(v\)∣Wg\]⏟=0\+𝔼​\[pg†​\(v\)​Ri​\(v\)−Ai−Ai​ε~i​\(v\)1\+Ai\|Wg\]\.\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\-p\_\{g\}^\{\\dagger\}\(v\)=\\underbrace\{p\_\{g\}^\{\\dagger\}\(v\)\\mathbb\{E\}\[\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\\mid W\_\{g\}\]\}\_\{=0\}\+\\mathbb\{E\}\\biggl\[p\_\{g\}^\{\\dagger\}\(v\)\\frac\{R\_\{i\}\(v\)\-A\_\{i\}\-A\_\{i\}\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\{1\+A\_\{i\}\}\\biggm\|W\_\{g\}\\biggr\]\.
SinceRi​\(v\)=eε~i​\(v\)−1−ε~i​\(v\)≥0R\_\{i\}\(v\)=e^\{\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\-1\-\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\\geq 0for allvv, we haveAi=∑upg†​\(u\)​Ri​\(u\)≥0A\_\{i\}=\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)R\_\{i\}\(u\)\\geq 0and thus

pg†​\(v\)​\|Ri​\(v\)−Ai−Ai​ε~i​\(v\)1\+Ai\|≤pg†​\(v\)​\(\|Ri​\(v\)\|\+\|Ai\|\+\|Ai​ε~i​\(v\)\|\)\.p\_\{g\}^\{\\dagger\}\(v\)\\left\|\\frac\{R\_\{i\}\(v\)\-A\_\{i\}\-A\_\{i\}\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\{1\+A\_\{i\}\}\\right\|\\leq p\_\{g\}^\{\\dagger\}\(v\)\\left\(\|R\_\{i\}\(v\)\|\+\|A\_\{i\}\|\+\|A\_\{i\}\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\|\\right\)\.
By Assumption[B\.1](https://arxiv.org/html/2605.30501#A2.Thmtheorem1)\(a\),‖δi‖∞≤ξ\\\|\\delta\_\{i\}\\\|\_\{\\infty\}\\leq\\xi\. Asεi​\(v\)=δi​\(v\)−𝔼​\[δi​\(v\)\|Wg\]\\varepsilon\_\{i\}\(v\)=\\delta\_\{i\}\(v\)\-\\mathbb\{E\}\[\\delta\_\{i\}\(v\)\|W\_\{g\}\], we have‖εi‖∞≤2​ξ\\\|\\varepsilon\_\{i\}\\\|\_\{\\infty\}\\leq 2\\xi\. It follows that‖ε~i‖∞≤4​ξ\\\|\\widetilde\{\\varepsilon\}\_\{i\}\\\|\_\{\\infty\}\\leq 4\\xi\. Then we have\|Ri​\(v\)\|=\|eε~i​\(v\)−1−ε~i​\(v\)\|≤Cξ​ε~i​\(v\)2\|R\_\{i\}\(v\)\|=\|e^\{\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\-1\-\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\|\\leq C\_\{\\xi\}\\widetilde\{\\varepsilon\}\_\{i\}\(v\)^\{2\}for allv∈𝒱v\\in\\mathcal\{V\}, whereCξC\_\{\\xi\}is a constant depending only onξ\\xi, and hence\|Ai\|=\|∑upg†​\(u\)​Ri​\(u\)\|≤∑upg†​\(u\)​\|Ri​\(u\)\|≤Cξ​∑upg†​\(u\)​ε~i​\(u\)2=Cξ​Varu∼pg†​\(εi​\(u\)\)\|A\_\{i\}\|=\\left\|\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)R\_\{i\}\(u\)\\right\|\\leq\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)\|R\_\{i\}\(u\)\|\\leq C\_\{\\xi\}\\sum\_\{u\}p\_\{g\}^\{\\dagger\}\(u\)\\widetilde\{\\varepsilon\}\_\{i\}\(u\)^\{2\}=C\_\{\\xi\}\\mathrm\{Var\}\_\{u\\sim p\_\{g\}^\{\\dagger\}\}\(\\varepsilon\_\{i\}\(u\)\)\. Using these bounds, we obtain

pg†​\(v\)​\|Ri​\(v\)−Ai−Ai​ε~i​\(v\)1\+Ai\|≤Cξ′​Varu∼pg†​\(εi​\(u\)\)\.p\_\{g\}^\{\\dagger\}\(v\)\\left\|\\frac\{R\_\{i\}\(v\)\-A\_\{i\}\-A\_\{i\}\\widetilde\{\\varepsilon\}\_\{i\}\(v\)\}\{1\+A\_\{i\}\}\\right\|\\leq C\_\{\\xi\}^\{\\prime\}\\mathrm\{Var\}\_\{u\\sim p\_\{g\}^\{\\dagger\}\}\(\\varepsilon\_\{i\}\(u\)\)\.
and hence

supv∈𝒱\|𝔼\[pi\(v\)∣Wg\]−pg†\(v\)\|≤Cξ′𝔼\[Varu∼pg†\(εi\(u\)\)∣Wg\]\.\\sup\_\{v\\in\\mathcal\{V\}\}\\left\|\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\-p\_\{g\}^\{\\dagger\}\(v\)\\right\|\\leq C\_\{\\xi\}^\{\\prime\}\\mathbb\{E\}\\left\[\\mathrm\{Var\}\_\{u\\sim p\_\{g\}^\{\\dagger\}\}\(\\varepsilon\_\{i\}\(u\)\)\\mid W\_\{g\}\\right\]\.\(18\)
We next relateVaru∼pg†​\(εi​\(u\)\)\\mathrm\{Var\}\_\{u\\sim p\_\{g\}^\{\\dagger\}\}\(\\varepsilon\_\{i\}\(u\)\)toVaru∼p∗​\(εi​\(u\)\)\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\varepsilon\_\{i\}\(u\)\)\. Since\|bg​\(u\)\|≤ξ\|b\_\{g\}\(u\)\|\\leq\\xi, on the support ofp∗p^\{\*\}we have

pg†​\(u\)p∗​\(u\)=exp⁡\(bg​\(u\)\)∑z∈𝒱p∗​\(z\)​exp⁡\(bg​\(z\)\)≤eξe−ξ=e2​ξ\.\\frac\{p\_\{g\}^\{\\dagger\}\(u\)\}\{p^\{\*\}\(u\)\}=\\frac\{\\exp\(b\_\{g\}\(u\)\)\}\{\\sum\_\{z\\in\\mathcal\{V\}\}p^\{\*\}\(z\)\\exp\(b\_\{g\}\(z\)\)\}\\leq\\frac\{e^\{\\xi\}\}\{e^\{\-\\xi\}\}=e^\{2\\xi\}\.Therefore,

Varu∼pg†​\(εi​\(u\)\)\\displaystyle\\mathrm\{Var\}\_\{u\\sim p\_\{g\}^\{\\dagger\}\}\(\\varepsilon\_\{i\}\(u\)\)=infa∈ℝ\{∑u∈𝒱pg†​\(u\)​\(εi​\(u\)−a\)2\}≤∑u∈𝒱pg†​\(u\)​\(εi​\(u\)−∑z∈𝒱p∗​\(z\)​εi​\(z\)\)2\\displaystyle=\\inf\_\{a\\in\\mathbb\{R\}\}\\biggl\\\{\\sum\_\{u\\in\\mathcal\{V\}\}p\_\{g\}^\{\\dagger\}\(u\)\(\\varepsilon\_\{i\}\(u\)\-a\)^\{2\}\\biggr\\\}\\leq\\sum\_\{u\\in\\mathcal\{V\}\}p\_\{g\}^\{\\dagger\}\(u\)\\left\(\\varepsilon\_\{i\}\(u\)\-\\sum\_\{z\\in\\mathcal\{V\}\}p^\{\*\}\(z\)\\varepsilon\_\{i\}\(z\)\\right\)^\{2\}≤e2​ξ​∑u∈𝒱p∗​\(u\)​\(εi​\(u\)−∑z∈𝒱p∗​\(z\)​εi​\(z\)\)2=e2​ξ​Varu∼p∗​\(εi​\(u\)\)\.\\displaystyle\\leq e^\{2\\xi\}\\sum\_\{u\\in\\mathcal\{V\}\}p^\{\*\}\(u\)\\left\(\\varepsilon\_\{i\}\(u\)\-\\sum\_\{z\\in\\mathcal\{V\}\}p^\{\*\}\(z\)\\varepsilon\_\{i\}\(z\)\\right\)^\{2\}=e^\{2\\xi\}\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\varepsilon\_\{i\}\(u\)\)\.Taking conditional expectations and using Assumption[B\.1](https://arxiv.org/html/2605.30501#A2.Thmtheorem1)\(d\), we obtain

𝔼\[Varu∼pg†\(εi\(u\)\)\|Wg\]≤e2​ξ𝔼\[Varu∼p∗\(εi\(u\)\)\|Wg\]≤e2​ξη2\.\\mathbb\{E\}\\left\[\\mathrm\{Var\}\_\{u\\sim p\_\{g\}^\{\\dagger\}\}\(\\varepsilon\_\{i\}\(u\)\)\\middle\|W\_\{g\}\\right\]\\leq e^\{2\\xi\}\\mathbb\{E\}\\left\[\\mathrm\{Var\}\_\{u\\sim p^\{\*\}\}\(\\varepsilon\_\{i\}\(u\)\)\\middle\|W\_\{g\}\\right\]\\leq e^\{2\\xi\}\\eta^\{2\}\.Substituting this into \([18](https://arxiv.org/html/2605.30501#A2.E18)\), we then havesupv∈𝒱\|𝔼\[pi\(v\)∣Wg\]−pg†\(v\)\|≤Cξ′e2​ξη2\\sup\_\{v\\in\\mathcal\{V\}\}\\left\|\\mathbb\{E\}\[p\_\{i\}\(v\)\\mid W\_\{g\}\]\-p\_\{g\}^\{\\dagger\}\(v\)\\right\|\\leq C\_\{\\xi\}^\{\\prime\}e^\{2\\xi\}\\eta^\{2\}\. Averaging over all providers gives

‖μW−p¯†‖∞≤Cξ′​e2​ξ​η2\.\\\|\\mu\_\{W\}\-\\bar\{p\}^\{\\dagger\}\\\|\_\{\\infty\}\\leq C\_\{\\xi\}^\{\\prime\}e^\{2\\xi\}\\eta^\{2\}\.\(19\)

#### Step 4: Final derivation\.

Substituting \([16](https://arxiv.org/html/2605.30501#A2.E16)\), \([19](https://arxiv.org/html/2605.30501#A2.E19)\), and the definition ofB​\(x\)B\(x\)into \([15](https://arxiv.org/html/2605.30501#A2.E15)\), we conclude that with probability at least1−δ1\-\\delta,

‖p¯N−p∗‖∞≤log⁡\(\|𝒱\|/δ\)2​N\+Cξ′​e2​ξ​η2\+B​\(x\)≲log⁡\(\|𝒱\|/δ\)N\+η2\+B​\(x\)\.\\\|\\bar\{p\}\_\{N\}\-p^\{\*\}\\\|\_\{\\infty\}\\leq\\sqrt\{\\frac\{\\log\(\|\\mathcal\{V\}\|/\\delta\)\}\{2N\}\}\+C^\{\\prime\}\_\{\\xi\}e^\{2\\xi\}\\eta^\{2\}\+B\(x\)\\lesssim\\sqrt\{\\frac\{\\log\(\|\\mathcal\{V\}\|/\\delta\)\}\{N\}\}\+\\eta^\{2\}\+B\(x\)\.
∎

## Appendix CRobustness to Biased Watermark Perturbations

Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(c\) assumes watermark perturbations as zero\-mean around the consensus distribution\. We stress\-test this condition by deploying a biased green\-red\-list watermark that consistently promotes a fixed set of tokens with a bias\. We evaluate the watermarked baseline andWASHwith both the accuracy on the GSM8K task, and the watermark detection z\-score\.

Table[8](https://arxiv.org/html/2605.30501#A3.T8)shows a utility\-detection trade\-off: stronger bias makes the baseline easier to detect, but rapidly hurts task performance\. This further validates the practical plausibility of Assumption[2\.2](https://arxiv.org/html/2605.30501#S2.Thmtheorem2)\(c\): rational providers have no incentive to deploy systematically biased perturbations, as the severe accuracy degradation \(from 0\.443 to 0\.023\) demonstrates that deviating from the zero\-mean condition directly harms generation quality, undermining their own service\. WASH remains effective across all bias values, keeping detector z\-scores below the no\-detection threshold \(z≤4z\\leq 4\) and improving accuracy over the corresponding biased baseline\.

This is consistent with the theoretical prediction in Appendix[B](https://arxiv.org/html/2605.30501#A2)when perturbations carry a shared bias, an irreducible termB​\(x\)B\(x\)persists in the convergence bound, explaining why the detection z\-scores under WASH do not vanish completely but remain below the detection threshold\.

Table 8:Robustness to biased watermark perturbations\.BiasWatermarked Accuracy↑\\uparrowWatermarked Detectionzz↓\\downarrowWASH Accuracy↑\\uparrowWASH Detectionzz↓\\downarrow2\.00\.4433\.710\.5631\.354\.00\.13711\.490\.5071\.926\.00\.01715\.550\.4032\.658\.00\.01015\.670\.2532\.9310\.00\.02315\.760\.2202\.95

## Appendix DExperiment Details

### D\.1Implementation Details

#### Detection thresholds\.

Forgeneration\-time attacks, we follow the z\-score protocol ofLiuet al\.\([2025](https://arxiv.org/html/2605.30501#bib.bib15)\)to quantify watermark signal strength\. Following the original definition, Detection confidence is categorised as high\-confidence identification \(z\>10z\>10\), low\-confidence identification \(4<z≤104<z\\leq 10\), and no detection \(z≤4z\\leq 4\)\.

Forfinal\-text rewrite attacks, perturbation detection on the small set of generated tokens is no longer compatible, so we additionally use native sequence detectors\(Panet al\.,[2024](https://arxiv.org/html/2605.30501#bib.bib37)\)\. FollowingLiuet al\.\([2025](https://arxiv.org/html/2605.30501#bib.bib15)\), we generate the sequence to be detected on the C4 dataset\(Raffelet al\.,[2020](https://arxiv.org/html/2605.30501#bib.bib26)\): we truncate each sample to 30 tokens as the prompt, generate 200 additional tokens with the watermarked model, and then perform detection on the generated sequence\. Since z\-score magnitudes vary substantially across watermarking schemes, directly comparing z\-scores across schemes is not meaningful\. We therefore report the removal effect using TPR@5% FPR, the true\-positive rate of identifying a watermarked sequence at a threshold calibrated to falsely flag 5% of unwatermarked sequences, which is widely used as a watermark robustness measurement due to its consistency across watermark types, text types, and lengths\(Kirchenbaueret al\.,[2023b](https://arxiv.org/html/2605.30501#bib.bib38)\)\. Lower TPR@5% FPR indicates stronger removal\. We categorise a strong watermark signal for TPR@5% FPR≥75%\\geq 75\\%, while a low\-confidence detection with50≤50\\leqTPR@5% FPR≤75%\\leq 75\\%\.

### D\.2Detailed Experiment Results

Table[9](https://arxiv.org/html/2605.30501#A4.T9)shows the detailed experiment results for Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(a\) and \(b\), the watermark detection evaluation with mixing diverse watermark schemes and base models\. For the fixed\-base model setting, we randomly sampled 15 ensemble combinations for each mixture amountNNand ran the detection task 5 times each to obtain stable z\-scores\. For the mixed\-base model setting, we experimented with a larger range ofNNfrom 1 to 8, and sampled 50 mixture combinations for eachNNdue to the large sampling space\. The results follow the scaling law: with a larger ensemble size, the watermark signal decays more\.

For the fixed\-base model setting \(Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(a\)\), the detection signal diminishes rapidly asNNincreases, with the Llama3\.1\-8B model dropping from an extremely high z\-score \(≈150\\approx 150\) to≈10\\approx 10atN=5N=5\. This empirical decay mirrors our theoretical prediction ofO​\(1/N\)O\(1/\\sqrt\{N\}\)\. However, we observe that the signal does not vanish as completely as in the heterogeneous setting\. This residual signal likely arises because the shared systematic bias \(δs​y​s\\delta\_\{sys\}\) of the identical base model persists across the ensemble, hindering the complete cancellation of artefacts\. More importantly, this single\-model scenario is rarely available in practice, as providers seldom expose multiple watermarked versions of the same model to end users\.

Table 9:Experiment results with a mixture of watermarks and base models for Figure[3](https://arxiv.org/html/2605.30501#S3.F3)\(a\) and \(b\)\.indicates high\-confidence watermark identification, andindicates low\-confidence watermark identification, while no colour indicates no watermark identified\.Base ModelMixture AmountMean Z\-scoreStd Z\-scoreLower BoundUpper BoundLlama3\.1\-8B1147\.352107\.82192\.787201\.917227\.41724\.89014\.81640\.018320\.51118\.85510\.96930\.053415\.89015\.3708\.11123\.66859\.3448\.9354\.82213\.865Qwen3\-8B124\.30427\.52510\.37438\.23326\.9346\.3053\.74410\.12533\.3084\.2661\.1505\.46741\.6502\.8620\.2013\.09851\.7161\.2841\.0662\.366Ministral3\-8B135\.72444\.90113\.00158\.447212\.49114\.2705\.26919\.71333\.2413\.7581\.3395\.14341\.2012\.1430\.1172\.28650\.4630\.965\-0\.0240\.951Mixed Model159\.90976\.43438\.72281\.095214\.95615\.02010\.79319\.12033\.8667\.4031\.8155\.91841\.8785\.5100\.3503\.40550\.0873\.666\-0\.9291\.1036\-0\.0662\.427\-0\.7390\.6067\-0\.3442\.174\-0\.9470\.2598\-0\.3422\.669\-1\.0820\.398

## Appendix EFluency\-Aware Routing Example

Figure[4](https://arxiv.org/html/2605.30501#A5.F4)shows two complete generation flows for a text completion task in the C4 dataset and a reasoning task on the MMLU Law subset, including the Fluency\-Aware Routing triggered in between\. These generations are conducted by a mixture of three watermarked models: Llama\-Aar, Ministral\-DIPMark, and Qwen\-KGW\.

The routing mechanism is triggered mostly on words that are tokenised differently across models\. For example, in the MMLU Law task, the word “negligent” is tokenised to \[‘negl’, ‘igent’\] by Ministral, but is maintained as a complete word token by Llama and Qwen\. During Fluency\-Aware Routing, the span is completed using only the specialist models that share the same tokenisation scheme for the target span\. After routing, the span is re\-synchronised by each model’s own tokeniser, ensuring that the span’s understanding won’t be damaged by misaligned tokenisations\.

Case study: Fluency\-Aware RoutingTask\.C4 CompletionPrefix\.*To Be Built By Professional Local Builders\! Post Modern Located On The Vanderbilt Little Neck Peninsula In Centerport\. SD\#6, Open Floor Plan Perfect For Entertaining With*Completion\.Open Kitchen, Family Room,Dining←\\leftarrow\[‘ Dining’\]And Bar Area\. Private MasterBedroom←\\leftarrow\[‘ Bedroom’\], Two Additional En\-SuiteBedrooms←\\leftarrow\[‘ Bedrooms’\]And One Additional Bed Or Office\. All Bedrooms With Walk\-In Closets And Bathrooms With Dual Sinks warmer\. 1st Likewise With Powder Room,2nd←\\leftarrow\[‘2’, ‘nd’\]WithLaundry\.←\\leftarrow\[‘ Laundry’, ‘\.’\]2\.5←\\leftarrow\[‘2’, ‘\.’, ‘5’\]car garage\.15←\\leftarrow\[‘1’, ‘5’\]min to CommuterParking←\\leftarrow\[‘ Parking’\]and LIRR←\\leftarrow\[‘IRR’\]Task\.MMLU Law ReasoningReasoning\.The fan was55←\\leftarrow\[‘5’, ‘5’\]%negligent←\\leftarrow\[‘ negligent’\]\(speeding\) and the driver was45←\\leftarrow\[‘4’, ‘5’\]%negligent←\\leftarrow‘ negligent’\(red light violation\)\. The friend was not foundnegligent\.←\\leftarrow\[‘ negl’, ‘igent’, ‘\.’\]Under modified comparativenegligence←\\leftarrow\[‘ negl’, ‘igence’\], if the plaintiff is not more than50←\\leftarrow\[‘5’, ‘0’\]%negligent←\\leftarrow\[‘ negl’, ‘igent’\], they can recover damages, but they are entitled to a percentage of the damages according to their relativenegligence\.←\\leftarrow\[‘ negligence’, ‘\.’\]However, the question states that the jury found $100,000←\\leftarrow\[‘100’, ‘,’, ‘000’\]in damages\. If the fan was blooded, and the friend suffered $100←\\leftarrow\[‘1’, ‘0’, ‘0’\],000←\\leftarrow\[‘000’\]in damages, but the friend’snegligence←\\leftarrow\[‘ negl’, ‘igence’\]was only 0% \(not foundnegligent←\\leftarrow\[‘ negligent’\]\), then the friend can recover the full $100←\\leftarrow\[‘1’, ‘0’, ‘0’\],000\.←\\leftarrow\[‘000’, ‘\.’\]The answer is A\.Figure 4:Examples of Fluency\-Aware Routing on C4 completion and MMLU Law reasoning tasks\. The routed spans are shown in bold blue, with the list of routed tokens appended\.