Auditing Cross-Lingual Fairness in Language Model Watermarking

# Auditing Cross-Lingual Fairness in Language Model Watermarking
Source: [https://arxiv.org/html/2608.20047](https://arxiv.org/html/2608.20047)
Alexander Nemecek, Osama Zafar, Debargha Ganguly, Vikash Singh,Thanks:Corresponding author\.Affiliation:Vipin Chaudhary, Erman AydayAffiliation:Case Western Reserve UniversityEmail:[\{ajn98,oxz23,dxg512,vxs465,vxc204,exa208\}@case\.edu](mailto:)

###### Abstract

Watermarking schemes for large language model output are evaluated almost exclusively on English text using each scheme’s detection threshold and a narrow set of quality measurements\. Multilingual deployment exposes evaluation\-design choices that are inconsequential on English but determine conclusions cross\-lingually\. We propose an evaluation framework with four components: detection thresholds calibrated empirically per deployment context, a threshold\-independent companion measurement that distinguishes calibration failures from detection failures, three disjoint quality measurement paradigms \(distributional, paired\-semantic, and reference\-perplexity\), and a generalized\-entropy decomposition of cross\-language disparity over a typological family partition\. Applied to six watermarking schemes, three open\-weight generators, eleven languages spanning four scripts and eight typological families, and both base and instruction\-tuned regimes, the framework reveals failure modes that single\-language single\-paradigm evaluation cannot surface\. Across detection and quality, observed disparity is predominantly between\-family on the typological partition, indicating that cross\-lingual fairness gaps in watermarking are structural to language properties rather than idiosyncratic to particular languages\.

## 1Introduction

Recent advances in large language models \(LLMs\) have made AI\-generated text difficult to distinguish from human\-authored writing\. This raises concerns about misinformation, the erosion of trust in online content, and model collapse, in which the performance of models trained on partially machine\-generated data degrades over successive generations[1](https://arxiv.org/html/2608.20047#bib.bib2);[24](https://arxiv.org/html/2608.20047#bib.bib4);[22](https://arxiv.org/html/2608.20047#bib.bib1)\. The misinformation concern in particular is not hypothetical\. NewsGuard has identified over 3,000 AI\-generated news and information sites publishing in 16 languages, often under legitimate\-sounding names and with little to no human oversight[20](https://arxiv.org/html/2608.20047#bib.bib3)\.

Text watermarking offers one mitigation by embedding a recoverable statistical signature into model output\. A watermarking scheme modifies the generation process in a key\-dependent way, most commonly by biasing the next\-token distribution toward a pseudorandomly selected vocabulary subset[10](https://arxiv.org/html/2608.20047#bib.bib8);[15](https://arxiv.org/html/2608.20047#bib.bib27);[2](https://arxiv.org/html/2608.20047#bib.bib18)\. A paired detector then tests a candidate text against this signature and returns a score, which is thresholded to yield a detection decision[6](https://arxiv.org/html/2608.20047#bib.bib28)\. Schemes are typically evaluated on detectability, generation quality \(fluency, coherence, and distributional similarity to unwatermarked output\), and robustness to downstream perturbations[18](https://arxiv.org/html/2608.20047#bib.bib5);[25](https://arxiv.org/html/2608.20047#bib.bib6);[12](https://arxiv.org/html/2608.20047#bib.bib7)\.

These evaluations, however, are conducted almost exclusively on English prompts and English generated text\. The implicit assumption is that a scheme which detects reliably and preserves quality on English will do the same on other languages, but this assumption has not been empirically tested at scale\. This gap matters in practice, because watermarking is already deployed in settings where users prompt and generate in dozens of languages[2](https://arxiv.org/html/2608.20047#bib.bib18);[14](https://arxiv.org/html/2608.20047#bib.bib29), and the multilingual misinformation ecosystem flagged above is precisely the threat surface that motivates the technology in the first place\.

Our work\.We present the first systematic audit of cross\-lingual fairness in LLM text watermarking\. Our contribution is methodological and empirical: an evaluation framework that treats multilingual fairness as a first\-class object of measurement rather than an after\-the\-fact slice of an English\-centric benchmark, and a characterization of how current schemes behave under it\.

\-Framework\.We propose four evaluation components designed for cross\-lingual settings: empirically calibrated detection thresholds, a threshold\-independent companion measurement that distinguishes calibration failures from detection failures, three disjoint quality paradigms \(distributional, paired\-semantic, and reference\-perplexity\), and a generalized\-entropy decomposition of disparity over a pre\-registered typological partition\.

\-Audit\.We apply the framework to six watermarking schemes, three open\-weight generators, eleven languages spanning four scripts and eight typological families, and two generation regimes \(parallel continuation and native\-speaker instruction\), yielding≈200,000\\approx 200\{,\}000matched watermarked/unwatermarked generations on shared prompts\.

\-Findings\.Cross\-language disparity is large, scheme\-specific, and predominantly between\-family on the typological partition rather than idiosyncratic to particular languages, on both detection and quality\. Per\-scheme rankings depend on which evaluation choice \(threshold, paradigm, prompt regime\) is taken as canonical, and we identify failure modes invisible to single\-language, single\-paradigm evaluation\.

## 2Related Work

Watermarking schemes\.Two design families organize recent work on LLM text watermarking\. The green\-list logit\-bias family biases the next\-token distribution toward a pseudorandomly selected vocabulary subset, with variants differing in how the partition is constructed: per\-position from a sliding window of preceding tokens[10](https://arxiv.org/html/2608.20047#bib.bib8), statically across the entire vocabulary[29](https://arxiv.org/html/2608.20047#bib.bib14), or via a cross\-lingual semantic\-cluster lookup intended to preserve detectability under translation[7](https://arxiv.org/html/2608.20047#bib.bib15)\. The distortion\-free family aims to leave the marginal next\-token distribution unchanged in expectation, intervening at sampling time through permutation reparameterization[26](https://arxiv.org/html/2608.20047#bib.bib16), key\-conditioned tournament sampling[2](https://arxiv.org/html/2608.20047#bib.bib18), or inverse transform sampling against a pseudo\-random uniform stream[11](https://arxiv.org/html/2608.20047#bib.bib17)\.

Watermark evaluation\.Existing watermark evaluation frameworks share a methodological template: detection is reported at each scheme’s default threshold targeting a theoretical false\-positive rate, quality is reported under a single paradigm, and prompts and generations are English\. MarkMyWords[18](https://arxiv.org/html/2608.20047#bib.bib5)and WaterBench[25](https://arxiv.org/html/2608.20047#bib.bib6)instantiate this template at benchmark scale, and dedicated robustness studies extend it to adversarial perturbations while retaining the same threshold and quality conventions[12](https://arxiv.org/html/2608.20047#bib.bib7)\. Recent work has identified this gap, calling for cross\-lingual and demographic disaggregation[16](https://arxiv.org/html/2608.20047#bib.bib25)\. These choices are inconsequential on English under single\-paradigm reporting, and become consequential cross\-lingually\. Prior cross\-lingual evaluation of watermarking has addressed translation robustness, asking whether a signal embedded in one language survives a translation pass, rather than whether detection and quality are preserved across languages of original generation[7](https://arxiv.org/html/2608.20047#bib.bib15)\.

Fairness methodology\.A watermark detector is a binary classifier and a quality measurement is a real\-valued performance score, so per\-language detection and quality vectors are the watermarking analog of per\-group performance vectors in classical fair\-ML evaluation\. We import three tools from that literature\. Operating\-point calibration, which sets detection thresholds per group rather than globally to expose calibration\-driven disparity[5](https://arxiv.org/html/2608.20047#bib.bib26)\. The four\-fifths rule, which thresholds per\-group performance ratios against the maximum to flag disparate impact[3](https://arxiv.org/html/2608.20047#bib.bib12)\. Generalized\-entropy decomposition, which splits a per\-group dispersion statistic into within\-partition and between\-partition components against a registered partition of the groups[21](https://arxiv.org/html/2608.20047#bib.bib11)\. Each tool transfers directly with language as the group attribute and typological family as the partition\.

## 3Experimental Grid

We evaluate every cell of a11​languages×6​schemes×3​generators×2​regimes11\\text\{ languages\}\\times 6\\text\{ schemes\}\\times 3\\text\{ generators\}\\times 2\\text\{ regimes\}grid atn=500n=500matched pairs per cell \(one watermarked, one unwatermarked, on a shared prompt\)\. The grid’s axes are chosen so that the typological decomposition of §[4\.4](https://arxiv.org/html/2608.20047#S4.SS4)is well\-posed \(multiple languages per script and morphological class\), so that base\-versus\-instruct sensitivity is identifiable \(every generator appears in both regimes\), and so that scheme\-family effects are not confounded with any single tokenizer \(every scheme is run on three independent vocabularies\)\.

Languages:The eleven evaluation languages span four scripts, eight typological families, and two Joshi et al\. resource tiers[8](https://arxiv.org/html/2608.20047#bib.bib13); Table[1](https://arxiv.org/html/2608.20047#S3.T1)enumerates the assignments\. The set is the minimal configuration in which the between\-family component of the generalized\-entropy decomposition \(§[4\.4](https://arxiv.org/html/2608.20047#S4.SS4)\) is identifiable on more than one non\-singleton family: Germanic and Romance both carry within\-family signal, and the remaining six families enter as singletons\.

Table 1:The eleven evaluation languages with script, typological family, and Joshi resource tier\.Watermarking schemes:We evaluate six schemes covering the two design families introduced in §[2](https://arxiv.org/html/2608.20047#S2)\. From the green\-list logit\-bias family: KGW[10](https://arxiv.org/html/2608.20047#bib.bib8)\(per\-position sliding window\), Unigram[29](https://arxiv.org/html/2608.20047#bib.bib14)\(static green list\), and XSIR[7](https://arxiv.org/html/2608.20047#bib.bib15)\(cross\-lingual semantic\-cluster lookup, designed to preserve detectability under translation\)\. From the distortion\-free family: DIP[26](https://arxiv.org/html/2608.20047#bib.bib16)\(permutation reparameterization\), SynthID\-Text[2](https://arxiv.org/html/2608.20047#bib.bib18)\(key\-conditioned tournament sampling\), and EXPEdit[11](https://arxiv.org/html/2608.20047#bib.bib17)\(inverse transform sampling\)\. For EXPEdit we use the published fast\-path detector, which replaces the original permutation test with a closed\-form score; we validate the fast path against the slow path\. Together the six schemes vary along the axes whose interaction with cross\-lingual generation is the object of study: per\-position versus static green\-list construction, logit\-bias versus sampling\-time intervention, and English\-trained versus cross\-lingual partitioning\.

Generators and regimes:We use three open\-weight generator families with distinct tokenizers and meaningful coverage of our eleven languages: Mistral\-NeMo\-12B[13](https://arxiv.org/html/2608.20047#bib.bib19), Gemma\-3\-4B[4](https://arxiv.org/html/2608.20047#bib.bib21), and Qwen2\.5\-7B[27](https://arxiv.org/html/2608.20047#bib.bib22)\. Each is run in both its base and instruction\-tuned variant\. The two generation regimes are designed to control different sources of cross\-lingual confound\. Thebase regimefeeds each base generator a sentence from FLORES\+\+devtest[17](https://arxiv.org/html/2608.20047#bib.bib20)as continuation prompt; FLORES\+\+provides parallel translations of the same source sentences across all eleven languages, so prompt content is held constant across languages and any cross\-lingual variation in detection or quality cannot be attributed to prompt\-content variation\. Theinstruct regimefeeds each instruction\-tuned generator native\-speaker prompts from the AYA dataset[23](https://arxiv.org/html/2608.20047#bib.bib23)\(original\-annotationssubset only, no machine translations\); AYA provides linguistically natural prompt distributions written by speakers of the target language, so cross\-lingual variation reflects realistic deployment input\. Reporting both regimes lets us check whether cross\-lingual disparities are robust across content\-controlled continuation by base models, and in\-language instruction\-following by their tuned counterparts or are localized to one\.

Generation protocol and LID gating:For every cell we generaten=500n=500watermarked outputs and an additionaln=500n=500unwatermarked outputs sharing the same prompts and seeds, fixingtemperature=0\.7\\texttt\{temperature\}=0\.7,max\_new\_tokens=200\\texttt\{max\\\_new\\\_tokens\}=200, andmin\_new\_tokens=100\\texttt\{min\\\_new\\\_tokens\}=100\. The unwatermarked baseline is shared across the six schemes within each\(l,m,r\)\(l,m,r\)triple, so detection comparisons within a cell are matched\-prompt\. Every generation is passed through GlotLID\-v3[9](https://arxiv.org/html/2608.20047#bib.bib24)to assign a language identifier; the headline rollups in §[5](https://arxiv.org/html/2608.20047#S5)are computed onsubset = all\(no LID filtering\) to avoid post\-hoc selection effects on the disparity statistics, with on\-target and off\-target slices reported in Appendix[B](https://arxiv.org/html/2608.20047#A2)\. Cells with fewer than 200 LID\-passing unwatermarked generations are excluded from quantile calibration atα=0\.01\\alpha=0\.01\(100 atα=0\.05\\alpha=0\.05\); paired\-set quality measurements require at least 100 samples per side\.

## 4Evaluation Framework

We propose a four\-component framework: empirically calibrated detection thresholds \(§[4\.1](https://arxiv.org/html/2608.20047#S4.SS1)\), a threshold\-independent companion that distinguishes calibration failures from detection failures \(§[4\.2](https://arxiv.org/html/2608.20047#S4.SS2)\), three disjoint quality paradigms \(§[4\.3](https://arxiv.org/html/2608.20047#S4.SS3)\), and a generalized\-entropy decomposition of cross\-language disparity over a typological partition \(§[4\.4](https://arxiv.org/html/2608.20047#S4.SS4)\)\. §[4\.5](https://arxiv.org/html/2608.20047#S4.SS5)specifies four disparity measurements \(M1\-M4\) applied symmetrically to both detection and quality sides\.

### 4\.1Empirical FPR Threshold Calibration

We index each evaluation cell asc=\(l,s,m,r\)c=\(l,s,m,r\): language, scheme, generator, regime\. Each schemessdefines a scoreSs:𝒯→ℝS\_\{s\}:\\mathcal\{T\}\\to\\mathbb\{R\}with watermarked text receiving systematically larger scores than unwatermarked, and withinccwe observe a watermarked set𝒳c\+\\mathcal\{X\}^\{\+\}\_\{c\}and a matched\-prompt unwatermarked set𝒳c−\\mathcal\{X\}^\{\-\}\_\{c\}\. Each of the six schemes ships a hardcoded threshold targeting a theoretical FPR under an IID\-token null that cross\-lingual generation does not satisfy, and in §[5](https://arxiv.org/html/2608.20047#S5)we exhibit a scheme\-language pair where the null tail saturates at the score cap so no watermarked text can clear the default\. We calibrate empirically instead: at levelα∈\(0,1\)\\alpha\\in\(0,1\),

τc​\(α\):=Quantile1−α​\(\{Ss​\(x\):x∈𝒳c−\},lower\),\\tau\_\{c\}\(\\alpha\)\\;:=\\;\\mathrm\{Quantile\}\_\{1\-\\alpha\}\\\!\\left\(\\\{S\_\{s\}\(x\):x\\in\\mathcal\{X\}^\{\-\}\_\{c\}\\\},\\ \\mathrm\{lower\}\\right\),the largest cutoff with realised FPR on𝒳c−\\mathcal\{X\}^\{\-\}\_\{c\}at mostα\\alpha\. A per\-language thresholdτclang​\(α\)\\tau^\{\\mathrm\{lang\}\}\_\{c\}\(\\alpha\)models deployment with a separate detector per language; a global thresholdτ\(s,m,r\)global​\(α\)\\tau^\{\\mathrm\{global\}\}\_\{\(s,m,r\)\}\(\\alpha\)calibrated from the pooled null⋃l𝒳\(l,s,m,r\)−\\bigcup\_\{l\}\\mathcal\{X\}^\{\-\}\_\{\(l,s,m,r\)\}models cross\-lingual deployment with a single detector and is our headline for the deployment\-realistic analog of operating\-point calibration in fair classification[5](https://arxiv.org/html/2608.20047#bib.bib26)\. Cells with insufficient null samples are excluded from disparity rollups; sample\-size floors are stated in §[3](https://arxiv.org/html/2608.20047#S3)\.

### 4\.2Threshold\-Independent Companion Measurement

TheTPR\\mathrm\{TPR\}atτc​\(α\)\\tau\_\{c\}\(\\alpha\)can collapse to zero from null\-tail saturation even when the watermarked and unwatermarked score distributions are well separated in the body\. To diagnose this we report a threshold\-independent companion,

AUCc=Pr⁡\(Ss​\(x\+\)\>Ss​\(x−\)\),x\+∈𝒳c\+,x−∈𝒳c−,\\mathrm\{AUC\}\_\{c\}\\;=\\;\\Pr\\\!\\left\(S\_\{s\}\(x^\{\+\}\)\>S\_\{s\}\(x^\{\-\}\)\\right\),\\quad x^\{\+\}\\\!\\in\\\!\\mathcal\{X\}^\{\+\}\_\{c\},\\ x^\{\-\}\\\!\\in\\\!\\mathcal\{X\}^\{\-\}\_\{c\},estimated by the Mann\-WhitneyUUstatistic with average\-rank tie handling\. TheTPR\\mathrm\{TPR\}\-AUC\\mathrm\{AUC\}gap separates a calibration failure \(TPR≈0\\mathrm\{TPR\}\\\!\\approx\\\!0,AUC≫0\.5\\mathrm\{AUC\}\\\!\\gg\\\!0\.5\) from a detection failure \(TPR≈0\\mathrm\{TPR\}\\\!\\approx\\\!0,AUC≈0\.5\\mathrm\{AUC\}\\\!\\approx\\\!0\.5\); §[5](https://arxiv.org/html/2608.20047#S5)exhibits a concrete instance where this distinction reverses the diagnosis\.

### 4\.3Three Quality Paradigms

Quality preservation is conceptually multi\-faceted, and a single metric cannot adjudicate it\. We measure preservation under three paradigms chosen so that none subsumes another\.

Distributional \(MAUVE\)\.We compute MAUVE[19](https://arxiv.org/html/2608.20047#bib.bib9)between𝒳c\+\\mathcal\{X\}^\{\+\}\_\{c\}and𝒳c−\\mathcal\{X\}^\{\-\}\_\{c\}on mean\-pooled XLM\-R\-large embeddings\. MAUVE captures aggregate distributional shift but is insensitive to per\-prompt semantic deviation\.

Paired\-semantic \(BERTScore\)\.For each watermarked generation we compute BERTScore F1[28](https://arxiv.org/html/2608.20047#bib.bib10)against the matched\-prompt unwatermarked completion, with per\-language baselines estimated from FLORES non\-pairs for rescaling\. BERTScore captures per\-prompt semantic preservation but is insensitive to systematic distributional shift\.

Reference perplexity\.For a held\-out reference LMRR\(XGLM\-7\.5B as primary, mGPT\-1\.3B as cross\-reference\), letlog⁡PPLR​\(𝒳\)\\log\\mathrm\{PPL\}\_\{R\}\(\\mathcal\{X\}\)denote the macro\-averaged log\-perplexity of generations in𝒳\\mathcal\{X\}underRR\. We report the cell preservation score

PPL​\-​preservationc=exp⁡\(−\|log⁡PPLR​\(𝒳c\+\)−log⁡PPLR​\(𝒳c−\)\|\)∈\(0,1\]\.\\mathrm\{PPL\\text\{\-\}preservation\}\_\{c\}\\;=\\;\\exp\\\!\\left\(\-\\bigl\|\\log\\mathrm\{PPL\}\_\{R\}\(\\mathcal\{X\}^\{\+\}\_\{c\}\)\-\\log\\mathrm\{PPL\}\_\{R\}\(\\mathcal\{X\}^\{\-\}\_\{c\}\)\\bigr\|\\right\)\\;\\in\\;\(0,1\]\.PPL captures per\-token likelihood preservation under an external reference but inherits that reference’s biases\.

The three paradigms can disagree on which schemes are least distorting \(§[5](https://arxiv.org/html/2608.20047#S5)\); we therefore report all three rather than pick one\.

### 4\.4Generalized\-Entropy Decomposition

Given a per\-language vectory=\(y1,…,yL\)y=\(y\_\{1\},\\ldots,y\_\{L\}\)of detection or quality scores, we measure cross\-language disparity by the generalized entropy index atα=2\\alpha=2,GE2​\(y\)=\(2​L\)−1​∑i\[\(yi/y¯\)2−1\]\\mathrm\{GE\}\_\{2\}\(y\)=\(2L\)^\{\-1\}\\sum\_\{i\}\\\!\\left\[\(y\_\{i\}/\\bar\{y\}\)^\{2\}\-1\\right\], which equals half the squared coefficient of variation\. Under a partition of languages into typological familiesℱ\\mathcal\{F\},GE2\\mathrm\{GE\}\_\{2\}admits the standard within/between decompositionGE2​\(y\)=GE2within​\(y\)\+GE2between​\(y\)\\mathrm\{GE\}\_\{2\}\(y\)=\\mathrm\{GE\}\_\{2\}^\{\\mathrm\{within\}\}\(y\)\+\\mathrm\{GE\}\_\{2\}^\{\\mathrm\{between\}\}\(y\)[21](https://arxiv.org/html/2608.20047#bib.bib11)\. The between\-family share quantifies whether disparity is structural to typology \(script, morphology, tokenizer interactions\) or idiosyncratic; we register the typological partition \(Germanic, Romance, Indic, Semitic, Sinitic, Japonic, Turkic, Austroasiatic\) before analysis and report a script\-based partition as a robustness check\.

### 4\.5Disparity Measurements \(M1\-M4\)

Each per\-cell measurement applies to both detection \(per\-languageTPRc​\(α\)\\mathrm\{TPR\}\_\{c\}\(\\alpha\)\) and to each quality paradigm \(per\-language preservation, §[4\.3](https://arxiv.org/html/2608.20047#S4.SS3)\):M1, the per\-language vector;M2, its mean and minimum across languages;M3, the four\-fifths disparate\-impact count, equal to the number of languages with value below×max0\.8\\\!\\times\\\!\\max\([3](https://arxiv.org/html/2608.20047#bib.bib12)\); andM4,GE2\\mathrm\{GE\}\_\{2\}with within/between\-family decomposition \(§[4\.4](https://arxiv.org/html/2608.20047#S4.SS4)\)\.

## 5Findings

### 5\.1Detection

Table[2](https://arxiv.org/html/2608.20047#S5.T2)summarizes detection across both regimes\. The base\-FLORES panel shows that four of six schemes \(KGW, SynthID, EXPEdit, DIP\) attain mean TPR above 0\.93 in every generator cell atτg\\tau\_\{g\}, with no per\-language minimum below 0\.798\. Within this regime, Unigram and XSIR carry the analytically interesting failure modes; the instruct\-AYA panel inverts the picture, so the two regimes warrant separate discussion\.

AUCTPR atτg\\tau\_\{g\}\(global\)TPR atτl\\tau\_\{l\}\(per\-lang\)SchemeGen\.MeanMinMean TPRMin TPRDI<\.8GE2Btw\. %Mean TPRΔ\\DeltaBase regime \(FLORES\+ continuation prompts\)KGWMistral0\.9990\.9970\.992 \[0\.987, 0\.996\]0\.9780<0\.001<\\\!0\.00183\.60\.991\+0\.001\+0\.001Gemma0\.9970\.9910\.969 \[0\.959, 0\.979\]0\.9340<0\.001<\\\!0\.00172\.40\.974−0\.005\-0\.005Qwen0\.9960\.9870\.965 \[0\.939, 0\.986\]0\.8800<0\.001<\\\!0\.00185\.10\.966−0\.001\-0\.001UnigramMistral0\.9960\.9810\.868 \[0\.802, 0\.927\]0\.63830\.00861\.30\.971−0\.103\-0\.103Gemma0\.9860\.9100\.000 \[0\.000, 0\.000\]0\.00011——0\.799−0\.799\-0\.799Qwen0\.9940\.9720\.898 \[0\.845, 0\.942\]0\.72220\.00493\.80\.949−0\.051\-0\.051SynthIDMistral1\.0001\.0001\.000 \[0\.999, 1\.000\]0\.9980<0\.001<\\\!0\.001100\.01\.000−0\.000\-0\.000Gemma1\.0001\.0001\.000 \[0\.999, 1\.000\]0\.9980<0\.001<\\\!0\.001100\.01\.000\+0\.000\+0\.000Qwen1\.0000\.9990\.997 \[0\.995, 0\.999\]0\.9900<0\.001<\\\!0\.00197\.50\.998−0\.000\-0\.000EXPEditMistral1\.0000\.9980\.999 \[0\.999, 1\.000\]0\.9960<0\.001<\\\!0\.00188\.01\.000−0\.000\-0\.000Gemma1\.0000\.9981\.000 \[0\.999, 1\.000\]0\.9980<0\.001<\\\!0\.001100\.01\.000\+0\.000\+0\.000Qwen0\.9980\.9930\.991 \[0\.982, 0\.997\]0\.9460<0\.001<\\\!0\.00199\.20\.992−0\.001\-0\.001DIPMistral0\.9970\.9890\.975 \[0\.966, 0\.984\]0\.9380<0\.001<\\\!0\.00192\.60\.975\+0\.000\+0\.000Gemma0\.9920\.9650\.963 \[0\.947, 0\.977\]0\.9140<0\.001<\\\!0\.00199\.50\.965−0\.002\-0\.002Qwen0\.9920\.9770\.931 \[0\.891, 0\.968\]0\.79800\.00394\.70\.929\+0\.002\+0\.002XSIRMistral0\.9810\.9620\.784 \[0\.743, 0\.821\]0\.67220\.00491\.50\.833−0\.049\-0\.049Gemma0\.9840\.9770\.766 \[0\.712, 0\.817\]0\.58840\.00696\.00\.829−0\.063\-0\.063Qwen0\.9780\.9300\.215 \[0\.118, 0\.362\]0\.050100\.49398\.90\.794−0\.579\-0\.579Instruct regime \(AYA native\-speaker prompts\)KGWMistral0\.9530\.9090\.626 \[0\.540, 0\.707\]0\.38670\.02695\.30\.632−0\.005\-0\.005Gemma0\.9000\.8480\.389 \[0\.325, 0\.463\]0\.222100\.05382\.40\.411−0\.022\-0\.022Qwen0\.9660\.9370\.689 \[0\.628, 0\.753\]0\.51080\.01178\.60\.723−0\.034\-0\.034UnigramMistral0\.9090\.8650\.311 \[0\.171, 0\.453\]0\.02260\.29799\.80\.450−0\.139\-0\.139Gemma0\.8510\.7900\.037 \[0\.023, 0\.052\]0\.010100\.22681\.20\.151−0\.113\-0\.113Qwen0\.9430\.8800\.546 \[0\.455, 0\.621\]0\.21040\.03291\.70\.636−0\.090\-0\.090SynthIDMistral0\.9910\.9820\.905 \[0\.865, 0\.940\]0\.78210\.00377\.70\.903\+0\.002\+0\.002Gemma0\.9460\.9020\.602 \[0\.530, 0\.675\]0\.422100\.02079\.70\.597\+0\.005\+0\.005Qwen0\.9930\.9770\.941 \[0\.906, 0\.971\]0\.79210\.00291\.90\.942−0\.002\-0\.002EXPEditMistral0\.9180\.7820\.577 \[0\.454, 0\.698\]0\.23860\.06780\.00\.591−0\.014\-0\.014Gemma0\.6210\.4620\.225 \[0\.179, 0\.275\]0\.084100\.07184\.20\.237−0\.012\-0\.012Qwen0\.8390\.6830\.496 \[0\.379, 0\.620\]0\.17490\.08084\.40\.508−0\.012\-0\.012DIPMistral0\.9130\.8650\.455 \[0\.364, 0\.546\]0\.26490\.05899\.50\.477−0\.022\-0\.022Gemma0\.8230\.7700\.196 \[0\.154, 0\.247\]0\.110100\.08491\.60\.205−0\.010\-0\.010Qwen0\.9240\.8660\.513 \[0\.434, 0\.593\]0\.29680\.03599\.60\.521−0\.007\-0\.007XSIRMistral0\.8640\.7700\.266 \[0\.183, 0\.369\]0\.088100\.18591\.80\.300−0\.035\-0\.035Gemma0\.7750\.6990\.081 \[0\.055, 0\.113\]0\.016100\.17576\.50\.125−0\.043\-0\.043Qwen0\.8540\.7480\.043 \[0\.003, 0\.119\]0\.000103\.78799\.90\.286−0\.243\-0\.243Table 2:Detection fairness by scheme, generator, and regime, subsetall,α=0\.01\\alpha=0\.01\. Mean TPR shown with 95% bootstrap CIs \(B=1000B=1000resamples of the 11\-language vector\)\. Min TPR is the realized minimum\. DI<\.8counts languages withTPRl<0\.8⋅maxl⁡TPRl\\mathrm\{TPR\}\_\{l\}<0\.8\\cdot\\max\_\{l\}\\mathrm\{TPR\}\_\{l\}\. GE2uses the typological partition \(Germanic, Romance, Indic, Semitic, Sinitic, Japonic, Turkic, Austroasiatic\); Btw\. % is its between\-family share\.Δ=mean​TPR​\(τg\)−mean​TPR​\(τl\)\\Delta=\\mathrm\{mean\\,TPR\}\(\\tau\_\{g\}\)\-\\mathrm\{mean\\,TPR\}\(\\tau\_\{l\}\)\. Values below10−310^\{\-3\}reported as<0\.001<\\\!0\.001; entries marked are undefined when the per\-language mean is zero\.Calibration versus detection failure\.Two cells in the base regime would, under single\-threshold reporting, read as outright detection failures: Unigram\-Gemma \(mean TPR 0\.000, all eleven languages at zero\) and XSIR\-Qwen \(mean TPR 0\.215, ten of eleven languages flagged for disparate impact\)\. The AUC companion contradicts that classification\. Unigram\-Gemma’s mean AUC is 0\.986 with minimum 0\.910, and XSIR\-Qwen’s mean AUC is 0\.978; the watermark signal is present in both cells, and the global threshold simply sits above the unwatermarked\-pool support\. Per\-language recalibration recovers mean TPR to 0\.799 and 0\.794 respectively, producing the two largest\|Δ\|\|\\Delta\|values of 0\.799 and 0\.579 in the grid and the diagnostic signature for cells where single\-detector deployment is the wrong operating\-point choice\. Without the threshold\-independent companion of §[4\.2](https://arxiv.org/html/2608.20047#S4.SS2), both cells would have been mis\-attributed to the watermark rather than to the calibration regime\.

Instruction\-tuning collapses detection\.The instruct\-AYA panel shows mean TPR below 0\.7 in 16 of 18 cells and DI<\.8≥8\{\}\_\{<\.8\}\\geq 8in 12 of 18, with non\-SynthID cells dropping by roughly 0\.17 to 0\.78 in mean TPR relative to their base counterparts\. SynthID is the only partial exception: SynthID\-Mistral and SynthID\-Qwen retain mean TPR above 0\.9, and SynthID\-Gemma at 0\.602 is the strongest Gemma instruct cell across all six schemes, though still well below its base value of 1\.000\. AUC nonetheless remains above the chance baseline of 0\.5 in 197 of 198 cell\-language entries in the instruct panel \(the sole exception is EXPEdit\-Gemma on Japanese at 0\.462\)\. The instruct collapse is therefore predominantly calibrational: the elevated entropy of instruction\-tuned generation under native\-speaker prompts compresses the watermarked and unwatermarked score distributions together but does not erase their separation\. The base\-versus\-instruct axis is itself a substantial fairness disparity, comparable in magnitude to the cross\-language gaps motivating the framework, and it generalizes across five of six scheme families\.

Structural rather than idiosyncratic disparity\.For cells with GE2above the10−310^\{\-3\}floor, the between\-family share sits between 61\.3% and 99\.9% with median above 90% across both regimes\. Cross\-language disparity is therefore concentrated at the typological\-family level rather than scattering across individual languages\. The family\-versus\-script partition robustness check \(Table[12](https://arxiv.org/html/2608.20047#A3.T12)\) sharpens this: the family partition captures more between\-group variance than the script partition for every cell with non\-trivial total disparity, with the sole near\-tie being XSIR\-Qwen instruct, where Hindi is a singleton in both partitions and the structural component is therefore invariant to the choice\. The XSIR\-Qwen instruct cell, with GE2=3\.787\{\}\_\{2\}=3\.787, is an order of magnitude beyond any other entry in the grid; its 99\.9% between\-family share localizes the extreme dispersion to a small number of typological families rather than to any singular linguistic outlier\.

![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_q_5_2_combined_xglm_base.png)

![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_q_5_2_combined_xglm_instruct.png)

Figure 1:Per\-language quality preservation under three paradigms, base\-FLORES \(top\) and instruct\-AYA \(bottom\);n=500n=500matched pairs per cell\. Left: MAUVE on mean\-pooled XLM\-R\-large embeddings \(1\.01\.0identical,0\.00\.0fully separable\)\. Center: BERTScore F1, rescaled against per\-language FLORES non\-pair baselines \(negative: paired generations less similar than random non\-pairs\)\. Right: PPL preservation under XGLM\-7\.5B,exp⁡\(−\|log⁡PPLR​\(𝒳\+\)−log⁡PPLR​\(𝒳−\)\|\)∈\(0,1\]\\exp\(\-\|\\log\\mathrm\{PPL\}\_\{R\}\(\\mathcal\{X\}^\{\+\}\)\-\\log\\mathrm\{PPL\}\_\{R\}\(\\mathcal\{X\}^\{\-\}\)\|\)\\in\(0,1\]\. Rows: \(scheme, generator\) cells; columns: eleven languages grouped by typological family\.
### 5\.2Quality

Figures[1](https://arxiv.org/html/2608.20047#S5.F1)and[2](https://arxiv.org/html/2608.20047#S5.F2)show the per\-language preservation landscape and its typological decomposition under three paradigms; per\-cell aggregates appear in Appendix[D\.1](https://arxiv.org/html/2608.20047#A4.SS1)\. Four findings organize the discussion\.

Distortion\-free schemes are the most distorting empirically\.EXPEdit and SynthID \(the two distortion\-free schemes\) sit at the bottom of the quality ranking under every paradigm in the base regime\. EXPEdit\-Gemma collapses to MAUVE 0\.01\-0\.06 and PPL preservation 0\.08\-0\.21 across the row; SynthID\-Gemma sits at MAUVE 0\.03\-0\.42 and PPL 0\.19\-0\.37; on BERTScore both schemes show distortion an order of magnitude beyond any logit\-bias scheme\. The marginal\-distribution distortion\-freeness these schemes provide in expectation does not translate to preservation under finite\-sample distributional, paired\-semantic, or reference\-perplexity measurement\. The four logit\-bias schemes \(KGW, Unigram, DIP, XSIR\) all preserve quality substantially better\.

Instruct regime compression\.All three measurements compress upward in the instruct regime: MAUVE saturates above 0\.95 on KGW, Unigram, DIP, and XSIR; BERTScore\-rescaled values flip to large positive \(the non\-pair baseline drops because AYA prompts force tighter content overlap among non\-pairs\); PPL preservation rises above 0\.90 on DIP and XSIR cells and above 0\.88 on KGW and Unigram\. The compression parallels the detection\-side compression of §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)but does not erase cross\-scheme separation: SynthID and EXPEdit remain the most distorting, with SynthID\-Mistral on Arabic at MAUVE 0\.07 and EXPEdit\-Mistral on Arabic at MAUVE 0\.02\. Within schemes the compression tracks the detection\-collapse pattern where the watermark detects weakly \(e\.g\., SynthID\-Gemma instruct, TPR 0\.602\) show near\-perfect quality preservation, consistent with the watermark having less effect on the generation distribution\.

Disparity is structural across paradigms\.Figure[2](https://arxiv.org/html/2608.20047#S5.F2)reports GE2over the per\-language preservation vector, averaged across the three generators per scheme, with the typological\-partition decomposition shown by stacking\. Two findings track\. First, SynthID and EXPEdit dominate cross\-language disparity on MAUVE \(GE2peaks at 0\.66 for EXPEdit base\) and PPL \(≈\\approx0\.053\), an order of magnitude beyond any logit\-bias scheme on the same paradigm; they are the most distorting schemes both on average and disparately across languages\. Second, the typological partition captures the dominant share of cross\-language disparity across the grid: between\-family≥70%\\geq 70\\%in 32 of 36 \(scheme, paradigm, regime\) cells, with median≈85%\\approx 85\\%, supporting the structural\-disparity claim from §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)on the quality side; the four sub\-70% cells are all on MAUVE and split between high\-GE2rows \(SynthID/EXPEdit base, where within\-family dispersion registers more in absolute terms\) and very\-low\-GE2rows \(DIP/Unigram instruct, where the ratio is noisy\)\.

Figure 2:Per\-scheme disparity decomposition under three quality paradigms, base\-FLORES \(top\) and instruct\-AYA \(bottom\)\. Bar height is GE2over the per\-language preservation vector, averaged across the three generators per scheme\. Stacking shows the typological\-partition decomposition into between\-family \(dark\) and within\-family \(light\) components; the percentage above is the between\-family share\. Absolute disparity is orders of magnitude smaller than under MAUVE or PPL but the between\-family pattern persists\. Y\-axes differ across panels\.Cross\-paradigm rank agreement\.Table[3](https://arxiv.org/html/2608.20047#S5.T3)quantifies cross\-paradigm rank inversion\. Pooled across the grid, no paradigm pair clearsρ=0\.6\\rho=0\.6: MAUVE\-vs\-BERTScore averages\+0\.58\+0\.58, MAUVE\-vs\-PPL\+0\.38\+0\.38, and BERTScore\-vs\-PPL\+0\.37\+0\.37\. The disagreement splits cleanly by intervention site\. The two sampling\-time distortion\-free schemes \(SynthID, EXPEdit\) sit atρ∈\[0\.68,0\.81\]\\rho\\in\[0\.68,0\.81\]across all three pairs with strictly positive cell\-level minima: their per\-language preservation vectors are monotone\-equivalent under all three paradigms\. The three green\-list schemes \(KGW, Unigram, XSIR\) sit at PPL\-vs\-embedding mean correlations of≤\+0\.20\\leq\+0\.20with cell\-level minima between−0\.39\-0\.39and−0\.80\-0\.80, locating cells where reference perplexity and the embedding\-based paradigms produce opposite per\-language orderings\. DIP, distortion\-free in design but logit\-side in implementation, falls in between atρ∈\[0\.43,0\.50\]\\rho\\in\[0\.43,0\.50\]\. The split is a local\-fluency\-vs\-distributional\-shift effect: where the watermark intervenes at the logit level, PPL captures fluency under a held\-out reference while MAUVE and BERTScore capture divergence from the unwatermarked set, so the per\-language orderings can come apart\. Hindi is the cleanest cell\-level instance: KGW, Unigram, and XSIR all preserve Hindi PPL at≥0\.65\\geq 0\.65under both XGLM and mGPT references \(Appendix[D\.2](https://arxiv.org/html/2608.20047#A4.SS2)\) while MAUVE flags outliers on the same cells \(Unigram\-Qwen 0\.12, KGW\-Qwen 0\.47\)\.

Table 3:Cross\-paradigm rank agreement: per\-cell Spearmanρ\\rhobetween per\-language preservation vectors, mean across \(model, regime\) cells with per\-cell minimum in parentheses \(subset=all;n=6n=6cells per scheme,n=36n=36pooled\)\. Pairs use BERTScore F1 \(rescaled\) and PPL preservation under XGLM\-7\.5B; BS\-raw gives nearly identical rankings \(ρBS\-raw↔BS\-rescaled=\+0\.87\\rho\_\{\\text\{BS\-raw\}\\leftrightarrow\\text\{BS\-rescaled\}\}=\+0\.87\)\. Negative minima identify cells where two paradigms produce opposite per\-language orderings\.
### 5\.3Joint detection\-quality

The detection and quality axes of §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)\-§[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)do not determine the joint shape: a scheme balanced on each axis separately can still be unfair if its detection wins and quality wins fall on different language subsets\. Figure[3](https://arxiv.org/html/2608.20047#S5.F3)reports per\-language\(TPR​@​τg,MAUVE\)\(\\mathrm\{TPR\}\\,@\\,\\tau\_\{g\},\\mathrm\{MAUVE\}\)pairs per scheme with the per\-regime spreadss, the mean pairwise Euclidean distance over the 33 points \(11 languages×\\times3 generators\), annotated\.

Low spread is consistent with opposite topologies\.DIP base \(sB=0\.066s\_\{B\}\{=\}0\.066\) and EXPEdit base \(sB=0\.070s\_\{B\}\{=\}0\.070\) carry near\-identical scalar spread but cluster in opposite corners: DIP top\-right \(detect\-and\-preserve\), EXPEdit bottom\-right \(detect\-everywhere, preserve\-nowhere\)\. XSIR instruct \(sI=0\.159s\_\{I\}\{=\}0\.159\) is a third low\-spread topology: top\-left under uniform detection collapse\. The scalar must be read with the panel\.

Figure 3:Joint per\-language detection\-quality landscape, MAUVE paradigm\. Markers: per\-language\(TPR​@​τg,MAUVE\)\(\\mathrm\{TPR\}\\,@\\,\\tau\_\{g\},\\mathrm\{MAUVE\}\)pairs atα=0\.01\\alpha\{=\}0\.01\. Circles: base\-FLORES\. Triangles: instruct\-AYA\. Color: typological family\. AnnotationssBs\_\{B\},sIs\_\{I\}: mean pairwise Euclidean distance within each regime\. BERTScore and PPL\-preservation companion panels in Appendix[E](https://arxiv.org/html/2608.20047#A5)\.Spread is regime\-dependent\.Base\-regime spread is dominated by detection\-side calibration failure: Unigram \(sB=0\.551s\_\{B\}\{=\}0\.551\) and XSIR \(sB=0\.399s\_\{B\}\{=\}0\.399\) scatter horizontally with quality intact, reproducing the §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)calibration\-gap diagnosis in joint form, while the four other base panels are tight clusters \(sB≤0\.165s\_\{B\}\\leq 0\.165\)\. Instruction tuning swaps the ranking: EXPEdit \(sI=0\.442s\_\{I\}\{=\}0\.442\), SynthID \(sI=0\.365s\_\{I\}\{=\}0\.365\), and Unigram \(sI=0\.301s\_\{I\}\{=\}0\.301\) lead under instruct as detection collapse induces vertical scatter on a typology\-structured language subset\. Base\-vs\-instruct shifts the topology of joint unfairness, not its magnitude\.

## 6Discussions

Limitations:Two limitations bound our claims\. Our eleven languages span four scripts and eight typological families, but many of the world’s languages remain outside this set\. Our six schemes cover the dominant green\-list and distortion\-free design families but are not exhaustive of an actively growing literature\.

Societal impacts:Watermarking is a scalable mitigation for AI\-generated text currently being deployed, much of which is multilingual\. A detection gap structural to typological family rather than idiosyncratic to a single language means the same scheme offers materially weaker protection to speakers of some language families than others, and quality disparities mean the cost of being watermarked is unequally borne\. Because these gaps are between\-family rather than per\-language, they are unlikely to close by scaling data for individual languages and instead implicate architectural choices, such as tokenizer construction or partition design, that operate at the family level\.

## 7Conclusion

In this work, we introduce the first systematic cross\-lingual fairness audit of LLM text watermarking: a four\-component evaluation framework comprising empirical FPR calibration, a threshold\-independent AUC companion, three disjoint quality paradigms, and a generalized\-entropy decomposition over a typological partition\. We apply this framework to six schemes, three open\-weight generators, and eleven languages across four scripts and eight typological families, under two generation regimes\. The audit surfaces failure modes that single\-language, single\-paradigm evaluation cannot, including calibration failures misread as detection failures, distortion\-free schemes that are empirically the most distorting, cross\-paradigm rank inversion, and disparity dominated by between\-family variation\. Our work promotes further research toward fairness in LLM watermarking\.

## References

- \[1\]\(2024\)Combating misinformation in the age of llms: opportunities and challenges\.AI Magazine45\(3\),pp\. 354–368\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p1.1)\.
- \[2\]S\. Dathathri, A\. See, S\. Ghaisas, P\. Huang, R\. McAdam, J\. Welbl, V\. Bachani, A\. Kaskasoli, R\. Stanforth, T\. Matejovicova,et al\.\(2024\)Scalable watermarking for identifying large language model outputs\.Nature634\(8035\),pp\. 818–823\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1),[§1](https://arxiv.org/html/2608.20047#S1.p3.1),[§2](https://arxiv.org/html/2608.20047#S2.p1.1),[§3](https://arxiv.org/html/2608.20047#S3.p3.1)\.
- \[3\]M\. Feldman, S\. A\. Friedler, J\. Moeller, C\. Scheidegger, and S\. Venkatasubramanian\(2015\)Certifying and removing disparate impact\.Inproceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining,pp\. 259–268\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p3.1),[§4\.5](https://arxiv.org/html/2608.20047#S4.SS5.p1.1)\.
- \[4\]Gemma Team, Google DeepMind\(2025\)Gemma 3 technical report\.Technical ReportGoogle DeepMind\.External Links:[Link](https://storage.googleapis.com/deepmind-media/gemma/Gemma3Report.pdf)Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p4.1)\.
- \[5\]M\. Hardt, E\. Price, and N\. Srebro\(2016\)Equality of opportunity in supervised learning\.Advances in neural information processing systems29\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p3.1),[§4\.1](https://arxiv.org/html/2608.20047#S4.SS1.p1.2)\.
- \[6\]H\. He, S\. Xu, A\. Nemecek, J\. Li, E\. Ayday, and G\. Cheng\(2026\)Optimal watermark generation under type i and type ii errors\.Biometrika,pp\. asag049\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1)\.
- \[7\]Z\. He, B\. Zhou, H\. Hao, A\. Liu, X\. Wang, Z\. Tu, Z\. Zhang, and R\. Wang\(2024\)Can watermarks survive translation? on the cross\-lingual consistency of text watermark for large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 4115–4129\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p1.1),[§2](https://arxiv.org/html/2608.20047#S2.p2.1),[§3](https://arxiv.org/html/2608.20047#S3.p3.1)\.
- \[8\]P\. Joshi, S\. Santy, A\. Budhiraja, K\. Bali, and M\. Choudhury\(2020\)The state and fate of linguistic diversity and inclusion in the nlp world\.InProceedings of the 58th annual meeting of the association for computational linguistics,pp\. 6282–6293\.Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p2.1)\.
- \[9\]A\. H\. Kargaran, A\. Imani, F\. Yvon, and H\. Schuetze\(2023\)GlotLID: language identification for low\-resource languages\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 6155–6218\.Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p5.1)\.
- \[10\]J\. Kirchenbauer, J\. Geiping, Y\. Wen, J\. Katz, I\. Miers, and T\. Goldstein\(2023\)A watermark for large language models\.InInternational conference on machine learning,pp\. 17061–17084\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1),[§2](https://arxiv.org/html/2608.20047#S2.p1.1),[§3](https://arxiv.org/html/2608.20047#S3.p3.1)\.
- \[11\]R\. Kuditipudi, J\. Thickstun, T\. Hashimoto, and P\. Liang\(2023\)Robust distortion\-free watermarks for language models\.arXiv preprint arXiv:2307\.15593\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p1.1),[§3](https://arxiv.org/html/2608.20047#S3.p3.1)\.
- \[12\]J\. Liang, Z\. Wang, S\. Hong, S\. Ji, and T\. Wang\(2024\)Watermark under fire: a robustness evaluation of llm watermarking\.arXiv preprint arXiv:2411\.13425\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1),[§2](https://arxiv.org/html/2608.20047#S2.p2.1)\.
- \[13\]Mistral AI team\(2024\)Mistral NeMo\.Note:Accessed: 2026\-05\-05External Links:[Link](https://mistral.ai/news/mistral-nemo/)Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p4.1)\.
- \[14\]A\. Nemecek, Y\. Jiang, and E\. Ayday\(2025\)Watermarking without standards is not ai governance\.arXiv preprint arXiv:2505\.23814\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p3.1)\.
- \[15\]A\. Nemecek, Y\. Jiang, and E\. Ayday\(2026\)Topic\-based watermarks for large language models\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 24372–24402\.External Links:[Link](https://aclanthology.org/2026.findings-acl.1220/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1220),ISBN 979\-8\-89176\-395\-1Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1)\.
- \[16\]A\. Nemecek, O\. Zafar, Y\. Xu, W\. Li, and E\. Ayday\(2026\)Who gets flagged? the pluralistic evaluation gap in ai content watermarking\.arXiv preprint arXiv:2604\.13776\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p2.1)\.
- \[17\]NLLB Team, M\. R\. Costa\-jussà, J\. Cross, O\. Çelebi, M\. Elbayad, K\. Heafield, K\. Heffernan, E\. Kalbassi, J\. Lam, D\. Licht, J\. Maillard, A\. Sun, S\. Wang, G\. Wenzek, A\. Youngblood, B\. Akula, L\. Barrault, G\. M\. Gonzalez, P\. Hansanti, J\. Hoffman, S\. Jarrett, K\. R\. Sadagopan, D\. Rowe, S\. Spruit, C\. Tran, P\. Andrews, N\. F\. Ayan, S\. Bhosale, S\. Edunov, A\. Fan, C\. Gao, V\. Goswami, F\. Guzmán, P\. Koehn, A\. Mourachko, C\. Ropers, S\. Saleem, H\. Schwenk, and J\. Wang\(2024\)Scaling neural machine translation to 200 languages\.Nature630\(8018\),pp\. 841–846\.External Links:ISSN 1476\-4687,[Document](https://dx.doi.org/10.1038/s41586-024-07335-x),[Link](https://doi.org/10.1038/s41586-024-07335-x)Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p4.1)\.
- \[18\]J\. Piet, C\. Sitawarin, V\. Fang, N\. Mu, and D\. Wagner\(2025\)Markmywords: analyzing and evaluating language model watermarks\.In2025 IEEE Conference on Secure and Trustworthy Machine Learning \(SaTML\),pp\. 68–91\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1),[§2](https://arxiv.org/html/2608.20047#S2.p2.1)\.
- \[19\]K\. Pillutla, S\. Swayamdipta, R\. Zellers, J\. Thickstun, S\. Welleck, Y\. Choi, and Z\. Harchaoui\(2021\)Mauve: measuring the gap between neural text and human text using divergence frontiers\.Advances in Neural Information Processing Systems34,pp\. 4816–4828\.Cited by:[§4\.3](https://arxiv.org/html/2608.20047#S4.SS3.p2.1)\.
- \[20\]M\. Sadeghi, D\. Dimitriadis, V\. Padovese, G\. Pozzi, S\. Badilini, C\. Vercellone, N\. Huet, Z\. Fishman, L\. Pfaller, and N\. Adams\(2026\)Tracking AI\-enabled misinformation: 3,006 AI content farm sites \(and counting\), plus the top false claims generated by artificial intelligence tools\.Note:[https://www\.newsguardtech\.com/special\-reports/ai\-tracking\-center/](https://www.newsguardtech.com/special-reports/ai-tracking-center/)NewsGuard Technologies, Inc\. Last updated March 17, 2026\. Accessed: May 4, 2026Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p1.1)\.
- \[21\]A\. F\. Shorrocks\(1980\)The class of additively decomposable inequality measures\.Econometrica: Journal of the Econometric Society,pp\. 613–625\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p3.1),[§4\.4](https://arxiv.org/html/2608.20047#S4.SS4.p1.1)\.
- \[22\]I\. Shumailov, Z\. Shumaylov, Y\. Zhao, N\. Papernot, R\. Anderson, and Y\. Gal\(2024\)AI models collapse when trained on recursively generated data\.Nature631\(8022\),pp\. 755–759\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p1.1)\.
- \[23\]S\. Singh, F\. Vargus, D\. Dsouza, B\. F\. Karlsson, A\. Mahendiran, W\. Ko, H\. Shandilya, J\. Patel, D\. Mataciunas, L\. OMahony, M\. Zhang, R\. Hettiarachchi, J\. Wilson, M\. Machado, L\. S\. Moura, D\. Krzemiński, H\. Fadaei, I\. Ergün, I\. Okoh, A\. Alaagib, O\. Mudannayake, Z\. Alyafeai, V\. M\. Chien, S\. Ruder, S\. Guthikonda, E\. A\. Alghamdi, S\. Gehrmann, N\. Muennighoff, M\. Bartolo, J\. Kreutzer, A\. Üstün, M\. Fadaee, and S\. Hooker\(2024\)Aya dataset: an open\-access collection for multilingual instruction tuning\.External Links:2402\.06619Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p4.1)\.
- \[24\]G\. Spitale, N\. Biller\-Andorno, and F\. Germani\(2023\)AI model gpt\-3 \(dis\) informs us better than humans\.Science Advances9\(26\),pp\. eadh1850\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p1.1)\.
- \[25\]S\. Tu, Y\. Sun, Y\. Bai, J\. Yu, L\. Hou, and J\. Li\(2024\)Waterbench: towards holistic evaluation of watermarks for large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1517–1542\.Cited by:[§1](https://arxiv.org/html/2608.20047#S1.p2.1),[§2](https://arxiv.org/html/2608.20047#S2.p2.1)\.
- \[26\]Y\. Wu, Z\. Hu, J\. Guo, H\. Zhang, and H\. Huang\(2023\)A resilient and accessible distribution\-preserving watermark for large language models\.arXiv preprint arXiv:2310\.07710\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p1.1),[§3](https://arxiv.org/html/2608.20047#S3.p3.1)\.
- \[27\]A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.\(2024\)Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§3](https://arxiv.org/html/2608.20047#S3.p4.1)\.
- \[28\]T\. Zhang, V\. Kishore, F\. Wu, K\. Q\. Weinberger, and Y\. Artzi\(2019\)Bertscore: evaluating text generation with bert\.arXiv preprint arXiv:1904\.09675\.Cited by:[Table 13](https://arxiv.org/html/2608.20047#A4.T13),[Table 13](https://arxiv.org/html/2608.20047#A4.T13.20),[§4\.3](https://arxiv.org/html/2608.20047#S4.SS3.p3.1)\.
- \[29\]X\. Zhao, P\. Ananth, L\. Li, and Y\. Wang\(2023\)Provable robust watermarking for ai\-generated text\.arXiv preprint arXiv:2306\.17439\.Cited by:[§2](https://arxiv.org/html/2608.20047#S2.p1.1),[§3](https://arxiv.org/html/2608.20047#S3.p3.1)\.

## Appendix ACompute

Compute\.All generation and evaluation runs were conducted on a mix of NVIDIA H100 and A100 GPUs\.

## Appendix BGeneration Preprocessing

### B\.1Language adherence

Table[4](https://arxiv.org/html/2608.20047#A2.T4)reports per\-cell on\-target rates from GlotLID\-v3 for nowatermark generations \(NWM%\) and the mean shift across the six watermarked schemes \(Δ¯\\bar\{\\Delta\}, percentage points\)\. Baseline adherence varies systematically: generator\-mean rates run from 80\.9% \(Mistral instruct\) to 96\.5% \(Mistral base\), with cell\-level minima at Qwen\-Turkish base \(57\.2%\) and Mistral\-Hindi instruct \(52\.2%\)\. Instruction\-tuning does not uniformly raise adherence; Mistral and Qwen lose ground on several non\-Latin\-script cells while Gemma remains roughly stable\.

Watermark\-induced shifts are modest in base \(2 of 33 cells exceed\|Δ¯\|=5\|\\bar\{\\Delta\}\|=5pp\) and larger in instruct \(12 of 33\), concentrated on Mistral and Qwen non\-Latin cells while Gemma instruct stays close to its baseline \(Δ¯=−0\.4\\bar\{\\Delta\}=\-0\.4pp on average\)\. Three instruct cells lose more than 13 points: Mistral\-Arabic \(−15\.6\-15\.6\), Qwen\-Arabic \(−15\.0\-15\.0\), and Qwen\-Turkish \(−13\.5\-13\.5\)\. These are the cells where the on\-target subset thins toward then=200n=200sample\-size floor forα=0\.01\\alpha=0\.01calibration and produce the largest subset\-all\-vs\-on\-target gaps in the detection robustness analysis\.

Base regime \(FLORES\+ continuation prompts\)Instruct regime \(AYA native\-speaker prompts\)MistralGemmaQwenMistralGemmaQwenLangNWM%Δ¯\\bar\{\\Delta\}NWM%Δ¯\\bar\{\\Delta\}NWM%Δ¯\\bar\{\\Delta\}NWM%Δ¯\\bar\{\\Delta\}NWM%Δ¯\\bar\{\\Delta\}NWM%Δ¯\\bar\{\\Delta\}eng99\.6−0\.3\-0\.398\.8−2\.0\-2\.098\.4−0\.6\-0\.692\.6−0\.9\-0\.980\.6−1\.7\-1\.793\.2−2\.0\-2\.0fra99\.2−1\.1\-1\.197\.6−4\.0\-4\.089\.2−0\.2\-0\.291\.4\+0\.5\+0\.586\.2−1\.7\-1\.794\.8−4\.9\-4\.9spa100\.0−1\.4\-1\.496\.8−2\.6\-2\.691\.2\+1\.1\+1\.195\.2−1\.3\-1\.362\.6\+2\.7\+2\.792\.0−5\.0\-5\.0por99\.2−0\.5\-0\.596\.8−1\.0\-1\.098\.2−0\.8\-0\.896\.2−2\.0\-2\.070\.6\+3\.0\+3\.096\.4−4\.3\-4\.3nld96\.2−7\.0\-7\.095\.8\+1\.6\+1\.680\.8−4\.6\-4\.673\.2−3\.1\-3\.189\.2−0\.4\-0\.490\.4−5\.8\-5\.8zho97\.6−1\.8\-1\.892\.8−1\.2\-1\.296\.4−4\.2\-4\.289\.0−7\.9\-7\.995\.6−0\.4\-0\.498\.4−0\.6\-0\.6jpn96\.8−3\.3\-3\.395\.6−1\.6\-1\.699\.8−3\.3\-3\.371\.0−10\.7\-10\.794\.0−1\.9\-1\.998\.4−7\.4\-7\.4arb89\.2−3\.0\-3\.087\.6−3\.8\-3\.887\.8\+2\.9\+2\.967\.6−15\.6\-15\.684\.2−1\.2\-1\.281\.0−15\.0\-15\.0hin96\.2−9\.1\-9\.188\.2−2\.3\-2\.398\.8−3\.9\-3\.952\.2−6\.5\-6\.583\.8−1\.4\-1\.483\.2\+0\.1\+0\.1tur89\.0\+1\.3\+1\.396\.8−4\.3\-4\.357\.2−1\.1\-1\.181\.8−8\.1\-8\.192\.8−0\.6\-0\.695\.6−13\.5\-13\.5vie98\.6−4\.7\-4\.777\.0\+4\.7\+4\.799\.0−1\.0\-1\.079\.2−7\.3\-7\.398\.8−0\.6\-0\.695\.2−6\.4\-6\.4mean96\.5−2\.8\-2\.893\.1−1\.5\-1\.590\.6−1\.4\-1\.480\.9−5\.7\-5\.785\.3−0\.4\-0\.492\.6−5\.9\-5\.9Table 4:Per\-cell language adherence on the eleven evaluation languages, base\-FLORES \(left\) and instruct\-AYA \(right\)\. NWM% is the GlotLID\-v3 on\-target rate for nowatermark generations;Δ¯\\bar\{\\Delta\}is the mean shift in percentage points across the six watermarked schemes\.meanrows are unweighted means across the eleven languages\.
### B\.2Generation length statistics

Generations sit close to the maximum\-token cap in the base regime where language\-mean lengths are196\.7±16\.2196\.7\\pm 16\.2\(Mistral\),196\.6±16\.8196\.6\\pm 16\.8\(Gemma\), and187\.3±26\.6187\.3\\pm 26\.6\(Qwen\) tokens and shift to shorter, more dispersed distributions under instruction tuning \(167\.7±37\.8167\.7\\pm 37\.8,174\.6±39\.4174\.6\\pm 39\.4,180\.2±32\.0180\.2\\pm 32\.0respectively\)\. Per\-cell variation is largest on Mistral instruct, where mean lengths range from 152\.2 tokens \(Portuguese\) to 183\.2 \(English\)\.

Table 5:Length\-distribution similarity between LID subsets and the all\-subset baseline\. Each row is a \(regime, subset\) pair; ratio==subset\-mean length/all\-subset mean length, computed per \(regime, model, language, watermark\) cell and summarized across cells\. Cells requiren≥100n\\geq 100in both subsets to enter the summary;nnskippedcounts cells that fail this floor \(typically off\_target on high\-adherence cells\)\.Table[5](https://arxiv.org/html/2608.20047#A2.T5)reports the ratio of subset\-conditioned mean length to all\-subset mean length, computed per \(regime, model, language, watermark\) cell withn≥100n\\geq 100in both subsets\. On\-target subset lengths are within 0\.5% of all\-subset means in aggregate \(median ratio1\.0001\.000in both regimes; per\-cell range0\.9640\.964\-1\.0811\.081\), so the on\-target stratification used downstream is not confounded by systematic length differences\. The off\-target subset clears then≥100n\\geq 100floor on only 8 of 231 base cells and 26 of 231 instruct cells\. This is a consequence of high baseline adherence outside the cells flagged in §[B\.1](https://arxiv.org/html/2608.20047#A2.SS1)but where measurable its mean lengths track all\-subset means within∼7\\sim 7%\.

### B\.3Categorization robustness

The on\-target categorization depends on two thresholds: a GlotLID per\-sentence confidence floor \(conf\_floor\) and a per\-text on\-target sentence fraction \(on\_target\_ratio\)\. Table[6](https://arxiv.org/html/2608.20047#A2.T6)sweeps these on a3×33\\times 3grid and reports Spearman rank\-correlation against the default\-config per\-cell adherence vector\. Rank\-correlation stays≥0\.86\\geq 0\.86across all eight alternatives, and the mean on\-target rate across cells moves by at most 7\.3 pp \(82\.5%82\.5\\%at the strictest config to92\.0%92\.0\\%at the most permissive\)\. The cell\-level adherence patterns that drive §[B\.1](https://arxiv.org/html/2608.20047#A2.SS1)are not knife\-edge sensitive to the threshold choice\.

Table 6:LID categorization robustness: per\-cell adherence under a3×33\\times 3sweep of the GlotLID per\-sentence confidence floor \(conf\_floor\) and the per\-text on\-target sentence fraction \(on\_target\_ratio\)\.ρ\\rhois the Spearman rank\-correlation between the per\-cell on\-target\-rate vector under each configuration and the default \(†\\dagger\)\.Mean on\-target %is the unweighted mean across the 462 cells of the evaluation grid\.Mean/Max\|Δ\|\|\\Delta\|are the mean and maximum absolute per\-cell shift in on\-target rate \(pp\) relative to default\.

## Appendix CDetection: Per\-Language Detail and Robustness

This appendix supplies the detection\-side material that the main text’s headline summary \(Table[2](https://arxiv.org/html/2608.20047#S5.T2)\) and §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)collapse or defer\. It is organized around four questions, each answered by one or two tables\. First, what does the per\-language vector behind each headline cell actually look like \(Figures[4\(a\)](https://arxiv.org/html/2608.20047#A3.F4.sf1)and[4\(b\)](https://arxiv.org/html/2608.20047#A3.F4.sf2), Table[7](https://arxiv.org/html/2608.20047#A3.T7)\)? Second, do the calibration\-failure cells flagged in §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)survive alternative operating choices on the FPR target and the language\-adherence subset \(Tables[8](https://arxiv.org/html/2608.20047#A3.T8),[9](https://arxiv.org/html/2608.20047#A3.T9)\)? Third, do the per\-cell disparity statistics depend on the choice of anchor language for DI and on whether Min TPR is read as a single\-language outlier or a small low\-performing cluster \(Table[10](https://arxiv.org/html/2608.20047#A3.T10)\)? Fourth, does the structural\-disparity claim of §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)survive swapping the inequality measure \(GE2→\{\}\_\{2\}\\toGE0\) or the typological partition \(family→\\toscript\) used in the decomposition \(Tables[11](https://arxiv.org/html/2608.20047#A3.T11),[12](https://arxiv.org/html/2608.20047#A3.T12)\)? Each table’s caption carries the cell\-specific findings; the prose below signposts which group a given table belongs to and what it adjudicates\.

Table 7:Per\-language calibration gapΔl=TPRl​\(τl\)−TPRl​\(τg\)\\Delta\_\{l\}=\\mathrm\{TPR\}\_\{l\}\(\\tau\_\{l\}\)\-\\mathrm\{TPR\}\_\{l\}\(\\tau\_\{g\}\), base\-FLORES \(top\) and instruct\-AYA \(bottom\), subsetall,α=0\.01\\alpha=0\.01\. Positive: per\-language calibration recovers detection\. Negative: per\-language calibration worsens detection\. Zero: detection insensitive to calibration locality\. Row means equal−1×\-1\\timestheΔ\\Deltacolumn of Table[2](https://arxiv.org/html/2608.20047#S5.T2)\.![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_d_5_1_base.png)\(a\)Base\-FLORES regime\.
![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_d_5_1_instruct.png)\(b\)Instruct\-AYA regime\.

Figure 4:Per\-language detection diagnostics, subsetall,α=0\.01\\alpha=0\.01\. Within each panel, left: TPR at the global empirical\-FPR thresholdτg\\tau\_\{g\}; right: threshold\-free AUC\. Rows are \(scheme, generator\) cells; columns are the eleven evaluation languages, grouped by typological family\.### C\.1Per\-language detail

Figures[4\(a\)](https://arxiv.org/html/2608.20047#A3.F4.sf1)and[4\(b\)](https://arxiv.org/html/2608.20047#A3.F4.sf2)render the per\-language TPR and AUC vectors for both regimes\. The heatmap is the visual entry point to every cell\-language pattern that the headline table’s mean, min, and GE2statistics collapse: the Unigram\-Gemma all\-zero row under base, the XSIR\-Qwen Hindi outlier, the EXPEdit\-Gemma broad AUC degradation under instruct\. Table[7](https://arxiv.org/html/2608.20047#A3.T7)then shows the per\-languageΔl=TPRl​\(τl\)−TPRl​\(τg\)\\Delta\_\{l\}=\\mathrm\{TPR\}\_\{l\}\(\\tau\_\{l\}\)\-\\mathrm\{TPR\}\_\{l\}\(\\tau\_\{g\}\)matrix, decomposing the headlineΔ\\Deltacolumn into its per\-language sources\. Together the two items establish that calibration locality is doing structurally different work in different cells: a near\-noise correction of order±0\.03\\pm 0\.03on strong\-detection cells, a near\-total recovery on the base Unigram\-Gemma row, and a smaller\-scale broad recovery across most instruct cells\.

### C\.2Detection\-threshold robustness

The calibration\-failure diagnoses in §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)depend on two operating choices: the FPR targetα\\alphaand the LID\-adherence subset on which the null is calibrated\. Table[8](https://arxiv.org/html/2608.20047#A3.T8)repeats the headline summary atα=0\.05\\alpha=0\.05, and Table[9](https://arxiv.org/html/2608.20047#A3.T9)restricts the unwatermarked null to LID\-passing on\-target generations\. The first table addresses whether the score\-cap saturation underlying Unigram\-Gemma atα=0\.01\\alpha=0\.01persists at a looser FPR target; the second addresses whether that saturation is driven by off\-target generations contaminating the calibration pool\. Read together, the two robustness checks distinguish two distinct base\-regime failure modes: Unigram\-Gemma is an off\-target\-mixture artifact that dissolves under either operating change, while XSIR\-Qwen is a deeper threshold misalignment that persists across both\.

TPR atτg\\tau\_\{g\}\(global\)TPR atτl\\tau\_\{l\}\(per\-lang\)SchemeGen\.Mean TPRMin TPRDI<\.8GE2Btw\. %Mean TPRΔ\\DeltaBase regime \(FLORES\+ continuation prompts\)KGWMistral0\.9970\.9920<0\.001<\\\!0\.00161\.10\.997\+0\.000\+0\.000Gemma0\.9910\.9800<0\.001<\\\!0\.00193\.20\.993−0\.002\-0\.002Qwen0\.9870\.9520<0\.001<\\\!0\.00157\.80\.986\+0\.000\+0\.000UnigramMistral0\.9770\.9200<0\.001<\\\!0\.00154\.90\.988−0\.011\-0\.011Gemma0\.9880\.9760<0\.001<\\\!0\.00155\.90\.898\+0\.089\+0\.089Qwen0\.9610\.86000\.00186\.30\.979−0\.018\-0\.018SynthIDMistral1\.0000\.9980<0\.001<\\\!0\.001100\.01\.000\+0\.000\+0\.000Gemma1\.0000\.9980<0\.001<\\\!0\.001100\.01\.000\+0\.000\+0\.000Qwen0\.9990\.9960<0\.001<\\\!0\.00174\.30\.999\+0\.000\+0\.000EXPEditMistral1\.0000\.9960<0\.001<\\\!0\.001100\.01\.000\+0\.000\+0\.000Gemma1\.0000\.9980<0\.001<\\\!0\.001100\.01\.000\+0\.000\+0\.000Qwen0\.9950\.9660<0\.001<\\\!0\.00199\.20\.995−0\.000\-0\.000DIPMistral0\.9900\.9720<0\.001<\\\!0\.00194\.70\.989\+0\.001\+0\.001Gemma0\.9810\.9420<0\.001<\\\!0\.00197\.90\.981−0\.001\-0\.001Qwen0\.9660\.89600\.00191\.60\.966\+0\.000\+0\.000XSIRMistral0\.9240\.86800\.00183\.30\.935−0\.011\-0\.011Gemma0\.9290\.84000\.00192\.80\.944−0\.015\-0\.015Qwen0\.5030\.27690\.07694\.60\.919−0\.416\-0\.416Instruct regime \(AYA native\-speaker prompts\)KGWMistral0\.8010\.62220\.00685\.00\.813−0\.012\-0\.012Gemma0\.6160\.474100\.01579\.70\.617−0\.001\-0\.001Qwen0\.8600\.74010\.00378\.10\.869−0\.009\-0\.009UnigramMistral0\.5170\.09050\.14995\.80\.659−0\.143\-0\.143Gemma0\.3800\.15450\.05080\.30\.408−0\.028\-0\.028Qwen0\.7270\.36640\.01893\.80\.790−0\.063\-0\.063SynthIDMistral0\.9670\.9260<0\.001<\\\!0\.00177\.80\.969−0\.002\-0\.002Gemma0\.7910\.66240\.00575\.90\.785\+0\.005\+0\.005Qwen0\.9710\.8980<0\.001<\\\!0\.00195\.70\.973−0\.002\-0\.002EXPEditMistral0\.6650\.35440\.03676\.70\.682−0\.017\-0\.017Gemma0\.3410\.126100\.04983\.30\.339\+0\.002\+0\.002Qwen0\.5830\.23090\.05082\.50\.596−0\.013\-0\.013DIPMistral0\.6660\.47860\.01798\.80\.672−0\.006\-0\.006Gemma0\.4100\.302100\.03193\.00\.416−0\.007\-0\.007Qwen0\.6980\.51070\.01299\.70\.700−0\.002\-0\.002XSIRMistral0\.5100\.25690\.05186\.40\.523−0\.013\-0\.013Gemma0\.2670\.072100\.09294\.30\.300−0\.033\-0\.033Qwen0\.1690\.024100\.81197\.50\.513−0\.344\-0\.344Table 8:Detection fairness summary atα=0\.05\\alpha=0\.05, base\-FLORES \(top\) and instruct\-AYA \(bottom\), subsetall\. Format mirrors Table[2](https://arxiv.org/html/2608.20047#S5.T2); AUC columns are omitted \(threshold\-independent, identical to Table[2](https://arxiv.org/html/2608.20047#S5.T2)\) and bootstrap CIs on Mean TPR are omitted for compactness\.L=11L=11throughout\. Values below10−310^\{\-3\}reported as<0\.001<\\\!0\.001\.Base regime \(FLORES\+ continuation\)Instruct regime \(AYA native\-speaker\)SchemeGen\.Mean TPRMin TPRDI<\.8Δ\\Deltavs\.allMean TPRMin TPRDI<\.8Δ\\Deltavs\.allKGWMistral0\.9950\.9830\+0\.003\+0\.0030\.6810\.4256\+0\.055\+0\.055Gemma0\.9730\.9390\+0\.004\+0\.0040\.4260\.26410\+0\.037\+0\.037Qwen0\.9870\.9630\+0\.022\+0\.0220\.6970\.5208\+0\.008\+0\.008UnigramMistral0\.8850\.6542\+0\.017\+0\.0170\.3880\.0117\+0\.077\+0\.077Gemma0\.9870\.9740\+0\.987\+0\.9870\.0200\.0009−0\.017\-0\.017Qwen0\.9250\.7621\+0\.027\+0\.0270\.5470\.1764\+0\.001\+0\.001SynthIDMistral1\.0000\.99800\.0000\.0000\.9240\.7741\+0\.019\+0\.019Gemma1\.0000\.99800\.0000\.0000\.6100\.42610\+0\.008\+0\.008Qwen0\.9990\.9960\+0\.002\+0\.0020\.9410\.76410\.0000\.000EXPEditMistral1\.0001\.0000\+0\.001\+0\.0010\.6750\.2345\+0\.098\+0\.098Gemma1\.0000\.99700\.0000\.0000\.2320\.09110\+0\.007\+0\.007Qwen0\.9940\.9410\+0\.003\+0\.0030\.5080\.05910\+0\.012\+0\.012DIPMistral0\.9800\.9520\+0\.005\+0\.0050\.4880\.2669\+0\.033\+0\.033Gemma0\.9770\.9540\+0\.014\+0\.0140\.1980\.10510\+0\.002\+0\.002Qwen0\.9570\.8480\+0\.026\+0\.0260\.5070\.2008−0\.006\-0\.006XSIRMistral0\.7990\.6882\+0\.015\+0\.0150\.2960\.08110\+0\.030\+0\.030Gemma0\.7930\.6003\+0\.027\+0\.0270\.0920\.01910\+0\.011\+0\.011Qwen0\.2070\.04710−0\.008\-0\.0080\.0470\.00010\+0\.004\+0\.004Table 9:Detection summary on the on\-target subset,α=0\.01\\alpha=0\.01, global thresholdτg\\tau\_\{g\}, base\-FLORES \(left\) and instruct\-AYA \(right\)\. Thresholds re\-calibrated on each cell’s on\-target unwatermarked pool;L=11L=11throughout\. The off\-target subset is omitted: in the base regime no cell meets thenneg≥200n\_\{\\text\{neg\}\}\\geq 200inclusion floor, and in the instruct regime only six \(Mistral\-generator\) cells pass the relaxedα=0\.05\\alpha=0\.05floor withL≤3L\\leq 3, insufficient for the fairness aggregates this appendix reports\. TheΔ\\Deltavs\.allcolumns report change in Mean TPR relative to subsetall\(Table[2](https://arxiv.org/html/2608.20047#S5.T2)\)\.
### C\.3Floor\-statistic robustness

The headline table reports Min TPR as a worst\-language floor, which is sensitive to isolated single\-language outliers\. Table[10](https://arxiv.org/html/2608.20047#A3.T10)replaces Min TPR with the mean of the bottom two languages \(the bottom quintile atL=11L=11\), testing whether the headline minima are driven by isolated outliers or by low\-performing clusters\. The bottom\-quintile mean and Min TPR identify the same worst\-performing cells in both regimes, with the bottom\-quintile values typically within0\.050\.05of Min TPR on cells where the floor is genuine and substantially higher only on the handful of cells where a single language sits well below the rest\. The check supports the headline floor statistic rather than displacing it\.

Table 10:Bottom\-quintile\-mean Rawlsian floor as a stability companion to Min TPR \(Table[2](https://arxiv.org/html/2608.20047#S5.T2)\), base\-FLORES \(left\) and instruct\-AYA \(right\), subsetall,α=0\.01\\alpha=0\.01, global thresholdτg\\tau\_\{g\}\. WithL=11L=11evaluation languages the bottom quintile is⌊0\.2⋅11⌋=2\\lfloor 0\.2\\cdot 11\\rfloor=2languages; the columns report the mean TPR of the two worst\-performing languages per cell, with the languages listed minimum first \(arbitrary tie\-breaking\)\. The Unigram\-Gemma base cell is degenerate \(all eleven per\-language TPRs are zero\); its bottom\-quintile language identities are reported but uninformative\.
### C\.4Decomposition robustness

The structural\-disparity claim of §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)rests on a single decomposition: GE2over the typological\-family partition\. Table[11](https://arxiv.org/html/2608.20047#A3.T11)repeats it under GE0\(mean log deviation, more sensitive to inequality at the low end of the per\-language vector\), and Table[12](https://arxiv.org/html/2608.20047#A3.T12)repeats it under a script\-based partition\. Between\-family share remains the dominant component \(\>\>65% in every well\-defined cell\) under both inequality measures across both regimes\. The family partition consistently captures more between\-group variance than the script partition for cells with non\-trivial total disparity, locating the structural component at the typology level rather than the writing\-system level\.

Table 11:Robustness of the generalized\-entropy decomposition to choice of inequality measure:GE2\\mathrm\{GE\}\_\{2\}\(half the squared coefficient of variation, more sensitive to inequality at the top of the per\-language TPR distribution\) versusGE0\\mathrm\{GE\}\_\{0\}\(mean log deviation, more sensitive to inequality at the bottom\)\. Base\-FLORES \(left\) and instruct\-AYA \(right\), subsetall,α=0\.01\\alpha=0\.01, global thresholdτg\\tau\_\{g\}; typological partition as in Table[2](https://arxiv.org/html/2608.20047#S5.T2)\. Total values below10−310^\{\-3\}reported as<0\.001<\\\!0\.001\. The Unigram\-Gemma baseGE0\\mathrm\{GE\}\_\{0\}entry is obtained by replacing zero TPRs with a smallϵ\\epsilon\(per the strict\-positivity requirement ofGE0\\mathrm\{GE\}\_\{0\}\); the resultingTotal=0\\mathrm\{Total\}=0andBtw\.%=100\.0\\mathrm\{Btw\.\\,\\%\}=100\.0are regularization artifacts, marked†\.Table 12:Robustness of theGE2\\mathrm\{GE\}\_\{2\}between\-share to choice of partition: typological family \(8 groups: Germanic, Romance, Indic, Semitic, Sinitic, Japonic, Turkic, Austroasiatic, with 5 of 11 languages in non\-singleton groups\) vs\. script \(4 groups: Latin pooling eng, nld, fra, spa, por, tur, vie; Devanagari==hin; Arabic==arb; Han pooling zho, jpn; with 9 of 11 languages in non\-singleton groups\)\. Base\-FLORES \(left\) and instruct\-AYA \(right\), subsetall,α=0\.01\\alpha=0\.01, global thresholdτg\\tau\_\{g\}\.GE2\\mathrm\{GE\}\_\{2\}Total depends only on the per\-language vector and is partition\-independent \(shown once per regime\)\. Totals below10−310^\{\-3\}reported as<0\.001<\\\!0\.001; for these cells the between\-share values involve ratios of near\-zero variance components and should not be over\-interpreted\.

## Appendix DQuality: Per\-Paradigm Detail and Robustness

### D\.1Per\-cell quality fairness summary

MAUVEBERTScore F1 \(rescaled\)PPL preservation \(XGLM\-7\.5B\)SchemeGenmean \(min\)DI<<\.8GE2Btw\. %mean \(min\)mean \(min\)DI<<\.8GE2Btw\. %Base regime \(FLORES\+ continuation\)KGWMistral0\.947 \(0\.879\)jpn00\.00186\.0\-0\.041 \(\-0\.079\)vie0\.866 \(0\.793\)spa00\.00266\.4KGWGemma0\.855 \(0\.435\)arb10\.01398\.9\-0\.081 \(\-0\.194\)vie0\.899 \(0\.812\)fra00\.00291\.0KGWQwen0\.889 \(0\.471\)hin10\.01295\.6\-0\.034 \(\-0\.091\)tur0\.834 \(0\.686\)hin30\.00594\.0UnigramMistral0\.747 \(0\.420\)arb50\.02649\.5\-0\.070 \(\-0\.134\)arb0\.872 \(0\.811\)spa00\.00296\.4UnigramGemma0\.638 \(0\.444\)spa60\.01767\.8\-0\.098 \(\-0\.191\)jpn0\.911 \(0\.815\)nld00\.00268\.3UnigramQwen0\.734 \(0\.123\)hin30\.04397\.2\-0\.055 \(\-0\.128\)zho0\.896 \(0\.647\)hin10\.00598\.0SynthIDMistral0\.187 \(0\.020\)arb100\.43559\.0\-0\.115 \(\-0\.163\)vie0\.299 \(0\.145\)arb70\.04888\.4SynthIDGemma0\.095 \(0\.029\)tur100\.64143\.0\-0\.146 \(\-0\.254\)vie0\.269 \(0\.195\)tur70\.02173\.2SynthIDQwen0\.183 \(0\.035\)vie100\.45553\.0\-0\.112 \(\-0\.188\)tur0\.308 \(0\.203\)arb90\.02878\.2EXPEditMistral0\.079 \(0\.006\)arb100\.53657\.1\-0\.149 \(\-0\.228\)vie0\.218 \(0\.079\)arb60\.08599\.1EXPEditGemma0\.013 \(0\.006\)jpn100\.57443\.4\-0\.220 \(\-0\.333\)vie0\.135 \(0\.076\)zho90\.03382\.7EXPEditQwen0\.067 \(0\.013\)arb100\.87447\.9\-0\.145 \(\-0\.245\)tur0\.224 \(0\.135\)tur90\.04274\.6DIPMistral0\.973 \(0\.933\)hin00\.00095\.6\-0\.015 \(\-0\.040\)vie0\.809 \(0\.750\)hin00\.00177\.6DIPGemma0\.932 \(0\.878\)jpn00\.00194\.7\-0\.053 \(\-0\.183\)vie0\.774 \(0\.702\)nld00\.00168\.0DIPQwen0\.947 \(0\.864\)hin00\.00181\.6\-0\.008 \(\-0\.071\)tur0\.811 \(0\.714\)tur00\.00292\.7XSIRMistral0\.859 \(0\.642\)hin20\.00686\.6\-0\.062 \(\-0\.110\)hin0\.922 \(0\.816\)nld00\.00273\.2XSIRGemma0\.683 \(0\.461\)fra60\.01880\.9\-0\.123 \(\-0\.226\)vie0\.925 \(0\.785\)nld10\.00275\.8XSIRQwen0\.850 \(0\.603\)hin20\.00889\.9\-0\.042 \(\-0\.129\)tur0\.883 \(0\.745\)hin10\.00386\.4Instruct regime \(AYA native\-speaker\)KGWMistral0\.989 \(0\.970\)arb00\.00058\.90\.255 \(0\.088\)arb0\.892 \(0\.853\)jpn00\.00191\.0KGWGemma0\.990 \(0\.959\)arb00\.00065\.30\.397 \(0\.338\)tur0\.967 \(0\.941\)hin00\.00080\.0KGWQwen0\.987 \(0\.954\)hin00\.00090\.80\.300 \(0\.192\)tur0\.885 \(0\.749\)hin10\.00292\.5UnigramMistral0\.981 \(0\.938\)nld00\.00040\.30\.240 \(0\.087\)arb0\.897 \(0\.835\)hin00\.00197\.4UnigramGemma0\.996 \(0\.991\)jpn00\.00054\.10\.400 \(0\.329\)tur0\.965 \(0\.941\)nld00\.00078\.0UnigramQwen0\.980 \(0\.927\)hin00\.00097\.70\.297 \(0\.184\)tur0\.885 \(0\.700\)hin10\.00395\.9SynthIDMistral0\.721 \(0\.066\)arb40\.10799\.60\.149 \(\-0\.117\)arb0\.595 \(0\.185\)arb70\.07095\.4SynthIDGemma0\.994 \(0\.979\)fra00\.00050\.00\.365 \(0\.296\)tur0\.931 \(0\.885\)zho00\.00090\.0SynthIDQwen0\.833 \(0\.237\)tur20\.03397\.10\.219 \(0\.047\)tur0\.645 \(0\.442\)tur50\.01490\.7EXPEditMistral0\.480 \(0\.023\)arb100\.19886\.60\.082 \(\-0\.169\)arb0\.549 \(0\.198\)arb70\.07095\.7EXPEditGemma0\.840 \(0\.598\)arb30\.01290\.00\.319 \(0\.265\)jpn0\.742 \(0\.669\)arb00\.00178\.7EXPEditQwen0\.699 \(0\.170\)tur40\.06380\.60\.174 \(0\.016\)tur0\.651 \(0\.437\)tur50\.01499\.1DIPMistral0\.992 \(0\.956\)arb00\.00096\.60\.259 \(0\.108\)arb0\.937 \(0\.836\)arb00\.00194\.6DIPGemma0\.997 \(0\.991\)fra00\.00052\.20\.403 \(0\.328\)tur0\.982 \(0\.946\)nld00\.00061\.7DIPQwen0\.994 \(0\.985\)fra00\.00028\.00\.318 \(0\.210\)tur0\.925 \(0\.853\)tur00\.00194\.6XSIRMistral0\.991 \(0\.975\)arb00\.00091\.90\.258 \(0\.112\)arb0\.928 \(0\.876\)hin00\.00097\.7XSIRGemma0\.995 \(0\.975\)fra00\.00031\.10\.396 \(0\.333\)tur0\.979 \(0\.969\)tur00\.00044\.6XSIRQwen0\.991 \(0\.980\)tur00\.00090\.60\.311 \(0\.197\)tur0\.902 \(0\.826\)tur00\.00184\.3

Table 13:Quality fairness summary by scheme, generator, regime, and measurement, subsetall\. Each row reports mean per\-language preservation with the per\-cell minimum and floor language \(ISO 639\-3 superscript\) in parentheses\. DI<<\.8 counts languages with per\-language preservation below0\.8×maxl0\.8\\times\\max\_\{l\}\. GE2uses the typological partition \(Germanic, Romance, Indic, Semitic, Sinitic, Japonic, Turkic, Austroasiatic\); Btw\. % is its between\-family share\. BERTScore mean \(min\) is reported on rescaled F1\[[28](https://arxiv.org/html/2608.20047#bib.bib10)\]; DI<<\.8 and GE2are omitted for BERTScore because neither registers signal on the bounded raw scale and both behave pathologically on the rescaled scale \(§[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)\)\.Table[13](https://arxiv.org/html/2608.20047#A4.T13)reports the per\-\(scheme, generator, regime\) quality\-fairness summary that the heatmaps in Figure[1](https://arxiv.org/html/2608.20047#S5.F1)and the per\-scheme decomposition in Figure[2](https://arxiv.org/html/2608.20047#S5.F2)aggregate over\. Three quantities are recoverable here that are not directly readable from those figures: the per\-cell minimum\-language identity \(ISO 639\-3 superscript on each mean–min pair\), the four\-fifths disparate\-impact count DI<<\.8 against the per\-cell maximum, and the GE2total with its between\-family share at the \(scheme, generator\) granularity rather than the per\-scheme generator\-averaged values plotted in Figure[2](https://arxiv.org/html/2608.20047#S5.F2)\. The BERTScore DI<<\.8 and GE2columns are omitted: raw F1 has insufficient dynamic range to register either statistic, and the rescaled F1 used elsewhere is unbounded below and produces pathological ratios under both the four\-fifths anchor and the squared\-coefficient\-of\-variation normalization \(§[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)\)\.

The cell\-level numbers support three observations that the main\-text aggregates state at the scheme level\. First, the floor\-language column shows that the worst\-preserved language within a cell is not constant across \(scheme, generator\): Arabic, Hindi, Turkish, Vietnamese, and Japanese each appear as floor language in at least four cells across the grid, while English never does\. Second, the DI<<\.8 column localizes the distortion\-free collapse of §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)to its cell\-level extent: all six SynthID and EXPEdit base cells flag≥6\\geq 6languages on MAUVE, while all six DIP base cells flag zero\. Third, the per\-cell GE2values expose within\-scheme generator variation that the per\-scheme bars in Figure[2](https://arxiv.org/html/2608.20047#S5.F2)smooth over, the largest case being EXPEdit on MAUVE in the base regime, where Qwen \(0\.874\) sits substantially above Mistral \(0\.536\) and Gemma \(0\.574\) despite all three being labelled “EXPEdit base” in the headline plot\.

### D\.2PPL preservation under mGPT cross\-reference

The cross\-scheme ranking is invariant to reference choice\. Per\-scheme means across the 33 base cells: XSIR \(0\.93 mGPT, 0\.91 XGLM\), Unigram \(0\.92, 0\.89\), KGW \(0\.86, 0\.87\), DIP \(0\.81, 0\.80\), SynthID \(0\.32, 0\.29\), EXPEdit \(0\.22, 0\.19\)\. Per\-cell agreement is close throughout the grid\. The Hindi\-on\-PPL pattern flagged in §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)reproduces under mGPT: KGW, Unigram, and XSIR all preserve Hindi at≥0\.75\\geq 0\.75under both references despite weaker BERTScore and MAUVE preservation in the same cells\. The largest single\-cell shift between references is SynthID\-Mistral on Hindi \(0\.22 XGLM→\\to0\.42 mGPT\); the second\-largest is EXPEdit\-Qwen on Hindi \(0\.23→\\to0\.44\)\. Both shifts are toward higher preservation under mGPT and are concentrated on the schemes already flagged as most distorting, so the cross\-reference comparison reinforces rather than weakens the per\-scheme conclusions of §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)\. Two\-reference agreement supports the §[4](https://arxiv.org/html/2608.20047#S4)design choice of treating PPL as a single paradigm with one primary and one cross\-reference reading\.

![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_q_5_2_ppl_mgpt_base.png)\(a\)Base\-FLORES regime\.
![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_q_5_2_ppl_mgpt_instruct.png)\(b\)Instruct\-AYA regime\.

Figure 5:PPL preservation under mGPT\-1\.3B cross\-reference\. Format and color scale match Figure[1](https://arxiv.org/html/2608.20047#S5.F1)\.
### D\.3On\-target vs\. all\-subset stratification

The headline quality reporting in §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)is computed on subset=allper the pre\-registration\. Figure[6](https://arxiv.org/html/2608.20047#A4.F6)reports the per\-cell shiftΔ=preservationon\-target−preservationall\\Delta=\\text\{preservation\}\_\{\\text\{on\-target\}\}\-\\text\{preservation\}\_\{\\text\{all\}\}across the three paradigms in both regimes\.

In base, shifts fall within±0\.05\\pm 0\.05for the vast majority of cells, with isolated MAUVE exceptions on Unigram\-Gemma, XSIR\-Gemma, and XSIR\-Qwen that do not displace the cross\-scheme rankings of §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)\. In instruct, shifts concentrate on the most\-distorting schemes \(SynthID, EXPEdit\) and the lowest\-adherence cells flagged in Table[4](https://arxiv.org/html/2608.20047#A2.T4)\(Mistral and Qwen on Arabic, Hindi, Turkish, Vietnamese\)\. Two opposing patterns recur on these cells: MAUVE shifts are large and positive \(on\-target restriction raises apparent preservation, indicating that off\-target generations are distributionally far from baseline\), while PPL preservation shifts are predominantly negative \(on\-target generations are distributionally closer to baseline but exhibit lower fluency under the held\-out reference\)\. The bidirectionality reflects that off\-target removal trades distributional similarity against reference\-perplexity fluency in opposite directions on these cells, consistent with the green\-list/sampling\-time mechanism split\. The four logit\-bias schemes show shifts within±0\.05\\pm 0\.05on the great majority of entries in both regimes\.

With the single exception of the EXPEdit\-Mistral, Arabic, BERTScore entry \(omitted for falling below then≥100n\\geq 100paired\-set floor\), the conclusions of §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2)survive on\-target restriction: SynthID and EXPEdit remain the most distorting, the cross\-paradigm disagreement pattern persists, and the typological structure of cross\-language disparity is unchanged\.

![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_q_5_2_subset_delta_base.png)

![Refer to caption](https://arxiv.org/html/2608.20047v1/figure_q_5_2_subset_delta_instruct.png)

Figure 6:On\-target minus all\-subset preservation, base\-FLORES regime \(top\) and instruct\-AYA regime \(bottom\)\. Each cell isΔ=preservationon\-target−preservationall\\Delta=\\text\{preservation\}\_\{\\text\{on\-target\}\}\-\\text\{preservation\}\_\{\\text\{all\}\}for the indicated paradigm; rows are \(scheme, generator\), columns are languages\. Positive \(red\): on\-target restriction raises preservation; negative \(blue\): on\-target restriction lowers it\. Color scale fixed at\[−0\.3,\+0\.3\]\[\-0\.3,\+0\.3\]; values outside this range saturate\. The EXPEdit\-Mistral, Arabic, BERTScore F1 entry in the instruct panel is omitted \(—\) because the on\-target sample size falls below then≥100n\\geq 100paired\-set floor\.

## Appendix EJoint Detection\-Quality: Full Paradigm Grid

Figure[7](https://arxiv.org/html/2608.20047#A5.F7)reports the BERTScore\-rescaled and PPL\-XGLM companion panels to Figure[3](https://arxiv.org/html/2608.20047#S5.F3)\. The topology of §[5\.3](https://arxiv.org/html/2608.20047#S5.SS3)carries through on both paradigms\. Unigram and XSIR retain the highest base\-regime spread on every paradigm \(UnigramsB∈\[0\.484,0\.551\]s\_\{B\}\\in\[0\.484,0\.551\]; XSIRsB∈\[0\.354,0\.399\]s\_\{B\}\\in\[0\.354,0\.399\]\), with the four remaining schemes atsB≤0\.195s\_\{B\}\\leq 0\.195\. EXPEdit, SynthID, and Unigram lead the instruct\-regime spread on every paradigm, and XSIR carries the lowest instruct spread on every paradigm \(sI∈\[0\.181,0\.241\]s\_\{I\}\\in\[0\.181,0\.241\]\)\. The opposite\-corner low\-spread topologies of §[5\.3](https://arxiv.org/html/2608.20047#S5.SS3)\(DIP top\-right, EXPEdit bottom\-right\) reproduce on the BERTScore and PPL panels: DIP basesB≤0\.127s\_\{B\}\\leq 0\.127across paradigms, EXPEdit basesB≤0\.195s\_\{B\}\\leq 0\.195\. The headline finding is that joint disparity is regime\-shaped rather than scheme\-uniform is therefore not specific to MAUVE\.

Figure 7:Joint per\-language detection–quality landscape, BERTScore\-rescaled \(top\) and PPL\-XGLM \(bottom\) paradigms\. Format matches Figure[3](https://arxiv.org/html/2608.20047#S5.F3): markers are per\-language\(TPR​@​τg,quality\)\(\\mathrm\{TPR\}\\,@\\,\\tau\_\{g\},\\mathrm\{quality\}\)pairs atα=0\.01\\alpha\{=\}0\.01,n=33n\{=\}33per regime \(11 languages×\\times3 generators\); circles are base\-FLORES, triangles are instruct\-AYA; color encodes typological family;sBs\_\{B\},sIs\_\{I\}are mean pairwise Euclidean distance within each regime\.
## Appendix FMechanism Regression

The decompositions of §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)\-§[5\.3](https://arxiv.org/html/2608.20047#S5.SS3)establish that disparity is typology\-structured but do not identify which structural variable does the work\. This appendix reports per\-language OLS fits of detection TPR and quality preservation on four candidate covariates: tokenizer fertility \(tokens per FLORES\-devtest sentence\), unwatermarked\-generation surprisal under XGLM\-7\.5B, a Latin\-script indicator, and language\-adherence rate \(NWM% from Table[4](https://arxiv.org/html/2608.20047#A2.T4)\)\. Tables[14](https://arxiv.org/html/2608.20047#A6.T14)and[15](https://arxiv.org/html/2608.20047#A6.T15)report per\-regime single\-covariateR2R^\{2\}over 33 \(language, generator\) points; §[F\.1](https://arxiv.org/html/2608.20047#A6.SS1)repeats the quality\-side analysis on BERTScore\-rescaled and PPL\-XGLM as paradigm\-stability checks on the MAUVE headline\.

Detection and quality have different mechanisms\.In base, adherence is the dominant detection predictor for KGW, SynthID, and DIP \(R2R^\{2\}between0\.41∗⁣∗0\.41^\{\*\*\}and0\.60∗⁣∗0\.60^\{\*\*\}\); fertility is dominant for the distortion\-free schemes \(EXPEdit0\.83∗⁣∗0\.83^\{\*\*\}, DIP0\.24∗⁣∗0\.24^\{\*\*\}\)\. On quality, fertility predicts the green\-list schemes \(KGW0\.39∗⁣∗0\.39^\{\*\*\}, Unigram0\.25∗⁣∗0\.25^\{\*\*\}, DIP0\.23∗⁣∗0\.23^\{\*\*\}\), and adherence registers only for the distortion\-free schemes under instruct\. The covariate that drives detectability is not the one that drives preservation\.

Instruction tuning collapses detection\-side structure\.DetectionR2R^\{2\}falls below significance for five of six schemes under instruct \(XSIR adherence0\.29∗⁣∗0\.29^\{\*\*\}excepted\), consistent with the calibrational instruct collapse of §[5\.1](https://arxiv.org/html/2608.20047#S5.SS1)\. Quality\-side fertility persists for the green\-list family\. Latin script and NWM surprisal clearp<0\.05p<0\.05in 6 of 48 cells combined; both correlate with the typological partition but neither carries enough non\-collinear signal atn=33n\{=\}33to compete\. Pooling across regimes \(results omitted\) dilutes the regime\-specific detection mechanisms while preserving the regime\-stable quality\-side fertility effect, corroborating the asymmetry\.

Table 14:Single\-covariateR2R^\{2\}for per\-language detection TPR atτg\\tau\_\{g\},α=0\.01\\alpha\{=\}0\.01, subset all\. Each fit regresses the per\-language outcome vector on a single covariate over 33 \(language, generator\) points\. Stars from the slopepp\-value:∗p<0\.05\{\}^\{\*\}p<0\.05,p∗⁣∗<0\.01\{\}^\{\*\*\}p<0\.01\.Table 15:Single\-covariateR2R^\{2\}for per\-language MAUVE preservation\. Each fit regresses the per\-language outcome vector on a single covariate over 33 \(language, generator\) points\. Stars from the slopepp\-value:∗p<0\.05\{\}^\{\*\}p<0\.05,p∗⁣∗<0\.01\{\}^\{\*\*\}p<0\.01\. BERTScore\-rescaled and PPL\-XGLM paradigm\-stability tables in §[F\.1](https://arxiv.org/html/2608.20047#A6.SS1)\.### F\.1Paradigm\-stability check

Tables[16](https://arxiv.org/html/2608.20047#A6.T16)and[17](https://arxiv.org/html/2608.20047#A6.T17)repeat the single\-covariate quality\-side regression with BERTScore\-rescaled and PPL\-XGLM as the outcome\. PPL\-XGLM corroborates and strengthens the MAUVE story: fertility is the dominant predictor for the green\-list family on both regimes, with the largestR2R^\{2\}values in the entire mechanism analysis appearing under instruct \(KGW0\.446∗⁣∗0\.446^\{\*\*\}, Unigram0\.559∗⁣∗0\.559^\{\*\*\}, XSIR0\.340∗⁣∗0\.340^\{\*\*\}\)\. BERTScore\-rescaled diverges: adherence, not fertility, is the dominant predictor in base for five of six schemes \(R2R^\{2\}between0\.188∗0\.188^\{\*\}and0\.378∗⁣∗0\.378^\{\*\*\}\)\. The divergence is consistent with the narrow\-dynamic\-range observation of §[5\.2](https://arxiv.org/html/2608.20047#S5.SS2): BERTScore\-rescaled F1 is sensitive to the per\-language LID\-baseline that goes into rescaling, so its single\-covariateR2R^\{2\}captures adherence rather than the fertility\-driven distributional shift that MAUVE and PPL\-XGLM register\. The fertility\-on\-green\-list mechanism is therefore paradigm\-stable across the two paradigms that measure distributional or reference\-likelihood divergence directly\.

Table 16:Single\-covariateR2R^\{2\}for BERTScore\-rescaled preservation\. Stars from the slopepp\-value:∗p<0\.05\{\}^\{\*\}p<0\.05,p∗⁣∗<0\.01\{\}^\{\*\*\}p<0\.01\.Table 17:Single\-covariateR2R^\{2\}for PPL\-XGLM preservation\. Stars from the slopepp\-value:∗p<0\.05\{\}^\{\*\}p<0\.05,p∗⁣∗<0\.01\{\}^\{\*\*\}p<0\.01\.