Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck

# Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck
Source: [https://arxiv.org/html/2608.20362](https://arxiv.org/html/2608.20362)
11institutetext:School of Engineering, Institute of Science Tokyo, Japan
11email:zhou\.c\.76d6@m\.isct\.ac\.jp22institutetext:College of Control Science and Engineering, Zhejiang University, China
22email:jiangqiliang@zju\.edu\.cn33institutetext:Department of Electrical and Computer Engineering, National University of Singapore, Singapore
33email:zhouxu\_nus@u\.nus\.edu
\*Equal contribution\.†Corresponding authors\.###### Abstract

Reinforcement learning with verifiable rewards \(RLVR\) is a standard recipe for training large language models on mathematical reasoning, where an answer verifier serves as a language\-neutral reward function\. We show that this assumption fails in multilingual mathematical reasoning: an exact\-match verifier turns format and script variation into language\-dependent false\-negative reward noise\. We introduce a general\-purpose, reusable protocol for auditing multilingual RLVR rewards: a verifier\-robustness suite, a rollout\-diagnosis procedure, and language\-conditioned reward\-error metrics for Japanese, English, and Chinese answers, applicable unchanged to any model, verifier, or language set\. On MGSM EN/JP/CN rollouts withk=8k=8, the exact\-match proxy rejects trusted\-correct answers at sharply different rates by language across Qwen3\-4B, Qwen3\-8B, and Llama\-3\.1\-8B\-Instruct; for Qwen3\-8B, the false\-negative rate reaches0\.6420\.642on JP against0\.1220\.122on EN and0\.0730\.073on CN, a false\-negative\-rate verifier language bias \(VLB\) of0\.5690\.569\. A same\-split, plain\-numeric probe localizes the mechanism to the final\-answer interface: a final\-answer\-interface model drives the exact reward\-error VLB to zero while the residual language accuracy gap is unchanged\. We then expose a cross\-lingual*selection bottleneck*: on fresh MGSM250 rollouts, a target\-local aggregation rule that uses no trusted labels closes55%55\\%and78%78\\%of the average local\-majority selection gap for Qwen3\-8B and Llama\-3\.1\-8B\-Instruct, and63%63\\%and88%88\\%of the JP gap\. An autopsy shows that more than95%95\\%of these repairs require genuine cross\-lingual support rather than a within\-language majority, and the bottleneck replicates on a manually audited483483\-problem MATH\-500 set\. Finally, a controlled training audit shows that rule\-GRPO raises trusted accuracy from0\.6470\.647to0\.7350\.735while the exact\-match reward\-error VLB stays high\. The unifying message is operational: multilingual RLVR rewards should be audited by language and by answer interface*before*they are optimized\.

## 1Introduction

Verifiable reward functions are the central attraction of RLVR\. When a task admits an automatic verifier, a model can be optimized against a binary correctness signal without expensive human preference labels\. Mathematical reasoning is the canonical case: GSM8K, MATH, and GRPO\-style math RL provide widely used problem settings and optimization recipes\[[2](https://arxiv.org/html/2608.20362#bib.bib11),[7](https://arxiv.org/html/2608.20362#bib.bib12),[11](https://arxiv.org/html/2608.20362#bib.bib1)\]\. A final\-answer verifier maps a generated solution to a reward, and an RL algorithm such as GRPO optimizes the policy against that reward\.

This setup carries an implicit reliability assumption: the verifier is a stable proxy for task correctness across the languages and answer formats present in training\. In multilingual reasoning, that assumption is fragile\. The same numerical answer can surface with currency symbols, markdown emphasis, brackets, decimal variants, full\-width digits, CJK magnitude units, or language\-specific suffixes\. A strict exact\-match verifier rejects these semantically correct strings\. When such rejections are not distributed uniformly across languages, the verifier converts surface variation into language\-dependent reward pressure, and that pressure enters the policy gradient\.

We study this phenomenon as*multilingual verifier bias*in RLVR and follow it from measurement to mechanism to consequence\. We first build a verifier\-robustness suite and a rollout\-diagnosis protocol that compares a weak proxy verifier against a trusted canonical\-equivalence reward on real model samples\. We measure where reward error concentrates by language\. We then isolate the mechanism with a controlled plain\-numeric interface probe\. Finally, we ask what the noise costs at decision time, and find that much of the language gap is not a generation failure but a*selection*failure: the model often already produces the correct answer in some language but selects a wrong one\.

Our central empirical object is the language\-conditioned reward\-error profile and a single scalar summarizing its spread, the verifier language biasVLB​\(m\)=maxl⁡m​\(l\)−minl⁡m​\(l\)\\mathrm\{VLB\}\(m\)=\\max\_\{l\}m\(l\)\-\\min\_\{l\}m\(l\)for a per\-language metricmm\. The diagnosis is reproducible across three models and two model families, and the worst\-affected language is not a property of the benchmark but of the model output distribution\.

Taken together, the three measurements are not three separate studies but three views of one quantity—the language\-conditioned gap between proxy and trusted reward\. The same tuple schema\(π,\{xl\},k,v,r∗\)\(\\pi,\\\{x\_\{l\}\\\},k,v,r^\{\\ast\}\)\(Section[3](https://arxiv.org/html/2608.20362#S3)\) is the common instrument behind every result in this paper: the rollout diagnosis reads*where*the verifier injects language\-conditioned noise directly off the per\-language profile, the interface probe isolates*why*\(answer format, not arithmetic competence\), and the selection bottleneck and training audit then trace*what*that noise costs downstream—at decision time and under optimization\. This paper makes three contributions\.

1. 1\.A general\-purpose protocol for auditing multilingual RLVR rewards\.The protocol fixes a single tuple\(π,\{xl\},k,v,r∗\)\(\\pi,\\\{x\_\{l\}\\\},k,v,r^\{\\ast\}\)and a set of language\-conditioned reward\-error metrics—false\-negative rate, false\-positive rate, reward\-error rate, expected policy pressure, and the verifier language biasVLB\\mathrm\{VLB\}—and applies unchanged to any model, verifier, or language set\. We release the JP/EN/CN stress suite and three rule\-verifier families \(exact match, normalized numeric, hybrid\) with the paper\.
2. 2\.Reproducible diagnosis with an interface\-level mechanism\.On Qwen3\-4B, Qwen3\-8B, and Llama\-3\.1\-8B\-Instruct MGSM rollouts, exact\-match reward noise is false\-negative dominated and language\-conditioned\. A same\-split plain\-numeric probe localizes the exact\-match component to the final\-answer interface: an interface model drives exact reward\-errorVLB\\mathrm\{VLB\}to0\.0000\.000while the residual accuracy gap is unchanged, separating a format effect from a capability effect\.
3. 3\.The cross\-lingual selection bottleneck, and a label\-free procedure that exploits it\.On fresh MGSM250 rollouts, a simple target\-local cross\-lingual aggregation procedure—using no trusted labels—recovers5555–78%78\\%of the average local\-majority selection gap and6363–88%88\\%of the JP gap, replicating on a manually audited483483\-problem MATH\-500 set; an autopsy confirms that over95%95\\%of the repairs require genuine cross\-lingual agreement, so much of the language gap is a recoverable selection failure, not a generation deficit\.

Section[2](https://arxiv.org/html/2608.20362#S2)positions the work\. Section[3](https://arxiv.org/html/2608.20362#S3)formalizes the metrics\. Sections[4](https://arxiv.org/html/2608.20362#S4)–[7](https://arxiv.org/html/2608.20362#S7)present the suite, the rollout diagnosis, the interface mechanism, and the selection bottleneck\. Section[8](https://arxiv.org/html/2608.20362#S8)reports a controlled training audit, and Section[9](https://arxiv.org/html/2608.20362#S9)draws operational guidance\.

## 2Related Work

### Verifier noise in RLVR\.

Verifier noise is increasingly recognized as a first\-order concern in RLVR\. TinyV studies mathematical false negatives and shows that reducing them improves RL for LLM reasoning\[[15](https://arxiv.org/html/2608.20362#bib.bib7)\], and analyses of noisy supervision argue that RLVR is not robust to incorrect reward by default\[[17](https://arxiv.org/html/2608.20362#bib.bib8)\]\. A theoretical line models verifier unreliability as a stochastic reward channel: the phase\-transition statisticJ=TPR−FPRJ=\\mathrm\{TPR\}\-\\mathrm\{FPR\}governs whether incorrect modes are driven out or amplified\[[10](https://arxiv.org/html/2608.20362#bib.bib15)\], and asymmetric false\-positive/false\-negative rates admit backward and forward corrections that restore an unbiased policy gradient\[[1](https://arxiv.org/html/2608.20362#bib.bib14)\]\. Our verifier language bias is the language\-conditioned sharpening of exactly this picture: in the false\-negative\-dominated exact\-match regime \(FPR≈0\\mathrm\{FPR\}\\approx 0\),J​\(l\)=TPR​\(l\)−FPR​\(l\)≈TPR​\(l\)=1−FNR​\(l\)J\(l\)=\\mathrm\{TPR\}\(l\)\-\\mathrm\{FPR\}\(l\)\\approx\\mathrm\{TPR\}\(l\)=1\-\\mathrm\{FNR\}\(l\), so a globally low\-noise verifier sits in different reward regimes across languages, and a single global correction cannot track that spread\. Reward\-hacking studies further motivate a clean separation between a proxy reward and a trusted reward: policies exploit verifier weaknesses, and evaluating that exploitation requires a trusted reference\[[6](https://arxiv.org/html/2608.20362#bib.bib10)\]\. Our work shares this proxy\-versus\-trusted framing but adds a language axis: we measure how the proxy/trusted gap is distributed across JP/EN/CN and how it enters the policy gradient as language\-dependent pressure\. Two distinctions matter\. First, prior work characterizes verifier reliability largely as a global property of the reward channel; we condition it on language and show that the spread, not the mean, is what biases a multilingual policy\. Second, where reward\-hacking work studies false positives—rewards granted to wrong answers that the policy then exploits—the multilingual exact\-match regime we observe is the opposite, dominated by false negatives, where correct answers are denied reward\. The two failure modes call for different audits: false positives are caught by a stronger trusted reference, while the language\-conditioned false negatives we study are caught only by conditioning the audit on language and on answer interface\.

### Multilingual reasoning and judges\.

MGSM established multilingual chain\-of\-thought math evaluation\[[12](https://arxiv.org/html/2608.20362#bib.bib6)\], and cross\-lingual collapse studies show that language\-centric pretraining shapes reasoning behavior across languages\[[9](https://arxiv.org/html/2608.20362#bib.bib5)\]\. On the evaluation side, multilingual LLM\-as\-a\-judge work documents that judge reliability varies by language\[[5](https://arxiv.org/html/2608.20362#bib.bib9)\]\. These results concern judges as evaluators; we instead study a rule verifier used as a*reward*and quantify its language\-conditioned failure modes during rollouts\.

### Cross\-lingual consistency and aggregation\.

A parallel line improves multilingual reasoning by enforcing agreement across languages\. Inference\-time frameworks integrate multilingual reasoning paths by majority vote to lift accuracy\[[16](https://arxiv.org/html/2608.20362#bib.bib16)\], and training\-time methods enforce cross\-lingual self\-consistency as an unsupervised RL objective, self\-translating each prompt and rewarding cross\-language answer agreement\[[4](https://arxiv.org/html/2608.20362#bib.bib17)\]\. Our cross\-lingual aggregation \(Section[7](https://arxiv.org/html/2608.20362#S7)\) is neither an accuracy\-maximizing decoder nor a training objective: it is a label\-free*diagnostic*that selects only among answer clusters already present in the target language, imports no answer from another language \(borrowed count0\), and trains nothing\. Its purpose is to measure how much of the residual language gap is a recoverable selection failure rather than a generation deficit—which is evidence for, not a competitor to, the cross\-lingual signal these methods exploit, and which our protocol isolates from the format and capability components\.

### Verifier\-free and self\-verified rewards\.

A growing line removes or softens the external verifier: NOVER trains without a verifier\[[8](https://arxiv.org/html/2608.20362#bib.bib2)\], VIGOR uses an intrinsic gradient\-norm reward\[[14](https://arxiv.org/html/2608.20362#bib.bib3)\], and Soft\-SVeRL uses self\-verified soft rewards\[[3](https://arxiv.org/html/2608.20362#bib.bib4)\]\. These are training\-time rewards and mark a boundary rather than a competitor to our work\. Our cross\-lingual aggregation is a label\-free, inference\-time procedure that recovers much of the selection gap, and our central contribution is the auditing protocol around it\.

### Math benchmarks across languages\.

Our suite builds on GSM8K and MATH\[[2](https://arxiv.org/html/2608.20362#bib.bib11),[7](https://arxiv.org/html/2608.20362#bib.bib12)\], and on the broader argument, made for Portuguese by MATH\-PT, that multilingual math evaluation must not rely on English\-centric assumptions\[[13](https://arxiv.org/html/2608.20362#bib.bib13)\]\. Our focus is complementary: not a new test set, but a protocol for auditing the*reward*that a multilingual RLVR pipeline applies to model outputs\.

## 3Problem Formulation

Letxlx\_\{l\}be a mathematical prompt in languagel∈\{EN,JP,CN\}l\\in\\\{\\mathrm\{EN\},\\mathrm\{JP\},\\mathrm\{CN\}\\\}, letyybe a sampled completion, and leta​\(y\)a\(y\)be its extracted final answer\. We distinguish a*trusted*rewardr∗​\(y\)r^\{\\ast\}\(y\), based on canonical answer equivalence, from a*proxy*rewardrv​\(y\)r\_\{v\}\(y\)produced by a verifiervvsuch as exact string match\. The trusted reward canonicalizes both prediction and reference \(Arabic\-digit normalization, CJK magnitude expansion, percentage and sign handling\) before comparing, and serves as the reference against which proxy errors are defined\.

For a fixed languagell, a*false negative*\(FN\) is a completion withr∗=1r^\{\\ast\}=1butrv=0r\_\{v\}=0\(a correct answer the verifier rejects\), and a*false positive*\(FP\) is a completion withr∗=0r^\{\\ast\}=0butrv=1r\_\{v\}=1\. We report the per\-language false\-negative and false\-positive rates, conditioned on the trusted label,

FNR​\(l\)\\displaystyle\\mathrm\{FNR\}\(l\)=Pry∼π\(⋅∣xl\)⁡\[rv​\(y\)=0∣r∗​\(y\)=1\],\\displaystyle=\\Pr\\nolimits\_\{y\\sim\\pi\(\\cdot\\mid x\_\{l\}\)\}\\\!\\big\[r\_\{v\}\(y\)=0\\mid r^\{\\ast\}\(y\)=1\\big\],FPR​\(l\)\\displaystyle\\mathrm\{FPR\}\(l\)=Pry∼π\(⋅∣xl\)⁡\[rv​\(y\)=1∣r∗​\(y\)=0\],\\displaystyle=\\Pr\\nolimits\_\{y\\sim\\pi\(\\cdot\\mid x\_\{l\}\)\}\\\!\\big\[r\_\{v\}\(y\)=1\\mid r^\{\\ast\}\(y\)=0\\big\],so thatFNR​\(l\)\\mathrm\{FNR\}\(l\)is the share of trusted\-correct completions the verifier rejects\. The unconditional reward\-error rate isErr​\(l\)=Pr⁡\[rv​\(y\)≠r∗​\(y\)\]\\mathrm\{Err\}\(l\)=\\Pr\[r\_\{v\}\(y\)\\neq r^\{\\ast\}\(y\)\], which for an FN\-dominated verifier \(FPR≈0\\mathrm\{FPR\}\\approx 0\) reduces toFNR​\(l\)⋅acc​\(l\)\\mathrm\{FNR\}\(l\)\\cdot\\mathrm\{acc\}\(l\), the withheld reward mass\. The expected policy pressure−Err​\(l\)\-\\mathrm\{Err\}\(l\)is the signed reward the policy loses per sample on correct\-but\-rejected answers, and it is the quantity that enters a GRPO\-style advantage: a language with a high false\-negative rate contributes a systematic negative signal on completions that are in fact correct, biasing the gradient against the formatting habits the model uses in that language\.

A diagnosis run is therefore fully specified by a tuple\(π,\{xl\},k,v,r∗\)\(\\pi,\\\{x\_\{l\}\\\},k,v,r^\{\\ast\}\)and computed by Algorithm[1](https://arxiv.org/html/2608.20362#alg1)\. Its per\-language profile, theVLB\\mathrm\{VLB\}summaries, and the dominant failure mode \(FN versus FP\) are the protocol’s output: the rollout and interface tables instantiate this profile directly, while the selection and training tables report its downstream consequences\.

To summarize the language spread of any per\-language metricmm, we define theverifier language bias

VLB​\(m\)=maxl⁡m​\(l\)−minl⁡m​\(l\)\.\\mathrm\{VLB\}\(m\)=\\max\_\{l\}m\(l\)\-\\min\_\{l\}m\(l\)\.The two quantities of primary interest are the reward\-error VLB,VLB​\(Err\)\\mathrm\{VLB\}\(\\mathrm\{Err\}\), and the false\-negative\-rate VLB,VLB​\(FNR\)\\mathrm\{VLB\}\(\\mathrm\{FNR\}\)\. A verifier withVLB≈0\\mathrm\{VLB\}\\approx 0treats languages symmetrically even if it is globally noisy; a verifier with largeVLB\\mathrm\{VLB\}injects language\-specific gradient pressure\. In the MGSM setting exact\-match errors are FN\-dominated \(FPR≈0\\mathrm\{FPR\}\\approx 0\), so the false\-negative\-rate VLB is the most informative single number\. Throughout, per\-language rates are reported to three decimals, and VLBs, gaps, and language deltas are the differences of those displayed values; confidence intervals are percentile bootstrap\.

Rolloutdiagnosis\(§[5](https://arxiv.org/html/2608.20362#S5)\)exact\-matchvvon live rolloutsInterfaceprobe\(§[6](https://arxiv.org/html/2608.20362#S6)\)control theanswer formatSelection\(§[7](https://arxiv.org/html/2608.20362#S7)\)label\-free cross\-lingual aggregationTrainingaudit\(§[8](https://arxiv.org/html/2608.20362#S8)\)optimize againstvvPer\-language profile:fnr,fpr,err,VLB\\mathrm\{VLB\}Protocol tuple\(π,\{xl\},k,v,r∗\)\(\\pi,\\\{x\_\{l\}\\\},k,v,r^\{\\ast\}\)Figure 1:One protocol, four uses\. A single tuple schema\(π,\{xl\},k,v,r∗\)\(\\pi,\\\{x\_\{l\}\\\},k,v,r^\{\\ast\}\)produces a per\-language reward\-error profile; varyingπ\\piandvvinstantiates the four analyses\. The metrics are model\- and verifier\-agnostic, so the protocol transfers unchanged to a new model, verifier, or language set\.Algorithm 1Language\-Conditioned RLVR Reward Audit1:policy

π\\pi; multilingual prompts

\{xl\}l∈L\\\{x\_\{l\}\\\}\_\{l\\in L\}; sample budget

kk; proxy verifier

vv; trusted reward

r∗r^\{\\ast\}
2:per\-language profile and

VLB\\mathrm\{VLB\}summaries

3:foreach language

l∈Ll\\in Land prompt

x∈Xlx\\in X\_\{l\}do

4:draw

kkcompletions

y∼π\(⋅∣xl\)y\\sim\\pi\(\\cdot\\mid x\_\{l\}\); score each under

rvr\_\{v\}and

r∗r^\{\\ast\}
5:endfor

6:foreach language

l∈Ll\\in Ldo

7:

FNR​\(l\)←Pr⁡\[rv=0∣r∗=1\]\\mathrm\{FNR\}\(l\)\\leftarrow\\Pr\[r\_\{v\}\{=\}0\\mid r^\{\\ast\}\{=\}1\];

FPR​\(l\)←Pr⁡\[rv=1∣r∗=0\]\\mathrm\{FPR\}\(l\)\\leftarrow\\Pr\[r\_\{v\}\{=\}1\\mid r^\{\\ast\}\{=\}0\];

Err​\(l\)←Pr⁡\[rv≠r∗\]\\mathrm\{Err\}\(l\)\\leftarrow\\Pr\[r\_\{v\}\{\\neq\}r^\{\\ast\}\]
8:endfor

9:

VLB​\(m\)←maxl⁡m​\(l\)−minl⁡m​\(l\)\\mathrm\{VLB\}\(m\)\\leftarrow\\max\_\{l\}m\(l\)\-\\min\_\{l\}m\(l\)for

m∈\{FNR,FPR,Err\}m\\in\\\{\\mathrm\{FNR\},\\mathrm\{FPR\},\\mathrm\{Err\}\\\}; identify the dominant mode \(FN vs\. FP\)

10:if

FNR​\(l\)\>0\.3\\mathrm\{FNR\}\(l\)\>0\.3and

VLB​\(FNR\)\>0\.2\\mathrm\{VLB\}\(\\mathrm\{FNR\}\)\>0\.2for some

llthen

11:flag

ll; audit the answer interface \(§[6](https://arxiv.org/html/2608.20362#S6)\), then selection \(§[7](https://arxiv.org/html/2608.20362#S7)\)

12:endif

13:returnprofile

\{FNR​\(l\),FPR​\(l\),Err​\(l\)\}l\\\{\\mathrm\{FNR\}\(l\),\\mathrm\{FPR\}\(l\),\\mathrm\{Err\}\(l\)\\\}\_\{l\},

VLB\\mathrm\{VLB\}summaries, dominant mode, flagged languages

## 4A Multilingual Verifier\-Robustness Suite

The suite starts from public MGSM JP/EN/CN records and available multilingual MATH\-500 records\. For each seed problem it constructs answer candidates spanning correct answers, format variants, and near misses\. The transformations exercise full\-width digits, CJK digit and magnitude forms, percentages, explanatory answer text, mixed scripts, and near\-miss numeric values\. Release scope is the schema, the transformations, the configs, the scripts, and aggregate tables; upstream problem text remains under the original dataset licenses\. We treat the suite as a controlled verifier\-fragility instrument: a fixed battery on which different rule verifiers can be compared, rather than a sample of organic answer formats\.

The trusted reward that anchors every comparison is the canonical\-equivalence check of Section[3](https://arxiv.org/html/2608.20362#S3), which normalizes script, magnitude units, punctuation, and percentage forms on both prediction and reference before deciding equivalence on the normalized values\. The proxy verifiers are deliberately weaker so that the gap between proxy and trusted exposes exactly the surface phenomena—script, magnitude unit, punctuation, and wrapper text—that a strict matcher mishandles\. This is the design choice that lets a single battery quantify verifier fragility rather than model competence\.

We instrument three rule\-verifier families: exact string match, normalized numeric equivalence, and a hybrid that tries exact match first and falls back to numeric equivalence\. Table[1](https://arxiv.org/html/2608.20362#S4.T1)reports the deterministic cluster rerun\. Exact matching is brittle on the synthetic battery, with error around0\.710\.71in every language, while normalized numeric and hybrid verifiers reduce error to roughly0\.120\.12–0\.130\.13\. All three verifiers are FN\-dominated on this battery \(false\-positive rate near zero\), and the language spread is small \(VLB≤0\.013\\mathrm\{VLB\}\\leq 0\.013\)\. The suite provides a controlled fragility baseline; the language\-conditioned bias it enables is measured on live rollouts in Section[5](https://arxiv.org/html/2608.20362#S5)\.

Table 1:Verifier\-robustness suite \(deterministic cluster rerun\)\. Exact matching is brittle and false\-negative dominated; normalized numeric and hybrid verifiers recover most of the loss\. The language spread is small, so the suite is a verifier\-fragility baseline rather than the language\-bias result\.
## 5Rollout Reward Diagnosis

We now apply the protocol to real rollouts\. We sample Qwen3\-4B, Qwen3\-8B, and Llama\-3\.1\-8B\-Instruct on the same MGSM EN/JP/CN split, with8080seed problems per language andk=8k=8completions per prompt, giving19201920records per model\. The trusted reward uses canonical answer equivalence; the primary proxy is an exact\-match verifier \(rule\_exact\_match\_v1\), intentionally weaker than the trusted reward so that format fragility is observed as reward noise\. The Llama run provides a second\-family replication of the diagnosis\.

The suite \(Section[4](https://arxiv.org/html/2608.20362#S4)\) found exact match brittle but language\-symmetric \(VLB≤0\.013\\mathrm\{VLB\}\\leq 0\.013\): a fixed battery applies the same format variants to every language and so cannot expose a language\-conditioned effect\. Live outputs are different—each model develops its own answer\-formatting habits, and those habits differ by language—so the reward\-noise profile is set by the interaction between the verifier’s brittleness and the model’s language\-conditioned formatting, which only a live policy reveals\. The language symmetry of Table[1](https://arxiv.org/html/2608.20362#S4.T1)breaks sharply here\.

Table[2](https://arxiv.org/html/2608.20362#S5.T2)reports the per\-language profile, and Fig\.[2](https://arxiv.org/html/2608.20362#S5.F2)visualizes the false\-negative rate\. Two facts stand out\. First, the reward noise is essentially all false\-negative: the observed false\-positive rate is0for all three models in every language, so the verifier never rewards a wrong answer here—it only rejects correct ones\. Second, the false\-negative rate is strongly language\-conditioned, and the worst language depends on the model\. Among trusted\-correct completions, Qwen3\-4B is rejected most on EN \(FNR=0\.345\\mathrm\{FNR\}=0\.345\), Qwen3\-8B on JP \(FNR=0\.642\\mathrm\{FNR\}=0\.642, against0\.1220\.122on EN and0\.0730\.073on CN\), and Llama\-3\.1\-8B\-Instruct again on EN \(0\.3420\.342\)\. The corresponding false\-negative\-rate VLB ranges from0\.2340\.234\(Qwen3\-4B\) through0\.2580\.258\(Llama\) to0\.5690\.569\(Qwen3\-8B\)\. A problem\-level bootstrap \(B=4000B=4000, resampling the8080shared problems\) confirms these are not sampling artifacts: the VLB95%95\\%confidence intervals are\[0\.16,0\.32\]\[0\.16,0\.32\],\[0\.46,0\.67\]\[0\.46,0\.67\], and\[0\.17,0\.37\]\[0\.17,0\.37\]respectively, all excluding zero, and the Qwen3\-8B JP rate of0\.6420\.642has CI\[0\.55,0\.73\]\[0\.55,0\.73\]\.

The Qwen3\-8B JP figure is the sharpest case:64\.2%64\.2\\%of its trusted\-correct JP completions are rejected by exact match, so an RLVR loop driven by exact match would systematically suppress correct JP reasoning\. These rejections have a single dominant cause rather than diffuse arithmetic disagreement:307307of the308308rejected\-correct JP completions \(99\.7%99\.7\\%\) wrap the answer in angle brackets \(e\.g\.<18\>for1818\), and94\.8%94\.8\\%recover the gold answer after stripping non\-numeric characters—so a numeric\-equivalence fallback alone would readmit nearly all of them\. Each model’s worst\-affected language is dominated by one wrapper format: markdown or math delimiters for Qwen3\-4B \(EN,98%98\\%of its FN\) and Llama\-3\.1\-8B\-Instruct \(EN,93%93\\%\), and angle brackets for Qwen3\-8B \(JP,99\.7%99\.7\\%\); in every case numeric normalization readmits most of the rejected\-correct mass \(83%83\\%,70%70\\%, and95%95\\%respectively\)\. Across models, the verifier bias is one formatting habit per model, not a reasoning gap\. An aggregate verifier that looks acceptable on a static battery \(Table[1](https://arxiv.org/html/2608.20362#S4.T1)\) can still inject large, model\-specific language pressure once it meets a live policy\.

Trusted accuracy and the false\-negative rate are different axes\. Qwen3\-8B has its*highest*trusted accuracy on EN \(0\.9480\.948\) and a respectable0\.7500\.750on JP, yet its reward noise is overwhelmingly concentrated on JP\. The model is largely able to solve the JP problems; it is the exact\-match verifier that fails to credit the JP answers\. This is the regime in which verifier bias is most dangerous, because the loss is invisible to a trusted\-accuracy report and only appears when the proxy reward is inspected by language\. Conversely, Llama\-3\.1\-8B\-Instruct has low trusted accuracy on JP \(0\.4690\.469\) but a*low*JP false\-negative rate \(0\.0900\.090\): when the model rarely produces a correct JP answer in the first place, the reward error migrates to EN, where the model is both productive and idiosyncratic in its formatting\. The diagnosis thus distinguishes a capability\-limited language from a verifier\-penalized one, a distinction a single accuracy number cannot make\.

Because the observed false\-positive rate is zero, the entire reward error here is withheld deserved reward, never granted undeserved reward\. This is the more corrosive direction for RLVR: false negatives directly depress the advantage of correct trajectories and, concentrated in one language, inject a systematic language\-specific negative signal into the policy gradient\. The remaining sections trace where this withheld reward comes from \(Section[6](https://arxiv.org/html/2608.20362#S6)\) and what it costs at decision time \(Section[7](https://arxiv.org/html/2608.20362#S7)\)\.

![Refer to caption](https://arxiv.org/html/2608.20362v1/x1.png)Figure 2:False\-negative rate by language on MGSMk=8k=8rollouts\. The worst\-affected language shifts across models: EN for Qwen3\-4B and Llama\-3\.1\-8B\-Instruct, JP for Qwen3\-8B\.Table 2:Rollout reward diagnosis on MGSM EN/JP/CN,8080seed problems per language,k=8k=8\(19201920records per model\)\. FN is the false\-negative rate among trusted\-correct completions\. Exact\-match reward noise is false\-negative dominated \(observedFPR=0\\mathrm\{FPR\}=0throughout\) and language\-conditioned, and the worst language is model\-distribution dependent\.
## 6The Interface Mechanism

The rollout diagnosis shows*that*exact\-match reward noise is language\-conditioned, but not*why*\. We test two hypotheses with a controlled probe: does the noise reflect genuine arithmetic differences across languages, or a surface mismatch at the final\-answer interface?

We construct a leak\-free MGSM heldout split of2020problem identifiers, disjoint from any training prompts, and evaluate three Qwen3\-8B variants under a*plain\-numeric*prompt that requests a bare numeric final answer: the base model, a same\-split rule\-GRPO model, and a final\-answer\-interface SFT model that is supervised only on producing the answer in the plain\-numeric form\. Each variant is evaluated on240240records \(the2020identifiers in three languages withk=4k=4\) with no rollout failures, under both the exact\-match and normalized\-numeric diagnoses\. Table[3](https://arxiv.org/html/2608.20362#S6.T3)reports the result\.

Two effects separate cleanly\. Under the plain\-numeric interface the exact\-match noise collapses: the base model’s exact reward\-error VLB is already low at0\.0130\.013, and the interface SFT model drives both it and the exact false\-negative\-rate VLB to0\.0000\.000; a problem\-level bootstrap puts the interface model’s exact VLB at\[0,0\]\[0,0\]\(Base\[0,0\.05\]\[0,0\.05\], rule\-GRPO\[0,0\.14\]\[0,0\.14\]\), so the format component of the bias is robustly eliminated\. At the same time the language accuracy gap does not move with the interface fix: it is0\.1500\.150for both rule\-GRPO and the interface model, against0\.2130\.213for the base model\. In other words, controlling the answer interface removes the measured exact\-match verifier bias but leaves a residual accuracy difference between languages—a policy capability difference, not a verifier\-format artifact—whose scale and composition Section[7](https://arxiv.org/html/2608.20362#S7)establishes on fresh data at the483483\-problem scale\.

The per\-language trusted accuracies of the interface model make the residual concrete:0\.8750\.875on EN,0\.8250\.825on CN, and0\.7250\.725on JP, so the0\.1500\.150gap is the EN–JP difference that survives once formatting is held constant\. The contrast with the rollout diagnosis is informative\. In Section[5](https://arxiv.org/html/2608.20362#S5)the dominant Qwen3\-8B problem was a JP*verifier*penalty; here, under a controlled plain\-numeric interface, the JP verifier penalty is gone \(exactVLB​\(FNR\)=0\\mathrm\{VLB\}\(\\mathrm\{FNR\}\)=0\) and what remains is a smaller JP*accuracy*deficit\. The probe therefore does two things at once: it confirms that the large exact\-match component of the bias was a format artifact, and it isolates the genuine, much smaller, trusted\-accuracy gap that any reward design must still contend with\. The same split, the same model, and the same diagnosis pipeline are held fixed while only the answer interface varies, so the decomposition into format artifact and capability gap is direct\.

Table 3:Plain\-numeric, same\-split interface probe on Qwen3\-8B \(240240records per variant, zero rollout failures\)\. Controlling the final\-answer interface drives the exact\-match verifier bias to zero, while the residual language accuracy gap is unchanged\.The exact\-match component of multilingual verifier bias is an interface problem: it is removed by fixing how the final answer is written and parsed, not by changing the RL objective\. What remains after the interface is controlled is a residual trusted\-accuracy gap between languages, and Section[7](https://arxiv.org/html/2608.20362#S7)shows that a large share of*that*gap is itself recoverable at selection time\.

## 7The Cross\-Lingual Selection Bottleneck

If the residual gap in Section[6](https://arxiv.org/html/2608.20362#S6)is a capability gap, the next question is whether it is a*generation*gap \(the model never produces the correct answer in the weak language\) or a*selection*gap \(the model produces it among itskksamples but a majority vote picks a wrong one\)\. We answer this on fresh data\.

### Setup\.

We sample Qwen3\-8B and Llama\-3\.1\-8B\-Instruct on a fresh MGSM250 split \(250250problems per language, disjoint from the diagnosis split\) withk=8k=8\. The baseline decision rule is*local*majority: for each problem in languagell, take the most frequent canonicalized answer among thekksame\-language samples\. The comparison rule is*target\-local cross\-lingual aggregation*: the selector may consult same\-problem answer\-cluster counts from the other two languages, but it chooses only among clusters that actually appear in the target language, never uses trusted labels, and runs entirely at inference time\.

### Result\.

Table[4](https://arxiv.org/html/2608.20362#S7.T4)reports the outcome and Fig\.[3](https://arxiv.org/html/2608.20362#S7.F3)visualizes it\. For both models, aggregation lifts average accuracy and shrinks the language gap, and the gains concentrate on JP\. For Qwen3\-8B, average accuracy rises from0\.8670\.867to0\.8950\.895and the EN–JP gap falls from0\.1720\.172to0\.1240\.124; for Llama\-3\.1\-8B\-Instruct, average accuracy rises from0\.7760\.776to0\.8670\.867and the gap falls from0\.2360\.236to0\.1040\.104\. Measured against the pass@kkceiling, aggregation closes55\.3%55\.3\\%and78\.2%78\.2\\%of the average local\-majority selection gap, and63\.2%63\.2\\%and88\.4%88\.4\\%of the JP gap\. The effect is statistically strong: of the available selection opportunities, Qwen3\-8B fixes2424of3838and Llama7171of8787, against33regressions each \(paired sign\-testp=4\.9×10−5p=4\.9\\times 10^\{\-5\}for Qwen3\-8B andp<10−12p<10^\{\-12\}for Llama\)\. Among the decisions aggregation changes, the fraction corrected—24/2724/27and71/7471/74—is0\.8890\.889and0\.9590\.959\.

Table 4:Fresh MGSM250 selection bottleneck \(k=8k=8,250250problems per language\)\.*Local*is same\-language majority;*X\-ling\.*is target\-local cross\-lingual aggregation, which uses no trusted labels and only selects among clusters present in the target language\. “Gap closed” is the fraction of the local\-to\-pass@kkselection gap recovered\.![Refer to caption](https://arxiv.org/html/2608.20362v1/x2.png)Figure 3:Cross\-lingual selection bottleneck: per\-language accuracy under local majority \(open\) versus target\-local cross\-lingual aggregation \(filled\)\. Left: MGSM250 \(250250problems/language; Qwen3\-8B and Llama\-3\.1\-8B\-Instruct\)\. Right: the audited483483\-problem MATH\-500 set \(Qwen3\-8B\)\. The procedure uses no trusted labels and imports no cross\-language answer; the JP gain dominates\.
### Autopsy: the repairs are genuinely cross\-lingual\.

Could these repairs bek=8k=8sampling noise—the correct cluster already the within\-language plurality, only confirmed by aggregation? We tested every fixed case by comparing the target\-language vote for the selected cluster against its sibling support, and the data rule it out: for Qwen3\-8B,2323of2424fixes \(95\.8%95\.8\\%\) have a target\-language vote no greater than sibling support \(Llama:7070of7171,98\.6%98\.6\\%\), and both siblings support the chosen cluster in22/2422/24Qwen and60/7160/71Llama fixes\. Because the selector chooses only among clusters present in the target language, no answer is ever imported \(borrowed count0\)\. The repairs are cross\-lingual selection signal, not within\-language sampling artifacts\.

### Robustness\.

The aggregation rule weights sibling support by a single exponentα\\alpha; the result is insensitive to it, with average accuracy changing by≤0\.004\\leq 0\.004acrossα∈\{0\.5,1\.0,2\.0\}\\alpha\\in\\\{0\.5,1\.0,2\.0\\\}and at least91\.7%91\.7\\%of fixes preserved \(we reportα=1\.0\\alpha=1\.0\)\. A conservative variant that triggers aggregation only when both sibling languages support a target cluster retains2222of2424Qwen3\-8B fixes while reducing regressions from33to22at the same gap reduction, so sibling agreement is itself a usable reliability gate\.

### Replication on a harder benchmark\.

The same target\-local rule replicates on a harder external set: the full MATH\-500 in EN/JP/CN\.111We use the public multilingual releaseappier\-ai\-research/Multilingual\-MATH\-500; gold answers are unchanged\. The Japanese and Chinese problems are machine translations; we manually audit the Japanese problems, to which the exclusion below applies\.By a criterion independent of model outputs—Japanese answerability—two reviewers audited all500500problems and excluded1717whose Japanese rendering had dropped a figure\-defining Asymptote block or contained answer\-altering translation artifacts, leaving483483\. On this audited set, Qwen3\-8B local\-majority average accuracy is0\.6620\.662\(EN0\.7540\.754, JP0\.4780\.478, CN0\.7540\.754; EN–JP gap0\.2760\.276\), and aggregation raises it to0\.7370\.737\(EN0\.7760\.776, JP0\.6630\.663, CN0\.7720\.772; gap0\.1130\.113\), with123123corrections against1414regressions among the137137changed selection decisions \(paired sign\-testp<10−20p<10^\{\-20\}\)\. The JP gain \(\+0\.185\+0\.185\) again dominates and borrowing stays at0, so the selection bottleneck holds at scale on a harder benchmark, not only on MGSM\.

### Interpretation\.

A large fraction of the residual language gap is not the model failing to reason in the weak language; it is the model failing to*select*a correct answer it has already produced, which sibling\-language agreement identifies without any trusted label\. This completes a three\-way decomposition of the language\-conditioned reward gap: a*format artifact*\(Section[6](https://arxiv.org/html/2608.20362#S6), removable at the answer interface\), a*selection failure*\(this section, recoverable by a label\-free cross\-lingual signal\), and a residual*generation deficit*\(the part of the pass@kkceiling that neither addresses\)\. The diagnosis thus locates recoverable headroom and points future verifier\-free work toward selection and reranking rather than only toward generation quality\.

## 8Training\-Time Consequence

Once the verifier is actually used as an RLVR reward, the diagnosis and its interface mechanism carry direct consequences\. Two predictions follow from the preceding sections, and a controlled rule\-GRPO audit confirms both\.

### Reward variance \(a consequence of Section[5](https://arxiv.org/html/2608.20362#S5)\)\.

The diagnosis predicts a failure of the learning signal itself\. A Qwen3\-4B rule\-GRPO run on a single\-language split has zero reward standard deviation across all five logged steps—every sampled group receives identical reward, so there is no gradient \(gradient norm0\.0000\.000\)\. This is exactly the FN\-dominated regime of Section[5](https://arxiv.org/html/2608.20362#S5): when correct answers are uniformly accepted or uniformly rejected within a group, the group reward is constant\. Diagnosis\-guided mixed\-group sampling, which deliberately mixes languages and difficulty within a group, restores a nonzero signal for Qwen3\-8B—mean reward standard deviation0\.2580\.258, nonzero variance in55of1010steps, and gradient norm0\.1720\.172\.

### Persisting bias \(a consequence of Section[6](https://arxiv.org/html/2608.20362#S6)\)\.

The interface mechanism predicts that optimizing the proxy reward raises capability without touching the format\-induced bias\. On a held1717\-prompt mixed split withk=4k=4\(6868records per model, record\-weighted across languages\), trusted accuracy improves from0\.6470\.647to0\.7350\.735, with gains on CN, EN, and JP, yet the exact\-match reward\-error VLB worsens from0\.4170\.417to0\.5000\.500\(Table[5](https://arxiv.org/html/2608.20362#S8.T5)\)\. Capability and verifier bias are orthogonal: accuracy improvement does not imply bias reduction, and the selection headroom identified in Section[7](https://arxiv.org/html/2608.20362#S7)is not recovered by proxy optimization\. This is the operational argument for measuring the bias as its own quantity*before*optimizing against it\.

Table 5:Rule\-GRPO adapter evaluation \(1717\-prompt mixed split,k=4k=4,6868records per model\)\. Trusted accuracy improves while the exact\-match reward\-error VLB stays high\.

## 9Discussion

### The workflow\.

The results compose into one workflow\. The rollout diagnosis \(Section[5](https://arxiv.org/html/2608.20362#S5)\) locates where a verifier injects language\-conditioned reward error; the interface probe \(Section[6](https://arxiv.org/html/2608.20362#S6)\) splits that error into a fixable answer\-format problem and a residual capability gap; and the selection bottleneck \(Section[7](https://arxiv.org/html/2608.20362#S7)\) shows that much of the residual gap is recoverable headroom\. The training audit \(Section[8](https://arxiv.org/html/2608.20362#S8)\) shows that skipping these steps is costly: a verifier can silently zero out the learning signal, and optimizing against it raises accuracy while leaving the language bias untouched\.

### An operational rule of thumb\.

The practitioner trigger in Algorithm[1](https://arxiv.org/html/2608.20362#alg1)—a single language with false\-negative rate above∼0\.3\\sim 0\.3and false\-negative\-rate VLB above∼0\.2\\sim 0\.2—emerges from the three models, and the ordering it prescribes \(interface first, where the exact\-match component lives; then selection, where sibling\-language agreement recovers correct answers without labels\) held across all three here\.

### Implications for verifier\-free rewards\.

The selection bottleneck has a direct bearing on the verifier\-free line of work\. Methods that remove the external verifier—training without one\[[8](https://arxiv.org/html/2608.20362#bib.bib2)\], using an intrinsic gradient signal\[[14](https://arxiv.org/html/2608.20362#bib.bib3)\], or self\-verifying with soft rewards\[[3](https://arxiv.org/html/2608.20362#bib.bib4)\]—implicitly hope that an internal signal can stand in for trusted correctness\. Our diagnosis suggests where such a signal is most needed and most attainable: not in generating correct answers in the weak language, which the models already do at the pass@kklevel, but in*selecting*among answers the model has produced\. The cross\-lingual aggregation result is evidence that a label\-free signal with this property exists: sibling\-language agreement identifies the correct cluster in over95%95\\%of the repaired cases\.

### Reusability\.

The same tuple\(π,\{xl\},k,v,r∗\)\(\\pi,\\\{x\_\{l\}\\\},k,v,r^\{\\ast\}\)and the same model\- and verifier\-agnostic metrics—false\-negative rate, false\-positive rate, reward\-error rate, andVLB\\mathrm\{VLB\}—drive every table here, from a synthetic battery to three live policies, a controlled interface probe, and a training run, read on a common scale\. This uniformity is the protocol’s contribution: it turns “is my multilingual verifier fair?” into a measurement with a fixed output format\.

## 10Conclusion

We presented a reusable measurement loop for multilingual verifier bias in RLVR and used it to establish three results\. Exact\-match verifier reward creates false\-negative\-dominated, language\-conditioned pressure on real Qwen and Llama MGSM rollouts, with the worst language set by the model rather than the benchmark\. A controlled plain\-numeric probe localizes the exact\-match component to the final\-answer interface, separating a removable format effect from a residual capability gap\. On fresh data, a label\-free cross\-lingual selection rule closes5555–78%78\\%of the average local\-majority selection gap and6363–88%88\\%of the JP gap, with an autopsy confirming that the repairs are genuinely cross\-lingual\. Together these results argue for a concrete discipline: audit multilingual RLVR rewards by language and by answer interface before optimizing against them\. The diagnosis spans two model families on MGSM JP/EN/CN and is corroborated on a manually audited483483\-problem MATH\-500 set; the cross\-lingual aggregation runs at inference time on parallel multilingual samples, and turning its selection signal into a training\-time reward is a natural next step\.

## References

- \[1\]X\. Cai, W\. Wang, F\. Liu, T\. Liu, G\. Niu, and M\. Sugiyama\(2025\)Reinforcement learning with verifiable yet noisy rewards under imperfect verifiers\.External Links:2510\.00915,[Link](https://arxiv.org/abs/2510.00915)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px1.p1.3)\.
- \[2\]K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman\(2021\)Training verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§1](https://arxiv.org/html/2608.20362#S1.p1.1),[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px5.p1.1)\.
- \[3\]S\. Dash, P\. Clavier, J\. Dang, M\. Galle, M\. Fadaee, A\. Üstün, and B\. Ermis\(2026\)Soft\-sverl: self\-verified reinforcement learning with soft rewards\.External Links:2605\.28561,[Link](https://arxiv.org/abs/2605.28561)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px4.p1.1),[§9](https://arxiv.org/html/2608.20362#S9.SS0.SSS0.Px3.p1.2)\.
- \[4\]A\. Elhady, E\. Agirre, and M\. Artetxe\(2026\)Cross\-lingual self\-consistency for multilingual reasoning with language models\.External Links:2606\.01464,[Link](https://arxiv.org/abs/2606.01464)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px3.p1.1)\.
- \[5\]X\. Fu and W\. Liu\(2025\)How reliable is multilingual llm\-as\-a\-judge?\.External Links:2505\.12201,[Link](https://arxiv.org/abs/2505.12201)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px2.p1.1)\.
- \[6\]L\. Helff, Q\. Delfosse, D\. Steinmann, R\. Harle, H\. Shindo, P\. Schramowski, W\. Stammer, K\. Kersting, and F\. Friedrich\(2026\)LLMs gaming verifiers: rlvr can lead to reward hacking\.External Links:2604\.15149,[Link](https://arxiv.org/abs/2604.15149)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px1.p1.3)\.
- \[7\]D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt\(2021\)Measuring mathematical problem solving with the math dataset\.External Links:2103\.03874,[Link](https://arxiv.org/abs/2103.03874)Cited by:[§1](https://arxiv.org/html/2608.20362#S1.p1.1),[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px5.p1.1)\.
- \[8\]W\. Liu, S\. Qi, X\. Wang, C\. Qian, Y\. Du, and Y\. He\(2025\)NOVER: incentive training for language models via verifier\-free reinforcement learning\.External Links:2505\.16022,[Link](https://arxiv.org/abs/2505.16022)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px4.p1.1),[§9](https://arxiv.org/html/2608.20362#S9.SS0.SSS0.Px3.p1.2)\.
- \[9\]C\. Park, J\. Kim, J\. Lee, S\. Bae, J\. Choo, and K\. Yoo\(2025\)Cross\-lingual collapse: how language\-centric foundation models shape reasoning in large language models\.External Links:2506\.05850,[Link](https://arxiv.org/abs/2506.05850)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px2.p1.1)\.
- \[10\]A\. Rad, K\. Filom, D\. Keivan, P\. Mohajerin Esfahani, and E\. Kamalinejad\(2026\)Rate or fate? RLVεR: reinforcement learning with verifiable noisy rewards\.External Links:2601\.04411,[Link](https://arxiv.org/abs/2601.04411)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px1.p1.3)\.
- \[11\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§1](https://arxiv.org/html/2608.20362#S1.p1.1)\.
- \[12\]F\. Shi, M\. Suzgun, M\. Freitag, X\. Wang, S\. Srivats, S\. Vosoughi, H\. W\. Chung, Y\. Tay, S\. Ruder, D\. Zhou, D\. Das, and J\. Wei\(2022\)Language models are multilingual chain\-of\-thought reasoners\.External Links:2210\.03057,[Link](https://arxiv.org/abs/2210.03057)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px2.p1.1)\.
- \[13\]T\. Teixeira, A\. C\. Erthal, J\. Belieni, B\. Canaverde, D\. Mesquita, M\. Faria, E\. d\. S\. da Silva, and A\. F\. T\. Martins\(2026\)MATH\-pt: a math reasoning benchmark for european and brazilian portuguese\.External Links:2604\.25926,[Link](https://arxiv.org/abs/2604.25926)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px5.p1.1)\.
- \[14\]X\. Wen, H\. Yu, L\. Zhu, and G\. Wang\(2026\)Verifier\-free rl for llms via intrinsic gradient\-norm reward\.External Links:2605\.09920,[Link](https://arxiv.org/abs/2605.09920)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px4.p1.1),[§9](https://arxiv.org/html/2608.20362#S9.SS0.SSS0.Px3.p1.2)\.
- \[15\]Z\. Xu, Y\. Li, F\. Jiang, B\. Ramasubramanian, L\. Niu, B\. Y\. Lin, and R\. Poovendran\(2025\)TinyV: reducing false negatives in verification improves rl for llm reasoning\.External Links:2505\.14625,[Link](https://arxiv.org/abs/2505.14625)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px1.p1.3)\.
- \[16\]Z\. Yu, T\. Li, C\. Wang, H\. Chen, and L\. Zhou\(2025\)Cross\-lingual consistency: a novel inference framework for advancing reasoning in large language models\.External Links:2504\.01857,[Link](https://arxiv.org/abs/2504.01857)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px3.p1.1)\.
- \[17\]Y\. Zhu and D\. Kang\(2026\)Noisy data is destructive to reinforcement learning with verifiable rewards\.External Links:2603\.16140,[Link](https://arxiv.org/abs/2603.16140)Cited by:[§2](https://arxiv.org/html/2608.20362#S2.SS0.SSS0.Px1.p1.3)\.