Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?

# How Much of RLVR’sOut-of-Domain Gain Is Answer Format?
Source: [https://arxiv.org/html/2610.20639](https://arxiv.org/html/2610.20639)
Zongye LyuAffiliation:Faculty of Information Technology, Monash University, Melbourne, VIC 3800, AustraliaEmail:[lyuzongye@gmail\.com](mailto:)

###### Abstract

Reinforcement learning with verifiable rewards \(RLVR\) is credited with gains beyond the domain its rewards check, such as higher MMLU accuracy after training on mathematics\. Such accuracies are usually read out of the model’s text by an answer extractor\. A model that learns to write its answer where the extractor looks can therefore score higher without answering better\. We measure how much of the gain reported by extractors copied from public code arises this way\. We generate outputs from a base model and its RL model and score them twice: with those extractors, and with one extractor of our own that reads a stated answer whether it is in the answer box`\\boxed\{\}`or in the text\. On four released 7B SimpleRL base–RL pairs, 82% of the out\-of\-domain multiple\-choice gain that these extractors report disappears under ours: it is a gain in answer format, not in answers\. The base models already state most of their answers in text, and RL moves them into the box, where these extractors look first\. Independent LLM readers, in place of our extractor, put the figure at 77%\. With a prompt that itself asks for the box, our extractor puts it at 79%\. On a 32B pair trained with another recipe, almost none of the reported gain is format gain: its base model boxes its answers when asked, and its RL model rarely does when not\. A reported gain is partly a property of the extractor that read it, and re\-scoring stored outputs with a second extractor shows how much, without re\-running RL\.

## 1Introduction

Reinforcement learning with verifiable rewards \(RLVR\) trains a language model on problems whose answers a program can check, mostly mathematics and code\. Its released models are also compared with their base models on benchmarks outside that domain, such as MMLU, and a gain there is read as a sign that what the model learned transfers\([Zeng et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib24);[Hu et al\., 2026](https://arxiv.org/html/2610.20639#bib.bib4);[Huan et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib23)\)\.

Such accuracies are usually read out of the generated text\. A program, the*answer extractor*, finds the answer in the text and compares it with the key, and evaluation and training code each ship their own\. We call the extractors copied from such public code*harness extractors*, and the gain that one of them reports for an RL model over its base model, published or not, a*reported gain*\. If RL changes where a model writes its answer, a harness extractor can report a gain that the answers do not contain\.

Figure[1](https://arxiv.org/html/2610.20639#S1.F1)shows this on one question\. Asked an ARC\-Challenge question, the base model Qwen2\.5\-7B ends with “The correct answer is B\) 5 m/s\.” Its RL\-trained successor from the SimpleRL\-Zoo release\([Zeng et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib24)\)ends with “Anna’s average speed is`\\boxed\{B\}`\.” The two answers are the same, and both are correct\. An extractor that reads only`\\boxed\{\}`counts the second and misses the first\. Over the 500 ARC\-Challenge questions we evaluate, that extractor reports that RL raised accuracy by 30\.6 points\. An extractor that finds a stated answer in either form reads a gain of 9\.8 points from the same outputs\.

Figure 1:The same answer, counted for one model only\.One ARC\-Challenge item and the end of each model’s output: Qwen2\.5\-7B \(base\) and the model trained from it by RL in the SimpleRL\-Zoo release \(RL\) both answer B\. A harness extractor that reads only\\boxed\{\}\(left\) does not read the base model’s answer and scores it wrong; our fixed extractor \(right\) reads a stated answer in either form\. Bottom: over the 500 ARC\-Challenge items we evaluate, the harness extractor reports that RL raised accuracy by 30\.6 points, and the fixed extractor reads 9\.8 from the same outputs\. The difference, 20\.8 points, is the format gain\.A gain on an out\-of\-domain benchmark can therefore mean two different things\. If the RL model gives more correct answers, what it learned transfers\. If it gives the same answers but writes more of them where the extractor looks, the benchmark has rewarded a change of format, not better answers\.

That RL can learn a format is known; how much of a reported gain the format accounts for, on released models, has not been measured\. On MATH\-500, a reward for merely producing a`\\boxed\{\}`expression recovers about half of the gain of the true reward\([Shao et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib1)\), and in RLVR with one training example a reward for format alone recovers most of it\([Wang et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib38)\)\. On their own RL runs,[Du et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib16)find that RL mostly adds formatting scaffolding and that what transfers from GSM8K to MATH\-500 is mostly format\. An essay by[Chandak et al\. \(2025b\)](https://arxiv.org/html/2610.20639#bib.bib42)compares the base\-model accuracies that seven RL papers report with official ones and argues that part of the reported gains comes from evaluations that under\-read the base model, answer format and parsing among the causes\. None of these takes a released checkpoint’s own outputs, with no weights changed, and measures how much of the gain reported on them is produced by the extractor\. To our knowledge, this paper is the first to do so\.

The two kinds of gain can be told apart on outputs that already exist\. We re\-read the same outputs of a base model and its RL model with one*fixed extractor*, the same for both models, which reads a stated answer whether it is in`\\boxed\{\}`or in the text\. The part of the reported gain that this extractor does not find is the*format gain*, and its fraction of the reported gain is the*format share*\(Section[2](https://arxiv.org/html/2610.20639#S2)\)\. We apply this to released base–RL pairs, with harness extractors copied from six public repositories \(Section[3](https://arxiv.org/html/2610.20639#S3)\), and find:

Most of the out\-of\-domain multiple\-choice gain is format gain\.On MMLU, MMLU\-Pro and ARC\-Challenge, 82% of the gain that harness extractors report for four released 7B SimpleRL base–RL pairs is format gain \(Figure[2](https://arxiv.org/html/2610.20639#S4.F2)\)\.

RL moves the stated answer into\\boxed\{\}\.The base models state most of their answers in the text, and the RL models write most of theirs in`\\boxed\{\}`\. In in\-domain free\-form mathematics, where both already box their answers, an extractor that reads only`\\boxed\{\}`shows a format gain close to zero \(Figure[3](https://arxiv.org/html/2610.20639#S4.F3)\)\.

Most of the gain remains format gain under our other choices and under independent readers\.The share moves by a few points when the fixed extractor reads more or less strictly, when one pair or benchmark is left out, or when only one of our two prompts is used\. The 82% pools only the comparisons with a clearly positive reported gain \(Section[2\.3](https://arxiv.org/html/2610.20639#S2.SS3)\); pooling every comparison whose reported gain is not zero lowers it the most, to 75%\. Independent LLM readers of the whole responses, in place of the fixed extractor, find 77% \(Figure[4](https://arxiv.org/html/2610.20639#S4.F4)\)\.

A reported gain depends on its extractor\.On identical outputs, the largest and smallest gains that harness extractors report for the same pair, benchmark and prompt differ by a median of 15 points\. An extractor that also reads letters in the text reports much less format gain \(Figure[5](https://arxiv.org/html/2610.20639#S4.F5)\)\.

These are the gains that harness extractors report on outputs we generated from the released checkpoints\. The two prompts are ours, and so is the use of the extractors that read`\\boxed\{\}`first on these benchmarks, which their repositories do not evaluate\. They are not the numbers a release publishes, whose own evaluations we did not reproduce\. They are also gains of 7B models: on the one larger pair with gains to split, a 32B model trained with another recipe, almost none of the reported gain is format gain \(Section[6](https://arxiv.org/html/2610.20639#S6)\)\. What does carry over to published numbers is how they are produced: where the releases we surveyed say how, their out\-of\-domain multiple\-choice results were almost always read out of generated text \(Section[4\.4](https://arxiv.org/html/2610.20639#S4.SS4)\), so a format gain can arise in them too\.

Our contribution is this measurement: how much of the out\-of\-domain multiple\-choice gain that harness extractors report for released RLVR checkpoints is format gain\. It needs only the two models’ outputs, re\-read by one extractor on a CPU, and no RL run\.

## 2Splitting a reported gain

We read the same outputs twice: once with a harness extractor, whose gain is the reported one, and once with a fixed extractor that treats the two models alike\. What the harness extractor reports beyond the fixed extractor’s gain is the format gain\.

### 2\.1Two gains from the same outputs

In Figure[1](https://arxiv.org/html/2610.20639#S1.F1), the 20\.8 points between the harness extractor’s 30\.6 and the fixed extractor’s 9\.8 are gain that exists only for the extractor that looks in the box\. We call them the*format gain*, and their fraction of the reported gain, here 68%, the*format share*\.

In general, fix a base–RL pair, a benchmark and a prompt, and lethhbe the harness extractor andω\\omegathe fixed one\. Each extractor gives each model an accuracy, and the two gains are

grep=acc⁡\(RL,h\)−acc⁡\(base,h\),gfix=acc⁡\(RL,ω\)−acc⁡\(base,ω\)\.g\_\{\\mathrm\{rep\}\}=\\mathrm\{acc\}\(\\text\{RL\},h\)\-\\mathrm\{acc\}\(\\text\{base\},h\),\\qquad g\_\{\\mathrm\{fix\}\}=\\mathrm\{acc\}\(\\text\{RL\},\\omega\)\-\\mathrm\{acc\}\(\\text\{base\},\\omega\)\.\(1\)The format gain isD=grep−gfixD=g\_\{\\mathrm\{rep\}\}\-g\_\{\\mathrm\{fix\}\}, and the format share isD/grepD/g\_\{\\mathrm\{rep\}\}\. Becauseω\\omegaapplies the same rules to both models’ outputs,gfixg\_\{\\mathrm\{fix\}\}is the gain in the answers asω\\omegareads them, andDDis the rest\.

We call one pair on one benchmark under one prompt, scored by one harness extractor, a*comparison*; Figure[1](https://arxiv.org/html/2610.20639#S1.F1)shows one\. Over many comparisons we sum before dividing, so the pooled format share is∑D/∑grep\\sum D/\\sum g\_\{\\mathrm\{rep\}\}\. This weights each comparison by its reported gain, so a comparison with a gain near zero cannot dominate the result\.

### 2\.2The fixed extractor

The fixed extractor has one job: to count the same answer the same way for both models\. It therefore reads an answer only where one is explicitly stated: inside`\\boxed\{\}`, after “the answer is” or “Answer:”, on a line of its own, or after “option” or “choice”\. Appendix[B](https://arxiv.org/html/2610.20639#A2)gives its rules in full, and two looser versions of it\.

Where it finds no answer, it abstains, and the item counts as wrong for either model\. It never drops an item, because an extractor allowed to skip items could raise its own accuracy by skipping the hard ones\. It abstains about equally often on the two models: on the comparisons behind the 82%, it reads an answer in 87% of the base models’ outputs and 88% of the RL models’\. Section[4\.3](https://arxiv.org/html/2610.20639#S4.SS3)checks what its abstentions cost each model\.

### 2\.3Which comparisons we pool

A format share is a fraction of a gain, so it means something only where there is a gain to split and where the fixed extractor can read both models\. We therefore pool the comparisons that meet two criteria\. The harness extractor must report a clearly positive gain, so that the ratio never divides by noise\. And the fixed extractor must find an answer in at least half of each model’s outputs, at rates within 20 points of each other, so that its gain is not read from a few outputs of one model\. \(The share of a model’s outputs in which an extractor finds an answer is that extractor’s*extraction rate*on the model\.\) Table[1](https://arxiv.org/html/2610.20639#S3.T1)gives the thresholds, and Section[4\.3](https://arxiv.org/html/2610.20639#S4.SS3)changes both\. Our 95% intervals resample items within each benchmark\. They do not cover variation between pairs or between benchmarks, which we show instead by leaving each one out\.

## 3Setup

To see what RL alone changes, we compare released pairs in which the RL model was trained from a named public base model by RL with verifiable rewards, with no supervised fine\-tuning or distillation in between\. We study seven such pairs from three releases \(Table[1](https://arxiv.org/html/2610.20639#S3.T1)\): SimpleRL\-Zoo\([Zeng et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib24)\), SimpleRL\-Zero from the same group, and Open\-Reasoner\-Zero\([Hu et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib25)\)\. Their base models are Qwen2\.5\-Math\-7B, Qwen2\.5\-7B and Qwen2\.5\-1\.5B\([Yang et al\., 2024b](https://arxiv.org/html/2610.20639#bib.bib27);[Yang et al\., 2024a](https://arxiv.org/html/2610.20639#bib.bib28)\), Mistral\-7B\-v0\.1\([Jiang et al\., 2023](https://arxiv.org/html/2610.20639#bib.bib29)\)and DeepSeek\-Math\-7B\([Shao et al\., 2024](https://arxiv.org/html/2610.20639#bib.bib30)\)\. Twelve other candidates fail this rule\. Three more pairs meet it, two larger pairs and a Llama\-3\.1\-8B pilot, and Section[6](https://arxiv.org/html/2610.20639#S6)reports them \(Appendix[A](https://arxiv.org/html/2610.20639#A1)\)\.

Table 1:What we ran\.Every model answers every item under both prompts, and every harness extractor that reads the benchmark’s kind of answer scores the same outputs\. The 82% pools the out\-of\-domain multiple\-choice comparisons of the four pairs marked∗\\astthat meet both criteria in the last two rows; the other three pairs show no clear out\-of\-domain multiple\-choice gain that the fixed extractor can read \(Section[3](https://arxiv.org/html/2610.20639#S3)\)\.base–RL pairsQwen2\.5\-Math\-7B with SimpleRL\-Zoo∗and with SimpleRL\-Zero∗; Qwen2\.5\-7B with SimpleRL\-Zoo∗; Mistral\-7B\-v0\.1 with SimpleRL\-Zoo∗; DeepSeek\-Math\-7B with SimpleRL\-Zoo; Qwen2\.5\-1\.5B with SimpleRL\-Zoo and with Open\-Reasoner\-Zerobenchmarksout of domain: MMLU, MMLU\-Pro and ARC\-Challenge \(multiple choice\), BBH \(free\-form\); in domain: MATH\-500 and GSM8K \(free\-form\), AQuA\-RAT and SAT\-Math \(multiple choice\); up to 500 items eachharness extractors for multiple choice, and where each looksMATH evaluation code: the last\\boxed\{\}
Open\-Reasoner\-Zero: the last\\boxed\{\}
Qwen2\.5\-Math: the last\\boxed\{\}first; if that gives no letter, a letter from the text \(on MMLU, the letter first\)
simple\-evals: a line “Answer: X”
\(five more read free\-form answers only; Appendix[E](https://arxiv.org/html/2610.20639#A5)\)promptsboxed\-instruction: asks for the final answer inside\\boxed\{\}, on multiple choice as a single letter; plain: no format request of its own\. Greedy decoding, at most 2,048 new tokenssignificant\-gain criterionthe reported gain’s lower 95% bound is at least 2 pointsextraction\-rate criterionthe fixed extractor reads an answer in at least 50% of each model’s outputs, and the two rates are within 20 points \(a comparison that meets it is*readable*\)The claim concerns three multiple\-choice benchmarks outside the domain that the rewards check: MMLU\([Hendrycks et al\., 2021a](https://arxiv.org/html/2610.20639#bib.bib8)\), MMLU\-Pro\([Wang et al\., 2024](https://arxiv.org/html/2610.20639#bib.bib9)\)and ARC\-Challenge\([Clark et al\., 2018](https://arxiv.org/html/2610.20639#bib.bib10)\)\. To tell whether a format gain goes with the domain or with the answer form, we add benchmarks for the other three combinations of the two\. Out of domain, BBH\([Suzgun et al\., 2022](https://arxiv.org/html/2610.20639#bib.bib11)\)supplies free\-form tasks; in domain, MATH\-500\([Hendrycks et al\., 2021b](https://arxiv.org/html/2610.20639#bib.bib12);[Lightman et al\., 2023](https://arxiv.org/html/2610.20639#bib.bib13)\)and GSM8K\([Cobbe et al\., 2021](https://arxiv.org/html/2610.20639#bib.bib14)\)supply free\-form mathematics, and AQuA\-RAT\([Ling et al\., 2017](https://arxiv.org/html/2610.20639#bib.bib36)\)and SAT\-Math\([Zhong et al\., 2024](https://arxiv.org/html/2610.20639#bib.bib37)\)multiple choice\. The items were frozen before any output was generated\.

The harness extractors come from six public repositories: the MATH evaluation code\([Hendrycks et al\., 2021b](https://arxiv.org/html/2610.20639#bib.bib12)\), Open\-Reasoner\-Zero, Qwen2\.5\-Math\([Yang et al\., 2024b](https://arxiv.org/html/2610.20639#bib.bib27)\), simple\-evals\([OpenAI, 2024](https://arxiv.org/html/2610.20639#bib.bib34)\), lm\-evaluation\-harness\([Biderman et al\., 2024](https://arxiv.org/html/2610.20639#bib.bib33)\)and verl\([Sheng et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib32)\)\. The four that read multiple choice look for the answer in different places \(Table[1](https://arxiv.org/html/2610.20639#S3.T1)\): two read only the last`\\boxed\{\}`, Qwen2\.5\-Math’s reads`\\boxed\{\}`first and then letters in the text \(on MMLU, the letter first\), and simple\-evals’ looks for a line “Answer: X”\. We run Qwen2\.5\-Math’s extractor as its repository does, with the*letter step*of its evaluation loop, which reads a letter from the text when the extraction function returns none\. We keep a verbatim copy of each extractor at a pinned commit and score with an executable version tested against it \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\)\. We apply every multiple\-choice extractor to all three benchmarks\. Where an extractor reads`\\boxed\{\}`first, this pairing is ours: its repository does not use it on these benchmarks\.

Each model answers each item under two prompts that differ in one sentence\. On multiple\-choice benchmarks the*boxed\-instruction prompt*adds “Reason step by step, then give your final answer as a single letter inside`\\boxed\{\}`”; the*plain prompt*adds no request for a format\. We use both to see whether a format gain remains when the prompt itself asks for the box\. Both are sent in the checkpoint’s chat template, where it ships one, and keep the template’s default system message\. For Qwen2\.5\-Math\-7B and its two RL models that message itself asks for the final answer “within`\\boxed\{\}`”, so for these two pairs the plain prompt also carries one request for the box\.

Four of the seven pairs show a clear out\-of\-domain multiple\-choice gain that the fixed extractor can read, and the 82% pools them: Qwen2\.5\-Math\-7B with its SimpleRL\-Zoo and SimpleRL\-Zero successors, and Qwen2\.5\-7B and Mistral\-7B\-v0\.1, each with SimpleRL\-Zoo\. On our outputs the DeepSeek\-Math\-7B pair shows no such gain to split, and the fixed extractor cannot read the Qwen2\.5\-1\.5B base model, whose outputs almost all run to the token limit \(Appendix[A](https://arxiv.org/html/2610.20639#A1)\)\.

## 4Results

### 4\.1Most of the out\-of\-domain multiple\-choice gain is format gain

Figure 2:Most of the reported gain disappears when the same outputs are read by one fixed extractor\.Mean gain of the RL model over its base model per comparison, on out\-of\-domain multiple choice, as the harness extractors report it \(orange\) and as the fixed extractor reads the same outputs \(blue\), by base–RL pair and by benchmark\. Each bar averages several comparisons; that the Qwen2\.5\-7B Zoo bar reports 30\.6, as Figure[1](https://arxiv.org/html/2610.20639#S1.F1)does, is a coincidence\. Zoo and Zero are the SimpleRL\-Zoo and SimpleRL\-Zero successors; Mistral\-7B contributes 2 comparisons\. Pooled over the 46 comparisons, the format share is 82\.2% \(95% CI 77\.4–87\.2%\); it is between 81% and 84% on each benchmark and between 76% and 87% for each Qwen pair\.We first ask how much of a reported gain is left when one extractor reads both models\. Little is left \(Figure[2](https://arxiv.org/html/2610.20639#S4.F2)\)\. On MMLU, MMLU\-Pro and ARC\-Challenge, the harness extractors report that RL raised accuracy by 22\.5 points per comparison on average; read by the fixed extractor, the same outputs show a gain of 4\.0 points\. Pooled, 82% of the reported gain is format gain, and it is at least 76% on each benchmark and for each of the three Qwen pairs\. Most of what these extractors report as an out\-of\-domain gain is a change in where the answers are written, not better answers\.

This holds for benchmarks that are both out of domain and multiple choice\. On the study’s other benchmarks the share is not stable enough to quote: it moves by 50 points or more with the rule for which comparisons are pooled \(Appendix[F](https://arxiv.org/html/2610.20639#A6)\)\. The study therefore cannot say whether the format gain goes with out\-of\-domain benchmarks or with multiple\-choice answers\.

### 4\.2RL moves the stated answer into\\boxed\{\}

Figure 3:The base models state most of their answers in the text; the RL models box them\.Each bar divides the outputs in which the fixed extractor reads an answer: those that contain a\\boxed\{\}, those with none whose answer is stated in the text, and other \(a line holding only the letter, or an answer after “option” or “choice”\)\. Out of domain, an extractor that reads only\\boxed\{\}cannot see most of the base models’ answers; in in\-domain free\-form mathematics both models box almost every answer\. Both prompts are pooled, over every pair whose two models receive the same input, which is all but Open\-Reasoner\-Zero’s, whose RL model ships a different chat template\. Out of domain, the four pairs of the 82% alone show the same pattern, slightly stronger \(Appendix[A](https://arxiv.org/html/2610.20639#A1)\)\.Why does a harness extractor see a gain that the fixed extractor does not? Because the two models write their answers in different places \(Figure[3](https://arxiv.org/html/2610.20639#S4.F3)\)\. Among the out\-of\-domain multiple\-choice outputs in which the fixed extractor reads an answer, 26\.6% of the base models’ contain a`\\boxed\{\}`, against 58\.3% of the RL models’\. Nearly all of the base models’ other answers are stated in the text, after “the answer is” or “Answer:”, with no`\\boxed\{\}`anywhere in the output\. An extractor that reads only`\\boxed\{\}`therefore misses most of the base models’ answers and fewer of the RL models’, and that difference is the format gain\.

The token limit could also leave answers unboxed, but it does not explain the difference\. An output cut off before its end has no final`\\boxed\{\}`\. Yet on the items where neither model reached the limit, the difference between the two models is wider, not narrower: 34\.7% against 81\.9% \(Figure[3](https://arxiv.org/html/2610.20639#S4.F3), middle rows\)\.

In\-domain free\-form mathematics is the contrast\. On MATH\-500 and GSM8K, 96% to 97% of the outputs in which the fixed extractor reads an answer contain a`\\boxed\{\}`, for both models\. The MATH and Open\-Reasoner\-Zero extractors, which read the last`\\boxed\{\}`, then see what the fixed extractor sees, and their format gain is close to zero\.

### 4\.3Most of the gain remains format gain under our other choices and under independent readers

Figure 4:The format share under each of our choices, and under independent readers\.Each row changes one thing from the pooled result at the top \(the band is its 95% interval\) and recomputes the share on the comparisons that remain, counted on the right\. A share above 100% means that the fixed extractor reads a loss\. The rows on the fixed extractor use the 18 comparisons that all three of its versions can read, on which our version gives 85\.5%\. Table[8](https://arxiv.org/html/2610.20639#A4.T8)in Appendix[D](https://arxiv.org/html/2610.20639#A4)gives every value and the conditions of each row\.The 82% rests on choices we made and on an extractor we wrote\. We therefore changed the choices shown in Figure[4](https://arxiv.org/html/2610.20639#S4.F4), one at a time, and then replaced the extractor with independent readers\. In every row of Figure[4](https://arxiv.org/html/2610.20639#S4.F4)most of the reported gain remains format gain: the share stays between 75% and 87%\.

Most choices move the share by a few points\. How strictly the fixed extractor reads, how the extraction\-rate criterion is set, and which pair or benchmark is left out each move it by at most 3\.2 points \(Appendix[D](https://arxiv.org/html/2610.20639#A4)\)\. Which comparisons are pooled moves the share most: pooling every comparison with a non\-zero reported gain lowers it to 75% \(Appendix[D](https://arxiv.org/html/2610.20639#A4)\)\.

The prompt matters too, and in the direction the mechanism predicts\. A base model that is asked for`\\boxed\{\}`uses it more often, so fewer of its answers are hidden from an extractor that reads only the box, and less of the gain is format gain\. The share is 79% under the boxed\-instruction prompt and 86% under the plain one\. Even when the prompt asks for the box, the RL models box more of their answers than the base models do, and most of the gain is still format gain\. A third prompt, from a pilot, gives a much lower share on one pair and benchmark \(Section[6](https://arxiv.org/html/2610.20639#S6)\)\.

The fixed extractor itself could be producing the result, so we also had the responses read by readers that share no code with it\. Two LLM readers each read 2,720 responses sampled from the outputs behind the 82%, the same items for a base model and its RL model, and a third settled their disagreements \(Appendix[C](https://arxiv.org/html/2610.20639#A3)\)\. Each saw the question, its options and one response, and never the model, the key or any extractor’s reading\. Scored by their readings, 77% of the reported gain is format gain\.

The readers and the fixed extractor read the same answer almost everywhere the extractor reads one: in 98\.6% of those responses\. They differ where it abstains\. In about a tenth of the responses, for base and RL models alike, the readers find an answer where the fixed extractor finds none\. Those answers are more often correct for the RL models, so the readers credit RL with a larger gain than the fixed extractor does, and a smaller part of the reported gain is left as format gain\.

By either reading, then, most of the reported gain is format gain\. Our sample of responses is too small to say whether 77% and 82% differ \(Appendix[C](https://arxiv.org/html/2610.20639#A3)\)\.

### 4\.4A reported gain depends on its extractor

Figure 5:Different extractors report different gains on the same outputs\.Left: the gain each multiple\-choice harness extractor reports on the outputs of Figure[1](https://arxiv.org/html/2610.20639#S1.F1)\(Qwen2\.5\-7B and SimpleRL\-Zoo, ARC\-Challenge, boxed\-instruction prompt, 500 items\); the dashed line is the fixed extractor’s reading\. Right: over all 112 combinations of pair, benchmark and prompt, the largest minus the smallest gain that the harness extractors report; the median is 15\.1 points\. On each extractor’s own comparisons in the 82%, the format share is 85% for the MATH extractor, 83% for Open\-Reasoner\-Zero’s and 56% for Qwen2\.5\-Math’s \(Table[13](https://arxiv.org/html/2610.20639#A4.T13)\)\.If a format gain comes from where the answers are written, the gain should change with the extractor that reads them\. It does, and sometimes even in sign \(Figure[5](https://arxiv.org/html/2610.20639#S4.F5)\)\. On the outputs of Figure[1](https://arxiv.org/html/2610.20639#S1.F1), the two extractors that read only`\\boxed\{\}`report a gain of 30\.6 points, and Qwen2\.5\-Math’s, which also takes letters from the text, reports 1\.0\. It reads base\-model answers that the fixed extractor misses there, mostly an option written out on a line of its own \(“B\) The dog becomes ill, and the tapeworm benefits\.”\), so on these outputs the fixed extractor’s 9\.8 overstates the gain in answers \(Appendix[B](https://arxiv.org/html/2610.20639#A2)\)\.

Simple\-evals’ extractor, which looks for the “Answer:” line that its own prompt asks for and ours do not, reports a loss of 1\.8: the RL models write their answers in`\\boxed\{\}`, and in most of the readable out\-of\-domain comparisons that it scores it gives them exactly 0%\. Across all pairs, benchmarks and prompts, the largest and smallest gains reported on the same outputs differ by a median of 15 points, and by 10 without simple\-evals’ two extractors\.

An extractor that also reads letters in the text therefore reports less format gain\. Of the gain that Qwen2\.5\-Math’s extractor reports on its own comparisons, 56% is format gain; for the two extractors that read only the box it is 85% and 83% \(Appendix[D](https://arxiv.org/html/2610.20639#A4)\)\. Across the seven combinations of base model and harness extractor that contribute comparisons to the 82%, the share is lower where the extractor’s extraction rate on the base model is higher\. This rests mostly on Qwen2\.5\-Math’s extractor, which reads the most of the base models’ outputs and reports the least format gain \(Appendix[H](https://arxiv.org/html/2610.20639#A8)\)\.

How are published out\-of\-domain multiple\-choice numbers produced? The 18 recent RLVR and reasoning\-model releases that we surveyed report 17 such results: 13 were computed by reading the answer out of generated text, one by scoring log\-likelihoods, and three do not say how \(Appendix[I](https://arxiv.org/html/2610.20639#A9)\)\. A format gain can arise wherever the answer is read out of text\.

## 5Related work

#### RL’s contribution and its transfer\.

[Yue et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib35)find that RLVR\-trained models beat their base models in pass@kkat smallkkwhile the base models score higher at largekk, and conclude that current RLVR does not elicit fundamentally new reasoning patterns\.[Hu et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib4)find that the benefits of RL post\-training are largely limited to the domains represented in its data, and[Huan et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib23)that RL\-tuned models transfer across domains better than supervised ones\. These studies ask what RL adds and whether it transfers; we ask how much of a measured gain the evaluation’s extractor produces\.

#### Measurement of RLVR gains\.

[Hochlehnert et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib5)show that reasoning benchmarks are highly sensitive to implementation choices and warn that formatting artifacts can produce spurious gains\.[Wu et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib2)argue that headline RLVR gains conflate policy improvement with measurement confounds, and[Zhu et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib20)find that small differences in evaluation conditions move results substantially\.[Chandak et al\. \(2025b\)](https://arxiv.org/html/2610.20639#bib.bib42)examine seven RL papers, find the base\-model accuracies they report below official ones in most, name answer format and parsing among the causes, and ask for sample\-level outputs to be released\. On the input side,[Li et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib3)and[Chilaka et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib21)control the prompt format given to each model and find that part of a gain remains when they do\.[Wang and Heckel \(2026\)](https://arxiv.org/html/2610.20639#bib.bib18)first adapt every model, the base model included, to the benchmark’s task format and distribution with test\-time RL, which changes the weights\. The gap between fine\-tuned models, SimpleRL\-Zoo among them, and their base models then largely disappears\. We hold the outputs fixed and change only the extractor\.

#### Format in RL training\.

[Shao et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib1)show that RL with a reward for merely producing a`\\boxed\{\}`expression raises Qwen2\.5\-Math\-7B’s MATH\-500 accuracy by 13\.8 points, about half of what the ground\-truth reward gives\.[Du et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib16)track RL checkpoints with one sparse autoencoder and find that RL mainly strengthens formatting features such as answer delimiters; their strict accuracy, which counts an answer only in the trained format, rises far more than their lenient accuracy\.[Xiaomi LLM\-Core Team \(2025\)](https://arxiv.org/html/2610.20639#bib.bib41)observe that their model’s first RL steps mainly learn to adapt to the answer extraction function\.[Chen et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib17)split the accuracy change of self\-revision into a content part and format parts, where revision makes an answer readable or unreadable\. Concurrently,[Chang et al\. \(2026\)](https://arxiv.org/html/2610.20639#bib.bib39)find, on their own RL runs \(GRPO\) and with human calibration, that most of the in\-domain gain a strict scorer measures is that scorer’s accuracy rising toward a lenient scorer’s\. We measure the same kind of split under three other conditions: on released checkpoints, as read by extractors copied from public code, and on out\-of\-domain multiple choice\.

#### The extractor as instrument\.

[Yu et al\. \(2025a\)](https://arxiv.org/html/2610.20639#bib.bib15)measure the accuracy of regular\-expression extractors and replace them with a trained one, and[Huang et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib40)find that rule\-based verifiers miss equivalent answers written in other formats\.[Molfese et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib22)find that rule\-based extraction of free\-text multiple\-choice answers often underestimates models, while LLM\-based extractors make systematic errors\.[Hua et al\. \(2025\)](https://arxiv.org/html/2610.20639#bib.bib7)trace much prompt sensitivity to heuristic scoring, including rigid matching of extracted answers, and[Chandak et al\. \(2025a\)](https://arxiv.org/html/2610.20639#bib.bib6)argue that answer matching should replace multiple choice\.[Cichoń and Dmitruk \(2026\)](https://arxiv.org/html/2610.20639#bib.bib19)find that changing the extractor moves a score only where a response’s reasoning and its final verdict line disagree or no such line exists\. An extractor’s accuracy does not say how much of a particular gain it produced, which turns on how the two models differ in where they write their answers\.

## 6Limitations

#### No published gain is decomposed\.

The gains we split are those that harness extractors report on outputs we generated from the checkpoints; the two prompts and the pairing of the box\-first extractors with these benchmarks are ours \(Section[3](https://arxiv.org/html/2610.20639#S3)\)\. The SimpleRL\-Zoo paper reports MMLU, MMLU\-STEM and GPQA\-Diamond accuracy before and after RL for its checkpoints and reads the gains as generalization\([Zeng et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib24)\)\. It describes its evaluation as using its own prompt, sampling at temperature 1\.0, and a script built on Qwen2\.5\-Math’s evaluation code, which reads letters in the text\. We did not reproduce that evaluation, so the 82% is not an estimate for its numbers\. Appendix[I](https://arxiv.org/html/2610.20639#A9)sets the published accuracies beside our readings of the same pairs\.

#### Extractors that read the box first\.

Nearly all of the gain behind the 82% is reported by extractors that read`\\boxed\{\}`first\. An extractor that also reads letters in the text reports less format gain, as Qwen2\.5\-Math’s does \(Section[4\.4](https://arxiv.org/html/2610.20639#S4.SS4)\)\. The share also depends on running that extractor as its repository runs it: its extraction function alone, without the letter step, gives a pooled share of 85% \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\)\.

#### One release family at 7B\.

The four pairs come from the SimpleRL releases and rest on three 7B base models, and the Qwen base models carry nearly all of the reported gain\. Whether the share holds in other model families is open\.

#### Prompts\.

Both of our prompts keep the chat template’s default system message, which for Qwen2\.5\-Math\-7B itself asks for`\\boxed\{\}`\. A pilot that replaced it with its own request for the letter in`\\boxed\{\}`gave a share of 31% for Qwen2\.5\-Math\-7B with SimpleRL\-Zoo on MMLU, against 88% and 86% under our boxed\-instruction and plain prompts\. Under the pilot’s prompt the base model stated no answer that the fixed extractor could read in 45% of its outputs, so most of the gain that the fixed extractor reads there was a gain in stating an answer at all \(Appendix[B](https://arxiv.org/html/2610.20639#A2)\)\. The share is thus a property of the pair, the prompt and the extractors together\.

#### Scale\.

The 82% does not hold on the one larger pair with gains to split\. On Qwen2\.5\-32B trained with DAPO\([Yu et al\., 2025b](https://arxiv.org/html/2610.20639#bib.bib26)\), which we set apart from the four pairs in advance, the format share is 2\.1% over 9 comparisons \(95% CI−18\.5\-18\.5% to 19\.9%\): the fixed extractor finds almost all of the gain that the harness extractors report\. Before generating these outputs we declared that a share below 40% on this pair would make the result a property of models of 7B and below\. The pair also shows little of what the mechanism of Section[4\.2](https://arxiv.org/html/2610.20639#S4.SS2)needs, a difference in where the two models write their answers: asked for the box, the 32B base model already boxes nearly every answer, and without that request the DAPO model rarely boxes its own \(Appendix[G](https://arxiv.org/html/2610.20639#A7)\)\. One pair cannot say whether this comes from scale, the DAPO recipe or the base model\. Appendix[G](https://arxiv.org/html/2610.20639#A7)also gives a Mistral\-Small\-24B pair, which has too few comparisons that meet both criteria to give a share\.

#### The fixed extractor can fail\.

When RL moves answers away from the places the fixed extractor reads, the extractors can disagree even on the sign of the change\. This happened in a pilot on a Llama\-3\.1\-8B pair\([Grattafiori et al\., 2024](https://arxiv.org/html/2610.20639#bib.bib31)\), where the fixed extractor reports a loss of 19 points\. The extraction\-rate criterion keeps that pilot’s comparisons out of the 82% \(Appendix[G](https://arxiv.org/html/2610.20639#A7)\)\.

#### The readers\.

The independent check of the fixed extractor used language models; no person has read the responses\.

## 7Conclusion

When an RLVR model is credited with an out\-of\-domain gain, part of the credit can belong to the program that read its answers\. On four released 7B SimpleRL pairs, 82% of the out\-of\-domain multiple\-choice gain that harness extractors report is the RL model writing its answer where those extractors look, in`\\boxed\{\}`, while the base model states its answers in the text\. On the one larger pair with gains to split, a 32B model, almost none of the reported gain is format gain: its base model boxes its answers when asked, and its RL model rarely does when not\. Where the releases we surveyed say how, their out\-of\-domain multiple\-choice results were almost always read out of generated text, so a format gain can arise in them in the same way\. We have not measured its size there, and an extractor that also reads letters in the text reports less of it\. Three practices make a format gain visible, and none needs RL to be re\-run\. Name the extractor behind every reported number\. Report the harness extractor’s extraction rate on the base model: it can be measured before any RL model exists, and in our main result the extractor that reads the most of the base models’ outputs reports the least format gain \(Appendix[H](https://arxiv.org/html/2610.20639#A8)\)\. And re\-score the same outputs with a second extractor that reads both models’ stated answers, which measures the format gain itself\.

### Reproducibility

Generation and scoring are separate, and everything after generation runs on a laptop\. The ancillary files of this arXiv version hold the analysis code, the frozen items with their hashes, the result files and the records behind the appendices’ counts, the independent readers’ responses, and the SHA\-256 hash of each of the 264 generation files\. Eight of those files ship with them; the other 256, which together exceed arXiv’s size limit, are available from the author \(Appendix[J](https://arxiv.org/html/2610.20639#A10)\)\. Each generation records the hash of the item file it was built from, so the evaluation set provably did not change between the two models\.

### AI use statement

In this work, we used generative AI tools to help propose and refine hypotheses, to give feedback on the methodology and experiments, and to help interpret results\. We have not used generative AI tools to generate synthetic data sets or for translation: the responses that the extractors and the readers read are outputs of publicly released checkpoints, the object of this study\. Qualitative or thematic data analysis is not applicable to this work\. We also used generative AI tools to edit prose, create figures, search the literature and the released checkpoints, and run the experiment pipeline\. We have reviewed all AI\-assisted work: the study’s design and structure, the decisions recorded in its analysis files, the review and revision of all content and the verification of every result remain with us, and every reported number was re\-derived from the raw result files\. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI\. The study itself also uses language models: the independent readers of Section[4\.3](https://arxiv.org/html/2610.20639#S4.SS3)are language models by design\.

## References

- S\. Biderman, H\. Schoelkopf, L\. Sutawika, L\. Gao, J\. Tow, B\. Abbasi, A\. F\. Aji, P\. S\. Ammanamanchi, S\. Black, J\. Clive, A\. DiPofi, J\. Etxaniz, B\. Fattori, J\. Z\. Forde, C\. Foster, J\. Hsu, M\. Jaiswal, W\. Y\. Lee, H\. Li, C\. Lovering, N\. Muennighoff, E\. Pavlick, J\. Phang, A\. Skowron, S\. Tan, X\. Tang, K\. A\. Wang, G\. I\. Winata, F\. Yvon, and A\. ZouLessons from the trenches on reproducible evaluation of language models\.arXiv preprint arXiv:2405\.14782\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p3.1)\.
- Chandaket al\.\(2025a\)N\. Chandak, S\. Goel, A\. Prabhu, M\. Hardt, and J\. GeipingAnswer matching outperforms multiple choice for language model evaluation\.arXiv preprint arXiv:2507\.02856\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px4.p1.1)\.
- Chandaket al\.\(2025b\)N\. Chandak, S\. Goel, and A\. PrabhuIncorrect baseline evaluations call into question recent LLM\-RL claims\.Note:Essay, linked from LessWrong on 29 May 2025\.[https://www\.lesswrong\.com/posts/p8rcMDRwEGeFAzCQS/incorrect\-baseline\-evaluations\-call\-into\-question\-recent\-llm](https://www.lesswrong.com/posts/p8rcMDRwEGeFAzCQS/incorrect-baseline-evaluations-call-into-question-recent-llm)Cited by:[§1](https://arxiv.org/html/2610.20639#S1.p5.1),[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.
- Changet al\.\(2026\)Y\. Chang, W\. Zhang, and Y\. WuLearned reporting preferences in RLVR can conflict with the current request\.arXiv preprint arXiv:2609\.36587\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2026\)M\. Chen, B\. Qu, and L\. WangThe calibration floor: format repair can masquerade as self\-correction at small\-to\-mid scale\.arXiv preprint arXiv:2608\.04355\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px3.p1.1)\.
- Chilakaet al\.\(2026\)S\. Chilaka, N\. Patel, C\. Xue, and E\. ShenPass@kkis not a property of a model: prompt format, decoding, and statistical power in evaluations of RL for reasoning\.Note:Preprints\.org, preprint 202608\.1495, version 1External Links:[Document](https://dx.doi.org/10.20944/preprints202608.1495.v1)Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.
- Cichoń and Dmitruk \(2026\)M\. Cichoń and B\. DmitrukA function\-level vulnerability score measures flag rate more than the model: protocol effects on paired benchmarks\.arXiv preprint arXiv:2609\.32890\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px4.p1.1)\.
- Clarket al\.\(2018\)P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. TafjordThink you have solved question answering? try ARC, the AI2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Duet al\.\(2026\)L\. Du, Y\. Tang, X\. Duan, and D\. LiuWhat does an LLM learn from reinforcement learning? A mechanistic interpretability perspective with Fixed\-SAE track\.arXiv preprint arXiv:2609\.15064\.Cited by:[§1](https://arxiv.org/html/2610.20639#S1.p5.1),[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px3.p1.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§6](https://arxiv.org/html/2610.20639#S6.SS0.SSS0.Px6.p1.1)\.
- Hendryckset al\.\(2021a\)D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. SteinhardtMeasuring massive multitask language understanding\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Hendryckset al\.\(2021b\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.NeurIPS Datasets and Benchmarks\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1),[§3](https://arxiv.org/html/2610.20639#S3.p3.1)\.
- Hochlehnertet al\.\(2025\)A\. Hochlehnert, H\. Bhatnagar, V\. Udandarao, S\. Albanie, A\. Prabhu, and M\. BethgeA sober look at progress in language model reasoning: pitfalls and paths to reproducibility\.InCOLM,External Links:[Link](https://openreview.net/forum?id=90UrTTxp5O)Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.
- Huet al\.\(2026\)C\. Hu, Y\. Zhu, A\. Kellermann, C\. Biddulph, S\. Waiwitlikhit, J\. Benn, and D\. KangBreaking barriers: do reinforcement post training gains transfer to unseen domains?\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.20639#S1.p1.1),[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px1.p1.1)\.
- Huet al\.\(2025\)J\. Hu, Y\. Zhang, Q\. Han, D\. Jiang, X\. Zhang, and H\. ShumOpen\-Reasoner\-Zero: an open source approach to scaling up reinforcement learning on the base model\.arXiv preprint arXiv:2503\.24290\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p1.1)\.
- Huaet al\.\(2025\)A\. Hua, K\. Tang, C\. Gu, J\. Gu, E\. Wong, and Y\. QinFlaw or artifact? rethinking prompt sensitivity in evaluating LLMs\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 19889–19899\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1006)Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px4.p1.1)\.
- Huanet al\.\(2025\)M\. Huan, Y\. Li, T\. Zheng, X\. Xu, S\. Kim, M\. Du, R\. Poovendran, G\. Neubig, and X\. YueDoes math reasoning improve general LLM capabilities? Understanding transferability of LLM reasoning\.arXiv preprint arXiv:2507\.00432\.Cited by:[§1](https://arxiv.org/html/2610.20639#S1.p1.1),[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px1.p1.1)\.
- Huanget al\.\(2025\)Y\. Huang, W\. Zeng, X\. Zeng, Q\. Zhu, and J\. HeFrom accuracy to robustness: a study of rule\- and model\-based verifiers in mathematical reasoning\.arXiv preprint arXiv:2505\.22203\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px4.p1.1)\.
- Jianget al\.\(2023\)A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. Renard Lavaud, M\. Lachaux, P\. Stock, T\. Le Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. El SayedMistral 7B\.arXiv preprint arXiv:2310\.06825\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p1.1)\.
- Liet al\.\(2026\)Y\. Li, W\. Liu, J\. Xie, B\. Gao, Y\. Wang, T\. Fu, and Y\. LiReachability is not realization: tracing the sources of LLM benchmark gains\.arXiv preprint arXiv:2608\.03219\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.
- Lightmanet al\.\(2023\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.arXiv preprint arXiv:2305\.20050\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Linget al\.\(2017\)W\. Ling, D\. Yogatama, C\. Dyer, and P\. BlunsomProgram induction by rationale generation: learning to solve and explain algebraic word problems\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vancouver, Canada,pp\. 158–167\.External Links:[Document](https://dx.doi.org/10.18653/v1/P17-1015)Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Maet al\.\(2025\)X\. Ma, Q\. Liu, D\. Jiang, G\. Zhang, Z\. Ma, and W\. ChenGeneral\-Reasoner: advancing LLM reasoning across all domains\.arXiv preprint arXiv:2505\.14652\.Cited by:[Appendix I](https://arxiv.org/html/2610.20639#A9.SS0.SSS0.Px1.p1.1)\.
- Molfeseet al\.\(2025\)F\. M\. Molfese, L\. Moroni, L\. Gioffré, A\. Scirè, S\. Conia, and R\. NavigliRight answer, wrong score: uncovering the inconsistencies of LLM evaluation in multiple\-choice question answering\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 18477–18494\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.950)Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px4.p1.1)\.
- OpenAI \(2024\)OpenAISimple\-evals\.Note:GitHub repositoryExternal Links:[Link](https://github.com/openai/simple-evals)Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p3.1)\.
- Shaoet al\.\(2025\)R\. Shao, S\. S\. Li, R\. Xin, S\. Geng, Y\. Wang, S\. Oh, S\. S\. Du, N\. Lambert, S\. Min, R\. Krishna, Y\. Tsvetkov, H\. Hajishirzi, P\. W\. Koh, and L\. ZettlemoyerSpurious rewards: rethinking training signals in RLVR\.arXiv preprint arXiv:2506\.10947\.Cited by:[§1](https://arxiv.org/html/2610.20639#S1.p5.1),[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px3.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p1.1)\.
- Shenget al\.\(2025\)G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. WuHybridFlow: a flexible and efficient RLHF framework\.InProceedings of the Twentieth European Conference on Computer Systems,pp\. 1279–1297\.External Links:[Document](https://dx.doi.org/10.1145/3689031.3696075)Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p3.1)\.
- Suzgunet al\.\(2022\)M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. V\. Le, E\. H\. Chi, D\. Zhou, and J\. WeiChallenging BIG\-Bench tasks and whether chain\-of\-thought can solve them\.arXiv preprint arXiv:2210\.09261\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Wang and Heckel \(2026\)K\. Wang and R\. HeckelTest\-time RL alignment exposes task familiarity artifacts in LLM benchmarks\.arXiv preprint arXiv:2603\.12875\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025\)Y\. Wang, Q\. Yang, Z\. Zeng, L\. Ren, L\. Liu, B\. Peng, H\. Cheng, X\. He, K\. Wang, J\. Gao, W\. Chen, S\. Wang, S\. S\. Du, and Y\. ShenReinforcement learning for reasoning in large language models with one training example\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=IBrRNLr6JA)Cited by:[§1](https://arxiv.org/html/2610.20639#S1.p5.1)\.
- Wanget al\.\(2024\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. ChenMMLU\-Pro: a more robust and challenging multi\-task language understanding benchmark\.InAdvances in Neural Information Processing Systems 37, Datasets and Benchmarks Track,Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Wuet al\.\(2025\)F\. Wu, A\. Tu, W\. Xuan, H\. Qi, X\. Huang, Q\. Zeng, S\. Talaei, Y\. Xiao, P\. Xia, X\. Tang, Y\. Zhuang, Y\. Li, B\. Hu, H\. Cao, W\. Shi, R\. Yang, N\. Liu, H\. Yao, G\. Liu, L\. E\. Li, A\. Saberi, N\. Yokoya, J\. Leskovec, and Y\. ChoiPosition: the hidden costs and measurement gaps of reinforcement learning with verifiable rewards\.arXiv preprint arXiv:2509\.21882\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.
- Xiaomi LLM\-Core Team \(2025\)Xiaomi LLM\-Core TeamMiMo: unlocking the reasoning potential of language model – from pretraining to posttraining\.arXiv preprint arXiv:2505\.07608\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px3.p1.1)\.
- Yanget al\.\(2024a\)A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p1.1)\.
- Yanget al\.\(2024b\)A\. Yang, B\. Zhang, B\. Hui, B\. Gao, B\. Yu, C\. Li, D\. Liu, J\. Tu, J\. Zhou, J\. Lin, K\. Lu, M\. Xue, R\. Lin, T\. Liu, X\. Ren, and Z\. ZhangQwen2\.5\-Math technical report: toward mathematical expert model via self\-improvement\.arXiv preprint arXiv:2409\.12122\.Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p1.1),[§3](https://arxiv.org/html/2610.20639#S3.p3.1)\.
- Yuet al\.\(2025a\)Q\. Yu, Z\. Zheng, S\. Song, Z\. Li, F\. Xiong, B\. Tang, and D\. ChenxFinder: large language models as automated evaluators for reliable evaluation\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px4.p1.1)\.
- Yuet al\.\(2025b\)Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, L\. Liu,et al\.DAPO: an open\-source LLM reinforcement learning system at scale\.arXiv preprint arXiv:2503\.14476\.Cited by:[§6](https://arxiv.org/html/2610.20639#S6.SS0.SSS0.Px5.p1.1)\.
- Yueet al\.\(2025\)Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. HuangDoes reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 57654–57689\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px1.p1.1)\.
- Zenget al\.\(2025\)W\. Zeng, Y\. Huang, Q\. Liu, W\. Liu, K\. He, Z\. Ma, and J\. HeSimpleRL\-Zoo: investigating and taming zero reinforcement learning for open base models in the wild\.InCOLM,External Links:[Link](https://openreview.net/forum?id=vSMCBUgrQj)Cited by:[Appendix I](https://arxiv.org/html/2610.20639#A9.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.20639#S1.p1.1),[§1](https://arxiv.org/html/2610.20639#S1.p3.1),[§3](https://arxiv.org/html/2610.20639#S3.p1.1),[§6](https://arxiv.org/html/2610.20639#S6.SS0.SSS0.Px1.p1.1)\.
- Zhonget al\.\(2024\)W\. Zhong, R\. Cui, Y\. Guo, Y\. Liang, S\. Lu, Y\. Wang, A\. Saied, W\. Chen, and N\. DuanAGIEval: a human\-centric benchmark for evaluating foundation models\.InFindings of the Association for Computational Linguistics: NAACL 2024,Mexico City, Mexico,pp\. 2299–2314\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-naacl.149)Cited by:[§3](https://arxiv.org/html/2610.20639#S3.p2.1)\.
- Zhuet al\.\(2025\)Y\. Zhu, L\. Sun, J\. Wu, W\. Lin, X\. Jian, G\. Zhao, C\. Jia, L\. Zhang, S\. Hu, Y\. Wu, and X\. ZhangEvaluation is all you need: strategic overclaiming of LLM reasoning capabilities through evaluation design\.arXiv preprint arXiv:2506\.04734\.Cited by:[§5](https://arxiv.org/html/2610.20639#S5.SS0.SSS0.Px2.p1.1)\.

In the appendices, the*main result*is the 82% of Section[4\.1](https://arxiv.org/html/2610.20639#S4.SS1), pooled over its 46 comparisons; the*grid*is everything Table[1](https://arxiv.org/html/2610.20639#S3.T1)lists; and the*letter step*is the step that Qwen2\.5\-Math’s evaluation loop runs after the extraction function: when the item’s key is a letter from A to E and the function’s answer is not such a letter, it reads a letter from the text \(Table[16](https://arxiv.org/html/2610.20639#A5.T16)gives the rule\)\. The step is part of how that extractor is run \(Section[3](https://arxiv.org/html/2610.20639#S3)and Appendix[E](https://arxiv.org/html/2610.20639#A5)\), and*the extraction function alone*is the extractor without it\. Shares are given as fractions, so that 0\.822 is the 82\.2% of the main text; Tables[8](https://arxiv.org/html/2610.20639#A4.T8)and[19](https://arxiv.org/html/2610.20639#A6.T19), which serve the body, keep percentages\.

## Appendix AHow the population narrows

This appendix gives the grid in counts \(Table[2](https://arxiv.org/html/2610.20639#A1.T2)\), traces the 46 comparisons behind the main result back to the 192 generation runs of the grid \(Table[3](https://arxiv.org/html/2610.20639#A1.T3)\), and lists the release candidates that the admission rule of Section[3](https://arxiv.org/html/2610.20639#S3)turned away\. A*combination*is one base–RL pair on one benchmark under one prompt; a comparison is a combination read by one harness extractor\.

Table 2:The grid in counts\.Table[1](https://arxiv.org/html/2610.20639#S3.T1)names what each row counts\. A comparison is one pair, benchmark, prompt and harness extractor\.base–RL pairs7, on 5 base models, from 3 releasesbenchmarks8: 4 out of domain, 4 in domain; up to 500 items eachprompts2: boxed\-instruction and plainharness extractors9, from 6 repositories; 4 read multiple choice, 8 free\-form answerscomparisons616out\-of\-domain multiple choice, readable100 comparisonsand with a clearly positive reported gain: the main result46 comparisons, from 4 pairsTable 3:From generation runs to the 46 comparisons of the main result\.Each row follows from the one above it; the text explains the second row\. The four groups are those of Table[19](https://arxiv.org/html/2610.20639#A6.T19)\.Of the first three rows, which build the population, only the second does not follow from the row above by multiplication: it pairs each RL model with its base model\. The 12 checkpoints are 5 base models and 7 RL models, and two of the base models have two RL successors each: Qwen2\.5\-Math\-7B \(SimpleRL\-Zoo and SimpleRL\-Zero\) and Qwen2\.5\-1\.5B \(SimpleRL\-Zoo and Open\-Reasoner\-Zero\)\. The outputs of these two base models therefore enter two pairs each, so the 12 checkpoints form 7 pairs, and each pair has 16 combinations of benchmark and prompt\. Multiple\-choice combinations are read by the 4 harness extractors that read letters and free\-form ones by 8 \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\), so each pair contributes 88 comparisons\.

By pair, the comparisons that meet the extraction\-rate criterion number 80 for Qwen2\.5\-Math\-7B with SimpleRL\-Zero, 68 for Qwen2\.5\-Math\-7B with SimpleRL\-Zoo, 52 for Qwen2\.5\-7B, 28 for Mistral\-7B\-v0\.1 and 12 for DeepSeek\-Math\-7B\. Neither pair on Qwen2\.5\-1\.5B has any, because that base model runs to the token limit on almost every item \(Appendix[H](https://arxiv.org/html/2610.20639#A8)\)\. Open\-Reasoner\-Zero\-1\.5B’s 88 comparisons are among the 616, but the pair is kept out of every pooled share: its base and RL models ship different chat templates, so the two would not receive the same input\. Figure[3](https://arxiv.org/html/2610.20639#S4.F3)is drawn over the other six pairs\. On the four pairs of the main result alone, 29\.5% of the base models’ out\-of\-domain multiple\-choice outputs thatω\\omegareads contain a`\\boxed\{\}`, against 66\.9% of the RL models’\. DeepSeek\-Math\-7B’s 4 readable out\-of\-domain multiple\-choice comparisons, all on MMLU under the plain prompt, show no significant reported gain, and its 2 comparisons that meet both criteria are in domain\.

The 46 comparisons of the main result come from 20 combinations\. By pair, 15 each come from Qwen2\.5\-Math\-7B with SimpleRL\-Zoo and with SimpleRL\-Zero, 14 from Qwen2\.5\-7B with SimpleRL\-Zoo and 2 from Mistral\-7B\-v0\.1 with SimpleRL\-Zoo; the three Qwen pairs carry 98\.5% of the summed reported gain\. By benchmark, 17 are on ARC\-Challenge, 16 on MMLU\-Pro and 13 on MMLU\. By prompt, 23 use the boxed\-instruction prompt and 23 the plain prompt\. By harness extractor, 18 are read by the MATH extractor, 17 by Open\-Reasoner\-Zero’s, 9 by Qwen2\.5\-Math’s and 2 by simple\-evals’ multiple\-choice extractor\. All 46 have a positive format gain, and in 43 its 95% interval excludes zero\.

Candidates were screened in August 2026 against the admission rule of Section[3](https://arxiv.org/html/2610.20639#S3): an RL model trained by RL with verifiable rewards directly from a named public base model\. Twelve did not meet it, almost all because RL started from a checkpoint that was already distilled, instruction\-tuned or fine\-tuned: Eurus\-2\-7B\-PRIME \(from a supervised fine\-tuned model\); five Skywork\-OR1 models and AceReason\-Nemotron\-7B and \-14B \(from DeepSeek\-R1\-distilled models\); Light\-R1\-7B\-DS \(distilled, then fine\-tuned\); Light\-R1\-32B \(from an instruction\-tuned model, trained with supervised fine\-tuning and DPO\); AceReason\-Nemotron\-1\.1\-7B \(with a supervised stage\); and OpenRLHF’s public checkpoints \(RLHF or DPO after a supervised stage\)\. A Llama\-3\.1\-8B pair from the SimpleRL\-Zoo release met the rule, but its base model is access\-gated, and it was run as a pilot\. Two larger pairs met the rule and were set apart from the pool before their generation; Appendix[G](https://arxiv.org/html/2610.20639#A7)reports all three\. The pool rests on one release family for these reasons: the only pairs from other labs that met the rule, Open\-Reasoner\-Zero on Qwen2\.5\-1\.5B and DAPO on Qwen2\.5\-32B, are outside it, the first for its chat templates and the second as a larger pair\.

## Appendix BThe fixed extractor and its variants

This appendix states the rules of the fixed extractorω\\omega, compares it with three variants that read more or less of each response, and shows how far a fixed extractor that reads the two models unequally moves the format share\.

#### Rules\.

For a multiple\-choice item,ω\\omegalooks for six answer positions in this order, stops at the first that occurs anywhere in the response, and takes its last occurrence:

1. 1\.a letter alone inside`\\boxed\{\}`, as in`\\boxed\{B\}`or`\\boxed\{\(B\)\}`;
2. 2\.a letter inside`\\boxed\{\\text\{\}\}`;
3. 3\.a letter after “answer is”, “the answer is” or “the correct answer is”, with or without a colon, bold marks or parentheses;
4. 4\.a letter after “Answer:”;
5. 5\.a line that holds only the letter, as “B”, “\(B\)”, “B\.” or “B\)”;
6. 6\.a letter after “option” or “choice”\.

Only the item’s own option letters count, A to D for four options and up to A to J for ten, and a letter followed directly by another letter is not read, so “the answer is Because” yields nothing\. Positions 3, 4 and 6 ignore case\. If no position occurs,ω\\omegareturns nothing and the item is scored wrong for that model\. For a free\-form item,ω\\omegareads the content of the last`\\boxed\{\}`, with nested braces balanced and an enclosing`\\text\{\}`removed; failing that, the text after the last “the answer is” or “final answer is”, up to the next period or line end; failing that, the text after the last “answer:”; otherwise nothing\. Free\-form answers are compared with the key after a light normalisation \(dollar signs, spaces, a final period, a percent sign and thousands separators removed, and numbers compared as numbers\)\. The comparison is the same for every extractor and both models, except that the Qwen2\.5\-Math extractor also normalises the key as its repository does \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\)\.

#### Variants\.

Three variants change how muchω\\omegareads \(Table[4](https://arxiv.org/html/2610.20639#A2.T4)\)\. The strict variant reads only the last`\\boxed\{\}`\. It applies the same rule as the boxed\-only harness extractor it is compared with, so its format gain is zero by construction, and it admits no comparison: on every one it reads under half of one model’s outputs or reads the two models’ at rates more than 20 points apart\. The permissive variant keepsω\\omegaand, whereω\\omegafinds nothing, adds looser forms of the same cues that do not require the letter to stand alone \(“answer is”, “answer:” or “answer =” followed by a letter\) and a letter at the very end of the response; for free\-form items it adds the last number in the response\. The maximal variant adds, after those, the last option letter anywhere in the response, so it almost never abstains\. On the 18 comparisons that the three versions other than the strict one all admit, all from the three Qwen pairs, the maximal variant exceedsω\\omegaby 0\.0184 \(95% CI 0\.0004 to 0\.0357, from a paired bootstrap of 10,000 draws over items\)\. Within this family the share therefore moves by less than 0\.02 andω\\omegagives the lowest value, although the interval’s lower end is too close to zero to fix the direction on its own\. On its own comparisons the maximal variant reads lower thanω\\omegaonly because the 5 comparisons it adds have a summed format gain of−8\.8\-8\.8points\. All four versions were written for this study, so the table shows that the share is insensitive within one family of rules, not that the family is right; Appendices[C](https://arxiv.org/html/2610.20639#A3)and[D](https://arxiv.org/html/2610.20639#A4)checkω\\omegaagainst readers and an extractor from outside it\.

Table 4:Reading more of each response moves the format share little, and upward on the same comparisons\.Each version admits comparisons under its own extraction\-rate criterion \(own comparisons, with that version’s extraction rate on them\); the last column holds the 18 comparisons that the other three versions all admit\. The harness extractor is the boxed\-only rule, applied as in the strict variant; the transcribed MATH extractor gives 0\.854 on its own 18 comparisons of the main result \(Table[8](https://arxiv.org/html/2610.20639#A4.T8)\)\.
#### A fixed extractor that reads the two models unequally\.

An extractor whose extraction rate differs between the two models moves the share far more than these variants do \(Table[5](https://arxiv.org/html/2610.20639#A2.T5)\)\. A pilot on Qwen2\.5\-Math\-7B and its SimpleRL\-Zoo successor on MMLU had two fixed extractors of its own\. Its permissive extractor falls back on loose patterns, such as “answer is” or “=” followed by any token\. Its abstaining extractor reads five ofω\\omega’s six answer positions, all but`\\boxed\{\\text\{\}\}`, and nothing else\. On the grid’s outputs for that pair and benchmark under the boxed\-instruction prompt \(500 items\), the permissive extractor gives a format share of 0\.5535, whereω\\omegagives 0\.8805\. It reads 80\.4% of the base model’s outputs and 99\.8% of the RL model’s, a 19\.4\-point gap that the extraction\-rate criterion admits, and so credits the RL model with a fixed\-extractor gain of 14\.2 points whereω\\omegafinds 3\.8\. It is not the permissive variant of Table[4](https://arxiv.org/html/2610.20639#A2.T4): on these outputs it reads less of the base model’s outputs thanω\\omegadoes\. On the pilot’s own outputs it read an answer into responses that state none in 10\.0% of the base model’s outputs and 4\.2% of the RL model’s\. Which model a guessing extractor favours depends on the outputs, soω\\omegaabstains instead\.

#### The pilot’s own result\.

The pilot generated its own outputs for the same pair and benchmark: 500 MMLU items drawn separately from the grid’s, at most 1,024 new tokens, and one prompt, which asks for the letter inside`\\boxed\{\}`in a system message\. That message replaces the default system message of the checkpoint’s chat template, which both prompts of the grid keep\. On those outputs the MATH extractor reports a gain of 18\.0 points andω\\omegareads 12\.4, a format share of 0\.311 \(95% CI 0\.084 to 0\.536\), against 0\.881 and 0\.864 for the same pair and benchmark under the grid’s two prompts\. Under the pilot’s prompt the base model states no answer thatω\\omegacan read in 45\.2% of its outputs \(24\.2% for the RL model\); whereω\\omegareads an answer, the two models’ accuracies are 55\.8% and 56\.7%\. Most of the RL model’s gain underω\\omegais then a gain in stating an answer at all, which the format share does not count as format\.ω\\omegareads 54\.8% of the base model’s outputs and 75\.8% of the RL model’s, a gap of 21\.0 points, so this comparison misses the extraction\-rate criterion by one point and would not enter the main result\. We set that criterion after the pilot, for free\-form comparisons in whichω\\omegareads one model and not the other, and it is applied without regard to the sign of the format gain\. The share is thus a property of the pair, the prompt and the two extractors together: under three prompts, this pair and benchmark give 0\.31, 0\.88 and 0\.86\.

Table 5:A fixed extractor that reads more of the RL model’s outputs lowers the format share\.The grid’s outputs for Qwen2\.5\-Math\-7B and its SimpleRL\-Zoo successor on MMLU under the boxed\-instruction prompt \(500 items\), read by a boxed\-only harness extractor that reports a gain of 31\.8 points\.
#### Abstention inside the main result\.

Over the 46 comparisons of the main result,ω\\omegaabstains on 13\.1% of the base models’ items and 12\.0% of the RL models’ on average, a difference of−1\.06\-1\.06points, or−1\.81\-1\.81weighted by reported gain\. In 17 comparisons, from 9 combinations, the two abstention rates differ by more than a quarter of the reported gain, in both directions \(from−9\.8\-9\.8to\+11\.4\+11\.4points, RL minus base\)\. The comparison of Figure[1](https://arxiv.org/html/2610.20639#S1.F1)is the most negative of them:ω\\omegafinds no answer in 56 of the base model’s 500 outputs and in 7 of the RL model’s\. On 43 of the 56, Qwen2\.5\-Math’s extractor reads the key, and these 43 are the 8\.6 points by which it scores the base model aboveω\\omega, 87\.0% against 78\.4% \(Section[4\.4](https://arxiv.org/html/2610.20639#S4.SS4)\)\. They state the answer in forms thatω\\omegadoes not read: an option written out on a line of its own after a line ending in a colon \(40 of them\), or “boxed\{B\}” without its backslash \(27\)\. On this comparison’s 80 sampled items the independent readers find the two models equally accurate, 90\.0% each, whereω\\omegareads a gain of 7\.5 points on the same responses\. On these outputs, then,ω\\omega’s gain of 9\.8 points overstates the gain in answers, and the share of 0\.68 understates the format gain\. No comparison is dropped for this\. Whereω\\omegaabstains more on the RL model, it can produce a share above one \(Appendix[D](https://arxiv.org/html/2610.20639#A4)\), and Appendix[C](https://arxiv.org/html/2610.20639#A3)measures what its abstentions cost each model\.

## Appendix CIndependent readers

This appendix gives the design of the check in Section[4\.3](https://arxiv.org/html/2610.20639#S4.SS3)as it was declared before the sample was drawn, the readers’ instructions, and the full results\.

#### Population and sample\.

The check was declared on the 50 comparisons that meet both criteria when Qwen2\.5\-Math’s extractor is scored with the extraction function alone\. They come from 20 combinations and rest on 34 generation runs: 14 of base models and 20 of RL models\. The 46 comparisons of the main result are among these 50 and come from the same 20 combinations, so the same sampled responses cover both\. For each of the 14 base\-model runs, 80 items were drawn uniformly without replacement \(seed 20260930\) from the items present in that run and in every RL\-model run that shares it, and the same items were read in all of these runs\. The sample holds 2,720 responses: 1,120 from base models and 1,600 from RL models\.

#### Blinding and readers\.

A reader saw the question, its lettered options and one response\. It never saw the key, which model wrote the response, or any extractor’s reading\. Responses carried random identifiers and were shuffled across all 34 runs before being cut into batches of ten\. Two readers read every response, Claude Sonnet 5 \(claude\-sonnet\-5\) and Claude Haiku 4\.5 \(claude\-haiku\-4\-5\-20251001\)\. Where they differed, Claude Opus 5\.5 \(claude\-opus\-5\-5\) read the response with the same instructions, and its reading decided\. Each batch was a fresh call of the Claude command\-line client in print mode \(claude \-p\), started in an empty temporary folder with session persistence off, so that a reader saw nothing but its own batch\. A response not returned in a readable form after three attempts was to be recorded as missing\.

#### Statistic and declared outcomes\.

The readers’ format share keeps each comparison’s reported gain on all items and correctsω\\omega’s fixed\-extractor gain by the sample estimate, on the comparison’s combination, of how much the readers’ reading changes the gain:

greaders=gfix−δ,δ=\(accω−accreaders\)RL−\(accω−accreaders\)base,g\_\{\\mathrm\{readers\}\}=g\_\{\\mathrm\{fix\}\}\-\\delta,\\qquad\\delta=\\left\(\\mathrm\{acc\}\_\{\\omega\}\-\\mathrm\{acc\}\_\{\\mathrm\{readers\}\}\\right\)\_\{\\mathrm\{RL\}\}\-\\left\(\\mathrm\{acc\}\_\{\\omega\}\-\\mathrm\{acc\}\_\{\\mathrm\{readers\}\}\\right\)\_\{\\mathrm\{base\}\},with accuracies in points on the combination’s sampled items\. The share is then pooled over the 50 comparisons as in Section[2](https://arxiv.org/html/2610.20639#S2), and the statistic is its differenceΔ\\Deltafromω\\omega’s share\. The 95% interval ofΔ\\Deltacomes from 10,000 bootstrap draws of the sampled items within each base\-model run, the same draw applied to that run and to every RL\-model run that shares it \(seed 20260930\)\. Three outcomes were declared before the run: the readers agree withω\\omegaif\|Δ\|≤0\.05\|\\Delta\|\\leq 0\.05and the interval lies inside\[−0\.10,\+0\.10\]\[\-0\.10,\+0\.10\]; they move the share if\|Δ\|\>0\.05\|\\Delta\|\>0\.05and the interval excludes zero; every other result is inconclusive\. Whatever the outcome, the main result staysω\\omega’s\. The declaration also fixed the statistics of Tables[6](https://arxiv.org/html/2610.20639#A3.T6)and[7](https://arxiv.org/html/2610.20639#A3.T7), except the rows marked descriptive\. Its SHA\-256 is written into the program that ran the readers, which refuses to run if the declaration has changed\.

#### Instructions\.

Every reader received the following instructions, shown here with lines broken for print, then the batch’s items, each with its identifier, question, options and the response between markers, and last the line “Now return one JSON line per item, in the order given, and nothing else\.”

```
You are checking what answer a language model gave to multiple-choice
questions. Below are several items. Each item has an id, a question,
its lettered options, and one response written by a model.

For each item, report the option letter that the response commits to
as its final answer to the question shown, or NONE.

Rules:
1. Do not solve the question and do not judge whether the answer is
   right. Report what the response says.
2. If the response gives an answer and later changes it, report the
   last answer it gives to this question.
3. If the response goes on to write and answer other questions, ignore
   them. Report only its answer to the question shown.
4. If the response names an option by its text rather than by its
   letter, report that option’s letter.
5. If the response commits to no option, or to more than one option,
   report NONE.

Return exactly one line of JSON per item, in the order given, and
nothing else:
{"id": "<id>", "final": "<one letter or NONE>", "quote": "<the
shortest phrase from the response that shows this answer, or an empty
string>"}
```

Table 6:Whereω\\omegareads a letter, the readers read the same one; its misses are abstentions\.Percentages of the responses read in each group, except in the descriptive rows, whose denominator is the count above them\. The descriptive rows were computed after the declared statistics had been read\. The two groups are not matched one to one \(a sampled Qwen2\.5\-Math\-7B base response is compared with two RL models\), so the difference between their accuracies is not a gain\.base modelsRL modelsresponses read1,1201,600ω\\omegaand the readers agree \(the same letter, or neither finds one\)88\.388\.8ω\\omegafinds no answer, the readers find one10\.59\.9ω\\omegareads a letter, the readers find none0\.60\.3ω\\omegaand the readers read different letters0\.50\.9accuracy underω\\omega53\.755\.1accuracy under the readers60\.062\.3*descriptive*responses in whichω\\omegareads a letter9411,391the readers read the same letter \(%\)98\.698\.6responses in whichω\\omegafinds no answer179209the readers find a letter \(%\)65\.976\.1the readers’ letter is the key \(%\)39\.152\.2Table 7:Agreement with the readers by the answer positionω\\omegaread\.Agreement is the percentage of responses on which the readers’ reading equalsω\\omega’s: the same letter, or, in the last row, no answer\. Positions follow the order of Appendix[B](https://arxiv.org/html/2610.20639#A2)\. Rates over fewer than 20 responses are in parentheses\.
#### Result\.

Under the readers’ reading the format share over the 50 comparisons of the declaration is 0\.801 \(95% CI 0\.735 to 0\.869\), against 0\.847 underω\\omega:Δ=−0\.0465\\Delta=\-0\.0465\(4\.7 points\), with an interval from−0\.1125\-0\.1125to\+0\.0213\+0\.0213\(its lower end 11\.2 points below zero\)\. The readers’ interval isω\\omega’s share plus the 95% interval of the difference between the two, from the bootstrap over the 2,720 responses they read with each comparison’s full\-sample gains held fixed; unlike the main result’s interval, it resamples only the sampled items, not the items behind each comparison’s gains\. The point lies inside±0\.05\\pm 0\.05, but the interval’s lower end passes−0\.10\-0\.10and the interval contains zero, so the declared outcome is inconclusive\. The same statistic over the 46 comparisons of the main result, computed after the readings were known, puts the readers’ share at 0\.771 \(95% CI 0\.696 to 0\.849\), against 0\.822 underω\\omega:Δ=−0\.0508\\Delta=\-0\.0508, with an interval from−0\.1259\-0\.1259to\+0\.0266\+0\.0266\. Its point lies just outside±0\.05\\pm 0\.05and its interval contains zero, so the declared rule would read it as inconclusive too\. The first two readers gave the same reading on 98\.2% of the 2,719 responses that both returned \(Cohen’sκ\\kappa0\.978\)\. The tie\-break reader read the 51 responses on which they differed or one of them returned no reading, and no response is missing after it\.

Whereω\\omegareads a letter, the readers read the same letter on 98\.6% of responses in both groups \(Table[6](https://arxiv.org/html/2610.20639#A3.T6)\)\. Most ofω\\omega’s disagreements with the readers are abstentions, and where it abstains, the readers’ letter is the key more often for the RL models \(52\.2%\) than for the base models \(39\.1%\)\. The abstentions therefore cost the RL models slightly more accuracy than the base models, which is why the readers’ share is lower thanω\\omega’s\. By answer position \(Table[7](https://arxiv.org/html/2610.20639#A3.T7)\), the readers agree withω\\omegaon at least 96\.8% of the responses in whichω\\omegareads`\\boxed\{\}`or “the answer is”\. Among the positions read in at least 20 responses, they agree least often whereω\\omegareads “Answer:” in an RL model’s response \(84\.0% of 50\); on the three responses in whichω\\omegareads a line holding only the letter, they never agree\. The release also holds a blind sample of 120 responses, 60 from base and 60 from RL models \(seed 20260931\), prepared for a human check\. We have not run that check, and no result here depends on it\.

## Appendix DRobustness in full

This appendix gives every check of Section[4\.3](https://arxiv.org/html/2610.20639#S4.SS3)in full\. Each check changes one thing and keeps the rest of the main result: which comparisons enter the ratio \(Tables[9](https://arxiv.org/html/2610.20639#A4.T9)to[11](https://arxiv.org/html/2610.20639#A4.T11)\), leaving out one pair, benchmark, extractor or prompt \(Table[12](https://arxiv.org/html/2610.20639#A4.T12)\), the harness extractor \(Table[13](https://arxiv.org/html/2610.20639#A4.T13)\), shares above one \(Table[14](https://arxiv.org/html/2610.20639#A4.T14)\), and an external fixed extractor \(Table[15](https://arxiv.org/html/2610.20639#A4.T15)\)\.

#### Intervals\.

Our 95% intervals on the format share come from a bootstrap that resamples items within each benchmark and applies the same draw to every comparison built on that benchmark, because every extractor reads the same outputs\. They measure the uncertainty from item sampling only\. Variation between pairs and between benchmarks, of which the main result has four and three, is shown by leaving each one out \(Table[12](https://arxiv.org/html/2610.20639#A4.T12)\)\.

Table 8:The values of Figure[4](https://arxiv.org/html/2610.20639#S4.F4)\.One thing changes per row; everything else is as in the main result \(first row\)\. The three rows on the harness extractors one by one and on Qwen2\.5\-Math’s extraction function alone are not drawn in the figure\. Rows that admit more comparisons can include pairs outside the main result’s four, from the six of Figure[3](https://arxiv.org/html/2610.20639#S4.F3)\. Shares in percent; changes are computed before rounding\. The readers’ interval is built differently from the first row’s, from the sampled responses they read rather than from the benchmarks’ resampled items \(Appendix[C](https://arxiv.org/html/2610.20639#A3)\)\.∗On the 18 comparisons of the three Qwen pairs that all three versions admit, with the\\boxed\{\}rule that MATH and Open\-Reasoner\-Zero share as the harness extractor\.
#### Which comparisons enter the ratio\.

The three rules for which comparisons enter the ratio move the share by at most 0\.073, and both other rules lower it \(Table[9](https://arxiv.org/html/2610.20639#A4.T9)\); under the same rules each of the other three domain and answer\-form groups moves by 0\.515 to 5\.434 \(Appendix[F](https://arxiv.org/html/2610.20639#A6)\)\. The 4 comparisons with a reported gain of exactly zero add nothing to the summed reported gain but 32\.0 points to the summed format gain, which separates the second rule from the third\. The 54 comparisons that the second rule adds to the main result’s 46 include 37 with a negative reported gain, 20 of them read by simple\-evals’ extractor\. Summed over all 54 added comparisons, the reported gain is−280\.6\-280\.6points and the format gain−253\.0\-253\.0, a ratio of 0\.902\. That ratio is above the main result’s 0\.822, so admitting them takes away a larger part of the summed format gain than of the summed reported gain, and the share falls\.

Table 9:Admitting comparisons without a significant reported gain lowers the share\.Out\-of\-domain multiple choice, all comparisons that meet the extraction\-rate criterion\.Five settings of the extraction\-rate criterion move the share within 0\.7933 to 0\.8222, and the setting used, at least 50% of each model’s outputs read with the two rates within 20 points, gives the highest of the five \(Table[10](https://arxiv.org/html/2610.20639#A4.T10)\)\. Without any criterion, four comparisons enter, all under the boxed\-instruction prompt\. Three are of DeepSeek\-Math\-7B: MMLU and ARC\-Challenge read by Open\-Reasoner\-Zero’s extractor, and ARC\-Challenge read by Qwen2\.5\-Math’s\. On these two benchmarksω\\omegareads 55\.2% and 51\.6% of the base model’s outputs, against 77\.6% and 85\.6% of the RL model’s\. The fourth is of Qwen2\.5\-1\.5B with SimpleRL\-Zoo, on MMLU\-Pro read by Qwen2\.5\-Math’s extractor, whereω\\omegareads 3\.6% of the base model’s outputs and 10\.8% of the RL model’s\. Dropping the chat\-template rule of Appendix[A](https://arxiv.org/html/2610.20639#A1)as well readmits Open\-Reasoner\-Zero\-1\.5B, whose base model’s outputsω\\omegareads at 3\.6–9\.2% against 81\.8–98\.0% for the RL model\. Its fixed\-extractor gain then rests on a base\-model accuracy read from under a tenth of the outputs, so its format gain is not identified; with it included, the setting without any criterion reads 0\.3564 over 68 comparisons\.

Table 10:The extraction\-rate criterion used gives the highest share of five settings\.Out\-of\-domain multiple choice, comparisons with a significant reported gain\.Because the ratio weights each comparison by its reported gain, comparisons near the significant\-gain threshold carry little weight: those with a reported gain under 5 points hold 0\.4% of the summed reported gain, and those under 10 points 8\.8%\. Dropping the comparisons below 5, 10, 15 or 20 points keeps the share within 0\.0455 of 0\.8222, and not in one direction \(Table[11](https://arxiv.org/html/2610.20639#A4.T11)\)\.

Table 11:Comparisons with small reported gains carry little weight\.The main result with the comparisons below each reported gain removed\.reported gain belowcomparisonstheir share of thecomparisonsformat share\(points\)removedsummed reported gain \(%\)kept510\.4450\.819110118\.8350\.8387151817\.0280\.8495202527\.5210\.8677
#### Leaving out one pair, benchmark, extractor or prompt\.

Leaving out any one pair moves the share by at most 0\.032, any one benchmark by at most 0\.016, and any one harness extractor by at most 0\.031 \(Table[12](https://arxiv.org/html/2610.20639#A4.T12)\); these bounds are the largest changes rounded up\. Leaving out one harness extractor never removes the last\-`\\boxed\{\}`rule, which the MATH and Open\-Reasoner\-Zero extractors share; leaving out both leaves Qwen2\.5\-Math’s 9 comparisons and simple\-evals’ 2, with a share of 0\.634\. Leaving out either Qwen2\.5\-Math\-7B pair does not leave out that base model, since its SimpleRL\-Zoo and SimpleRL\-Zero pairs share its outputs\. Leaving out one prompt moves the share most, to 0\.7879 or 0\.8557, because the two prompts’ shares differ \(Section[4\.3](https://arxiv.org/html/2610.20639#S4.SS3)\) and each carries about half of the summed reported gain, the plain prompt 50\.6%\. Under the boxed\-instruction prompt, fewer of the base models’ outputs thatω\\omegareads contain a`\\boxed\{\}`than of the RL models’, 45\.4% against 69\.5%, over the pairs of Figure[3](https://arxiv.org/html/2610.20639#S4.F3)\.

Table 12:Leaving out any one pair, benchmark or harness extractor moves the share by at most 0\.032\.The main result \(0\.8222\) without each group\. Zoo and Zero are the SimpleRL\-Zoo and SimpleRL\-Zero RL models\.groupleft outformat sharepairQwen2\.5\-Math\-7B, Zoo0\.8345Qwen2\.5\-Math\-7B, Zero0\.8410Qwen2\.5\-7B, Zoo0\.7907Mistral\-7B\-v0\.1, Zoo0\.8182benchmarkMMLU0\.8137MMLU\-Pro0\.8203ARC\-Challenge0\.8376harness extractorMATH \(\\boxed\{\}only\)0\.7920Open\-Reasoner\-Zero0\.8137Qwen2\.5\-Math0\.8493simple\-evals, multiple choice0\.8182promptplain \(boxed\-instruction prompt only\)0\.7879boxed\-instruction \(plain prompt only\)0\.8557
#### The harness extractor\.

The three extractors that carry the result do not agree \(Table[13](https://arxiv.org/html/2610.20639#A4.T13)\)\. The two that read only the last`\\boxed\{\}`give 0\.854 and 0\.835 on their own comparisons, and Qwen2\.5\-Math’s extractor, which also reads letters in the text, gives 0\.562 on its own\. The 8 comparisons that all three admit show the same gap\. The two that agree run the same algorithm \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\), so theirs is not the agreement of two independent rules\. Together they carry 89\.1% of the summed reported gain of the main result, and Qwen2\.5\-Math’s extractor carries 9\.4%\. Eight of its 9 comparisons are on ARC\-Challenge and MMLU\-Pro, which its repository does not evaluate; there it applies its free\-form rules, which read the last`\\boxed\{\}`before any “the answer is” \(Table[16](https://arxiv.org/html/2610.20639#A5.T16)\), and then the letter step\. Its one comparison on MMLU, the pairing that its repository also makes, gives 0\.378\. Extractors that read`\\boxed\{\}`first thus carry 97\.6% of the summed reported gain\.

Run as its extraction function alone, without the letter step, Qwen2\.5\-Math’s extractor gives 0\.842 over 13 comparisons, against 0\.562 over 9 as its repository runs it\. The three extractors then give 0\.829 to 0\.854 on the 12 comparisons that all three admit\. No comparison is admitted for all four extractors, because simple\-evals’ extractor meets the significant\-gain criterion in only 2 of its 25 readable comparisons\.

Table 13:Qwen2\.5\-Math’s extractor, which also reads letters in the text, reports less format gain than the two that read only the box; its extraction function alone reports about as much\.Out\-of\-domain multiple choice\. Each extractor has 25 comparisons that meet the extraction\-rate criterion; the significant\-gain criterion removes some \(removed\) and keeps the rest \(own comparisons\)\. Weight is the extractor’s part of the summed reported gain of the main result\.
#### Shares above one\.

Six of the 46 comparisons have a share above one, becauseω\\omegareads a small negative gain where the harness extractor reads a positive one \(Table[14](https://arxiv.org/html/2610.20639#A4.T14)\)\. They carry 10\.9% of the summed format gain and 7\.8% of the summed reported gain, and without them the share is 0\.7947\. Five of the six come from two MMLU\-Pro combinations under the plain prompt, in whichω\\omegareads fewer of the RL model’s outputs than of the base model’s: 66\.8% against 76\.4% for Qwen2\.5\-Math\-7B with SimpleRL\-Zoo, and 69\.0% against 80\.4% for Qwen2\.5\-7B\. There the fixed\-extractor gains are−2\.4\-2\.4and−1\.6\-1\.6points, with intervals that include zero\. The six pass both criteria and are kept: dropping comparisons by their share would select on the outcome\.

Table 14:The six comparisons with a share above one\.Gains in accuracy points, grouped by combination\.harness extractorreported gainfixed\-extractor gainformat share*Qwen2\.5\-7B with SimpleRL\-Zoo, MMLU\-Pro, plain prompt*MATH \(\\boxed\{\}only\)25\.2−1\.6\-1\.61\.063Open\-Reasoner\-Zero23\.2−1\.6\-1\.61\.069Qwen2\.5\-Math7\.6−1\.6\-1\.61\.211*Qwen2\.5\-Math\-7B with SimpleRL\-Zoo, MMLU\-Pro, plain prompt*MATH \(\\boxed\{\}only\)11\.4−2\.4\-2\.41\.211Open\-Reasoner\-Zero8\.4−2\.4\-2\.41\.286*Mistral\-7B\-v0\.1 with SimpleRL\-Zoo, MMLU\-Pro, boxed\-instruction prompt*simple\-evals, multiple choice4\.4−2\.4\-2\.41\.545
#### An external fixed extractor\.

Qwen2\.5\-Math’s extractor, which this study did not write \(the authors of the Qwen2\.5\-Math checkpoints ship it, in evaluation code adapted from math\-evaluation\-harness\), gives 0\.775 inω\\omega’s place over its own 25 comparisons, againstω\\omega’s 0\.855 over its 18 \(Table[15](https://arxiv.org/html/2610.20639#A4.T15)\)\. The two sets share 18 comparisons, all ofω\\omega’s, and on those the pooled shares are within 0\.03 of each other, with an interval that includes zero\. That agreement is a cancellation\. On the 6 MMLU comparisons, where the external extractor applies its letter rule, it gives 1\.053 againstω\\omega’s 0\.858; on the 12 MMLU\-Pro and ARC\-Challenge comparisons, where it applies its free\-form rules and then the letter step, it gives 0\.718 against 0\.854\. Comparison by comparison the two shares differ by 0\.25 on average, and in 5 of the 18 the two extractors disagree on the sign of the RL model’s gain\.

Item by item, over every out\-of\-domain multiple\-choice output of the six grid pairs other than Open\-Reasoner\-Zero \(36 combinations\), the two give the same score on 85\.1% of the base models’ outputs and 86\.1% of the RL models’\.ω\\omegascores the base models 0\.91 points higher than the Qwen2\.5\-Math extractor does and the RL models 0\.76 points higher, so item by item the two extractors differ by about as much on either model\. The check does not show, however, that an extractor written outside this study reproducesω\\omega’s share comparison by comparison\. The check ofω\\omegathat remains is the independent readers of Appendix[C](https://arxiv.org/html/2610.20639#A3), who read the same letter asω\\omegain 98\.6% of the responses in which it reads one\.

Table 15:With an external extractor inω\\omega’s place, the pooled share on the comparisons both admit is within 0\.03 ofω\\omega’s, by offsetting differences\.Out\-of\-domain multiple choice, the six grid pairs other than Open\-Reasoner\-Zero, with the boxed\-only harness extractor as in Table[4](https://arxiv.org/html/2610.20639#A2.T4);ω\\omegaadmits comparisons of the three Qwen pairs only\. Each difference is Qwen2\.5\-Math’s share minusω\\omega’s, with an interval paired over items \(10,000 bootstrap draws\)\. The last two blocks split the 18 comparisons that both admit: on MMLU the external extractor applies its letter rule, on the other two benchmarks its free\-form rules and then the letter step\.

## Appendix EThe harness extractors

This appendix lists the nine harness extractors \(Table[16](https://arxiv.org/html/2610.20639#A5.T16)\), what each returns when it finds no answer, how they were copied, where the executable versions depart from the source and by how much \(Table[17](https://arxiv.org/html/2610.20639#A5.T17)\), and how multiple choice is scored\.

Table 16:The nine harness extractors\.Each is copied from its repository at the commit shown; files are given relative to the repository root\. “Nothing” means that the extractor returns no value\.extractorsource and rolewhat it readswhen nothing matches*multiple choice and free\-form*MATH \(\\boxed\{\}only\)hendrycks/math@ 940d1bf \(modeling/dataset/util\.py,modeling/eval\_math\_gpt\.py\); the MATH benchmark’s evaluation codethe last\\boxed\{\}nothingQwen2\.5\-MathQwenLM/Qwen2\.5\-Math@ cb098e1 \(evaluation/parser\.py,evaluation/math\_eval\.py\); evaluation code, with a copy in the SimpleRL\-Zoo code; the extraction function, then the letter step of the loop that calls itbenchmark\-aware: for AQuA\-RAT, SAT\-Math and MMLU \(run as MMLU\-STEM\), the letter rule: a letter A–E standing alone, the first after the last “answer is” or “choice is” \(only in the first paragraph when either occurs twice\), else the last in the response, read case\-insensitively; otherwise, in turn, “final answer is $X$\. I hope”, the last\\boxed\{\}, the last “the answer is”, the last “final answer is” \(or its Chinese form\), the last number; on MMLU\-Pro the result is narrowed to the last letter A–E in it; then normalised\. Then, on an item keyed A–E whose result is not such a letter, the letter step applies the letter rule to the whole response; this can change an answer only on ARC\-Challenge and on the 287 of MMLU\-Pro’s 500 items keyed A–Eon items keyed A–E, the response’s own text when it holds no letter; otherwise the empty stringOpen\-Reasoner\-ZeroOpen\-Reasoner\-Zero/Open\-Reasoner\-Zero@ f6d1ec7 \(orz/ppo/tools/math\_utils\.py\); RL training code, which applies it only to a\\boxed\{\}inside<answer\>tags; we apply it to the whole responsethe last\\boxed\{\}the whole response*multiple choice only*simple\-evals, multiple choiceopenai/simple\-evals@ 5e623c2 \(common\.py,gpqa\_eval\.py\); evaluation harnessthe first “Answer: X”, with X one of A–Dnothing*free\-form only*simple\-evals, mathopenai/simple\-evals@ 5e623c2 \(common\.py,math\_eval\.py\); evaluation harnessthe text after the first “Answer:”, to the line endnothinglm\-evaluation\-harness GSM8K, strict\-matchEleutherAI/lm\-evaluation\-harness@ b733598 \(lm\_eval/tasks/gsm8k/gsm8k\.yaml,lm\_eval/filters/extraction\.py\); evaluation harnessthe first “\#\#\#\# number”the string “\[invalid\]”lm\-evaluation\-harness GSM8K, flexible\-extractas abovethe last run of two or more digits, dollar signs, points or commas, or else of digits“\[invalid\]”, or a run of punctuation with no digitverl GSM8K, strictverl’s reward function, as shipped inhkust\-nlp/simpleRL\-reason@ 2204dd1 \(verl/utils/reward\_score/gsm8k\.py\); RL training rewardthe first “\#\#\#\# number”nothingverl GSM8K, flexibleas abovethe last run of digits, points or commas that is not a bare pointnothing, or “\.” when every run is a bare point#### When nothing matches\.

Given a response that holds no answer they can read, the nine extractors do five different things\. lm\-evaluation\-harness’s two GSM8K rules return the string “\[invalid\]” on most such inputs; Qwen2\.5\-Math’s returns the empty string except on items keyed A–E, where it returns any letter in the response or else the response’s text; simple\-evals’ two extractors, MATH’s and verl’s strict rule return nothing; the two flexible rules can return a run of punctuation with no digit, because their number patterns admit one; and Open\-Reasoner\-Zero’s`\\boxed\{\}`reader, applied to a whole response as we apply it, returns that response\. For lm\-evaluation\-harness’s flexible rule, responses with no digit return “\[invalid\]” 99\.02% of the time and a punctuation run 0\.98% of the time, over 76,428 responses from 11 checkpoints\. In this study’s scoring, a returned nothing is an abstention and is scored wrong; a returned string, empty or not, is an answer, and it is scored wrong unless it matches the key\.

#### lm\-evaluation\-harness’s default MMLU\.

The task thatlm\_eval \-\-tasks mmluruns has no extraction step: it scores the log\-likelihood of each of the four options and takes the most likely, so it never reads generated text and has no format gain to produce\. Its generative variant,mmlu\_generative, has to be selected by name\. The harness’s default MMLU\-Pro and BBH tasks, by contrast, generate text and extract the answer from it\. This holds at the pinned commit and on the harness’s main branch as read in August 2026\.

#### Copying\.

Each extractor was transcribed verbatim from the files of its repository fetched at the pinned commit\. The record of these copies holds the nine extractors with their helper functions and task configurations; its 11 entries that hold source text match the same files fetched again at the same commits\. The executable versions used for scoring were written from these copies and tested against them, and every departure from the source is listed with its measured effect in Table[17](https://arxiv.org/html/2610.20639#A5.T17)\. The source of an extractor is its function together with the step of its evaluation script that reads the answer\. For Qwen2\.5\-Math that step is the letter step of its evaluation loop, and its executable version includes it, so its extractor is run as its repository runs it; the first row of the table gives the effect of leaving the step out\. Open\-Reasoner\-Zero’s training code applies its function only to a`\\boxed\{\}`inside<answer\>tags, and we apply it to the whole response \(Table[16](https://arxiv.org/html/2610.20639#A5.T16)\)\.

Table 17:Every departure of the executable extractors from their source, with its measured effect\.
#### First\-character scoring\.

Multiple choice is scored on the first character of the extracted answer, compared with the key, for every extractor and both models\. This matters for Open\-Reasoner\-Zero’s extractor, which returns the whole response when it finds no`\\boxed\{\}`: a response that opens with “Answer”, for instance, is then scored as option A\. Over the 33,000 out\-of\-domain multiple\-choice outputs of the 11 checkpoints whose pair receives the same input \(every grid checkpoint except Open\-Reasoner\-Zero\-1\.5B\), this extractor returns a whole response longer than 40 characters on 78\.76% of the base models’ outputs and 49\.67% of the RL models’, and credits a correct answer this way on 6\.99% and 4\.61%\. The extra credit falls more on the base models, which lowers the reported gain and therefore the format share; its size on the main result is not computed\. Under exact matching, the MATH and Open\-Reasoner\-Zero extractors give the same correctness on 100\.00% of items, and under first\-character scoring on 94\.31%\. The two therefore run one algorithm, and the gap between their shares in Table[13](https://arxiv.org/html/2610.20639#A4.T13)comes from the scoring rule, not from extraction\.

#### The extractors on the same outputs\.

Table[18](https://arxiv.org/html/2610.20639#A5.T18)gives the values of Figure[5](https://arxiv.org/html/2610.20639#S4.F5)\. Without simple\-evals’ two extractors, which look for an “Answer:” line that neither of our prompts asks for and that simple\-evals’ own prompt does, the median spread between harness extractors is 9\.6 points\. The table’s indented rows give the values with Qwen2\.5\-Math’s extraction function alone\.

Table 18:Different extractors report different gains on the same outputs\.Top: on the outputs of Figure[1](https://arxiv.org/html/2610.20639#S1.F1)\(Qwen2\.5\-7B and SimpleRL\-Zoo, ARC\-Challenge, boxed\-instruction prompt, 500 items\), the gain each multiple\-choice harness extractor reports and the fixed extractor’s reading\. Middle: over the grid’s 112 combinations of pair, benchmark and prompt, the spread between harness extractors \(largest minus smallest reported gain\), and the readable out\-of\-domain comparisons in which simple\-evals scores the RL model at exactly 0%\. Bottom: the 17 out\-of\-domain multiple\-choice results reported in 18 recent RLVR and reasoning\-model releases, by how they were produced \(Appendix[I](https://arxiv.org/html/2610.20639#A9)\)\.

## Appendix FThe other groups

This appendix gives the three groups of Table[19](https://arxiv.org/html/2610.20639#A6.T19)other than out\-of\-domain multiple choice: how each moves under the rules for which comparisons enter the ratio \(Table[20](https://arxiv.org/html/2610.20639#A6.T20)\), why the domain contrast within multiple choice is unresolved, and what the in\-domain free\-form gains consist of \(Table[21](https://arxiv.org/html/2610.20639#A6.T21)\)\.

Table 19:Reported and fixed\-extractor gains by domain and answer form\.Mean gains per comparison, in accuracy points, over the comparisons that pass both criteria of Section[2\.3](https://arxiv.org/html/2610.20639#S2.SS3), with the fixed extractor’s extraction rate on the base and RL models’ outputs\. The format share is printed only where it is stable\. For each of the other three groups, the shares under the rule of Section[2\.3](https://arxiv.org/html/2610.20639#S2.SS3)and under the two alternatives of Appendix[D](https://arxiv.org/html/2610.20639#A4)lie more than 50 points apart \(Table[20](https://arxiv.org/html/2610.20639#A6.T20)\)\.Table 20:Across the three rules, the share of out\-of\-domain multiple choice moves by 0\.07, that of each other group by more than 0\.5\.Format share by group under each rule for which comparisons enter the ratio, with the number of comparisons in parentheses, and the spread across the three rules\. The first row is the main result\.significant reportedevery readableany non\-zerogroupgain \[95% CI\]comparisonreported gainspread*out of domain*multiple choice0\.8222 \(46\) \[0\.7735, 0\.8716\]0\.7926 \(100\)0\.7501 \(96\)0\.0721free\-form1\.3438 \(2\), not quoted6\.7778 \(32\)5\.6389 \(21\)5\.434*in domain*multiple choice0\.7150 \(10\) \[0\.5736, 0\.8867\]0\.1996 \(28\)0\.2308 \(26\)0\.5154free\-form−0\.1435\-0\.1435\(35\) \[−0\.208\-0\.208,−0\.086\-0\.086\]−0\.8701\-0\.8701\(80\)−0\.1688\-0\.1688\(50\)0\.7266#### Out of domain, free\-form\.

Free\-form answers out of domain rest on BBH alone\. Of its 32 comparisons that meet the extraction\-rate criterion, 2 meet the significant\-gain criterion, both from one combination \(Qwen2\.5\-Math\-7B with SimpleRL\-Zero, boxed\-instruction prompt\), whose reported gain of 6\.4 pointsω\\omegareads as−2\.2\-2\.2while reading 78\.0% of the base model’s outputs and 92\.8% of the RL model’s\. Two comparisons from one combination with a 14\.8\-point gap in extraction rate cannot carry a share, so whether the format gain reaches free\-form answers out of domain is untested\.

#### In domain, multiple choice\.

This group has 10 comparisons\. Its share moves from 0\.369 to 0\.829 when one pair is left out and from 0\.465 to 0\.771 when one benchmark is left out, a movement of up to 0\.346 that is wider than its own interval\.

#### The domain contrast is unresolved\.

Within multiple choice, out\-of\-domain minus in\-domain is\+0\.107\+0\.107\(95% CI−0\.057\-0\.057to 0\.271\) under the significant\-gain criterion\. Under the other two rules the in\-domain share falls to 0\.200 and 0\.231, the contrast becomes 0\.593 and 0\.519, and the two groups’ intervals no longer overlap: 0\.7926 \[0\.6712, 0\.9249\] against 0\.1996 \[−0\.0469\-0\.0469, 0\.4917\] for every readable comparison, 0\.7501 \[0\.6345, 0\.8745\] against 0\.2308 \[0\.0021, 0\.5033\] for any non\-zero reported gain\. Which way the contrast reads is thus set by the rule, a consequence of 10 in\-domain comparisons against 46, and the data cannot say whether the format gain is specific to out\-of\-domain evaluation\.

#### In domain, free\-form\.

The share of−0\.144\-0\.144is a mixture of extractors \(Table[21](https://arxiv.org/html/2610.20639#A6.T21)\)\. The two boxed extractors read almost exactly whatω\\omegareads \(a share of\+0\.013\+0\.013\)\. Three extractors report smaller gains thanω\\omegadoes\. The table gives the shortfall over each extractor’s own reported gain; overω\\omega’s gain it is 20\.6% \(Qwen2\.5\-Math’s\), 23\.5% \(lm\-evaluation\-harness’s flexible GSM8K rule\) and 26\.8% \(verl’s flexible rule\)\. The two flexible rules take the last number anywhere in the response; Qwen2\.5\-Math’s reads`\\boxed\{\}`and “the answer is” first and falls back on the last number only when neither occurs\. The other three free\-form extractors, simple\-evals’ and the two strict GSM8K rules, have no comparison that meets both criteria\. The 35 comparisons come from the three Qwen pairs, on MATH\-500 and GSM8K\.

Table 21:The in\-domain free\-form share is a mixture of extractors\.The 35 in\-domain free\-form comparisons that meet both criteria, by harness extractor; gains summed over comparisons, in accuracy points\.The grid also holds contrasts larger than the main result whose intervals exclude zero: multiple choice minus free\-form in domain, 0\.8585 \[0\.6906, 1\.0264\], and out\-of\-domain minus in\-domain for free\-form answers, 1\.4873 \[0\.383, 2\.5916\]\. Each rests on a group that moves by an order of magnitude more than the main result under the same rules, so neither is quoted as a finding\. Multiple choice minus free\-form out of domain is−0\.5216\-0\.5216\[−1\.6519\-1\.6519, 0\.6087\]\.

## Appendix GBeyond the grid: larger pairs and the Llama pilot

This appendix reports three pairs outside the grid: Mistral\-Small\-24B with SimpleRL\-Zoo and Qwen2\.5\-32B with DAPO, generated separately, each model on two GPUs, and a 100\-item pilot on Llama\-3\.1\-8B with SimpleRL\-Zoo\.

#### The larger pairs\.

Both were generated with the grid’s items, prompts, decoding \(greedy,float16, at most 2048 new tokens\), harness extractors andω\\omega; only the number of GPUs per model differs\. Both were set apart from the pooled share before their generation, whatever they show, and a declaration fixed then how each result would be read\. A pair with fewer than 5 comparisons meeting both criteria has no gain to decompose\. With 5 or more, a share of at least 0\.60 counts as the result holding and a share below 0\.40 as it failing: for Mistral\-Small\-24B across family and scale together, for Qwen2\.5\-32B across scale within the Qwen family\. Between 0\.40 and 0\.60, the declaration reads Mistral\-Small\-24B as indeterminate and declares no reading for Qwen2\.5\-32B\. Its hash was written into the generation job before the job ran\. Table[22](https://arxiv.org/html/2610.20639#A7.T22)gives the outcome\.

Table 22:Neither larger pair reproduces the main result\.Out\-of\-domain multiple choice\. The last row scores Qwen2\.5\-Math’s extractor with its extraction function alone \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\)\.On Qwen2\.5\-32B almost none of the reported gain is format gain\. The pair’s 9 comparisons that meet both criteria have a summed format gain of\+2\.0\+2\.0points over a summed reported gain of 94\.0, a share of 0\.021 \(Table[22](https://arxiv.org/html/2610.20639#A7.T22)\)\. Leaving out any one harness extractor, or taking either prompt alone, gives a share between−0\.04\-0\.04and 0\.10 \(Table[23](https://arxiv.org/html/2610.20639#A7.T23)\), so the main result is not reproduced on this pair under any single extractor’s removal\. Under the declaration, a share below 0\.40 on this pair makes the main result a property of models of 7B and below \(Section[6](https://arxiv.org/html/2610.20639#S6)\)\.

The pair also shows little of the difference in where the two models write their answers on which a format gain rests \(Section[4\.2](https://arxiv.org/html/2610.20639#S4.SS2)\)\. Under the boxed\-instruction prompt the 32B base model writes`\\boxed\{\}`in 99\.2% of its out\-of\-domain multiple\-choice outputs, against 42\.2% for the base models of the four pairs of the main result, and its two comparisons there report\+6\.4\+6\.4points whereω\\omegareads\+5\.8\+5\.8\. Under the plain prompt the DAPO model writes`\\boxed\{\}`in 14\.8% of its outputs \(the four pairs’ RL models 54\.8%, its base model 1\.1%\)\. MATH’s extractor then reads 0% to 3% of the base model’s outputs and 12% to 19% of the RL model’s, so the gains it reports there, 9\.8 to 12\.2 points, come from the RL model’s few boxed answers\.ω\\omegareads most of both models’ outputs and finds gains of about the same size, 10\.4 to 12\.8 points\. The two gains agree in size without resting on the same answers\. The pair differs from the grid’s in scale, in RL recipe and in lab, and one pair cannot separate the three\.

With Qwen2\.5\-Math’s extraction function alone, the pair has 10 comparisons that meet both criteria and a share of 0\.436, between the two cut\-offs, where nothing was declared \(Table[22](https://arxiv.org/html/2610.20639#A7.T22)\)\. Two comparisons then carry the share: the two read by Qwen2\.5\-Math’s extractor have a summed format gain of\+82\.4\+82\.4points, more than the pair’s total of\+79\.2\+79\.2over a summed reported gain of 181\.8\. Without that extractor the share is−0\.042\-0\.042; without MATH’s it is 0\.555, and without Open\-Reasoner\-Zero’s 0\.568\. The letter step changes both of these comparisons \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\): with it, one no longer meets the significant\-gain criterion, and the other’s format gain is\+5\.2\+5\.2points\. Either way, the two extractors that read only the box show no format gain on this pair \(Table[23](https://arxiv.org/html/2610.20639#A7.T23)\), so the main result does not carry over to it\.

On Mistral\-Small\-24B,ω\\omegameets the extraction\-rate criterion in 8 of the 24 comparisons, and 3 of those meet the significant\-gain criterion, below the declared floor of 5, so the pair’s share of 0\.560 is reported and not used\. Two of the three are read by simple\-evals’ extractor \(summed format gain\+28\.2\+28\.2points\) and one by Open\-Reasoner\-Zero’s \(−2\.0\-2\.0\), all under the plain prompt\. The declared reading for this outcome is that the pair has no gain to decompose on our outputs, like DeepSeek\-Math\-7B in the grid; it is not evidence about the share\. The SimpleRL\-Zoo paper reports MMLU gains for all three pairs under its own evaluation, which we did not reproduce \(Table[28](https://arxiv.org/html/2610.20639#A9.T28)\)\.

Table 23:On the 32B pair, almost none of the reported gain is format gain, whichever extractor is left out\.Qwen2\.5\-32B with DAPO, the 9 comparisons that meet both criteria\.harness extractorits comparisonsits summed format gainformat share without itMATH \(\\boxed\{\}only\)4\+0\.6\+0\.60\.0260Open\-Reasoner\-Zero4−3\.8\-3\.80\.0997Qwen2\.5\-Math1\+5\.2\+5\.2−0\.0421\-0\.0421promptcomparisonsshare of the weight \(%\)format shareplain7860\.0099boxed\-instruction2140\.0937all90\.0213
#### The Llama pilot\.

The Llama\-3\.1\-8B pair of the SimpleRL\-Zoo release met the admission rule, but its base model’s weights are access\-gated; they were taken from a public mirror whose four weight files have the same SHA\-256 hashes as the gated repository’s\. The pilot generated 50 items each of ARC\-Challenge, MMLU and MMLU\-Pro under the boxed\-instruction prompt; Table[24](https://arxiv.org/html/2610.20639#A7.T24)covers the first two\. RL moved this pair’s answers away from the positionsω\\omegareads, the opposite of every Qwen pair:ω\\omegareads 90% of the base model’s ARC\-Challenge outputs and 68% of the RL model’s, and 70% and 38% on MMLU, so both benchmarks fail the extraction\-rate criterion, with gaps of 22 and 32 points\. MMLU\-Pro is left out becauseω\\omegareads only 6% of its base model’s outputs \(the pilot ranω\\omegawith the four letters A to D on this ten\-option benchmark\); with it, over 150 items,ω\\omega’s reading under that setting is−13\.3\-13\.3points \(95% CI−21\.3\-21\.3to−5\.3\-5\.3\)\. Neither model writes a readable`\\boxed\{\}`, even when asked\.

ω\\omega, which scores a missing answer as wrong, puts the RL model 19 points below its base model\. Conditional onω\\omegareading an item, the RL model is not worse: 75\.5% against 73\.8%,\+1\.7\+1\.7points, over 53 and 80 items\. The pilot therefore shows a collapse in extraction rate, not a loss of capability\. On the same outputs the MATH extractor reports exactly 0\.0, because it reads neither model, and Open\-Reasoner\-Zero’s reports\+5\.0\+5\.0, through its whole\-response fall\-back and first\-character scoring \(Appendix[E](https://arxiv.org/html/2610.20639#A5)\)\. A harness extractor can thus report no change whereω\\omegareports a large one, and can produce a rise\.

Table 24:One pair, five readings\.Gain of Llama\-3\.1\-8B’s SimpleRL\-Zoo successor over its base model on 100 items \(50 each of ARC\-Challenge and MMLU, boxed\-instruction prompt\), in accuracy points\.

## Appendix HWhat the share tracks

This appendix gives what the share tracks across base models and RL recipes: the base model’s boxing rate, its fit across pairs, the same base model under two RL recipes, the change in boxing rate between the two models, and the harness extractors’ extraction rates on the base models\.

#### The base model’s boxing rate\.

A combination’s boxing rate is the percentage of the base model’s outputs on it that contain`\\boxed\{\}`, and its share is pooled over its comparisons\. Across the 20 combinations of the main result, the share falls as the boxing rate rises:r=−0\.434r=\-0\.434\(95% CI−0\.687\-0\.687to−0\.082\-0\.082\), a slope of−0\.0042\-0\.0042per percentage point\. The rate runs from 1\.0% \(Qwen2\.5\-7B, ARC\-Challenge, plain prompt\) to 73\.2% \(Qwen2\.5\-Math\-7B, MMLU\-Pro, boxed\-instruction prompt\), and nine combinations exceed 38%\. The 20 rates take only 14 distinct values, because the SimpleRL\-Zoo and SimpleRL\-Zero pairs are compared with the same base\-model outputs, so that 12 combinations share their rate two by two; the bootstrap treats the 20 as independent, so the interval is narrower than the design supports\. Over the 46 comparisons, where each rate repeats once per harness extractor, the correlation is weaker:r=−0\.275r=\-0\.275\. The correlation is across combinations, so it does not rank base models\.

#### Across pairs\.

Averaged over each pair’s comparisons, the base models box 6\.0% of their outputs for Mistral\-7B, 29\.2% for Qwen2\.5\-7B, and 35\.9% \(SimpleRL\-Zero\) and 38\.3% \(SimpleRL\-Zoo\) for Qwen2\.5\-Math\-7B\. Qwen2\.5\-Math\-7B’s two values differ only because its two pairs do not have the same comparisons\. The pairs’ shares are 1\.091, 0\.867, 0\.762 \(SimpleRL\-Zero\) and 0\.798 \(SimpleRL\-Zoo\)\. Over these four pointsr=−0\.986r=\-0\.986, but one of them is Mistral\-7B, whose share comes from 2 comparisons and exceeds one; without itr=−0\.829r=\-0\.829, over three points from two base models\. The fit across pairs is therefore not evidence; only the weak correlation across combinations above is\.

#### The same base model, two RL recipes\.

SimpleRL\-Zoo and SimpleRL\-Zero both start from Qwen2\.5\-Math\-7B and have 15 comparisons each in the main result, 13 of them matched: the same combination read by the same harness extractor\. On the 13, with base\-model outputs, benchmark and extractor all fixed, the shares are 0\.8146 for SimpleRL\-Zoo and 0\.7904 for SimpleRL\-Zero, a gap of 0\.024 that can come only from the RL model\. On each pair’s own 15 comparisons the shares are 0\.7975 and 0\.7624, a gap of 0\.035, which differs from the matched gap by two unmatched comparisons on each side\. For SimpleRL\-Zoo these are MMLU under the plain prompt, read by Open\-Reasoner\-Zero’s extractor, and MMLU\-Pro under the boxed\-instruction prompt, read by Qwen2\.5\-Math’s; for SimpleRL\-Zero, MMLU and MMLU\-Pro under the plain prompt, both read by Qwen2\.5\-Math’s\. No interval is computed for the gap, so it is not shown to exceed sampling noise, and as evidence of a recipe effect it is exploratory\.

#### The change in boxing rate\.

The share does not track how much more often the RL model boxes than its base model \(Table[25](https://arxiv.org/html/2610.20639#A8.T25)\)\. Within out\-of\-domain multiple choice, over the 20 combinations with each combination’s share pooled as in the rest of this paper, the correlation between that change and the share isr=−0\.03r=\-0\.03\. The other rows of Table[25](https://arxiv.org/html/2610.20639#A8.T25)average each combination’s comparison shares instead, which gives−0\.109\-0\.109\(95% CI−0\.546\-0\.546to 0\.544\)\. Controlling each for the other, with pooled shares, the base model’s rate keeps its association with the share \(partialr=−0\.455r=\-0\.455\) and the change does not \(−0\.154\-0\.154\)\. A large positive correlation appears only across groups: over the 93 comparisons of all four groups that meet both criteria,r=0\.567r=0\.567, and over their 35 combinations, 0\.402\. That is a contrast between groups\. Within out\-of\-domain multiple choice the correlation is near zero over combinations, and within the groups the sign is not uniform: it reverses in in\-domain free\-form only, and is positive in in\-domain multiple choice\. No dose–response on the change is detectable within a group at this grid’s size\.

Table 25:The change in boxing rate tracks the share across groups, not within out\-of\-domain multiple choice\.Correlation between the change in the rate of\\boxed\{\}from base to RL model and the format share\. Across comparisons, each combination’s change repeats once per harness extractor; across combinations, each combination’s share is the mean of its comparisons’ shares, except in the row marked pooled, which sums gains as in the rest of the paper\.
#### Harness extractors’ extraction rates on the base models\.

Table[26](https://arxiv.org/html/2610.20639#A8.T26)gives, for each base model and multiple\-choice harness extractor, the percentage of the base model’s out\-of\-domain multiple\-choice outputs from which the extractor returns an answer whose first character is an option letter\. The count also takes in an empty return, which three of the extractors can give \(Qwen2\.5\-Math’s when it finds nothing, MATH’s and Open\-Reasoner\-Zero’s on an empty`\\boxed\{\}`\) and which no key matches\. The rates in parentheses leave empty returns out, and 15 of the 20 are lower\. Under the MATH extractor the rate runs from 3\.9% \(Mistral\-7B\) to 32\.0% \(Qwen2\.5\-Math\-7B\), or from 0\.0% \(Qwen2\.5\-1\.5B\) to 30\.5% without empty returns\. Under simple\-evals’ extractor, whose rates are the same under both counts, the base models change order, with Mistral\-7B first at 33\.8% and Qwen2\.5\-1\.5B last at 1\.6%\. Across the 20 entries, the extractor accounts for 78\.5% of the variance, the base model for 9\.4%, and the residual for 12\.1%; without empty returns the three are 75\.2%, 14\.1% and 10\.7%\. Changing the extractor with the base model fixed moves the rate by a median of 72\.7 points, and changing the base model with the extractor fixed by a median of 30\.15 points\. The ratio of the two medians, 2\.41, has a permutationpp\-value below 0\.0001: none of 20,000 shuffles of the 20 entries, with the row and column structure destroyed \(seed 20260818\), reached it\. Without empty returns the medians are 66\.7 and 31\.35 points, the ratio 2\.13 and thepp\-value 0\.0003\. Under either count the rate is a property of the pairing of base model and extractor, and it varies more between extractors than between base models\.

The extraction\-rate criterion does not select base models with high rates\. Mistral\-7B, at 3\.9% under the MATH extractor \(3\.6% without empty returns\), has all 24 of its out\-of\-domain multiple\-choice comparisons admitted, while the two pairs on Qwen2\.5\-1\.5B, at 21\.6% \(0\.0%\), have none, becauseω\\omegareads only 6\.7% of that base model’s outputs: it runs to the token limit on 98\.7% of items\.

A harness extractor’s extraction rate on the base model needs neither a fixed extractor nor an RL model, and in the main result it goes with the share\. Over the seven combinations of base model and extractor that contribute comparisons to the main result, the share is lower where the rate is higher:r=−0\.858r=\-0\.858with pooled shares\. The seven are not independent, and the interval of that value,−0\.987\-0\.987to−0\.376\-0\.376, treats them as if they were: four are one rule, the last`\\boxed\{\}`, on two base models, and two are Qwen2\.5\-Math’s extractor on two base models\. Without those two,r=−0\.461r=\-0\.461over five, so the relation rests mostly on that one extractor, which reads the most of the base models’ outputs and reports the least format gain\. Leaving out any one combination, including the one whose share exceeds one, gives−0\.934\-0\.934to−0\.765\-0\.765\. Without empty returnsr=−0\.845r=\-0\.845\(−0\.988\-0\.988to−0\.302\-0\.302\)\. A mean of per\-comparison shares would give−0\.783\-0\.783\(−0\.987\-0\.987to−0\.321\-0\.321\)\. Seven points do not support a prediction beyond these combinations\.

Table 26:A base model’s extraction rate depends on the extractor more than on the model\.Percentage of each base model’s out\-of\-domain multiple\-choice outputs \(3,000 per base model, both prompts\) from which the harness extractor returns an answer whose first character is an option letter, with an empty return counted among them; in parentheses, the percentage without empty returns\. Open\-Reasoner\-Zero’s rate includes whole responses returned when it finds no\\boxed\{\}\(Appendix[E](https://arxiv.org/html/2610.20639#A5)\)\.

## Appendix IHow released RLVR and reasoning\-model checkpoints report out\-of\-domain multiple choice

This appendix gives the reading of 18 recent RLVR and reasoning\-model releases behind the count in Section[4\.4](https://arxiv.org/html/2610.20639#S4.SS4)and in Table[18](https://arxiv.org/html/2610.20639#A5.T18), and sets the accuracies published for pairs we evaluate beside our readings\. For each release, the paper and, where one exists, its released evaluation code were read in August 2026 for how each reported out\-of\-domain multiple\-choice number was produced; the releases in which we first found none were read again, in every version, in September 2026\. A release is one paper or repository, so SimpleRL\-Zoo and SimpleRL\-Zero, which share a repository, count as one\. A result is a set of such numbers that one release produces the same way; four releases produce different benchmarks, or different model stages, differently and so contribute two results each\. A result counts as generated when the paper or the code shows that the model writes text from which the answer is read \(a stated sampling temperature, or an average over sampled runs, counts as showing this\); as log\-likelihood when the options are scored by their likelihood; and as unstated when neither says\.

Thirteen releases report at least one such number, in 17 results \(Table[27](https://arxiv.org/html/2610.20639#A9.T27)\): 13 are generated, 1 is scored by log\-likelihood \(Tulu 3’s headline MMLU, while the same paper’s GPQA and MMLU\-Pro numbers are generated\), and 3 are unstated\. Five releases report none\. Where these releases report an out\-of\-domain multiple\-choice number at all, it was almost always produced by generating text and reading the answer out of it, so a format gain can arise in them as it does on our outputs\. The count says nothing about how large the format gain is in these releases, and the 18 are not a systematic sample: they were not chosen by a stated rule\.

Table 27:How 18 releases produced their out\-of\-domain multiple\-choice numbers\.Benchmarks in parentheses\.#### What is published for the pairs we evaluate\.

The SimpleRL\-Zoo paper reports MMLU, MMLU\-STEM and GPQA\-Diamond accuracy for its base models and their RL models, and reads the gains as generalization\([Zeng et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib24), Table 2\)\. It describes its evaluation as a script based on Qwen2\.5\-Math’s, a sampling temperature of 1\.0, up to 16K tokens and the prompt template of training\. Its repository ships the MMLU\-STEM items for that script and neither items nor a script for MMLU or GPQA\-Diamond, and the paper does not say which decoding this table used\. We class the result as generated\. The SimpleRL\-Zero checkpoints have no report of their own\. General\-Reasoner\([Ma et al\., 2025](https://arxiv.org/html/2610.20639#bib.bib43)\)reports MMLU\-Pro for one of our pairs and does not say how answers were read\.

Table[28](https://arxiv.org/html/2610.20639#A9.T28)sets these accuracies beside three readings of our own outputs for the same pair and benchmark\. The two sides differ in items, prompt, decoding and extractor, so the table does not split any published gain\. Published out\-of\-domain multiple\-choice gains exist for these checkpoints, including the pairs for which our outputs show no gain to split\. No single reading of our outputs reproduces SimpleRL\-Zoo’s accuracies: for Qwen2\.5\-Math\-7B, the release prints 38\.0% and 56\.6% on MMLU, where under the boxed\-instruction prompt our outputs give 53\.6% and 57\.4% by the fixed extractor and 24\.0% and 55\.8% by the boxed\-only one\. The one number from an independent evaluation, General\-Reasoner’s gain of 3\.8 points on MMLU\-Pro, is of the size that the fixed extractor reads on our outputs and far below the boxed\-only reading\. Splitting the published gains would take outputs generated under the release’s own prompt and decoding\.

Table 28:Published accuracies for pairs of this study, beside our readings of the same pair and benchmark\.Published: the base model, its SimpleRL\-Zoo model and the gain in points\. Ours: the gain on our 500 items under the boxed\-instruction and the plain prompt, as read by the fixed extractor, by the boxed\-only MATH extractor, and by Qwen2\.5\-Math’s extractor, whose MMLU branch is the letter rule of the script that SimpleRL\-Zoo’s evaluation is based on\.†The fixed extractor’s extraction rates do not meet the criterion of Section[2\.3](https://arxiv.org/html/2610.20639#S2.SS3)\.publishedour outputs: gain, boxed\-instruction / plainbase modelbaseRLgainfixed extractorboxed onlyQwen2\.5\-Math*MMLU; published in SimpleRL\-Zoo, Table 2*Qwen2\.5\-Math\-7B38\.056\.6\+18\.6\+18\.6\+3\.8\+3\.8/\+3\.0\+3\.0\+31\.8\+31\.8/\+22\.0\+22\.0−12\.2\-12\.2/−12\.0\-12\.0Qwen2\.5\-7B38\.647\.0\+8\.4\+8\.4\+5\.8\+5\.8/\+3\.0\+3\.0\+28\.4\+28\.4/\+44\.8\+44\.8−0\.8\-0\.8/\+2\.2\+2\.2Mistral\-7B\-v0\.128\.034\.6\+6\.6\+6\.6\+1\.0\+1\.0/−12\.2\-12\.2−2\.8\-2\.8/0\.00\.0−4\.6\-4\.6/−8\.2\-8\.2DeepSeek\-Math\-7B22\.745\.5\+22\.8\+22\.8\+4\.2†\+4\.2^\{\\dagger\}/−2\.0\-2\.0\+2\.6\+2\.6/0\.00\.0−3\.6\-3\.6/\+0\.2\+0\.2Qwen2\.5\-1\.5B35\.445\.2\+9\.8\+9\.8\+0\.8†\+0\.8^\{\\dagger\}/−0\.2†\-0\.2^\{\\dagger\}\+1\.6\+1\.6/0\.00\.0\+3\.4\+3\.4/\+0\.8\+0\.8Mistral\-Small\-24B31\.778\.8\+47\.1\+47\.1\+25\.0†\+25\.0^\{\\dagger\}/\+7\.4\+7\.4\+40\.4\+40\.4/0\.00\.0\+4\.6\+4\.6/\+0\.8\+0\.8*MMLU\-Pro; published in General\-Reasoner, Table 2*Qwen2\.5\-7B47\.751\.5\+3\.8\+3\.8\+2\.4\+2\.4/−1\.6\-1\.6\+15\.6\+15\.6/\+25\.2\+25\.2\+1\.0\+1\.0/\+7\.6\+7\.6

## Appendix JCost and reproducibility

This appendix gives the cost of generation, the settings that fix every output and interval, the prompts, and what the release holds\.

#### Cost\.

The 192 generation runs of the grid took 29\.7 GPU\-hours of generation and 1\.5 hours of model loading\. Four of the 12 checkpoints took 89% of the generation time: Mistral\-7B, DeepSeek\-Math\-7B and their RL models, which ship no chat template and run to the 2048\-token limit on 61% to 86% of items\. Scoring needs no GPU: every step after generation runs on a CPU\.

#### Settings\.

Decoding is greedy, so no sampling seed enters generation\. All models run infloat16, because not every GPU used supportsbfloat16; both models of every pair use the same precision\. Each benchmark contributes 500 items, except AQuA\-RAT and SAT\-Math, whose 254 and 220 test items are used in full\. Items were frozen and hashed before any generation, and every generation file records the hash of the item file it was built from; all records match\. Every interval is seeded\. Each comparison’s bootstrap of 10,000 draws is seeded from a fixed seed and a hash of the comparison’s name, so that each interval is reproducible on its own; the pooled and paired bootstraps over each benchmark’s items use seed 20260816, the permutation test of Appendix[H](https://arxiv.org/html/2610.20639#A8)seed 20260818, and the readers’ sample and bootstrap seed 20260930\. The interval of the pilot’s format share in Appendix[B](https://arxiv.org/html/2610.20639#A2)is an exception: its seed includes a hash that Python salts per process, so it is the interval of the one recorded run, and a new run reproduces the share and moves the interval’s ends slightly\.

#### Prompts\.

Each prompt is wrapped in the checkpoint’s own chat template where the checkpoint ships one\. The boxed\-instruction prompt for multiple choice reads as follows, with its first line broken for print:

```
Answer the following multiple-choice question. Reason step by step,
then give your final answer as a single letter inside \boxed{}.

Question: <question>
A) <first option>
B) <second option>
...
```

The plain prompt for multiple choice omits the second sentence\. For free\-form items, both prompts begin “Solve the following problem\.” in place of the first sentence, write “Problem:” in place of “Question:”, list no options, and, in the boxed\-instruction prompt, ask for the final answer “inside`\\boxed\{\}`” without “as a single letter”\.

#### Release\.

The ancillary files of this arXiv version hold the analysis code, the 8 frozen item files with their hashes, the result files, the records behind the counts of Appendices[A](https://arxiv.org/html/2610.20639#A1),[B](https://arxiv.org/html/2610.20639#A2),[E](https://arxiv.org/html/2610.20639#A5)and[I](https://arxiv.org/html/2610.20639#A9), the independent readers’ material, the written declarations of the readers’ check and of how the two larger pairs would be read, the Llama pilot’s 8 generation files \(two are second copies of its ARC\-Challenge files\) with its item files, and the SHA\-256 hash of each of the 264 generation files \(192 from the grid, 64 from the two larger pairs and 8 from the Llama pilot\)\. The other 256 generation files, which together exceed arXiv’s size limit, are available from the author; each can be checked against its hash\. With them, any extractor, including one written after this paper, can be scored against the same outputs without a GPU\. The result files behind the paper’s numbers that the letter step can change are held twice: inresults/with\-letter\-step/as this paper reports them, written byharness/with\_letter\_step\.py, and inresults/with Qwen2\.5\-Math’s extraction function alone\. The others are held once, inresults/\.