SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning

# SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning
Source: [https://arxiv.org/html/2608.14277](https://arxiv.org/html/2608.14277)
Haonan He\*Haodi Lei\*Yun Luo†,‡\\ddaggerHaoran ZhangShunkai ZhangYizhuo LiShengji TangZhilin WangRunzhe ZhanLei BaiGanqu CuiFangchen Yu‡\\ddaggerYafu Li‡\\ddaggerPeng Ye‡\\ddaggerNing Ding‡\\ddaggerYu Cheng‡\\ddaggerSU\-01 Team, Shanghai Artificial Intelligence Laboratory\*Equal Contribution,†Project Lead,‡\\ddaggerCorresponding Authorluoyun@pjlab\.org\.cn

###### Abstract

On\-policy distillation \(OPD\) offers a promising way to transfer reasoning capabilities from stronger teacher models, but applying it to long\-context reasoning teachers and short\-context students introduces practical challenges, including tokenizer mismatch, teacher\-student distribution mismatch, response length explosion, and training instability\. In this work, we study this setting by transferring proof\-reasoning capabilities from the long\-context reasoning model SU\-01 to short\-context student models\. To handle tokenizer differences, we perform OPD in a shared text space and align only tokens that occupy identical text spans under the student and teacher tokenizers\. To mitigate the problem of excessive generation length and frequent truncation, we introduce a student reference KL loss and mask the advantages of special termination tokens such as</think\>\{\\scriptstyle<\}\\,\\mathrm\{/think\}\{\\scriptstyle\>\}and<\|im​\_​end\|\>\{\\scriptstyle<\}\\,\\mathrm\{\|im\\\_end\|\}\\,\{\\scriptstyle\>\}\. This strategy constrains the student from drifting excessively from its initial policy, thereby mitigating the teacher\-student distribution mismatch problem and fostering steady length growth\. Experiments on both same\-family and different\-family student models, including Qwen3, Qwen3\.5, Intern\-S2, GLM\-4\.7, Gemma\-4, show consistent gains in mathematical reasoning, especially natural\-language math proving\. Notably, Intern\-S2\-Preview improves by 21\.2 points on ProofBench, reaching 55\.2 and surpassing Gemini\-2\.5\-Pro\. It also improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain\.

[![[Uncaptioned image]](https://arxiv.org/html/2608.14277v1/icon/worlwide.png)Project Page](https://hhnqqq.github.io/SimpleOPD-project-page)[Code](https://github.com/hhnqqq/SimpleOPD)[Models](https://huggingface.co/collections/bingyang-lei/simpleopd)August, 2026

11footnotetext:Work done during an internship at Shanghai Artificial Intelligence Laboratory\.Figure 1:Evaluation performance on ProofBench using Gemini\-2\.5\-Pro as the judge, under the same evaluation setting as SU\-01\. With SimpleOPD, Intern\-S2\-OPD achieves performance comparable to DeepSeek\-V3\.2\-Speciale, demonstrating the effectiveness of our OPD strategy\.###### Contents

1. [1Introduction](https://arxiv.org/html/2608.14277#S1)
2. [2Method](https://arxiv.org/html/2608.14277#S2)1. [2\.1Cross\-tokenizer Alignment](https://arxiv.org/html/2608.14277#S2.SS1) 2. [2\.2On\-policy distillation objective](https://arxiv.org/html/2608.14277#S2.SS2)
3. [3Experiment](https://arxiv.org/html/2608.14277#S3)1. [3\.1Training Data](https://arxiv.org/html/2608.14277#S3.SS1) 2. [3\.2Implementation Details](https://arxiv.org/html/2608.14277#S3.SS2)
4. [4Experimental Results](https://arxiv.org/html/2608.14277#S4)1. [4\.1Training Instability in On\-Policy Distillation](https://arxiv.org/html/2608.14277#S4.SS1) 2. [4\.2Results on Same\-family Models](https://arxiv.org/html/2608.14277#S4.SS2) 3. [4\.3Results on Cross\-family Models](https://arxiv.org/html/2608.14277#S4.SS3)
5. [5Analysis](https://arxiv.org/html/2608.14277#S5)1. [5\.1Comparison with OPD Baselines](https://arxiv.org/html/2608.14277#S5.SS1) 2. [5\.2Lexical Overlap](https://arxiv.org/html/2608.14277#S5.SS2) 3. [5\.3Out\-of\-domain Generalization](https://arxiv.org/html/2608.14277#S5.SS3) 4. [5\.4Effect of Training Data](https://arxiv.org/html/2608.14277#S5.SS4) 5. [5\.5Effect of OPD Length](https://arxiv.org/html/2608.14277#S5.SS5) 6. [5\.6Distillation from DeepSeek\-V4\-Flash](https://arxiv.org/html/2608.14277#S5.SS6) 7. [5\.7Effect of Student\-Reference KL Loss](https://arxiv.org/html/2608.14277#S5.SS7)
6. [6Related Work](https://arxiv.org/html/2608.14277#S6)1. [6\.1Olympiad\-Level Long\-Context Reasoning](https://arxiv.org/html/2608.14277#S6.SS1) 2. [6\.2On\-policy Distillation](https://arxiv.org/html/2608.14277#S6.SS2)
7. [7Conclusion](https://arxiv.org/html/2608.14277#S7)
8. [References](https://arxiv.org/html/2608.14277#bib)
9. [ACross\-Tokenizer Scoring and Alignment](https://arxiv.org/html/2608.14277#A1)
10. [BModel Solutions for ProofBench](https://arxiv.org/html/2608.14277#A2)1. [B\.1ProofBench\-Basic\-001](https://arxiv.org/html/2608.14277#A2.SS1) 2. [B\.2ProofBench\-Advanced\-028](https://arxiv.org/html/2608.14277#A2.SS2)
11. [CCase Study](https://arxiv.org/html/2608.14277#A3)1. [C\.1Case A: correct answer, then 972 exact self\-check repeats \(AIME25\)](https://arxiv.org/html/2608.14277#A3.SS1) 2. [C\.2Case B: reasoning collapses into a one\-token loop \(AnswerBench\)](https://arxiv.org/html/2608.14277#A3.SS2)

## 1Introduction

On\-policy distillation \(OPD\)\([11](https://arxiv.org/html/2608.14277#bib.bib1);[1](https://arxiv.org/html/2608.14277#bib.bib3)\)presents a compelling paradigm in the post\-training landscape of LLMs for tasks such as strong\-to\-weak distillation\([18](https://arxiv.org/html/2608.14277#bib.bib35)\), multi\-teacher distillation\([22](https://arxiv.org/html/2608.14277#bib.bib36)\), and self\-distillation\([12](https://arxiv.org/html/2608.14277#bib.bib37)\)\. Unlike off\-policy approaches that distill knowledge from fixed teacher\-generated data, OPD actively queries the teacher to evaluate tokens in student\-generated trajectories, thereby providing dense token\-level supervision grounded in the student’s own policy\([11](https://arxiv.org/html/2608.14277#bib.bib1);[1](https://arxiv.org/html/2608.14277#bib.bib3)\), enabling efficient distillation of complex reasoning behaviors, as it incurs less forgetting and achieves better generalization than off\-policy settings\. However, prior work has primarily studied teacher–student pairs that share the same model family and vocabulary\. Distilling reasoning capabilities from long\-context teachers into short\-context students, especially across different model families, remains largely underexplored\([30](https://arxiv.org/html/2608.14277#bib.bib24)\)\.

Applying OPD to distilling long\-context reasoning teachers to short\-context reasoning students presents several challenges\. First, teacher and student models may use different tokenizers, making direct alignment between their token\-level distributions infeasible\([36](https://arxiv.org/html/2608.14277#bib.bib12);[5](https://arxiv.org/html/2608.14277#bib.bib13);[29](https://arxiv.org/html/2608.14277#bib.bib14)\)\. Second, differences in model capacity and context length introduce a substantial teacher–student distribution mismatch\([17](https://arxiv.org/html/2608.14277#bib.bib28)\)\. In particular, long\-context reasoning teachers often favor responses that exceed the student’s context budget\. Through experiments, we observe that direct OPD can cause rapid response\-length growth, frequent truncation, and incomplete reasoning trajectories, ultimately destabilizing training\. This issue is especially pronounced for termination tokens such as</think\>\{\\scriptstyle<\}\\,\\mathrm\{/think\}\{\\scriptstyle\>\}and<\|im​\_​end\|\>\{\\scriptstyle<\}\\,\\mathrm\{\|im\\\_end\|\}\\,\{\\scriptstyle\>\}, as teacher supervision may repeatedly discourage the student from terminating its thinking process or final response\.

In this work, we focus on the transfer of reasoning capabilities from the long\-context reasoning model SU\-01\([16](https://arxiv.org/html/2608.14277#bib.bib11)\)to short\-context student models\. As shown in Figure[2](https://arxiv.org/html/2608.14277#S1.F2), to enable distillation across different model families, we perform OPD in a shared text space and align teacher and student tokens only when they occupy identical text spans\. This provides reliable token\-level supervision without requiring an artificial correspondence between incompatible tokenizations\. To mitigate excessive response\-length growth and stabilize optimization, we introduce a reference KL\-divergence loss between the student and its initial policy\. We also mask the advantages of special termination tokens, preventing teacher supervision from directly suppressing termination\. Together, these techniques constrain excessive policy drift, alleviate teacher–student distribution mismatch, and promote steady response\-length growth within the student’s context budget\.

We evaluate SimpleOPD across both same\-family and cross\-family distillation settings, using student models from Qwen3, Qwen3\.5, Intern\-S2\-Preview, GLM\-4\.7, and Gemma\-4\([32](https://arxiv.org/html/2608.14277#bib.bib25);[27](https://arxiv.org/html/2608.14277#bib.bib23);[13](https://arxiv.org/html/2608.14277#bib.bib26);[35](https://arxiv.org/html/2608.14277#bib.bib27);[9](https://arxiv.org/html/2608.14277#bib.bib2)\)\. Across model families, SimpleOPD consistently improves mathematical reasoning performance, with particularly strong gains on natural\-language mathematical proof tasks\. Most notably, Intern\-S2\-Preview improves by 21\.2 points on ProofBench\([20](https://arxiv.org/html/2608.14277#bib.bib29)\), from 34\.0 to 55\.2, surpassing Gemini\-2\.5\-Pro and approaching the performance of substantially stronger reasoning models \(Figure[1](https://arxiv.org/html/2608.14277#S0.F1)\)\. The distilled students also achieve improvements on science\-oriented benchmarks, including HLE and HiPhO\([26](https://arxiv.org/html/2608.14277#bib.bib30);[34](https://arxiv.org/html/2608.14277#bib.bib31)\), despite being trained exclusively on mathematical reasoning data\. These results demonstrate that stable cross\-tokenizer OPD can transfer reasoning capabilities from long\-context teachers, and that the acquired capabilities generalize beyond the training domain\.

In summary, our observations and contributions are:

![Refer to caption](https://arxiv.org/html/2608.14277v1/simpleopd_intro.png)Figure 2:Overview of SimpleOPD\. The student generates responses under its own tokenizer, while the teacher evaluates the same response under its native tokenizer\. We align tokens with identical text spans, use teacher supervision for matched tokens, and stabilize training with KL regularization and termination\-token masking\.- •Cross\-tokenizer OPD can improve student models without full token\-level alignment by using student\-generated text as a shared space and aligning tokens with identical text spans\.
- •Direct OPD transfers long\-context reasoning capabilities, but it often causes excessive length growth, frequent truncation, and training instability\.
- •Termination\-token advantage masking and student reference KL loss stabilize training and reduce truncation\. Larger teacher–student gaps generally require stronger KL regularization\.
- •SimpleOPD shows consistent performance gains across model families without supervised fine\-tuning on teacher trajectories, while moderately increasing the distillation length can further improve long\-form reasoning\.

## 2Method

Let𝐱\\mathbf\{x\}denote the input conversation represented as a list of messages\. The student and teacher with different tokenizers may use different chat templates, denoted by𝒞θ\\mathcal\{C\}\_\{\\theta\}and𝒞ϕ\\mathcal\{C\}\_\{\\phi\}, respectively\. We first construct the student input context textcθ=𝒞θ​\(𝐱\)c\_\{\\theta\}=\\mathcal\{C\}\_\{\\theta\}\(\\mathbf\{x\}\)and sample a response\-token sequence with the student policy executed by the rollout engineπθroll\\pi\_\{\\theta\_\{\\mathrm\{roll\}\}\}:

y1:n=\(y1,…,yn\)∼πθroll\(⋅∣cθ\)\.y\_\{1:n\}=\(y\_\{1\},\\ldots,y\_\{n\}\)\\sim\\pi\_\{\\theta\_\{\\mathrm\{roll\}\}\}\(\\cdot\\mid c\_\{\\theta\}\)\.\(1\)
Decodingy1:ny\_\{1:n\}via the tokenizer decoder of the student gives the response surface strings=𝒟θ\(y1:n\)s=\\mathcal\{D\}\_\{\\theta\}\(y\_\{1:n\}\)\. Instead of passing the student context to the teacher, we reconstruct the teacher context textcϕc\_\{\\phi\}using its own chat template and append the student response:

cϕ=𝒞ϕ​\(𝐱\),uϕ=cϕ⊕s,c\_\{\\phi\}=\\mathcal\{C\}\_\{\\phi\}\(\\mathbf\{x\}\),\\quad u\_\{\\phi\}=c\_\{\\phi\}\\oplus s,where⊕\\oplusdenotes string concatenation\. The complete textuϕu\_\{\\phi\}is then provided to the teacher, which tokenizes it using the tokenizer encoderℰϕ\\mathcal\{E\}\_\{\\phi\}of the teacher\. Letℰϕ​\(s\|cϕ\)=\(z1,…,zm\)\\mathcal\{E\}\_\{\\phi\}\(s\|c\_\{\\phi\}\)=\(z\_\{1\},\\ldots,z\_\{m\}\)denote token list obtained by encoding the student response given the input contextcϕc\_\{\\phi\}using the teacher tokenizer\. This procedure allows the teacher to evaluate the student response under its native tokenizer and chat\-template\. Althoughcθc\_\{\\theta\}andcϕc\_\{\\phi\}may differ, the response text being evaluated is identical on both sides\. For token alignment, letτθ​\(yt\)\\tau\_\{\\theta\}\(y\_\{t\}\)andτϕ​\(zi\)\\tau\_\{\\phi\}\(z\_\{i\}\)denote the incremental text spans contributed by the corresponding response tokens\. They satisfy

⨁t=1nτθ​\(yt\)=⨁i=1mτϕ​\(zi\)=s\.\\bigoplus\_\{t=1\}^\{n\}\\tau\_\{\\theta\}\(y\_\{t\}\)=\\bigoplus\_\{i=1\}^\{m\}\\tau\_\{\\phi\}\(z\_\{i\}\)=s\.We perform no additional cleanup or normalization, so these spans define consistent offsets in the shared response string\. The teacher log\-probability associated with tokenziz\_\{i\}islog⁡πϕ​\(zi∣cϕ,z<i\)\\log\\pi\_\{\\phi\}\(z\_\{i\}\\mid c\_\{\\phi\},z\_\{<i\}\)\.

### 2\.1Cross\-tokenizer Alignment

Define the cumulative response text preceding each student and teacher token as

Pθ​\(t\)=⨁k=1t−1τθ​\(yk\),Pϕ​\(i\)=⨁k=1i−1τϕ​\(zk\),P\_\{\\theta\}\(t\)=\\bigoplus\_\{k=1\}^\{t\-1\}\\tau\_\{\\theta\}\(y\_\{k\}\),\\qquad P\_\{\\phi\}\(i\)=\\bigoplus\_\{k=1\}^\{i\-1\}\\tau\_\{\\phi\}\(z\_\{k\}\),withPθ​\(1\)=Pϕ​\(1\)=εP\_\{\\theta\}\(1\)=P\_\{\\phi\}\(1\)=\\varepsilon, whereε\\varepsilondenotes the empty string\. A teacher positioniiis aligned with a student positionttif both tokenizations have consumed the same response prefix and the current tokens contribute the same text span:

ℳ=\{\(i,t\):Pϕ​\(i\)=Pθ​\(t\)∧τϕ​\(zi\)=τθ​\(yt\)\}\.\\mathcal\{M\}=\\\{\(i,t\):P\_\{\\phi\}\(i\)=P\_\{\\theta\}\(t\)\\land\\tau\_\{\\phi\}\(z\_\{i\}\)=\\tau\_\{\\theta\}\(y\_\{t\}\)\\\}\.\(2\)Equivalently, an aligned teacher\-student pair covers the same start and end offsets in the shared response stringss\. Tokens that overlap only partially are not aligned because the log‑probability of one teacher token cannot be uniquely assigned to multiple student tokens, nor can the log‑probabilities of multiple teacher tokens be uniquely merged into a single student token\. Sinceyyandzzare ordered segmentations of the same response string,ℳ\\mathcal\{M\}is a partial one\-to\-one mapping: each token on either side can match at most one token on the other side\. We enumerateℳ\\mathcal\{M\}using a linear two\-pointer scan\. The scan maintains the accumulated response prefixes on both sides\. If the prefixes and current token spans agree, the two tokens are aligned and both pointers advance\. Otherwise, the side that has consumed less response text advances until the two prefixes meet again\. When both sides have consumed the same amount of text but the current token spans differ, both pointers advance\.

For each student positiontt, define the alignment indicator

at=𝟏\[∃isuch that\(i,t\)∈ℳ\]\.a\_\{t\}=\\mathbf\{1\}\[\\exists i\\text\{ such that \}\(i,t\)\\in\\mathcal\{M\}\]\.\(3\)Letlog⁡πθ​\(yt∣cθ,y<t\)\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\)andlog⁡πϕ​\(zi∣cϕ,z<i\)\\log\\pi\_\{\\phi\}\(z\_\{i\}\\mid c\_\{\\phi\},z\_\{<\{i\}\}\)denote the log\-probabilities of student policy and teacher policy, respectively\. We construct a student\-length teacher target as

ℓ~tϕ=\{log⁡πϕ​\(zi∣cϕ,z<i\),at=1,log⁡πθ​\(yt∣cθ,y<t\),at=0\.\\widetilde\{\\ell\}\_\{t\}^\{\\phi\}=\\begin\{cases\}\\log\\pi\_\{\\phi\}\(z\_\{i\}\\mid c\_\{\\phi\},z\_\{<\{i\}\}\),&a\_\{t\}=1,\\\\ \\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\),&a\_\{t\}=0\.\\end\{cases\}\(4\)Thus, aligned positions inherit the corresponding teacher log\-probability, while unmatched positions fall back to the student’s log\-probability\. We report the lexical overlap ratioρ=\|ℳ\|n,\\rho=\\frac\{\|\\mathcal\{M\}\|\}\{n\},which measures the fraction of student response tokens that receive teacher supervision\. The complete alignment procedure is provided in Algorithm[1](https://arxiv.org/html/2608.14277#algorithm1)in Appendix[A](https://arxiv.org/html/2608.14277#A1)\.

### 2\.2On\-policy distillation objective

The cross\-tokenizer distillation objective is defined over aligned response positions:

ℒDistill​\(θ\)=𝔼y∼πθ​\[∑t=1nlog⁡πθ​\(yt∣cθ,y<t\)−ℓ~tϕ\]\.\\mathcal\{L\}\_\{\\mathrm\{Distill\}\}\(\\theta\)=\\mathbb\{E\}\_\{y\\sim\\pi\_\{\\theta\}\}\\left\[\\sum\_\{t=1\}^\{n\}\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\)\-\\widetilde\{\\ell\}\_\{t\}^\{\\phi\}\\right\]\.\(5\)This objective is a token\-aligned surrogate for the reverse KL divergence\. It compares the teacher and student probabilities only at positions where the two tokenizers induce the same local segmentation of the response string\. When the tokenizers are identical, every student token is aligned with the corresponding teacher token, soat=1a\_\{t\}=1andzt=ytz\_\{t\}=y\_\{t\}for alltt\. The objective then reduces to

ℒDistill\(θ\)=𝔼y∼πθ\[logπθ​\(y∣cθ\)πϕ​\(y∣cϕ\)\]=DKL\(πθ\(⋅∣cθ\)∥πϕ\(⋅∣cϕ\)\)\.\\mathcal\{L\}\_\{\\mathrm\{Distill\}\}\(\\theta\)=\\mathbb\{E\}\_\{y\\sim\\pi\_\{\\theta\}\}\\left\[\\log\\frac\{\\pi\_\{\\theta\}\(y\\mid c\_\{\\theta\}\)\}\{\\pi\_\{\\phi\}\(y\\mid c\_\{\\phi\}\)\}\\right\]=D\_\{\\mathrm\{KL\}\}\\left\(\\pi\_\{\\theta\}\(\\cdot\\mid c\_\{\\theta\}\)\\parallel\\pi\_\{\\phi\}\(\\cdot\\mid c\_\{\\phi\}\)\\right\)\.\(6\)When the tokenizers differ, the same response string is factorized into different token sequences, so the exact token\-level KL is not directly computable\. The proposed objective instead applies teacher supervision only where the two tokenizations agree on both the token boundary and textual span\. To enable multiple policy updates on the same rollout batch, we substitute the online log\-probabilitylog⁡πθ​\(yt∣cθ,y<t\)\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\)inℓ~tϕ\\widetilde\{\\ell\}\_\{t\}^\{\\phi\}with its pre\-update counterpartlog⁡πθold​\(yt∣cθ,y<t\)\\log\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\)for all unmatched positions \(at=0a\_\{t\}=0\) and define the fixed policy advantages as

A^t=ℓ~tϕ−log⁡πθold​\(yt∣cθ,y<t\),\\widehat\{A\}\_\{t\}=\\widetilde\{\\ell\}\_\{t\}^\{\\phi\}\-\\log\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\),\(7\)using the PPO clipped policy loss\. The importance\-sampling ratio is

rt=πθ​\(yt∣cθ,y<t\)πθold​\(yt∣cθ,y<t\)\.r\_\{t\}=\\frac\{\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{t\}\\mid c\_\{\\theta\},y\_\{<t\}\)\}\.\(8\)
The resulting objective is

ℒθ=−𝔼⁡\[∑t=1nmin⁡\(rt​A^t,clip⁡\(rt,1−ϵ,1\+ϵ\)​A^t\)\]\.\\mathcal\{L\}\_\{\\theta\}=\-\\mathbb\{E\}\\Bigg\[\\sum\_\{t=1\}^\{n\}\\min\\Big\(r\_\{t\}\\widehat\{A\}\_\{t\},\\;\\operatorname\{clip\}\(r\_\{t\},1\-\\epsilon,1\+\\epsilon\)\\widehat\{A\}\_\{t\}\\Big\)\\Bigg\]\.\(9\)

## 3Experiment

We adopt SU\-01, an IMO gold\-medal\-level mathematical reasoning model developed by our team, as the teacher model\. Our goal is to transfer its mathematical reasoning and proof\-generation capabilities to a diverse set of student models through on\-policy distillation\.

We consider both same\-vocabulary and cross\-tokenizer settings\. For the same\-tokenizer setting, we distill SU\-01 into models from the Qwen3 family, including Qwen3\-4B\-thinking\-2507 and Qwen3\-30B\-A3B\-Thinking\-2507 \(abbreviated as Qwen3\-4B and Qwen3\-30B\-A3B, respectively\)\. For the cross\-tokenizer setting, we further evaluate models with different tokenizer designs, including Qwen3\.5\-4B, Qwen3\.5\-35B\-A3B, Intern\-S2\-Preview, GLM\-4\.7\-Flash, and Gemma\-4\-26B\-A4B\-it \(abbreviated as Gemma\-4\-26B\-A4B\)\. This setting allows us to examine whether cross\-tokenizer OPD can transfer mathematical proof capabilities across different model architectures and tokenization schemes\.

Figure 3:Training dynamics of Intern\-S2\-Preview during direct OPD training\.### 3\.1Training Data

All training examples are mathematical proof problems\. The collection combines curated proof corpora, community\-contributed problems, and olympiad training materials to provide a varied set of proof\-oriented prompts\. Our training set consists of:

1. 1\.OPC: 63 problems from the Open Proof Corpus\.
2. 2\.AoPS: 2,948 problems collected from the Art of Problem Solving community\.
3. 3\.Books: 900 problems collected from online mathematical competition training books\.
4. 4\.Shuzhimi: 617 problems sourced from the Shuzhimi Forum, an online Chinese mathematical problem\-solving community, and Evan Chen’s olympiad materials\.

### 3\.2Implementation Details

We perform OPD using responses sampled from the student policy, withSU\-01serving as the teacher model\. SU\-01 is a 30B\-A3B reasoning model based on Qwen3\-30B\-A3B with strong mathematical proof capabilities, achieving gold\-medal\-level performance on olympiad\-level evaluations and demonstrating particularly competitive results on proof\-oriented benchmarks such as ProofBench\. It is also capable of sustaining over 100K tokens of natural\-language reasoning for difficult Olympiad problems, making it a powerful yet highly long\-context teacher\. During distillation, SU\-01 provides token\-level supervision at positions that can be aligned across the teacher and student tokenizations\.

Figure 4:Training dynamics of Qwen3\.5\-35B\-A3B during direct OPD training\.We use Slime\([38](https://arxiv.org/html/2608.14277#bib.bib15)\)as the underlying framework for OPD and SGLang\([37](https://arxiv.org/html/2608.14277#bib.bib16)\)for evaluation rollout\. The models are trained for 100 rollout iterations with a constant learning rate of 1e\-6, a rollout batch size of 64, 4 responses sampled per prompt, and a maximum rollout length of 32K tokens for Qwen\-Series, 6k for GLM\-4\.7 and Gemma\-4\. The policy objective uses a PPO clipping coefficient of 0\.2\. The policy is updated four times per rollout step\.

In our evaluation, we primarily evaluate the non\-verifiable ProofBench and the verifiable AnswerBench from the IMO\-Bench suite\([20](https://arxiv.org/html/2608.14277#bib.bib29)\), together with the verifiable benchmarks AIME25\([23](https://arxiv.org/html/2608.14277#bib.bib33)\)and AMOBench\([2](https://arxiv.org/html/2608.14277#bib.bib32)\)\. For ProofBench, we use DeepSeek\-V4\-Flash as the judge for cost\-efficient evaluation\. To mitigate evaluation instability, we evaluate 4 times for the same proof rollout and choose the average result\. For the latter three, we first use a rule\-based verifier for testing; if the answer is correct, it is counted as correct, and if it is wrong, we then use GPT\-OSS\-120B\([24](https://arxiv.org/html/2608.14277#bib.bib38)\)for evaluation\. The reported results are the averages over 4 rollouts for ProofBench and 8 for the other three\.

We use the following evaluation settings: temperature 1\.0, top\-p 0\.95, repetition penalty 1\.0, and a maximum response length of 160,000 tokens\. The best checkpoint is selected based on the average score of AIME@4 and AnswerBench@1\.

## 4Experimental Results

### 4\.1Training Instability in On\-Policy Distillation

In this section, we first examine training instability in direct on\-policy distillation, including length explosion and truncation\. Accordingly, we propose two mitigation strategies, namely masking special tokens and adding a student reference KL loss\.

#### 4\.1\.1Initial Observation: Length Explosion and Truncation

During OPD training, the student model Intern\-S2\-Preview shows modest improvements in task performance, but these gains are accompanied by clear signs of degeneration \(Figure[3](https://arxiv.org/html/2608.14277#S3.F3)\)\. Specifically, the truncation rate and repetition rate both increase substantially as training progresses, while the average response length also grows sharply\. This phenomenon is even more pronounced for Qwen3\.5\-35B\-A3B \(Figure[4](https://arxiv.org/html/2608.14277#S3.F4)\), as its task performance deteriorates more noticeably, and the degradation is accompanied by a persistently high truncation rate and rapidly increasing response length\. These results suggest that apparent performance gains can come at the cost of increasingly verbose and unstable generation behavior, revealing a trade\-off between task performance and output quality\. Case studies could be found in Appendix[C](https://arxiv.org/html/2608.14277#A3)\.

#### 4\.1\.2Special Token Masking

During training, we observe that the student model’s output length keeps increasing, and many generated responses fail to emit termination tokens such as<\|im​\_​end\|\>\{\\scriptstyle<\}\\,\\mathrm\{\|im\\\_end\|\}\\,\{\\scriptstyle\>\}\. This suggests that the student is strongly affected by the long outputs from the teacher and gradually loses the ability to terminate properly\. To mitigate this issue, we mask the OPD loss on the structural tokens</think\>\{\\scriptstyle<\}\\,\\mathrm\{/think\}\{\\scriptstyle\>\}and<\|im​\_​end\|\>\{\\scriptstyle<\}\\,\\mathrm\{\|im\\\_end\|\}\\,\{\\scriptstyle\>\}, as these tokens primarily control the output format and termination behavior and do not need to match the teacher distribution exactly\.

The validation curve is shown in Figure[5](https://arxiv.org/html/2608.14277#S4.F5)\. The results suggest that the masking strategy helps mitigate length\-related instability\. However, in the later stage of training, the model’s length truncation rate still continues to increase sharply, indicating that special\-token masking alone cannot fully resolve the length expansion problem\. The evaluation results are shown in Table[1](https://arxiv.org/html/2608.14277#S4.T1)\. The results show that masking the advantages of special termination tokens improves OPD training stability, but special token masking alone cannot fully resolve the length\-expansion problem\.

Figure 5:Intern\-S2\-Preview OPD training dynamics when masking the special tokens such as</think\>\{\\scriptstyle<\}\\,\\mathrm\{/think\}\{\\scriptstyle\>\}and<\|im​\_​end\|\>\{\\scriptstyle<\}\\,\\mathrm\{\|im\\\_end\|\}\\,\{\\scriptstyle\>\}\. The special\-token masking alone helps mitigate length\-related instability but cannot resolve the length expansion problem\.Figure 6:Intern\-S2\-Preview OPD training dynamics when adding student reference KL loss to prevent the student policy from deviating excessively from its initial distribution\. The training could be stabilized, and the truncation rate is effectively reduced to nearly zero\.Table 1:OPD results from SU\-01 to Intern\-S2\-Preview with masking termination tokens and adding student reference KL loss\.ModelProofBench@4AnswerBench@8AIME25@8SU\-0145\.0077\.5094\.60Intern\-S2\-Preview21\.7076\.0388\.33OPD \+ Spec Mask38\.10\(\+Δ​16\.40\)\(\+\\Delta\\,16\.40\)77\.60\(\+Δ​1\.57\)\(\+\\Delta\\,1\.57\)95\.00\(\+Δ​6\.67\)\(\+\\Delta\\,6\.67\)OPD \+ Ref KL38\.50\(\+Δ​16\.80\)\(\+\\Delta\\,16\.80\)79\.10\(\+Δ​3\.07\)\(\+\\Delta\\,3\.07\)95\.80\(\+Δ​7\.47\)\(\+\\Delta\\,7\.47\)
#### 4\.1\.3Student\-Reference KL Loss

We then attempt to introduce a student reference KL loss to prevent the student policy from deviating excessively from its initial distribution, thereby preserving the student model’s general capabilities during distillation\. In this experiment, we set the student\-reference kl\-loss coefficient to 0\.5\.

The validation curves during training are shown in Figure[6](https://arxiv.org/html/2608.14277#S4.F6)\. As observed, the truncation rate is effectively reduced to nearly zero, while performance on AIME25 and AnswerBench is consistently improved\. This indicates that the student reference KL loss can effectively mitigate the dramatic increase in response length during OPD training\. The evaluation results are summarized in Table[1](https://arxiv.org/html/2608.14277#S4.T1)\. The results demonstrate that adding the reference KL loss substantially improves OPD training while preserving the student model’s original capabilities\. Specifically, OPD \+ Ref KL improves ProofBench@4 from 21\.70 to 38\.50, and further raises AnswerBench@8 and AIME25@8 to 79\.10 and 95\.80, respectively\.

### 4\.2Results on Same\-family Models

Finally, we combine special token masking with the student reference KL loss, which yields larger performance improvements\. We set kl\-loss\-coef to 0\.5 for the Qwen\-series models and Intern\-S2\-Preview\.

Table 2:Evaluation results of distilling SU\-01 to students on ProofBench, AnswerBench, AIME25, and AMOBench\. Green values indicate improvements over the corresponding base model\.ModelProofBench@4AnswerBench@8AIME25@8AMOBench@8SU\-0145\.0077\.5094\.6061\.75Same\-Tokenizer ModelsQwen3\-4B11\.4247\.5071\.2523\.00Qwen3\-4B\-OPD23\.72\(\+Δ​12\.30\)\(\+\\Delta\\,12\.30\)64\.50\(\+Δ​17\.00\)\(\+\\Delta\\,17\.00\)90\.83\(\+Δ​19\.58\)\(\+\\Delta\\,19\.58\)35\.00\(\+Δ​12\.00\)\(\+\\Delta\\,12\.00\)Qwen3\-30B\-A3B13\.8059\.1388\.3336\.50Qwen3\-30B\-A3B\-OPD36\.47\(\+Δ​22\.67\)\(\+\\Delta\\,22\.67\)74\.46\(\+Δ​15\.33\)\(\+\\Delta\\,15\.33\)93\.75\(\+Δ​5\.42\)\(\+\\Delta\\,5\.42\)52\.75\(\+Δ​16\.25\)\(\+\\Delta\\,16\.25\)Cross\-tokenizer ModelsQwen3\.5\-4B15\.9060\.9486\.6732\.00Qwen3\.5\-4B\-OPD28\.61\(\+Δ​12\.71\)\(\+\\Delta\\,12\.71\)67\.84\(\+Δ​6\.90\)\(\+\\Delta\\,6\.90\)91\.67\(\+Δ​5\.00\)\(\+\\Delta\\,5\.00\)51\.25\(\+Δ​19\.25\)\(\+\\Delta\\,19\.25\)Qwen3\.5\-35B\-A3B26\.7873\.1694\.6057\.25Qwen3\.5\-35B\-A3B\-OPD42\.39\(\+Δ​15\.61\)\(\+\\Delta\\,15\.61\)80\.15\(\+Δ​6\.99\)\(\+\\Delta\\,6\.99\)96\.66\(\+Δ​2\.06\)\(\+\\Delta\\,2\.06\)61\.25\(\+Δ​4\.00\)\(\+\\Delta\\,4\.00\)Intern\-S2\-Preview21\.7076\.0388\.3358\.00Intern\-S2\-OPD44\.50\(\+Δ​22\.80\)\(\+\\Delta\\,22\.80\)80\.10\(\+Δ​4\.07\)\(\+\\Delta\\,4\.07\)95\.00\(\+Δ​6\.67\)\(\+\\Delta\\,6\.67\)59\.50\(\+Δ​1\.50\)\(\+\\Delta\\,1\.50\)The experimental results are shown in Table[2](https://arxiv.org/html/2608.14277#S4.T2)\. Overall, SimpleOPD consistently improves performance across model sizes, tokenizer settings, and model families\. Among the same\-tokenizer models, Qwen3\-4B\-OPD gains 12\.30 points on ProofBench, 17\.00 points on AnswerBench, and 19\.58 points on AIME25\. Scaling the student to Qwen3\-30B\-A3B produces even greater proof\-reasoning improvements; its ProofBench score increases by 22\.67 points, from 13\.80 to 36\.47, while its AnswerBench and AIME25 scores improve by 15\.33 and 5\.42 points, respectively\.

The improvements also generalize to cross\-tokenizer distillation\. Qwen3\.5\-35B\-A3B\-OPD reaches 42\.39 on ProofBench, improving over its base model by 15\.61 points, and achieves additional gains of 6\.99 and 2\.06 points on AnswerBench and AIME25\. Intern\-S2\-Preview exhibits the largest ProofBench gain, improving by 22\.80 points from 21\.70 to 44\.50, nearly matching the teacher’s score of 45\.00\. It also reaches 80\.10 on AnswerBench and 95\.00 on AIME25, surpassing SU\-01 on both benchmarks\. These results demonstrate that our approach effectively transfers proof\-reasoning capabilities to diverse students without requiring shared tokenization or supervised fine\-tuning on teacher\-generated trajectories\.

We also evaluate ProofBench using Gemini\-2\.5\-Pro as the judge, following the same evaluation setting as SU\-01\. The results are shown in Figure[1](https://arxiv.org/html/2608.14277#S0.F1)\. The results show that OPD training brings a substantial improvement to Intern\-S2\-Preview on ProofBench\. Specifically, Intern\-S2\-OPD improves from 34\.0 to 55\.2, achieving a gain of 21\.2 points over the original Intern\-S2\-Preview model\. This demonstrates that OPD can effectively transfer proof\-reasoning capabilities from the long\-context teacher model to the student model\. Notably, Intern\-S2\-OPD also surpasses several strong frontier models, including Gemini\-2\.5\-Pro and GPT\-5\. Although it still trails SU\-01 and DeepSeek\-V3\.2\-Speciale, the gap is significantly narrowed after OPD training\. These results further confirm the effectiveness of OPD for improving natural\-language mathematical proof reasoning, especially in cross\-model capability transfer settings\.

### 4\.3Results on Cross\-family Models

We set student\-reference KL loss 1\.0 for GLM\-4\.7\-Flash and Gemma\-4\-26B\-A4B to further maintain stability\. Figure[7](https://arxiv.org/html/2608.14277#S4.F7)presents additional cross\-family distillation results on GLM\-4\.7\-Flash and Gemma\-4\-26B\-A4B\. GLM\-4\.7\-OPD improves consistently on both benchmarks, increasing from 30\.8 to 39\.7 on ProofBench and from 69\.6 to 72\.0 on AnswerBench\. Gemma\-4\-26B\-A4B\-OPD also achieves a substantial ProofBench gain, rising from 25\.5 to 34\.2, although its AnswerBench score decreases from 68\.8 to 67\.5\.

Figure 7:Cross\-family distillation results for GLM\-4\.7\-Flash and Gemma\-4\-26B\-A4B from SU\-01\.These results demonstrate the effectiveness of our method for cross\-family distillation, while also highlighting the challenge posed by tokenizer mismatch\. SU\-01 is based on the Qwen model family and uses a byte\-level BPE tokenizer\. GLM\-4\.7\-Flash also adopts a BPE\-based tokenizer, although with a different vocabulary, whereas Gemma uses a SentencePiece\-based tokenizer with substantially different vocabulary and segmentation behavior\. Accordingly, GLM benefits consistently across both benchmarks, while the more tokenizer\-dissimilar Gemma merely improves on ProofBench\. Nevertheless, the clear ProofBench improvements for both models demonstrate that our approach can transfer proof\-reasoning capabilities across model families and tokenizer boundaries, although larger tokenizer discrepancies may make such transfer more challenging\.

## 5Analysis

### 5\.1Comparison with OPD Baselines

Figure 8:Comparison of the Intern\-S2 base model distilled from SU\-01 with EOPD, G\-OPD, and SimpleOPD across four reasoning benchmarks\. All scores are reported as percentages\. Each panel uses an independently scaled vertical axis to make small performance differences visible\.We compare SimpleOPD with the unmodified Intern\-S2 model and two recent OPD variants: EOPD, which supplements reverse\-KL OPD with forward KL at positions where the teacher has high token\-level entropy\([14](https://arxiv.org/html/2608.14277#bib.bib4)\), and G\-OPD, which generalizes OPD through a flexible reference model and reward scaling\([33](https://arxiv.org/html/2608.14277#bib.bib5)\)\. As shown in Figure[8](https://arxiv.org/html/2608.14277#S5.F8), SimpleOPD improves the Intern\-S2 base model by 22\.80, 4\.07, 6\.67, and 1\.50 points on ProofBench, AnswerBench, AIME25, and AMOBench, respectively\. Among the OPD methods, SimpleOPD obtains 44\.50 on ProofBench, 80\.10 on AnswerBench, and 59\.50 on AMOBench, outperforming both EOPD and G\-OPD by a large margin\. On AIME25, SimpleOPD reaches 95\.00, which is 0\.33 points below EOPD and 0\.34 points above G\-OPD\. Overall, SimpleOPD achieves the best result on three of the four benchmarks, with its largest advantage appearing on the proof\-oriented ProofBench, while remaining competitive on AIME25\.

### 5\.2Lexical Overlap

The lexical\-overlap curves provide an estimate of how much token\-level supervision can be retained under cross\-tokenizer alignment\. As shown in Figure[9](https://arxiv.org/html/2608.14277#S5.F9), the models already exhibit a high aligned\-token ratio at the beginning of training, and the ratio increases further over time\. This indicates that, despite tokenizer differences, a large portion of the student\-generated text can be matched to teacher tokens through shared surface spans\. These results suggest that the partial alignment used in cross\-tokenizer OPD preserves a substantial amount of usable training signal in practice, while avoiding the requirement of full tokenizer compatibility\.

### 5\.3Out\-of\-domain Generalization

Although OPD training is conducted only on mathematical reasoning data, the resulting model also shows consistent gains on out\-of\-domain scientific reasoning benchmarks\. The results are shown in Table[3](https://arxiv.org/html/2608.14277#S5.T3)\. Compared with Intern\-S2\-Preview, Intern\-S2\-OPD achieves a slight improvement on FrontierScience\-Olympiad and HLE\([31](https://arxiv.org/html/2608.14277#bib.bib34);[26](https://arxiv.org/html/2608.14277#bib.bib30)\), and a clearer gain on HiPhO\([34](https://arxiv.org/html/2608.14277#bib.bib31)\), improving from 38\.6 to 41\.1\. On FrontierScience Research\([31](https://arxiv.org/html/2608.14277#bib.bib34)\), the score also improves from 1\.7 to 5\.0\. This suggests that the OPD\-trained model does not lose its broader scientific reasoning ability despite being trained only on math data, and may further benefit physics\-oriented reasoning\. In particular, Intern\-S2\-OPD outperforms SU\-01 on HiPhO, indicating that the student model retains its own strengths while gaining additional reasoning capability through SimpleOPD\.

Figure 9:The lexical\-overlap curves of Qwen3\.5\-35B\-A3B, Intern\-S2\-Preview and GLM\-4\.7\-Flash during OPD training from the teacher SU\-01\.Table 3:Evaluation results of Intern\-S2\-OPD distilled from SU\-01 on scientific reasoning benchmarks\.ModelFrontierScienceOlympiadFrontierScienceResearchHLE\(text\-only\)HiPhOSU\-0161\.511\.720\.735\.0Intern\-S2\-Preview60\.61\.719\.638\.6Intern\-S2\-OPD60\.9\(\+Δ​0\.3\)\(\+\\Delta\\,0\.3\)5\.0\(\+Δ​3\.3\)\(\+\\Delta\\,3\.3\)20\.5\(\+Δ​0\.9\)\(\+\\Delta\\,0\.9\)41\.1\(\+Δ​2\.5\)\(\+\\Delta\\,2\.5\)
### 5\.4Effect of Training Data

We further analyze the effect of data composition in distillation\. Specifically, we include the verifiable math data used in SU\-01 during the OPD process in addition to the proof data\.

The results show that both data compositions substantially improve Intern\-S2\-Preview over the base model, confirming the effectiveness of OPD distillation\. Using only proof data achieves the best performance on ProofBench@4, improving Intern\-S2 from 21\.70 to 44\.50 and approaching the teacher model SU\-01\. This suggests that proof\-focused data is more beneficial for transferring natural\-language proof\-reasoning ability\. Adding verifiable math data brings only a marginal improvement on AnswerBench@8, increasing the score from 80\.10 to 81\.10, while AIME25@8 remains unchanged at 95\.00\. However, this mixed\-data setting significantly underperforms proof\-only data on ProofBench@4, dropping from 44\.50 to 38\.50\. These results indicate that adding verifiable data provides limited additional benefit for general mathematical reasoning, while potentially weakening the transfer of proof\-reasoning ability\.

### 5\.5Effect of OPD Length

We analyze the effect of the distillation length on the student models \(Table[4](https://arxiv.org/html/2608.14277#S5.T4)\)\. By comparison, we find that for long\-context proof\-reasoning tasks, a 6k distillation length is insufficient to fully capture the teacher model’s reasoning patterns\. Specifically, increasing the distillation length from 6k to 32k improves the student model consistently across all evaluated benchmarks when the distillation is stable\. In Qwen3\.5\-OPD, ProofBench@4 increases from 40\.07 to 42\.39, AnswerBench@8 from 77\.97 to 80\.16, and AIME25@8 from 96\.25 to 96\.67\. The improvement is most pronounced on ProofBench, suggesting that longer distillation sequences are particularly important for transferring complex, multi\-step proof reasoning behavior\. These results indicate that preserving longer reasoning traces during distillation helps the student model better approximate the teacher’s long\-context reasoning capability\.

Table 4:Results of different distillation lengths on Intern\-S2\-Preview and Qwen3\.5\-35B\-A3B distilled from the teacher SU\-01\.ModelLengthProofBench@4AnswerBench@8AIME25@8SU\-01–45\.0077\.5094\.60Intern\-S2\-Preview–21\.7076\.0388\.33Qwen3\.5\-35B\-A3B–26\.7873\.1694\.606k38\.80\(\+Δ​17\.10\)\(\+\\Delta\\,17\.10\)77\.25\(\+Δ​1\.22\)\(\+\\Delta\\,1\.22\)95\.00\(\+Δ​6\.67\)\(\+\\Delta\\,6\.67\)Intern\-S2\-OPD32k44\.50\(\+Δ​22\.80\)\(\+\\Delta\\,22\.80\)80\.10\(\+Δ​4\.07\)\(\+\\Delta\\,4\.07\)95\.00\(\+Δ​6\.67\)\(\+\\Delta\\,6\.67\)6k40\.07\(\+Δ​13\.29\)\(\+\\Delta\\,13\.29\)77\.97\(\+Δ​4\.81\)\(\+\\Delta\\,4\.81\)96\.25\(\+Δ​1\.65\)\(\+\\Delta\\,1\.65\)Qwen3\.5\-35B\-A3B\-OPD32k42\.39\(\+Δ​15\.61\)\(\+\\Delta\\,15\.61\)80\.16\(\+Δ​7\.00\)\(\+\\Delta\\,7\.00\)96\.67\(\+Δ​2\.07\)\(\+\\Delta\\,2\.07\)
### 5\.6Distillation from DeepSeek\-V4\-Flash

Table[5](https://arxiv.org/html/2608.14277#S5.T5)reports the results of distilling DeepSeek\-V4\-Flash\([7](https://arxiv.org/html/2608.14277#bib.bib22)\)into Intern\-S2\-Preview using a 6K distillation length for computational efficiency, together with termination\-token masking and student\-reference KL regularization\. Despite the shorter distillation length, Intern\-S2\-DS\-OPD consistently outperforms its base model across all three benchmarks\. Its ProofBench@4 score increases from 21\.70 to 39\.71, corresponding to a gain of 18\.01 points\. AnswerBench@8 improves from 76\.03 to 77\.94, while AIME25@8 increases from 88\.33 to 97\.50, yielding gains of 1\.91 and 9\.17 points, respectively\. Moreover, its ProofBench@4 score of 39\.71 is 0\.91 points higher than the 38\.80 achieved using SU\-01 as the teacher with the same 6K distillation length in Table[4](https://arxiv.org/html/2608.14277#S5.T4), suggesting that a stronger teacher can lead to greater improvements in the student\. Overall, these results demonstrate that SimpleOPD can effectively transfer reasoning capabilities across distinct model families and from a substantially larger teacher \(158B\) to a smaller student \(30B\), even with a relatively short distillation length\.

### 5\.7Effect of Student\-Reference KL Loss

We further investigate how the student\-reference KL coefficient affects GLM distillation\. This coefficient controls the extent to which the student is regularized toward its reference policy and therefore influences the balance between preserving the base model’s behavior and learning from the teacher\. We evaluate coefficients of 0\.5, 1\.0, and 1\.2 to study this trade\-off\.

As shown in Table[6](https://arxiv.org/html/2608.14277#S5.T6), GLM\-4\.7\-Flash\-OPD consistently outperforms the base model under all three settings\. Among them, a coefficient of 1\.0 achieves the best overall performance, improving ProofBench@4 from 30\.75 to 39\.71 and AnswerBench@8 from 69\.59 to 71\.97, with gains of 8\.96 and 2\.38 points, respectively\. A smaller coefficient of 0\.5 provides insufficient regularization and yields lower ProofBench and AnswerBench scores\. Conversely, increasing the coefficient to 1\.2 imposes a stronger constraint on the student policy, which may limit its ability to absorb the teacher’s reasoning capabilities\. Although coefficients of 0\.5 and 1\.2 achieve the highest AIME25@8 score of 94\.17, their overall performance is less balanced than that obtained with a coefficient of 1\.0\. These results suggest that the student\-reference KL coefficient should be neither too small nor too large, and that a moderate value provides the best balance between retaining the student’s original capabilities and acquiring knowledge from the teacher\.

Table 5:SimpleOPD results from DeepSeek\-V4\-Flash to Intern\-S2\-Preview with masking termination tokens and adding student reference KL loss\. The distillation length is 6k for computational efficiency\.ModelProofBench@4AnswerBench@8AIME25@8DeepSeek\-V4\-Flash52\.1877\.0099\.17Intern\-S2\-Preview21\.7076\.0388\.33Intern\-S2\-DS\-OPD39\.71\(\+Δ​18\.01\)\(\+\\Delta\\,18\.01\)77\.94\(\+Δ​1\.91\)\(\+\\Delta\\,1\.91\)97\.50\(\+Δ​9\.17\)\(\+\\Delta\\,9\.17\)Table 6:Ablation study on the student\-reference KL coefficient in the setting from SU\-01 to GLM\-4\.7\-Flash\. Green values indicate improvements over the GLM\-4\.7\-Flash base model\.ModelStudent KLCoefficientProofBench@4AnswerBench@8AIME25@8SU\-01–45\.0077\.5094\.60GLM\-4\.7\-Flash–30\.7569\.5992\.080\.537\.15\(\+Δ​6\.40\)\(\+\\Delta\\,6\.40\)71\.28\(\+Δ​1\.69\)\(\+\\Delta\\,1\.69\)94\.17\(\+Δ​2\.09\)\(\+\\Delta\\,2\.09\)1\.039\.71\(\+Δ​8\.96\)\(\+\\Delta\\,8\.96\)71\.97\(\+Δ​2\.38\)\(\+\\Delta\\,2\.38\)92\.91\(\+Δ​0\.83\)\(\+\\Delta\\,0\.83\)GLM\-4\.7\-Flash\-OPD1\.237\.96\(\+Δ​7\.21\)\(\+\\Delta\\,7\.21\)71\.19\(\+Δ​1\.60\)\(\+\\Delta\\,1\.60\)94\.17\(\+Δ​2\.09\)\(\+\\Delta\\,2\.09\)

## 6Related Work

### 6\.1Olympiad\-Level Long\-Context Reasoning

Early works such as OpenAI o1 and DeepSeek\-R1 established long chain\-of\-thought reasoning as a central test\-time scaling paradigm\([25](https://arxiv.org/html/2608.14277#bib.bib8);[6](https://arxiv.org/html/2608.14277#bib.bib9)\)\. This approach has enabled LLMs to tackle challenging real\-world tasks, including long\-horizon agentic tasks and Olympiad\-level reasoning\. Recent studies have further advanced reasoning capabilities in mathematics and science\. Gemini Deep Think was among the first systems to achieve gold\-medal\-level performance at the IMO\([21](https://arxiv.org/html/2608.14277#bib.bib17)\)\. Nemotron\-Math\-v2 and MiniMax subsequently reported similarly strong results on mathematical Olympiad benchmarks\([8](https://arxiv.org/html/2608.14277#bib.bib18);[4](https://arxiv.org/html/2608.14277#bib.bib19)\)\. Beyond mathematics, P1 and P1\-VL achieved gold\-medal\-level performance across several physics Olympiads, while SU\-01 introduced a unified recipe for eliciting Olympiad\-level proof behavior from a 30B model\([3](https://arxiv.org/html/2608.14277#bib.bib10);[19](https://arxiv.org/html/2608.14277#bib.bib20);[16](https://arxiv.org/html/2608.14277#bib.bib11)\)\. However, these capabilities typically rely on costly SFT and RL pipelines, including long\-trajectory curation, repeated data\-selection rollouts, and RL training\. The cost of each stage increases substantially with reasoning length\. This motivates our exploration of on\-policy distillation for efficiently transferring Olympiad\-level proof capabilities across diverse models\.

### 6\.2On\-policy Distillation

Recently, on\-policy distillation \(OPD\), introduced by MiniLLM and GKD\([11](https://arxiv.org/html/2608.14277#bib.bib1);[1](https://arxiv.org/html/2608.14277#bib.bib3)\), has emerged as an important component of LLM post\-training and has been adopted by flagship models such as GLM\-5, DeepSeek\-V4, and Qwen3\.5\([10](https://arxiv.org/html/2608.14277#bib.bib21);[7](https://arxiv.org/html/2608.14277#bib.bib22);[27](https://arxiv.org/html/2608.14277#bib.bib23)\)\. By optimizing the reverse KL divergence between teacher and student distributions over student\-generated rollouts, OPD combines on\-policy exploration with dense token\-level supervision, offering a stable and generalizable alternative to conventional SFT and RL\([11](https://arxiv.org/html/2608.14277#bib.bib1);[1](https://arxiv.org/html/2608.14277#bib.bib3)\)\. Subsequent work has broadened OPD beyond standard autoregressive students: Draft\-OPD adapts it to training\-based speculative draft models through verification\-error replay\([15](https://arxiv.org/html/2608.14277#bib.bib7)\), while[28](https://arxiv.org/html/2608.14277#bib.bib6)show that OPD follows a characteristic low\-dimensional update geometry distinct from SFT and RLVR\.

However, its effectiveness can be limited by mismatched reasoning patterns between teacher and student and unreliable supervision over long sequences\([1](https://arxiv.org/html/2608.14277#bib.bib3);[15](https://arxiv.org/html/2608.14277#bib.bib7)\)\. Moreover, most existing studies assume a shared vocabulary, typically distilling models within the same family or combining domain\-specific RL teachers\([11](https://arxiv.org/html/2608.14277#bib.bib1);[1](https://arxiv.org/html/2608.14277#bib.bib3);[7](https://arxiv.org/html/2608.14277#bib.bib22)\)\. OPD between heterogeneous models with different architectures and vocabularies remains underexplored\. In this work, we investigate heterogeneous OPD for transferring long\-context reasoning capabilities across diverse models\.

## 7Conclusion

In this work, we studied on\-policy distillation from a long\-context reasoning model, SU\-01, to short\-context student models\. To handle tokenizer differences, we performed OPD in a shared text space and aligned only tokens that occupy identical text spans under the student and teacher tokenizers\. We found that naive distillation led to severe training instability: the student’s output length kept increasing, termination tokens were often missing, and many responses were truncated\. To address this issue, we introduced two simple but effective stabilization techniques: \(1\) masking the OPD loss on structural termination tokens, including</think\>\{\\scriptstyle<\}\\,\\mathrm\{/think\}\{\\scriptstyle\>\}and<\|im​\_​end\|\>\{\\scriptstyle<\}\\,\\mathrm\{\|im\\\_end\|\}\\,\{\\scriptstyle\>\}, \(2\) adding a reference KL loss to constrain the student policy\. These techniques substantially reduced length explosion and truncation while improving reasoning performance\. Experiments on both same\-family and different\-family student models showed consistent gains in mathematical reasoning ability, with Intern\-S2\-Preview achieving a 21\.2\-point improvement on ProofBench\. These results demonstrate that our approach provides an effective and generalizable way to transfer long\-context reasoning capabilities to short\-context models across different tokenizers and model families\.

## Acknowledgments

This work was supported by the Shanghai Artificial Intelligence Laboratory\. We are grateful to the authors and open\-source communities whose work made this project possible\.

## References

- R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos Garea, M\. Geist, and O\. BachemOn\-policy distillation of language models: learning from self\-generated mistakes\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=3zKtaqxLhW)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p2.1)\.
- Anet al\.\(2025\)S\. An, X\. Cai, X\. Cao, X\. Li, Y\. Lin, J\. Liu, X\. Lv, D\. Ma, X\. Wang, Z\. Wang, and S\. ZhouAMO\-Bench: large language models still struggle in high school math competitions\.arXiv preprint arXiv:2510\.26768\.External Links:[Link](https://arxiv.org/abs/2510.26768)Cited by:[§3\.2](https://arxiv.org/html/2608.14277#S3.SS2.p3.1)\.
- Chenet al\.\(2025a\)J\. Chen, Q\. Cheng, F\. Yu, H\. Wan, Y\. Zhang, S\. Zheng, J\. Yao, Q\. Zhang, H\. He, Y\. Luo,et al\.P1: mastering physics olympiads with reinforcement learning\.arXiv preprint arXiv:2511\.13612\.External Links:[Link](https://arxiv.org/abs/2511.13612)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Chenet al\.\(2026\)J\. Chen, X\. Zhang, S\. Zhang, Y\. Wang, L\. Li, T\. Qin, Q\. Wang, Z\. Zhu, T\. Li, J\. Li,et al\.MaxProof: scaling mathematical proof with generative\-verifier RL and population\-level test\-time scaling\.arXiv preprint arXiv:2606\.13473\.External Links:[Link](https://arxiv.org/abs/2606.13473)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Chenet al\.\(2025b\)Y\. Chen, Y\. Liu, F\. Meng, Y\. Chen, J\. Xu, and J\. ZhouEnhancing cross\-tokenizer knowledge distillation with contextual dynamical mapping\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 8005–8018\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.419),[Link](https://aclanthology.org/2025.findings-acl.419/)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p2.1)\.
- DeepSeek\-AIet al\.\(2025\)DeepSeek\-AI, D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma,et al\.DeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.Nature645,pp\. 633–638\.External Links:[Link](https://arxiv.org/abs/2501.12948)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- DeepSeek\-AI \(2026\)DeepSeek\-AIDeepSeek\-V4: towards highly efficient million\-token context intelligence\.Note:Technical reportExternal Links:[Link](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main/DeepSeek_V4.pdf)Cited by:[§5\.6](https://arxiv.org/html/2608.14277#S5.SS6.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p2.1)\.
- Duet al\.\(2025\)W\. Du, S\. Toshniwal, B\. Kisacanin, S\. Mahdavi, I\. Moshkov, G\. Armstrong, S\. Ge, E\. Minasyan, F\. Chen, and I\. GitmanNemotron\-Math: efficient long\-context distillation of mathematical reasoning from multi\-mode supervision\.arXiv preprint arXiv:2512\.15489\.External Links:[Link](https://arxiv.org/abs/2512.15489)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Gemma Team \(2026\)Gemma TeamGemma 4 technical report\.External Links:2607\.02770,[Link](https://arxiv.org/abs/2607.02770)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1)\.
- GLM\-5 Team \(2026\)GLM\-5 TeamGLM\-5: from vibe coding to agentic engineering\.arXiv preprint arXiv:2602\.15763\.External Links:[Link](https://arxiv.org/abs/2602.15763)Cited by:[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1)\.
- Guet al\.\(2024\)Y\. Gu, L\. Dong, F\. Wei, and M\. HuangMiniLLM: knowledge distillation of large language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=5h0qf7IBZZ)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p2.1)\.
- Hübotteret al\.\(2026\)J\. Hübotter, F\. Lübeck, L\. Behric, A\. Baumann, M\. Bagatella, D\. Marta, I\. Hakimi, I\. Shenfeld, T\. K\. Buening, C\. Guestrin, and A\. KrauseReinforcement learning via self\-distillation\.External Links:2601\.20802,[Link](https://arxiv.org/abs/2601.20802)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p1.1)\.
- InternLM Team \(2026\)InternLM TeamIntern\-S2\-Preview\.Note:Hugging Face model cardExternal Links:[Link](https://huggingface.co/internlm/Intern-S2-Preview)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1)\.
- Jinet al\.\(2026\)W\. Jin, T\. Min, Y\. Yang, D\. Wei, Y\. Zhou, S\. R\. Kadhe, N\. Baracaldo, and K\. LeeEntropy\-aware on\-policy distillation of language models\.arXiv preprint arXiv:2603\.07079\.External Links:[Link](https://arxiv.org/abs/2603.07079)Cited by:[§5\.1](https://arxiv.org/html/2608.14277#S5.SS1.p1.1)\.
- Leiet al\.\(2026\)H\. Lei, Y\. Li, H\. Zhang, S\. Zhang, Q\. Cheng,et al\.Draft\-opd: on\-policy distillation for speculative draft models\.External Links:2605\.29343,[Link](https://arxiv.org/abs/2605.29343)Cited by:[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p2.1)\.
- Liet al\.\(2026a\)Y\. Li, R\. Zhan, H\. Zhang, S\. Zhang, Y\. Li, Z\. Wang, J\. Chen, F\. Wang, X\. Hu, Y\. Fan,et al\.Achieving gold\-medal\-level olympiad reasoning via simple and unified scaling\.arXiv preprint arXiv:2605\.13301\.External Links:[Link](https://arxiv.org/abs/2605.13301)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p3.1),[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Liet al\.\(2026b\)Y\. Li, Y\. Zuo, B\. He, J\. Zhang, C\. Xiao, C\. Qian, T\. Yu, H\. Gao, W\. Yang, Z\. Liu, and N\. DingRethinking on\-policy distillation of large language models: phenomenology, mechanism, and recipe\.arXiv preprint arXiv:2604\.13016\.External Links:[Link](https://arxiv.org/abs/2604.13016)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p2.1)\.
- Lu and Lab \(2025\)K\. Lu and T\. M\. LabOn\-policy distillation\.Thinking Machines Lab: Connectionism\.Note:https://thinkingmachines\.ai/blog/on\-policy\-distillationExternal Links:[Document](https://dx.doi.org/10.64434/tml.20251026)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p1.1)\.
- Luoet al\.\(2026\)Y\. Luo, F\. Wang, Q\. Cheng, F\. Yu, H\. Lei, J\. Yan, C\. Li, J\. Chen, Y\. Zhao, H\. Wan,et al\.P1\-VL: bridging visual perception and scientific reasoning in physics olympiads\.arXiv preprint arXiv:2602\.09443\.External Links:[Link](https://arxiv.org/abs/2602.09443)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Luonget al\.\(2025\)T\. Luong, D\. Hwang, H\. H\. Nguyen, G\. Ghiasi, Y\. Chervonyi, I\. Seo, J\. Kim, G\. Bingham, J\. Lee, S\. Mishra, A\. Zhai, H\. Hu, H\. Michalewski, J\. Kim, J\. Ahn, J\. Bae, X\. Song, T\. H\. Trinh, Q\. V\. Le, and J\. JungTowards robust mathematical reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 35418–35442\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1794),[Link](https://aclanthology.org/2025.emnlp-main.1794/)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1),[§3\.2](https://arxiv.org/html/2608.14277#S3.SS2.p3.1)\.
- Luong and Lockhart \(2025\)T\. Luong and E\. LockhartAdvanced version of Gemini with Deep Think officially achieves gold\-medal standard at the international mathematical olympiad\.Note:Google DeepMind BlogExternal Links:[Link](https://deepmind.google/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Maet al\.\(2026\)W\. Ma, J\. Wei, L\. Zhao, H\. Zhang, B\. Xiao, L\. Li, Q\. Yang, B\. Gao, Y\. Wang, R\. Li, J\. Dong, Z\. Sui, and F\. LuoMOPD: multi\-teacher on\-policy distillation for capability integration in llm post\-training\.External Links:2606\.30406,[Link](https://arxiv.org/abs/2606.30406)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p1.1)\.
- Mathematical Association of America \(2025\)Mathematical Association of AmericaAmerican invitational mathematics examination \(aime\) 2025\.External Links:[Link](https://maa.org/math-competitions/american-invitational-mathematics-examination-aime)Cited by:[§3\.2](https://arxiv.org/html/2608.14277#S3.SS2.p3.1)\.
- OpenAIet al\.\(2025\)OpenAI, :, S\. Agarwal, L\. Ahmad, J\. Ai, S\. Altman, A\. Applebaum, E\. Arbus, R\. K\. Arora, Y\. Bai, B\. Baker, H\. Bao, B\. Barak, A\. Bennett, T\. Bertao, N\. Brett, E\. Brevdo, G\. Brockman, S\. Bubeck, C\. Chang, K\. Chen, M\. Chen, E\. Cheung, A\. Clark, D\. Cook, M\. Dukhan, C\. Dvorak, K\. Fives, V\. Fomenko, T\. Garipov, K\. Georgiev, M\. Glaese, T\. Gogineni, A\. Goucher, L\. Gross, K\. G\. Guzman, J\. Hallman, J\. Hehir, J\. Heidecke, A\. Helyar, H\. Hu, R\. Huet, J\. Huh, S\. Jain, Z\. Johnson, C\. Koch, I\. Kofman, D\. Kundel, J\. Kwon, V\. Kyrylov, E\. Y\. Le, G\. Leclerc, J\. P\. Lennon, S\. Lessans, M\. Lezcano\-Casado, Y\. Li, Z\. Li, J\. Lin, J\. Liss, Lily, Liu, J\. Liu, K\. Lu, C\. Lu, Z\. Martinovic, L\. McCallum, J\. McGrath, S\. McKinney, A\. McLaughlin, S\. Mei, S\. Mostovoy, T\. Mu, G\. Myles, A\. Neitz, A\. Nichol, J\. Pachocki, A\. Paino, D\. Palmie, A\. Pantuliano, G\. Parascandolo, J\. Park, L\. Pathak, C\. Paz, L\. Peran, D\. Pimenov, M\. Pokrass, E\. Proehl, H\. Qiu, G\. Raila, F\. Raso, H\. Ren, K\. Richardson, D\. Robinson, B\. Rotsted, H\. Salman, S\. Sanjeev, M\. Schwarzer, D\. Sculley, H\. Sikchi, K\. Simon, K\. Singhal, Y\. Song, D\. Stuckey, Z\. Sun, P\. Tillet, S\. Toizer, F\. Tsimpourlas, N\. Vyas, E\. Wallace, X\. Wang, M\. Wang, O\. Watkins, K\. Weil, A\. Wendling, K\. Whinnery, C\. Whitney, H\. Wong, L\. Yang, Y\. Yang, M\. Yasunaga, K\. Ying, W\. Zaremba, W\. Zhan, C\. Zhang, B\. Zhang, E\. Zhang, and S\. ZhaoGpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Link](https://arxiv.org/abs/2508.10925)Cited by:[§3\.2](https://arxiv.org/html/2608.14277#S3.SS2.p3.1)\.
- OpenAI \(2024\)OpenAIOpenAI o1 system card\.External Links:[Link](https://openai.com/index/openai-o1-system-card/)Cited by:[§6\.1](https://arxiv.org/html/2608.14277#S6.SS1.p1.1)\.
- Phanet al\.\(2025\)L\. Phan, A\. Gatti, Z\. Han, N\. Li, J\. Hu, H\. Zhang, C\. B\. C\. Zhang, M\. Shaaban, J\. Ling, S\. Shi,et al\.Humanity’s last exam\.arXiv preprint arXiv:2501\.14249\.External Links:[Link](https://arxiv.org/abs/2501.14249)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1),[§5\.3](https://arxiv.org/html/2608.14277#S5.SS3.p1.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.5\.Note:Model releaseExternal Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1),[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1)\.
- Shenet al\.\(2026\)Z\. Shen, Y\. Li, Q\. Yin, C\. T\. Leong, Z\. Wang, Y\. Chen, R\. Han, S\. Lee, and Y\. R\. FungOn the geometry of on\-policy distillation\.arXiv preprint arXiv:2606\.07082\.External Links:[Link](https://arxiv.org/abs/2606.07082)Cited by:[§6\.2](https://arxiv.org/html/2608.14277#S6.SS2.p1.1)\.
- Singhet al\.\(2026\)A\. K\. Singh, Y\. Wu, A\. Cioba, A\. Bernacchia, and D\. BuffelliCross\-tokenizer LLM distillation through a byte\-level interface\.InProceedings of the Second Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual,San Diego, California, USA,pp\. 84–96\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.customnlp4u-1.9),[Link](https://aclanthology.org/2026.customnlp4u-1.9/)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p2.1)\.
- Sunet al\.\(2026\)J\. Sun, M\. Zheng, M\. Song, Q\. Zhong, Y\. Cheng, B\. Feng, P\. Liu, J\. Fang, and X\. WangSimCT: recovering lost supervision for cross\-tokenizer on\-policy distillation\.External Links:2605\.07711,[Link](https://arxiv.org/abs/2605.07711)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p1.1)\.
- Wanget al\.\(2026\)M\. Wang, R\. Lin, K\. Hu, J\. Jiao, N\. Chowdhury, E\. Chang, and T\. PatwardhanFrontierScience: evaluating AI’s ability to perform expert\-level scientific tasks\.arXiv preprint arXiv:2601\.21165\.External Links:[Link](https://arxiv.org/abs/2601.21165)Cited by:[§5\.3](https://arxiv.org/html/2608.14277#S5.SS3.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:[Link](https://arxiv.org/abs/2505.09388)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1)\.
- Yanget al\.\(2026\)W\. Yang, W\. Liu, R\. Xie, K\. Yang, S\. Yang, and Y\. LinLearning beyond teacher: generalized on\-policy distillation with reward extrapolation\.arXiv preprint arXiv:2602\.12125\.External Links:[Link](https://arxiv.org/abs/2602.12125)Cited by:[§5\.1](https://arxiv.org/html/2608.14277#S5.SS1.p1.1)\.
- Yuet al\.\(2025\)F\. Yu, H\. Wan, Q\. Cheng, Y\. Zhang, J\. Chen, F\. Han, Y\. Wu, J\. Yao, R\. Hu, N\. Ding, Y\. Cheng, T\. Chen, L\. Bai, D\. Zhou, Y\. Luo, G\. Cui, and P\. YeHiPhO: how far are \(M\)LLMs from humans in the latest high school physics olympiad benchmark?\.arXiv preprint arXiv:2509\.07894\.External Links:[Link](https://arxiv.org/abs/2509.07894)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1),[§5\.3](https://arxiv.org/html/2608.14277#S5.SS3.p1.1)\.
- Z\.ai \(2026\)Z\.aiGLM\-4\.7\-Flash\.Note:Hugging Face model cardExternal Links:[Link](https://huggingface.co/zai-org/GLM-4.7-Flash)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p4.1)\.
- Zhanget al\.\(2024\)S\. Zhang, X\. Zhang, Z\. Sun, Y\. Chen, and J\. XuDual\-space knowledge distillation for large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 18164–18181\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1010),[Link](https://aclanthology.org/2024.emnlp-main.1010/)Cited by:[§1](https://arxiv.org/html/2608.14277#S1.p2.1)\.
- Zhenget al\.\(2024\)L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. ShengSGLang: efficient execution of structured language model programs\.InAdvances in Neural Information Processing Systems,External Links:[Document](https://dx.doi.org/10.48550/arXiv.2312.07104),[Link](https://arxiv.org/abs/2312.07104)Cited by:[§3\.2](https://arxiv.org/html/2608.14277#S3.SS2.p2.1)\.
- Zhuet al\.\(2025\)Z\. Zhu, C\. Xie, X\. Lv, and slime ContributorsSlime: an LLM post\-training framework for RL scaling\.Note:[https://github\.com/THUDM/slime](https://github.com/THUDM/slime)GitHub repositoryCited by:[§3\.2](https://arxiv.org/html/2608.14277#S3.SS2.p2.1)\.

## Appendix ACross\-Tokenizer Scoring and Alignment

The complete procedure of cross\-tokenizer scoring and alignment is shown in Algorithm 1\.

Inputs :message sequence

𝐱\\mathbf\{x\}; student response tokens

y1:ny\_\{1:n\}; student log\-probabilities

ℓ1:nθ\\ell\_\{1:n\}^\{\\theta\}\.

Outputs :aligned teacher log\-probabilities

ℓ~1:nϕ\\widetilde\{\\ell\}\_\{1:n\}^\{\\phi\}; matched set

ℳ\\mathcal\{M\}\.

⊳\\trianglerightToken\-out, text\-in teacher scoring

s←𝒟θ\(y1:n\),cϕ←𝒞ϕ\(𝐱\),uϕ←cϕ⊕ss\\leftarrow\\mathcal\{D\}\_\{\\theta\}\(y\_\{1:n\}\),\\quad c\_\{\\phi\}\\leftarrow\\mathcal\{C\}\_\{\\phi\}\(\\mathbf\{x\}\),\\quad u\_\{\\phi\}\\leftarrow c\_\{\\phi\}\\oplus s

z=\(z1,…,zm\)←ℰϕ​\(s∣cϕ\)z=\(z\_\{1\},\\ldots,z\_\{m\}\)\\leftarrow\\mathcal\{E\}\_\{\\phi\}\(s\\mid c\_\{\\phi\}\)

ℓiϕ←log⁡πϕ​\(zi∣cϕ,z<i\),∀i∈\{1,…,m\}\\ell\_\{i\}^\{\\phi\}\\leftarrow\\log\\pi\_\{\\phi\}\(z\_\{i\}\\mid c\_\{\\phi\},z\_\{<i\}\),\\quad\\forall i\\in\\\{1,\\ldots,m\\\}

⊳\\trianglerightInitialize all positions with student log\-probabilities

ℓ~1:nϕ←ℓ1:nθ,ℳ←∅\\widetilde\{\\ell\}\_\{1:n\}^\{\\phi\}\\leftarrow\\ell\_\{1:n\}^\{\\theta\},\\quad\\mathcal\{M\}\\leftarrow\\emptyset

i←1,j←1,Hϕ←ε,Hθ←εi\\leftarrow 1,\\quad j\\leftarrow 1,\\quad H\_\{\\phi\}\\leftarrow\\varepsilon,\\quad H\_\{\\theta\}\\leftarrow\\varepsilon

⊳\\trianglerightτϕ\\tau\_\{\\phi\}andτθ\\tau\_\{\\theta\}return tokenizer\-native token pieces

while*i≤m∧j≤ni\\leq m\\land j\\leq n*do

pϕ←τϕ​\(zi\),pθ←τθ​\(yj\)p\_\{\\phi\}\\leftarrow\\tau\_\{\\phi\}\(z\_\{i\}\),\\quad p\_\{\\theta\}\\leftarrow\\tau\_\{\\theta\}\(y\_\{j\}\)

if*Hϕ=Hθ∧pϕ=pθH\_\{\\phi\}=H\_\{\\theta\}\\land p\_\{\\phi\}=p\_\{\\theta\}*then

ℓ~jϕ←ℓiϕ,ℳ←ℳ∪\{\(i,j\)\}\\widetilde\{\\ell\}\_\{j\}^\{\\phi\}\\leftarrow\\ell\_\{i\}^\{\\phi\},\\quad\\mathcal\{M\}\\leftarrow\\mathcal\{M\}\\cup\\\{\(i,j\)\\\}
Hϕ←Hϕ⊕pϕ,Hθ←Hθ⊕pθ,i←i\+1,j←j\+1H\_\{\\phi\}\\leftarrow H\_\{\\phi\}\\oplus p\_\{\\phi\},\\quad H\_\{\\theta\}\\leftarrow H\_\{\\theta\}\\oplus p\_\{\\theta\},\\quad i\\leftarrow i\+1,\\quad j\\leftarrow j\+1
else if*\|Hϕ\|\>\|Hθ\|\\lvert H\_\{\\phi\}\\rvert\>\\lvert H\_\{\\theta\}\\rvert*then

Hθ←Hθ⊕pθ,j←j\+1H\_\{\\theta\}\\leftarrow H\_\{\\theta\}\\oplus p\_\{\\theta\},\\quad j\\leftarrow j\+1
else if*\|Hϕ\|<\|Hθ\|\\lvert H\_\{\\phi\}\\rvert<\\lvert H\_\{\\theta\}\\rvert*then

Hϕ←Hϕ⊕pϕ,i←i\+1H\_\{\\phi\}\\leftarrow H\_\{\\phi\}\\oplus p\_\{\\phi\},\\quad i\\leftarrow i\+1
else

Hϕ←Hϕ⊕pϕ,Hθ←Hθ⊕pθ,i←i\+1,j←j\+1H\_\{\\phi\}\\leftarrow H\_\{\\phi\}\\oplus p\_\{\\phi\},\\quad H\_\{\\theta\}\\leftarrow H\_\{\\theta\}\\oplus p\_\{\\theta\},\\quad i\\leftarrow i\+1,\\quad j\\leftarrow j\+1
end if

end while

return

ℓ~1:nϕ,ℳ\\widetilde\{\\ell\}\_\{1:n\}^\{\\phi\},\\mathcal\{M\}

Algorithm 1Cross\-tokenizer teacher scoring and greedy token\-piece alignmentAs shown in Figure[10](https://arxiv.org/html/2608.14277#A1.F10), which plots an example of cross\-tokenizer scoring in the distillation from SU\-01 to Intern\-S2\-Preview, the visualization confirms that our alignment process allows most semantically informative tokens to receive supervision from the teacher\.

![Refer to caption](https://arxiv.org/html/2608.14277v1/token_kl_visualization.png)Figure 10:Per\-token reverse KL divergence in cross\-vocabulary distillation\. Colored boxes show aligned tokens with KL divergence values, while gold boxes indicate tokens that could not be aligned between teacher and student models\.
## Appendix BModel Solutions for ProofBench

This section presents two ProofBench examples to provide a qualitative view of the mathematical proof\-generation behavior learned through SimpleOPD\. Both ProofBench\-Basic\-001 and ProofBench\-Advanced\-028 are outputs generated by Intern\-S2\-OPD, the Intern\-S2\-Preview student distilled from SU\-01; each case includes the original problem statement, a concise solution overview, and the complete model\-generated proof\.

### B\.1ProofBench\-Basic\-001

QuestionDetermine all functionsf:ℤ→ℤf\\colon\\mathbb\{Z\}\\to\\mathbb\{Z\}such that, for allx,y∈ℤx,y\\in\\mathbb\{Z\},f⁡\(2​x\)\+2​f​\(y\)=f⁡\(f⁡\(x\+y\)\)\.f\(2x\)\+2f\(y\)=f\(f\(x\+y\)\)\.

Solution OverviewThe solution first evaluates the equation at carefully chosen arguments to derive a doubling identity and a translated Cauchy relation\. Subtracting the constantf⁡\(0\)f\(0\)then reduces the problem to an additive function onℤ\\mathbb\{Z\}, which must be linear\. Comparing coefficients and directly verifying the resulting candidates completes the classification\.

Intern\-S2\-OPD SolutionSolution\.For allx,y∈ℤx,y\\in\\mathbb\{Z\}, letP⁡\(x,y\)P\(x,y\)denote the assertionf⁡\(2​x\)\+2​f​\(y\)=f⁡\(f⁡\(x\+y\)\)f\(2x\)\+2f\(y\)=f\(f\(x\+y\)\)\(B\.1\)Step 1\. Two useful identities\.For everyt∈ℤt\\in\\mathbb\{Z\},P⁡\(0,t\)P\(0,t\)andP⁡\(t,0\)P\(t,0\)give, respectively,f⁡\(f⁡\(t\)\)\\displaystyle f\(f\(t\)\)=f⁡\(0\)\+2​f​\(t\),\\displaystyle=f\(0\)\+2f\(t\),\(B\.2\)f⁡\(f⁡\(t\)\)\\displaystyle f\(f\(t\)\)=f⁡\(2​t\)\+2​f​\(0\)\.\\displaystyle=f\(2t\)\+2f\(0\)\.\(B\.3\)Equating these expressions gives, for everyt∈ℤt\\in\\mathbb\{Z\},f⁡\(2​t\)=2​f​\(t\)−f⁡\(0\)\.f\(2t\)=2f\(t\)\-f\(0\)\.\(B\.4\)Step 2\. Deriving a Cauchy\-type relation\.Substituting \([B\.4](https://arxiv.org/html/2608.14277#A2.E4)\) into \([B\.1](https://arxiv.org/html/2608.14277#A2.E1)\) yieldsf⁡\(f⁡\(x\+y\)\)=2​f​\(x\)\+2​f​\(y\)−f⁡\(0\)\.f\(f\(x\+y\)\)=2f\(x\)\+2f\(y\)\-f\(0\)\.\(B\.5\)On the other hand, applying \([B\.2](https://arxiv.org/html/2608.14277#A2.E2)\) withttreplaced byx\+yx\+ygivesf⁡\(f⁡\(x\+y\)\)=f⁡\(0\)\+2​f​\(x\+y\)\.f\(f\(x\+y\)\)=f\(0\)\+2f\(x\+y\)\.\(B\.6\)Equating \([B\.5](https://arxiv.org/html/2608.14277#A2.E5)\) and \([B\.6](https://arxiv.org/html/2608.14277#A2.E6)\), we obtain, for allx,y∈ℤx,y\\in\\mathbb\{Z\},f⁡\(0\)\+2​f​\(x\+y\)=2​f​\(x\)\+2​f​\(y\)−f⁡\(0\),f\(0\)\+2f\(x\+y\)=2f\(x\)\+2f\(y\)\-f\(0\),and hencef⁡\(x\+y\)=f⁡\(x\)\+f⁡\(y\)−f⁡\(0\)f\(x\+y\)=f\(x\)\+f\(y\)\-f\(0\)\(B\.7\)Step 3\. Additivity after subtracting the constant\.Defineg⁡\(x\)=f⁡\(x\)−f⁡\(0\)g\(x\)=f\(x\)\-f\(0\)\. Theng⁡\(0\)=0g\(0\)=0, and, for allx,y∈ℤx,y\\in\\mathbb\{Z\}, \([B\.7](https://arxiv.org/html/2608.14277#A2.E7)\) becomesg⁡\(x\+y\)=g⁡\(x\)\+g⁡\(y\)\.g\(x\+y\)=g\(x\)\+g\(y\)\.Thusggis additive onℤ\\mathbb\{Z\}\. Sinceℤ\\mathbb\{Z\}is cyclic, any additive mapℤ→ℤ\\mathbb\{Z\}\\to\\mathbb\{Z\}is determined by its value at11\. In particular, for everyn∈ℤn\\in\\mathbb\{Z\},g⁡\(n\)=n​g​\(1\)\.g\(n\)=ng\(1\)\.Leta=g⁡\(1\)∈ℤa=g\(1\)\\in\\mathbb\{Z\}andb=f⁡\(0\)b=f\(0\)\. Consequently, for everyx∈ℤx\\in\\mathbb\{Z\},f⁡\(x\)=a​x\+b\.f\(x\)=ax\+b\.Step 4\. Determining the parametersaaandbb\.Substitutingf⁡\(x\)=a​x\+bf\(x\)=ax\+binto \([B\.1](https://arxiv.org/html/2608.14277#A2.E1)\), the left\-hand side isf⁡\(2​x\)\+2​f​\(y\)\\displaystyle f\(2x\)\+2f\(y\)=a⁡\(2​x\)\+b\+2​\(a​y\+b\)\\displaystyle=a\(2x\)\+b\+2\(ay\+b\)=2​a​\(x\+y\)\+3​b,\\displaystyle=2a\(x\+y\)\+3b,while the right\-hand side isf⁡\(f⁡\(x\+y\)\)\\displaystyle f\(f\(x\+y\)\)=f⁡\(a⁡\(x\+y\)\+b\)\\displaystyle=f\(a\(x\+y\)\+b\)=a2​\(x\+y\)\+a​b\+b\.\\displaystyle=a^\{2\}\(x\+y\)\+ab\+b\.Sincet=x\+yt=x\+yranges overℤ\\mathbb\{Z\}, for everyt∈ℤt\\in\\mathbb\{Z\}we have2​a​t\+3​b=a2​t\+a​b\+b\.2at\+3b=a^\{2\}t\+ab\+b\.Therefore,2​a−a2\\displaystyle 2a\-a^\{2\}=0,\\displaystyle=0,2​b−a​b\\displaystyle 2b\-ab=0,\\displaystyle=0,\(B\.8\)a⁡\(2−a\)\\displaystyle a\(2\-a\)=0,\\displaystyle=0,b⁡\(2−a\)\\displaystyle b\(2\-a\)=0\.\\displaystyle=0\.It follows thata=0a=0ora=2a=2\. Ifa=0a=0, then2​b=02b=0, sob=0b=0andf≡0f\\equiv 0\. Ifa=2a=2, the second condition is automatically satisfied for everyb∈ℤb\\in\\mathbb\{Z\}, givingf⁡\(x\)=2​x\+bf\(x\)=2x\+b\.Step 5\. Verification\.Forf≡0f\\equiv 0, both sides of \([B\.1](https://arxiv.org/html/2608.14277#A2.E1)\) are zero\. Forf⁡\(x\)=2​x\+bf\(x\)=2x\+b,f⁡\(2​x\)\+2​f​\(y\)\\displaystyle f\(2x\)\+2f\(y\)=\(4​x\+b\)\+2​\(2​y\+b\)\\displaystyle=\(4x\+b\)\+2\(2y\+b\)=4​x\+4​y\+3​b,\\displaystyle=4x\+4y\+3b,andf⁡\(f⁡\(x\+y\)\)\\displaystyle f\(f\(x\+y\)\)=f​\(2​\(x\+y\)\+b\)\\displaystyle=f\(2\(x\+y\)\+b\)=4​x\+4​y\+3​b\.\\displaystyle=4x\+4y\+3b\.Hence both families satisfy the equation, and the complete set of solutions isf≡0,orf\(x\)=2x\+c,c∈ℤ\.\\boxed\{\\begin\{aligned\} &f\\equiv 0,\\quad\\text\{or\}\\\\ &f\(x\)=2x\+c,\\qquad c\\in\\mathbb\{Z\}\.\\end\{aligned\}\}

### B\.2ProofBench\-Advanced\-028

QuestionLetHHbe the orthocenter of acute triangleA​B​CABC, letFFbe the foot of the altitude fromCCtoA​BAB, and letPPbe the reflection ofHHacrossB​CBC\. Suppose that the circumcircle of triangleA​F​PAFPintersects lineB​CBCat two distinct pointsXXandYY\. Prove thatCCis the midpoint ofX​YXY\.

Solution OverviewThe solution placesB​CBCon the horizontal axis and computes the coordinates ofFF,HH, andPP\. It then shows that the circumcenter of△​A​F​P\\triangle AFPlies on the vertical line throughCC\. Since this line is perpendicular to the chordX​Y⊂B​CXY\\subset BC, the standard chord\-bisection property of a circle implies thatCCis the midpoint ofX​YXY\.

Intern\-S2\-OPD SolutionSolution\.Step 1\. Coordinate setup\.We place the triangle in a convenient coordinate system\. Since the statement involves only midpoints, collinearity, and perpendicularity, we may apply a similarity transformation without loss of generality\. Thus, letB=\(0,0\),C=\(1,0\),A=\(p,q\),B=\(0,0\),\\qquad C=\(1,0\),\\qquad A=\(p,q\),\(B\.2\.1\)whereq\>0q\>0\. Since the triangle is acute,0<p<10<p<1, although these inequalities are not essential for the algebra below\. DefineD=p2\+q2\.D=p^\{2\}\+q^\{2\}\.Step 2\. Coordinates of the auxiliary points\.The lineA​BABpasses through the origin and has direction\(p,q\)\(p,q\)\. Therefore, the projection ofC=\(1,0\)C=\(1,0\)ontoA​BABisF=C⋅AA⋅A​A=pp2\+q2​\(p,q\)=\(p2D,p​qD\)\.F=\\frac\{C\\mathbin\{\\cdot\}A\}\{A\\mathbin\{\\cdot\}A\}A=\\frac\{p\}\{p^\{2\}\+q^\{2\}\}\(p,q\)=\\left\(\\frac\{p^\{2\}\}\{D\},\\frac\{pq\}\{D\}\\right\)\.\(B\.2\.2\)BecauseB​CBCis horizontal, the altitude fromAAis the vertical linex=px=p\. The altitude fromBBis perpendicular toA​CAC\. SinceA​C=\(1−p,−q\)AC=\(1\-p,\-q\), a perpendicular direction is\(q,1−p\)\(q,1\-p\), and hence this altitude has equationy=1−pq​x\.y=\\frac\{1\-p\}\{q\}x\.Intersecting it withx=px=pgivesH=\(p,p⁡\(1−p\)q\)\.H=\\left\(p,\\frac\{p\(1\-p\)\}\{q\}\\right\)\.\(B\.2\.3\)ReflectingHHacrossB​CBC, which is thexx\-axis, yieldsP=\(p,−p⁡\(1−p\)q\)\.P=\\left\(p,\-\\frac\{p\(1\-p\)\}\{q\}\\right\)\.\(B\.2\.4\)Step 3\. A candidate for the circumcenter of△​A​F​P\\triangle AFP\.The pointsAAandPPhave the samexx\-coordinate, soA​PAPis vertical\. Its perpendicular bisector is the horizontal line through the midpointMA​P\\displaystyle M\_\{AP\}=\(p,q−p⁡\(1−p\)q2\)\\displaystyle=\\left\(p,\\frac\{q\-\\frac\{p\(1\-p\)\}\{q\}\}\{2\}\\right\)=\(p,q2−p⁡\(1−p\)2​q\)\.\\displaystyle=\\left\(p,\\frac\{q^\{2\}\-p\(1\-p\)\}\{2q\}\\right\)\.Sinceq2−p⁡\(1−p\)=p2\+q2−p=D−p,q^\{2\}\-p\(1\-p\)=p^\{2\}\+q^\{2\}\-p=D\-p,definey0=D−p2​q\.y\_\{0\}=\\frac\{D\-p\}\{2q\}\.\(B\.2\.5\)The perpendicular bisector ofA​PAPis thereforey=y0y=y\_\{0\}\. We claim thatO=\(1,y0\)O=\(1,y\_\{0\}\)\(B\.2\.6\)is the circumcenter of△​A​F​P\\triangle AFP\. By construction,OOlies on the perpendicular bisector ofA​PAP\. It remains to show thatOOis equidistant fromAAandFF\.Step 4\. Verifying thatO​A=O​FOA=OF\.From equations \([B\.2\.1](https://arxiv.org/html/2608.14277#A2.E1a)\), \([B\.2\.2](https://arxiv.org/html/2608.14277#A2.E2a)\), and \([B\.2\.6](https://arxiv.org/html/2608.14277#A2.E6a)\),O​A2\\displaystyle OA^\{2\}=\(1−p\)2\+\(y0−q\)2,\\displaystyle=\(1\-p\)^\{2\}\+\(y\_\{0\}\-q\)^\{2\},O​F2\\displaystyle OF^\{2\}=\(1−p2D\)2\+\(y0−p​qD\)2\.\\displaystyle=\\left\(1\-\\frac\{p^\{2\}\}\{D\}\\right\)^\{2\}\+\\left\(y\_\{0\}\-\\frac\{pq\}\{D\}\\right\)^\{2\}\.\(B\.2\.7\)First observe that1−p2D=D−p2D=q2D\.1\-\\frac\{p^\{2\}\}\{D\}=\\frac\{D\-p^\{2\}\}\{D\}=\\frac\{q^\{2\}\}\{D\}\.\(B\.2\.8\)Consider the differenceΔ\\displaystyle\\Delta=O​A2−O​F2\\displaystyle=OA^\{2\}\-OF^\{2\}=\[\(1−p\)2−\(1−p2D\)2\]\+\[\(y0−q\)2−\(y0−p​qD\)2\]\.\\displaystyle=\\left\[\(1\-p\)^\{2\}\-\\left\(1\-\\frac\{p^\{2\}\}\{D\}\\right\)^\{2\}\\right\]\+\\left\[\(y\_\{0\}\-q\)^\{2\}\-\\left\(y\_\{0\}\-\\frac\{pq\}\{D\}\\right\)^\{2\}\\right\]\.\(B\.2\.9\)Factoring the first bracket as a difference of squares gives\(1−p\)2−\(1−p2D\)2\\displaystyle\(1\-p\)^\{2\}\-\\left\(1\-\\frac\{p^\{2\}\}\{D\}\\right\)^\{2\}=\(p2D−p\)​\(2−p−p2D\)\\displaystyle=\\left\(\\frac\{p^\{2\}\}\{D\}\-p\\right\)\\left\(2\-p\-\\frac\{p^\{2\}\}\{D\}\\right\)=−p⁡\(D−p\)D​\(2−p−p2D\)\.\\displaystyle=\-\\frac\{p\(D\-p\)\}\{D\}\\left\(2\-p\-\\frac\{p^\{2\}\}\{D\}\\right\)\.\(B\.2\.10\)Similarly,\(y0−q\)2−\(y0−p​qD\)2\\displaystyle\(y\_\{0\}\-q\)^\{2\}\-\\left\(y\_\{0\}\-\\frac\{pq\}\{D\}\\right\)^\{2\}=\(p​qD−q\)​\(2​y0−q−p​qD\)\\displaystyle=\\left\(\\frac\{pq\}\{D\}\-q\\right\)\\left\(2y\_\{0\}\-q\-\\frac\{pq\}\{D\}\\right\)=−q⁡\(D−p\)D​\(2​y0−q−p​qD\)\.\\displaystyle=\-\\frac\{q\(D\-p\)\}\{D\}\\left\(2y\_\{0\}\-q\-\\frac\{pq\}\{D\}\\right\)\.\(B\.2\.11\)Substituting equations \([B\.2\.10](https://arxiv.org/html/2608.14277#A2.E10)\) and \([B\.2\.11](https://arxiv.org/html/2608.14277#A2.E11)\) into equation \([B\.2\.9](https://arxiv.org/html/2608.14277#A2.E9)\), we obtainΔ=−D−pD\[\\displaystyle\\Delta=\-\\frac\{D\-p\}\{D\}\\Bigg\[p⁡\(2−p−p2D\)\\displaystyle p\\left\(2\-p\-\\frac\{p^\{2\}\}\{D\}\\right\)\+q\(2y0−q−p​qD\)\]\.\\displaystyle\+q\\left\(2y\_\{0\}\-q\-\\frac\{pq\}\{D\}\\right\)\\Bigg\]\.\(B\.2\.12\)Using equation \([B\.2\.5](https://arxiv.org/html/2608.14277#A2.E5a)\),q⁡\(2​y0−q−p​qD\)\\displaystyle q\\left\(2y\_\{0\}\-q\-\\frac\{pq\}\{D\}\\right\)=q⁡\(D−pq−q−p​qD\)\\displaystyle=q\\left\(\\frac\{D\-p\}\{q\}\-q\-\\frac\{pq\}\{D\}\\right\)=\(D−p\)−q2−p​q2D\.\\displaystyle=\(D\-p\)\-q^\{2\}\-\\frac\{pq^\{2\}\}\{D\}\.\(B\.2\.13\)Consequently, the bracket in equation \([B\.2\.12](https://arxiv.org/html/2608.14277#A2.E12)\) equalsp⁡\(2−p−p2D\)\+\(D−p\)−q2−p​q2D\\displaystyle p\\left\(2\-p\-\\frac\{p^\{2\}\}\{D\}\\right\)\+\(D\-p\)\-q^\{2\}\-\\frac\{pq^\{2\}\}\{D\}=2​p−p2−p3D\+D−p−q2−p​q2D\\displaystyle=2p\-p^\{2\}\-\\frac\{p^\{3\}\}\{D\}\+D\-p\-q^\{2\}\-\\frac\{pq^\{2\}\}\{D\}=p\+\(D−p2−q2\)−p⁡\(p2\+q2\)D\\displaystyle=p\+\\left\(D\-p^\{2\}\-q^\{2\}\\right\)\-\\frac\{p\(p^\{2\}\+q^\{2\}\)\}\{D\}=p\+0−p=0\.\\displaystyle=p\+0\-p=0\.It follows thatΔ=0\\Delta=0, and henceO​A2=O​F2OA^\{2\}=OF^\{2\}\. Therefore,OOlies on the perpendicular bisector ofA​FAFas well as that ofA​PAP, soOOis the circumcenter of△​A​F​P\\triangle AFP\.Step 5\. Bisecting the chordX​YXY\.The circumcenterO=\(1,y0\)O=\(1,y\_\{0\}\)and the pointC=\(1,0\)C=\(1,0\)have the samexx\-coordinate\. SinceB​CBCis thexx\-axis, the lineO​COCis perpendicular toB​CBC\. The chordX​YXYlies onB​CBC, and the perpendicular from the center of a circle to a chord bisects that chord\. Thus,CCis the midpoint ofX​YXY\.C​is the midpoint of​X​Y\.\\boxed\{C\\text\{ is the midpoint of \}XY\.\}

## Appendix CCase Study

This appendix presents two representative instances of direct OPD when the teacher model SU\-01 is distilled to Qwen3\.5\-35B\-A3B\. In both cases, the student degenerates into an endless repetition—either a fixed multi‑sentence self‑check block or a single‑token loop\. The rollouts terminate only when hitting the 160k‑token generation limit\.

### C\.1Case A: correct answer, then 972 exact self\-check repeats \(AIME25\)

The model solves an AIME 2025 divisibility problem, reaches the correct numerical answer279\\boxed\{279\}, but then appends the same ten‑sentence “verification” unit verbatim 972 times\. This loop occupies nearly the entire remaining generation budget and is truncated mid‑unit\.

QuestionDetermine the difference betweenNNand20252025, whereNNis the number of eight\-digit positive integers using each of the digits1,2,…,81,2,\\dots,8exactly once that are divisible by2222\.

Model Output \(excerpt\)Beginning of output\.We need to solve: there are8\!=403208\!=40320eight\-digit positive integers that use each of the digits1,2,3,4,5,6,7,81,2,3,4,5,6,7,8exactly once\. LetNNbe the number of these integers that are divisible by2222\. Find the difference betweenNNand20252025\. So we need to computeNN, then computeN−2025N\-2025\. Since22=2⋅1122=2\\cdot 11, we need divisibility by22and by1111\.At the failure onset \(response chars 75,000–77,150\)\.Consequently,N=144×16=2304N=144\\times 16=2304\. Finally,N−2025=2304−2025=279N\-2025=2304\-2025=279\. Thus the answer is279\\boxed\{279\}\. I’ll also ensure that the solution is not missing any step\.Repeating pattern — exact 10\-sentence self\-check unit \(542 characters\), repeated 972 times\.“I’ll also ensure that the solution is not missing any step\.” “I’ll also ensure that the solution is clear\.” “I’ll also ensure that the solution is correct\.” “I’ll also ensure that the solution is comprehensive\.” “I’ll also ensure that the solution is easy to follow\.” “I’ll also ensure that the solution is in the required format\.” “I’ll also ensure that the solution is not too long\.” “I’ll also ensure that the solution is self\-contained\.” “I’ll also ensure that the solution is not missing any justification\.” “I’ll also ensure that the solution is correct\.”The model reaches the correct answer279279, then repeats the same ten\-sentence self\-check unit exactly 972 times\. The rollout hits the 160k\-token generation limit inside this unit and is recorded as truncated; the response ends mid\-unit with “… not missing any justification\. I’ll also ensure that the”\.

### C\.2Case B: reasoning collapses into a one\-token loop \(AnswerBench\)

In this case, the model engages with a complex number‑theoretic definition \(“exotic integers”\)\. After some initial reasoning, it gets stuck on a sub‑problem involving multiplicative orders and begins emitting the token sequence “2?2^\{?\}” indefinitely\. The repetition is atomic \(one token\) and repeats over 34,000 times with no progress or termination\.

QuestionFor a positive integernn, callg:ℤ→ℤg\\colon\\mathbb\{Z\}\\to\\mathbb\{Z\}annn\-good function ifg⁡\(1\)=1g\(1\)=1and, for distinct integersaaandbb,g⁡\(a\)−g⁡\(b\)g\(a\)\-g\(b\)dividesan−bna^\{n\}\-b^\{n\}\. Callnnexotic if the number ofnn\-good functions is twice an odd integer\. Find the 132nd exotic integer\.

Model Output \(excerpt\)Beginning of output\.We need to solve: for positive integernn, defineg:ℤ→ℤg\\colon\\mathbb\{Z\}\\to\\mathbb\{Z\}such thatg⁡\(1\)=1g\(1\)=1and for any two distinct integersaaandbb,g⁡\(a\)−g⁡\(b\)g\(a\)\-g\(b\)dividesan−bna^\{n\}\-b^\{n\}\. We callnnan exotic integer if the number ofnn\-good functions is twice an odd integer\. Find the 132nd exotic integer\. We need to understand the condition and count the number of such functions for a givennn\.At the failure onset \(response tokens 840–919\)\.Form=8m=8:28−1=2552^\{8\}\-1=255, order of22mod255255?28=256≡1\(mod255\)2^\{8\}=256\\equiv 1\\pmod\{255\}, but24=16\(mod255\)2^\{4\}=16\\pmod\{255\}, not11; … but could be smaller?2?2^\{?\}Actually …2?2^\{?\}2?2^\{?\}2?2^\{?\}Repeating pattern — exact one\-token unit, repeated many times\.2?​2?​2?​2?​2?​2?​2?​2?​2?​2?⋯2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\;\\;2^\{?\}\\quad\\cdots\\quadThe identical token occupies one uninterrupted run; no new claim, calculation, or answer is produced after the loop begins\. The rollout reaches the 160k\-token generation limit inside this run and is recorded as truncated; the response ends with “…2?2^\{?\}2?2^\{?\}2?2^\{?\}2?2^\{?\}2?2^\{?\}”\.