SWE-Ladder：通过渐进式特权自蒸馏学习定位、复现与修补

# Learning to Localize, Reproduce, and Patch viaProgressive Privileged Self-Distillation
Source: [https://arxiv.org/html/2610.33667](https://arxiv.org/html/2610.33667)
## SWE\-Ladder: Learning to Localize, Reproduce, and Patch via Progressive Privileged Self\-Distillation

Jun Xiao Yueting Zhuang Yongliang ShenEmail:[\{ccdown, zhengxilu, syl\}@zju\.edu\.cn](mailto:)Affiliation:Zhejiang University

###### Abstract

Resolving a repository\-level issue typically requires a software\-engineering \(SWE\) agent to localize the faulty code, reproduce the failure, and patch the code\. Reinforcement learning trains such agents from a single test result on the final patch, while on\-policy self\-distillation \(OPSD\) adds token\-level supervision from a teacher with privileged evidence, exposing the full reference patch at every step\. Neither provides stage\-specific guidance for localization or reproduction\. We proposeSWE\-Ladder, which turns the verified repair of each training instance into stage\-specific evidence: the changed files and lines for localization, the fail\-to\-pass tests for reproduction, and the diff for patching\. A rule\-based router assigns each step of an end\-to\-end rollout to a stage based on its tool call, a frozen teacher sees only the matching evidence, and the stages are distilled in repair order while GRPO optimizes the full trajectory\. With Qwen3\-4B\-Instruct\-2507 on SWE\-bench Verified under the R2E\-Gym scaffold,SWE\-Ladderimproves pass@1 from9\.6%9\.6\\%to16\.4%16\.4\\%, compared with10\.4%10\.4\\%for GRPO and12\.4%12\.4\\%for OPSD with the full patch\. Under the OpenHands scaffold, it improves pass@1 from13\.0%13\.0\\%to17\.6%17\.6\\%\. Reversing the stage order under R2E\-Gym reduces pass@1 to11\.4%11\.4\\%\. Code is available at[https://github\.com/sekongger/SWE\-Ladder](https://github.com/sekongger/SWE-Ladder)\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.33667v1/figures/intro.png)Figure 1:Supervision signals compared in this paper\.GRPO assigns one reward to the whole trajectory; GRPO\+OPSD shows the teacher the full reference patch at every step;SWE\-Laddergives the teacher stage\-specific evidence and introduces the stages one at a time\.Figure 2:Stage\-wise diagnostics on SWE\-bench Verified\.Bars show changes from the base model at the final checkpoint; labels mark the changes discussed in Section[1](https://arxiv.org/html/2610.33667#S1)\. Base values appear under each metric\. Definitions are in Appendix[A\.1](https://arxiv.org/html/2610.33667#A1.SS1)\.Reinforcement learning with verifiable rewards \(RLVR\) has become an effective approach to training software\-engineering \(SWE\) agents\([Luo et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib7);[Golubev et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib8)\)\. The agent edits a repository over many tool calls, and the only training signal is whether its final patch passes the designated tests\([Jimenez et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib1);[Chowdhury et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib2)\)\. Over dozens of turns, it is difficult to identify which decision caused a failure\([Xia et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib18)\)\. On\-policy self\-distillation \(OPSD\) offers denser supervision\([Zhao et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib13);[Hübotter et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib30)\)\. The same model acts as a teacher that receives privileged evidence withheld from the student and scores every token of the student’s trajectory\([Lu and Thinking Machines Lab, 2025](https://arxiv.org/html/2610.33667#bib.bib24);[Agarwal et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib25)\)\. In SWE, the available evidence is the verified repair of each training instance, and the natural choice is to show the teacher the full reference patch at every step \(Figure[1](https://arxiv.org/html/2610.33667#S1.F1)\)\.

To see what these signals teach, we measure three intermediate behaviors \(Figure[2](https://arxiv.org/html/2610.33667#S1.F2)\)\. They record whether the agent edits the files changed by the reference repair, reproduces the failure with a script, and re\-runs the script successfully after its last edit\. We find two patterns\. First, the outcome reward does not consistently improve these behaviors: under GRPO, pass@1 barely moves \(from 9\.6% to 10\.4%\), while reproduction falls from 59\.4% to 48\.6%\. Second, showing the teacher the full patch raises pass@1 but lowers post\-edit verification from 22\.6% to 16\.2%\.

The full patch conflates the evidence for three different skills\. Localization needs to know where the fault is, reproduction needs to know how the failure shows up, and patching needs to know what to change\. During a localization step, a teacher that sees the full patch receives not only localization cues but also patch\-generation details, and may therefore favor editing over searching\. A similar effect has been reported in single\-turn reasoning, where a teacher conditioned on the full reference makes the student skip intermediate steps and verify less\([Zhang, 2026](https://arxiv.org/html/2610.33667#bib.bib31)\)\.*At each step, the teacher should therefore see only the part of the verified repair that the current stage needs\.*

The three skills support one another: localization identifies the code to investigate, reproduction makes the failure concrete, and the resulting test provides a basis for checking a patch\. We therefore organize privileged supervision into a progressive training schedule: localization, then reproduction, then patch generation\.

Following these observations, we proposeSWE\-Ladder, which trains a single end\-to\-end agent with progressive privileged self\-distillation\. Prior work obtains stage\-level supervision by fixing a localize\-then\-repair pipeline at inference time\([Xia et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib18);[Ma et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib20);[He et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib17)\), which limits how the agent can explore\.SWE\-Ladderleaves the agent’s actions unconstrained and decomposes only the teacher’s evidence\. A deterministic router labels each step as localization \(LOC\), reproduction \(REP\), or patch generation \(PATCH\) from its tool call\. The teacher then sees only the slice of the repair for that stage\. Training proceeds through the three stages in order, distilling one stage at a time, while GRPO optimizes the full trajectory\.

With Qwen3\-4B\-Instruct\-2507 on SWE\-bench Verified, the gain depends on how the teacher receives evidence\. Without privileged evidence, GRPO improves pass@1 from 9\.6% to 10\.4%\. With the full patch at every step, GRPO\+OPSD reaches 12\.4%, and training all three stages at once also reaches 12\.4%\. Introducing the stages in order,SWE\-Ladderreaches16\.4%and is the only method that improves both localization and verification without lowering reproduction \(Figure[2](https://arxiv.org/html/2610.33667#S1.F2)\)\. Reversing the order lowers pass@1 to 11\.4%, and on the OpenHands scaffoldSWE\-Ladderimproves pass@1 from 13\.0% to 17\.6%\.

Our contributions are threefold:

- •We show with stage\-wise diagnostics that outcome rewards do not consistently improve intermediate repair behavior and that a full\-patch teacher lowers post\-edit verification\.
- •We proposeSWE\-Ladder, which shows the teacher one stage’s slice of the verified repair at a time, following the order of repair\.
- •SWE\-Ladderoutperforms GRPO and full\-patch GRPO\+OPSD on two agent scaffolds, and removing any stage, reversing the order, or training all stages at once lowers pass@1\.

## 2Related Work

### 2\.1Post\-Training for SWE Agents

Recent datasets and environments expand the supply of executable repair tasks, trajectories, and verifiers for SWE training\([Pan et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib9);[Jain et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib10);[Yang et al\., 2026a](https://arxiv.org/html/2610.33667#bib.bib11);[Badertdinov et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib4);[Chen et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib14);[Sonwane et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib15)\)\. SWE\-RL applies GRPO with patch\-similarity rewards, DeepSWE uses verifier\-based RL, and long\-context SWE training combines rejection fine\-tuning with multi\-turn RL\([Wei et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib6);[Luo et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib7);[Golubev et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib8)\)\. Kimi\-Dev transfers an Agentless\-based skill prior to SWE\-Agent through lightweight SFT, without additional multi\-turn RL\([Yang et al\., 2026b](https://arxiv.org/html/2610.33667#bib.bib16)\)\. For RL\-based agents, feedback remains largely trajectory\-level, with limited supervision for intermediate tool decisions\.

### 2\.2On\-Policy Distillation for Agents

Agentic distillation methods regulate teacher guidance across multi\-turn trajectories\. SDAR gates an OPSD auxiliary loss with detached token statistics while retaining verifier\-driven RL; StepOPSD redistributes preference credit over action\-centered steps using hindsight\-enriched contexts\([Lu et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib26);[Zhang et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib27)\)\. SOD reweights OPD by step\-level divergence, while TCOD progressively expands the exposed trajectory depth\([Zhong et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib28);[Wang et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib29)\)\. Another line asks what the teacher should see: AR\-OPD replaces full\-reference imitation with anchored residual guidance, ATESD learns how much of the reference to reveal, and EviSD gives search and answer actions different evidence\([Zhang, 2026](https://arxiv.org/html/2610.33667#bib.bib31);[Han et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib34);[Xie et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib32)\)\. These methods keep the exposure fixed during training, whereasSWE\-Ladderorders the evidence across stages\.

### 2\.3Subtask\-Oriented Software Repair

Agentless organizes repair into localization, patch generation, and validation, while AutoCodeRover iteratively retrieves program structure before patching\([Xia et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib18);[Zhang et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib23)\)\. SWE\-Fixer follows a retrieve\-then\-edit workflow, and SoRFT makes file\-, function\-, and line\-level localization explicit before editing\([Xie et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib19);[Ma et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib20)\)\. For testing, SWT\-Bench evaluates reproduction through generated tests, while AssertFlip converts passing tests into fail\-to\-pass reproduction tests\([Mündler et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib21);[Khatib et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib22)\)\. SWE\-Swiss trains localization, repair, and test generation through multi\-task SFT followed by targeted RL\([He et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib17)\)\. These works make repair subgoals explicit and fix them into the inference pipeline or the training data\.SWE\-Ladderkeeps the agent end\-to\-end and uses this structure only to align privileged teacher evidence with subtask\-specific response tokens and progressively schedule distillation\. Curricula for LLM reinforcement learning usually order problems by difficulty\([Simonds and Yoshiyama, 2025](https://arxiv.org/html/2610.33667#bib.bib33)\);SWE\-Ladderkeeps each task fixed and orders the evidence that the teacher sees\.

## 3Method

Following the analysis in Section[1](https://arxiv.org/html/2610.33667#S1)and the staged organization of Agentless\([Xia et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib18)\), we organize end\-to\-end software repair into three training\-time subtasks: fault localization \(LOC\), failure reproduction \(REP\), and patch generation \(PATCH\)\. We collect a distinct source of task\-specific privileged evidence for each subtask\.SWE\-Ladderthen trains a single policy in the end\-to\-end repair setting while progressively targeting one subtask at a time with its corresponding privileged evidence \(Figure[3](https://arxiv.org/html/2610.33667#S3.F3)\)\. To target a subtask within an end\-to\-end rollout, we first assign each assistant step a repair\-stage label using its tool call\. At each training iteration, the schedule selects an active stage; its evidence conditions the teacher, and only response tokens labeled with that stage receive the OPSD loss\. GRPO continues to optimize all valid response tokens, so stage\-specific supervision is integrated into full\-trajectory learning\.

### 3\.1Rule\-Based Stage Routing

SWE\-Ladderuses a deterministic tool\-level state machine to assign each generated step one ofLOC,REP, orPATCH, in that order, while allowing state\-dependent backtracking\. The router labels response tokens and routes only the OPSD teacher context; it does not constrain tool choices, rollouts, or rewards\. Marker rules, backtracking, malformed\-tool handling, and token alignment are given in Appendix[A\.2](https://arxiv.org/html/2610.33667#A1.SS2)\. On held\-out stage\-labeled steps, the router agrees with the reference labels on 92\.0% of steps; a neural router reaches higher agreement but trains a worse agent \(Section[4\.4](https://arxiv.org/html/2610.33667#S4.SS4)\)\.

### 3\.2Stage\-Specific Privileged Evidence

We construct three evidence views from each training instance’s verified repair metadata\. We use only factual content from the repository, tests, and verified repair; collecting this evidence introduces no knowledge generated by external large language models\. Each view serves one subgoal:

- •Localization evidence\.We extract changed non\-test file paths and the old\-side hunk ranges from the verified repair, together with the corresponding pre\-patch source lines\. These clues identify where to investigate without revealing the replacement code\.
- •Reproduction evidence\.We collect fail\-to\-pass test identifiers and names, and include reference test code and candidate test changes when available\. This evidence describes the failing behavior and provides a starting point for constructing a reproduction script\.
- •Patch evidence\.We provide the verified repair diff, which specifies the fix\.

The evidence fields are stored with each instance, but teacher prompts are rendered at training time\. All three teacher variants share the student’s system instructions and end\-to\-end task prompt; only an appended evidence block differs\. Appendix[B](https://arxiv.org/html/2610.33667#A2)shows a routed trajectory, the evidence fields for each stage, and their token cost\. The active training phase selects which evidence view is used for distillation\.

![Refer to caption](https://arxiv.org/html/2610.33667v1/figures/main_figure.png)Figure 3:Stage\-routed privileged distillation for SWE\.A rule\-based router labels each step of an end\-to\-end rollout as LOC, REP, or PATCH\. In each phase, a frozen teacher receives one stage’s evidence and distills only that stage’s tokens, while GRPO optimizes the full trajectory\.
### 3\.3Full\-Trajectory GRPO Objective

We denote the standard GRPO objective\([Shao et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib12)\)for rolloutggat training iterationuubyℒGRPOu,g​\(θ\)\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}^\{u,g\}\(\\theta\)\. It acts on all valid model\-generated response tokens throughout the end\-to\-end trajectory, regardless of the active training stage\. The stage schedule changes only the auxiliary OPSD term defined below; the GRPO loss and its token coverage remain unchanged\. In particular, stage labels are annotations for distillation, not early\-stopping conditions: a rollout is never truncated at a stage boundary and always runs to normal termination or the configured resource limit\.

### 3\.4Stage\-Conditioned OPSD and Joint Objective

For a fixed training iterationuuand rolloutgg, we suppress\(u,g\)\(u,g\)in token\-level quantities and per\-rollout losses\. The active training stagea⁡\(u\)a\(u\), defined in Section[3\.5](https://arxiv.org/html/2610.33667#S3.SS5), selects the teacher’s privileged context, soct=ca⁡\(u\)c\_\{t\}=c\_\{a\(u\)\}throughout the rollout\. The evidence views are described in Section[3\.2](https://arxiv.org/html/2610.33667#S3.SS2)and Appendix[B](https://arxiv.org/html/2610.33667#A2)\. Letzt∈\{LOC,REP,PATCH\}z\_\{t\}\\in\\\{\\mathrm\{LOC\},\\mathrm\{REP\},\\mathrm\{PATCH\}\\\}denote the stage label at token positiontt, assigned by the step\-level router in Section[3\.1](https://arxiv.org/html/2610.33667#S3.SS1)\.

The teacherqϕ0q\_\{\\phi\_\{0\}\}is initialized from the student’s base checkpoint and remains frozen throughout training\. Lethth\_\{t\}denote the student\-sampled interaction prefix before tokentt, including earlier tool observations\. We abbreviate the student and teacher distributions aspt​\(v\)=pθ​\(v∣ht\)p\_\{t\}\(v\)=p\_\{\\theta\}\(v\\mid h\_\{t\}\)andqt​\(v\)=qϕ0​\(v∣ht,ct\)q\_\{t\}\(v\)=q\_\{\\phi\_\{0\}\}\(v\\mid h\_\{t\},c\_\{t\}\), respectively, with the dependence ofptp\_\{t\}onθ\\thetaimplicit\. The teacher selects itsKKhighest\-probability vocabulary tokens:

𝒦t=TopKv⁡qt​\(v\)\.\\mathcal\{K\}\_\{t\}=\\operatorname\{TopK\}\_\{v\}\\,q\_\{t\}\(v\)\.\(1\)Its distribution is then normalized over this set:

q~t​\(v\)=qt​\(v\)∑w∈𝒦tqt​\(w\),v∈𝒦t\.\\widetilde\{q\}\_\{t\}\(v\)=\\frac\{q\_\{t\}\(v\)\}\{\\sum\_\{w\\in\\mathcal\{K\}\_\{t\}\}q\_\{t\}\(w\)\},\\qquad v\\in\\mathcal\{K\}\_\{t\}\.\(2\)The student probabilities retain their full\-vocabulary normalization\. We use the following stop\-gradient top\-KKsurrogate:

dt​\(θ\)\\displaystyle d\_\{t\}\(\\theta\)=∑v∈𝒦tpt​\(v\)\\displaystyle=\\sum\_\{v\\in\\mathcal\{K\}\_\{t\}\}p\_\{t\}\(v\)\(3\)×\[sg⁡\(log⁡pt​\(v\)\)−sg⁡\(log⁡q~t​\(v\)\)\]\.\\displaystyle\}\{\\displaystyle\\times\\bigl\[\\operatorname\{sg\}\(\\log p\_\{t\}\(v\)\)\-\\operatorname\{sg\}\(\\log\\widetilde\{q\}\_\{t\}\(v\)\)\\bigr\]\.Here,sg\\operatorname\{sg\}denotes stop\-gradient: both log terms are detached, while gradients flow through the outer student probability factorpt​\(v\)p\_\{t\}\(v\)\. Implementation details are provided in Appendix[A\.6](https://arxiv.org/html/2610.33667#A1.SS6)\.

To restrict distillation to the active subtask, letmtrespm\_\{t\}^\{\\mathrm\{resp\}\}equal one at valid model\-generated response positions and zero elsewhere\. The OPSD mask and normalizer are

mtOPSD\\displaystyle m\_\{t\}^\{\\mathrm\{OPSD\}\}=mtresp1\{zt=a\(u\)\},\\displaystyle=m\_\{t\}^\{\\mathrm\{resp\}\}\\,\\mathbf\{1\}\\\{z\_\{t\}=a\(u\)\\\},\(4\)NOPSD\\displaystyle N^\{\\mathrm\{OPSD\}\}=max⁡\{1,∑tmtOPSD\}\.\\displaystyle=\\max\\bigl\\\{1,\\textstyle\\sum\_\{t\}m\_\{t\}^\{\\mathrm\{OPSD\}\}\\bigr\\\}\.The per\-rollout distillation loss and joint objective are

ℒOPSD​\(θ\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{OPSD\}\}\(\\theta\)=1NOPSD​∑tmtOPSD​dt​\(θ\),\\displaystyle=\\frac\{1\}\{N^\{\\mathrm\{OPSD\}\}\}\\sum\_\{t\}m\_\{t\}^\{\\mathrm\{OPSD\}\}d\_\{t\}\(\\theta\),\(5\)ℒ⁡\(θ\)\\displaystyle\\mathcal\{L\}\(\\theta\)=ℒGRPO​\(θ\)\+λ​ℒOPSD​\(θ\),\\displaystyle=\\mathcal\{L\}\_\{\\mathrm\{GRPO\}\}\(\\theta\)\+\\lambda\\mathcal\{L\}\_\{\\mathrm\{OPSD\}\}\(\\theta\),whereλ\\lambdacontrols the distillation weight\. GRPO remains active on all valid response tokens, including those outside the active subtask\.

### 3\.5Progressive Privilege Scheduling

SWE\-Ladderapplies privileged stages in the fixed orderLOC→REP→PATCH\\mathrm\{LOC\}\\rightarrow\\mathrm\{REP\}\\rightarrow\\mathrm\{PATCH\}, with each phase continuing from the preceding checkpoint\. The active stagea⁡\(u\)a\(u\)selects both the teacher’s privileged evidence and the response tokens used for OPSD via Eq\. \([4](https://arxiv.org/html/2610.33667#S3.E4)\)\. The order follows how repair uses the evidence\. On\-policy distillation supervises only the states that the student reaches, so patch evidence helps only once the student can find the code to change, and a reproduction script is what the student later runs to check its patch\. Section[4\.4](https://arxiv.org/html/2610.33667#S4.SS4)compares alternative orders\. The formal schedule, phase budgets, and training and inference details are given in Appendix[A\.7](https://arxiv.org/html/2610.33667#A1.SS7)\.

## 4Experiments

### 4\.1Experimental Setup

Table 1:Training data composition\.The collection contains 1,149 issues from four software\-repair corpora\.Benchmark and protocol\.We evaluate on SWE\-bench Verified , which contains500500human\-validated repository issues, and report pass@1\([Jimenez et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib1);[Chowdhury et al\., 2024](https://arxiv.org/html/2610.33667#bib.bib2)\)\. An issue is counted as solved only when the submitted patch passes all fail\-to\-pass and pass\-to\-pass unit tests\. The primary RL comparisons use the same issues, R2E\-Gym\([Jain et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib10)\)agent scaffold, sandbox, and evaluation harness\.

Model and training\.We use the Qwen3\-4B\-Instruct\-2507 policy backbone\([Yang et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib3)\)\. There is no additional task\-specific supervised fine\-tuning before RL\.SWE\-Ladderuses a frozen teacher initialized from the same base checkpoint as the student\. Each training iteration uses a batch size of1616tasks and a group size of44rollouts per task, yielding6464rollouts per iteration\. We use a teacher top\-set ofK=16K=16tokens, an entropy\-bonus coefficient of0\.0050\.005, andλ=0\.15\\lambda=0\.15for the OPSD term\. The training collection contains1,1491\{,\}149issues from four software\-repair corpora; their composition is summarized in Table[1](https://arxiv.org/html/2610.33667#S4.T1)\.

Progressive training\.SWE\-Ladderuses the fixed LOC→\\rightarrowREP→\\rightarrowPATCH schedule with3030,2020, and2020global iterations, respectively\. Every rollout remains end to end through normal termination or the configured resource limit\. The rule\-based router assigns stages to assistant steps; OPSD selects only the response tokens assigned to the active phase, while GRPO retains all valid response tokens\. Detailed scheduling and training configuration are given in Appendix[A\.7](https://arxiv.org/html/2610.33667#A1.SS7)\.

Comparison scope\.Our primary baselines are GRPO and GRPO\+OPSD, which exposes the reference patch at every step without stage\-specific routing\. We additionally include SDAR\([Lu et al\., 2026](https://arxiv.org/html/2610.33667#bib.bib26)\), a gated OPSD method for agents, with the same patch evidence\. All methods start from the same base model and are evaluated with the same scaffold and harness; training curves cover a common 70\-iteration horizon \(Appendix[C](https://arxiv.org/html/2610.33667#A3)\)\.

### 4\.2Main Results

#### SWE\-Ladderimproves end\-to\-end repair on both scaffolds\.

On R2E\-Gym,SWE\-Ladderraises pass@1 from 9\.6% to 16\.4%, compared with 10\.4% for GRPO and 12\.4% for GRPO\+OPSD \(Table[2](https://arxiv.org/html/2610.33667#S4.T2)\), without task\-specific supervised fine\-tuning\. It also uses fewer steps than the base model \(19\.3 vs\. 26\.5\) and completes 98\.6% of its trajectories, whereas GRPO completes only 36\.0% before the resource limit\. With the same patch evidence, SDAR falls below the base model \(4\.6%\)\.

Note\.Colored arrows show absolute changes from the Qwen3 base within each scaffold: percentage points for pass@1 andenv\_done, and steps for Avg\. steps\. For Avg\. steps, a downward arrow is favorable\. All OpenHands rows use the common evaluation protocol in Appendix[A\.4](https://arxiv.org/html/2610.33667#A1.SS4)\.

Table 2:SWE\-bench Verified results\.We report results for both the R2E\-Gym and OpenHands\([Wang et al\., 2025](https://arxiv.org/html/2610.33667#bib.bib5)\)scaffolds\. Each configuration is evaluated in our environment with the scaffold listed in the table\. All reported baselines andSWE\-Laddervariants are evaluated in the same environment and protocol\.On the OpenHands scaffold, the three baselines stay at or below the base model: the base policy reaches13\.0%13\.0\\%pass@1, and GRPO, GRPO\+OPSD, and SDAR reach11\.2%11\.2\\%,12\.4%12\.4\\%, and6\.8%6\.8\\%, respectively, whereasSWE\-Ladderfrom Base reaches17\.6%17\.6\\%with16\.616\.6steps and99\.4%99\.4\\%completion\.

Figure 4:Four\-view training dynamics on SWE\-bench Verified\.Localization, reproduction, verification, and pass@1, evaluated every 10 training iterations, for SWE\-Ladder, SDAR, GRPO\+OPSD, and the all\-stages\-at\-once variant\. Vertical guides mark the LOC, REP, and PATCH phases ofSWE\-Ladder\.Figure[4](https://arxiv.org/html/2610.33667#S4.F4)tracks the four metrics during training\. The localization ofSWE\-Ladderrises from 33\.6% to 52\.0%\. Its reproduction and verification dip during the REP phase and recover during the PATCH phase, ending above the base model\. GRPO\+OPSD raises reproduction but lowers verification, SDAR and the all\-stages\-at\-once variant fall below the base model on both, and onlySWE\-Ladderends above it on all four metrics\.

### 4\.3Training Dynamics

Figure 5:Training\-dynamics diagnostics\.Entropy, gradient norm, response length, and shaped training reward of GRPO, GRPO\+OPSD, SDAR, andSWE\-Ladderover 70 iterations\. Shaded regions mark the LOC, REP, and PATCH phases ofSWE\-Ladder\.To investigate training stability, we compareSWE\-Ladderwith GRPO, GRPO\+OPSD, and SDAR\. Figure[5](https://arxiv.org/html/2610.33667#S4.F5)tracks policy entropy, gradient norm, response length, and shaped training reward across individual training runs \(Appendix[C\.1](https://arxiv.org/html/2610.33667#A3.SS1)\)\.

GRPO exhibits substantial deterioration in its training dynamics\. Comparing the first and last ten iterations, its mean entropy decreases from0\.03000\.0300to0\.00610\.0061, while its gradient norm drops from1\.8201\.820to0\.1870\.187\. Meanwhile, the average response length nearly doubles from8\.298\.29k to16\.7316\.73k tokens, and training reward declines\.

In contrast,SWE\-Ladderexhibits more consistent optimization dynamics, with relatively stable entropy \(0\.0251→0\.02210\.0251\\rightarrow 0\.0221\) and gradient norm \(2\.353→2\.3972\.353\\rightarrow 2\.397\), alongside a modest increase in response length \(10\.0910\.09k to11\.6211\.62k tokens\)\. Although its training reward remains variable,SWE\-Ladderdoes not exhibit the pronounced diagnostic deterioration observed under GRPO\.

GRPO\+OPSD and SDAR also retain optimization signals late in training\. Thus, maintaining entropy or gradient magnitude alone does not explainSWE\-Ladder’s superior held\-out performance\. These observations suggest more consistent optimization behavior under the tested configuration, without establishing a general stability guarantee\.

### 4\.4Ablation Study

Figure 6:Action counts per turn across checkpoints\.For the base model,LOC30,REP50, andPATCH70, each panel stacks the number of trajectories taking a search/read/view, file\-edit, test/reproduction, or other action at each turn on SWE\-bench Verified \(n=500n=500\)\.![Refer to caption](https://arxiv.org/html/2610.33667v1/checkpoint_capability_heatmap.png)Figure 7:Capability changes across checkpoints\.Cells show changes \(%\) fromBasefor localization \(Loc\.\), reproduction \(Rep\.\), verification \(Verify\.\), and pass@1 atBase,LOC30,REP50, andPATCH70\. Darker blue indicates a larger signed change\.We organize the ablations around three questions: whether all three training stages contribute, whether a deterministic router is preferable to a neural router for training, and whether the privileged information is matched to the stage whose response tokens it supervises\.

#### All three stages contribute to capability gains\.

We remove each stage from the progressive schedule in turn and compare with a variant that trains all three stages jointly\. In this variant, the LOC, REP, and PATCH teacher contexts are available concurrently, and each routed response\-token subset is distilled from its corresponding context; it does not use the progressive continuation schedule\. We refer to it as*all stages at once*\. The variants share the Qwen3\-4B\-Instruct\-2507 base and are evaluated with the same R2E\-Gym scaffold and protocol\.

Table 3:Stage ablation on SWE\-bench Verified\.Qwen3\-4B\-Instruct\-2507 with R2E\-Gym,n=500n=500\. We report Loc\., Rep\., Verify\., and pass@1; arrows indicate changes relative to Base\. “All at once” applies the LOC, REP, and PATCH teacher contexts concurrently, with each routed response\-token subset distilled from its corresponding context\.Table[3](https://arxiv.org/html/2610.33667#S4.T3)shows that removing LOC, REP, or PATCH reduces pass@1 from16\.4%16\.4\\%to11\.2%11\.2\\%,13\.2%13\.2\\%, or14\.0%14\.0\\%, respectively, and every variant falls below the base model on reproduction and verification\. Removing LOC causes the largest drop, consistent with localization being the stage on which the others depend\. Training all stages at once reaches 12\.4% and also loses reproduction \(35\.6%\) and verification \(12\.6%\), so routing the evidence without the order recovers neither pass@1 nor the intermediate behaviors\. The REP phase does not raise the behavior metrics by itself \(Section[4\.5](https://arxiv.org/html/2610.33667#S4.SS5)\), but removing it still lowers the final pass@1\.

#### Stage order matters\.

We examine whether the order of privileged supervision affects training outcomes by comparing the proposed LOC→\\rightarrowREP→\\rightarrowPATCH schedule with PATCH→\\rightarrowREP→\\rightarrowLOC and REP→\\rightarrowLOC→\\rightarrowPATCH\. All schedules allocate 30 iterations to LOC and 20 each to REP and PATCH\.

As shown in Table[4](https://arxiv.org/html/2610.33667#S4.T4), the proposed order achieves16\.4%16\.4\\%pass@1, outperforming the reverse order \(11\.4%11\.4\\%\) and the REP\-first order \(12\.6%12\.6\\%\)\. It also requires fewer interaction steps and achieves a higher environment completion rate\. Notably, REP→\\rightarrowLOC→\\rightarrowPATCH retains PATCH as the final phase but still underperforms the proposed schedule, highlighting the importance of the preceding stage order\. These results support introducing localization before reproduction, followed by patch generation\.

Table 4:Effect of training\-stage order\.All schedules use 70 training iterations: 30 for LOC and 20 each for REP and PATCH\. The blue row is the reference\. Arrows show changes in percentage points for Solve and env\_done, and in steps for Avg\. steps\.
#### Rule\-based routing is more effective for training\.

The neural router achieves higher agreement with the stage annotations than the rule\-based router \(98\.63%98\.63\\%vs\.92\.0%92\.0\\%\), yet yields lower downstream pass@1\. This gap may reflect more fragmented stage assignments under independent step prediction, compared with stateful rule\-based routing\. Annotation agreement alone therefore does not determine training utility\. Details are provided in Appendix[A\.3](https://arxiv.org/html/2610.33667#A1.SS3)\.

#### Privilege should match the target stage\.

We compare matched and swapped LOC/PATCH evidence under a 30\-iteration budget \(Table[5](https://arxiv.org/html/2610.33667#S4.T5)\)\. For LOC targets, matched LOC PI achieves14\.4%14\.4\\%pass@1, compared with13\.6%13\.6\\%when using PATCH PI, indicating that patch evidence retains some localization cues\. For PATCH targets, matched PATCH PI reaches13\.2%13\.2\\%, compared with11\.8%11\.8\\%for LOC PI, showing that localization evidence alone exposes too little information for patch generation\. Although the matched PATCH condition uses longer trajectories and has lowerenv\_donethan the mismatched condition, it achieves higher repair success, indicating that shorter interactions alone do not imply better repairs\.

Table 5:Privilege–target stage alignment\.All variants use a 30\-step training budget\. Rows are grouped by target stage; the aligned row in each group is the reference, and arrows show absolute changes for the mismatched PI condition\.

### 4\.5Analysis

To examine how behavior changes along the progressive sequence, we use four held\-out evaluation snapshots and refer to them consistently asBase,LOC30,REP50, andPATCH70\. Figure[6](https://arxiv.org/html/2610.33667#S4.F6)provides a complementary view of the same checkpoints from their audited trajectories\. The stacks count search/read/view, file\-edit, test/reproduction, and other/environment actions at each event turn; they are counts rather than normalized rates\. The action taxonomy comes from the collected records and is not a stage\-router annotation\.

Figure[7](https://arxiv.org/html/2610.33667#S4.F7)shows how the four metrics change along the progressive sequence, at the end of each phase \(LOC30,REP50, andPATCH70\)\. The LOC phase improves all four metrics\. The REP phase further improves localization, but reproduction and verification fall below the base model\. The PATCH phase recovers both, andPATCH70exceeds the base model on all four metrics\.

## 5Conclusion

We studied what a privileged teacher should see when training an end\-to\-end SWE agent\.SWE\-Laddergives the teacher only the evidence for the current stage and introduces the stages in repair order\. It improves pass@1 on SWE\-bench Verified from 9\.6% to 16\.4% with R2E\-Gym and from 13\.0% to 17\.6% with OpenHands, and ablations show that both stage\-matched evidence and the stage order matter\.

## Limitations

The evidence is bounded to one Qwen3\-4B policy, the SWE\-bench Verified benchmark, and two agent scaffolds\. Although the OpenHands results provide a second scaffold check, the main ablations and the stage\-wise diagnostics use the R2E\-Gym protocol\. We therefore do not establish transfer to larger backbones, other repair benchmarks, or substantially different tool interfaces\. The reported training curves and router comparison each follow one checkpoint lineage without independent random\-seed repeats or confidence intervals, so the observed optimization trends and downstream differences should be interpreted as configuration\-specific evidence\.

The method also depends on verified repair metadata to construct the three privileged views and on a parser that maps tool calls to repair stages\. The neural\-router accuracy is agreement with Qwen3\.8\-Max annotations rather than an independently adjudicated semantic label, while the rule\-based router can miss unfamiliar tools or atypical backtracking patterns\. Future work should test neural and hybrid routers across models and benchmarks, repeat the training with independent seeds, and quantify the cost and robustness of evidence construction\.

## Ethics Statement

This work uses publicly available SWE\-bench Verified issues, open\-weight Qwen3 models, the hosted Qwen3\.8\-Max model for rollout stage annotation, and public R2E\-Gym and OpenHands scaffolds\. We do not recruit human participants or collect new personal data\. Training and evaluation are conducted in isolated benchmark sandboxes and do not take actions on external systems\. Privileged repair evidence is used only as training\-time context and is absent at inference\. Deploying agents trained withSWE\-Ladderbeyond these benchmark sandboxes would require additional safety validation, permission controls, and human oversight\.

## References

- Agarwalet al\.\(2024\)R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos Garea, M\. Geist, and O\. BachemOn\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 21246–21263\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1)\.
- Badertdinovet al\.\(2026\)I\. Badertdinov, A\. Golubev, M\. Nekrashevich, A\. Shevtsov, S\. Karasik, A\. Andriushchenko, M\. Trofimova, D\. Litvintseva, and B\. YangelSwe\-rebench: an automated pipeline for task collection and decontaminated evaluation of software engineering agents\.Advances in Neural Information Processing Systems38\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.33667#S4.T1.2.1.3.1)\.
- Chenet al\.\(2026\)M\. Chen, L\. Zhang, Y\. Feng, X\. Wang, W\. Zhao, R\. Cao, J\. Yang, J\. Chen, M\. Li, Z\. Ma,et al\.Swe\-universe: scale real\-world verifiable environments to millions\.arXiv preprint arXiv:2602\.02361\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1)\.
- Chowdhuryet al\.\(2024\)N\. Chowdhury, J\. Aung, J\. S\. Chan, O\. Jaffe, D\. Sherburn, G\. Starace, E\. Mays, R\. Dias, M\. Aljubeh, M\. Glaese, C\. E\. Jimenez, J\. Yang, L\. Ho, T\. Patwardhan, K\. Liu, and A\. MadryIntroducing SWE\-bench verified\.Note:OpenAI BlogPublished August 13, 2024; updated February 24, 2025External Links:[Link](https://openai.com/index/introducing-swe-bench-verified/)Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1),[§4\.1](https://arxiv.org/html/2610.33667#S4.SS1.p1.1)\.
- Golubevet al\.\(2025\)A\. Golubev, M\. Trofimova, S\. Polezhaev, I\. Badertdinov, M\. Nekrashevich, A\. Shevtsov, S\. Karasik, S\. Abramov, A\. Andriushchenko, F\. Fisin,et al\.Training long\-context, multi\-turn software engineering agents with reinforcement learning\.arXiv preprint arXiv:2508\.03501\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1)\.
- Hanet al\.\(2026\)Z\. Han, T\. Zhang, H\. Wang, and Y\. SunAdaptive teacher exposure for self\-distillation in llm reasoning\.arXiv preprint arXiv:2605\.11458\.Cited by:[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1)\.
- Heet al\.\(2026\)Z\. He, Q\. Yang, W\. Shen, X\. Zhong, K\. Zhang, C\. An, W\. Shi, T\. Cai, D\. He, J\. Chen,et al\.Swe\-swiss: a multi\-task fine\-tuning and rl recipe for high\-performance issue resolution\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 16102–16114\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p5.1),[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Hübotteret al\.\(2026\)J\. Hübotter, F\. Lübeck, L\. Behric, A\. Baumann, M\. Bagatella, D\. Marta, I\. Hakimi, I\. Shenfeld, T\. K\. Buening, C\. Guestrin,et al\.Reinforcement learning via self\-distillation\.arXiv preprint arXiv:2601\.20802\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1)\.
- Jainet al\.\(2025\)N\. Jain, J\. Singh, M\. Shetty, L\. Zheng, K\. Sen, and I\. StoicaR2e\-gym: procedural environments and hybrid verifiers for scaling open\-weights swe agents\.arXiv preprint arXiv:2504\.07164\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.33667#S4.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.33667#S4.T1.2.1.4.1)\.
- Jimenezet al\.\(2024\)C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. NarasimhanSwe\-bench: can language models resolve real\-world github issues?\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 54107–54157\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1),[§4\.1](https://arxiv.org/html/2610.33667#S4.SS1.p1.1)\.
- Khatibet al\.\(2026\)L\. Khatib, N\. S\. Mathews, and M\. NagappanAssertflip: reproducing bugs via inversion of llm\-generated passing tests\.InProceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering,pp\. 3825–3835\.Cited by:[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Lu and Thinking Machines Lab \(2025\)K\. Lu and Thinking Machines LabOn\-policy distillation\.Thinking Machines Lab: Connectionism\.External Links:[Document](https://dx.doi.org/10.64434/tml.20251026),[Link](https://thinkingmachines.ai/blog/on-policy-distillation/)Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1)\.
- Luet al\.\(2026\)Z\. Lu, Z\. Yao, Z\. Han, Z\. Wang, J\. Wu, Q\. Gu, X\. Cai, W\. Lu, J\. Xiao, Y\. Zhuang,et al\.Self\-distilled agentic reinforcement learning\.arXiv preprint arXiv:2605\.15155\.Cited by:[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.33667#S4.SS1.p4.1)\.
- Luoet al\.\(2025\)M\. Luo, N\. Jain, J\. Singh, S\. Tan, A\. Patel, Q\. Wu, A\. Ariyak, C\. Cai, T\. Venkat, S\. Zhu, B\. Athiwaratkun, M\. Roongta, C\. Zhang, L\. E\. Li, R\. A\. Popa, K\. Sen, and I\. StoicaDeepSWE: training a fully open\-sourced, state\-of\-the\-art coding agent by scaling RL\.Note:Together AI BlogPublished July 2, 2025External Links:[Link](https://www.together.ai/blog/deepswe)Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1)\.
- Maet al\.\(2025\)Z\. Ma, C\. Peng, P\. Gao, X\. Meng, Y\. Zou, and B\. XieSorft: issue resolving with subtask\-oriented reinforced fine\-tuning\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 11427–11441\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p5.1),[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Mündleret al\.\(2024\)N\. Mündler, M\. N\. Müller, J\. He, and M\. VechevSwt\-bench: testing and validating real\-world bug\-fixes with code agents\.Advances in Neural Information Processing Systems37,pp\. 81857–81887\.Cited by:[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Panet al\.\(2024\)J\. Pan, X\. Wang, G\. Neubig, N\. Jaitly, H\. Ji, A\. Suhr, and Y\. ZhangTraining software engineering agents and verifiers with swe\-gym\.arXiv preprint arXiv:2412\.21139\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.33667#S4.T1.2.1.5.1)\.
- Qwen Team \(2026\)Qwen TeamQwen3\.8\-Max: a new bar for coding and cowork\.Note:Qwen BlogExternal Links:[Link](https://qwen.ai/blog?id=qwen3.8)Cited by:[§A\.3](https://arxiv.org/html/2610.33667#A1.SS3.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§3\.3](https://arxiv.org/html/2610.33667#S3.SS3.p1.1)\.
- Simonds and Yoshiyama \(2025\)T\. Simonds and A\. YoshiyamaLadder: self\-improving llms through recursive problem decomposition\.arXiv preprint arXiv:2503\.00735\.Cited by:[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Sonwaneet al\.\(2025\)A\. Sonwane, I\. White, H\. Lee, M\. Pereira, L\. Caccia, M\. Kim, Z\. Shi, C\. Singh, A\. Sordoni, M\. Côté,et al\.Bugpilot: complex bug generation for efficient learning of swe skills\.arXiv preprint arXiv:2510\.19898\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1)\.
- Wanget al\.\(2026\)J\. Wang, W\. Zhang, W\. Shi, Y\. Li, and J\. ChengTcod: exploring temporal curriculum in on\-policy distillation for multi\-turn autonomous agents\.arXiv preprint arXiv:2604\.24005\.Cited by:[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1)\.
- Wanget al\.\(2025\)X\. Wang, B\. Li, Y\. Song, F\. F\. Xu, X\. Tang, M\. Zhuge, J\. Pan, Y\. Song, B\. Li, J\. Singh,et al\.Openhands: an open platform for ai software developers as generalist agents\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 65882–65919\.Cited by:[Table 2](https://arxiv.org/html/2610.33667#S4.T2)\.
- Weiet al\.\(2026\)Y\. Wei, O\. Duchenne, J\. Copet, Q\. Carbonneaux, L\. Zhang, D\. Fried, G\. Synnaeve, R\. Singh, and S\. WangSwe\-rl: advancing llm reasoning via reinforcement learning on open software evolution\.Advances in Neural Information Processing Systems38,pp\. 78500–78525\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1)\.
- Xiaet al\.\(2024\)C\. S\. Xia, Y\. Deng, S\. Dunn, and L\. ZhangAgentless: demystifying llm\-based software engineering agents\.arXiv preprint arXiv:2407\.01489\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1),[§1](https://arxiv.org/html/2610.33667#S1.p5.1),[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1),[§3](https://arxiv.org/html/2610.33667#S3.p1.1)\.
- Xieet al\.\(2025\)C\. Xie, B\. Li, C\. Gao, H\. Du, W\. Lam, D\. Zou, and K\. ChenSwe\-fixer: training open\-source llms for effective and efficient github issue resolution\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 1123–1139\.Cited by:[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Xieet al\.\(2026\)J\. Xie, X\. Sun, Z\. Chen, X\. Zheng, S\. Wu, B\. Song, and L\. WangEviSD: evidence\-conditioned self\-distillation for search\-augmented agents\.arXiv preprint arXiv:2608\.01359\.Cited by:[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui,et al\.Qwen3 technical report\.External Links:2505\.09388,[Document](https://dx.doi.org/10.48550/arXiv.2505.09388),[Link](http://arxiv.org/abs/2505.09388)Cited by:[§4\.1](https://arxiv.org/html/2610.33667#S4.SS1.p2.1)\.
- Yanget al\.\(2026a\)J\. Yang, K\. Lieret, C\. Jimenez, A\. Wettig, K\. Khandpur, Y\. Zhang, B\. Hui, O\. Press, L\. Schmidt, and D\. YangSwe\-smith: scaling data for software engineering agents\.Advances in Neural Information Processing Systems38\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.33667#S4.T1.2.1.2.1)\.
- Yanget al\.\(2026b\)Z\. Yang, S\. Wang, K\. Fu, W\. He, W\. Xiong, Y\. Liu, Y\. Miao, B\. Gao, Y\. Wang, Y\. Li,et al\.Kimi\-dev: agentless training as skill prior for swe\-agents\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 148369–148436\.Cited by:[§2\.1](https://arxiv.org/html/2610.33667#S2.SS1.p1.1)\.
- Zhang \(2026\)W\. ZhangBeyond absolute imitation: anchored residual guidance for privileged on\-policy distillation\.arXiv preprint arXiv:2606\.10385\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p3.1),[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1)\.
- Zhanget al\.\(2026\)Y\. Zhang, X\. Lin, and C\. WuStepOPSD: step\-aware online preference distillation for agent reinforcement learning\.arXiv preprint arXiv:2605\.27140\.Cited by:[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1)\.
- Zhanget al\.\(2024\)Y\. Zhang, H\. Ruan, Z\. Fan, and A\. RoychoudhuryAutocoderover: autonomous program improvement, 2024\.URL https://arxiv\. org/abs/2404\.05427\.Cited by:[§2\.3](https://arxiv.org/html/2610.33667#S2.SS3.p1.1)\.
- Zhaoet al\.\(2026\)S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. GroverSelf\-distilled reasoner: on\-policy self\-distillation for large language models\.arXiv preprint arXiv:2601\.18734\.Cited by:[§1](https://arxiv.org/html/2610.33667#S1.p1.1)\.
- Zhonget al\.\(2026\)Q\. Zhong, M\. Zheng, M\. Song, X\. Lin, J\. Sun, H\. Jiang, X\. Wang, and J\. FangSod: step\-wise on\-policy distillation for small language model agents\.arXiv preprint arXiv:2605\.07725\.Cited by:[§2\.2](https://arxiv.org/html/2610.33667#S2.SS2.p1.1)\.

## Appendix AImplementation Details for End\-to\-End Privileged Distillation

### A\.1Stage\-wise Capability Evaluation

Figure[2](https://arxiv.org/html/2610.33667#S1.F2)reports four complementary capability measurements on the same 500 held\-out SWE\-bench Verified tasks\. For each method, we use its selected evaluation model and evaluate one agent trajectory per task under the common evaluation scaffold\. The first three measurements are behavioral diagnostics extracted from the trajectory; they are not additional verifier rewards\.

- •Localization \(gold\-file recall\)\.LetFFbe the set of non\-test files changed by the reference patch andEEthe set of files changed by the agent’s final patch\. We report\|F∩E\|/\|F\|\|F\\cap E\|/\|F\|, averaged over tasks with a nonempty reference set\. This measures whether the agent edits the files implicated by the issue, rather than whether the final patch is correct\.
- •Reproduction \(ran failing repro\)\.We count a trajectory as reproducing the issue when the agent creates a reproduction script, executes it, and observes a failure signal such as a traceback, assertion failure, failed test, exception, or nonzero command result\.
- •Verification \(re\-ran repro, passed\)\.After the last source edit, we check whether the agent re\-executes a previously created reproduction script without a failure signal\. This captures an explicit post\-edit verification action and does not require the final patch to pass the hidden benchmark tests\.
- •Solve \(pass@1\)\.We use the official SWE\-bench verifier result: a trajectory counts as solved when its submitted patch satisfies the designated fail\-to\-pass tests without introducing a regression\.

We report each quantity as a percentage, but the aggregation differs by metric: localization is the macro\-average of task\-level recalls over tasks with a nonempty reference set, whereas reproduction, verification, and solve are trajectory\-level percentages\. The diagnostic definitions use only the agent trajectory, reference patch file paths, and verifier outcome; they do not expose privileged teacher evidence to the student\. Consequently, the figure should be read as a stage\-wise behavioral comparison: a method can improve solve rate while becoming less reliable on an earlier subtask, which is the failure mode the routed objective is designed to avoid\.

### A\.2Stage Annotation and Token Alignment

The main method uses the deterministic rule\-based router described in Section[3\.1](https://arxiv.org/html/2610.33667#S3.SS1)\. The parser consumes only the parsed tool call: function name and arguments \(the editor command andpath/file\_path, or bashcmd/command\)\. It accepts both standard and malformed inline parameter forms and supports the r2egym and OpenHands tool dialects\. Creating or editing a file whose basename containsreproorreproduce, executing the corresponding Python script, creating or editing a test file, or running the project’s test suite proposesREP\. A test file is under/tests/or/test/, or has basenametest\_\*,\*\_test\.py,\*\_tests\.py,\*tests\.py,edge\_case\*,conftest\.py, or containsfixture\. Creating or editing any other source file proposesPATCH\. Search, editor views, and read\-only shell commands whose first token is in the exploration set are exploration actions; all other tools emit no proposal\.

A proposed stage advances the current state only if it is later in the orderLOC→\\rightarrowREP→\\rightarrowPATCH; other actions normally inherit the current state, andLOC→\\rightarrowPATCHcan therefore skipREP\. Backtracking is state\-dependent: inREP, three consecutive exploration actions return the state toLOC; inPATCH, a single test run demotes it toREP, while a single exploration action returns it toLOC\. These transitions are applied after the current step’s proposal\.

The state after processing a step labels that same step\. The label is copied to its assistant reasoning and action tokens and to the environment\-token positions concatenated into the response tensor\. A separate response mask excludes environment observations from optimization: GRPO uses all valid model\-generated response positions, while OPSD intersects this mask with the active\-stage indicator in Eq\. \([4](https://arxiv.org/html/2610.33667#S3.E4)\)\. The annotation does not constrain tool choices, rollouts, or rewards, and it routes only the OPD teacher context\.

### A\.3Neural Router and Rule\-Based Router Comparison

We also evaluate a lightweight neural router as an alternative to the rules, rather than as a component of the main method\. Qwen3\.8\-Max\([Qwen Team, 2026](https://arxiv.org/html/2610.33667#bib.bib35)\)labels assistant steps from end\-to\-end Qwen3\-4B\-Instruct\-2507 trajectories asLOC,REP, orPATCH, using the issue excerpt together with action and observation snippets\. The resulting data contain 10,440 valid labels from 731 trajectories\. We use a deterministic trajectory\-level 80/20 hash split, giving 8,179 training labels and 2,261 held\-out labels\. The router concatenates the parsed tool function name with the lower\-cased assistant action, applies a TF–IDF unigram/bigram transform with 50,000 maximum features, and predicts the three labels with a class\-balanced linear classifier \(C=4C=4\)\. Its held\-out accuracy against the Qwen3\.8\-Max annotations is 98\.63%\. The deterministic rule\-based router obtains 92\.0% agreement on the same 2,261 held\-out records\. These figures measure agreement with the annotator on held\-out trajectories, not independently adjudicated semantic accuracy or generalization to unseen issues\. Qwen3\.8\-Max is used only to produce these rollout annotations; it is not an OPSD teacher\.

After each assistant message is generated, this classifier independently predicts one canonical stage for the step, without temporal smoothing or a monotonicity constraint\. Consequently, small changes in an action or its parsed tool call can change the predicted stage and fragment a trajectory into isolated labels\. The rule\-based router instead uses an ordered state machine with state\-dependent backtracking; its assignments are consequently robust to local action variation and tend to form contiguous stage blocks\. Because the assistant action text includes any reasoning tokens present in that message, the classifier input includes that text as well as the parsed function name; it does not use model hidden states\. The predicted label is copied to every assistant and environment\-token position generated by the step\. If the classifier cannot be loaded or produces an invalid label, the implementation falls back to the rule\-based state machine\. Its annotation\-agreement score and downstream solve rate assess different properties: high agreement with the annotator does not by itself establish better training supervision\.

Figure[8](https://arxiv.org/html/2610.33667#A1.F8)reports the supplied neural\-router evaluation points alongside the complete rule\-routedSWE\-Ladderschedule\. Despite its lower label\-agreement score, the rule\-based router gives the stronger downstream training result in this comparison, so annotation accuracy alone is not a sufficient proxy for training utility\.

### A\.4Evaluation Protocols

For the OpenHands rows in Table[2](https://arxiv.org/html/2610.33667#S4.T2), we run the public agent implementation and evaluate every configuration in the same environment and protocol\. The OpenHands scaffold and its evaluation harness are kept fixed across the base, baseline, andSWE\-Ladderrows, with a128128K context window, at most150150agent steps, and temperature0\.70\.7\. The R2E\-Gym rows use the R2E\-Gym scaffold and its evaluation harness, with a6464K context window, at most150150agent steps, and temperature0\.70\.7\. The two scaffold groups are reported separately in Table[2](https://arxiv.org/html/2610.33667#S4.T2)\.

Figure 8:Neural router versusSWE\-Ladderevaluation performance\.Solve rates on SWE\-bench Verified with the R2E\-Gym scaffold\. Both curves use the LOC→\\rightarrowREP→\\rightarrowPATCH training schedule and share a9\.6%9\.6\\%Qwen3\-4B\-Instruct\-2507 base point; they differ only in the router, neural versus rule\-based\. Lines connect the reported evaluation points\.
### A\.5Termination\-Aware Reward Shaping

For a fixed rollout, we suppress the indices\(u,g\)\(u,g\)\. LetV∈\{0,1\}V\\in\\\{0,1\\\}be the verifier result,ξ\\xithe termination reason, andeean indicator of a successful file\-edit call, used as a nonempty\-patch proxy\. Let𝒯o\\mathcal\{T\}\_\{\\mathrm\{o\}\}contain the soft\-overlong reasonsTRUNCATION,MAX\_STEPS, andTRAJ\_TIMEOUT\. Let𝒯d\\mathcal\{T\}\_\{\\mathrm\{d\}\}containENV\_DONEand the empty reason∅\\varnothing\. The reward shaping is

R\(0\)\\displaystyle R^\{\(0\)\}=\{0\.5​V,ξ∈𝒯o,V−0\.2,ξ∈𝒯d,V=e=0,V,otherwise,\\displaystyle=\\begin\{cases\}0\.5V,&\\xi\\in\\mathcal\{T\}\_\{\\mathrm\{o\}\},\\\\ V\-0\.2,&\\xi\\in\\mathcal\{T\}\_\{\\mathrm\{d\}\},\\;V=e=0,\\\\ V,&\\text\{otherwise\},\\end\{cases\}\(6\)R\\displaystyle R=clip⁡\(R\(0\),−1,1\)\.\\displaystyle=\\operatorname\{clip\}\(R^\{\(0\)\},\-1,1\)\.Thus, Eq\. \([6](https://arxiv.org/html/2610.33667#A1.E6)\) discounts a successful verifier result from one to0\.50\.5under soft\-overlong termination, while an unsolved result remains zero\. The−0\.2\-0\.2term applies only when the agent voluntarily finishes \(ENV\_DONEor an empty reason\), remains unsolved, and has made no successful edit\. Exploration\-tool reward and the no\-tool\-call penalty are both configured to zero in the main training chain\.

The overlong filter treatsNO\_CONTAINER,PROMPT\_TRUNCATION, andENCODE\_ERRORas hard failures\. For these cases, the implementation clears the complete response mask instead of treating the trajectory as an ordinary optimization sample\. After reward shaping, group\-relative advantages use the Bessel\-corrected sample standard deviation, with denominatorG−1G\-1, whereG=4G=4is the group size\. GRPO is averaged over each rollout’s valid response tokens; OPSD is averaged over the subset assigned to the active stage\. A rollout with no active\-stage response tokens contributes zero OPSD loss\. The actor also includes the configured0\.0050\.005response\-token entropy bonus in the standard GRPO objective; this term is separate from privileged distillation\.

### A\.6Top\-KKDistillation and Systems Configuration

Using the shorthand from Section[3\.4](https://arxiv.org/html/2610.33667#S3.SS4), letpt​\(v\)p\_\{t\}\(v\)andqt​\(v\)q\_\{t\}\(v\)denote the student and teacher probabilities at the same student\-sampled prefix\. Forv∈𝒦tv\\in\\mathcal\{K\}\_\{t\}, the implementation computes

log⁡q~t​\(v\)=log⁡qt​\(v\)−LSEw∈𝒦t⁡log⁡qt​\(w\),\\log\\widetilde\{q\}\_\{t\}\(v\)=\\log q\_\{t\}\(v\)\-\\operatorname\{LSE\}\_\{w\\in\\mathcal\{K\}\_\{t\}\}\\log q\_\{t\}\(w\),\(7\)whereLSE\\operatorname\{LSE\}denotes log\-sum\-exp over the indexed tokens\. The implemented token contribution is

dt​\(θ\)\\displaystyle d\_\{t\}\(\\theta\)=∑v∈𝒦texp⁡\(log⁡pt​\(v\)\)\\displaystyle=\\sum\_\{v\\in\\mathcal\{K\}\_\{t\}\}\\exp\\\!\\bigl\(\\log p\_\{t\}\(v\)\\bigr\)\(8\)×\[sg⁡\(log⁡pt​\(v\)\)−sg⁡\(log⁡q~t​\(v\)\)\]\.\\displaystyle\}\{\\displaystyle\\times\\bigl\[\\operatorname\{sg\}\(\\log p\_\{t\}\(v\)\)\-\\operatorname\{sg\}\(\\log\\widetilde\{q\}\_\{t\}\(v\)\)\\bigr\]\.Both log terms are detached, so gradients flow through the student probability factors in the surrogate\. The teacher top\-KKtargets are aligned with the corresponding student token probabilities at each sampled prefix\.

The OPSD loss mask is the elementwise product of the valid model\-response mask and the active\-stage indicator\. The teacher’s top\-KKtargets are conditioned on the active stage’s privileged evidence and aligned to the student\-sampled response positions\. Only selected positions contribute to the loss, with no additional per\-sample OPSD token cap\. Targets at non\-active positions, if computed, are excluded from distillation\. This stage\-masked objective is distinct from the historical full\-token distillation variants\.

The student vocabulary normalizer is accumulated in chunks, and probabilities at the teacher token ids are gathered directly\. The reported eight\-NPU setup enables activation offload, FSDP parameter offload for the actor and frozen reference policy, and optimizer offload for the actor\. Ulysses sequence parallelism has degree eight and operates within each rollout sequence rather than across rollouts\.

GRPO and OPSD use per\-rollout means over their respective response\-token and active\-stage masks\. The training hyperparameters are given in Appendix[A\.7](https://arxiv.org/html/2610.33667#A1.SS7)\.

### A\.7Progressive Privilege Scheduling and Training Configuration

SWE\-Ladderapplies privileged stages in the fixed orderLOC→REP→PATCH\\mathrm\{LOC\}\\rightarrow\\mathrm\{REP\}\\rightarrow\\mathrm\{PATCH\}\. Given phase budgetsUL,UR,UPU\_\{L\},U\_\{R\},U\_\{P\}, define the cumulative boundariesb1=ULb\_\{1\}=U\_\{L\},b2=UL\+URb\_\{2\}=U\_\{L\}\+U\_\{R\}, andb3=UL\+UR\+UPb\_\{3\}=U\_\{L\}\+U\_\{R\}\+U\_\{P\}\. The active training stage is

a⁡\(u\)=\{LOC,0≤u<b1,REP,b1≤u<b2,PATCH,b2≤u<b3\.a\(u\)=\\begin\{cases\}\\mathrm\{LOC\},&0\\leq u<b\_\{1\},\\\\ \\mathrm\{REP\},&b\_\{1\}\\leq u<b\_\{2\},\\\\ \\mathrm\{PATCH\},&b\_\{2\}\\leq u<b\_\{3\}\.\\end\{cases\}\(9\)Suppressing the trajectory index, the teacher context for distillation is

cu,t=ca⁡\(u\)\.c\_\{u,t\}=c\_\{a\(u\)\}\.\(10\)whereca⁡\(u\)c\_\{a\(u\)\}is the teacher context selected by the active stage, so Eq\. \([10](https://arxiv.org/html/2610.33667#A1.E10)\) holds at every token of the phase\. The corresponding teacher predictions contribute to the loss only at tokens selected by Eq\. \([4](https://arxiv.org/html/2610.33667#S3.E4)\)\. Thus, the LOC phase distills LOC\-labeled response tokens, the REP phase distills REP\-labeled response tokens, and the PATCH phase distills PATCH\-labeled response tokens\. No unprivileged distillation term is applied to non\-active stages\. This schedule changes the active\-stage mask \(and, when configured, the teacher context\) while preserving end\-to\-end rollouts and the full\-response GRPO objective\.

Each phase continuation warm\-starts from its immediate predecessor, yielding a single checkpoint sequence over the fixed schedule in Eq\. \([9](https://arxiv.org/html/2610.33667#A1.E9)\)\. The active privilegea⁡\(u\)a\(u\)is fixed for all rollouts collected at global training iterationuu\.

#### Training configuration\.

We use\(UL,UR,UP\)=\(30,20,20\)\(U\_\{L\},U\_\{R\},U\_\{P\}\)=\(30,20,20\)global training iterations,K=16K=16, andλ=0\.15\\lambda=0\.15\. Each iteration samples 16 tasks with four rollouts per task, yielding 64 rollouts, and the actor mini\-batch size is one rollout\. The figures index completed training iterations from11: LOC occupies steps11–3030, REP steps3131–5050, and PATCH steps5151–7070\. This is equivalent to the zero\-based update indexuuin Eq\. \([9](https://arxiv.org/html/2610.33667#A1.E9)\)\.

#### Inference\.

At inference, we deploy one end\-to\-end policy with the same rollout termination conditions: submission, environment completion, or the configured resource limit\. Inference uses no teacher, no stage\-specific deployed policies, and no extra trainable parameters\.

## Appendix BStage\-Routed Privileged Information

The main text describes how the three stage\-specific evidence views are constructed\. This appendix gives a representative router trace \(Table[7](https://arxiv.org/html/2610.33667#A2.T7)\), the evidence fields supplied for each stage \(Table[6](https://arxiv.org/html/2610.33667#A2.T6)and Figures[9](https://arxiv.org/html/2610.33667#A2.F9)–[11](https://arxiv.org/html/2610.33667#A2.F11)\), and the token cost that each evidence block adds to the teacher prompt \(Table[8](https://arxiv.org/html/2610.33667#A2.T8)\)\.

Table 6:Schematic examples of the stage\-specific privileged information\. The actual fields are populated from each instance’s verified repair metadata\.Table 7:A representative2121\-step trajectory \(anoauthlibissue\), labeled by the rule\-based router of Appendix[A\.2](https://arxiv.org/html/2610.33667#A1.SS2)\. Re\-running the reproduction script while inPATCHdemotes the state toREP\(step 12\), and the third consecutive inspection step returns it toLOC\(step 15\), so the label sequence is not monotone\.Table 8:Tokens added by each privileged evidence block over the shared task prompt across the1,1491\{,\}149training instances \(Qwen3\-4B\-Instruct\-2507 tokenizer\)\.### B\.1Representative Privileged\-Information Views

The following examples show the evidence blocks supplied to the frozen teacher for the three repair stages\. They are rendered from the structured fields described in the main text; the student prompt does not include these blocks\.

![Refer to caption](https://arxiv.org/html/2610.33667v1/figures/LOC_PI.png)Figure 9:Localization privileged information\.Candidate files, old\-side hunk ranges, and source fragments identify where the issue should be investigated without revealing the replacement patch\.![Refer to caption](https://arxiv.org/html/2610.33667v1/figures/REP_PI.png)Figure 10:Reproduction privileged information\.Fail\-to\-pass tests, reference test snippets, and candidate test changes describe the failing behavior and provide a starting point for reproducing it\.![Refer to caption](https://arxiv.org/html/2610.33667v1/figures/PATCH_PI.png)Figure 11:Patch privileged information\.The verified repair diff specifies the intended source\-code correction available to the patch\-stage teacher\.

## Appendix CAdditional Training Diagnostics

### C\.1Training Dynamics

Figure[5](https://arxiv.org/html/2610.33667#S4.F5)compares GRPO, GRPO\+OPSD, SDAR, and the completeSWE\-Laddersequence over a common7070\-iteration horizon\. We distinguish model\-generated tokens from tool/environment output and plot shaped training reward separately from held\-out pass@1\. Curves show individual training runs, not averages over independent seeds\.

The GRPO run exhibits a pronounced decline in its optimization diagnostics\. Comparing iterations11–1010with6161–7070, its mean entropy diagnostic falls from0\.03000\.0300to0\.00610\.0061, mean gradient norm from1\.8201\.820to0\.1870\.187, and mean training reward from0\.2360\.236to0\.0550\.055\. Meanwhile, mean generated tokens increase from8\.298\.29k to16\.7316\.73k\.SWE\-Ladder’s corresponding entropy means are0\.02510\.0251and0\.02210\.0221, and its gradient\-norm means are2\.3532\.353and2\.3972\.397\. Its mean generated length increases from10\.0910\.09k to11\.6211\.62k\. This sequence does not exhibit the same diagnostic decline as the GRPO run, although its reward remains noisy and does not increase monotonically\.

The other privileged baselines provide an important qualification\. The GRPO\+OPSD and SDAR traces retain additional optimization signal late in training, but preserving entropy or gradient magnitude alone does not explainSWE\-Ladder’s held\-out performance\. These plots therefore do not establish a general failure of either baseline; they provide diagnostic context for the stage\-aligned comparison\. Appendix[C\.2](https://arxiv.org/html/2610.33667#A3.SS2)defines the plotted metrics\.

### C\.2Metric Definitions

The panels report logged actor entropy and gradient norm, the mean number of model\-generated tokens, and the mean shaped training reward; the plotting manifest records their exact source fields\. We retain the implementation’s logged scale for entropy and gradient norm rather than treating them as independently recalculated policy statistics\. In particular, generated\-token counts exclude the environment output included in the broaderresponse\_length/meanfield\. The reward includes termination\-aware shaping \(Appendix[A\.5](https://arxiv.org/html/2610.33667#A1.SS5)\) and must not be read as SWE\-bench pass@1\. The teacher\-top\-KKloss is not plotted as a calibrated KL distance for the reasons discussed in Appendix[A\.6](https://arxiv.org/html/2610.33667#A1.SS6)\.