One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents

Hugging Face Daily Papers Papers
software-engineering reinforcement-learning ai-agents category-aware expert-training multi-teacher-distillation

Summary

This paper introduces a category-aware expert training framework for software engineering agents to mitigate uneven progress across task categories, using iterative training and multi-teacher distillation, with significant performance gains on Pro-618 and SWE-bench Multilingual benchmarks.

Repository-level software engineering (SWE) comprises heterogeneous task categories, whose progress under pooled agentic reinforcement learning can be uneven: gains in some categories coincide with regressions in others, while aggregate resolution obscures these changes. Motivated by this category see-saw, we develop a category-aware expert-training and policy-integration framework. Executable task construction and SWE Labeler, an evidence-grounded multi-axis labeling system, organize the training pools. Initial category-specific RL improves average training success while leaving uneven instance-level progress, motivating explicit consolidation of successful behavior and policy-adaptive task selection. Same-origin category experts alternate long-horizon Agentic-miniRL with Refresh-Repair-Expand (RRE): the updated policy refreshes instance mastery, reuses its own verified successful trajectories for Repair SFT, and reselects tasks for further RL. Label-routed multi-teacher on-policy distillation (MOPD) consolidates the experts into one deployable student, with ReLU-gated reward extrapolation keeping only each teacher's improving direction over the reference. Expert training and policy integration require no external model to provide solution trajectories or action targets. We evaluate Pooled RL and Balanced RL, expert development, and single-model integration through aggregate and per-category resolution, the minimum category lift over each joint-RL baseline, and expert-gain recovery. The final MOPD policy achieves mean resolution of 58.04% on Pro-618 and 59.00% on SWE-bench Multilingual, improving over the base model by 5.39 and 2.78 percentage points, respectively.
Original Article
View Cached Full Text

Cached at: 09/22/26, 03:25 AM

Paper page - One to More, More to One: Category-Aware Iterative Expert Training for Software Engineering Agents

Source: https://huggingface.co/papers/2609.23377

Abstract

Repository-levelsoftwareengineering(SWE)comprisesheterogeneoustaskcategories,whoseprogressunderpooledagenticreinforcementlearningcanbeuneven:gainsinsomecategoriescoincidewithregressionsinothers,whileaggregateresolutionobscuresthesechanges.Motivatedbythiscategorysee-saw,wedevelopacategory-awareexpert-trainingandpolicy-integrationframework.ExecutabletaskconstructionandSWELabeler,anevidence-groundedmulti-axislabelingsystem,organizethetrainingpools.Initialcategory-specificRLimprovesaveragetrainingsuccesswhileleavinguneveninstance-levelprogress,motivatingexplicitconsolidationofsuccessfulbehaviorandpolicy-adaptivetaskselection.Same-origincategoryexpertsalternatelong-horizonAgentic-miniRLwithRefresh-Repair-Expand(RRE):theupdatedpolicyrefreshesinstancemastery,reusesitsownverifiedsuccessfultrajectoriesforRepairSFT,andreselectstasksforfurtherRL.Label-routedmulti-teacheron-policydistillation(MOPD)consolidatestheexpertsintoonedeployablestudent,withReLU-gatedrewardextrapolationkeepingonlyeachteacher’simprovingdirectionoverthereference.Experttrainingandpolicyintegrationrequirenoexternalmodeltoprovidesolutiontrajectoriesoractiontargets.WeevaluatePooledRLandBalancedRL,expertdevelopment,andsingle-modelintegrationthroughaggregateandper-categoryresolution,theminimumcategoryliftovereachjoint-RLbaseline,andexpert-gainrecovery.ThefinalMOPDpolicyachievesmeanresolutionof58.04%onPro-618and59.00%onSWE-benchMultilingual,improvingoverthebasemodelby5.39and2.78percentagepoints,respectively.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2609\.23377

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### Logics-MLLM/Logics-SWE-Qwen3.6-27B Reinforcement Learning• 27B• Updatedabout 1 hour ago • 30 • 5

Datasets citing this paper1

#### Logics-MLLM/Logics-SWE-Env-2.5K Viewer• Updatedabout 1 hour ago • 2.55k • 1

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.23377 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AgentBrew: Lifelong Knowledge Brewing from Strong Teachers to Weak LLM Agents

arXiv cs.AI

AgentBrew proposes a training-free method for distilling knowledge from strong teacher LLM agents to weak student agents, using a failure-triggered reflection loop and student-aware synthesis to create executable notes, improving performance on coding, math, and tool-use tasks without weight updates.

Sample-Efficient Learning from Agent Experience

Hugging Face Daily Papers

Proposes Experience Distillation, a method that internalizes in-context learning gains from agent interaction histories into model weights without requiring additional environment interaction, achieving significant sample efficiency improvements on software engineering and text-adventure tasks.