基于模型切片的协作学习与推理

# Collaborative Learning and Reasoning over Model Slices
Source: [https://arxiv.org/html/2610.12927](https://arxiv.org/html/2610.12927)
Yuheng Wang†Runde Yang†Zhuohan BaoQiran ZhangHuatao LiYiyang ZhengLin WuJingru FanShu YaoYihan LiZhi Li

###### Abstract

Large language models \(LLMs\) excel across diverse tasks, benefiting from knowledge priors, in\-context reasoning, and test\-time computation\. Despite these strengths, single\-model reasoning remains vulnerable to error propagation, as later steps depend on the model’s earlier outputs, leaving reasoning blind spots difficult to independently reassess\. To address this limitation, we developcollaborative learning, encompassing the construction of heterogeneous collaborators and the learning of effective cross\-model collaboration\. Specifically,model slicingreplaces a large model with smaller, heterogeneous models under a comparable parameter budget, enabling prior reasoning to be reassessed across distinct model parameterizations\.Pyramid supervisioncombines protocol distillation with step\-level supervision to refine reasoning and interaction decisions throughout collaboration\. We evaluate our approach across four task categories: multi\-hop question answering, mathematical reasoning, code generation, and open\-ended constrained generation\. The trained heterogeneous models achieve comparable or better task performance than the original large model and established multi\-agent baselines, including reinforcement\-learning\-based methods, while also achieving higher inference efficiency than the multi\-agent baselines and exhibiting differentiated collaboration patterns across tasks\. These findings show that learned collaboration turns model heterogeneity into complementary reasoning capabilities\. Code is available at:[github\.com/SJTUFreshman/collaborative\-learning](https://github.com/SJTUFreshman/collaborative-learning)\.

††footnotetext:†Equal contribution\.††footnotetext:🖂 Corresponding author\.## 1Introduction

Large language models \(LLMs\) have demonstrated capabilities across tasks\([Wang et al\., 2024b](https://arxiv.org/html/2610.12927#bib.bib2)\)\. Advances highlight the benefits of scaling test\-time computation\([Snell et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib3);[Muennighoff et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib4);[Wu et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib11)\)\. Many approaches seek to improve task performance by allowing a single model to structure its reasoning process\([Besta et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib1)\)and generate, inspect, and revise intermediate steps\([Zhou et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib14);[Zhang et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib29);[Kumar et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib7)\)\. However, when the same model both generates and evaluates a solution, it may exhibit self\-bias, favoring its own outputs and limiting its ability to identify and correct reasoning errors\([Huang et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib8);[Xu et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib9)\)\. Consequently, errors in intermediate steps may remain undetected and influence subsequent reasoning\([Zheng et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib12)\)\. This motivatesreasoning\-state isolationacross models, allowing prior reasoning to be independently reassessed under distinct model parameterizations rather than repeatedly evaluated within the same model’s reasoning space\([Du et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib10);[Liang et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib21)\)\.

Accordingly, we develop acollaborative learningframework for reasoning and learning across heterogeneous models, as illustrated in Figure[2](https://arxiv.org/html/2610.12927#footnote2)\. Model slicing replaces a single large model with several smaller, heterogeneous models under a comparable total parameter budget\([Liu et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib26)\)\. Through explicit handoffs of task state between models\([Chen et al\., 2024c](https://arxiv.org/html/2610.12927#bib.bib22)\), prior reasoning can be independently reassessed under distinct model parameterizations, thereby enabling model\-specific reasoning\-state isolation while producing interaction trajectories that capture reasoning and interaction decisions\. Together, this isolation and the resulting trajectories provide the structural basis for training heterogeneous models to leverage complementary capabilities\([Wang et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib20);[Abdulaal et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib17);[Chen et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib15)\)when questioning, extending, or revising prior conclusions\.

![Refer to caption](https://arxiv.org/html/2610.12927v1/Figure1.png)Figure 1:Collaborative Learning and Reasoning over Model Slices\.Model slicing reallocates a single large model into multiple smaller, heterogeneous models under a comparable parameter budget, enabling reasoning to be reassessed across model parameterizations\. Building on this, collaborative learning trains heterogeneous models through pyramid supervision, combining protocol distillation for interaction initialization with step\-level supervision over collaboration trajectories222Short linear, long linear, short cyclic, and long cyclic trajectories are defined in Section[3\.5](https://arxiv.org/html/2610.12927#S3.SS5)\.\.Although this structure provides a foundation for cross\-model collaboration, outcome\-level supervision alone provides insufficient guidance for intermediate reasoning and interaction decisions throughout the collaborative process\([Lightman et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib6);[Wang et al\., 2024a](https://arxiv.org/html/2610.12927#bib.bib30);[Setlur et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib13)\)\. To address this limitation, we proposepyramid supervision, which uses supervisory signals from larger models to guide the heterogeneous models produced by model slicing toward effective cross\-model collaboration\. It combines interaction initialization with fine\-grained supervision over reasoning and interaction decisions along model\-generated collaboration trajectories, while integrating task\-level rewards to jointly optimize intermediate decision quality and overall task performance\. Through this training mechanism, heterogeneous models learn effective collaborative reasoning and develop differentiated interaction patterns across tasks\.

We evaluate our approach across four task categories: multi\-hop question answering, mathematical reasoning, code generation, and open\-ended constrained generation\. With a total parameter budget comparable to that of the original large model, the trained heterogeneous models collectively match or outperform the original model as well as established multi\-agent system \(MAS\) and multi\-agent reinforcement learning \(MARL\) baselines\. Analysis of interaction trajectories further reveals reduced reasoning redundancy and differentiated collaboration patterns characterized by questioning, complementing, and revising prior reasoning\. Together, these results suggest that collaborative reasoning across heterogeneous models can improve task performance while enabling complementary contributions and the reassessment of prior reasoning across distinct model parameterizations\.

Our key contributions are summarized in three points:

- •We propose a trainable paradigm for collaborative reasoning and learning across heterogeneous models, enabling independent reassessment across distinct model parameterizations and progressive learning of effective cross\-model collaboration from interaction trajectories\.
- •Collaborative learning comprises model slicing and pyramid supervision\. Model slicing constructs heterogeneous models under a comparable budget and enables reasoning\-state isolation through task\-state handoffs\. Pyramid supervision combines interaction initialization with fine\-grained supervision from the original large model over reasoning and interaction decisions\.
- •We validate collaborative learning across four task categories\. The trained heterogeneous models achieve comparable or better performance than the original large model, established MAS baselines, and MARL methods, while demonstrating higher inference efficiency than MAS and MARL baselines and exhibiting emergent task\-adaptive cross\-model collaboration patterns\.

## 2Method

![Refer to caption](https://arxiv.org/html/2610.12927v1/Figure2.png)Figure 2:Overview of collaborative learning\.\(a\) Model slicing constructs smaller, heterogeneous models under a comparable total parameter budget\. \(b\) Pyramid supervision initializes collaboration through protocol distillation and refines it through step\-level supervision\. The original large model evaluates each step’s reasoning and interaction decisions within the complete trajectory\. Step\-level scores and final task rewards jointly guide collaborative policy refinement\.Collaborative reasoning characterizes an iterative reasoning process among heterogeneous models, where explicit task\-state handoffs under a unified interaction protocol allow prior reasoning to be continually reassessed by models with different parameters \(⊳\\triangleright[Section2\.1](https://arxiv.org/html/2610.12927#S2.SS1)\)\. To realize this process, collaborative learning constructs a heterogeneous model set through model slicing and learns effective cross\-model collaboration through pyramid supervision \(⊳\\triangleright[Section2\.2](https://arxiv.org/html/2610.12927#S2.SS2)\)\.

### 2\.1Collaborative Reasoning

Collaborative reasoning refers to a process in which multiple models jointly construct a solution through successive contributions\([Yin et al\., 2023](https://arxiv.org/html/2610.12927#bib.bib65);[Chen et al\., 2024a](https://arxiv.org/html/2610.12927#bib.bib66)\)\. We define a unified interaction protocol that specifies how heterogeneous models generate reasoning and retain, transfer, or terminate control\. For a task inputqq, each interaction stepttis represented by

zt=\(it,xt,yt,at\)z\_\{t\}=\(i\_\{t\},x\_\{t\},y\_\{t\},a\_\{t\}\)\(1\)whereiti\_\{t\}denotes the active modelMitM\_\{i\_\{t\}\},xtx\_\{t\}contains the task input and preceding interaction history,yty\_\{t\}is the current reasoning, andata\_\{t\}is the interaction action\. The action space is defined as

at∈\{continue,terminate\}∪\{handoff\(j\)∣j∈\{1,…,N\},j≠it\}a\_\{t\}\\in\\\{\\texttt\{continue\},\\texttt\{terminate\}\\\}\\cup\\\{\\texttt\{handoff\}\(j\)\\mid j\\in\\\{1,\\ldots,N\\\},\\ j\\neq i\_\{t\}\\\}\(2\)wherecontinuekeeps control with the model,handoff\(j\)transfers the task state and control toMjM\_\{j\}, andterminateends collaborative reasoning\. A complete trajectory is represented as

τ=\(z1,z2,…,zT\)\\tau=\(z\_\{1\},z\_\{2\},\\ldots,z\_\{T\}\)\(3\)whereTTdenotes the total number of interaction steps recorded in the complete trajectory\.

Throughhandoff, models use distinct parameters to extend, question, or revise prior conclusions, enabling cross\-model reassessment across the evolving reasoning process\. The resulting trajectories record reasoning and interaction decisions for subsequent collaborative learning\.

### 2\.2Collaborative Learning

Collaborative learning encompasses both the construction of heterogeneous collaborators and the learning of cross\-model collaboration, with the pipeline illustrated in Figure[2](https://arxiv.org/html/2610.12927#S2.F2)\. Model slicing constructs heterogeneous collaborators under a comparable parameter budget, providing the structural basis for cross\-model reasoning\. Pyramid supervision guides collaborative learning through protocol distillation and step\-level supervision, progressively refining models’ reasoning and interaction decisions while integrating task\-level rewards throughout the collaborative reasoning process\.

#### 2\.2\.1Model Slicing

Model slicing replaces a large model with smaller, independently parameterized models under a comparable parameter budget, each operating as a standalone model and interacting through explicit task\-state handoffs\. LetMLM\_\{L\}denote the large model with parametersθL\\theta\_\{L\}\. We have

ℳ=\{M1,…,MN\},∑i=1N\|θi\|≈\|θL\|\\mathcal\{M\}=\\\{M\_\{1\},\\ldots,M\_\{N\}\\\},\\qquad\\sum\_\{i=1\}^\{N\}\|\\theta\_\{i\}\|\\approx\|\\theta\_\{L\}\|\(4\)whereθi\\theta\_\{i\}denotes the parameters ofMiM\_\{i\}, and\|θ\|\|\\theta\|denotes the parameter count\.

These models differ in size and parameters and are not obtained by directly partitioning the weights ofMLM\_\{L\}\. Together with task\-state handoffs, this heterogeneous model set enables model\-specificreasoning\-state isolationacross distinct model parameterizations during collaboration\.

#### 2\.2\.2Pyramid Supervision

We introducepyramid supervision, which refines collaboration through supervision across model scales\. Protocol distillation uses the largest model slice to initialize cross\-model interaction, while step\-level supervision uses the original large model to evaluate collaboration trajectories\.

Protocol Distillation\.Although the interaction protocol specifies how models communicate and transfer control, heterogeneous models do not naturally learn to use it effectively as reasoning evolves\. In particular, they must learn when to retain control, hand off the task state, or terminate based on the evolving reasoning context\([Zeng et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib68)\)\. We therefore use protocol distillation to initialize cross\-model interaction from teacher\-generated trajectories\([Chen et al\., 2024b](https://arxiv.org/html/2610.12927#bib.bib67)\)\.

We select the largest model inℳ\\mathcal\{M\}by parameter count as the teacherM⋆M^\{\\star\}and instantiateNNhomogeneous copies to form the teacher system:

ℳteacher=\{M1⋆,…,MN⋆\},Mj⋆≡M⋆,j=1,…,N\\mathcal\{M\}\_\{\\mathrm\{teacher\}\}=\\\{M\_\{1\}^\{\\star\},\\ldots,M\_\{N\}^\{\\star\}\\\},\\qquad M\_\{j\}^\{\\star\}\\equiv M^\{\\star\},\\quad j=1,\\ldots,N\(5\)
Following the same interaction protocol as the target models, the teacher system generates multi\-step trajectories for distillation inputs𝒬distill=\{q\(m\)\}m=1K\\mathcal\{Q\}\_\{\\mathrm\{distill\}\}=\\\{q^\{\(m\)\}\\\}\_\{m=1\}^\{K\}:

τ\(m\)∼Pℳteacher\(τ∣q\(m\)\),m=1,…,K\\tau^\{\(m\)\}\\sim P\_\{\\mathcal\{M\}\_\{\\mathrm\{teacher\}\}\}\\left\(\\tau\\mid q^\{\(m\)\}\\right\),\\qquad m=1,\\ldots,K\(6\)We retain qualified trajectories as demonstrations for subsequent distillation\.

To avoid a fixed correspondence between teacher slots and target models, we sample a uniform permutation for each trajectory:

π\(m\)∼Uniform\(SN\),i~t\(m\)=π\(m\)\(it\(m\)\),t=1,…,Tm\\pi^\{\(m\)\}\\sim\\mathrm\{Uniform\}\(S\_\{N\}\),\\qquad\\widetilde\{i\}\_\{t\}^\{\(m\)\}=\\pi^\{\(m\)\}\\left\(i\_\{t\}^\{\(m\)\}\\right\),\\quad t=1,\\ldots,T\_\{m\}\(7\)whereSNS\_\{N\}denotes the set of permutations over\{1,…,N\}\\\{1,\\ldots,N\\\}\. Within each trajectory, the same permutation is applied consistently to all occurrences associated with each teacher slot, including model indices, handoff targets, and corresponding references in the interaction history\. This operation changes only the correspondence between teacher slots and target models across trajectories, while preserving the reasoning content, step order, and handoff structure of the original trajectory\.

Letx~t\(m\)\\widetilde\{x\}\_\{t\}^\{\(m\)\}ando~t\(m\)\\widetilde\{o\}\_\{t\}^\{\(m\)\}denote the reassigned context and response\. Supervision covers the complete response, including reasoning and interaction decisions\([Chen et al\., 2023](https://arxiv.org/html/2610.12927#bib.bib69)\)\. For each modelMiM\_\{i\}, the reassigned context–response pairs satisfyingi~t\(m\)=i\\widetilde\{i\}\_\{t\}^\{\(m\)\}=iform its training set𝒟ipro\\mathcal\{D\}\_\{i\}^\{\\mathrm\{pro\}\}\. Following sequence\-level knowledge distillation\([Kim and Rush, 2016](https://arxiv.org/html/2610.12927#bib.bib70)\), each model minimizes

ℒipro=−𝔼\(x,o\)∼𝒟ipro​\[log⁡pθi​\(o∣x\)\]\\mathcal\{L\}\_\{i\}^\{\\mathrm\{pro\}\}=\-\\mathbb\{E\}\_\{\(x,o\)\\sim\\mathcal\{D\}\_\{i\}^\{\\mathrm\{pro\}\}\}\\left\[\\log p\_\{\\theta\_\{i\}\}\(o\\mid x\)\\right\]\(8\)
This objective initializes cross\-model interaction capabilities in the heterogeneous models, providing a stable starting point for trajectory\-level refinement of reasoning and interaction decisions\.

Step\-Level Supervision\.After protocol distillation, the heterogeneous models perform collaborative reasoning under the shared interaction protocol, generating trajectories for subsequent optimization\. For a given training taskqq, a collaborative trajectory is sampled as

τ=\(z1,…,zT\)∼Pℳ​\(τ∣q\)\\tau=\(z\_\{1\},\\ldots,z\_\{T\}\)\\sim P\_\{\\mathcal\{M\}\}\(\\tau\\mid q\)\(9\)wherePℳP\_\{\\mathcal\{M\}\}is induced by the current models and the shared interaction protocol\.

Learning effective collaboration from these trajectories requires identifying the contribution of each reasoning and interaction decision to the overall process\([Foerster et al\., 2018](https://arxiv.org/html/2610.12927#bib.bib72)\)\. The value of an individual step depends not only on its local correctness but also on how it influences subsequent collaboration\. We therefore retain the original large modelMLM\_\{L\}as an evaluator𝒥\\mathcal\{J\}, which assesses each step in the context of the complete trajectory\. Given the taskqqand trajectoryτ\\tau, the evaluator produces two complementary scores:

\{\(strea,stact\)\}t=1T=𝒥⁡\(q,τ\),strea,stact∈\[−1,1\]\\left\\\{\\left\(s\_\{t\}^\{\\mathrm\{rea\}\},s\_\{t\}^\{\\mathrm\{act\}\}\\right\)\\right\\\}\_\{t=1\}^\{T\}=\\mathcal\{J\}\(q,\\tau\),\\qquad s\_\{t\}^\{\\mathrm\{rea\}\},s\_\{t\}^\{\\mathrm\{act\}\}\\in\[\-1,1\]\(10\)
The reasoning scorestreas\_\{t\}^\{\\mathrm\{rea\}\}evaluates the model’s reasoning contribution in terms of correctness, informativeness, and relevance to the task state and interaction history\. The interaction scorestacts\_\{t\}^\{\\mathrm\{act\}\}evaluates whether its decision to continue, hand off the task state, or terminate appropriately advances the collaborative process\. Separating these dimensions avoids conflating reasoning quality with interaction quality\. We average the two scores to obtain a step\-level score:

stdel=12​\(strea\+stact\)s\_\{t\}^\{\\mathrm\{del\}\}=\\frac\{1\}\{2\}\\left\(s\_\{t\}^\{\\mathrm\{rea\}\}\+s\_\{t\}^\{\\mathrm\{act\}\}\\right\)\(11\)
Reward integration\.Step\-level scores provide fine\-grained supervision for intermediate reasoning and interaction decisions, but local decision quality does not necessarily translate into successful task completion\. To provide decision\-specific feedback while also accounting for the overall task objective, we integrate each step\-level score with the final task reward\. Letm⁡\(q,y^\)∈\[0,1\]m\(q,\\hat\{y\}\)\\in\[0,1\]denote the task\-specific evaluation metric for the final predictiony^\\hat\{y\}\. We map it to the same\[−1,1\]\[\-1,1\]range as the step\-level score and define

rtask=2​m​\(q,y^\)−1,rt=α​rtask\+\(1−α\)​stdelr\_\{\\mathrm\{task\}\}=2m\(q,\\hat\{y\}\)\-1,\\qquad r\_\{t\}=\\alpha r\_\{\\mathrm\{task\}\}\+\(1\-\\alpha\)s\_\{t\}^\{\\mathrm\{del\}\}\(12\)whereα∈\[0,1\]\\alpha\\in\[0,1\]controls the trade\-off between the task\-level reward and the step\-level score\. The resulting reward distinguishes the contributions of intermediate steps across the complete collaborative trajectory while aligning their optimization with the final task objective\.

Collaborative Policy Refinement\.Using the combined rewards, we extract the steps at which each model was active from the collaborative trajectories and construct model\-specific training data\. For modelMiM\_\{i\}, the corresponding training examples are collected as

𝒟i=\{\(xt\(m\),ot\(m\),rt\(m\)\)∣it\(m\)=i\}\\mathcal\{D\}\_\{i\}=\\left\\\{\(x\_\{t\}^\{\(m\)\},o\_\{t\}^\{\(m\)\},r\_\{t\}^\{\(m\)\}\)\\mid i\_\{t\}^\{\(m\)\}=i\\right\\\}\(13\)whereot\(m\)o\_\{t\}^\{\(m\)\}denotes the complete response at stepttof trajectorymm, serialized under the shared interaction protocol and including its reasoning, answer, and interaction fields\. Following the score\-function policy\-gradient form, we optimize each model with the reward\-weighted objective

ℒicol​\(θi\)=−𝔼\(x,o,r\)∼𝒟i​\[r​log⁡pθi​\(o∣x\)\]\\mathcal\{L\}\_\{i\}^\{\\mathrm\{col\}\}\(\\theta\_\{i\}\)=\-\\mathbb\{E\}\_\{\(x,o,r\)\\sim\\mathcal\{D\}\_\{i\}\}\\left\[r\\log p\_\{\\theta\_\{i\}\}\(o\\mid x\)\\right\]\(14\)
This objective increases the likelihood of responses assigned positive rewards and decreases the likelihood of those assigned negative rewards\. Because each reward combines step\-level evaluation conditioned on the complete trajectory with the final task outcome, each model can adapt its policy according to how its own decisions contribute to the collaborative result, allowing differentiated collaboration patterns to progressively emerge through training\([Cui et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib73)\)\.

## 3Experiments

### 3\.1Experimental Setup

Datasets\.We use five benchmarks across four task categories: multi\-hop question answering \(MuSiQue\([Trivedi et al\., 2022](https://arxiv.org/html/2610.12927#bib.bib59)\)\), mathematical reasoning \(GSM\-Hard\([Gao et al\., 2023](https://arxiv.org/html/2610.12927#bib.bib60)\)and MATH\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.12927#bib.bib61)\)\), code generation \(an eight\-language MultiPL\-E subset\([Cassano et al\., 2023](https://arxiv.org/html/2610.12927#bib.bib62)\)\)333Following the language selection in the Qwen3 technical report\([Yang et al\., 2025a](https://arxiv.org/html/2610.12927#bib.bib64)\), we restrict code\-generation evaluation to Python, C\+\+, Java, PHP, TypeScript, C\#, Bash, and JavaScript\., and open\-ended constrained generation \(Conifer\([Sun et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib63)\)\)\.

Baselines\.\(1\) Single\-agent baselines: Qwen3\-14B\([Yang et al\., 2025a](https://arxiv.org/html/2610.12927#bib.bib64)\)and its self\-evaluated RL variant \(SE\-RL\)444SE\-RL uses Qwen3\-14B self\-feedback for comparison with cross\-model feedback\.\. \(2\) A homogeneous MAS of three Qwen3\-8B models \(24B total\) without collaborative training, serving as a higher\-capacity multi\-model baseline\. \(3\) A homogeneous collaborative\-learning system \(Homo\.CL\) of three Qwen3\-4B models \(12B total\), trained with the same procedure as our heterogeneous system to assess the effect of model heterogeneity in collaborative reasoning\. \(4\) Established multi\-agent frameworks: MAD\([Du et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib10)\), AgentVerse\([Chen et al\., 2024c](https://arxiv.org/html/2610.12927#bib.bib22)\), AFlow\([Zhang et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib24)\), and GPTSwarm\([Zhuge et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib23)\)\. \(5\) MARL methods: MAGRPO\([Liu et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib26)\), MAPoRL\([Park et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib74)\), and AT\-GRPO\([Zhao et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib28)\)\.

Metrics\.We report accuracy and F1 on MuSiQue\([Trivedi et al\., 2022](https://arxiv.org/html/2610.12927#bib.bib59)\), GSM\-Hard\([Gao et al\., 2023](https://arxiv.org/html/2610.12927#bib.bib60)\), and MATH\([Hendrycks et al\., 2021](https://arxiv.org/html/2610.12927#bib.bib61)\)\. We use weighted pass@1 and macro\-language pass@1 for MultiPL\-E\([Cassano et al\., 2023](https://arxiv.org/html/2610.12927#bib.bib62)\), and Coverage and Explicit for Conifer\([Sun et al\., 2024](https://arxiv.org/html/2610.12927#bib.bib63)\)\. Beyond task performance, we analyze handoffs, anomaly rates, reasoning redundancy, and model\-wise behavioral distributions to characterize training\-induced changes in collaboration patterns\.

Implementation Details\.Model slicing replaces Qwen3\-14B \(MLM\_\{L\}\)\([Yang et al\., 2025a](https://arxiv.org/html/2610.12927#bib.bib64)\)with Qwen3\-1\.7B, Qwen3\-4B, and Qwen3\-8B \(13\.7B total\)\. The three collaborating models share the same prompt and action space\. Three Qwen3\-8B instances generate protocol distillation demonstrations, while Qwen3\-14B evaluates reasoning and interaction decisions within complete trajectories for step\-level supervision\. Experiments run on8×8\\timesNVIDIA A800\-SXM4\-80GB GPUs\. Each model is trained with a rank\-64 LoRA adapter\([Hu et al\., 2022](https://arxiv.org/html/2610.12927#bib.bib75)\)while its base parameters remain frozen\.

### 3\.2Does Collaborative Learning Lead to Improved Performance?

Table 1:Performance comparison with baseline methods\.Scores are percentages \(%\); higher is better\. Acc\. denotes accuracy; Wtd\. and Mac\. denote weighted and macro\-language pass@1, respectively; Cov\. and Expl\. denote Coverage and Explicit, respectively\.CL\.denotes collaborative learning\. Best and second\-best results are shown inboldandunderline, respectively\.↑\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\uparrow\}and↓\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\downarrow\}indicate scores above and below the corresponding baseline mean for each metric in the table\.†\\daggerindicates that the corresponding baseline differs significantly fromCL\.\(p≤0\.05p\\leq 0\.05\)\.\\rectanglecolorCadetBlue\!201\-12\-114,6,8,10,12,14\\Block\[l\]2\-1MethodMuSiQueGSM\-HardMATHMultiPL\-EConiferAcc\.F1Acc\.F1Acc\.F1Wtd\.Mac\.Cov\.Expl\.Qwen3\-14B39\.55†↑0\.87\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.87$\}\}\}49\.70†↑0\.47\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.47$\}\}\}68\.94↑1\.65\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.65$\}\}\}80\.69†↑3\.19\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.19$\}\}\}74\.60†↑4\.40\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.40$\}\}\}82\.86†↑3\.25\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.25$\}\}\}55\.18†↑1\.08\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.08$\}\}\}57\.34†↑2\.59\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.59$\}\}\}88\.00†↑0\.17\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.17$\}\}\}95\.72↑0\.62\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.62$\}\}\}SE\-RL41\.75↑3\.07\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.07$\}\}\}52\.71†↑3\.48\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.48$\}\}\}68\.18↑0\.89\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.89$\}\}\}79\.67†↑2\.17\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.17$\}\}\}76\.00↑5\.80\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!5\.80$\}\}\}80\.70↑1\.09\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.09$\}\}\}60\.43†↑6\.33\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!6\.33$\}\}\}62\.55†↑7\.80\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!7\.80$\}\}\}88\.00↑0\.17\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.17$\}\}\}95\.81↑0\.71\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.71$\}\}\}Homo\. MAS26\.40†↓12\.28\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!12\.28$\}\}\}32\.82↓16\.41\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!16\.41$\}\}\}71\.21↑3\.92\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.92$\}\}\}79\.93†↑2\.43\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.43$\}\}\}68\.20†↓2\.00\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.00$\}\}\}76\.41†↓3\.20\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!3\.20$\}\}\}56\.36†↑2\.26\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.26$\}\}\}57\.20↑2\.45\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.45$\}\}\}88\.02†↑0\.19\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.19$\}\}\}94\.27†↓0\.83\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.83$\}\}\}Homo\.CL\.36\.00†↓2\.68\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.68$\}\}\}47\.19†↓2\.04\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.04$\}\}\}60\.61†↓6\.68\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!6\.68$\}\}\}76\.51↓0\.99\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.99$\}\}\}68\.60†↓1\.60\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.60$\}\}\}77\.95↓1\.66\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.66$\}\}\}53\.33†↓0\.77\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.77$\}\}\}52\.65†↓2\.10\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.10$\}\}\}88\.15↑0\.32\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.32$\}\}\}95\.22†↑0\.12\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.12$\}\}\}MAD39\.51†↑0\.83\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.83$\}\}\}50\.59†↑1\.36\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.36$\}\}\}66\.67†↓0\.62\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.62$\}\}\}74\.18↓3\.32\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!3\.32$\}\}\}72\.40†↑2\.20\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.20$\}\}\}81\.32↑1\.71\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.71$\}\}\}55\.03†↑0\.93\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.93$\}\}\}55\.35†↑0\.60\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.60$\}\}\}88\.55↑0\.72\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.72$\}\}\}95\.84↑0\.74\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.74$\}\}\}AgentVerse41\.99↑3\.31\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.31$\}\}\}52\.85†↑3\.62\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.62$\}\}\}71\.97↑4\.68\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.68$\}\}\}78\.44†↑0\.94\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.94$\}\}\}68\.20†↓2\.00\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.00$\}\}\}79\.42↓0\.19\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.19$\}\}\}59\.25↑5\.15\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!5\.15$\}\}\}59\.83†↑5\.08\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!5\.08$\}\}\}87\.74†↓0\.09\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.09$\}\}\}95\.86†↑0\.76\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.76$\}\}\}AFlow41\.83†↑3\.15\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.15$\}\}\}50\.36†↑1\.13\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.13$\}\}\}68\.18↑0\.89\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.89$\}\}\}76\.04↓1\.46\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.46$\}\}\}65\.00†↓5\.20\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!5\.20$\}\}\}76\.53†↓3\.08\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!3\.08$\}\}\}42\.83↓11\.27\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!11\.27$\}\}\}42\.46†↓12\.29\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!12\.29$\}\}\}87\.92†↑0\.09\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.09$\}\}\}96\.51†↑1\.41\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.41$\}\}\}GPTSwarm36\.20†↓2\.48\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.48$\}\}\}47\.37†↓1\.86\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.86$\}\}\}70\.45↑3\.16\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.16$\}\}\}78\.39†↑0\.89\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.89$\}\}\}69\.20†↓1\.00\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.00$\}\}\}79\.90↑0\.29\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.29$\}\}\}51\.63†↓2\.47\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.47$\}\}\}52\.43↓2\.32\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.32$\}\}\}88\.17†↑0\.34\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.34$\}\}\}95\.96†↑0\.86\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.86$\}\}\}MAGRPO40\.13†↑1\.45\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.45$\}\}\}51\.74↑2\.51\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.51$\}\}\}65\.91↓1\.38\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.38$\}\}\}76\.44↓1\.06\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.06$\}\}\}65\.00†↓5\.20\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!5\.20$\}\}\}76\.01†↓3\.60\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!3\.60$\}\}\}49\.93↓4\.17\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!4\.17$\}\}\}50\.48†↓4\.27\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!4\.27$\}\}\}88\.58†↑0\.75\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.75$\}\}\}95\.82†↑0\.72\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.72$\}\}\}MAPoRL40\.59†↑1\.91\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.91$\}\}\}52\.74†↑3\.51\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.51$\}\}\}63\.64†↓3\.65\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!3\.65$\}\}\}75\.34†↓2\.16\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.16$\}\}\}70\.80†↑0\.60\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.60$\}\}\}80\.87†↑1\.26\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.26$\}\}\}53\.25†↓0\.85\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.85$\}\}\}53\.27†↓1\.48\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.48$\}\}\}86\.37†↓1\.46\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.46$\}\}\}89\.19†↓5\.91\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!5\.91$\}\}\}AT\-GRPO41\.54↑2\.86\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!2\.86$\}\}\}53\.51†↑4\.28\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.28$\}\}\}64\.39†↓2\.90\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!2\.90$\}\}\}76\.86↓0\.64\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!0\.64$\}\}\}74\.20†↑4\.00\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.00$\}\}\}83\.70†↑4\.09\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.09$\}\}\}57\.91†↑3\.81\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.81$\}\}\}58\.74†↑3\.99\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.99$\}\}\}86\.60†↓1\.23\{\}\_\{\{\\color\[rgb\]\{0\.0039,0\.7031,0\.7266\}\\scalebox\{1\.1\}\{$\\scriptstyle\\downarrow\\\!1\.23$\}\}\}95\.85†↑0\.75\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.75$\}\}\}CL\.42\.37↑3\.69\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.69$\}\}\}53\.43↑4\.20\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.20$\}\}\}71\.97↑4\.68\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.68$\}\}\}81\.15↑3\.65\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!3\.65$\}\}\}78\.40↑8\.20\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!8\.20$\}\}\}85\.25↑5\.64\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!5\.64$\}\}\}59\.39↑5\.29\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!5\.29$\}\}\}59\.63↑4\.88\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!4\.88$\}\}\}88\.90↑1\.07\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!1\.07$\}\}\}95\.90↑0\.80\{\}\_\{\{\\color\[rgb\]\{0\.9453,0\.375,0\.2031\}\\scalebox\{1\.1\}\{$\\scriptstyle\\uparrow\\\!0\.80$\}\}\}

As shown in Table[1](https://arxiv.org/html/2610.12927#S3.T1),collaborative learning achieves the highest average performance across five benchmarks, with best or joint\-best results on most metrics\. With a total parameter budget of 13\.7B, it outperforms the original Qwen3\-14B on all ten metrics, with an average absolute gain of 2\.38%, and surpasses SE\-RL on eight\. It also outperforms the 24B homogeneous MAS without collaborative training on all ten metrics, despite using fewer parameters\. Compared with Homo\. CL, which trains homogeneous models using the same procedure, the heterogeneous collaborative system achieves higher scores across all ten metrics, with an average absolute gain of 6\.02%\. Its average absolute gains over established MAS and MARL baselines range from 2\.08% to 6\.87% across the ten metrics, demonstrating competitive performance against both classes\.

Trajectory analysis reveals task\-specific characteristics of collaboration and how intermediate behaviors translate into final outcomes\. On MuSiQue, subsequent interactions correct initial errors, while on MATH, the trained heterogeneous models better preserve correct intermediate answers than MAPoRL\. On MuSiQue and GSM\-Hard, differences in partial\-match scores on questions missed by both the collaborative system and the baseline help explain why accuracy and F1 rankings do not fully align\. Code\-generation performance varies across task subsets: the collaborative system solves more MBPP tasks but fewer HumanEval tasks than SE\-RL\. On most tasks solved only by SE\-RL, it produces no passing intermediate program, suggesting that this gap primarily reflects failure to generate correct solutions rather than the loss of correct candidates during subsequent interactions\. On Conifer, the collaborative system achieves the highest Coverage and almost always maintains compliance in the final output when an intermediate answer already satisfies all constraints\.

### 3\.3How Much Does Each Mechanism Contribute to Performance?

As shown in Figure[3](https://arxiv.org/html/2610.12927#S3.F3), we compare three ablation variants: without collaborative training \(w/o CL\.\), without protocol distillation \(w/o PD\.\), and without the step\-level score \(w/o SLS\.\)\. Averaged over the ten reported metrics, these variants incur absolute score drops of 9\.37, 8\.85, and 5\.41 points, respectively, and each underperforms the full method on every metric\. These results indicate thatheterogeneity alone is insufficient to achieve the full performance gains, and that both protocol distillation and step\-level scoring contribute meaningfully to learned collaboration\.

Figure 3:Ablation study of the mechanism across five benchmarks\.We compare a zero\-shot variant that receives no training \(w/o CL\.\), a variant without Protocol Distillation \(w/o PD\.\), and a variant without the step\-level score \(w/o SLS\.\)\. Arrows indicate the gain ofcollaborative learningover the arithmetic mean of the available variants\. For each benchmark,denotes Acc\., Wtd\., and Cov\., whiledenotes the corresponding second metric \(F1, Mac\., or Expl\.\)\.Beyond aggregate performance, trajectory\-level behavior provides a more direct view of how the two training components shape collaboration throughout collaborative reasoning\. For example, without protocol distillation, first\-answer EM on MuSiQue increases from 21\.02% to 25\.24%, yet the number of initially incorrect answers corrected by the final response falls from 594 to 171\. Meanwhile, the proportion of Conifer trajectories that initially satisfy all constraints but fail at termination increases, and protocol\-following failures on MATH also become more frequent\. These observations suggest that protocol distillation primarily supports effective downstream interaction rather than simply improving the quality of the initial response\. Similarly, removing the step\-level score increases the average MuSiQue trajectory length from 3\.4 to 3\.8 steps while reducing the number of corrected answers by 18\.5%, indicating that step\-level feedback helps models use interaction steps more productively and achieve more corrections with fewer interactions on this task\.

### 3\.4How Efficient and Effective Is Collaborative Reasoning?

Alongside improvements in task performance, collaborative learning leads to more efficient inference\. As shown in Figure[4](https://arxiv.org/html/2610.12927#S3.F4), the collaborative system requires fewer model invocations than MAD, AgentVerse, AFlow, and GPTSwarm across all five benchmarks, with reductions ranging from 39\.1% to 84\.8%\. Specifically, it requires an average of 4\.0, 4\.1, 1\.8, 3\.0, and 3\.3 invocations per question on MuSiQue, GSM\-Hard, MultiPL\-E, MATH, and Conifer, respectively\. This efficiency advantage also extends to the evaluated MARL baselines, with invocation reductions ranging from 46\.7% to 86\.0%\. In terms of parameter\-weighted output tokens,the collaborative system achieves the lowest count among the compared methods on four of five benchmarks\. Compared with the MARL baselines, it further reduces parameter\-weighted output tokens by 8\.2%–89\.4%\.

We further compare reasoning\-overlap rates before and after collaborative learning to examine redundancy across successive reasoning steps\. On MuSiQue, the overlap rates of the three collaborating models decrease from 48\.3%, 24\.1%, and 55\.2% to 0\.2%, 0\.3%, and 0\.5%, respectively\. On MultiPL\-E, they decrease from 34\.8%, 2\.3%, and 17\.6% to 7\.4%, 1\.8%, and 10\.7%\. These results suggest thatcollaborative learning substantially reduces repetitive reasoning across model interactions, yielding less redundant subsequent contributions throughout the process\.

To further assess collaborative effectiveness, we measure correct\-answer retention, defined as the proportion of trajectories whose final answer remains correct among those containing a correct intermediate candidate\. On MuSiQue, GSM\-Hard, MultiPL\-E, and MATH, the trained heterogeneous models achieve 80\.42%, 97\.94%, 98\.26%, and 98\.00% retention, respectively, ranking first on the latter two tasks, second on GSM\-Hard, and third on MuSiQue among comparison methods with recoverable intermediate candidates555Conifer is excluded because its open\-ended outputs lack an intermediate\-candidate correctness criterion\.\. These results show thatthe trained models generally preserve available correct intermediate answers throughout subsequent collaborative interactions\.

Figure 4:Computation cost versus task performance\.The center of each circle indicates task performance and parameter\-weighted output tokens for the collaborative system and four multi\-agent baselines across five benchmarks; its area is proportional to the number of model invocations\.
### 3\.5What Collaboration Patterns Emerge from Collaborative Learning?

Figure 5:Post\-training interaction topologies across tasks\.,,, anddenote short linear, long linear, short cyclic, and long cyclic trajectories, respectively\. Bar height indicates the proportion of each trajectory type, while depth indicates the average number of interaction steps\.To examine how collaborative learning reorganizes model interactions across tasks, we compare trajectory topologies and interaction lengths before and after training\. We categorize trajectories as short linear, long linear, short cyclic, or long cyclic\. Linear trajectories involve each model at most once, with at most two interaction steps considered short; cyclic trajectories revisit a previously participating model, with at most four steps considered short\. On MuSiQue, the proportion of long linear trajectories increases from 9\.18% to 60\.12%, showing a shift toward longer linear interactions\. In contrast, this proportion decreases from 10\.95% to 8\.58% on MultiPL\-E\. Short cyclic trajectories evolve differently across tasks: their average interaction length decreases from 4\.00 to 3\.67 steps on GSM\-Hard, but increases from 3\.33 to 4\.00 on MATH\. These results show that collaborative learning does not uniformly increase interaction complexity or length, but instead induces task\-dependent collaboration topologies rather than converging to a single universal interaction pattern\.

Task\-dependent functional specialization also emerges across models during collaborative learning\.On MuSiQue, Qwen3\-4B’s answer\-revision rate increases from 13% to 69%, its handoff rate from 48% to 96%, and its termination rate decreases from 50% to 4%, indicating a shift toward intermediate revision and handoff\. Meanwhile, Qwen3\-8B increasingly provides the final answer, with its share rising from 13\.0% to 58\.4%\. This division of labor changes across tasks: on GSM\-Hard, Qwen3\-4B instead becomes the more frequent final\-answer provider, with its share increasing from 34\.09% to 64\.39%\. Thus, the same model can assume different functions across tasks, revealing task\-dependent divisions of labor rather than a fixed collaboration structure\.

## 4Related Work

Reasoning in LLMs\.Existing work improves LLM reasoning by eliciting existing capabilities or strengthening them through training\([Zhao et al\., 2026a](https://arxiv.org/html/2610.12927#bib.bib41);[Li et al\., 2026c](https://arxiv.org/html/2610.12927#bib.bib35);[Liu et al\., 2026a](https://arxiv.org/html/2610.12927#bib.bib44)\)\. Inference\-time methods use chain\-of\-thought prompting\([Wei et al\., 2022](https://arxiv.org/html/2610.12927#bib.bib76)\), systematic exploration\([Hariri et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib53);[Li et al\., 2026a](https://arxiv.org/html/2610.12927#bib.bib54)\), evaluation\([Wu and Raghunathan, 2026](https://arxiv.org/html/2610.12927#bib.bib47)\), and revision\([Yang et al\., 2026a](https://arxiv.org/html/2610.12927#bib.bib33)\); training\-based methods learn from bootstrapped demonstrations\([Zelikman et al\., 2022](https://arxiv.org/html/2610.12927#bib.bib71);[Dong et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib48)\), distill teacher reasoning\([Yang et al\., 2025b](https://arxiv.org/html/2610.12927#bib.bib5);[Liu et al\., 2026c](https://arxiv.org/html/2610.12927#bib.bib34);[Fu et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib43)\), or use outcome\- and process\-level supervision\. Yet these methods remain bounded by a single model’s knowledge and reasoning biases, leaving blind spots unaddressed and allowing intermediate errors to persist through subsequent reasoning\([Xie et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib49)\)\.

From Single\-Model Reasoning to Multi\-Model Composition\.These limitations motivate moving beyond single\-model reasoning toward multi\-model composition, which exploits complementary knowledge and task\-specific strengths across models\([Yang et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib39);[Zhang et al\., 2026c](https://arxiv.org/html/2610.12927#bib.bib50);[Huot et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib36)\)\. Existing approaches select models suited to an input\([Ong et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib18);[Cao et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib38);[Feng et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib37);[Fisch et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib46)\), distribute tasks or subtasks\([Agashe et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib31);[Chen et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib55)\), organize sequential model invocations\([Dekoninck et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib19);[Li et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib56)\), determine when additional models are needed\([Rajib et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib32);[Qin et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib57)\), and aggregate candidate outputs\([Zhang et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib51)\)\. However, system\-level composition does not necessarily constitute collaborative reasoning in which models jointly develop, evaluate, and revise a shared solution\.

Multi\-Model Collaborative Reasoning\.Collaborative reasoning directly addresses how models jointly develop solutions through interaction\. Existing approaches range from hand\-designed roles and protocols to optimized workflows\([Hu et al\., 2025](https://arxiv.org/html/2610.12927#bib.bib25);[Zhou et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib16);[Lee et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib45);[Koh et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib52)\)and learned collaborative policies\([Xue et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib27);[Zhang et al\., 2026a](https://arxiv.org/html/2610.12927#bib.bib58)\)\. Representative systems, however, often retain an explicitly prescribed division of labor\([Zhao et al\., 2026b](https://arxiv.org/html/2610.12927#bib.bib28);[Feng et al\., 2026a](https://arxiv.org/html/2610.12927#bib.bib40);[Ding et al\., 2026](https://arxiv.org/html/2610.12927#bib.bib42)\)\. In contrast, we train heterogeneous small models to collaborate without predefined roles under a total parameter budget comparable to that of a single large model\.

## 5Conclusion

We introduced a collaborative learning framework for reasoning across heterogeneous models\. Model slicing replaces a large model with smaller heterogeneous models under a comparable parameter budget, enabling reasoning\-state isolation through task\-state handoffs\. Pyramid supervision enables cross\-model collaboration by combining protocol distillation for interaction initialization with step\-level supervision over model\-generated trajectories\. Across four task categories, the trained heterogeneous models match or outperform the large model and MAS and MARL baselines, while requiring fewer model invocations than multi\-agent frameworks\. Analysis reveals reduced reasoning redundancy, stronger correct\-answer retention, and task\-dependent interaction topologies and functional specialization\. These results suggest that collaborative learning can transform model heterogeneity into complementary reasoning capabilities across distinct model parameterizations\.

Our study covers three model sizes and five benchmarks, leaving generalization to larger model pools, longer interaction horizons, and unseen task distributions unexplored\. Pyramid supervision depends on inspector and evaluator quality, whose biases may affect step\-level feedback\. Comparable parameter counts do not imply equal computational cost, as collaborative inference adds latency and memory overhead beyond our parameter\-weighted output\-token metric\. Future work will study generalization and robust supervision through multiple inspectors or uncertainty\-aware aggregation\.

## References

- Abdulaalet al\.\(2025\)A\. Abdulaal, C\. Jin, N\. Montaña\-Brown, A\. P\. Gema, D\. Castro, D\. Alexander, P\. Teare, T\. Diethe, D\. Oglic, and A\. SaseendranBalancing act: diversity and consistency in large language model ensembles\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/a5554e55d7a21e62d0d7a028ec0ea1c7-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p2.1)\.
- Agasheet al\.\(2025\)S\. Agashe, K\. Wong, V\. Tu, J\. Yang, A\. Li, and X\. E\. WangAgent S2: a compositional generalist\-specialist framework for computer use agents\.InConference on Language Modeling \(COLM\),External Links:[Link](https://openreview.net/forum?id=zg5is4GJ3R)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Bestaet al\.\(2024\)M\. Besta, N\. Blach, A\. Kubicek, R\. Gerstenberger, M\. Podstawski, L\. Gianinazzi, J\. Gajda, T\. Lehmann, H\. Niewiadomski, P\. Nyczyk, and T\. HoeflerGraph of thoughts: solving elaborate problems with large language models\.InAAAI Conference on Artificial Intelligence \(AAAI\),External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/29720)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Caoet al\.\(2026\)Q\. Cao, S\. Zhang, R\. Zhou, R\. Zhang, P\. Qin, and P\. XieModels under SCOPE: scalable and controllable routing via pre\-hoc reasoning\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://arxiv.org/abs/2601.22323)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Cassanoet al\.\(2023\)F\. Cassano, J\. Gouwar, D\. Nguyen, S\. Nguyen, L\. Phipps\-Costin, D\. Pinckney, M\. Yee, Y\. Zi, C\. J\. Anderson, M\. Q\. Feldman, A\. Guha, M\. Greenberg, and A\. JangdaMultiPL\-E: a scalable and polyglot approach to benchmarking neural code generation\.InIEEE Transactions on Software Engineering \(TSE\),External Links:[Link](https://doi.org/10.1109/TSE.2023.3267446)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p3.1)\.
- Chenet al\.\(2023\)B\. Chen, C\. Shu, E\. Shareghi, N\. Collier, K\. Narasimhan, and S\. YaoFireAct: toward language agent fine\-tuning\.InarXiv preprint arXiv:2310\.05915,External Links:[Link](https://arxiv.org/abs/2310.05915)Cited by:[§2\.2\.2](https://arxiv.org/html/2610.12927#S2.SS2.SSS2.p6.1)\.
- Chenet al\.\(2026\)G\. Chen, S\. Yang, C\. Li, W\. Liu, J\. Luan, and Z\. XuEnd\-to\-end optimization of LLM\-driven multi\-agent search systems via heterogeneous\-group\-based reinforcement learning\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2026.acl-long.1399/)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Chenet al\.\(2024a\)J\. Chen, S\. Saha, and M\. BansalReConcile: round\-table conference improves reasoning via consensus among diverse LLMs\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2024.acl-long.381/)Cited by:[§2\.1](https://arxiv.org/html/2610.12927#S2.SS1.p1.1)\.
- Chenet al\.\(2024b\)J\. Chen, S\. Saha, E\. Stengel\-Eskin, and M\. BansalMAGDi: structured distillation of multi\-agent interaction graphs improves reasoning in smaller language models\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://proceedings.mlr.press/v235/chen24ah.html)Cited by:[§2\.2\.2](https://arxiv.org/html/2610.12927#S2.SS2.SSS2.p2.1)\.
- Chenet al\.\(2024c\)W\. Chen, Y\. Su, J\. Zuo, C\. Yang, C\. Yuan, C\. Chan, H\. Yu, Y\. Lu, Y\. Hung, C\. Qian, Y\. Qin, X\. Cong, R\. Xie, Z\. Liu, M\. Sun, and J\. ZhouAgentVerse: facilitating multi\-agent collaboration and exploring emergent behaviors\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/578e65cdee35d00c708d4c64bce32971-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1)\.
- Chenet al\.\(2025\)W\. Chen, Z\. You, R\. Li, Y\. Guan, C\. Qian, C\. Zhao, C\. Yang, R\. Xie, Z\. Liu, and M\. SunInternet of agents: weaving a web of heterogeneous agents for collaborative intelligence\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=o1Et3MogPw)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p2.1)\.
- Cuiet al\.\(2026\)G\. Cui, L\. Yuan, Z\. Wang, H\. Wang, Y\. Zhang, J\. Chen, W\. Li, B\. He, Y\. Fan, T\. Yu, Q\. Xu, W\. Chen, J\. Yuan, H\. Chen, K\. Zhang, X\. Lv, S\. Wang, Y\. Yao, X\. Han, H\. Peng, Y\. Cheng, Z\. Liu, M\. Sun, B\. Zhou, and N\. DingProcess reinforcement through implicit rewards\.InTransactions on Machine Learning Research \(TMLR\),External Links:[Link](https://openreview.net/forum?id=9SkkifLopZ)Cited by:[§2\.2\.2](https://arxiv.org/html/2610.12927#S2.SS2.SSS2.p15.1)\.
- Dekonincket al\.\(2025\)J\. Dekoninck, M\. Baader, and M\. VechevA unified approach to routing and cascading for LLMs\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://proceedings.mlr.press/v267/dekoninck25a.html)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Dinget al\.\(2026\)J\. Ding, R\. Sun, X\. Zhang, Z\. Zhang, and X\. LiuAPEx: distillation of agent procedural experience for adaptive deep research question answering\.InarXiv preprint arXiv:2609\.02253,External Links:[Link](https://arxiv.org/abs/2609.02253)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Donget al\.\(2026\)L\. Dong, D\. Zhang, Y\. Sun, Q\. Wang, Y\. Wang, L\. Chen, and J\. ZhuBetter starts, better ends: bootstrapped iterative self\-reasoning distillation for compressed reasoning\.InarXiv preprint arXiv:2607\.15736,External Links:[Link](https://arxiv.org/abs/2607.15736)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Duet al\.\(2024\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://proceedings.mlr.press/v235/du24e.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1)\.
- Fenget al\.\(2026a\)L\. Feng, L\. Zheng, S\. He, F\. Zhang, and B\. AnDr\. MAS: stable reinforcement learning for multi\-agent LLM systems\.InarXiv preprint arXiv:2602\.08847,External Links:[Link](https://arxiv.org/abs/2602.08847)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Fenget al\.\(2026b\)T\. Feng, F\. Yu, H\. Zhang, Z\. Dai, L\. Yuan, Z\. Lei, W\. Zhang, K\. Zhu, H\. Yue, K\. Xuan, G\. Liu, and J\. YouLLMRouter: unified infrastructure for developing, evaluating, and deploying LLM routers\.InarXiv preprint arXiv:2608\.06867,External Links:[Link](https://arxiv.org/abs/2608.06867)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Fischet al\.\(2026\)A\. Fisch, S\. Trivedi, F\. Huot, W\. W\. Cohen, M\. Kaisers, M\. Lapata, K\. Larson, and J\. EisensteinPandora’s AI model routing box: efficient allocation with costly value estimation\.InarXiv preprint arXiv:2608\.20316,External Links:[Link](https://arxiv.org/abs/2608.20316)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Foersteret al\.\(2018\)J\. Foerster, G\. Farquhar, T\. Afouras, N\. Nardelli, and S\. WhitesonCounterfactual multi\-agent policy gradients\.InAAAI Conference on Artificial Intelligence \(AAAI\),External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/11794)Cited by:[§2\.2\.2](https://arxiv.org/html/2610.12927#S2.SS2.SSS2.p10.1)\.
- Fuet al\.\(2026\)Z\. Fu, B\. He, Y\. Zuo, H\. Huang, J\. Zhang, R\. Xiao, C\. Qian, Q\. Luo, H\. Gao, Y\. Wang, Z\. Liu, N\. Ding, and C\. XiaoRethinking on\-policy distillation of large language models II: one training example\.InarXiv preprint arXiv:2609\.04172,External Links:[Link](https://arxiv.org/abs/2609.04172)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Gaoet al\.\(2023\)L\. Gao, A\. Madaan, S\. Zhou, U\. Alon, P\. Liu, Y\. Yang, J\. Callan, and G\. NeubigPAL: program\-aided language models\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://proceedings.mlr.press/v202/gao23f.html)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p3.1)\.
- Haririet al\.\(2026\)M\. Hariri, W\. Chen, N\. Shahini, V\. Singh, K\. Ye, A\. Samandar, D\. Ganguly, S\. Sankar, Y\. Zhang, S\. Wang, J\. Peng, B\. Zhang, M\. Hinczewski, and V\. ChaudharyTest\-time scaling in reasoning LLMs: inference regimes, evaluation, and reproducibility\.InarXiv preprint arXiv:2608\.04001,External Links:[Link](https://arxiv.org/abs/2608.04001)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InConference on Neural Information Processing Systems, Datasets and Benchmarks Track \(NeurIPS\),External Links:[Link](https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/hash/be83ab3ecd0db773eb2dc1b0a17836a1-Abstract-round2.html)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p3.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p4.1)\.
- Huet al\.\(2025\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/36b7acf6f6010652b3f2a433774a66fe-Abstract-Conference.html)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Huanget al\.\(2024\)J\. Huang, X\. Chen, S\. Mishra, H\. S\. Zheng, A\. Yu, X\. Song, and D\. ZhouLarge language models cannot self\-correct reasoning yet\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/8b4add8b0aa8749d80a34ca5d941c355-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Huotet al\.\(2026\)F\. Huot, M\. Kaisers, and M\. LapataWhen is routing meaningful? diversity and robustness in language model societies\.InarXiv preprint arXiv:2607\.09197,External Links:[Link](https://arxiv.org/abs/2607.09197)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Kim and Rush \(2016\)Y\. Kim and A\. M\. RushSequence\-level knowledge distillation\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://aclanthology.org/D16-1139/)Cited by:[§2\.2\.2](https://arxiv.org/html/2610.12927#S2.SS2.SSS2.p6.1)\.
- Kohet al\.\(2026\)D\. Koh, G\. Kim, J\. Bak, S\. Na, T\. Yang, H\. Jang, and C\. ParkGlobal optimization and inference\-time region grafting for agentic workflows\.InarXiv preprint arXiv:2608\.02353,External Links:[Link](https://arxiv.org/abs/2608.02353)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Kumaret al\.\(2025\)A\. Kumar, V\. Zhuang, R\. Agarwal, Y\. Su, J\. D\. Co\-Reyes, A\. Singh, K\. Baumli, S\. Iqbal, C\. Bishop, R\. Roelofs, L\. Zhang, K\. McKinney, D\. Shrivastava, C\. Paduraru, G\. Tucker, D\. Precup, F\. Behbahani, and A\. FaustTraining language models to self\-correct via reinforcement learning\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/871ac99fdc5282d0301934d23945ebaa-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Leeet al\.\(2026\)J\. H\. Lee, K\. H\. Lee, and G\. H\. YooMEGA: self\-evolving agent optimization infrastructure via wisdom graph\.InarXiv preprint arXiv:2608\.10504,External Links:[Link](https://arxiv.org/abs/2608.10504)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Liet al\.\(2026a\)M\. Li, J\. Qian, and W\. MouPredicting and improving test\-time scaling laws via reward tail\-guided search\.InarXiv preprint arXiv:2602\.01485,External Links:[Link](https://arxiv.org/abs/2602.01485)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Liet al\.\(2026b\)S\. Li, A\. M\. Abdelmoniem, and S\. WangProgRouter: online progress\-guided orchestration for multi\-agent LLM workflows under quality\-cost tradeoffs\.InFindings of the Association for Computational Linguistics: Conference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://arxiv.org/abs/2608.25992)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Liet al\.\(2026c\)Y\. Li, S\. Yavuz, and S\. JotyRISE: recursive improvement via self\-extrapolating policy distillation\.InarXiv preprint arXiv:2609\.05295,External Links:[Link](https://arxiv.org/abs/2609.05295)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Lianget al\.\(2024\)T\. Liang, Z\. He, W\. Jiao, X\. Wang, Y\. Wang, R\. Wang, Y\. Yang, S\. Shi, and Z\. TuEncouraging divergent thinking in large language models through multi\-agent debate\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://aclanthology.org/2024.emnlp-main.992/)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/aca97732e30bcf1303bc22ac3924fd16-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p3.1)\.
- Liuet al\.\(2026a\)B\. Liu, S\. Yu, Y\. Jiang, A\. Qu, A\. Zhao, Z\. Liu, J\. Kim, Z\. Zhou, S\. Kim, T\. Ren, M\. Liu, H\. Yu, Z\. Chen, W\. Shi, P\. P\. Liang, L\. Zettlemoyer, Y\. Choi, and N\. JaquesSPADE: self\-play in adaptive synthetic executable environments\.InarXiv preprint arXiv:2608\.19197,External Links:[Link](https://arxiv.org/abs/2608.19197)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Liuet al\.\(2026b\)S\. Liu, Z\. Liang, X\. Lyu, and C\. AmatoLLM collaboration with multi\-agent reinforcement learning\.InAAAI Conference on Artificial Intelligence \(AAAI\),External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/40487)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p2.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1)\.
- Liuet al\.\(2026c\)Z\. Liu, X\. Xu, M\. S\. Seyfioglu, P\. Xu, and K\. BouyarmaneExtremely sparse supervision incentivizes reasoning ability\.InarXiv preprint arXiv:2609\.04565,External Links:[Link](https://arxiv.org/abs/2609.04565)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Muennighoffet al\.\(2025\)N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candès, and T\. Hashimotos1: simple test\-time scaling\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://aclanthology.org/2025.emnlp-main.1025/)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Onget al\.\(2025\)I\. Ong, A\. Almahairi, V\. Wu, W\. Chiang, T\. Wu, J\. E\. Gonzalez, M\. Kadous, and I\. StoicaRouteLLM: learning to route LLMs from preference data\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/5503a7c69d48a2f86fc00b3dc09de686-Abstract-Conference.html)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Parket al\.\(2025\)C\. Park, S\. Han, X\. Guo, A\. E\. Ozdaglar, K\. Zhang, and J\. KimMAPoRL: multi\-agent post\-co\-training for collaborative large language models with reinforcement learning\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2025.acl-long.1459/)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1)\.
- Qinet al\.\(2026\)P\. Qin, Q\. Cao, and P\. XieATLAS: agentic test\-time learning\-to\-allocate scaling\.InarXiv preprint arXiv:2606\.01667,External Links:[Link](https://arxiv.org/abs/2606.01667)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Rajibet al\.\(2026\)R\. H\. Rajib, M\. Zheng, and Q\. LouLearning what to retain: gated\-memory routing for efficient collaboration in multi\-agent LLM systems\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://arxiv.org/abs/2609.00237)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Setluret al\.\(2025\)A\. Setlur, C\. Nagpal, A\. Fisch, X\. Geng, J\. Eisenstein, R\. Agarwal, A\. Agarwal, J\. Berant, and A\. KumarRewarding progress: scaling automated process verifiers for LLM reasoning\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/98711dea460bdefe0e651ca23ec98ba2-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p3.1)\.
- Snellet al\.\(2025\)C\. Snell, J\. Lee, K\. Xu, and A\. KumarScaling LLM test\-time compute optimally can be more effective than scaling parameters for reasoning\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/1b623663fd9b874366f3ce019fdfdd44-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Sunet al\.\(2024\)H\. Sun, L\. Liu, J\. Li, F\. Wang, B\. Dong, R\. Lin, and R\. HuangConifer: improving complex constrained instruction\-following ability of large language models\.InarXiv preprint arXiv:2404\.02823,External Links:[Link](https://arxiv.org/abs/2404.02823)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p3.1)\.
- Trivediet al\.\(2022\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalMuSiQue: multihop questions via single\-hop question composition\.InTransactions of the Association for Computational Linguistics \(TACL\),External Links:[Link](https://aclanthology.org/2022.tacl-1.31/)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p3.1)\.
- Wanget al\.\(2025\)J\. Wang, J\. Wang, B\. Athiwaratkun, C\. Zhang, and J\. Y\. ZouMixture\-of\-agents enhances large language model capabilities\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/5434be94e82c54327bb9dcaf7fca52b6-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p2.1)\.
- Wanget al\.\(2024a\)P\. Wang, L\. Li, Z\. Shao, R\. Xu, D\. Dai, Y\. Li, D\. Chen, Y\. Wu, and Z\. SuiMath\-Shepherd: verify and reinforce LLMs step\-by\-step without human annotations\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2024.acl-long.510/)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p3.1)\.
- Wanget al\.\(2024b\)Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang, T\. Li, M\. Ku, K\. Wang, A\. Zhuang, R\. Fan, X\. Yue, and W\. ChenMMLU\-Pro: a more robust and challenging multi\-task language understanding benchmark\.InConference on Neural Information Processing Systems, Datasets and Benchmarks Track \(NeurIPS\),External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/ad236edc564f3e3156e1b2feafb99a24-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.InConference on Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Wu and Raghunathan \(2026\)C\. H\. Wu and A\. RaghunathanSelf\-trained verification for training\- and test\-time self\-improvement\.InarXiv preprint arXiv:2605\.30290,External Links:[Link](https://arxiv.org/abs/2605.30290)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Wuet al\.\(2025\)Y\. Wu, Z\. Sun, S\. Li, S\. Welleck, and Y\. YangInference scaling laws: an empirical analysis of compute\-optimal inference for LLM problem\-solving\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/8c3caae2f725c8e2a55ecd600563d172-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Xieet al\.\(2026\)S\. Xie, J\. Liu, B\. Yang, W\. Yuan, F\. Yang, T\. Gao, and M\. WuFrom reasoning depth to reasoning breadth: evaluating multi\-point associative reasoning in large language models\.InarXiv preprint arXiv:2608\.10444,External Links:[Link](https://arxiv.org/abs/2608.10444)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Xuet al\.\(2024\)W\. Xu, G\. Zhu, X\. Zhao, L\. Pan, L\. Li, and W\. WangPride and prejudice: LLM amplifies self\-bias in self\-refinement\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2024.acl-long.826/)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Xueet al\.\(2026\)X\. Xue, Y\. Zhou, G\. Zhang, Z\. Zhang, Y\. Li, C\. Zhang, Z\. Yin, P\. Torr, W\. Ouyang, and L\. BaiCoMAS: co\-evolving multi\-agent systems via interaction rewards\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=ihwAzktmWc)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Yanget al\.\(2026a\)A\. Yang, Q\. Guo, Z\. Huang, Y\. Chen, S\. Qian, and J\. CaoSelf\-correcting long\-horizon search agents via tree\-structured memory\.InarXiv preprint arXiv:2608\.10676,External Links:[Link](https://arxiv.org/abs/2608.10676)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Yanget al\.\(2025a\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.InarXiv preprint arXiv:2505\.09388,External Links:[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1),[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p4.1),[footnote 3](https://arxiv.org/html/2610.12927#footnote3)\.
- Yanget al\.\(2025b\)L\. Yang, Z\. Yu, T\. Zhang, M\. Xu, J\. E\. Gonzalez, B\. Cui, and S\. YanSuperCorrect: advancing small LLM reasoning with thought template distillation and self\-correction\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/0967d7c8b171dd81b77c43067c02bebf-Abstract-Conference.html)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Yanget al\.\(2026b\)Y\. Yang, Y\. Bian, Y\. Tian, D\. Fu, T\. Huang, Y\. Shi, Z\. Xiao, N\. Vasconcelos, and Y\. LiCo\-RL: unsupervised reasoning emerges from diverse cohort in multi\-agent RL\.InarXiv preprint arXiv:2608\.17253,External Links:[Link](https://arxiv.org/abs/2608.17253)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Yinet al\.\(2023\)Z\. Yin, Q\. Sun, C\. Chang, Q\. Guo, J\. Dai, X\. Huang, and X\. QiuExchange\-of\-thought: enhancing large language model capabilities through cross\-model communication\.InConference on Empirical Methods in Natural Language Processing \(EMNLP\),External Links:[Link](https://aclanthology.org/2023.emnlp-main.936/)Cited by:[§2\.1](https://arxiv.org/html/2610.12927#S2.SS1.p1.1)\.
- Zelikmanet al\.\(2022\)E\. Zelikman, Y\. Wu, J\. Mu, and N\. GoodmanSTaR: bootstrapping reasoning with reasoning\.InConference on Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/639a9a172c044fbb64175b5fad42e9a5-Abstract-Conference.html)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Zenget al\.\(2024\)A\. Zeng, M\. Liu, R\. Lu, B\. Wang, X\. Liu, Y\. Dong, and J\. TangAgentTuning: enabling generalized agent abilities for LLMs\.InFindings of the Association for Computational Linguistics: Annual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2024.findings-acl.181/)Cited by:[§2\.2\.2](https://arxiv.org/html/2610.12927#S2.SS2.SSS2.p2.1)\.
- Zhanget al\.\(2024\)D\. Zhang, S\. Zhoubian, Z\. Hu, Y\. Yue, Y\. Dong, and J\. TangReST\-MCTS\*: LLM self\-training via process reward guided tree search\.InConference on Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/76ec4dc30e9faaf0e4b6093eaa377218-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Zhanget al\.\(2025\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang, B\. Zheng, B\. Liu, Y\. Luo, and C\. WuAFlow: automating agentic workflow generation\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/5492ecbce4439401798dcd2c90be94cd-Abstract-Conference.html)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1)\.
- Zhanget al\.\(2026a\)Y\. Zhang, Y\. Zhao, N\. Wang, Y\. Wu, J\. Chang, Y\. Chen, Q\. Wu, J\. Zhao, and H\. WangMetaAgent\-X: breaking the ceiling of automatic multi\-agent systems via end\-to\-end reinforcement learning\.InarXiv preprint arXiv:2605\.14212,External Links:[Link](https://arxiv.org/abs/2605.14212)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Zhanget al\.\(2026b\)Z\. Zhang, K\. Shi, Z\. Yuan, Z\. Wang, T\. Ma, K\. Murugesan, V\. Galassi, C\. Zhang, and Y\. YeAgentRouter: a knowledge\-graph\-guided LLM router for collaborative multi\-agent question answering\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2026.acl-long.33/)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Zhanget al\.\(2026c\)Z\. Zhang, Z\. Huang, G\. Li, H\. Wang, C\. Yuan, X\. Xia, D\. Wang, F\. Zhuang, S\. Ma, N\. Ding, Y\. Yang, J\. Li, and Y\. BanHeterogeneous agent collaborative reinforcement learning\.InarXiv preprint arXiv:2603\.02604,External Links:[Link](https://arxiv.org/abs/2603.02604)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p2.1)\.
- Zhaoet al\.\(2026a\)S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. GroverSelf\-distilled reasoner: on\-policy self\-distillation for large language models\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://openreview.net/forum?id=Jpxfof0EaS)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p1.1)\.
- Zhaoet al\.\(2026b\)Y\. Zhao, L\. Hu, Y\. Wang, M\. Hou, H\. Zhang, K\. Ding, and J\. ZhaoStronger\-MAS: multi\-agent reinforcement learning for collaborative LLMs\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=IdF6JqXWzx)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1),[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Zhenget al\.\(2025\)C\. Zheng, Z\. Zhang, B\. Zhang, R\. Lin, K\. Lu, B\. Yu, D\. Liu, J\. Zhou, and J\. LinProcessBench: identifying process errors in mathematical reasoning\.InAnnual Meeting of the Association for Computational Linguistics \(ACL\),External Links:[Link](https://aclanthology.org/2025.acl-long.50/)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Zhouet al\.\(2026\)H\. Zhou, X\. Wan, R\. Sun, H\. Palangi, S\. Iqbal, I\. Vulić, A\. Korhonen, and S\. ArikMulti\-agent design: optimizing agents with better prompts and topologies\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://iclr.cc/virtual/2026/poster/10010348)Cited by:[§4](https://arxiv.org/html/2610.12927#S4.p3.1)\.
- Zhouet al\.\(2024\)P\. Zhou, J\. Pujara, X\. Ren, X\. Chen, H\. Cheng, Q\. V\. Le, E\. H\. Chi, D\. Zhou, S\. Mishra, and H\. S\. ZhengSELF\-DISCOVER: large language models self\-compose reasoning structures\.InConference on Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/e41efb03e20ca3c231940a3c6917ef6f-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2610.12927#S1.p1.1)\.
- Zhugeet al\.\(2024\)M\. Zhuge, W\. Wang, L\. Kirsch, F\. Faccio, D\. Khizbullin, and J\. SchmidhuberGPTSwarm: language agents as optimizable graphs\.InInternational Conference on Machine Learning \(ICML\),External Links:[Link](https://proceedings.mlr.press/v235/zhuge24a.html)Cited by:[§3\.1](https://arxiv.org/html/2610.12927#S3.SS1.p2.1)\.