DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents

# Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents
Source: [https://arxiv.org/html/2608.18524](https://arxiv.org/html/2608.18524)
1\]ByteDance 2\]University of Science and Technology of China\\contribution\[⋆\]Equal contribution\\contribution\[†\]Corresponding author\\correspondenceYang Yang at , Yan Song at

Jiarui WangYang YangChuanbo ZhuFangda Chen Ziqi WuJingming CaiYan SongAffiliation:\[Affiliation:\[Email:[yang\.yves@bytedance\.com](mailto:[email protected])Email:[clksong@gmail\.com](mailto:[email protected])

###### Abstract

Equipping Large Language Models \(LLMs\) with multi\-turn tool\-calling capabilities is essential for building autonomous agents\. However, progress is fundamentally limited by the reliance on full\-length trajectory imitation\. For tasks involving multiple order\-independent sub\-goals, the optimal solution space forms a vast combinatorial diamond lattice\. Forcing this rich topology into monolithic trajectories causes a severetopological collapse, indiscriminately penalizing valid alternative explorations and severely degrading policy diversity\. To address this, we proposeDART\-SD\(Diamond\-topologyAwareRetrieval andTuning forSelf\-Distillation\), a novel framework that shifts the paradigm from global forcing to topology\-guided localized correction\. DART\-SD first models the execution process as a convergingInteraction\-State Transition Graph\(ISTG\), faithfully capturing the inherent diamond topology of successful and failed exploratory paths\. During autonomous rollouts, the framework identifies theCritical Topological Breakpoint\(CTB\) and retrieves success\-supported recovery references\. Finally, we introduce aprogressive self\-distillation paradigmthrough CTB\-guided localized supervision, ensuring that the training loss is calculated exclusively on the generated recovery steps while strictly protecting the valid reasoning prefix from destructive gradient updates\. Experiments on complex multi\-turn tool\-calling benchmarks demonstrate that DART\-SD significantly outperforms traditional full\-trajectory baselines\.

## 1Introduction

Equipping Large Language Models \(LLMs\) with multi\-turn tool\-calling capabilities is an essential milestone toward building autonomous agents capable of solving complex, real\-world tasks\[[32](https://arxiv.org/html/2608.18524#bib.bib14),[21](https://arxiv.org/html/2608.18524#bib.bib15),[30](https://arxiv.org/html/2608.18524#bib.bib33)\]\. Through sequential interactions with external environments, tool\-augmented agents can retrieve up\-to\-date information, execute code, and manipulate APIs to transcend their static parametric knowledge\[[4](https://arxiv.org/html/2608.18524#bib.bib39),[34](https://arxiv.org/html/2608.18524#bib.bib24),[8](https://arxiv.org/html/2608.18524#bib.bib25),[31](https://arxiv.org/html/2608.18524#bib.bib16),[3](https://arxiv.org/html/2608.18524#bib.bib35)\]\. Currently, state\-of\-the\-art tool\-calling performance is predominantly achieved by massive and expensive frontier models\[[6](https://arxiv.org/html/2608.18524#bib.bib21),[1](https://arxiv.org/html/2608.18524#bib.bib22)\]\. Consequently, agent distillation has gained significant research momentum, aiming to transfer these sophisticated planning and interaction capabilities into efficient, compact open\-source models\[[34](https://arxiv.org/html/2608.18524#bib.bib24),[39](https://arxiv.org/html/2608.18524#bib.bib23),[12](https://arxiv.org/html/2608.18524#bib.bib9)\]\.

![Refer to caption](https://arxiv.org/html/2608.18524v1/dart_fig2.png)Figure 1:Comparison of training paradigms\. \(a\) SFT Teacher Boosting applies an indiscriminate global loss, which overwrites valid exploration\. \(b\) Standard\-RL \(GRPO\) misassigns credit through a uniformly distributed reward spread\. \(c\) DART\-SD dynamically identifies the Critical Topological Breakpoints \(CTB\) and applies localized correction while preserving valid exploration\.The prevalent agent distillation paradigm relies on full trajectory imitation via Behavior Cloning \(BC\) or Supervised Fine\-Tuning \(SFT\)\[[22](https://arxiv.org/html/2608.18524#bib.bib17)\]\. In long\-horizon tasks, global forcing paradigms apply an indiscriminate global loss that overwrites the student model’s valid exploratory steps\[[17](https://arxiv.org/html/2608.18524#bib.bib26)\], causing models to memorize redundant trajectories instead of extracting the task’s core logical backbone\. Recent efforts attempt to explore hindsight\-based scaffolding or Reinforcement Learning \(RL\)\[[37](https://arxiv.org/html/2608.18524#bib.bib1),[44](https://arxiv.org/html/2608.18524#bib.bib12),[28](https://arxiv.org/html/2608.18524#bib.bib13)\]\. However, traditional hindsight methods\[[40](https://arxiv.org/html/2608.18524#bib.bib10)\]still treat multi\-turn interactions as strict linear sequences\. This rigid linearity forces models to confuse fatal errors with harmless exploration, severely degrading token efficiency and internal knowledge consistency\[[18](https://arxiv.org/html/2608.18524#bib.bib28),[45](https://arxiv.org/html/2608.18524#bib.bib27)\]\. Meanwhile, standard RL approaches\[[26](https://arxiv.org/html/2608.18524#bib.bib19),[29](https://arxiv.org/html/2608.18524#bib.bib44)\], such as Group Relative Policy Optimization \(GRPO\), frequently suffer from credit misassignment due to uniformly distributed rewards spread across all intermediate tool calls and inadvertently penalize valid steps within failed trajectories \(see Figure[1](https://arxiv.org/html/2608.18524#S1.F1)\)\.

The monolithic SFT and standard RL paradigms are fundamentally limited by misrepresenting the multi\-turn tool\-calling process as a collection of isolated linear trajectories\[[2](https://arxiv.org/html/2608.18524#bib.bib29),[11](https://arxiv.org/html/2608.18524#bib.bib30)\]\. This oversimplified perspective ignores the inherent graph structure of state transitions\[[9](https://arxiv.org/html/2608.18524#bib.bib32),[13](https://arxiv.org/html/2608.18524#bib.bib31)\]\. For tasks with order\-independent sub\-goals, distinct trajectories frequently intersect at shared intermediate states, structuring the optimal solution space into a vast combinatorial diamond lattice\[[7](https://arxiv.org/html/2608.18524#bib.bib34)\]\. Forcing models to flatten this interconnected topological space into single linear paths, traditional methods inevitably cause a severetopological collapse\. This structural oversight induces myopic credit assignment\[[29](https://arxiv.org/html/2608.18524#bib.bib44)\], arbitrarily penalizing valid alternative explorations and severely degrading policy diversity\[[15](https://arxiv.org/html/2608.18524#bib.bib37),[41](https://arxiv.org/html/2608.18524#bib.bib36)\]\.

To align with this topological reality, we proposeDART\-SD\(Diamond\-topologyAwareRetrieval andTuning forSelf\-Distillation\), a framework that shifts the paradigm from linear imitation to topology\-aware localized correction\. DART\-SD extracts a structural prior from teacher rollouts, instantiated as anInteraction\-State Transition Graph\(ISTG\) covering both successful and failed trajectories\. By defining nodes as cumulative interaction states rather than transient actions, the ISTG captures the diamond topology, naturally resolving order\-dependency conflicts\. During self\-distillation, DART\-SD projects student states onto the success\-reachable region to identify theCritical Topological Breakpoint\(CTB\) and its recovery anchors\. From each anchor, it searches the ISTG for a success\-supported continuation as a reference for recovery generation\. Finally, DART\-SD applies aprogressive self\-distillation paradigmthrough CTB\-guided localized supervision, ensuring that the loss is calculated only on recovery steps to protect valid reasoning prefixes\.

In summary, our main contributions are as follows:

- •We identify a fundamental topological collapse in existing agent\-training paradigms\. To resolve this, we introduce the Interaction\-State Transition Graph \(ISTG\), which represents tool execution through cumulative interaction states and captures the diamond topology induced by order\-independent exploration\.
- •We define theCritical Topological Breakpoint\(CTB\) by projecting student interaction states onto the empirical success\-reachable region\. We proposeCTB\-guided localized supervision, which retrieves success\-supported references and supervises the generated recovery steps while strictly preserving valid prefixes\.
- •We propose aprogressive self\-distillation paradigmthat repeatedly rolls out the student, identifies its evolving capability boundary, and performs CTB\-guided localized supervision\. This iterative process enables the student to continuously extend valid interaction prefixes and progressively master more complex tool\-use behaviors\.
- •Extensive experiments across five benchmarks and two model scales demonstrate that DART\-SD consistently outperforms supervised and reinforcement\-learning baselines, while reducing redundant tool calls and extending valid interaction prefixes\.

## 2Related Work

Distillation\-Based Paradigms\.Early attempts at building autonomous agents rely on Behavior Cloning \(BC\) or Supervised Fine\-Tuning \(SFT\) over complete expert trajectories\[[22](https://arxiv.org/html/2608.18524#bib.bib17)\]\. However, this global forcing paradigm forces students to mimic entire sequences, often leading to compounding errors in long\-horizon tasks as deviations propagate autoregressively\[[16](https://arxiv.org/html/2608.18524#bib.bib11)\]\. Furthermore, indiscriminate token\-level imitation triggers exposure bias\[[25](https://arxiv.org/html/2608.18524#bib.bib20)\]and overwrites the student’s valid exploration habits\. To address the limitations of static global forcing, recent distillation methods have shifted toward student\-centered and dynamic strategies\[[14](https://arxiv.org/html/2608.18524#bib.bib5)\]\. SCoRe\-SFT\[[16](https://arxiv.org/html/2608.18524#bib.bib11)\]provides post\-hoc scaffolding through student\-centered knowledge distillation, while On\-Policy Self\-Distillation \(OPSD\)\[[44](https://arxiv.org/html/2608.18524#bib.bib12)\]mitigates offline distribution shifts by dynamically rolling out and distilling trajectories from the current student policy\. Similarly, hindsight\-based distillation methods such as HINT\-SD\[[40](https://arxiv.org/html/2608.18524#bib.bib10)\]leverage targeted scaffolding for correction\.

Reinforcement Learning Paradigms\.To further refine policy optimization beyond distillation, Reinforcement Learning \(RL\) approaches leverage environment feedback to directly align agent behaviors\. Methods such as FTRL\-GRPO\[[37](https://arxiv.org/html/2608.18524#bib.bib1)\]and ToolRL\[[20](https://arxiv.org/html/2608.18524#bib.bib38)\]optimize agents using sparse outcome rewards and verifiable execution feedback\. However, sparse terminal signals in these methods often lead to severe credit misassignment, inadvertently penalizing valid intermediate steps within failed trajectories\[[27](https://arxiv.org/html/2608.18524#bib.bib7),[43](https://arxiv.org/html/2608.18524#bib.bib8),[5](https://arxiv.org/html/2608.18524#bib.bib6),[26](https://arxiv.org/html/2608.18524#bib.bib19),[37](https://arxiv.org/html/2608.18524#bib.bib1),[17](https://arxiv.org/html/2608.18524#bib.bib26)\]\. Even finer\-grained RL methods such as MatchTIR\[[23](https://arxiv.org/html/2608.18524#bib.bib41)\], which aligns trajectories via bipartite matching to provide turn\-level rewards, still fundamentally treat multi\-turn interactions as rigid linear sequences, thereby overlooking the inherent topology of exploration\. In contrast, our proposed DART\-SD shifts the paradigm from linear imitation and complex matching to topology\-aware localized correction\. By calculating the optimization loss exclusively on retrieved recovery steps, DART\-SD strictly protects valid reasoning prefixes from destructive gradient updates\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2608.18524v1/main.png)Figure 2:Overview of DART\-SD\. \(1\) DART\-SD constructs an ISTG from teacher rollouts, where main and auxiliary nodes model information acquisition and useless exploration\. \(2–3\) Failed student rollouts are collected and replayed in the same interaction\-state space, then projected onto the budget\-filtered success\-reachable regionℛx\+\\mathcal\{R\}\_\{x\}^\{\+\}, where the first projectable\-to\-non\-projectable transition defines the CTB\. \(4–5\) Conditioned on the retained student prefix and privileged teacher references, DART\-SD generates a recovery continuation after the CTB and applies localized supervision only to the generated assistant tokens\. \(6\) This CTB\-guided SFT loop is progressively repeated across self\-distillation rounds\.Figure[2](https://arxiv.org/html/2608.18524#S3.F2)provides an overview of the DART\-SD framework\. DART\-SD first constructs an ISTG from teacher rollouts, then identifies the CTB of failed student rollouts through success\-reachable projection\. The identified CTB is used to generate localized recovery continuations and to drive progressive self\-distillation\.

### 3\.1ISTG Abstraction and Construction

Information Atom Abstraction\.For each taskxx, useful facts extracted from tool responses are normalized into a task\-specific set of information atoms𝒦x\\mathcal\{K\}\_\{x\}\. Semantically equivalent responses share the same atom, while non\-informative responses contribute none\. We realize this in two stages\. A deterministic stage canonicalizes each tool response and decides whether it carries task\-usable data at all\. Parsing the response into fields, it is treated as non\-informative only when every field is either a status signal or an empty or placeholder value, so a payload retaining a single substantive value still qualifies as informative, whereas an error or not\-found message does not\. Non\-informative responses are collapsed per tool into a single class, which removes the bulk of the corpus without semantic judgement\.

A semantic stage then assigns atoms to the surviving candidates\. Leteedenote a tool call, with tool identitytl⁡\(e\)\\operatorname\{tl\}\(e\)and canonicalized responseo¯​\(e\)\\bar\{o\}\(e\)\. All candidates of a task are examined jointly, conditioned on the task question and one successful rollout, yielding a set\-valued atom map

αx:\(tl⁡\(e\),o¯​\(e\)\)⟼αx​\(e\)⊆𝒦x,\|αx​\(e\)\|≤1\.\\alpha\_\{x\}:\\ \\bigl\(\\operatorname\{tl\}\(e\),\\bar\{o\}\(e\)\\bigr\)\\ \\longmapsto\\ \\alpha\_\{x\}\(e\)\\subseteq\\mathcal\{K\}\_\{x\},\\qquad\\lvert\\alpha\_\{x\}\(e\)\\rvert\\leq 1\.\(1\)An informative response is mapped to the singleton containing its atom, while a non\-informative response, or one left unjudged, is mapped to∅\\varnothing\. Responses supplying the same fact are constrained to receive the same atom, so each element of𝒦x\\mathcal\{K\}\_\{x\}is the canonical representative of one class of information\-equivalent calls, namely those whose images underαx\\alpha\_\{x\}coincide and are non\-empty\. Judging a task’s candidates jointly is what makes this equivalence decidable: atoms are assigned with all alternatives in view, rather than labeling responses independently and relying on the labels to agree\.

The mapαx\\alpha\_\{x\}supplies the acquisition increment used in Eq\. \([4](https://arxiv.org/html/2608.18524#S3.E4)\)\. WritingBtB\_\{t\}for the tool call or concurrent call bundle executed at steptt,

ΔIt=\{k∈𝒦x\|∃e∈Bt,k∈αx\(e\),k∉It−1\},It=It−1∪ΔIt\.\\Delta I\_\{t\}=\\left\\\{k\\in\\mathcal\{K\}\_\{x\}\\;\\middle\|\\;\\exists e\\in B\_\{t\},\\;k\\in\\alpha\_\{x\}\(e\),\\;k\\notin I\_\{t\-1\}\\right\\\},\\qquad I\_\{t\}=I\_\{t\-1\}\\cup\\Delta I\_\{t\}\.\(2\)Two properties of this construction determine the resulting topology\. First,αx\\alpha\_\{x\}identifies responses across surface form and across tools, so the same fact returned as free text by one tool and as a structured record by another yields one atom; the two acquisition paths therefore reconverge at the same main information state instead of forking into states the projection would treat as unrelated\. Second, mapping non\-informative and unjudged responses to∅\\varnothingis what makes them contribute none: such a call contributes no element toΔ​It\\Delta I\_\{t\}in Eq\. \([2](https://arxiv.org/html/2608.18524#S3.E2)\) and leaves the information set unchanged, instead of receiving an identifier derived from its own text, which would manufacture a distinct information state that no evidence supports and inflate the information dimension of the graph\. Because only informative responses yield atoms, residual judgement errors predominantly omit an atom rather than invent one, leaving the graph sparser but never fabricating states or reachability\.

Main and Auxiliary Nodes\.For a taskxx, the interaction state at stepttis represented as

Xt=\(It,Ut\),X\_\{t\}=\(I\_\{t\},U\_\{t\}\),\(3\)whereItI\_\{t\}is the set of canonical information atoms acquired up to steptt, andUtU\_\{t\}records the multiset of useless operations performed since the most recent main node\.

The root stateX0=\(∅,∅\)X\_\{0\}=\(\\varnothing,\\varnothing\)is defined as a main node\. Fort≥1t\\geq 1, the type ofXtX\_\{t\}is determined by the preceding transition:

type⁡\(Xt\)=\{main,Δ​It≠∅,aux,Δ​It=∅​and​Ut≠∅\.\\operatorname\{type\}\(X\_\{t\}\)=\\begin\{cases\}\\mathrm\{main\},&\\Delta I\_\{t\}\\neq\\varnothing,\\\\ \\mathrm\{aux\},&\\Delta I\_\{t\}=\\varnothing\\ \\text\{and\}\\ U\_\{t\}\\neq\\varnothing\.\\end\{cases\}\(4\)whereΔ​It\\Delta I\_\{t\}is the set of previously unseen information atoms acquired at steptt, given by Eq\. \([2](https://arxiv.org/html/2608.18524#S3.E2)\)\.

Main nodes include mixed steps acquiring new information alongside useless operations and are represented withUt=∅U\_\{t\}=\\varnothing, whereas auxiliary nodes record useless operations after the main node without changing the information set\.

Interaction\-State Transition Graph \(ISTG\)\.For each taskxx, we construct a directed multigraph

Gx=\(Vx,Ex\)\.G\_\{x\}=\(V\_\{x\},E\_\{x\}\)\.\(5\)The graph is built from all successful and failed teacher rollouts\. All rollouts share a common root and terminate at either a success or a failure terminal\. Each vertexv∈Vxv\\in V\_\{x\}represents a stateX⁡\(v\)=\(I⁡\(v\),U⁡\(v\)\)X\(v\)=\(I\(v\),U\(v\)\)with typetype⁡\(v\)∈\{main,aux\}\\operatorname\{type\}\(v\)\\in\\\{\\mathrm\{main\},\\mathrm\{aux\}\\\}, and each edge represents one tool call or concurrent call bundle\. Parallel edges are retained\.

Main nodes form the information\-acquisition backbone, while auxiliary nodes represent useless exploration attached to the most recent main node\. Both node types and transitions are retained, since auxiliary exploration contributes to execution depth and matching\. The set\-valued information component allows order\-independent acquisition paths to diverge and reconverge at the same main information state, producing the diamond structures of the ISTG\. A student rollout is replayed under the same state\-update rule:

Pxs=\(s0,…,sT\),X⁡\(st\)=\(Its,Uts\)\.P\_\{x\}^\{s\}=\(s\_\{0\},\\ldots,s\_\{T\}\),\\qquad X\(s\_\{t\}\)=\(I\_\{t\}^\{s\},U\_\{t\}^\{s\}\)\.\(6\)
The node type ofsts\_\{t\}is determined by the preceding transition\. Teacher and student executions are therefore embedded in a shared interaction\-state space, enabling separate matching of information acquisition and useless exploration without requiring action\-wise correspondence\.

### 3\.2Success\-Reachable Projection and CTB

Empirical Success\-Reachable Region\.Let𝒯x\+\\mathcal\{T\}\_\{x\}^\{\+\}denote the successful teacher rollouts for taskxx, and letdxmind\_\{x\}^\{\\min\}be the minimum number of graph transitions from the root to a success terminal among these rollouts\. We define the task\-specific reachability budget as

Bx=min⁡\(dxmin\+Δx,Bxmax\),B\_\{x\}=\\min\\bigl\(d\_\{x\}^\{\\min\}\+\\Delta\_\{x\},\\;B\_\{x\}^\{\\max\}\\bigr\),\(7\)whereΔx\\Delta\_\{x\}is the task\-dependent allowance beyond the shortest successful depth andBxmaxB\_\{x\}^\{\\max\}is the budget cap\. For a nodevvon a successful rolloutτ\\tau, letrτ​\(v\)r\_\{\\tau\}\(v\)denote the number of graph transitions remaining fromvvto the success terminal ofτ\\tau\. The empirical success\-reachable region is defined by

ℛx\+=\{v∈Vx\|∃τ∈𝒯x\+,v∈τ,rτ\(v\)≤Bx\}\.\\mathcal\{R\}\_\{x\}^\{\+\}=\\left\\\{v\\in V\_\{x\}\\;\\middle\|\\;\\exists\\,\\tau\\in\\mathcal\{T\}\_\{x\}^\{\+\},\\;v\\in\\tau,\\;r\_\{\\tau\}\(v\)\\leq B\_\{x\}\\right\\\}\.\(8\)Thus,ℛx\+\\mathcal\{R\}\_\{x\}^\{\+\}contains both main and auxiliary nodes on successful teacher rollouts whose remaining distance to success is at mostBxB\_\{x\}\. The distance is measured backward from the success terminal along the rollout, rather than as forward depth from the root\. States observed exclusively on failed rollouts remain inGxG\_\{x\}, but are excluded fromℛx\+\\mathcal\{R\}\_\{x\}^\{\+\}and cannot serve as recovery anchors\.

Type\-specific State Projection\.For each student statest=\(Its,Uts\)s\_\{t\}=\(I\_\{t\}^\{s\},U\_\{t\}^\{s\}\), we further apply the projection rule associated with its node type\. Letℛx,main\+\\mathcal\{R\}\_\{x,\\mathrm\{main\}\}^\{\+\}andℛx,aux\+\\mathcal\{R\}\_\{x,\\mathrm\{aux\}\}^\{\+\}denote the main and auxiliary nodes in the budget\-filtered empirical success\-reachable region\.

For a student main node, the acceptable teacher anchors are defined over non\-root successful nodes with valid information acquisition:

𝒜tmain=\{v∈ℛx,main\+\|I⁡\(v\)⊆Its\}\.\\mathcal\{A\}\_\{t\}^\{\\mathrm\{main\}\}=\\left\\\{v\\in\\mathcal\{R\}\_\{x,\\mathrm\{main\}\}^\{\+\}\\;\\middle\|\\;I\(v\)\\subseteq I\_\{t\}^\{s\}\\right\\\}\.\(9\)Thus, a student main node can be aligned with any reachable teacher main node whose information set is contained in the student’s acquired information\. Equality of information sets and the useless\-operation component are not required\. For mixed transitions, the criterion depends only on the information component, since any transition with information gain is classified as a main transition\.

For a student auxiliary node, we first consider the reachable teacher main nodes whose information is contained inItsI\_\{t\}^\{s\}:

ℳt=\{m∈ℛx,main\+\|I⁡\(m\)⊆Its\}\.\\mathcal\{M\}\_\{t\}=\\left\\\{m\\in\\mathcal\{R\}\_\{x,\\mathrm\{main\}\}^\{\+\}\\;\\middle\|\\;I\(m\)\\subseteq I\_\{t\}^\{s\}\\right\\\}\.\(10\)Ifℳt≠∅\\mathcal\{M\}\_\{t\}\\neq\\varnothing, letmtm\_\{t\}be a node inℳt\\mathcal\{M\}\_\{t\}with the largest information\-set cardinality\. We then define the acceptable auxiliary anchors as:

𝒜taux=\{v∈ℛx,aux\+\|par\(v\)=mt,\|U\(v\)\|=\|Uts\|\},\\mathcal\{A\}\_\{t\}^\{\\mathrm\{aux\}\}=\\left\\\{v\\in\\mathcal\{R\}\_\{x,\\mathrm\{aux\}\}^\{\+\}\\;\\middle\|\\;\\operatorname\{par\}\(v\)=m\_\{t\},\\;\|U\(v\)\|=\|U\_\{t\}^\{s\}\|\\right\\\},\(11\)wherepar⁡\(v\)\\operatorname\{par\}\(v\)denotes the reachable teacher main node from which the auxiliary nodevvis reached\. Thus, the projection compares only the number of useless operations performed aftermtm\_\{t\}, without requiring the unsuccessful tools to be identical\. Ifℳt=∅\\mathcal\{M\}\_\{t\}=\\varnothing, then𝒜taux=∅\\mathcal\{A\}\_\{t\}^\{\\mathrm\{aux\}\}=\\varnothing\. Let𝒜t\\mathcal\{A\}\_\{t\}be𝒜tmain\\mathcal\{A\}\_\{t\}^\{\\mathrm\{main\}\}or𝒜taux\\mathcal\{A\}\_\{t\}^\{\\mathrm\{aux\}\}according to the type ofsts\_\{t\}, and define

ρt=𝕀\[𝒜t≠∅\]\.\\rho\_\{t\}=\\mathbb\{I\}\\\!\\left\[\\mathcal\{A\}\_\{t\}\\neq\\varnothing\\right\]\.\(12\)Ifρt=1\\rho\_\{t\}=1, the student state corresponds to at least one reachable teacher node of the same type\. We formally denote the selected valid teacher anchor asπ⁡\(st\)\\pi\(s\_\{t\}\): for a main node,π⁡\(st\)\\pi\(s\_\{t\}\)is chosen from𝒜tmain\\mathcal\{A\}\_\{t\}^\{\\mathrm\{main\}\}as a node with the maximal information set; for an auxiliary node,π⁡\(st\)\\pi\(s\_\{t\}\)can be any anchor in𝒜taux\\mathcal\{A\}\_\{t\}^\{\\mathrm\{aux\}\}\. Ifρt=0\\rho\_\{t\}=0, no valid same\-type anchor exists and the student state is treated as outside the success\-reachable region\.

Critical Topological Breakpoint \(CTB\)\.Since the initial state is shared by the teacher and student,ρ0=1\\rho\_\{0\}=1\. The CTB is defined as the first transition from a projectable to a non\-projectable student state:

tC\\displaystyle t\_\{\\mathrm\{C\}\}=min\{t:ρt−1=1,ρt=0\},\\displaystyle=\\min\\\{t:\\rho\_\{t\-1\}=1,\\ \\rho\_\{t\}=0\\\},\(13\)aC\\displaystyle a\_\{\\mathrm\{C\}\}=π⁡\(stC−1\),\\displaystyle=\\pi\(s\_\{t\_\{\\mathrm\{C\}\}\-1\}\),whereπ⁡\(⋅\)\\pi\(\\cdot\)denotes the projection mapping that assigns a valid student node to its teacher anchor\. Thus,tCt\_\{\\mathrm\{C\}\}identifies the first departure from the teacher\-supported region, and the anchoraCa\_\{\\mathrm\{C\}\}is the last valid teacher projection before that departure\. This boundary captures the earliest point at which the student either acquires an unsupported information combination or exceeds the teacher\-supported exploration count under the corresponding main node\. If the student remains projectable until the end of the rollout but still fails, we instead view the terminal state as the correction boundary and use its latest valid projection as the recovery anchor\.

### 3\.3CTB\-Guided Localized Supervision

Privileged\-context Retrieval\.We randomly sample successful and failed teacher traces from the teacher graph as privileged references, denoted as𝒞priv\\mathcal\{C\}^\{\\mathrm\{priv\}\}\. The reference traces are not directly concatenated with the student prefix, since the two trajectories may contain different calls and observations\. Instead, an augmented generator is conditioned on the task, the student prefix retained before the CTB, and the privileged references:

ctC∗=AugGen⁡\(x,τ<tCs,𝒞priv\)\.c\_\{t\_\{\\mathrm\{C\}\}\}^\{\*\}=\\operatorname\{AugGen\}\\bigl\(x,\\tau\_\{<t\_\{\\mathrm\{C\}\}\}^\{s\},\\mathcal\{C\}^\{\\mathrm\{priv\}\}\\bigr\)\.\(14\)The training trajectory is formed by appendingctC∗c\_\{t\_\{\\mathrm\{C\}\}\}^\{\*\}to the retained student prefix\. The trajectory is then tokenized, and CTB\-localized supervision is applied to the resulting training sequence\. Thus, the realized student context is preserved, while the sampled teacher traces provide privileged information for generating the subsequent continuation\.

CTB\-Localized Supervision\.Lety~=\(y~1,…,y~L\)\\widetilde\{y\}=\(\\widetilde\{y\}\_\{1\},\\ldots,\\widetilde\{y\}\_\{L\}\)denote the tokenized training trajectory\. We train with the masked causal language modeling objective:

ℒDART=−∑i=1Lmilogpθ\(y~i∣y~<i,x\)\.\\mathcal\{L\}\_\{\\mathrm\{DART\}\}=\-\\sum\_\{i=1\}^\{L\}m\_\{i\}\\log p\_\{\\theta\}\\bigl\(\\widetilde\{y\}\_\{i\}\\mid\\widetilde\{y\}\_\{<i\},x\\bigr\)\.\(15\)Although the loss is computed at the token level, the mask is defined over response steps:mi=1m\_\{i\}=1only if tokeny~i\\widetilde\{y\}\_\{i\}belongs to an assistant response step generated after the CTB and before the final\-answer step\. Tokens in the retained student prefix, user messages, tool observations, and the final\-answer step receive zero weight\. Thus, supervision is restricted to post\-CTB assistant response steps, while the pre\-CTB behavior and final answer are not directly optimized\.

### 3\.4Progressive Self\-Distillation Paradigm

The task\-specific ISTG is maintained throughout the self\-distillation\. At each iteration, the current student produces new rollouts, which are mapped into the shared interaction\-state space and processed by the type\-specific projection and breakpoint\-localization procedure\. Failed rollouts yield breakpoint\-localized privileged\-context distillation instances\. As the student improves, its states remain projectable for longer portions of the interaction trajectory\. The detected CTBs therefore track the evolving boundary at which the current policy first departs from the teacher\-supported region\. DART\-SD consequently induces a self\-paced curriculum that progressively shifts supervision toward recovery behaviors beyond the current tool\-use capability\.

MethodsFTRLBFCLToolHopτ\\tau\-benchRoTBenchAvg\.Solve\-PSolve\-RSolve\-F1Multi\-TurnACPass^1TSPICFQwen3\-4BBase21\.0026\.5421\.8110\.1420\.2015\.1569\.5226\.3116\.0725\.19♠\\spadesuitSFT34\.2649\.0237\.9614\.5740\.5021\.8271\.6744\.4024\.4037\.62♠\\spadesuitSCoRe\-SFT23\.7928\.2024\.6112\.7525\.8311\.5260\.3631\.1918\.3326\.29♠\\spadesuitOPSD22\.3729\.2123\.5011\.0020\.2023\.0368\.9335\.2421\.6728\.35♢\\diamondsuitFTRL\-GRPO36\.8341\.7137\.8413\.5029\.2520\.6170\.3632\.6220\.2433\.66♢\\diamondsuitToolRL26\.8933\.2328\.479\.8820\.6018\.1869\.6438\.2124\.0529\.91♢\\diamondsuitMatchTIR \(OT\)23\.5529\.2724\.9510\.3826\.2316\.3672\.0232\.7420\.3628\.43♢\\diamondsuitMatchTIR \(KM\)25\.5431\.1026\.5010\.0026\.6321\.8272\.0233\.8121\.4329\.87♠\\spadesuitDART\-SD \(Ours\)36\.7048\.1639\.7723\.8842\.1123\.0372\.0242\.3824\.5239\.17Qwen3\-8BBase21\.1830\.7123\.4818\.3828\.5410\.1375\.5236\.2922\.1929\.60♠\\spadesuitSFT38\.0850\.9541\.8919\.2543\.5226\.0675\.9548\.8130\.2441\.64♠\\spadesuitSCoRe\-SFT31\.2834\.4831\.5819\.2530\.2518\.1870\.3640\.4824\.5233\.38♠\\spadesuitOPSD24\.3734\.8826\.6820\.5041\.1121\.2175\.2443\.2127\.3834\.95♢\\diamondsuitFTRL\-GRPO37\.6645\.4940\.2235\.2534\.5723\.0377\.0242\.7427\.0240\.33♢\\diamondsuitToolRL32\.4941\.0735\.0020\.5044\.7225\.4575\.7151\.4333\.1039\.94♢\\diamondsuitMatchTIR \(OT\)29\.4534\.8930\.6521\.3838\.7924\.2475\.0040\.3625\.3635\.57♢\\diamondsuitMatchTIR \(KM\)33\.0740\.4235\.3723\.2541\.7126\.0675\.7142\.2626\.9038\.31♠\\spadesuitDART\-SD \(Ours\)42\.0054\.1345\.6627\.6345\.0327\.1275\.8357\.3835\.4845\.58

Table 1:Performance comparison of different training methods on five tool\-use benchmarks using Qwen3\-4B and Qwen3\-8B backbones\. Training\-based baselines are organized into distillation\-based♠\\spadesuitand reinforcement learning♢\\diamondsuitparadigms\. All trainable methods are trained on FTRL and evaluated on both the in\-domain FTRL test set and four out\-of\-domain benchmarks\. Thebestandsecond\-bestresults within each backbone are highlighted\.

## 4Experiments

### 4\.1Experimental Setup

Datasets\.We train the models on theFTRLdataset\[[37](https://arxiv.org/html/2608.18524#bib.bib1)\], which comprises over 2,000 automatically constructed tool\-use environments with verifiable feedback\. FTRL covers four types of task structures: Single, containing a single sub\-question; Para\-Single, consisting of multiple independent sub\-questions executable in parallel; Multi, comprising a sequence of dependent sub\-questions; and Para\-Multi, combining both independent and dependent sub\-questions\. We evaluate DART\-SD on five tool\-use benchmarks, including FTRL as the in\-domain test set, andBFCL\[[19](https://arxiv.org/html/2608.18524#bib.bib2)\],ToolHop\[[36](https://arxiv.org/html/2608.18524#bib.bib4)\],τ\\tau\-bench\[[35](https://arxiv.org/html/2608.18524#bib.bib3)\], andRoTBench\[[38](https://arxiv.org/html/2608.18524#bib.bib18)\]as out\-of\-domain benchmarks to assess generalization\.

Figure 3:Performance comparison of Qwen3\-8B, DART\-SD, and the teacher across five tool\-use benchmarks\. DART\-SD improves upon Qwen3\-8B on all benchmarks andsurpassesthe teacher on FTRL, ToolHop, andτ\\tau\-bench\.Baselines\.We evaluate DART\-SD using both Qwen3\-4B and Qwen3\-8B backbones\[[33](https://arxiv.org/html/2608.18524#bib.bib40)\]\. For each backbone, we first report the performance of the pretrained model without task\-specific adaptation, denoted asBase\. The remaining baselines are grouped into two optimization paradigms\.\(1\) Distillation\-based Paradigms:standardSFT,SCoRe\-SFT\[[16](https://arxiv.org/html/2608.18524#bib.bib11)\], andOPSD\[[44](https://arxiv.org/html/2608.18524#bib.bib12)\]\.\(2\) RL Paradigms:FTRL\-GRPO\[[37](https://arxiv.org/html/2608.18524#bib.bib1)\],ToolRL\[[20](https://arxiv.org/html/2608.18524#bib.bib38)\], andMatchTIR\[[23](https://arxiv.org/html/2608.18524#bib.bib41)\]\(evaluated using bothMatchTIR\-OTandMatchTIR\-KM\)\. To ensure a strictly controlled comparison, all methods are trained and evaluated under a no\-thinking configuration\.

Evaluation Metrics\.To ensure a rigorous and standardized assessment, we strictly adhere to the official evaluation protocols for each respective benchmark\.\(1\) FTRL:We evaluate trajectory precision and task completeness usingSolve\-PandSolve\-Ralong with their harmonic mean,Solve\-F1\.\(2\) BFCL:We report the averageMulti\-Turnscore over Base, Miss Function, Miss Parameter and Long Context\.\(3\) ToolHop:We evaluate the model’s ability to resolve complex multi\-hop tool dependencies usingAnswer Correctness \(AC\)\.\(4\)τ\\tau\-bench:We assess performance on real\-world API orchestration tasks usingPass^1\.\(5\) RoTBench:We evaluate fine\-grained execution robustness throughTool Selection \(TS\),Parameter Identification \(PI\), andContent Filling \(CF\)\.

Implementation Details\.We initialize the student model from Qwen3\-4B and Qwen3\-8B\. Teacher trajectories are collected from a mixed pool of Qwen3\.6\-27B\[[24](https://arxiv.org/html/2608.18524#bib.bib43)\]and GLM\-5\.2\[[42](https://arxiv.org/html/2608.18524#bib.bib42)\]\. We perform five iterations of CTB\-guided localized SFT over all 2,215 tasks in the FTRL training set\. At each iteration, the current student checkpoint generates eight trajectories per task with a temperature of 0\.7, a maximum generation length of 4,096 tokens, and at most nine interaction turns\. The model is then fine\-tuned for one epoch using breakpoint\-localized loss masking, with a learning rate of5×10−75\\times 10^\{\-7\}and a batch size of 32\. Each augmented training context contains two positive references and one negative reference, and each positive reference is accompanied by a teacher\-generated analysis\.

### 4\.2Experimental Results

Overall Performance\.As shown in Table[1](https://arxiv.org/html/2608.18524#S3.T1), DART\-SD demonstrates robust and superior performance on both the in\-domain FTRL dataset and four out\-of\-domain tool\-use benchmarks\. Compared with distillation\-based methods and reinforcement learning methods, DART\-SD delivers consistent empirical gains and achieves the strongest overall performance\. This robust cross\-domain superiority confirms that our topology\-aware localized tuning learns generalizable and transferable tool\-use capabilities, rather than merely overfitting to the FTRL training distribution\. Crucially, DART\-SD maintains strong performance across different parameter scales\. It achieves the highest average performance with both the Qwen3\-8B and Qwen3\-4B backbones, while obtaining the best or competitive results on most individual metrics\. These results indicate that the effectiveness of topology\-aware localized tuning is not limited to a particular model size and remains robust when applied to a smaller backbone\.

Effectiveness of Capability Distillation\.Figure[3](https://arxiv.org/html/2608.18524#S4.F3)compares the performance of the base Qwen3\-8B student, DART\-SD, and the teacher across five tool\-use benchmarks\. For visualization, we report Solve\-F1 on FTRL, the average Multi\-Turn score on BFCL, AC on ToolHop, Pass^1 onτ\\tau\-bench, and PI on RoTBench\. Initialized from the same 8B base model, DART\-SD delivers substantial improvements across all five benchmarks\. Notably, the resulting student surpasses the teacher on FTRL, ToolHop, andτ\\tau\-bench, despite using substantially fewer parameters\. These results suggest that DART\-SD goes beyond direct trajectory imitation\. By leveraging topology\-aware recovery paths, it effectively extracts and generalizes reusable tool\-use behaviors from the structural knowledge encoded in the ISTG, enabling the smaller student to outperform its teacher on several benchmarks\.

TaskDART\-SDGoldenIter1Iter2Iter3Iter4Iter5Single1\.231\.151\.101\.091\.071\.00Multi3\.834\.634\.544\.494\.414\.71Para\-Single3\.292\.652\.272\.262\.222\.11Para\-Multi7\.256\.485\.975\.825\.626\.97Overall4\.233\.993\.713\.653\.554\.02Solve\-F140\.3742\.9543\.7844\.6745\.66–Table 2:Average tool\-call length of successful trajectories across progressive SFT iterations on the FTRL test set\. DART\-SD progressively improves Solve\-F1 while shortening its tool traces, eventually producing more efficient traces than the golden solutions given during data construction\.Tool\-Call Length of Successful Trajectories\.To verify whether DART\-SD improves reasoning rather than memorizing references, we track the evolution of successful trajectories across iterations\. We measure task performance \(Solve\-F1\) and efficiency \(average tool calls\) on the FTRL test set, and compare the final model against the golden references\. The golden reference denotes the tool trajectory provided by the FTRL construction pipeline\. As shown in Table[2](https://arxiv.org/html/2608.18524#S4.T2),Solve\-F1steadily improves while trajectory length decreases\. The average number of tool calls drops from 4\.23 at Iter1 to 3\.55 at Iter5\. This confirms that the model learnsmore efficient tool\-use strategiesinstead of relying on brute\-force exploration or longer execution chains\. This reduction is especially clear in complex multi\-step and parallel tasks, suggesting that our method effectively removes redundant tool calls while maintaining high success rates\. Remarkably, the trajectories of our final model are evenshorter than the golden references\(3\.55 vs\. 4\.02\)\. This highlights that DART\-SD successfully discovers optimized shortcuts rather than blindly following the provided subtask structures\.

TaskCTB PositionIter1Iter2Iter3Iter4Iter5Single0\.0340\.1860\.1370\.2490\.077Multi0\.5001\.6241\.7291\.8761\.953Para\-Single0\.0950\.2190\.2990\.3280\.313Para\-Multi0\.3951\.5241\.6921\.7911\.816Overall0\.3481\.1851\.3101\.4211\.452Δ\\Deltavs\. Iter1–\+0\.837\+0\.962\+1\.073\+1\.104Table 3:Average CTB positions of failed training trajectories across progressive SFT\. Larger values indicate that the first departure from empirically recoverable behavior occurs later, meaning that the model correctly executes a longer trajectory prefix before localized recovery is required\.CTB Position Shifts within Failed Trajectories\.To analyze how the student’s capability boundary expands during progressive training, we track the average CTB position within failed trajectories across iterations\. A later CTB position indicates that the model remains within the empirically recoverable region for a longer prefix before its first departure\. As shown in Table[3](https://arxiv.org/html/2608.18524#S4.T3), the average CTB position steadily advances from 0\.348 at Iter1 to 1\.452 at Iter5, indicating that the first departure from recoverable behavior is progressively delayed\. This trend is particularly evident in complex multi\-hop and parallel multi\-hop tasks, where increasingly longer valid prefixes are learned\. These results validate the effectiveness of CTB\-guided localized supervision\. By identifying the current capability boundary, DART\-SD performs teacher\-guided resampling from the first departure point and supervises only the recovery suffix while preserving already\-mastered prefixes\. As the boundary moves deeper across iterations, the student progressively acquires more complex tool\-use behaviors, demonstrating the effectiveness of DART\-SD for long\-horizon tasks\.

MethodFTRLBFCLToolHopQwen3\-8B29\.7440\.0042\.21FTRL\-GRPO32\.8541\.5036\.72ToolRL26\.7234\.2532\.93MatchTIR \(KM\)37\.3347\.1346\.16DART\-SD41\.0349\.7546\.43Table 4:Performance comparison under the thinking setting\. Results are reported using FTRL Solve\-F1, the average BFCL Multi\-Turn score, and ToolHop AC\.Effect of Thinking Mode\.Table[4](https://arxiv.org/html/2608.18524#S4.T4)evaluates DART\-SD under the thinking setting\. All trainable baselines are trained with thinking enabled, whereas the Qwen3\-8B base model enables thinking only at inference time\. For DART\-SD, teacher trajectories are generated in thinking mode, and both the teacher’s reasoning traces and tool\-call trajectories are incorporated into the resampling context for student training\. The effect of explicit thinking varies across methods and benchmarks\. To provide a fair and controlled comparison, we therefore use the no\-thinking configuration in all other experiments unless otherwise specified, allowing us to more directly evaluate the proposed training framework\. Importantly, DART\-SD still achieves the best performance across all three benchmarks under the thinking setting, demonstrating that CTB\-guided localized supervision remains effective when explicit thinking is enabled\.

MethodIFEvalAIME24AIME25MMLUAvg\.Qwen3\-8B34\.7546\.6723\.3370\.9443\.92SFT35\.3043\.3326\.6771\.4344\.18DART\-SD45\.2950\.0030\.0074\.2749\.89Table 5:General capability evaluation on representative benchmarks\. Thebestresults are highlighted\.General Capability Preservation\.To evaluate whether DART\-SD preserves the model’s general capabilities beyond tool\-use tasks, we further evaluate it under a thinking\-enabled setting on four standard benchmarks: IFEval\[[46](https://arxiv.org/html/2608.18524#bib.bib45)\]for instruction following, AIME24 and AIME25 for mathematical reasoning, and MMLU\[[10](https://arxiv.org/html/2608.18524#bib.bib46)\]for general knowledge\. We report strict prompt\-level accuracy on IFEval, pass@10 on AIME24 and AIME25, and accuracy on MMLU\. As shown in Table[5](https://arxiv.org/html/2608.18524#S4.T5), DART\-SD consistently outperforms both the pretrained model and standard SFT across all benchmarks, improving the average score from 43\.92 to 49\.89\. These results suggest that topology\-aware localized supervision not only enhances tool\-use ability but also preserves the model’s general reasoning and instruction\-following capabilities\.

MethodSolve\-PSolve\-RSolve\-F1Qwen3\-8B21\.1830\.7123\.48\+SD36\.2344\.4638\.10\+CTB36\.6246\.3239\.51\+Progressive SFT41\.3249\.6543\.93\+ISTG \(Ours\)42\.0054\.1345\.66Table 6:Component ablation of DART\-SD on the FTRL test set using Solve\-P, Solve\-R, and Solve\-F1\.Ablation Study on Different Components\.Table[6](https://arxiv.org/html/2608.18524#S4.T6)presents the component ablation of DART\-SD to evaluate the contribution of each core design\. First, applying Self\-Distillation \(SD\) directly over the base model yields substantial gains, demonstrating the efficacy of learning from the agent’s own exploratory trajectories\. Second, introducing CTB\-guided localized supervision further improves performance by restricting the loss calculation exclusively to the post\-breakpoint recovery suffix, thereby avoiding direct global supervision on already\-mastered prefixes\. Third, Progressive Self\-Distillation introduces iterative rollout and localized correction rounds, confirming the necessity of a continuous iterative adaptation paradigm over one\-shot training\. Finally, ISTG replaces LLM\-judge\-based breakpoint detection with topology\-aware state projection and structured recovery, enabling more accurate CTB identification and structurally compatible supervision\. Together, these complementary components yield the strongest performance\.

## 5Conclusion

In this paper, we proposeDART\-SD, a topology\-aware self\-distillation framework for multi\-turn tool\-calling agents\. DART\-SD models execution as anInteraction\-State Transition Graph\(ISTG\), revealing the diamond topology induced by order\-independent valid exploration\. Building on the ISTG, DART\-SD identifies theCritical Topological Breakpoint\(CTB\), retrieves recovery references from the ISTG, and performs progressive self\-distillation via CTB\-guided localized supervision while preserving the valid interaction prefix\. Extensive experiments demonstrate that DART\-SD consistently outperforms both distillation and reinforcement learning baselines across multiple in\-domain and out\-of\-domain tool\-use benchmarks\. Beyond improving task success, DART\-SD enables more efficient tool\-use behaviors by preserving exploration, reducing redundant tool calls, and progressively mastering more complex tool\-use behaviors throughout self\-distillation\. More broadly, our findings suggest that effective agent distillation should be guided by interaction\-state topology rather than rigid trajectory imitation\. We hope this perspective encourages future research on structure\-aware agent training and graph representations for long\-horizon reasoning and decision\-making\.

## References

- \[1\]Anthropic\(2025\)Introducing claude 4\.External Links:[Link](https://www.anthropic.com/news/claude-4)Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[2\]H\. Chai, Z\. Cao, M\. Ran, Y\. Yang, J\. Lin, X\. Peng, H\. Wang, R\. Ding, Z\. Wan, M\. Wen,et al\.\(2025\)PARL\-mt: learning to call functions in multi\-turn conversation with progress awareness\.arXiv preprint arXiv:2509\.23206\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[3\]Z\. Chen, Z\. Wu, Z\. Liao, S\. Jiang, H\. Xu, Y\. Chen, C\. Su, X\. Liu, and H\. Wang\(2026\)R3G: a reasoning\-retrieval\-reranking framework for vision\-centric answer generation\.InICASSP 2026 \- 2026 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 8602–8606\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP55912.2026.11463453)Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[4\]Z\. Chen, Z\. Wu, Y\. Yu, H\. Xu, Z\. Liao, Z\. Liu, X\. Deng, P\. Jiao, and H\. Wang\(2026\)Learning to search: a decision\-based agent for knowledge\-based visual question answering\.arXiv preprint arXiv:2604\.07146\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[5\]X\. Cheng, S\. He, L\. Feng, H\. Xu, M\. Yan, L\. Feng, and B\. An\(2026\)Beyond trajectory\-level attribution: graph\-based credit assignment for agentic reinforcement learning\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p2.1)\.
- \[6\]G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen,et al\.\(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[7\]J\. V\. Gallart, S\. Kia, R\. Bent, and M\. Grosskopf\(2026\)Chain\-based adaptive reconfiguration over lattices for hallucination reduction\.arXiv preprint arXiv:2605\.27706\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[8\]Z\. Gekhman, R\. Aharoni, E\. Ofek, M\. Geva, R\. Reichart, and J\. Herzig\(2026\)Thinking to recall: how reasoning unlocks parametric knowledge in llms\.arXiv preprint arXiv:2603\.09906\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[9\]L\. Gupta, L\. Li, Y\. Liu, S\. G\. Subramanian, K\. Suleman, Z\. Zhang, H\. Lu, and S\. Pasupalak\(2026\)World of workflows: a benchmark for bringing world models to enterprise systems\.arXiv preprint arXiv:2601\.22130\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[10\]D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt\(2020\)Measuring massive multitask language understanding\.arXiv preprint arXiv:2009\.03300\.Cited by:[§4\.2](https://arxiv.org/html/2608.18524#S4.SS2.p6.1)\.
- \[11\]H\. Hu, C\. Liu, N\. Li, and Y\. Wang\(2025\)Training task reasoning llm agents for multi\-turn task planning via single\-turn reinforcement learning\.IEEE Control Systems Letters9,pp\. 2879–2884\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[12\]M\. Kang, J\. Jeong, S\. Lee, J\. Cho, and S\. J\. Hwang\(2026\)Distilling llm agent into small models with retrieval and code tools\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.38,pp\. 106501–106538\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[13\]C\. Kausik, A\. Swaminathan, and N\. Kallus\(2026\)The context gathering decision process: a pomdp framework for agentic search\.arXiv preprint arXiv:2605\.07042\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[14\]B\. Liao, H\. Dong, C\. Monz, X\. Xu, L\. Dong, and F\. Wei\(2026\)Multi\-turn on\-policy distillation with prefix replay\.arXiv preprint arXiv:2607\.04763\.Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p1.1)\.
- \[15\]H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe\(2024\)Let’s verify step by step\.InInternational Conference on Learning Representations \(ICLR\),pp\. 39578–39601\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[16\]Y\. Lyu, C\. Wang, J\. Huang, and T\. Xu\(2025\)From correction to mastery: reinforced distillation of large language model agents\.arXiv preprint arXiv:2509\.14257\.Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p2.1)\.
- \[17\]H\. Meng, K\. Huang, S\. Wei, C\. Ma, S\. Yang, X\. Wang, G\. Wang, B\. Ding, and J\. Zhou\(2026\)Sparse but critical: a token\-level analysis of distributional shifts in rlvr fine\-tuning of llms\.arXiv preprint arXiv:2603\.22446\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§2](https://arxiv.org/html/2608.18524#S2.p2.1)\.
- \[18\]S\. M\. Mousavi, S\. Alghisi, and G\. Riccardi\(2026\)What does loss optimization actually teach, if anything? knowledge dynamics in continual pre\-training of llms\.arXiv preprint arXiv:2601\.03858\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1)\.
- \[19\]S\. G\. Patil, H\. Mao, F\. Yan, C\. C\. Ji, V\. Suresh, I\. Stoica, and J\. E\. Gonzalez\(2025\)The berkeley function calling leaderboard \(bfcl\): from tool use to agentic evaluation of large language models\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p1.1)\.
- \[20\]C\. Qian, E\. C\. Acikgoz, Q\. He, H\. Wang, X\. Chen, D\. Hakkani\-Tur, G\. Tur, and H\. Ji\(2026\)Toolrl: reward is all tool learning needs\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.38,pp\. 105523–105553\.Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p2.1),[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p2.1)\.
- \[21\]Y\. Qin, S\. Hu, Y\. Lin, W\. Chen, N\. Ding, G\. Cui, Z\. Zeng, X\. Zhou, Y\. Huang, C\. Xiao,et al\.\(2024\)Tool learning with foundation models\.ACM Computing Surveys57\(4\),pp\. 1–40\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[22\]Y\. Qin, S\. Liang, Y\. Ye, K\. Zhu, L\. Yan, Y\. Lu, Y\. Lin, X\. Cong, X\. Tang, B\. Qian,et al\.\(2024\)Toolllm: facilitating large language models to master 16000\+ real\-world apis\.InInternational Conference on Learning Representations \(ICLR\),pp\. 9695–9717\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§2](https://arxiv.org/html/2608.18524#S2.p1.1)\.
- \[23\]C\. Qu, S\. Dai, H\. Cai, J\. Xu, S\. Wang, and D\. Yin\(2026\)MatchTIR: fine\-grained supervision for tool\-integrated reasoning via bipartite matching\.arXiv preprint arXiv:2601\.10712\.Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p2.1),[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p2.1)\.
- \[24\]Qwen Team\(2026\)Qwen3\.6\-27B: flagship\-level coding in a 27B dense model\.External Links:[Link](https://qwen.ai/blog?id=qwen3.6-27b)Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p4.1)\.
- \[25\]M\. Ranzato, S\. Chopra, M\. Auli, and W\. Zaremba\(2015\)Sequence level training with recurrent neural networks\.arXiv preprint arXiv:1511\.06732\.Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p1.1)\.
- \[26\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§2](https://arxiv.org/html/2608.18524#S2.p2.1)\.
- \[27\]G\. Wang, S\. Dai, G\. Ye, Z\. Gan, W\. Yao, Y\. Deng, X\. Wu, and Z\. Ying\(2025\)Information gain\-based policy optimization: a simple and effective approach for multi\-turn llm agents\.arXiv preprint arXiv:2510\.14967\.Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p2.1)\.
- \[28\]S\. Wang, G\. Li, Z\. Yang, and Y\. Gao\(2026\)Hindsight hint distillation: scaffolded reasoning for swe agents from cot\-free answers\.arXiv preprint arXiv:2605\.11556\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1)\.
- \[29\]Y\. Wang, M\. Song, Z\. Zhang, S\. Huang, H\. Huang, F\. Wei, W\. Deng, F\. Sun, and Q\. Zhang\(2026\)Group\-graph policy optimization for long\-horizon agentic reinforcement learning\.arXiv preprint arXiv:2606\.22995\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[30\]Z\. Wu, K\. Shi, C\. Zhang, Z\. Liao, J\. Yang, N\. Yang, Q\. Peng, L\. Zhang, H\. Xu, T\. Su,et al\.\(2026\)When models judge themselves: unsupervised self\-evolution for multimodal reasoning\.arXiv preprint arXiv:2603\.21289\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[31\]Z\. Wu, H\. Xu, K\. Shi, Z\. Chen, Y\. Yu, C\. Zhang, Z\. Liao, J\. Yang, Z\. Yang, H\. Lu,et al\.\(2026\)ProMSA: progressive multimodal search agents for knowledge\-based visual question answering\.arXiv preprint arXiv:2606\.27974\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[32\]Z\. Xi, W\. Chen, X\. Guo, W\. He, Y\. Ding, B\. Hong, M\. Zhang, J\. Wang, S\. Jin, E\. Zhou,et al\.\(2025\)The rise and potential of large language model based agents: a survey\.Science China Information Sciences68\(2\),pp\. 121101\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[33\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p2.1)\.
- \[34\]W\. Yang, H\. Zang, J\. Zhang, W\. Shi, D\. Su, J\. Wang, X\. Cheng, and F\. Sun\(2026\)Beyond reasoning: reinforcement learning unlocks parametric knowledge in llms\.arXiv preprint arXiv:2605\.07153\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[35\]S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhan\(2024\)τ\\tau\-Bench: a benchmark for tool\-agent\-user interaction in real\-world domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p1.1)\.
- \[36\]J\. Ye, Z\. Du, X\. Yao, W\. Lin, Y\. Xu, Z\. Chen, Z\. Wang, S\. Zhu, Z\. Xi, S\. Yuan,et al\.\(2025\)ToolHop: a query\-driven benchmark for evaluating large language models in multi\-hop tool use\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2995–3021\.Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p1.1)\.
- \[37\]J\. Ye, C\. Jiang, Z\. Du, Y\. Xu, X\. Yao, Z\. Xi, X\. Fan, Q\. Zhang, T\. Gui, X\. Huang,et al\.\(2025\)Feedback\-driven tool\-use improvements in large language models via automated build environments\.arXiv preprint arXiv:2508\.08791\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§2](https://arxiv.org/html/2608.18524#S2.p2.1),[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p2.1)\.
- \[38\]J\. Ye, Y\. Wu, S\. Gao, C\. Huang, S\. Li, G\. Li, X\. Fan, Q\. Zhang, T\. Gui, and X\. Huang\(2024\)RoTBench: a multi\-level benchmark for evaluating the robustness of large language models in tool learning\.InProceedings of the Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 313–333\.Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p1.1)\.
- \[39\]J\. Ye, Y\. Wu, S\. Li, Y\. Yang, Z\. Xi, T\. Gui, Q\. Zhang, X\. Huang, P\. Wang, Z\. Shi,et al\.\(2024\)Tl\-training: a task\-feature\-based framework for training large language models in tool use\.arXiv preprint arXiv:2412\.15495\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p1.1)\.
- \[40\]W\. Yeo, Y\. Choi, T\. Ki, and S\. J\. Hwang\(2026\)HINT\-sd: targeted hindsight self\-distillation for long\-horizon agents\.arXiv preprint arXiv:2605\.17873\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§2](https://arxiv.org/html/2608.18524#S2.p1.1)\.
- \[41\]J\. Yu, Z\. Cheng, X\. Wu, and X\. Xing\(2025\)Building coding agents via entropy\-enhanced multi\-turn preference optimization\.arXiv preprint arXiv:2509\.12434\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p3.1)\.
- \[42\]A\. Zeng, X\. Lv, Z\. Hou, Z\. Du, Q\. Zheng, B\. Chen, D\. Yin, C\. Ge, C\. Huang, C\. Xie,et al\.\(2026\)Glm\-5: from vibe coding to agentic engineering\.arXiv preprint arXiv:2602\.15763\.Cited by:[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p4.1)\.
- \[43\]S\. Zeng, Q\. Wei, W\. Brown, O\. Frunza, Y\. Nevmyvaka, Y\. K\. Zhao, and M\. Hong\(2025\)Reinforcing multi\-turn reasoning in llm agents via turn\-level credit assignment\.InICML 2025 Workshop on Computer Use Agents,Cited by:[§2](https://arxiv.org/html/2608.18524#S2.p2.1)\.
- \[44\]S\. Zhao, Z\. Xie, M\. Liu, J\. Huang, G\. Pang, F\. Chen, and A\. Grover\(2026\)Self\-distilled reasoner: on\-policy self\-distillation for large language models\.arXiv preprint arXiv:2601\.18734\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1),[§2](https://arxiv.org/html/2608.18524#S2.p1.1),[§4\.1](https://arxiv.org/html/2608.18524#S4.SS1.p2.1)\.
- \[45\]D\. Zheng, M\. Lapata, and J\. Z\. Pan\(2024\)How reliable are llms as knowledge bases? re\-thinking facutality and consistency\.arXiv preprint arXiv:2407\.13578\.Cited by:[§1](https://arxiv.org/html/2608.18524#S1.p2.1)\.
- \[46\]J\. Zhou, T\. Lu, S\. Mishra, S\. Brahma, S\. Basu, Y\. Luan, D\. Zhou, and L\. Hou\(2023\)Instruction\-following evaluation for large language models\.arXiv preprint arXiv:2311\.07911\.Cited by:[§4\.2](https://arxiv.org/html/2608.18524#S4.SS2.p6.1)\.