SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL

# 1 Introduction
Source: [https://arxiv.org/html/2609.29050](https://arxiv.org/html/2609.29050)
![[Uncaptioned image]](https://arxiv.org/html/2609.29050v1/assets/pku_logo.png)![[Uncaptioned image]](https://arxiv.org/html/2609.29050v1/assets/szu_logo.png)![[Uncaptioned image]](https://arxiv.org/html/2609.29050v1/assets/qq_logo.png)![[Uncaptioned image]](https://arxiv.org/html/2609.29050v1/assets/logo_slca_grpo_square.png)SLCA\-GRPO: Resolving Cross\-Segment Credit Misattribution in Tool\-Calling RLYan Zhan1,\*Shaobo Liu2,\*Qiunan Liu3Yuanjun Shi3Siqi Xu3WeiYi Hou3Xiang Xu3Zekang Li3Weizhou Pan3,\+Jiahong Yan31Peking University2Shenzhen University3Tencent PCG QQ TeamTool\-calling agents produce heterogeneous outputs, interleaving structured tool invocations with user\-facing natural language summaries\. This output heterogeneity presents a structural failure mode in standard on\-policy Reinforcement Learning \(RL\): algorithms like GRPO indiscriminately broadcast a homogeneous trajectory\-level scalar advantage to all tokens\. Consequently, gradient noise from summary generation leaks into tool\-decision tokens, causing cross\-segment credit misattribution and brittle optimization\. In this work, we proposeSLCA\-GRPO, a framework incorporating Segment\-Locked Credit Assignment \(SLCA\)\. To enable scalable exploration without costly real APIs and stable training, we first construct the Schema\-Guided LLM Simulator \(SGLS\) as foundational training infrastructure\. Building on this, SLCA decouples advantage estimation at the*structural segment level*within a single group of rollouts, without requiring additional rollouts from intermediate states\. Supported by Hierarchical Rewards \(HierR\), SLCA routes execution advantages to tool tokens and preference advantages to summary tokens, eliminating advantage contamination \(the dominant cross\-segment credit misattribution channel\) within each policy update\. On a 7B backbone,SLCA\-GRPOaccelerates convergence and outperforms standard GRPO, ToolPO, and RLTR by \+2\.53 pp on in\-domain evaluation, \+1\.36 pp on the Berkeley Function\-Calling Leaderboard \(BFCL\), and \+9\.15 pp onτ2\\tau^\{2\}\-Bench under the same training budgets, achieving higher accuracy with reduced tool redundancy and costs\.Contact:[josephpan@tencent\.com](mailto:[email protected])Code:[github\.com/SLCA\-GRPO/SLCA\-GRPO](https://github.com/SLCA-GRPO/SLCA-GRPO)![[Uncaptioned image]](https://arxiv.org/html/2609.29050v1/assets/hf_logo.png)Dataset:[huggingface\.co/datasets/YanZhanPKU/SLCA\-GRPO\-Datasets](https://huggingface.co/datasets/YanZhanPKU/SLCA-GRPO-Datasets)

††footnotetext:\*Equal contribution\. Work done during Yan Zhan and Shaobo Liu’s internships at Tencent\.\+Correspondence to: Weizhou Pan \(josephpan@tencent\.com\)\.The evolution of Large Language Models \(LLMs\) from passive chatbots to active agents hinges on their ability to plan and execute actions via external tools\([Qin et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib23);[Schick et al\., 2023](https://arxiv.org/html/2609.29050#bib.bib29);[Yao et al\., 2023](https://arxiv.org/html/2609.29050#bib.bib40);[Patil et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib21)\)\. Although Supervised Fine\-Tuning \(SFT\) establishes a strong initialization, it depends on imitating fixed patterns and is fragile against the large, changing namespaces of ecosystems such as the Model Context Protocol \(MCP\)\([Hou et al\., 2026](https://arxiv.org/html/2609.29050#bib.bib9)\)\. To transcend mimicry, post\-training via on\-policy Reinforcement Learning \(RL\) is essential\([Ouyang et al\., 2022](https://arxiv.org/html/2609.29050#bib.bib19);[Schulman et al\., 2017](https://arxiv.org/html/2609.29050#bib.bib31);[Yu et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib41)\)\. Tool\-calling resembles reasoning tasks in that trajectories can contain intermediate work before a final answer, but it adds externally verifiable boundaries: structured API calls, environment\-injected observations, and schema constraints\. These boundaries naturally decompose a rollout intoy=\[ytool⊕ysum\]y=\\big\[\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\\oplus\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}\\big\], whereytool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}encodes the operational trajectory \(reasoning traces interleaved with structured tool calls\), andysum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}constitutes the user\-facing natural language response\.

Applying standard RL algorithms to this two\-segment structure reveals a fundamental mismatch\. Dominant approaches such as Group Relative Policy Optimization \(GRPO\)\([Guo et al\., 2025a](https://arxiv.org/html/2609.29050#bib.bib6)\)broadcast a unified trajectory\-level advantage to all tokens; even Process Reward Models \(PRMs\) aggregate signals \(Rtotal=Rtool\+RsumR\_\{\\text\{total\}\}=\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\) before advantage estimation\. We argue that thisGlobal Signal Conflationis a structural failure mode: summary\-reward variation can enter the tool\-token advantage, producingCross\-Segment Credit Misattribution\. In sign\-conflict regimes, a failed tool call can be reinforced when followed by a correct summary, while a correct tool trajectory can be penalized when followed by an incorrect summary\. Existing mitigations do not close this pathway: temporal methods \(VinePPO\([Kazemnejad et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib10)\), GiGPO\([Feng et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib5)\), SPO\([Guo et al\., 2025b](https://arxiv.org/html/2609.29050#bib.bib7)\)\) address step\-wise credit while retainingRtotalR\_\{\\text\{total\}\}; ToolPO\([Li et al\., 2026b](https://arxiv.org/html/2609.29050#bib.bib14)\)adds local tool rewards but still lets summary\-dependent noise reach tool tokens; and RLTR\([Li et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib15)\)separates planner and summarizer, abandoning a unified backbone\.

We verify this failure through diagnostic cases: standard GRPO assigns a single aggregated advantage to both tool and summary tokens, so a correct summary can mask an inefficient or wrong tool trajectory, and an incorrect summary can suppress otherwise correct tool use\. Gradient diagnostics show instability spikes while tool/summary gradient directions remain near\-orthogonal, consistent with advantage contamination as a source of the observed instability \(App\.[B](https://arxiv.org/html/2609.29050#A2)\)\. Two questions therefore arise:*\(1\)*can cross\-segment advantage contamination be structurally eliminated within a single unified policy, and*\(2\)*does eliminating it translate into measurable gains beyond what additive augmentation delivers?

![Refer to caption](https://arxiv.org/html/2609.29050v1/case_study_main_tex.png)Diagnostic cases\.Unified reward aggregation can misassign credit across tool and summary segments\. Left: an unnecessary tool call is paired with a correct delivery summary\. Right: correct tool evidence is paired with an incorrect refund summary\. Under standard GRPO, the mixed advantage can reward bad tool use or penalize good tool use\.To resolve this, we proposeSLCA\-GRPO\([Figure1](https://arxiv.org/html/2609.29050#S3.F1)\)\. Its core, Segment\-Locked Credit Assignment \(SLCA\), routes tool\-side advantages \(A^tool\\hat\{A\}\_\{\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathrm\{tool\}\}\}\) exclusively toytool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}tokens and summary\-side advantages \(A^sum\\hat\{A\}\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{sum\}\}\}\) exclusively toysum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}tokens, blocking the defined summary\-to\-tool advantage path*before*the backward pass, at zero additional rollout cost and within a single unified policy \(§[3\.3](https://arxiv.org/html/2609.29050#S3.SS3)\)\. This routing is a deliberate bias–variance trade\-off: SLCA does not claim that tool calls are causally irrelevant to final answers; rather, it assumes that a dense execution reward is the lower\-variance and sufficiently informative signal for updating tool\-decision tokens, while summary rewards train the articulation segment\. Crucially, SLCA operates on a*structural*axis \(execution vs\. articulation\) that is orthogonal to the*temporal*axis addressed by VinePPO / SPO / GiGPO: it therefore*composes with*rather than replaces these methods, and can be combined with temporal credit\-assignment within each segment\. A Schema\-Guided LLM Simulator \(SGLS, §[3\.4](https://arxiv.org/html/2609.29050#S3.SS4)\) and Hierarchical Rewards \(HierR, §[3\.2](https://arxiv.org/html/2609.29050#S3.SS2)\) supply the infrastructure that makes this routing practical at scale\.

Across three backbones \(Qwen2\.5\-3B/7B\-Instruct, Qwen3\-8B\-Base\) and three benchmarks,SLCA\-GRPOhas higher reported means than matched GRPO on the main comparisons\. On the 7B backbone, the matched mean gaps are \+2\.53 pp on in\-domain Toucan, \+1\.36 pp on BFCL, and \+9\.15 pp onτ2\\tau^\{2\}\-Bench\. The corresponding Toucan gaps are \+2\.35 pp \(3B\) and \+2\.05 pp \(8B\); complete multi\-run results and method\-specific baseline protocols are reported in the experiments and appendix\. Our contributions are threefold:

- •We identifyCross\-Segment Credit Misattributionas a structural failure mode of on\-policy RL for tool\-calling agents, arising from advantage contamination rather than gradient\-direction conflict\.
- •We proposeSLCA\-GRPO, an intra\-trajectory estimator that separately normalizes and routes segment advantages without additional rollouts, complemented by the SGLS simulator and HierR rewards\.
- •We evaluateSLCA\-GRPOacross three scales and three benchmarks, with ablations \(w/o SLCA / SGLS / HierR\) and a reward\-protocol sensitivity experiment\.

## 2 Related Work

### 2\.1 Tool\-Calling Post\-Training

Tool\-calling agents are commonly initialized with SFT on annotated tool\-use trajectories\([Schick et al\., 2023](https://arxiv.org/html/2609.29050#bib.bib29);[Qin et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib23)\), often followed by preference\- or verification\-driven optimization\([Ouyang et al\., 2022](https://arxiv.org/html/2609.29050#bib.bib19);[Christiano et al\., 2017](https://arxiv.org/html/2609.29050#bib.bib3);[Stiennon et al\., 2020](https://arxiv.org/html/2609.29050#bib.bib32);[Rafailov et al\., 2023](https://arxiv.org/html/2609.29050#bib.bib26)\)\. However, SFT can be brittle under rule variants and OOD shifts due to memorization and exposure bias\([Chu et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib4);[Bengio et al\., 2015](https://arxiv.org/html/2609.29050#bib.bib2);[Ross et al\., 2011](https://arxiv.org/html/2609.29050#bib.bib27);[Wei et al\., 2025b](https://arxiv.org/html/2609.29050#bib.bib37);[Lightman et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib16);[Guo et al\., 2025a](https://arxiv.org/html/2609.29050#bib.bib6)\)\. As tool namespaces scale, executable benchmarks such as ToolBench, API\-Bank, and BFCL make post\-training robustness increasingly central\([Qin et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib23);[Li et al\., 2023](https://arxiv.org/html/2609.29050#bib.bib13);[Patil et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib20)\)\. We target this post\-training setting, where tool trajectories and final summaries form heterogeneous segments but standard end\-to\-end objectives still broadcast a single advantage to all tokens\. Scalable tool\-agent training also relies on simulated or emulated tool environments, since live API interaction can be costly, unstable, or unavailable at RL scale\([Guo et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib8);[Ruan et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib28);[Li et al\., 2026b](https://arxiv.org/html/2609.29050#bib.bib14)\)\. Our SGLS follows this line but uses schemas as a control plane so that the simulator supplies segment\-specific feedback for credit\-assignment analysis\.

### 2\.2 Credit Assignment for Agentic RL

On\-policy RL for tool use inherits credit\-assignment issues from sparse trajectory\-level optimization\([Williams, 1992](https://arxiv.org/html/2609.29050#bib.bib38);[Schulman et al\., 2015](https://arxiv.org/html/2609.29050#bib.bib30);[Schulman et al\., 2017](https://arxiv.org/html/2609.29050#bib.bib31);[Guo et al\., 2025a](https://arxiv.org/html/2609.29050#bib.bib6)\)\. Process supervision, PRMs, and reward\-centric tool RL provide denser feedback\([Lightman et al\., 2024](https://arxiv.org/html/2609.29050#bib.bib16);[Uesato et al\., 2022](https://arxiv.org/html/2609.29050#bib.bib33);[Qian et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib22);[Lin et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib17)\), but dense rewards do not by themselves prevent aggregation before advantage estimation\. Existing credit\-assignment methods mainly operate along a*temporal*axis\([Kazemnejad et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib10);[Feng et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib5);[Guo et al\., 2025b](https://arxiv.org/html/2609.29050#bib.bib7)\)or use token\-level refinements, while ToolPO\([Li et al\., 2026b](https://arxiv.org/html/2609.29050#bib.bib14)\)adds local tool rewards and RLTR\([Li et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib15)\)separates planner and summarizer into a pipeline\. SLCA instead decouples advantage estimation along a*structural*axis within the same sampled rollout group: it routes independently normalized tool and summary advantages to their corresponding token segments without intermediate\-state rollouts\. This structural decomposition targets a different axis from temporal credit assignment; we do not evaluate a combined method here\.

## 3 Methodology

#### Core Design

SLCA\-GRPOturns the diagnosis above into three design requirements: expose structural token segments, obtain segment\-specific feedback, and block cross\-segment advantages before optimization\. As summarized in[Figure1](https://arxiv.org/html/2609.29050#S3.F1), the framework implements these requirements through mask\-based segmentation, HierR segment returns, SLCA routing, SGLS rollouts, and a unified PPO\-style objective, in contrast to post\-hoc gradient\-projection methods\([Yu et al\., 2020](https://arxiv.org/html/2609.29050#bib.bib42)\); a schematic comparison is provided in App\.[4](https://arxiv.org/html/2609.29050#A1.F4)\.

![Refer to caption](https://arxiv.org/html/2609.29050v1/framework_arch.png)Figure 1:Overview of theSLCA\-GRPOFramework\. \(Top\) Infrastructure: SGLS enables scalable exploration via schema\-constrained simulation, generating interleaved trajectories\. \(Middle\) Signal: HierR decouples feedback into dense execution rewards \(Rtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\) and terminal outcome rewards \(Rsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\)\. \(Bottom\) Optimization: SLCA decouples segment\-wise advantages\. By normalizing and routing advantages independently \(A^tool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}^\{\\mathrm\{tool\}\}\}vs\.A^sum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}\), it blocks the defined summary\-to\-tool support path within each policy update\.
### 3\.1 Problem Formulation & Segment Decomposition

#### Heterogeneous Rollouts

To lock credit to the right semantic target, the structural boundary must first be made explicit\. Given an instructionx∼𝒟x\\sim\\mathcal\{D\}, the policyπθ\\pi\_\{\\theta\}generates a trajectoryy=\(y1,…,yT\)y=\(y\_\{1\},\\dots,y\_\{T\}\)that decomposes asy=\[ytool⊕ysum\]y=\[\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\\oplus\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}\], whereytool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}comprises reasoning traces and structured tool calls, andysum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}is the final user\-facing response\.

#### Gradient Masking

To align optimization with this structure, a binary maskmi,t∈\{0,1\}m\_\{i,t\}\\in\\\{0,1\\\}identifies*learnable policy tokens*\(mi,t=1m\_\{i,t\}=1\) versus*frozen environment contexts*\(mi,t=0m\_\{i,t\}=0\)\. This mask restricts gradient propagation to agent actions and provides the structural signal for our automatic segment decomposition\.

#### Automatic Segment Decomposition

This mask partitions learnable tokens into disjoint semantic sets without training auxiliary segmenters\. For a rolloutii, theSummary Segment𝒯i,sum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathcal\{T\}\_\{i,\\mathrm\{sum\}\}\}is the*final contiguous run*of learnable tokens\. TheTool Segment𝒯i,tool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\}is the union of*all*preceding learnable runs, capturing the entire reasoning\-action chain including any intermediate thinking blocks between tool calls\. This deterministic mapping ensures that every gradient\-bearing token is uniquely assigned to a semantic role, as long as the trajectory culminates in a single final\-answer block \(see App\.[A\.1](https://arxiv.org/html/2609.29050#A1.SS1)for edge cases\)\.Scope\.SLCA targets the*structural*axis \(execution vs\. articulation\) rather than the*temporal*axis within the tool segment \(e\.g\., Action 1 vs\. Action 2 in a ReAct chain\)\. Thus, SLCA is not a variant of VinePPO/GiGPO/SPO, which address temporal credit assignment under different grouping assumptions \(§[2](https://arxiv.org/html/2609.29050#S2)\)\.

### 3\.2 HierR and Segment Returns

To make segment\-locked routing meaningful, the reward must also separate execution quality from response articulation\. We decompose the return into two semantically distinct components:R⁡\(y\)=Rtool​\(ytool\)\+Rsum​\(ysum∣ytool,o\)R\(y\)=\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\)\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\(\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\},o\)\.HierRinstantiates these signals as:\(i\)Process Reward: A dense, structure\-aware score for tool correctness and efficiency\.\(ii\)Summary Preference: A terminal score evaluating the final answer quality\.

For theii\-th sampled rolloutyiy\_\{i\}, the segment rewards are computed by applying these functions directly:

Ritool=Sprocess​\(yi,tool\),Risum=Ssummary​\(yi,sum∣yi,tool,oi\)\.\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{i\}^\{\\mathrm\{tool\}\}\}=S\_\{\\text\{process\}\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{i,\\mathrm\{tool\}\}\}\),\\quad\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{i\}^\{\\mathrm\{sum\}\}\}=S\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{summary\}\}\}\(\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{i,\\mathrm\{sum\}\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{i,\\mathrm\{tool\}\}\},o\_\{i\}\)\.\(1\)Full reward definitions and matching logic are detailed in App\.[A\.4](https://arxiv.org/html/2609.29050#A1.SS4)\. Together, SGLS and HierR build on prior tool\-use simulation and process\-supervision work, but serve a specific role here: providing scalable, segment\-specific feedback for testing structural credit routing\.

### 3\.3SLCA\-GRPO

Given segment boundaries and segment returns, SLCA directly blocks the contamination channel identified in the introduction\. Standard GRPO broadcasts a single trajectory\-level advantage derived fromRtotalR\_\{\\text\{total\}\}to all learnable tokens, coupling tool\-token gradients with summary rewards\.

#### Decoupled Advantage Estimation

For each rolloutii, the two segment rewards\(Ritool,Risum\)\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{i\}^\{\\mathrm\{tool\}\}\},\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{i\}^\{\\mathrm\{sum\}\}\}\)from Eq\.[1](https://arxiv.org/html/2609.29050#S3.E1)are normalized*separately*within the group:

A^itool=Ritool−μgtoolσgtool\+ϵnorm,A^isum=Risum−μgsumσgsum\+ϵnorm,\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}\_\{i\}^\{\\mathrm\{tool\}\}\}=\\frac\{\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{i\}^\{\\mathrm\{tool\}\}\}\-\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mu\_\{g\}^\{\\mathrm\{tool\}\}\}\}\{\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\sigma\_\{g\}^\{\\mathrm\{tool\}\}\}\+\\epsilon\_\{\\text\{norm\}\}\},\\quad\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}\_\{i\}^\{\\mathrm\{sum\}\}\}=\\frac\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{i\}^\{\\mathrm\{sum\}\}\}\-\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mu\_\{g\}^\{\\mathrm\{sum\}\}\}\}\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\sigma\_\{g\}^\{\\mathrm\{sum\}\}\}\+\\epsilon\_\{\\text\{norm\}\}\},\(2\)whereμg\\mu\_\{g\}andσg\\sigma\_\{g\}denote the mean and standard deviation within the group, andϵnorm\\epsilon\_\{\\text\{norm\}\}is a numerical floor\. The token\-level advantageA^i,tSLCA\\hat\{A\}^\{\\text\{SLCA\}\}\_\{i,t\}is then constructed by routing these signals exclusively to their semantic counterparts:

A^i,tSLCA=\{λtool​A^itool,t∈𝒯i,tool,λsum​A^isum,t∈𝒯i,sum,0,mi,t=0,\\hat\{A\}^\{\\text\{SLCA\}\}\_\{i,t\}=\\begin\{cases\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\lambda\_\{\\mathrm\{tool\}\}\}\\,\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}\_\{i\}^\{\\mathrm\{tool\}\}\},&t\\in\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\},\\\\ \{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\lambda\_\{\\mathrm\{sum\}\}\}\\,\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}\_\{i\}^\{\\mathrm\{sum\}\}\},&t\\in\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathcal\{T\}\_\{i,\\mathrm\{sum\}\}\},\\\\ 0,&m\_\{i,t\}=0,\\end\{cases\}\(3\)This construction changes only the scalar advantage attached to each token; it does not split the model or require additional rollouts\. Tool tokens are updated only by execution quality, while summary tokens are updated only by response quality\. Thus, summary rewards can still train final\-answer articulation, but no longer supply gradients to tool\-decision tokens\.

#### Robust Optimization Mechanisms

For sparse\-tool stability, the implementation uses presence filtering, post\-normalization weighting, and omission\-penalty routing; details are in App\.[A\.3](https://arxiv.org/html/2609.29050#A1.SS3)\.

#### Theoretical Properties

The key per\-update guarantee of SLCA is advantage isolation: the tool\-token gradient component is functionally independent of the summary reward,

∂gtoolSLCA∂Rsum=𝟎\.\\frac\{\\partial\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}g\_\{\\mathrm\{tool\}\}\}^\{\\text\{SLCA\}\}\}\{\\partial\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\}=\\mathbf\{0\}\.\(4\)Beyond this isolation property, local score\-function analysis shows that SLCA removes a positive summary\-noise variance term from tool updates and preserves the tool\-execution direction in sign\-conflict regimes\. These are conditional, per\-update guarantees rather than claims of global bias\-free optimization; full assumptions and proofs are in App\.[A\.2](https://arxiv.org/html/2609.29050#A1.SS2)\.

### 3\.4 Scalable Exploration via SGLS

To test structural credit routing at scale, SGLS supplies schema\-consistent observations without relying on live APIs\. It combines deterministic schema validation with frozen cross\-family LLM mocking, so invalid calls receive immediate structured feedback and valid calls receive plausible tool responses; deployment details and alignment examples are in App\.[C\.3](https://arxiv.org/html/2609.29050#A3.SS3)\. The cross\-family design mitigates implicit leakage while preserving the semantic topology needed for transfer\.

### 3\.5 Objective and Training Algorithm

Finally, a single unified policy is trained by replacing GRPO’s unified scalar advantage with the routed segment advantage\. The objective is PPO\-clip\([Schulman et al\., 2017](https://arxiv.org/html/2609.29050#bib.bib31)\)with segment\-locked advantages\. Letρi,t​\(θ\)=πθ​\(yi,t∣hi,t\)πold​\(yi,t∣hi,t\)\\rho\_\{i,t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid h\_\{i,t\}\)\}\{\\pi\_\{\\text\{old\}\}\(y\_\{i,t\}\\mid h\_\{i,t\}\)\}; a per\-token KL penaltyβ𝔻KL\(πθ∥πref\)\\beta\\mathbb\{D\}\_\{\\text\{KL\}\}\(\\pi\_\{\\theta\}\\\|\\pi\_\{\\text\{ref\}\}\)is enforced but omitted below for brevity:

𝒥\(θ\)=𝔼x∼𝒟,\{yi\}i=1G∼πθold\(⋅∣x\)\[1N∑i,tmi,t⋅min\(ρi,tA^i,tSLCA,clip\(ρi,t,1−ϵclip,1\+ϵclip\)A^i,tSLCA\)\],\\mathcal\{J\}\(\\theta\)=\\mathbb\{E\}\_\{x\\sim\\mathcal\{D\},\\,\\\{y\_\{i\}\\\}\_\{i=1\}^\{G\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid x\)\}\\Bigg\[\\frac\{1\}\{N\}\\sum\_\{i,t\}m\_\{i,t\}\\cdot\\min\\Big\(\\rho\_\{i,t\}\\hat\{A\}^\{\\text\{SLCA\}\}\_\{i,t\},\\text\{clip\}\(\\rho\_\{i,t\},1\-\\epsilon\_\{\\text\{clip\}\},1\+\\epsilon\_\{\\text\{clip\}\}\)\\hat\{A\}^\{\\text\{SLCA\}\}\_\{i,t\}\\Big\)\\Bigg\],\(5\)whereGGis the rollout group size andN=∑i,tmi,tN=\\sum\_\{i,t\}m\_\{i,t\}is the total valid token count\. The complete training procedure is summarized in Algorithm[1](https://arxiv.org/html/2609.29050#alg1)\.

## 4 Experiments

Our experiments proceed in three parts: we examine the diagnosed failure under unified or additive credit signals, test transfer across schemas and long\-horizon interaction, and ablate SLCA, SGLS, and HierR\.

### 4\.1 Experimental Setup

#### Datasets and Benchmarks

SLCA\-GRPOis evaluated across three dimensions: in\-domain mastery, cross\-distribution generalization, and collaborative robustness\.\(i\)Toucan\-1\.5M \(In\-Domain\):The Toucan dataset\([Xu et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib39)\)is used for both SFT initialization and RL post\-training\. A multi\-stage filtering pipeline \(App\.[C\.1](https://arxiv.org/html/2609.29050#A3.SS1)\) yields 42,423 SFT samples and 31,818 RL training samples \(spanning single\-turn and decomposed multi\-turn trajectories\)\. Evaluation uses a held\-out Toucan\-Test set of 4,000 samples\.\(ii\)BFCL V3 \(Generalization\):Generalization is evaluated on the Berkeley Function\-Calling Leaderboard\([Patil et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib20)\)\.111For the main BFCL V3 comparison, relevance detection is excluded and evaluation is restricted tosingle\-turnsamples\. This isolatesatomicgeneralization to unseen schemas \(e\.g\., AST\); the appendix additionally reports BFCL Multi\-Turn accuracy, whileτ2\\tau^\{2\}\-Bench covers longer\-horizon collaboration\.\(iii\)τ2\\tau^\{2\}\-Bench \(Robustness\):Robustness is measured in dynamic dual\-control environments \(Airline, Retail, Telecom\) where agents must collaborate with users to manipulate state, reporting Pass1\([Barres et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib1)\)\.Full protocols are in App\.[C\.5](https://arxiv.org/html/2609.29050#A3.SS5)\.

#### Baselines and Training

The methods are implemented on Qwen2\.5\-7B\-Instruct\([Qwen Team, 2025a](https://arxiv.org/html/2609.29050#bib.bib24)\)\(default\), Qwen2\.5\-3B\-Instruct, and Qwen3\-8B\-Base\([Qwen Team, 2025b](https://arxiv.org/html/2609.29050#bib.bib25)\)to verify scalability, comparing six paradigms \(details in App\.[C\.2](https://arxiv.org/html/2609.29050#A3.SS2)\):\(i\)Original Backbones: The original model weights evaluated directly without any exposure to the Toucan training set\.\(ii\)SFT \(Toucan\): A behavioral cloning baseline fine\-tuned on the union of the standard SFT partition \(42k\) and the raw source trajectories of the RL partition \(31k\)\.\(iii\)SFT\+GRPO \(Baseline\): The standard GRPO algorithm using the same HierR rewards and SGLS environment asSLCA\-GRPO, with a unified advantage computed fromRtotal=Rtool\+RsumR\_\{\\text\{total\}\}=\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\.\(iv\)ToolPO\([Li et al\., 2026b](https://arxiv.org/html/2609.29050#bib.bib14)\): An additive credit\-assignment baseline evaluated with its global outcome and local tool rewards\. Tool tokens receive their sum, while summary tokens inherit the global term\. The outcome\-reward protocol and its sensitivity diagnostic are documented in App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)\.\(v\)RLTR \(adapted\)\([Li et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib15)\): A planner\-summarizer baseline with a dedicated planner\-only SFT initialization and a frozen summarizer, following a two\-stage planning pipeline\.\(vi\)SFT\+SLCA\-GRPO\(Ours\): Our proposed framework employing Segment\-Locked Credit Assignment\.The controlled isolation comparison is between SFT\+GRPO and SFT\+SLCA\-GRPO: these two rows share the same data partitions, SFT initialization, SGLS endpoint, mocker configuration, decoding settings, evaluation protocol, rollout group sizeG=16G=16, one RL epoch, and HierR rewards, differing in unified versus segment\-locked advantage computation\. The matched ablations use the same per\-backbone run set; w/o HierR removes HierR, and w/o SGLS removes schema conditioning and deterministic validation\. ToolPO uses the LLM\-judge outcome reward in the main tables, while RLTR retains its method\-specific protocol; the reference\-rule ToolPO sensitivity diagnostic is reported in App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)\.

### 4\.2 Main Results

#### Toucan\-Test

The in\-domain evaluation first asks whether structural routing improves execution under matched data, simulator, and reward conditions\. Table[1](https://arxiv.org/html/2609.29050#S4.T1)reports the matched GRPO comparison together with the method\-specific baseline rows \(full results in App\.[D](https://arxiv.org/html/2609.29050#A4)\)\. On Qwen2\.5\-7B\-Instruct,SLCA\-GRPOhas a 79\.13% three\-run mean for strict Success@0\.9, 2\.53 pp above matched GRPO\. The corresponding mean gaps are \+2\.35 pp on Qwen2\.5\-3B\-Instruct and \+2\.05 pp on Qwen3\-8B\-Base\.

Table 1:Toucan\-Test Results\. Metrics: Name F1: Multiset F1 score measuring precision and recall of predicted tool\-name occurrences against gold names; ArgMatch: Arithmetic mean of argument key and value matching scores; Process: The dense tool\-segment reward \(SprocessS\_\{\\text\{process\}\}\) defined in Eq\.[22](https://arxiv.org/html/2609.29050#A1.E22)\(App\.[A\.4](https://arxiv.org/html/2609.29050#A1.SS4)\); Success: Strict binary indicator \(𝕀\[Sprocess≥0\.9\]\\mathbb\{I\}\[S\_\{\\text\{process\}\}\\geq 0\.9\]\) obtained by thresholding the process score\. All trained entries report mean±\\pmstd over three runs; Original rows are point evaluations\. The matched rows share the per\-backbone SFT initialization, data split, SGLS endpoint, mocker configuration, decoding settings, evaluation protocol,G=16G=16, one RL epoch, and run set\. GRPO,SLCA\-GRPO, w/o SLCA, and w/o SGLS also share the HierR definition and subweights; w/o HierR removes HierR\. ToolPO and RLTR retain their method\-specific protocols\. The ToolPO rows use the LLM\-judge outcome reward; App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)reports how this baseline responds to a rule\-based outcome reward under the same estimator\.
#### Comparison with Prior Credit\-Assignment Baselines

The next comparison evaluates prior paradigms that either augment or separate the tool\-planning signal rather than route segment advantages\. RLTR \(pipeline separation\) and ToolPO \(additive augmentation\) provide complementary reference points for the segment\-locked estimator\. Their training traces show capacity\-dependent format and invocation patterns \(App\.[D\.3](https://arxiv.org/html/2609.29050#A4.SS3)\); the main\-table ToolPO rows use the LLM\-judge outcome reward described in App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)\.

#### Training Dynamics and Efficiency

Training traces reveal whether the final gains arise from stable correction of the misattribution channel rather than late\-stage overfitting\.[Figure2](https://arxiv.org/html/2609.29050#S4.F2)contrasts training behavior from two views\. In the cross\-scale training curves \([Figure2\(a\)](https://arxiv.org/html/2609.29050#S4.F2.sf1)\), the plotted traces are representative single runs\.SLCA\-GRPOrises across the three backbones\. The 7B ToolPO curve is a separate diagnostic: it declines after step 40, with the structural divergence appearing after step 80 \(App\.[D\.3](https://arxiv.org/html/2609.29050#A4.SS3)\); RLTR changes more slowly under its coarse completeness reward\. Across the representative traces \(3B/7B/8B;[Figure2\(b\)](https://arxiv.org/html/2609.29050#S4.F2.sf2)\),SLCA\-GRPOends with*fewer*average tool turns and*higher*success rates than standard GRPO\. The gap in trajectory length is narrow, and the ordering is consistent across the plotted traces\. Standard GRPO stabilizes at longer trajectories without a corresponding success gain, compatible with “performative execution”: padding trajectories to exploit summary rewards rather than improving tool\-call precision\. These dynamics provide evidence for the failure mode diagnosed in the introduction: additive or unified signals can improve proxy rewards while degrading executable tool behavior\.

\(a\)Training curves across 3B/7B/8B\.Representative single\-run traces\.\(b\)Across 3B/7B/8B: success vs\. tool turns\.The plotted traces showSLCA\-GRPO\(solid\) ending with shorter trajectories than GRPO \(dashed\)\.
Figure 2:Training Dynamics\.\(a\) Baseline comparison across 3B/7B/8Bfor SLCA / ToolPO / RLTR \(the ToolPO diagnostic is described in App\.[D\.3](https://arxiv.org/html/2609.29050#A4.SS3)\), and\(b\) cross\-scale dynamicsof success rate and average tool turns over training steps\.
#### Generalization on BFCL andτ2\\tau^\{2\}\-Bench

The OOD evaluation then tests whether correcting structural credit assignment transfers beyond the in\-domain simulator setting\.[Figure3](https://arxiv.org/html/2609.29050#S4.F3)comparesSLCA\-GRPOwith matched GRPO and the method\-specific baselines on two OOD benchmarks across scales \(protocols in App\.[C\.5](https://arxiv.org/html/2609.29050#A3.SS5)\)\. On BFCL \([Figure3\(a\)](https://arxiv.org/html/2609.29050#S4.F3.sf1)\), SLCA reaches70\.31±\\pm0\.50%Overall Accuracy on Qwen3\-8B\-Base, compared with 66\.96±\\pm0\.32% for matched GRPO and 68\.50±\\pm0\.35% for SFT\. The 7B BFCL gap over standard GRPO is \+1\.36 pp; the corresponding gaps are \+0\.40 pp on 3B and \+3\.35 pp on 8B\. Onτ2\\tau^\{2\}\-Bench, the matched gaps are \+1\.09 pp \(3B\), \+9\.15 pp \(7B\), and \+10\.03 pp \(8B\), all based on three\-run means\. ToolPO outcome\-reward details and the LLM\-judge protocol used in the scale rows are documented in App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)\. Multi\-Turn BFCL results and per\-domainτ2\\tau^\{2\}\-Bench breakdowns are in App\.[D\.5](https://arxiv.org/html/2609.29050#A4.SS5)and App\.[D\.6](https://arxiv.org/html/2609.29050#A4.SS6)\. Robustness under larger candidate tool spaces \(KKup to 150\) with hard\-negative distractors is evaluated in App\.[D\.11](https://arxiv.org/html/2609.29050#A4.SS11)\.

\(a\)BFCL V3 \(Single\-Turn; mean±\\pmstd\)\.Trained bars show three\-run means with one\-standard\-deviation error bars; Original is a point evaluation\.\(b\)τ2\\tau^\{2\}\-Bench \(Overall Pass1; mean±\\pmstd\)\.Trained rows are three\-run means with one\-standard\-deviation error bars; Original is a point evaluation\. The 8B Original point is omitted because its four domain scores are zero\.
Figure 3:OOD Generalization across Scales\.\(a\) Atomic generalization on BFCL V3and\(b\) Collaboration robustness onτ2\\tau^\{2\}\-Benchfor the three backbones \(3B, 7B, 8B\), comparingSLCA\-GRPOagainst standard GRPO, RLTR, and ToolPO\. Per\-domainτ2\\tau^\{2\}\-Bench details are in App\.[D\.6](https://arxiv.org/html/2609.29050#A4.SS6)\.

### 4\.3 Ablation Studies

Finally, the ablations target the method’s three design requirements: routing, scalable simulation, and dense segment feedback\. Qwen2\.5\-7B\-Instruct is used as the primary testbed \(Table[2](https://arxiv.org/html/2609.29050#S4.T2)\), with detailed ablation across scales in App\.[D\.10](https://arxiv.org/html/2609.29050#A4.SS10)\.

\(i\)w/o SLCA \(Unified Advantage\): Reverting to the standard unified advantage \(Rtool\+Rsum\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\) changes both normalization and token support; it lowers Toucan Success by 2\.53 pp on 7B under the matched setting\. The support\-control experiment in App\.[D\.9](https://arxiv.org/html/2609.29050#A4.SS9)separately measures the contribution of removing the summary\-to\-tool support path\.\(ii\)w/o SGLS:Removing deterministic schema validation and schema\-conditioned tool\-response prompting lowers the reported OOD transfer metrics \(Table[2](https://arxiv.org/html/2609.29050#S4.T2)\); its Multi\-Turn effect is reported in App\.[D\.5](https://arxiv.org/html/2609.29050#A4.SS5)\. Schema\-constrained simulation supplies the controlled exploration interface\.\(iii\)w/o HierR:Relying solely on sparse summary rewards leads to slow convergence and poor argument alignment on complex queries\.

The matched “w/o SLCA” condition compares the unified and segment\-locked estimators under fixed reward definitions, so it measures the combined estimator change rather than either operation in isolation; the unified reward ratio sweep in App\.[D\.8](https://arxiv.org/html/2609.29050#A4.SS8)examines this distinction directly\. Together, these ablations test the three design requirements in Section[3](https://arxiv.org/html/2609.29050#S3): routing addresses the cross\-segment path, SGLS stabilizes schema\-grounded exploration, and HierR supplies dense execution feedback\. The summary advantage ablation and the 7B support control are reported in Apps\.[D\.2](https://arxiv.org/html/2609.29050#A4.SS2)and[D\.9](https://arxiv.org/html/2609.29050#A4.SS9)\.

Table 2:Ablation Studies on Qwen2\.5\-7B\-Instruct\. Entries report mean±\\pmstd over three independent runs\. Full results are provided in App\.[D\.10](https://arxiv.org/html/2609.29050#A4.SS10)\. Process is an evaluation metric computed post hoc for all settings, including w/o HierR\.#### Reward and response\-mocker checks

We also test SLCA with a binaryRsuccR\_\{\\text\{succ\}\}judge without per\-example gold\-call matching and with a GPT\-OSS\-120B response mocker; the matched results are reported in Apps\.[D\.12](https://arxiv.org/html/2609.29050#A4.SS12)and[C\.3](https://arxiv.org/html/2609.29050#A3.SS3)\.

## 5 Conclusion

We identified Global Signal Conflation as a structural pathology in tool\-calling RL and proposedSLCA\-GRPO, which decouples advantage estimation at the segment level\. Across three backbones,SLCA\-GRPOimproves success by\+2\.53 ppon 7B in\-domain Toucan,\+1\.36 ppon BFCL, and\+9\.15 pponτ2\\tau^\{2\}\-Bench\. The corresponding matched gaps are \+2\.35/\+0\.40/\+1\.09 pp on 3B and \+2\.05/\+3\.35/\+10\.03 pp on 8B for Toucan, BFCL, andτ2\\tau^\{2\}\-Bench, respectively\. The Toucan breakdown shows higher mean Process and Success forSLCA\-GRPOacross the reported backbones, while higher Summary scores for matched GRPO on 3B and 8B do not translate into higher task Success\. Controlled comparisons and ablations support the role of segment\-level routing, and the reward\-swap check shows that the improvement is not tied to a single reward formulation\. SLCA is complementary to temporal credit\-assignment methods\.

## 6 Limitations

Segment decomposition\.SLCA assumes a clear boundary between tool calls and free\-form text\. Settings without one, such as inline code generation, would require learned segmentation\.Reward and bias\.SprocessS\_\{\\text\{process\}\}relies on matching to gold trajectories and may under\-reward alternative valid tool sequences when multiple API plans solve the same request\. The execution\-based judge check in App\.[D\.12](https://arxiv.org/html/2609.29050#A4.SS12)provides a complementary reward formulation\. Toucan strict Success@0\.9 thresholds this score instead of using an independent task outcome label\.A^sum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}inherits a grouping bias in summary tokens \(App\.[A\.2](https://arxiv.org/html/2609.29050#A1.SS2.SSS0.Px5)\); explicit subgrouping would shrink group size exponentially\.Intra\-tool temporal credit\.SLCA routes one scalarA^tool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}^\{\\mathrm\{tool\}\}\}to every tool token, so it does not distinguish a correct Action 1 from a failing Action 2\. This temporal issue is separate from cross\-segment credit\. A combination with VinePPO/GiGPO/SPO remains untested\.Scale and simulation\.Experiments use models up to 8B with SGLS\. Larger\-scale training and direct real\-API comparisons remain future work\.

## References

- Barres et al\. \(2025\)Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan\. 2025\.τ2\\tau^\{2\}\-Bench: Evaluating conversational agents in a dual\-control environment\.*arXiv preprint arXiv:2506\.07982*\.
- Bengio et al\. \(2015\)Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer\. 2015\.Scheduled sampling for sequence prediction with recurrent neural networks\.In*Advances in Neural Information Processing Systems*, volume 28, pages 1171–1179\. Curran Associates, Inc\.
- Christiano et al\. \(2017\)Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei\. 2017\.Deep reinforcement learning from human preferences\.In*Advances in Neural Information Processing Systems*, volume 30, pages 4299–4307\. Curran Associates, Inc\.
- Chu et al\. \(2025\)Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma\. 2025\.SFT memorizes, RL generalizes: A comparative study of foundation model post\-training\.In*Proceedings of the 42nd International Conference on Machine Learning*, volume 267, pages 10818–10838\.
- Feng et al\. \(2025\)Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An\. 2025\.[Group\-in\-group policy optimization for LLM agent training](https://doi.org/10.52202/085713-1544)\.In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*, volume 38, pages 51797–51830\.
- Guo et al\. \(2025a\)Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, et al\. 2025a\.[DeepSeek\-R1 incentivizes reasoning in LLMs through reinforcement learning](https://doi.org/10.1038/s41586-025-09422-z)\.*Nature*, 645:633–638\.
- Guo et al\. \(2025b\)Yiran Guo, Lijie Xu, Jie Liu, Dan Ye, and Shuang Qiu\. 2025b\.[Segment policy optimization: Effective segment\-level credit assignment in RL for large language models](https://doi.org/10.52202/085713-3815)\.In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*, volume 38, pages 126866–126898\.
- Guo et al\. \(2024\)Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu\. 2024\.[StableToolBench: Towards stable large\-scale benchmarking on tool learning of large language models](https://doi.org/10.18653/v1/2024.findings-acl.664)\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 11143–11156\. Association for Computational Linguistics\.
- Hou et al\. \(2026\)Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang\. 2026\.[Model Context Protocol \(MCP\): Landscape, security threats, and future research directions](https://doi.org/10.1145/3796519)\.*ACM Transactions on Software Engineering and Methodology*\.
- Kazemnejad et al\. \(2025\)Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux\. 2025\.VinePPO: Refining credit assignment in RL training of LLMs\.In*Proceedings of the 42nd International Conference on Machine Learning*, volume 267, pages 29557–29590\.
- Kingma and Ba \(2015\)Diederik P\. Kingma and Jimmy Ba\. 2015\.Adam: A method for stochastic optimization\.In*3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7\-9, 2015, Conference Track Proceedings*\.
- Li et al\. \(2026a\)Dawei Li, Renliang Sun, Yue Huang, Ming Zhong, Bohan Jiang, Jiawei Han, Xiangliang Zhang, Wei Wang, and Huan Liu\. 2026a\.[Preference leakage: A contamination problem in LLM\-as\-a\-judge](https://openreview.net/forum?id=grIvSXVJ65)\.In*International Conference on Learning Representations*\.
- Li et al\. \(2023\)Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li\. 2023\.API\-Bank: A comprehensive benchmark for tool\-augmented LLMs\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 3102–3116\. Association for Computational Linguistics\.
- Li et al\. \(2026b\)Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Guanting Dong, Jiajie Jin, Yinuo Wang, Hao Wang, Yutao Zhu, Ji\-Rong Wen, Yuan Lu, and Zhicheng Dou\. 2026b\.[DeepAgent: A general reasoning agent with scalable toolsets](https://doi.org/10.1145/3774904.3792460)\.In*Proceedings of the ACM Web Conference 2026*, pages 2219–2230\. Association for Computing Machinery\.
- Li et al\. \(2025\)Zhiwei Li, Yong Hu, and Wenqing Wang\. 2025\.[Encouraging good processes without the need for good answers: Reinforcement learning for LLM agent planning](https://doi.org/10.18653/v1/2025.emnlp-industry.116)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track*, pages 1654–1666\. Association for Computational Linguistics\.
- Lightman et al\. \(2024\)Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe\. 2024\.Let’s verify step by step\.In*The Twelfth International Conference on Learning Representations*\.
- Lin et al\. \(2025\)Zihan Lin, Xiaohan Wang, Hexiong Yang, Jiajun Chai, Jie Cao, Guojun Yin, Wei Lin, and Ran He\. 2025\.AWPO: Enhancing tool\-use of large language models through adaptive integration of reasoning rewards\.*arXiv preprint arXiv:2512\.19126*\.
- Loshchilov and Hutter \(2019\)Ilya Loshchilov and Frank Hutter\. 2019\.Decoupled weight decay regularization\.In*International Conference on Learning Representations*\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe\. 2022\.Training language models to follow instructions with human feedback\.In*Advances in Neural Information Processing Systems*, volume 35, pages 27730–27744\. Curran Associates, Inc\.
- Patil et al\. \(2025\)Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng\-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E\. Gonzalez\. 2025\.[The Berkeley function calling leaderboard \(BFCL\): From tool use to agentic evaluation of large language models](https://openreview.net/forum?id=2GmDdhBdDk)\.In*Proceedings of the 42nd International Conference on Machine Learning*, volume 267, pages 48371–48392\.
- Patil et al\. \(2024\)Shishir G\. Patil, Tianjun Zhang, Xin Wang, and Joseph E\. Gonzalez\. 2024\.Gorilla: Large language model connected with massive APIs\.In*Advances in Neural Information Processing Systems*, volume 37, pages 126544–126565\. Curran Associates, Inc\.
- Qian et al\. \(2025\)Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani\-Tür, Gokhan Tur, and Heng Ji\. 2025\.[ToolRL: Reward is all tool learning needs](https://doi.org/10.52202/085713-3524)\.In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*, volume 38, pages 116896–116926\.
- Qin et al\. \(2024\)Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun\. 2024\.ToolLLM: Facilitating large language models to master 16000\+ real\-world APIs\.In*The Twelfth International Conference on Learning Representations*\.
- Qwen Team \(2025a\)Qwen Team\. 2025a\.Qwen2\.5 technical report\.*arXiv preprint arXiv:2412\.15115*\.
- Qwen Team \(2025b\)Qwen Team\. 2025b\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*\.
- Rafailov et al\. \(2023\)Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn\. 2023\.Direct preference optimization: Your language model is secretly a reward model\.In*Advances in Neural Information Processing Systems*, volume 36, pages 53728–53741\. Curran Associates, Inc\.
- Ross et al\. \(2011\)Stephane Ross, Geoffrey Gordon, and Drew Bagnell\. 2011\.A reduction of imitation learning and structured prediction to no\-regret online learning\.In*Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics*, volume 15, pages 627–635\. PMLR\.
- Ruan et al\. \(2024\)Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris Maddison, and Tatsunori Hashimoto\. 2024\.Identifying the risks of LM agents with an LM\-emulated sandbox\.In*International Conference on Learning Representations*, volume 2024, pages 27031–27098\.
- Schick et al\. \(2023\)Timo Schick, Jane Dwivedi\-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom\. 2023\.Toolformer: Language models can teach themselves to use tools\.In*Advances in Neural Information Processing Systems*, volume 36, pages 68539–68551\. Curran Associates, Inc\.
- Schulman et al\. \(2015\)John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz\. 2015\.Trust region policy optimization\.In*Proceedings of the 32nd International Conference on Machine Learning*, volume 37, pages 1889–1897\. PMLR\.
- Schulman et al\. \(2017\)John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov\. 2017\.Proximal policy optimization algorithms\.In*arXiv preprint arXiv:1707\.06347*\.
- Stiennon et al\. \(2020\)Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano\. 2020\.Learning to summarize with human feedback\.In*Advances in Neural Information Processing Systems*, volume 33, pages 3008–3021\. Curran Associates, Inc\.
- Uesato et al\. \(2022\)Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins\. 2022\.Solving math word problems with process\- and outcome\-based feedback\.*arXiv preprint arXiv:2211\.14275*\.
- Wang et al\. \(2025\)Xiaoxuan Wang, Bo Liu, Song Jiang, Jingzhou Liu, Jingyuan Qi, Xia Chen, and Baosheng He\. 2025\.From solving to verifying: A unified objective for robust reasoning in LLMs\.*arXiv preprint arXiv:2511\.15137*\.
- Wataoka et al\. \(2025\)Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri\. 2025\.Self\-preference bias in LLM\-as\-a\-judge\.*arXiv preprint arXiv:2410\.21819*\.
- Wei et al\. \(2025a\)Quan Wei, Siliang Zeng, Chenliang Li, Zhongruo Wang, William Brown, Oana Frunza, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Yang Katie Zhao, Alfredo Garcia, and Mingyi Hong\. 2025a\.Reinforcing multi\-turn reasoning in LLM agents via fine\-grained reward structure and credit assignment\.*arXiv preprint arXiv:2505\.11821*\.
- Wei et al\. \(2025b\)Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li\. 2025b\.[WebAgent\-R1: Training web agents via end\-to\-end multi\-turn reinforcement learning](https://doi.org/10.18653/v1/2025.emnlp-main.401)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 7909–7928\. Association for Computational Linguistics\.
- Williams \(1992\)Ronald J\. Williams\. 1992\.Simple statistical gradient\-following algorithms for connectionist reinforcement learning\.*Machine Learning*, 8\(3\):229–256\.
- Xu et al\. \(2025\)Zhangchen Xu, Adriana Meza Soria, Shawn Tan, Anurag Roy, Ashish Sunil Agrawal, Radha Poovendran, and Rameswar Panda\. 2025\.TOUCAN: Synthesizing 1\.5m tool\-agentic data from real\-world MCP environments\.*arXiv preprint arXiv:2510\.01179*\.
- Yao et al\. \(2023\)Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao\. 2023\.ReAct: Synergizing reasoning and acting in language models\.In*The Eleventh International Conference on Learning Representations*\.
- Yu et al\. \(2025\)Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, LingJun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, and 17 others\. 2025\.[DAPO: An open\-source LLM reinforcement learning system at scale](https://doi.org/10.52202/085713-3775)\.In*The Thirty\-ninth Annual Conference on Neural Information Processing Systems*, volume 38, pages 125532–125554\.
- Yu et al\. \(2020\)Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn\. 2020\.Gradient surgery for multi\-task learning\.In*Advances in Neural Information Processing Systems*, volume 33, pages 5824–5836\. Curran Associates, Inc\.

## Appendix AAdditional Formalism and Proofs

![Refer to caption](https://arxiv.org/html/2609.29050v1/comparison_grpo.png)Figure 4:Mechanism of Cross\-Segment Credit Misattribution and segment\-locked routing\. Unified advantage broadcast leaks summary rewards into tool tokens, whereas SLCA blocks this path by routing segment\-wise advantages only to their corresponding tokens\.### A\.1 Mask\-Based Segment Decomposition

We derive segment boundaries using only the standard token\-level maskmi,t∈\{0,1\}m\_\{i,t\}\\in\\\{0,1\\\}from the agent loop\. Heremi,t=1m\_\{i,t\}=1denotes a policy\-generated \(learnable\) token andmi,t=0m\_\{i,t\}=0denotes an environment\-injected token\. LetTiT\_\{i\}be the unpadded length of rolloutii\.

#### Runs of ones

Consider the maximal contiguous runs wheremi,t=1m\_\{i,t\}=1\. Let\{\(si,k,ei,k\)\}k=1Ki\\\{\(s\_\{i,k\},e\_\{i,k\}\)\\\}\_\{k=1\}^\{K\_\{i\}\}denote the start and end indices \(left\-closed, right\-open\) of these runs so that

mi,t=1⇔t∈⋃k=1Ki\[si,k,ei,k\),m\_\{i,t\}=1\\iff t\\in\\bigcup\_\{k=1\}^\{K\_\{i\}\}\[s\_\{i,k\},e\_\{i,k\}\),\(6\)with disjoint intervals ordered by time\.

#### Segment definition

We define theSummarysegment as the last run of learnable tokens:

𝒯i,sum≜\[si,Ki,ei,Ki\),\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathcal\{T\}\_\{i,\\mathrm\{sum\}\}\}\\triangleq\[s\_\{i,K\_\{i\}\},e\_\{i,K\_\{i\}\}\),\(7\)and theToolsegment as the union of all earlier learnable runs:

𝒯i,tool≜⋃k=1Ki−1\[si,k,ei,k\)\.\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\}\\triangleq\\bigcup\_\{k=1\}^\{K\_\{i\}\-1\}\[s\_\{i,k\},e\_\{i,k\}\)\.\(8\)

#### Edge cases

IfKi=1K\_\{i\}=1\(no environment injection\), then𝒯i,tool=∅\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\}=\\emptysetand the entire generation is treated as Summary\. If tool usage is required but the rollout contains no parseable tool call, we trigger the omission\-penalty guard \(Appendix[A\.3](https://arxiv.org/html/2609.29050#A1.SS3)\)\.

### A\.2 Theoretical Analysis: Proofs of Main Properties

This section provides the proofs and modeling assumptions behind Proposition[A\.1](https://arxiv.org/html/2609.29050#A1.Thmtheorem1)\(Per\-Step Advantage Isolation\), Theorem[A\.3](https://arxiv.org/html/2609.29050#A1.Thmtheorem3)\(Nuisance\-Variance Removal\), and Corollary[A\.4](https://arxiv.org/html/2609.29050#A1.Thmtheorem4)\(Directional Fidelity\), which are summarized in §[3\.3](https://arxiv.org/html/2609.29050#S3.SS3)\. The variance and direction results analyze a local score\-function estimator that isolates the noise channel induced by unified reward broadcasting; exact GRPO group normalization introduces cross\-sample dependencies, which we discuss explicitly below\.

#### Setup and Definitions

Let a trajectoryyybe partitioned into a tool segmentytool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}and a summary segmentysum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}\. The total reward isR=Rtool\+RsumR=\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\. Letut=∇θ​log​πθ​\(yt∣ht\)u\_\{t\}=\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid h\_\{t\}\)be the score function\. For the tool\-token component, the accumulated score for trajectoryiiisUi,tool=∑t∈𝒯i,toolui,t\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{i,\\mathrm\{tool\}\}\}=\\sum\_\{t\\in\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\}\}u\_\{i,t\}\. Letwiw\_\{i\}be the importance weight \(e\.g\., length normalization\)\. Letℱtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}be theσ\\sigma\-algebra generated byytool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\.

###### Proposition A\.1\(Per\-Step Advantage Isolation\)\.

Under SLCA routing \(Eq\.[3](https://arxiv.org/html/2609.29050#S3.E3)\), the tool\-token component of the gradient estimator has no functional dependence onRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\.

###### Proof\.

The SLCA\-routed tool\-token component can be written as

gtoolSLCA=∑iwi​λtool​A^itool​∑t∈𝒯i,toolui,t\.\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}g\_\{\\mathrm\{tool\}\}\}^\{\\text\{SLCA\}\}=\\sum\_\{i\}w\_\{i\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\lambda\_\{\\mathrm\{tool\}\}\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}\_\{i\}^\{\\mathrm\{tool\}\}\}\\sum\_\{t\\in\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\}\}u\_\{i,t\}\.\(9\)The routed advantageA^itool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}\_\{i\}^\{\\mathrm\{tool\}\}\}is computed only from the tool rewards\{Rjtool\}\\\{\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{j\}^\{\\mathrm\{tool\}\}\}\\\}within the rollout group, while summary tokens are outside the summation\. The tool\-token gradient componentgtoolSLCA\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}g\_\{\\mathrm\{tool\}\}\}^\{\\text\{SLCA\}\}has no functional dependence onRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}, and∂gtoolSLCA/∂Rsum=𝟎\\partial\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}g\_\{\\mathrm\{tool\}\}\}^\{\\text\{SLCA\}\}/\\partial\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}=\\mathbf\{0\}\. Shared parameters may still transfer information across future optimization steps; the proposition is a per\-update statement about the routed gradient estimator\. ∎

#### Part I: Conditional Nuisance\-Variance Removal

We model the summary rewardRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}as containing a stochastic noise termξ\\xi\(e\.g\., phrasing randomness\) that is independent of the tool execution logic\.

###### Assumption A\.2\(Nuisance Noise Decomposition\)\.

Conditioned on the tool trajectoryytool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}, the summary reward decomposes into a deterministic expectation and a zero\-mean noise term:

Rsum=R¯sum​\(ytool\)\+ξ,where​𝔼​\[ξ∣ytool\]=0,𝕍⁡\[ξ∣ytool\]=σξ2\>0\.\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}=\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\)\+\\xi,\\quad\\text\{where \}\\mathbb\{E\}\[\\xi\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\]=0,\\ \\mathbb\{V\}\[\\xi\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\]=\\sigma\_\{\\xi\}^\{2\}\>0\.\(10\)

###### Theorem A\.3\(Nuisance\-Variance Removal: restated from Theorem[A\.3](https://arxiv.org/html/2609.29050#A1.Thmtheorem3)\)\.

LetgSTDg\_\{\\text\{STD\}\}andgSLCAg\_\{\\text\{SLCA\}\}be local score\-function gradient estimators for tool tokens under unified reward broadcasting and SLCA, respectively\. In this local analysis, SLCA removes the irreducible conditional variance term induced by summary\-reward noise:

𝔼⁡\[𝕍⁡\(gSTD∣ℱtool\)\]−𝔼⁡\[𝕍⁡\(gSLCA∣ℱtool\)\]=𝔼⁡\[wi2​‖Utool‖2⋅σξ2\]\>0\.\\mathbb\{E\}\\\!\\left\[\\mathbb\{V\}\(g\_\{\\text\{STD\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\)\\right\]\-\\mathbb\{E\}\\\!\\left\[\\mathbb\{V\}\(g\_\{\\text\{SLCA\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\)\\right\]=\\mathbb\{E\}\\left\[w\_\{i\}^\{2\}\\\|\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\\\|^\{2\}\\cdot\\sigma\_\{\\xi\}^\{2\}\\right\]\>0\.\(11\)

###### Proof\.

We analyze the estimators for a single trajectoryiiafter fixing the local normalization scale of the update:

gSTD\\displaystyle g\_\{\\text\{STD\}\}=wi​\(Rtool\+R¯sum\+ξ\)​Utool,\\displaystyle=w\_\{i\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\}\+\\xi\)\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\},\(12\)gSLCA\\displaystyle g\_\{\\text\{SLCA\}\}=wi​Rtool​Utool\.\\displaystyle=w\_\{i\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\.\(13\)This is the local score\-function form of the GRPO update after suppressing group mean and standard\-deviation terms\. Exact GRPO uses group\-normalized rewards, soμg\\mu\_\{g\}andσg\\sigma\_\{g\}are functions of all samples in the group and create cross\-sample dependencies\. This theorem does not give an exact closed form for the fully normalized GRPO covariance; it isolates the summary\-noise channel that unified broadcasting necessarily exposes to tool\-token gradients and SLCA removes\. Conditioned onℱtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}, the termswi,Rtool,R¯sum,Utoolw\_\{i\},\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\},\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\},\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}are deterministic\. The only random variable isξ\\xi\. For vector\-valued gradients,𝕍⁡\[⋅\]\\mathbb\{V\}\[\\cdot\]denotes the trace covariance, equivalently the expected squaredL2L\_\{2\}deviation from the conditional mean\. We apply the Law of Total Variance:𝕍⁡\[X\]=𝔼⁡\[𝕍⁡\[X\|ℱ\]\]\+𝕍⁡\[𝔼⁡\[X\|ℱ\]\]\\mathbb\{V\}\[X\]=\\mathbb\{E\}\[\\mathbb\{V\}\[X\|\\mathcal\{F\}\]\]\+\\mathbb\{V\}\[\\mathbb\{E\}\[X\|\\mathcal\{F\}\]\]\.

1\. Conditional variance of the standard estimator:

- •Conditional Variance:𝕍⁡\[gSTD∣ℱtool\]=𝕍⁡\[wi​ξ​Utool∣ℱtool\]=wi2​‖Utool‖2​σξ2\\mathbb\{V\}\[g\_\{\\text\{STD\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\]=\\mathbb\{V\}\[w\_\{i\}\\xi\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\]=w\_\{i\}^\{2\}\\\|\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\\\|^\{2\}\\sigma\_\{\\xi\}^\{2\}\.
- •Conditional Expectation:𝔼⁡\[gSTD∣ℱtool\]=wi​\(Rtool\+R¯sum\)​Utool\\mathbb\{E\}\[g\_\{\\text\{STD\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\]=w\_\{i\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\}\)\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\.

Thus, by the law of total variance,

𝕍⁡\[gSTD\]=𝔼⁡\[wi2​‖Utool‖2​σξ2\]\+𝕍⁡\[wi​\(Rtool\+R¯sum\)​Utool\]\.\\mathbb\{V\}\[g\_\{\\text\{STD\}\}\]=\\mathbb\{E\}\[w\_\{i\}^\{2\}\\\|\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\\\|^\{2\}\\sigma\_\{\\xi\}^\{2\}\]\+\\mathbb\{V\}\[w\_\{i\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\}\)\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\]\.\(14\)
2\. Conditional variance of the SLCA estimator:

- •SincegSLCAg\_\{\\text\{SLCA\}\}isℱtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\-measurable \(independent ofξ\\xi\):

𝕍⁡\[gSLCA∣ℱtool\]=0\.\\mathbb\{V\}\[g\_\{\\text\{SLCA\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\]=0\.\(15\)
3\. Comparison:Taking expectations overℱtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}gives the stated identity:

𝔼⁡\[𝕍⁡\(gSTD∣ℱtool\)\]−𝔼⁡\[𝕍⁡\(gSLCA∣ℱtool\)\]=𝔼⁡\[wi2​‖Utool‖2​σξ2\]\.\\mathbb\{E\}\\\!\\left\[\\mathbb\{V\}\(g\_\{\\text\{STD\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\)\\right\]\-\\mathbb\{E\}\\\!\\left\[\\mathbb\{V\}\(g\_\{\\text\{SLCA\}\}\\mid\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{F\}\_\{\\mathrm\{tool\}\}\}\)\\right\]=\\mathbb\{E\}\[w\_\{i\}^\{2\}\\\|\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\\\|^\{2\}\\sigma\_\{\\xi\}^\{2\}\]\.\(16\)Since generation noiseσξ2\>0\\sigma\_\{\\xi\}^\{2\}\>0and gradient norm‖Utool‖2\>0\\\|\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\\\|^\{2\}\>0almost everywhere, the removed nuisance\-variance term is strictly positive\. This theorem intentionally makes a conditional variance claim: the systematic term𝕍⁡\[wi​\(Rtool\+R¯sum\)​Utool\]\\mathbb\{V\}\[w\_\{i\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\}\)\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\]may differ from𝕍⁡\[wi​Rtool​Utool\]\\mathbb\{V\}\[w\_\{i\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}U\_\{\\mathrm\{tool\}\}\}\]depending on task structure and covariance, which is why we frame SLCA as a bias–variance trade\-off rather than an unconditional variance dominance statement\. ∎

#### Optimization implication

For SGD on non\-convexLL\-smooth functions, standard convergence bounds scale with the gradient\-variance term \(e\.g\.,O⁡\(σ2/ϵ4\)O\(\\sigma^\{2\}/\\epsilon^\{4\}\)to reach anϵ\\epsilon\-stationary point\)\. Theorem[A\.3](https://arxiv.org/html/2609.29050#A1.Thmtheorem3)shows that SLCA removes a positive conditional variance component from tool\-token gradients; when this nuisance component dominates systematic covariance changes, the resulting smoother gradients should translate into faster and more stable optimization\.

#### Part II: Resolution of Advantage Confusion \(Directional Fidelity\)

We define a Conflict Regime where the tool segment receives a below\-group\-mean normalized advantage \(A^tool<0\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}^\{\\mathrm\{tool\}\}\}<0\) but the summary segment receives an above\-group\-mean normalized advantage \(A^sum\>0\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}\>0\)\.

###### Theorem A\.4\(Directional Fidelity: restated from Corollary[A\.4](https://arxiv.org/html/2609.29050#A1.Thmtheorem4)\)\.

Let𝐝∗=−∇log⁡π​\(ytool\)\\mathbf\{d\}^\{\*\}=\-\\nabla\\log\\pi\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\)be the ideal descent direction to penalize tool failure\. Let normalized scores beA^tool=−α\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}^\{\\mathrm\{tool\}\}\}=\-\\alpha\(α\>0\\alpha\>0, penalty\) andA^sum=β\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}=\\beta\(β\>0\\beta\>0, reward\)\. Letσtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\sigma\_\{\\mathrm\{tool\}\}\}andσsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\sigma\_\{\\mathrm\{sum\}\}\}denote the group standard deviations of the raw segment rewards\. The expectations below are taken over the stated rollout randomness, with the trajectory partition, group standard deviations, andα,β\\alpha,\\betaheld fixed\.

- •SLCA Update:𝔼\[gSLCA\]∝−α∇logπ\\mathbb\{E\}\[g\_\{\\text\{SLCA\}\}\]\\propto\-\\alpha\\nabla\\log\\pi\. Inner product with𝐝∗\\mathbf\{d\}^\{\*\}:α​‖∇‖2\>0\\alpha\\\|\\nabla\\\|^\{2\}\>0\.\(Correct Direction\)
- •Standard GRPO Update:The trajectory\-level raw deviation is proportional to−α​σtool\+β​σsum\-\\alpha\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\sigma\_\{\\mathrm\{tool\}\}\}\+\\beta\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\sigma\_\{\\mathrm\{sum\}\}\}, so𝔼\[gSTD\]∝\(βσsum−ασtool\)∇logπ\\mathbb\{E\}\[g\_\{\\text\{STD\}\}\]\\propto\(\\beta\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\sigma\_\{\\mathrm\{sum\}\}\}\-\\alpha\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\sigma\_\{\\mathrm\{tool\}\}\}\)\\nabla\\log\\pi\. Inner product with𝐝∗\\mathbf\{d\}^\{\*\}is\(α​σtool−β​σsum\)​‖∇‖2\(\\alpha\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\sigma\_\{\\mathrm\{tool\}\}\}\-\\beta\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\sigma\_\{\\mathrm\{sum\}\}\}\)\\\|\\nabla\\\|^\{2\}\.

Ifβ​σsum\>α​σtool\\beta\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\sigma\_\{\\mathrm\{sum\}\}\}\>\\alpha\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\sigma\_\{\\mathrm\{tool\}\}\}\(the raw summary deviation dominates the raw tool penalty\), Standard GRPO updates in the wrong direction \(<0<0\), reinforcing the failure\. The simpler conditionβ\>α\\beta\>\\alphais recovered when the two raw reward components have equal group standard deviations\. SLCA maintains fidelity with respect to the tool\-execution signal\.

A concrete instantiation of the conflict regime in Theorem[A\.4](https://arxiv.org/html/2609.29050#A1.Thmtheorem4)is visualized in Appendix[D\.4](https://arxiv.org/html/2609.29050#A4.SS4): under an imperfect tool trajectory \(Rtool=0\.65\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\{=\}0\.65, below the group mean\) paired with a perfect summary \(Rsum=1\.0\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\{=\}1\.0, above the group mean\), standard GRPO reinforces the inefficient policy, whereas SLCA routes the tool\-level penalty intoA^tool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}^\{\\mathrm\{tool\}\}\}alone and recovers the correct single parallel invocation\.

#### Bias–Variance Trade\-off of theA^sum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}Estimator

Within a rollout group, trajectories may diverge after the tool segment and produce distinct post\-tool states\. SLCA still normalizesRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}across the group rather than within subgroups that share an identical post\-tool state, soA^sum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}is not a state\-conditional value estimate and can conflate summary quality with post\-tool state variation\.

This cross\-state normalization is not unique to SLCA: standard GRPO normalizesRtotalR\_\{\\text\{total\}\}across the same group, and methods such as MT\-GRPO\([Wei et al\., 2025a](https://arxiv.org/html/2609.29050#bib.bib36)\)and GRPO\-Verif\([Wang et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib34)\)also do not compute exact state\-conditional advantages without a learned value function or tree\-based rollouts\. SLCA*narrows*the scope of this bias:

- •Under standard GRPO, both tool and summary tokens receiveAtotalA\_\{\\text\{total\}\}, the z\-score of\(Rtool\+Rsum\)\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}\+\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\), which includes cross\-state variation inRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}\.
- •Under SLCA, tool tokens receiveA^tool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}^\{\\mathrm\{tool\}\}\}, the z\-score ofRtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R^\{\\mathrm\{tool\}\}\}alone, computed across rollouts from the same initial state \(shared promptxx\), without cross\-state summary bias\. Only summary tokens receive the biasedA^sum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}^\{\\mathrm\{sum\}\}\}\.

We frame this as a bias–variance trade\-off\. The unified estimator avoids grouping bias but exposes tool\-token gradients to conditional nuisance variance from the summary\-reward channel \(Theorem[A\.3](https://arxiv.org/html/2609.29050#A1.Thmtheorem3)\)\. SLCA accepts summary\-side grouping bias but confines it to summary tokens and removes that direct variance channel\. The support\-control experiment in Table[21](https://arxiv.org/html/2609.29050#A4.T21)provides a cleaner decomposition: closing the summary\-to\-toolS→TS\\\!\\to\\\!Tsupport path has a 5\.51 pp main effect onτ2\\tau^\{2\}\-Bench, whereas the matched w/o SLCA comparison changes normalization and token support together\.

Explicit subgroup partitioning, which would normalizeRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}only within trajectories sharing the same tool outcome, reduces the effective group size toG/CG/Cper subgroup, whereCCis the number of distinct tool outcomes\. For multi\-tool\-call settings withKKcalls, the effective size decays asG/CKG/C^\{K\}, making explicit partitioning impractical without increasingGG\. This is a design trade\-off; soft\-grouping or hierarchical alternatives remain open\.

### A\.3 SLCA Practical Details

This section details the numerical stability mechanisms and edge\-case handling strategies implemented inSLCA\-GRPO\.

#### Presence\-Based Filtering \(Numerical Stability\)

Tool usage behavior is naturally sparse and dynamic; some rollouts may lack a tool segment \(i\.e\.,\|𝒯i,tool\|=0\|\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\}\|=0\) due to omission or early termination\. Naively normalizing rewards within a group where only one sample contains a tool segment leads to a variance collapse \(denominator becomes 0 or unstable\), or yields an unnormalized advantage equal to the raw score\. To resolve this, we strictly normalize each segment type using only the subset of samples where that segment is present\.

LetLis≜\|𝒯i,s\|L\_\{i\}^\{s\}\\triangleq\|\\mathcal\{T\}\_\{i,s\}\|denote the length of segments∈\{tool,sum\}s\\in\\\{\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathrm\{tool\}\},\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{sum\}\}\\\}in rolloutii\. We define the valid indices set for groupggas:

ℐgs≜\{j∣g⁡\(j\)=g​and​Ljs\>0\}\.\\mathcal\{I\}\_\{g\}^\{s\}\\triangleq\\\{j\\mid g\(j\)=g\\text\{ and \}L\_\{j\}^\{s\}\>0\\\}\.\(17\)We compute the segment\-wise advantageA^is\\hat\{A\}\_\{i\}^\{s\}using a conditional normalization scheme\. With a small stability constantϵnorm\\epsilon\_\{\\text\{norm\}\}, the formulation is:

A^is=\{0if​Lis=0​or​\|ℐg⁡\(i\)s\|<2,Ris−μg⁡\(i\)sσg⁡\(i\)s\+ϵnormotherwise,\\hat\{A\}\_\{i\}^\{s\}=\\begin\{cases\}0&\\text\{if \}L\_\{i\}^\{s\}=0\\text\{ or \}\|\\mathcal\{I\}\_\{g\(i\)\}^\{s\}\|<2,\\\\\[8\.0pt\] \\dfrac\{R\_\{i\}^\{s\}\-\\mu\_\{g\(i\)\}^\{s\}\}\{\\sigma\_\{g\(i\)\}^\{s\}\+\\epsilon\_\{\\text\{norm\}\}\}&\\text\{otherwise\},\\end\{cases\}\(18\)where the meanμgs\\mu\_\{g\}^\{s\}and standard deviationσgs\\sigma\_\{g\}^\{s\}are computed exclusively over the valid setℐgs\\mathcal\{I\}\_\{g\}^\{s\}:

μgs=1\|ℐgs\|​∑j∈ℐgsRjs,σgs=Std⁡\(\{Rjs\}j∈ℐgs\)\.\\mu\_\{g\}^\{s\}=\\frac\{1\}\{\|\\mathcal\{I\}\_\{g\}^\{s\}\|\}\\sum\_\{j\\in\\mathcal\{I\}\_\{g\}^\{s\}\}R\_\{j\}^\{s\},\\quad\\sigma\_\{g\}^\{s\}=\\operatorname\{Std\}\\left\(\\\{R\_\{j\}^\{s\}\\\}\_\{j\\in\\mathcal\{I\}\_\{g\}^\{s\}\}\\right\)\.\(19\)Hereϵnorm\\epsilon\_\{\\text\{norm\}\}denotes a numerical floor for stable division, not a tuned reward or optimization hyperparameter\. When the effective group size\|ℐgs\|<2\|\\mathcal\{I\}\_\{g\}^\{s\}\|<2, we explicitly zero out the advantage \(A^is=0\\hat\{A\}\_\{i\}^\{s\}=0\) to disable updates, rather than falling back to standard statistics \(e\.g\.,μ=0,σ=1\\mu=0,\\sigma=1\), which would amplify raw scores as advantages\.

#### Post\-Normalization Weighting

The segment weightsλtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\lambda\_\{\\mathrm\{tool\}\}\}andλsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\lambda\_\{\\mathrm\{sum\}\}\}must be applied*after*the normalization step\. Applying linear scaling*before*z\-score normalization cancels the scale in the idealized case without a numerical floor:

Norm⁡\(λ⋅X\)=λ​X−Mean⁡\(λ​X\)Std⁡\(λ​X\)=λ⁡\(X−μ\)λ​σ=Norm⁡\(X\)\.\\operatorname\{Norm\}\(\\lambda\\cdot X\)=\\frac\{\\lambda X\-\\operatorname\{Mean\}\(\\lambda X\)\}\{\\operatorname\{Std\}\(\\lambda X\)\}=\\frac\{\\lambda\(X\-\\mu\)\}\{\\lambda\\sigma\}=\\operatorname\{Norm\}\(X\)\.\(20\)With the fixedϵnorm\\epsilon\_\{\\text\{norm\}\}used in our implementation, this identity holds up to the numerical floor\. We construct the final routed advantageA^i,tSLCA\\hat\{A\}^\{\\text\{SLCA\}\}\_\{i,t\}by multiplying the normalized scalarA^is\\hat\{A\}\_\{i\}^\{s\}byλs\\lambda\_\{s\}during the token\-level routing phase \(as shown in Eq\.[3](https://arxiv.org/html/2609.29050#S3.E3)\)\.

#### Anti\-Hacking Omission Penalty Guard

We strictly route the omission penaltyRpen<0R\_\{\\text\{pen\}\}<0to the Summary Segment:

Ritool=0,Risum=Rpen\.\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{i\}^\{\\mathrm\{tool\}\}\}=0,\\qquad\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{i\}^\{\\mathrm\{sum\}\}\}=R\_\{\\text\{pen\}\}\.\(21\)In our implementation,Rpen=−0\.5R\_\{\\text\{pen\}\}=\-0\.5\. When the ground truth requires tool use but the rollout contains no<tool\_call\>opening tag, this value directly*overrides*the weighted trajectory score rather than being subtracted as an additive penalty; the normalized success score is then clipped to zero\. By this design, the negative signal propagates only through the summary generation tokens\. This effectively penalizes the decision to "answer immediately without calling," correctly aligning the gradient direction to suppress premature summarization\. In a representative SLCA trace, no\-call episodes start at roughly 10–15% and approach zero by update 50\. In the uniform sensitivity setting, the parallelism weight isSpar=0\.20S\_\{\\mathrm\{par\}\}=0\.20; its BFCL andτ2\\tau^\{2\}\-Bench results remain above matched GRPO \(App\.[D\.7](https://arxiv.org/html/2609.29050#A4.SS7)\)\.

### A\.4 HierR: Process Reward Definitions and Edge Cases

HierR provides dense, structure\-preserving shaping for the tool segment and a terminal preference score for the summary\. We define a process score as a weighted sum of five components:

Sprocess=0\.10​Sfmt\+0\.25​Sname\+0\.15​Skey\+0\.20​Sval\+0\.30​Spar\.S\_\{\\text\{process\}\}=0\.10\\,S\_\{\\text\{fmt\}\}\+0\.25\\,S\_\{\\text\{name\}\}\+0\.15\\,S\_\{\\text\{key\}\}\+0\.20\\,S\_\{\\text\{val\}\}\+0\.30\\,S\_\{\\text\{par\}\}\.\(22\)Let𝒴\\mathcal\{Y\}be the predicted tool\-call multiset and𝒢\\mathcal\{G\}be the ground\-truth multiset\.

#### \(1\) Format adherence \(SfmtS\_\{\\text\{fmt\}\}\)

We measure syntactic validity and schema\-parseability:

Sfmt=Nvalidmax⁡\(Nopen,Nclose\)∈\[0,1\],S\_\{\\text\{fmt\}\}=\\frac\{N\_\{\\text\{valid\}\}\}\{\\max\(N\_\{\\text\{open\}\},N\_\{\\text\{close\}\}\)\}\\in\[0,1\],\(23\)whereNvalidN\_\{\\text\{valid\}\}is the number of call blocks that pass JSON parsing and schema validation, andNopen,NcloseN\_\{\\text\{open\}\},N\_\{\\text\{close\}\}count<tool\_call\>and</tool\_call\>tags\.

#### \(2\) Tool\-name match \(SnameS\_\{\\text\{name\}\}\)

We compute an F1\-style score over tool\-name multisets𝒩Y,𝒩G\\mathcal\{N\}\_\{Y\},\\mathcal\{N\}\_\{G\}:

Sname=2​\|𝒩Y∩𝒩G\|\|𝒩Y\|\+\|𝒩G\|∈\[0,1\],S\_\{\\text\{name\}\}=\\frac\{2\|\\mathcal\{N\}\_\{Y\}\\cap\\mathcal\{N\}\_\{G\}\|\}\{\|\\mathcal\{N\}\_\{Y\}\|\+\|\\mathcal\{N\}\_\{G\}\|\}\\in\[0,1\],\(24\)where∩\\capdenotes multiset intersection\.

#### \(3\) Argument\-key match \(SkeyS\_\{\\text\{key\}\}\) and value match \(SvalS\_\{\\text\{val\}\}\)

For each gold callg∈𝒢g\\in\\mathcal\{G\}, match it to the best predicted cally∗∈𝒴y^\{\*\}\\in\\mathcal\{Y\}with the same tool name\. Let𝒦y,𝒦g\\mathcal\{K\}\_\{y\},\\mathcal\{K\}\_\{g\}be argument\-key sets\. We define key match as Jaccard:

Skey​\(y,g\)=\|𝒦y∩𝒦g\|\|𝒦y∪𝒦g\|\.S\_\{\\text\{key\}\}\(y,g\)=\\frac\{\|\\mathcal\{K\}\_\{y\}\\cap\\mathcal\{K\}\_\{g\}\|\}\{\|\\mathcal\{K\}\_\{y\}\\cup\\mathcal\{K\}\_\{g\}\|\}\.\(25\)Value match averages a loose indicator over matched keys:

Sval​\(y,g\)=1\|𝒦g\|​∑k∈𝒦y∩𝒦g𝕀loose​\(y⁡\[k\],g⁡\[k\]\),S\_\{\\text\{val\}\}\(y,g\)=\\frac\{1\}\{\|\\mathcal\{K\}\_\{g\}\|\}\\sum\_\{k\\in\\mathcal\{K\}\_\{y\}\\cap\\mathcal\{K\}\_\{g\}\}\\mathbb\{I\}\_\{\\text\{loose\}\}\\big\(y\[k\],g\[k\]\\big\),\(26\)where𝕀loose\\mathbb\{I\}\_\{\\text\{loose\}\}allows simple type casting and string normalization\. We aggregate over allggby averaging the best\-match scores\. Extra predicted calls are not directly averaged intoSkeyS\_\{\\text\{key\}\}orSvalS\_\{\\text\{val\}\}once all gold calls have been matched; they are penalized throughSnameS\_\{\\text\{name\}\},SparS\_\{\\text\{par\}\}, and, when invalid,SfmtS\_\{\\text\{fmt\}\}\.

#### \(4\) Parallelism constraint \(SparS\_\{\\text\{par\}\}\)

To prevent redundancy or “parallelism collapse” at the initial tool step, we impose a hard cardinality check:

Spar=𝕀⁡\(\|Yinit\|=\|Ginit\|\)∈\{0,1\},S\_\{\\text\{par\}\}=\\mathbb\{I\}\\big\(\|Y\_\{\\text\{init\}\}\|=\|G\_\{\\text\{init\}\}\|\\big\)\\in\\\{0,1\\\},\(27\)whereYinitY\_\{\\text\{init\}\}andGinitG\_\{\\text\{init\}\}denote the predicted and gold call lists in the first call step\.

### A\.5 Summary score and omission edge case

We derive the summary preference scoreSsummary∈\[0,1\]S\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{summary\}\}\}\\in\[0,1\]using an LLM\-as\-a\-judge approach\([Wataoka et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib35);[Li et al\., 2026a](https://arxiv.org/html/2609.29050#bib.bib12)\)\. Specifically, we employ a structured evaluation prompt to assess the utility and faithfulness of the final response, normalizing the raw ratings to the unit interval\. We query the judge for every rollout, including cases in which mandatory tool use is omitted\. For a rollout that triggers the omission guard, the policy reward is overridden byRsum=Rpen=−0\.5\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}=R\_\{\\text\{pen\}\}=\-0\.5; the normalized guard advantage is retained for the policy update\.

#### Judge Model and Deployment

The judge is GPT\-OSS\-120B \(OpenAI, Apache 2\.0\), a 120B\-parameter open\-weight language model served via a vLLM OpenAI\-compatible API endpoint\. It is a frozen reward producer: its parameters are not optimized by policy training, although its scalarRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}enters the policy loss\. The role can be served by a local model or an API endpoint and is separate from the response mocker used by SGLS\. The deployment and decoding parameters are listed in Table[3](https://arxiv.org/html/2609.29050#A1.T3)\. The deterministic setting \(temperature=0\.0=0\.0\) makes the judge output deterministic for a fixed input; rollout and summary sampling can still vary across runs\.

Table 3:Summary Judge: Deployment and Decoding Parameters\.
#### Prompt Structure and Input Variables

The full evaluation prompt is provided in the Prompts section\. Three variables are injected into the template:\{question\_content\}\(the user’s original question\),\{tool\_responses\}\(tool return values only, excluding<tool\_call\>content\), and\{final\_response\}\(the model’s final summary\)\. The judge evaluates along three dimensions \(*Result Interpretation*,*Information Completeness*, and*Expression Quality*\) and produces a categorical rating on a 5\-point scale\.

#### Factual vs\. Stylistic Dimensions

The three evaluation dimensions differ in nature:*Result Interpretation*and*Information Completeness*are factual; the latter is causally dependent on tool execution quality, since incomplete tool results necessarily limit response completeness\.*Expression Quality*is partly stylistic \(clarity, organization\)\. SLCA’s decoupling guarantee \(Proposition[A\.1](https://arxiv.org/html/2609.29050#A1.Thmtheorem1)\) holds regardless of whatRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R^\{\\mathrm\{sum\}\}\}measures: it prevents any component ofSsummaryS\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{summary\}\}\}, whether factual or stylistic, from contaminating tool\-token advantages\. The legitimate causal link \(correct tools→\\tobetter summaries\) is preserved via the systematic componentR¯sum​\(ytool\)\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\bar\{R\}^\{\\mathrm\{sum\}\}\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}y\_\{\\mathrm\{tool\}\}\}\)in Assumption[A\.2](https://arxiv.org/html/2609.29050#A1.Thmtheorem2); the direct summary\-to\-tool pathway is blocked\. BFCL andτ2\\tau^\{2\}\-Bench use judge\-free evaluation metrics, providing checks beyond the judge\-based metric\.

#### Score Normalization

The categorical rating is mapped to\[0,1\]\[0,1\]via linear normalization\(r−1\)/4\(r\-1\)/4, whererris the raw integer score\. Table[4](https://arxiv.org/html/2609.29050#A1.T4)lists the complete mapping\.

Table 4:Summary Judge: Rating\-to\-Score Mapping\.
#### Output Parsing and Failure Handling

The judge output is parsed via the regex pattern:

<response\_quality\>\.\*?<rating\>\(\.\*?\)</rating\>\.\*?</response\_quality\>

Only the five predefined categorical values listed in Table[4](https://arxiv.org/html/2609.29050#A1.T4)are accepted\. Any other output \(e\.g\., numeric scores, free\-text descriptions, or malformed XML\) is treated as a parse failure and assignedJUDGE\_FAILURE\_SCORE=0\.0\\texttt\{JUDGE\\\_FAILURE\\\_SCORE\}=0\.0\. This conservative default prevents noisy or malformed judge outputs from injecting spurious reward signals into training\.

#### Omission Guard

The guard overrides the policy reward when a required tool call is absent, so a summary cannot receive a positive policy reward for answering without tool support\. The judge call is still made for the rollout; its output does not replace the omission reward\. This keeps the reward protocol and group normalization statistics fixed across the compared conditions\.

#### Safeguards Against Judge Artifacts

We employ the following safeguards to mitigate known failure modes of LLM\-as\-a\-judge:

1. 1\.Verbosity/style bias:The structured rubric evaluates three factual dimensions rather than holistic preference, constraining the judge to content\-level assessment\. The judge receives only tool responses and the final summary \(no intermediate reasoning, chain\-of\-thought, or<tool\_call\>content\), limiting stylistic influence\. The discrete 5\-point categorical scale further constrains output granularity\.
2. 2\.Reward hacking:Two safeguards are used: \(a\) the omission guard overrides the policy reward withRpenR\_\{\\text\{pen\}\}and routes its normalized signal to the available Summary tokens when mandatory tool use is absent; \(b\) SLCA’s advantage routing \(Eq\.[3](https://arxiv.org/html/2609.29050#S3.E3)\) isolatesSsummaryS\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{summary\}\}\}from tool\-token gradients\.
3. 3\.Preference leakage:The same judge model, prompt template, and decoding parameters are used identically across all compared methods\. The judge weights are frozen throughout training\. For cross\-domain benchmarks,SsummaryS\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{summary\}\}\}is not used: BFCL evaluates tool accuracy directly, andτ2\\tau^\{2\}\-Bench uses the officialPass1\\text\{Pass\}^\{1\}metric\.

### A\.6 SGLS Scope and Evaluation

SGLS is a schema\-guided simulator: it preserves tool names, required fields, and response structure while allowing concrete observations to differ from those of a live API\. We evaluate this structural alignment empirically through thew/o SGLSablation, the cross\-domain benchmarks, and the alignment examples in App\.[C\.3](https://arxiv.org/html/2609.29050#A3.SS3)\.

#### Scope of the semantic argument

Raw simulated and real observations can have disjoint supports: random hashes, timestamps, and instance IDs can makeDTV​\(Preal,Psim\)=1D\_\{\\mathrm\{TV\}\}\(P\_\{\\mathrm\{real\}\},P\_\{\\mathrm\{sim\}\}\)=1, so a standard simulation lemma on raw observations is vacuous\. The useful alignment is at the schema and semantic\-interaction level: tool names, required fields, call structure, and response roles remain aligned even when concrete values differ\. A Wasserstein\-style metric on a semantic representation would be more meaningful than total variation, but we do not define such a representation or claim a bound here\. SGLS treats the schema as a control plane, and its contribution is assessed through thew/o SGLSablation, the cross\-domain benchmarks, and the alignment examples reported in the rest of this appendix\.

### A\.7 Algorithm Details

Algorithm[1](https://arxiv.org/html/2609.29050#alg1)summarizes the completeSLCA\-GRPOtraining procedure with SGLS rollouts and HierR segment rewards\.

Algorithm 1SLCA\-GRPOtraining with SGLS and HierR1:Input:dataset

𝒟\\mathcal\{D\}, schemas

Σ⁡\(x\)\\Sigma\(x\), group size

GG, PPO clip

ϵclip\\epsilon\_\{\\text\{clip\}\}
2:foreach iterationdo

3:sample

x∼𝒟x\\sim\\mathcal\{D\}
4:for

i=1i=1to

GGdo

5:rollout

yi∼πθold\(⋅∣x\)y\_\{i\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid x\)by interacting with SGLS \(schema validation \+ tool responses\)

6:record mask

mi,tm\_\{i,t\}and segment sets

𝒯i,tool,𝒯i,sum\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\mathcal\{T\}\_\{i,\\mathrm\{tool\}\}\},\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathcal\{T\}\_\{i,\\mathrm\{sum\}\}\}
7:compute HierR segment returns

\(Sitool,Sisum\)\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}S\_\{i\}^\{\\mathrm\{tool\}\}\},\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}S\_\{i\}^\{\\mathrm\{sum\}\}\}\)with omission\-penalty guard if needed

8:endfor

9:compute

A^itool=Norm⁡\(Sitool\)\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\hat\{A\}\_\{i\}^\{\\mathrm\{tool\}\}\}=\\mathrm\{Norm\}\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}S\_\{i\}^\{\\mathrm\{tool\}\}\}\)and

A^isum=Norm⁡\(Sisum\)\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\hat\{A\}\_\{i\}^\{\\mathrm\{sum\}\}\}=\\mathrm\{Norm\}\(\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}S\_\{i\}^\{\\mathrm\{sum\}\}\}\)with presence\-based filtering

10:route advantages to tokens via Eq\.[3](https://arxiv.org/html/2609.29050#S3.E3)to obtain

A^i,tSLCA\\hat\{A\}^\{\\text\{SLCA\}\}\_\{i,t\}
11:update

θ\\thetaby maximizing Eq\.[5](https://arxiv.org/html/2609.29050#S3.E5)

12:endfor

## Appendix BEmpirical Analysis of Optimization Stability

As an optimization diagnostic, we track theL2L\_\{2\}norm of the gradient vector‖∇θ‖2\\\|\\nabla\_\{\\theta\}\\\|\_\{2\}throughout training across all model scales\. A lower and smoother gradient norm is one descriptive indicator of update variability\.

#### Visual Analysis\.

Figure[5](https://arxiv.org/html/2609.29050#A2.F5)illustrates the training dynamics\.

- •SFT\+GRPO \(w/o SLCA\) \(Orange\):Exhibits frequent high\-magnitude spikes and broader fluctuations\. These patterns are compatible with updates in which summary and tool signals point to different changes\.
- •SLCA\-GRPO\(Blue\):Maintains a generally smoother trajectory in these runs\. Its segment\-specific support removes the direct summary\-to\-tool contribution to tool\-token updates\.

Figure 5:Gradient Norm Dynamics \(‖∇θ‖2\\\|\\nabla\_\{\\theta\}\\\|\_\{2\}\) during Training\. We compare the gradient norms of SFT\+GRPO \(w/o SLCA\) \(Orange\) andSLCA\-GRPO\(Blue\) across Qwen2\.5\-3B, 7B, and Qwen3\-8B\. These are representative single\-run traces and provide a descriptive view of update variability under the two estimators\.
#### Quantitative Gradient Stability

We quantify this diagnostic by computing the standard deviation \(Std\) of the gradient norm over the training steps\. As reported in Table[5](https://arxiv.org/html/2609.29050#A2.T5),SLCA\-GRPOhas a lower observed standard deviation across all model scales in these runs\.

OnQwen2\.5\-3B\-Instruct, gradient\-norm volatility is reduced by25\.0%\. OnQwen3\-8B\-Base, which has the highest baseline Std in this table, the reduction is 2\.5%\.

#### Cross\-Scale Analysis: Gradient Stability vs\. Task Performance

The relative reduction decreases across the three backbones \(25\.0%→\\to13\.1%→\\to2\.5%\)\. Gradient\-norm standard deviation captures one aggregate variability measure, so these values do not identify the contribution of each noise source\.

The matchedτ2\\tau^\{2\}\-Bench gaps are \+1\.09 pp \(3B\), \+9\.15 pp \(7B\), and \+10\.03 pp \(8B\), all based on three\-run means\. These aggregate results do not establish a scale trend for gradient stability\. On the simpler Toucan single\-turn evaluation, the three\-run Success gaps range from \+2\.05 to \+2\.53 pp\. Separately, we measured the gradient cosine similaritycos⁡\(∇tool,∇sum\)\\cos\(\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\nabla\_\{\\mathrm\{tool\}\}\},\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\nabla\_\{\\mathrm\{sum\}\}\}\)on the 3B backbone and found it near zero \(\[−0\.03,0\.08\]\[\-0\.03,0\.08\]\) throughout training for*both*SLCA and standard GRPO\. In this diagnostic, advantage magnitudes, rather than gradient direction, are the more visible source of cross\-segment mismatch\. Because the measured tool/summary gradient cosine is near zero, we refer to SLCA as an “advantage\-level firewall” rather than a “gradient firewall”: the diagnostic concerns the magnitude of the assigned advantages, not a measured directional conflict between the two gradient components\.

Table 5:Quantification of Gradient Stability\.We report the Standard Deviation \(Std\) of the gradient norm\.SLCA\-GRPOhas lower observed volatility in these representative runs\.

## Appendix CDatasets and Training Details

### C\.1 Data Processing Pipeline

As illustrated in Figure[6](https://arxiv.org/html/2609.29050#A3.F6), this pipeline is designed to strictly prevent test\-set leakage and optimize data composition for different learning stages\.

The pipeline consists of four distinct phases:

#### Phase 1: Noise Filtering and Schema Validation

Starting with 119,279 Toucan raw trajectories, we first remove approximately 40,000 samples identified as irrelevant or containing low\-quality formatting\. The remaining 79,279 candidates undergo strict filtering, which removes 1,038 samples containing broken JSON, non\-standard tool tokens, or hallucinations to ensure simulator stability\. This results in a high\-quality valid pool of78,241trajectories\.

#### Phase 2: Evaluation Isolation\.

To ensure rigorous zero\-shot evaluation, we randomly sample a held\-out Toucan\-Test set of4,000samples from the valid pool before any training splits are made\. This set comprises:

- •3,000 Native Single\-turn samples:Testing atomic tool\-use capabilities\.
- •1,000 Decomposed Multi\-turn samples:Extracted from distinct multi\-turn dialogues to test context retention and state tracking\.

The remaining 74,241 samples constitute the Total Training Pool\.

#### Phase 3: Trajectory Decomposition Strategy

For Reinforcement Learning, training on long\-horizon dialogs from scratch is unstable\. We decompose multi\-turn dialogs into atomicContext\-Actionpairs\. We exclude the first tool\-call turn during decomposition and extract samples starting from the second tool invocation \(Turn≥\\geq2\)\. Each decomposed sample provided to the RL agent thus contains historical context from previous user\-agent interactions\. The Toucan trajectories use real MCP teacher traces\. The segment mask is derived deterministically from the structured trajectory trace and the environment boundary; no annotated segment or token labels and no learned segmenter are used\.

![Refer to caption](https://arxiv.org/html/2609.29050v1/data_process.png)Figure 6:Data Processing Pipeline\.The workflow applies strict relevance filtering and format check before isolating the evaluation set\. The remaining training pool is split into SFT and RL partitions; the RL set combines native single\-turn and decomposed multi\-turn trajectories formatted for the SGLS\.
#### Phase 4: SFT vs\. RL Partitioning

We split the training pool \(74,241 samples\) into two partitions:

- •SFT Partition \(42,423 samples\):Representing approximately57%57\\%of the data, this set prioritizes diversity in format \(Human/GPT styles\) and includes the majority of native multi\-turn data to establish a robust instruction\-following prior\.
- •RL Partition \(31,818 samples\):Optimized for exploration, this set is constructed with a specific compositional ratio: RL Composition≈23×Native Single\-turn\+13×Decomposed Multi\-turn\\text\{RL Composition\}\\approx\\frac\{2\}\{3\}\\times\\text\{Native Single\-turn\}\+\\frac\{1\}\{3\}\\times\\text\{Decomposed Multi\-turn\}Specifically, it contains21,212native single\-turn samples and10,606decomposed multi\-turn samples\. This stratified mix balances the learning of simple function execution with complex, history\-dependent reasoning\.

#### Total Supervision Visibility

As noted in the experimental setup, our SFT baseline is trained on the union of both partitions \(Total 74,241 samples\), so the comparison uses matched data exposure\.

### C\.2 Training Implementation Details

#### Implementation Environment and Hyperparameters

We fine\-tune the model on the Toucan supervision trajectories using the standard cross\-entropy objective\. The SFT training is conducted on 8 NVIDIA H20 GPUs withbfloat16precision\. We employ the AdamW optimizer\([Kingma and Ba, 2015](https://arxiv.org/html/2609.29050#bib.bib11);[Loshchilov and Hutter, 2019](https://arxiv.org/html/2609.29050#bib.bib18)\)with a cosine learning rate scheduler\. For the RL stage, training is scaled across 32 NVIDIA H20 GPUs \(4 nodes\)\. The maximum sequence lengths for prompts and generations are extended to 16k and 8k tokens, respectively, to accommodate long\-context reasoning\. Detailed hyperparameters for both stages are listed in Table[6](https://arxiv.org/html/2609.29050#A3.T6)\.

#### Baseline Configurations and Controlled Variables

The matched set comprises SFT\+GRPO, SFT\+SLCA\-GRPO, w/o SLCA, w/o SGLS, and w/o HierR\. Within each backbone, all five use that backbone’s SFT checkpoint, the same data split, SGLS endpoint, mocker configuration, decoding settings, evaluation protocol, rollout group sizeG=16G=16, one RL epoch, and the same run set\. GRPO,SLCA\-GRPO, w/o SLCA, and w/o SGLS use the same HierR definition and subweights; w/o HierR removes HierR, and w/o SGLS removes schema conditioning and deterministic validation:

- •SFT \(Data\-Equivalent Baseline\):By exposing the model to the full dataset in a supervised manner, this baseline serves as a data\-equivalent benchmark to determine whether RL optimization yields gains beyond simply scaling up demonstration data\.
- •SFT\+GRPO \(Standard Baseline\):This baseline uses the same SGLS environment and HierR reward definitions as our method\. Its estimator aggregates the two rewards into an unweighted sum \(Rtotal=1\.0⋅Rtool\+1\.0⋅RsumR\_\{\\text\{total\}\}=1\.0\\cdot\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\+1\.0\\cdot\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\) and broadcasts the resulting advantage to all learnable tokens\.
- •SFT\+SLCA\-GRPO\(Ours\):Following the same data split as SFT\+GRPO, it computes independent advantages for execution and summarization\. The matched comparison uses the same 1:1 weighting \(λtool=λsum=1\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}\\lambda\_\{\\mathrm\{tool\}\}\}=\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\lambda\_\{\\mathrm\{sum\}\}\}=1\) and routes signals via Eq\.[3](https://arxiv.org/html/2609.29050#S3.E3)\.
- •Ablation definitions:w/o SLCA uses the unified estimator, w/o SGLS removes schema conditioning and deterministic validation while retaining the mocker endpoint, and w/o HierR removes the hierarchical reward\.

Table 6:Detailed Hyperparameters and Environment Settings\. We list the configurations for both the supervised fine\-tuning \(SFT\) and the subsequent RL training stages\.Fine\-Tuning HyperparametersRL HyperparametersParameterValueParameterValueHardware Resources8×\\timesH20 GPUsHardware Resources32×\\timesH20 \(4 Nodes\)OptimizerAdamWPrecisionbfloat16Total Batch Size32Total Batch Size128Per\-Device Batch1Mini\-batch Size32Grad Accumulation4Rollout Group Size \(GG\)16Learning Rate5×10−65\\times 10^\{\-6\}Learning Rate1×10−61\\times 10^\{\-6\}Number of Epochs2Number of Epochs1LR SchedulerCosineMax Response Len8192Warmup Ratio0\.1KL Coefficient \(βKL\\beta\_\{\\text\{KL\}\}\)0\.001Max Context Len16384Max Prompt Len16384–PPO Clip \(ϵclip\\epsilon\_\{\\text\{clip\}\}\)0\.2 \(symmetric\)–Dual\-Clip Constant \(cc\)3\.0–KL Estimatorlow\-var KL–Loss Aggregationtoken\-mean–No\-Call Penalty \(RpenR\_\{\\text\{pen\}\}\)−0\.5\-0\.5\(override\)The PPO clip is symmetric:ϵclip=ϵlow=ϵhigh=0\.2\\epsilon\_\{\\text\{clip\}\}=\\epsilon\_\{\\text\{low\}\}=\\epsilon\_\{\\text\{high\}\}=0\.2, without asymmetric DAPO\-style clipping\. The dual\-clip constantc=3\.0c=3\.0applies only to negative\-advantage samples when the likelihood ratio is large\. The no\-call valueRpen=−0\.5R\_\{\\text\{pen\}\}=\-0\.5is used when tool use is required but no tool call is emitted; it overrides the weighted score rather than subtracting from it\.

#### Training\-step convention

All RL curves report policy\-update steps, not PPO inner mini\-batch passes\. The main runs use rollout group sizeG=16G=16and one RL epoch over the post\-filtered rollout stream; this corresponds to approximately 100 logged policy\-update steps in the default 7B setting\. We use this convention consistently across all training\-dynamics figures\.

### C\.3 SGLS Implementation Details

#### Frozen LLM Specification

The frozen LLM used in SGLS isQwen3\-235B\-A22B\-Instruct\(July 2025 checkpoint\), a 235B\-parameter Mixture\-of\-Experts \(MoE\) model with 22B active parameters per forward pass\. This model belongs to a*different architectural family*from the policy backbone \(Qwen2\.5\-7B\-Instruct, dense architecture, September 2024\)\. The cross\-family, cross\-generational design limits implicit leakage and same\-backbone favoritism: \(i\) the representation spaces are structurally distinct \(MoE routing vs\. dense layers\), \(ii\) the models use different tokenizers and pre\-training corpora, and \(iii\) the frozen LLM’s role is limited to template\-constrained mocking of valid tool calls; it never provides planning guidance or reward signals\. The Toucan teacher trajectories used to construct the training data are real MCP trajectories curated for tool coverage and quality\. They provide the tool\-reference source forRtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}and are not an input to the SLCA estimator\.

#### Deployment Configuration

The frozen LLM is served via vLLM 0\.11\.0 with the deployment parameters listed in Table[7](https://arxiv.org/html/2609.29050#A3.T7)\.

Table 7:SGLS Frozen LLM: Deployment and Decoding Parameters\.
#### Two\-Stage Pipeline

SGLS operates as a two\-stage pipeline to minimize unnecessary LLM invocations:

1. 1\.Stage 1: Deterministic Schema Validation \(no LLM\):Checks tool existence against the current instance’s available tool set; if the tool name is not found, returns a hard\-coded error string\. Then verifies required parameters against the schema definition; missing parameters trigger a deterministic error message listing the absent fields\.
2. 2\.Stage 2: Template\-Constrained LLM Mocking \(valid calls only\):For calls that pass Stage 1, the tool definition JSON and the parsed<tool\_call\>payload are injected into a fixed prompt template\. The frozen LLM generates a mock<tool\_response\>output\. Post\-processing extracts the response content via tag matching and handles truncation edge cases \(e\.g\., responses with opening but no closing tags\)\.

This hybrid architecture gives schema violations in early\-training rollouts deterministic feedback, while the LLM is invoked only for semantically plausible mock responses to correctly formed calls\.

#### w/o SGLSfallback environment

Thew/o SGLSablation keeps the same frozen mocker and rollout interface, and removes only schema conditioning and deterministic validation\. Both variants call the same external vLLM/v1/chat/completionsendpoint with the same Qwen3\-235B\-A22B model, temperature \(0\.6\), max\-token budget \(4,096\), and timeout \(540s\)\. Thus the ablation is*prompt\-unconstrained*, not model\-unconstrained: the API\-server role instruction,<tool\_response\>output contract, prohibitions against explanations or<tool\_call\>emission, few\-shot examples, incoming\-request format, and rollout limits are unchanged\. The only prompt\-side removal is theTool DefinitionJSON\-schema block; the corresponding code path also removes per\-instance schema caching and tool\-definition parsing\. As a result, unknown tools, missing required parameters, and type\-invalid arguments no longer trigger deterministic error strings; every call is forwarded to the mocker, which generates a<tool\_response\>shell whose content is determined by the LLM\. Post\-processing only strips/extracts tool\-response tags and does not validate JSON fields\.

![Refer to caption](https://arxiv.org/html/2609.29050v1/sgls_alignment.png)Figure 7:SGLS mock responses align with real\-API outputs across three representative tool domains\.The Semantic Mappingϕ⁡\(o\)\\phi\(o\)extracts schemas, facts, and logic, realizing three levels of alignment \(Mapping Marks\):*Identical Value*for deterministic tools \(*calculator*\),*Identical Schema*for query\-style tools \(*weather*; individual values need not match a specific real\-API snapshot but remain within plausible ranges\), and*Semantic Equivalence*for open\-ended tools \(*search*; factually consistent content differing only in phrasing\)\.
#### Response\-mocker robustness

An alternate response\-mocker check replaces the frozen response mocker with GPT\-OSS\-120B while keeping the policy initialization, data, validators, rewards, rollout budget, and evaluation protocol fixed\. The response mocker generates tool observations; it is separate from the summary judge that producesRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\. In this check the mocker and judge are intentionally from the same GPT\-OSS\-120B family; BFCL andτ2\\tau^\{2\}\-Bench do not use the judge reward, so their transfer measurements are not affected by that shared\-family choice\. The summary\-judge prompt and decoding remain unchanged\. Table[8](https://arxiv.org/html/2609.29050#A3.T8)reports the matched results\.

Table 8:Response\-mocker check on Qwen2\.5\-7B\-Instruct\. GPT\-OSS\-120B replaces the response mocker; entries are mean±\\pmstandard deviation under the matched policy setup\.The SLCA advantage remains positive on all three benchmarks after the response\-mocker replacement\.

### C\.4 Baseline Comparison: RLTR and ToolPO Implementation

To provide context for the comparisons in §[4\.2](https://arxiv.org/html/2609.29050#S4.SS2), we re\-implement RLTR\([Li et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib15)\)and ToolPO\([Li et al\., 2026b](https://arxiv.org/html/2609.29050#bib.bib14)\)within the same verl training framework\. The matched GRPO andSLCA\-GRPOruns share the controls listed in Table[6](https://arxiv.org/html/2609.29050#A3.T6); RLTR and ToolPO retain their method\-specific initialization and reward/protocol structure\. The ToolPO rows are our implementations under the stated outcome\-reward choices, not reproductions of the original ToolPO environment\.

#### SFT Initialization

For each backbone, the matched methods start from that backbone’s SFT checkpoint \(e\.g\.,qwen2\.5\-7B\-sft\); the RLTR planner checkpoint follows its method\-specific construction\. RLTR requires a dedicated*Planner SFT*: we remove summary tokens from the SFT data and append an<answer\>marker to delineate the planner’s output boundary, following the original paper’s design\.

#### RLTR Implementation

We re\-implement the Planner\-Summarizer pipeline:

- •Reward:R=Rcomp\+Rrepeat\+RerrorR=R\_\{\\text\{comp\}\}\+R\_\{\\text\{repeat\}\}\+R\_\{\\text\{error\}\}, whereRcompR\_\{\\text\{comp\}\}is an LLM\-based Completeness Checker \(Qwen3\-30B\-A3B,N=3N\{=\}3samples averaged\),Rrepeat=−0\.1×repeat\_countR\_\{\\text\{repeat\}\}=\-0\.1\\times\\text\{repeat\\\_count\}, andRerror=−0\.2×error\_countR\_\{\\text\{error\}\}=\-0\.2\\times\\text\{error\\\_count\}\. Format violations yieldR=−1\.0R=\-1\.0\.
- •Advantage:Only process \(tool\) tokens receive non\-zero advantages; summary token advantages are set to zero\.
- •Evaluation:A 2\-stage pipeline: the Planner generates the tool trajectory, then a frozen SFT Summarizer produces the final answer\. For BFCL, only the Planner checkpoint is used\.

#### ToolPO Implementation

We report a reference\-rule ToolPO condition and an LLM\-judge condition as a protocol\-sensitivity diagnostic on 7B\. The 3B and 8B ToolPO rows in the scale tables use the LLM\-judge Solved/Unsolved outcome reward; no reference\-rule runs are available for those scales\.

- •Outcome reward protocols:The reference\-rule condition uses the available gold tool calls, while the sensitivity condition uses an LLM judge that classifies each rollout as Solved or Unsolved, denotedRsuccjudgeR\_\{\\text\{succ\}\}^\{\\text\{judge\}\}\. Both are evaluated on the 7B backbone below\.
- •C⁡\(atcall\)C\(a\_\{t\}^\{\\text\{call\}\}\)adaptation:Tool\-call correctness via strict matching against gold tool calls \(exact name match\+\+all gold keys present\+\+lenient value matching\)\.Raction=∑C/\|gold\_flat\|R\_\{\\text\{action\}\}=\\sum C/\|\\text\{gold\\\_flat\}\|, normalized to\[0,1\]\[0,1\]\.
- •Advantage:Atool=Aglobal\+λ⋅Atool\_local\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}A\_\{\\mathrm\{tool\}\}\}=A\_\{\\text\{global\}\}\+\\lambda\\cdot A\_\{\\text\{tool\\\_local\}\}for tool tokens,Asum=Aglobal\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}A\_\{\\mathrm\{sum\}\}\}=A\_\{\\text\{global\}\}for summary tokens, withλ=1\.0\\lambda=1\.0and group\-wise normalization\. We adoptλ=1\.0\\lambda\{=\}1\.0following the original ToolPO specification\([Li et al\., 2026b](https://arxiv.org/html/2609.29050#bib.bib14)\), which gives the local action reward parity with the global outcome reward; we did not further tuneλ\\lambdato avoid introducing a confound absent from the original design\.

#### Reference rule protocol

We implement the reference rule as an explicit gold\-call conjunction using the same matching primitives as the HierR process reward\. Let𝒩Y\\mathcal\{N\}\_\{Y\}and𝒩G\\mathcal\{N\}\_\{G\}be the predicted and gold tool\-name multisets, and lety∗:𝒢→𝒴y^\{\*\}:\\mathcal\{G\}\\to\\mathcal\{Y\}be the one\-to\-one matching used above, pairing each gold call with the best unused predicted call of the same name\. LetYinitY\_\{\\mathrm\{init\}\}andGinitG\_\{\\mathrm\{init\}\}denote first\-round calls:

Rsuccrule=\\displaystyle R\_\{\\mathrm\{succ\}\}^\{\\mathrm\{rule\}\}=\{\}𝕀\[Nvalid=Nopen=Nclose,𝒩Y=𝒩G,\|Yinit\|=\|Ginit\|,\\displaystyle\\mathbb\{I\}\\\!\\big\[N\_\{\\mathrm\{valid\}\}=N\_\{\\mathrm\{open\}\}=N\_\{\\mathrm\{close\}\},\\;\\mathcal\{N\}\_\{Y\}=\\mathcal\{N\}\_\{G\},\\;\|Y\_\{\\mathrm\{init\}\}\|=\|G\_\{\\mathrm\{init\}\}\|,\(28\)∀g∈𝒢:\[𝒦reqg⊆𝒦y∗​\(g\)∧∀k∈𝒦reqg,𝕀loose\(y∗\(g\)\[k\],g\[k\]\)=1\]\]\.\\displaystyle\\forall g\\in\\mathcal\{G\}:\\;\\left\[\\mathcal\{K\}^\{\\mathrm\{req\}\}\_\{g\}\\subseteq\\mathcal\{K\}\_\{y^\{\*\}\(g\)\}\\ \\land\\ \\forall k\\in\\mathcal\{K\}^\{\\mathrm\{req\}\}\_\{g\},\\;\\mathbb\{I\}\_\{\\mathrm\{loose\}\}\\\!\\left\(y^\{\*\}\(g\)\[k\],g\[k\]\\right\)=1\\right\]\\big\]\.HereNvalid=Nopen=NcloseN\_\{\\mathrm\{valid\}\}=N\_\{\\mathrm\{open\}\}=N\_\{\\mathrm\{close\}\}requires every parsed call to be valid with balanced opening and closing tags\. The name multiset equality rejects missing or extra calls and incorrect multiplicities;𝒦greq\\mathcal\{K\}^\{\\mathrm\{req\}\}\_\{g\}is the set of required keys in gold callgg, and the matched\-call key condition requires all of them\. Values use the same case\-insensitive, type\-coercing matcher as HierR\. The initial cardinality check enforces the parallel\-call count\. Cross\-round call order is not checked\. The conjunction reuses the gold\-call checks already used byRactionR\_\{\\text\{action\}\}, so this row is a protocol comparison rather than an independent reward source\.

Table 9:ToolPO outcome reward protocol comparison on Qwen2\.5\-7B\-Instruct\. Entries are mean±\\pmstandard deviation over three matched runs\. The reference rule row uses Eq\.[28](https://arxiv.org/html/2609.29050#A3.E28); Unified GRPO andSLCA\-GRPOare shown as contextual matched baselines under their standard reward protocol\.The two ToolPO rows use the same additive estimator, initialization, data, training budget, and evaluation\. Replacing the LLM\-judge outcome reward with the reference rule removes the observed 7B collapse in this ToolPO comparison; the rule\-based ToolPO remains close to Unified GRPO and below SLCA onτ2\\tau^\{2\}\-Bench\. This comparison isolates sensitivity to the outcome\-reward protocol\.

#### Discussion: ToolPO\-7B Format Behavior

Under the LLM\-judge protocol, ToolPO has low format validity on Qwen2\.5\-7B\-Instruct \(format\_passed=\\,\{=\}\\,38%\), whereas the 3B and 8B runs retain higher format validity\. The matched GRPO andSLCA\-GRPOrows use the same 7B SFT initialization, SGLS environment, and RL data; ToolPO retains its method\-specific initialization, additive estimator, and outcome reward protocol\. The 7B result is also reflected in the three\-run mean \(Toucan Success0\.200±0\.0140\.200\{\\pm\}0\.014\)\. The training trace shows an increase in unmatched opening tags after step 80 \(App\.[D\.3](https://arxiv.org/html/2609.29050#A4.SS3)\)\. This is a diagnostic of the LLM\-judge condition\. The reference\-rule comparison above keeps the ToolPO estimator fixed while changing the outcome reward\.

#### Transparency Note

We document the following necessary adaptations for reproducibility:

1. 1\.The reference\-rule condition uses the available gold tool calls; the LLM\-judge condition is retained as a protocol\-sensitivity diagnostic\.
2. 2\.RLTR’s Planner SFT is trained with summary removal and<answer\>markers, rather than using the general SFT directly\.
3. 3\.RLTR’s penalty coefficients \(λrepeat=0\.1\\lambda\_\{\\text\{repeat\}\}\{=\}0\.1,μerror=0\.2\\mu\_\{\\text\{error\}\}\{=\}0\.2\) are set to reasonable values, as the original paper does not specify them\.
4. 4\.We do not force HierR into ToolPO’s additive update, because doing so would change ToolPO’s original reward protocol rather than evaluate the method\-faithful baseline\.
5. 5\.All methods use vanilla policy loss mode, so the policy\-loss implementation is held fixed\.

### C\.5 Evaluation Protocols

For full reproducibility, we document the complete evaluation protocol for each benchmark, including software versions, model configurations, and scoring logic\. The matched GRPO andSLCA\-GRPOrows share the benchmark protocol; the ToolPO and RLTR choices are listed in App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)\.

#### τ2\\tau^\{2\}\-Bench Protocol

We use the officialτ2\\tau^\{2\}\-Bench framework\([Barres et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib1)\)without modifications to prompts, tools, tasks, or scoring logic\. Table[10](https://arxiv.org/html/2609.29050#A3.T10)summarizes the configuration\.

Table 10:τ2\\tau^\{2\}\-Bench evaluation protocol\.
#### BFCL V3 Protocol

We evaluate on BFCL V3\([Patil et al\., 2025](https://arxiv.org/html/2609.29050#bib.bib20)\)using the function\-calling protocol\. Table[11](https://arxiv.org/html/2609.29050#A3.T11)lists the configuration\.

Table 11:BFCL V3 evaluation protocol\.Chat template alignment\.The vLLM serving endpoint uses a Toucan\-specific chat template that injects a fixed system prompt and skips upstream system messages\. This is achat template alignmentchoice ensuring inference consistency with training, not a modification of the BFCL benchmark protocol\. The skipped upstream system messages contain only generic formatting instructions; skipping them is a compatibility step and does not change instance\-specific content or evaluation semantics\. BFCL tool definitions are injected through the standardtoolsparameter in the function\-calling interface, which the template processes correctly\. Theunderscore\_to\_dot=Trueflag reconciles naming\-convention differences between training\-time schemas and BFCL’s schema format\. For the main BFCL V3 comparison, we exclude the relevance\-detection subset and restrict evaluation to single\-turn samples\. These choices are held fixed for the matched GRPO andSLCA\-GRPOrows; the Multi\-Turn extension is reported separately below\.

#### Toucan In\-Domain Protocol

The Toucan evaluation uses a custom asynchronous evaluation system \(independent of evalscope\)\. Table[12](https://arxiv.org/html/2609.29050#A3.T12)lists the agent and environment configuration\.

Table 12:Toucan in\-domain evaluation protocol\.The primary Success metric in Table[1](https://arxiv.org/html/2609.29050#S4.T1)is𝕀\[Sprocess≥0\.9\]\\mathbb\{I\}\[S\_\{\\text\{process\}\}\\geq 0\.9\]; it is distinct from the summary\-judge threshold above\.

Process Score sub\-item weights\.TheSprocessS\_\{\\text\{process\}\}metric \(Eq\.[22](https://arxiv.org/html/2609.29050#A1.E22)\) decomposes into five sub\-items: format \(w=0\.10w\{=\}0\.10\), name \(w=0\.25w\{=\}0\.25\), key \(w=0\.15w\{=\}0\.15\), value \(w=0\.20w\{=\}0\.20\), and parallel \(w=0\.30w\{=\}0\.30\)\. Name matching uses multiset F1 over predicted and gold tool\-name occurrences; key matching uses gold\-call greedy Jaccard similarity; value matching uses lenient comparison \(case\-insensitive, automatic numeric type conversion\)\.

### C\.6 Details on Tool Space Scalability

We evaluate tool\-space scalability via a controlled candidate\-size sweep \(K\-sweep\) under two distractor sampling strategies \(Hard vs\. Random\)\. This appendix documents the dataset statistics, the K\-sweep candidate\-set construction procedure, the hard\-negative mining mechanism, and the context\-length profiling\.

#### Dataset and Global Tool Pool

All K\-sweep datasets are derived from the Toucan evaluation set containing 4,000 instances\. The global tool pool comprises 1,568 tools in total, exhibiting a long\-tailed distribution of schema lengths\. In the original data, the allowed\-tool list contains 5\.44 tools on average, and the gold\-tool set size is 1\.54 on average\. These statistics motivate a controlled augmentation protocol to probe robustness under larger candidate sets\.

#### Controlled Candidate\-Set Construction \(K\-sweep\)

For each instanceii, let𝒜i\\mathcal\{A\}\_\{i\}denote the original allowed tool set and𝒢i⊆𝒜i\\mathcal\{G\}\_\{i\}\\subseteq\\mathcal\{A\}\_\{i\}denote the gold tool set\. Given a target candidate sizeKK, we construct a controlled candidate set𝒜~i\\tilde\{\\mathcal\{A\}\}\_\{i\}as follows:

- •Gold preservation:Always retain all gold tools𝒢i\\mathcal\{G\}\_\{i\}in𝒜~i\\tilde\{\\mathcal\{A\}\}\_\{i\}\.
- •Original baseline \(K=0K=0\):IfK=0K=0, keep the instance unchanged, i\.e\.,𝒜~i=𝒜i\\tilde\{\\mathcal\{A\}\}\_\{i\}=\\mathcal\{A\}\_\{i\}\.
- •Expansion \(\|𝒜i\|<K\|\\mathcal\{A\}\_\{i\}\|<K\):AddK−\|𝒜i\|K\-\|\\mathcal\{A\}\_\{i\}\|distractor tools sampled from the global tool pool, excluding tools already present in𝒜i\\mathcal\{A\}\_\{i\}\.
- •Contraction \(\|𝒜i\|\>K\|\\mathcal\{A\}\_\{i\}\|\>K\):Keep all gold tools and uniformly subsampleK−\|𝒢i\|K\-\|\\mathcal\{G\}\_\{i\}\|tools from𝒜i∖𝒢i\\mathcal\{A\}\_\{i\}\\setminus\\mathcal\{G\}\_\{i\}\.

This construction enforces a precise candidate\-set size while guaranteeing that the target tool\(s\) remain available, thereby isolating the effect of tool\-space size on tool selection\.

#### Distractor Sampling Strategies\.

We implement two strategies for selecting distractors in the expansion step:

- •Random Sampling \(Control\):Distractors are drawn uniformly at random \(without replacement\) from the global tool pool after excluding𝒜i\\mathcal\{A\}\_\{i\}\.
- •Hard\-Negative Sampling \(Semantic Distractors\):We mine semantically similar distractors based on tool\-description similarity\. Specifically, we encode each tool’s description text \(tool name concatenated with its functionality docstring\) using the sentence\-transformersentence\-transformers/all\-MiniLM\-L6\-v2, and precompute a cosine similarity matrix over the global pool\. For each instance, we retrieve the most similar non\-gold tools to the gold tool\(s\) \(excluding tools already in𝒜i\\mathcal\{A\}\_\{i\}\) and take the top\-scoring tools as distractors until the required budget is met\. This procedure produces functionally related distractors, maximizing the probability of semantic interference\.

#### Context Length Profiling and Skip Policy

All token statistics below are computed using the Qwen2\.5\-7B\-Instruct tokenizer with a maximum context length of 32,768 tokens\. During K\-sweep dataset generation, if an instance exceeds the context limit after tool\-schema injection, it is automatically skipped\. Tables[13](https://arxiv.org/html/2609.29050#A3.T13)and[14](https://arxiv.org/html/2609.29050#A3.T14)report empirical system\-prompt length statistics and skip rates for Random and Hard strategies, respectively\.

Table 13:System prompt length statistics vs\. candidate set size \(KK\) forRandomdistractors \(Qwen2\.5 tokenizer\)\.Table 14:System prompt length statistics vs\. candidate set size \(KK\) forHarddistractors \(Qwen2\.5 tokenizer\)\.Hard sampling tends to yield shorter prompts and lower skip rates at fixedKKthan Random sampling, since semantically similar tools are more likely to match the length profile of tools already present in the original candidate set\. We report main\-paper scalability results up toK=100K=100and treatK=150K=150as a stress\-test regime with high context\-limit attrition\.

## Appendix DAdditional Experimental Results

### D\.1 Toucan\-Test: full HierR decomposition

Table[15](https://arxiv.org/html/2609.29050#A4.T15)expands Table[1](https://arxiv.org/html/2609.29050#S4.T1)by reporting two additional components: \(i\)*Parallel*, which captures the parallelism/cardinality constraints in HierR, and \(ii\)*Summary*, an LLM\-judge score over the user\-facing segmentysum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}y\_\{\\mathrm\{sum\}\}\}\. We include RLTR and ToolPO as prior credit\-assignment baselines\. Reporting these terms separately is useful here: RLTR does not optimize the summary segment inside RL, while ToolPO can attain a non\-trivial summary score even when tool execution degrades\.

Table 15:Comprehensive Results on Toucan\-Test across Scales\. This table expands upon the main results by including all sub\-metrics\. In addition to Name F1, ArgMatch, Process, and Success \(defined in Table[1](https://arxiv.org/html/2609.29050#S4.T1)\), we report: Parallel: The parallelism/cardinality constraint score defined in Eq\.[27](https://arxiv.org/html/2609.29050#A1.E27); Summary: The user\-facing response quality score evaluated by the LLM\-judge\.SLCA\-GRPOhas the highest mean Success and Process values across scales\. RLTR and ToolPO are included for context; their method\-specific protocols are described in App\.[C\.4](https://arxiv.org/html/2609.29050#A3.SS4)\. RLTR’s Summary column is marked “–” as its frozen summarizer is not optimized during RL\. Trained rows report mean±\\pmstd over three runs; Original rows are point evaluations\.#### Process and summary scores\.

Across scales,SLCA\-GRPOhas higher mean*Process*and strict*Success*scores than matched GRPO, and higher mean*Parallel*scores\. These metrics capture structural constraints beyond tool\-name matching\. On Qwen2\.5\-3B and Qwen3\-8B\-Base, standard GRPO attains a higher*Summary*score but a lower*Success*rate thanSLCA\-GRPO\. This pattern is compatible with Cross\-Segment Credit Misattribution: a plausible summary can coexist with erroneous tool behavior when a mixed advantage is broadcast to all tokens\. Under the LLM\-judge diagnostic, ToolPO shows the same separation on Qwen2\.5\-7B: it reaches*Summary*0\.6054 while its*Process*score is 0\.3408\. On Qwen2\.5\-7B,SLCA\-GRPOhas higher reported means for both execution and*Summary*; the two scores move in the same direction in this comparison\.

### D\.2 Summary Advantage Ablation

We isolate the learned summary\-reward branch while keeping the omission safeguard unchanged\. Let𝒪\\mathcal\{O\}denote rollouts that trigger the omission guard\. After computing the ordinary group\-normalized summary advantages, we setA^isum=0\\hat\{A\}\_\{i\}^\{\\mathrm\{sum\}\}=0fori​o​t​i​n​𝒪iotin\\mathcal\{O\}and retain the computed guard advantage fori∈𝒪i\\in\\mathcal\{O\}\. The summary tokens remain in the loss mask and under the per\-token KL constraint\. We query the judge for every rollout, including omission cases; its parameters remain frozen\. For a guard rollout, the policy reward is overridden byRsum=Rpen=−0\.5R\_\{\\mathrm\{sum\}\}=R\_\{\\mathrm\{pen\}\}=\-0\.5\.

Table 16:Summary advantage ablation on Qwen2\.5\-7B\-Instruct\. Process and Summary are unit\-interval scores; benchmark columns are percentages\. The no\-call column is the end\-of\-training rate\. Entries in the score columns are mean±\\pmstandard deviation over three matched runs\.The omission\-exception condition remains close to SLCA on Process, Toucan, and BFCL, while itsτ2\\tau^\{2\}\-Bench score is lower by 4\.78 pp and its Summary score is close to the SFT value\. The full\-zero condition is reported as a guard diagnostic: removing both the summary advantage and the guard raises the no\-call rate to 14\.2% and lowers all listed scores\. Its no\-call rollouts receive no advantage\-weighted policy gradient, while the KL term remains active\. The two rows are not interpreted as a comparison of the summary branch alone\.

### D\.3 Training Dynamics against RLTR and ToolPO

Static benchmark numbers can obscure how proxy objectives and execution quality move during training\. We analyze the full training traces across all three model scales and observe different behaviors for ToolPO’s additive advantage design\.

#### ToolPO training trace

Figure 8:ToolPO training behavior under the LLM\-judge diagnostic \(representative single\-run traces\)\.\(a\)Success@0\.9 across three scales: a decline on 3B \(0\.65→0\.510\.65\{\\to\}0\.51\), a sharper decline on 7B \(0\.72→0\.200\.72\{\\to\}0\.20\), and an overall decline with a brief mid\-training rebound on 8B, ending near0\.320\.32\.\(b\)7B:tag\_open\_ndiverges fromtag\_close\_nafter step 80, reducing the valid parsed\-call count from 1\.91 to 0\.73\.\(c\)8B:pred\_tool\_countsurges1\.71→3\.981\.71\{\\to\}3\.98\(gold≈2\.0\{\\approx\}2\.0\), driving Name F1 and Parallel Score down\.\(d\)3B: Process Score plateaus at∼0\.84\{\\sim\}0\.84, below theSprocess≥0\.9S\_\{\\text\{process\}\}\\geq 0\.9threshold, explaining why proxy metrics improve while strict Success declines\.Heretag\_valid\_pair\_ndenotes the number of parsed, schema\-valid call blocks\. It is not a count formed by pairing the opening and closing tag totals\.

The traces show different failure patterns across model scales \(Figure[8](https://arxiv.org/html/2609.29050#A4.F8)a\):

- •3B: Silent degradation \(Figure[8](https://arxiv.org/html/2609.29050#A4.F8)d\)\.The proxy metrics increase \(RsuccR\_\{\\text\{succ\}\}:0\.24→0\.500\.24\{\\to\}0\.50; summary:0\.56→0\.620\.56\{\\to\}0\.62\), andprocess\_scorerises from 0\.75 to a plateau of∼0\.84\{\\sim\}0\.84\. Strict Success@0\.9 declines \(0\.65→0\.510\.65\{\\to\}0\.51\)\. The observed gap is consistent with the thresholdSprocess≥0\.9S\_\{\\text\{process\}\}\\geq 0\.9: the Process Score remains near∼0\.84\{\\sim\}0\.84\. Format integrity is preserved \(format\_passed≈0\.98\\approx 0\.98,tag\_valid\_pair\_n≈2\.0\\approx 2\.0\), so the decline is not accompanied by a structural\-format change in this trace\.
- •7B: Structural format decline \(Figure[8](https://arxiv.org/html/2609.29050#A4.F8)b\)\.The trace shows more opening than closing tool\-call tags \(tag\_open\_n:1\.50→3\.341\.50\{\\to\}3\.34;tag\_close\_n:1\.46→0\.741\.46\{\\to\}0\.74\), while the summary score remains non\-zero\. The valid parsed\-call count drops from 1\.91 to 0\.73, and Success@0\.9 drops from 0\.72 to 0\.20\. The change is concentrated between steps 80–100, when the gap betweentag\_open\_nandtag\_close\_nwidens to 2\.60\.
- •8B: Redundant tool invocation \(Figure[8](https://arxiv.org/html/2609.29050#A4.F8)c\)\.Qwen3\-8B\-Base keepstag\_open\_nandtag\_close\_nclose in this trace\. The trace instead shows more predicted tool calls \(pred\_tool\_count:1\.71→3\.981\.71\{\\to\}3\.98, gold≈2\.0\{\\approx\}2\.0\)\. Success@0\.9 declines overall, with a brief mid\-training rebound, and ends near∼0\.32\{\\sim\}0\.32as redundant calls accumulate\. At steps 70–80,pred\_tool\_countexceeds 3\.5 and Name F1 falls from0\.800\.80to0\.690\.69\.

These observations are specific to the LLM\-judge diagnostic and do not imply that ToolPO always fails\. They motivate the separate matched GRPO comparison, where the summary\-to\-tool support path is blocked by SLCA’s routing\.

#### RLTR: Coarse\-Grained Reward and Training Stagnation

As visualized in Figure[2\(a\)](https://arxiv.org/html/2609.29050#S4.F2.sf1), RLTR avoids direct summary\-to\-tool leakage via pipeline separation, but its planning reward remains sequence\-level and coarse\. In our runs,rltr\_composcillates between 0\.49 and 0\.64, while*Process*and*Parallel*improve much more slowly than underSLCA\-GRPO\. On Qwen3\-8B\-Base,RcompR\_\{\\text\{comp\}\}doubles \(0\.22→\\to0\.43\) whileerror\_countincreases from 0\.39 to 0\.59, suggesting that the completeness reward does not penalize structurally invalid calls \(cf\. the frozen\-summarizer limitation noted in Table[15](https://arxiv.org/html/2609.29050#A4.T15)\)\.

### D\.4 Visualizing Credit Misattribution

[Figure9](https://arxiv.org/html/2609.29050#A4.F9)presents a diagnostic case on a parallel information\-retrieval task \(comparing the current stock prices of Apple and Microsoft\), illustrating how Cross\-Segment Credit Misattribution manifests as*tool\-use inefficiency masked by a perfect summary*\. The reference trajectory consists of a single parallelget\_stock\_priceinvocation covering both tickers, and the tool\-level reward is decomposed asRtool=0\.10​Rformat\+0\.25​Rname\+0\.15​Rkey\+0\.20​Rvalue\+0\.30​Rparallel\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}=0\.10\\,R\_\{\\text\{format\}\}\+0\.25\\,R\_\{\\text\{name\}\}\+0\.15\\,R\_\{\\text\{key\}\}\+0\.20\\,R\_\{\\text\{value\}\}\+0\.30\\,R\_\{\\text\{parallel\}\}\.

![Refer to caption](https://arxiv.org/html/2609.29050v1/case_study.png)Figure 9:Diagnostic Case Study: Resolving Cross\-Segment Credit Misattribution on a parallel tool\-use task\.\(Top\) Reference trajectory and rewards\.\(Middle\) Standard GRPO\.The agent adopts a redundant, trial\-and\-error trajectory \(first issuing an unnecessarysearch\_company\_infocall, then retrieving the two stock prices through*serial*get\_stock\_priceinvocations in separate turns\), yet still produces a fluent, factually correct comparison \(Rsum=1\.00\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\{=\}1\.00\)\. The tool\-level decomposition exposes where the trajectory fails: theformatreward is unaffected \(1\.001\.00\), thekeyandvaluerewards remain high on the individual calls that do match the schema \(1\.001\.00each\), thenamereward is diluted by the redundantsearch\_company\_infocall \(0\.800\.80\), and theparallelreward is0\.00\\mathbf\{0\.00\}because the two required stock\-price queries are issued as separate sequential turns \(further preceded by a redundant exploration call\), rather than batched into a single parallel invocation\. Aggregated under the fixed weights above, this yieldsRtool=0\.65\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\{=\}0\.65\. Because standard GRPO mergesRtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}andRsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}into a single trajectory\-level advantage, the high summary reward can offset the structural tool penalty in the unified advantage, allowing the inefficient pattern to be reinforced\.\(Bottom\)SLCA\-GRPO\(Ours\)\.The same model, trained under our advantage\-isolation objective, produces a trajectory in which both tickers are known a priori and issues a*single parallel tool call*covering AAPL and MSFT simultaneously\. All five tool sub\-rewards saturate at1\.001\.00, givingRtool=1\.00\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}\{=\}1\.00while preservingRsum=1\.00\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}\{=\}1\.00\. By routingAtool\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}A\_\{\\mathrm\{tool\}\}\}andAsum\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}A\_\{\\mathrm\{sum\}\}\}through separate optimization pathways, segment routing keeps the summary signal from entering the tool\-token update in this example\. The tool\-execution score increases by\+0\.35\+0\.35while the summary score remains1\.001\.00\. The trajectory is qualitatively similar to the representative single\-runpred\_tool\_countincrease observed under ToolPO on 8B \([Figure8](https://arxiv.org/html/2609.29050#A4.F8)c\); the two diagnostics use different protocols\.
### D\.5 BFCL: Robustness in Multi\-Turn Scenarios

While the main paper \([Figure3\(a\)](https://arxiv.org/html/2609.29050#S4.F3.sf1)\) focuses on atomic schema generalization,[Table17](https://arxiv.org/html/2609.29050#A4.T17)additionally reports Multi\-Turn accuracy, now including RLTR and ToolPO\. On Qwen3\-8B\-Base, standard GRPO has a lower Multi\-Turn mean than SFT \(0\.0822±\\pm0\.0035 vs\. 0\.1544±\\pm0\.0102\), whileSLCA\-GRPOreaches 0\.1489±\\pm0\.0025\. The SLCA mean is close to the SFT value in this three\-run comparison; the table does not isolate the source of the Multi\-Turn difference\. On Qwen2\.5\-3B\-Instruct,SLCA\-GRPOalso remains below SFT on Multi\-Turn \(0\.0622±\\pm0\.0025 vs\. 0\.0867±\\pm0\.0100\)\. RLTR remains competitive on AST Non\-Live, suggesting that planner\-only RL can learn basic tool selection under unseen schemas, but it lags on Multi\-Turn where end\-to\-end coordination matters\. Under the reported ToolPO evaluation, the Multi\-Turn means are 0\.0961±\\pm0\.0067 on 3B and 0\.0083±\\pm0\.0067 on 8B\. This spread is reported as context rather than as a claim about the original method across model scales\.

Table 17:Comprehensive Generalization Results on BFCL V3 across Scales\. This table details BFCL generalization performance\. In addition to Overall Acc, AST Live, and AST Non\-Live \(reported in[Figure3\(a\)](https://arxiv.org/html/2609.29050#S4.F3.sf1)\), we include Multi\-Turn accuracy to assess state tracking under schema generalization\. Trained rows report mean±\\pmstd over three runs; Original rows are point evaluations\. ToolPO and RLTR retain their method\-specific protocols\.
### D\.6 Additional Analysis onτ2\\tau^\{2\}\-Bench Results

#### Baseline Configurations and Omissions

The Qwen3\-8B\-BaseOriginalbaseline has a 0\.00 Pass1rate in every domain; it is shown explicitly in[Table18](https://arxiv.org/html/2609.29050#A4.T18)\. The SFT baseline serves as the starting point for the 8B scale comparisons\.

Table 18:Comprehensive Collaboration Results onτ2\\tau^\{2\}\-Bench across Scales\. We report Pass1rates for dual\-control scenarios across three distinct domains: Airline \(User Retention\), Retail \(Order Modification\), and Telecom \(Plan Management\)\. Pass1is the proportion of successful tasks in the finite official task set; because each run evaluates a finite set, the reported values are discrete proportions and their standard deviations reflect variation of those proportions across runs\. The Qwen3\-8B\-Base Original row is included and is zero in all four reported metrics\. Trained rows are mean±\\pmstd over three runs\. ToolPO and RLTR retain their method\-specific protocols\.
#### Domain\-Specific Performance Analysis

TheSLCA\-GRPOmean is higher than the matched GRPO mean on Overall Pass1for all three backbones\. Domain\-level orderings vary by scale: on Qwen2\.5\-3B\-Instruct,SLCA\-GRPOties GRPO on Airline, trails it on Retail, and leads on Telecom; RLTR has the highest Airline mean\. On Qwen2\.5\-7B\-Instruct, the Airline mean is below GRPO while Retail and Telecom are higher\. On Qwen3\-8B\-Base,SLCA\-GRPOleads on Airline and Telecom, while ToolPO has the highest Retail mean\.

The inclusion of RLTR and ToolPO reveals additional patterns\. RLTR trails SLCA on 7B \(0\.3372 vs\. 0\.4102\) and 3B \(0\.2850 vs\. 0\.3563\), constrained by its frozen summarizer in tasks requiring full conversational agent loops\. On Qwen3\-8B\-Base, the reported ToolPO andSLCA\-GRPOOverall means are 44\.32±\\pm1\.86% and 44\.69±\\pm0\.84%, a difference of 0\.37 pp\. ToolPO has the higher Retail mean \(58\.91±\\pm3\.03% vs\. 53\.74±\\pm2\.17%\), whileSLCA\-GRPOis higher on Airline and Telecom\. These results show that the relative ordering depends on the domain and protocol\.

### D\.7 HierR Weight Sensitivity Analysis

To examine sensitivity to HierR weight choices, we evaluate threeSprocessS\_\{\\text\{process\}\}weight configurations on Qwen2\.5\-7B\-Instruct \(three\-run mean±\\pmstd\), summarized in Table[19](https://arxiv.org/html/2609.29050#A4.T19)\.

Table 19:HierR weight sensitivity on Qwen2\.5\-7B\-Instruct\. Entries are mean±\\pmstd over three runs\.The Uniform configuration setsSpar=0\.20S\_\{\\text\{par\}\}=0\.20and remains above the matched GRPO baseline on BFCL andτ2\\tau^\{2\}\-Bench \(\+0\.77 pp and \+6\.75 pp, respectively\)\. The Default configuration has the highest values among the tested settings\.

### D\.8 Unified Reward Ratio Sensitivity

We vary the relative weight of the two raw rewards in the unified GRPO estimator,

A^iuni​\(α\)=Norm⁡\(α​Ritool\+Risum\),α=λtoolλsum,\\hat\{A\}\_\{i\}^\{\\mathrm\{uni\}\}\(\\alpha\)=\\operatorname\{Norm\}\\\!\\left\(\\alpha R\_\{i\}^\{\\mathrm\{tool\}\}\+R\_\{i\}^\{\\mathrm\{sum\}\}\\right\),\\qquad\\alpha=\\frac\{\\lambda\_\{\\mathrm\{tool\}\}\}\{\\lambda\_\{\\mathrm\{sum\}\}\},\(29\)while keeping the SFT initialization, data split, SGLS, HierR components, rollout budget, and evaluation protocol fixed\. The displayed ratios include the existing1:11\{:\}1condition and the three additional tested ratios\.

Table 20:Unified reward ratio sensitivity on Qwen2\.5\-7B\-Instruct\. Entries are mean±\\pmstandard deviation over three matched runs\. The ratio is applied before the unified group normalization; the scale\-invariance identity holds up to the numerical floor described in App\.[A\.3](https://arxiv.org/html/2609.29050#A1.SS3)\.The unified reward ratio sweep recovers part of the SLCA gap within the tested ratios\. The best tested unified ratio remains below SLCA on BFCL andτ2\\tau^\{2\}\-Bench, while increasing the tool weight to5:15\{:\}1reduces theτ2\\tau^\{2\}\-Bench score relative to the best tested ratio\. The pattern indicates a trade\-off between tool and summary rewards in the unified estimator rather than a single uniformly favorable weight\.

### D\.9 Support Control on Qwen2\.5\-7B\-Instruct

We evaluate four support conditions at the primary 7B scale\. LetT=λtool​Norm⁡\(Rtool\)T=\\lambda\_\{\\mathrm\{tool\}\}\\operatorname\{Norm\}\(R\_\{\\mathrm\{tool\}\}\)andS=λsum​Norm⁡\(Rsum\)S=\\lambda\_\{\\mathrm\{sum\}\}\\operatorname\{Norm\}\(R\_\{\\mathrm\{sum\}\}\), with both weights set to one\. The four cells differ only in which normalized components reach each token segment; additive sums are left unrescaled, as in the tested estimator\. All cells use the same initialization, data, rollout budget, and evaluation protocol\. Unified is shown as a joint\-normalization reference and is not part of the factorial effects\.

Table 21:Support control on Qwen2\.5\-7B\-Instruct\. The four support cells use separately normalized components\. Unified is a joint\-normalization reference\. Entries are mean±\\pmstandard deviation over three matched runs\.The factorial effects are computed from the four support cells rather than from a diagonal contrast:

Table 22:Factor effects from the 7B support control\. Values are computed from the cell means in Table[21](https://arxiv.org/html/2609.29050#A4.T21)and are reported in percentage points\. Main effects average the two simple effects over the other factor; interaction entries use the unhalved difference\-in\-differences contrast\.Closing theS→TS\\mathbin\{\\to\}Tpath gives the larger estimated effect on all three benchmarks in this control\. Opening theT→ST\\mathbin\{\\to\}Spath has a smaller effect, and the difference between that condition and SLCA is−0\.11/\+0\.07/\+0\.34\-0\.11/\+0\.07/\+0\.34pp on Toucan, BFCL, andτ2\\tau^\{2\}\-Bench\. These results are support control effects under the tested additive estimator; they are not presented as a routing\-only causal estimate\.

### D\.10 Full Ablation Studies Across Scales

#### Component contributions across scales\.

Table[23](https://arxiv.org/html/2609.29050#A4.T23)reports ablation results across the three backbones\. The w/o SLCA rows have lower reported Toucan Success means by 2\.05 to 2\.53 pp across the three backbones\. The matched BFCL andτ2\\tau^\{2\}\-Bench means are also lower in the reported blocks; the 3B BFCL gap is 0\.40 pp\. The w/o HierR rows show the largest Toucan Success drops in each block and lower means on most other Toucan metrics\. Removing deterministic schema validation and schema\-conditioned tool\-response prompting has a smaller effect on in\-domain Success; it also changes BFCL andτ2\\tau^\{2\}\-Bench, with the largest BFCL drop at 8B\.

Table 23:Comprehensive Ablation Studies Across Model Families\. Comparison of FullSLCA\-GRPOagainst variants removing key components\. Metrics include Toucan\-Test \(Name F1, ArgMatch, Process, Success\), BFCL \(Accuracy\), andτ2\\tau^\{2\}\-Bench \(Pass1\)\. Process is computed as a post\-hoc evaluation metric for all settings, including w/o HierR\. Trained rows in all three blocks report mean±\\pmstd over three runs; Original evaluations are point values where shown\. The w/o SLCA condition is the SFT\+GRPO estimator\.

### D\.11 Scaling to Large Tool Spaces Across Families

Table[24](https://arxiv.org/html/2609.29050#A4.T24)extends the scalability analysis across all three backbones, including an extreme stress test atK=150K=150\. The table compares the two sampling strategies at each candidate\-set size:

- •Semantic interference:Across all models, performance drops under Hard\-Negative sampling compared to Random sampling \(e\.g\., Qwen3\-8B drops from∼\\sim60% to∼\\sim36% atK=100K=100\)\. The stress test makes semantically similar tools harder to distinguish than random candidates at the same set size\.
- •Effect of context limits:AtK=150K=150for Qwen2\.5\-7B, the Random\-sampling rows are near\-tied, while the Hard\-sampling gap is \+2\.50 pp\. These results show how candidate\-set construction and context length interact with the estimator\.
- •Across tested families:SLCA\-GRPOis higher than GRPO in the reported Hard\-sampling rows \(e\.g\., \+2\.25 pp for 3B atK=150K=150\)\.

Table 24:Comprehensive Scalability Results Across Model Families\. We reportSuccess@@0\.9as the candidate set sizeKKincreases\. Comparison is made between Hard\-Negative \(Semantic\) and Random sampling to isolate the impact of semantic interference\. TheOriginalcolumn reports the multi\-run mean atK=0K=0, matching the main results; theK\>0K\>0columns retain the stress\-test evaluations\. ForK≥100K\\geq 100, especiallyK=150K=150, these stress\-test values are computed on context\-valid survivors after skip filtering\.
### D\.12 Execution\-Based Reward Experiment

To test whether SLCA’s gains extend beyond gold\-trajectory matching rewards, we replaceRtool=Sprocess\{\\color\[rgb\]\{0\.418,0\.3711,0\.8203\}R\_\{\\mathrm\{tool\}\}\}=S\_\{\\text\{process\}\}with execution\-basedRsuccR\_\{\\text\{succ\}\}\.Rsum=Ssummary\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}R\_\{\\mathrm\{sum\}\}\}=S\_\{\{\\color\[rgb\]\{0\.3711,0\.6406,0\.3047\}\\mathrm\{summary\}\}\}remains unchanged\. Both methods use Qwen2\.5\-7B\-Instruct with identical data, SGLS, and compute\.

For this test, the binaryRsuccR\_\{\\text\{succ\}\}judge labels each rollout as Solved or Unsolved without per\-example gold\-call matching\. Table[25](https://arxiv.org/html/2609.29050#A4.T25)reports the matched comparison\.

Table 25:Execution\-basedRsuccR\_\{\\text\{succ\}\}comparison on Qwen2\.5\-7B\-Instruct\. Entries are mean±\\pmstandard deviation over three runs; data, initialization, SGLS, and compute are matched\.The SLCA advantage persists under the execution\-based reward\. Absolute scores are lower than under dense HierR rewards because the execution signal is sparser\.

#### Training Dynamics underRsuccR\_\{\\text\{succ\}\}

Figure 10:Training dynamics under execution\-basedRsuccR\_\{\\text\{succ\}\}\(7B\)\.The curves are single\-run checkpoint evaluations on the same fixed set of 3,991 valid Toucan validation examples\. The plottedRsuccR\_\{\\text\{succ\}\}is distinct from Toucan Success@0\.9\. At step 100,SLCA\-GRPOreachesRsucc=0\.593R\_\{\\text\{succ\}\}\{=\}0\.593, compared with0\.4240\.424for SFT\+GRPO \(w/o SLCA\)\.TheRsuccR\_\{\\text\{succ\}\}learning curve \(Figure[10](https://arxiv.org/html/2609.29050#A4.F10)\) reveals a clear divergence: at step 0, both methods start atRsucc≈0\.29R\_\{\\text\{succ\}\}\\approx 0\.29; by step 100,SLCA\-GRPOreaches 0\.592583 \(2365/3991\), whereas the SFT\+GRPO \(w/o SLCA\) baseline reaches 0\.423703 \(1691/3991\), a \+16\.9 pp gap\. The logged validation series isval\-aux/toucan\_eval\_v4/reward/tool\_call/r\_succ/mean@1; each checkpoint reports the exact empirical proportion on the same fixed set of 3,991 examples from one run per method, rather than an across\-run mean\. This execution\-based metric is separate from Toucan’s strict Success@0\.9 measure\.

## Appendix EPrompts

### E\.1 Summary Judge Prompts

\#\#Task

EvaluatethequalityofanAIassistant’sfinalresponsebasedonthetoolresultsitreceived\.

\#\#AssessmentCriteria

\#\#\#ResponseQuality\(1\-5scale\)

\-1\(verypoor\):Responseiscompletelywrong,irrelevant,orignorestoolresults

\-2\(poor\):Seriouslymisinterpretstoolresultsormissescriticalinformation

\-3\(acceptable\):Basicallycorrectbuthasminorerrorsorincompletecoverage

\-4\(good\):Correctlyusestoolresults,responseisclearandcomplete

\-5\(excellent\):Perfectinterpretationofresults,professionalandhelpfulresponse

\#\#UserQuestion

‘‘‘

\{question\_content\}

‘‘‘

\#\#ToolResults

‘‘‘

\{tool\_responses\}

‘‘‘

\#\#Model’sFinalResponse

‘‘‘

\{final\_response\}

‘‘‘

\#\#EvaluationPoints

1\.\*\*ResultInterpretation\*\*:Doesthemodelcorrectlyunderstandandexplainthedatareturnedbytools?

2\.\*\*InformationCompleteness\*\*:Doestheresponseaddressallpartsoftheuser’squestionbasedonthetoolresults?

3\.\*\*ExpressionQuality\*\*:Istheresponseclear,well\-organized,andhelpfultotheuser?

\#\#OutputFormat

<response\>

<response\_quality\>

<reasoning\>Briefanalysisoftheresponsequalitybasedonthethreeevaluationpointsabove\.</reasoning\>

<rating\><\!\-\-oneof:verypoor,poor,acceptable,good,excellent\-\-\></rating\>

</response\_quality\>

</response\>

### E\.2 SGLS Simulator Prompts

\#CRITICALROLE:YouareanAPISERVER,NOTanAIassistantoragent\.

Youarerole\-playingasa\*\*backendAPIserver\*\*thatexecutesfunctioncallsandreturnsresults\.

YouareNOTmakingtoolcalls\-youareRESPONDINGtothemasifyouweretheactualAPIendpoint\.

\#\#YOURONLYOUTPUTFORMAT:

<tool\_response\>\{"key":"value",\.\.\.\}</tool\_response\>

\#\#ABSOLUTEPROHIBITIONS:

\-NEVERoutput<tool\_call\>tags\(youaretheSERVER,nottheclient\)

\-NEVERoutputmarkdowncodeblocks

\-NEVERoutputexplanationsorconversationaltext

\-ONLYoutputthe<tool\_response\>tagwithJSONinside

\#\#ExamplesofCORRECTserverresponses:

Input:calculator\(expression="124\*5"\)

<tool\_response\>\{"result":620\}</tool\_response\>

Input:get\_weather\(city="Seattle"\)

<tool\_response\>\{"temp":"12$^\\circ$C","condition":"Rainy"\}</tool\_response\>

Input:google\_search\(query="capitalofAustralia"\)

<tool\_response\>\{"snippets":\["CanberraisthecapitalcityofAustralia\."\]\}</tool\_response\>

\-\-\-

\#\#NOWEXECUTETHISAPICALL:

ToolDefinition:

%s

IncomingRequest:

%s

Returntheexecutionresultin<tool\_response\>\.\.\.</tool\_response\>format: