Diagnosing and Calibrating Tool-Call Boundary Drift in Multi-Teacher On-Policy Distillation

Hugging Face Daily Papers Papers
tool-call boundary-drift multi-teacher knowledge-distillation agentic calibration language-models

Summary

This paper diagnoses and proposes SoftClamp, a calibration method that reduces tool-call boundary drift in multi-teacher on-policy distillation for agentic language models, decreasing over-calling while maintaining accuracy.

Agentic language models must learn when to call tools, when to consume tool responses, and when to answer directly. This makes multi-teacher on-policy distillation a natural training strategy: one teacher can specialize in tool calls, another in direct responses, and the student can learn from both on its own generated distribution. We show that this strategy can induce a behavior shift that is invisible from aggregate losses alone. In a two-teacher tool-use setting, vanilla generalized knowledge distillation improves tool-call recall but also moves the model toward over-calling, where it calls tools on examples that should be answered directly. Aggregate explanations are insufficient: tool-call samples do not receive more token exposure, and full-sequence per-token divergence is not larger for the tool-call teacher. We instead analyze behavior leverage imbalance: local token-level signals at mode- entry and structural positions, such as <tool_call> and function names, can have disproportionate control over the global generation mode. We propose Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero gradients. On APIGen-MT, Soft Clamp reduces over-calling from 13.7% to 9.0% relative to vanilla GKD while matching its decision accuracy. In a BFCL multi-turn diagnostic, it also lowers tool-call loops and repeated calls among GKD variants. These results suggest that multi-teacher OPD should monitor where teacher signals act, not only how large they are in aggregate.
Original Article
View Cached Full Text

Cached at: 07/21/26, 02:37 PM

Paper page - Diagnosing and Calibrating Tool-Call Boundary Drift in Multi-Teacher On-Policy Distillation

Source: https://huggingface.co/papers/2607.07050 Published on Jul 15

·

Submitted byhttps://huggingface.co/jiabin

shenon Jul 21

Abstract

Agenticlanguagemodelsmustlearnwhentocalltools,whentoconsumetoolresponses,andwhentoanswerdirectly.Thismakesmulti-teacheron-policydistillationanaturaltrainingstrategy:oneteachercanspecializeintoolcalls,anotherindirectresponses,andthestudentcanlearnfrombothonitsowngenerateddistribution.Weshowthatthisstrategycaninduceabehaviorshiftthatisinvisiblefromaggregatelossesalone.Inatwo-teachertool-usesetting,vanillageneralizedknowledgedistillationimprovestool-callrecallbutalsomovesthemodeltowardover-calling,whereitcallstoolsonexamplesthatshouldbeanswereddirectly.Aggregateexplanationsareinsufficient:tool-callsamplesdonotreceivemoretokenexposure,andfull-sequenceper-tokendivergenceisnotlargerforthetool-callteacher.Weinsteadanalyzebehaviorleverageimbalance:localtoken-levelsignalsatmode-entryandstructuralpositions,suchas<tool_call>andfunctionnames,canhavedisproportionatecontrolovertheglobalgenerationmode.WeproposeSoftClamp,aper-tokendivergencecalibrationmethodthatdynamicallycompressesextremetoken-levelJensen-Shannondivergencewhilepreservingnonzerogradients.OnAPIGen-MT,SoftClampreducesover-callingfrom13.7%to9.0%relativetovanillaGKDwhilematchingitsdecisionaccuracy.InaBFCLmulti-turndiagnostic,italsolowerstool-callloopsandrepeatedcallsamongGKDvariants.Theseresultssuggestthatmulti-teacherOPDshouldmonitorwhereteachersignalsact,notonlyhowlargetheyareinaggregate.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.07050

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.07050 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.07050 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.07050 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation

arXiv cs.CL

This paper identifies a failure mode in multi-teacher on-policy distillation for agentic language models where the student becomes over-reliant on tool-calling, and proposes Soft Clamp, a per-token divergence calibration method that reduces over-calling without sacrificing accuracy.

The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes

Hugging Face Daily Papers

This paper presents a comprehensive empirical study on on-policy distillation for large language models, identifying failure mechanisms like distribution mismatch and optimization instability, and proposing fixes such as stop-gradient objectives and RLVR-adapted teachers.

Blockwise Policy-Drift Gating for On-Policy Distillation

arXiv cs.LG

This paper introduces blockwise policy-drift gating, a lightweight method to improve on-policy distillation for language models by weighting loss based on old-current student probability shifts, achieving improved reasoning accuracy on math benchmarks.

On-Policy Distillation (5 minute read)

TLDR AI

This paper introduces on-policy distillation, which trains a student model on its own trajectories with teacher token-level KL supervision to fix train-inference mismatch, unifying forward-KL, reverse-KL, and JSD losses, with reverse-KL favored for smaller students.