Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation

Hugging Face Daily Papers Papers
multi-teacher-distillation on-policy-distillation gradient-conflict llm-training knowledge-distillation credit-assignment

Summary

提出 RoadMOPD（Cross-Teacher Guarding），通过引入非路由教师作为跨域批评者，在 token 级信用分配与预防性梯度投影协调多教师监督，缓解多教师在线蒸馏中的梯次冲突与能力退化，在高冲突场景下提升超过 5 个百分点。

Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) by routing domain-specific data to specialized teachers, integrating their capabilities through dense token-level supervision on student-generated trajectories. However, domain routing only distinguishes the supervision sources for different domains; updates from all domains still act on shared parameters. When teachers differ substantially in response style and domain expertise, supervision that is effective in isolation may still lead to persistent gradient conflicts and capability degradation during joint optimization. To address this challenge, we propose Cross-Teacher Guarding for MOPD (Road MOPD), which introduces non-routed teachers as auxiliary cross-domain critics on the same student trajectories and coordinates multi-teacher supervision through token-level credit assignment and gradient updates. Specifically, cross-teacher-guided credit assignment uses teacher agreement to strengthen token-level corrections jointly supported by the teachers and attenuate conflicting signals; preventive multi-teacher gradient projection uses auxiliary gradients from the same trajectories to constrain potentially conflicting updates and adaptively controls projection strength to retain target-domain learning signals. In experiments, Road MOPD mitigates degradation in the disadvantaged domain, gains over 5 percentage points under high conflict, and improves overall performance in low-conflict and three-domain settings.
Original Article
View Cached Full Text

Cached at: 10/15/26, 02:55 AM

Paper page - Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation

Source: https://huggingface.co/papers/2610.13521 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Multi-teacheron-policydistillation(MOPD)extendson-policydistillation(OPD)byroutingdomain-specificdatatospecializedteachers,integratingtheircapabilitiesthroughdensetoken-levelsupervisiononstudent-generatedtrajectories.However,domainroutingonlydistinguishesthesupervisionsourcesfordifferentdomains;updatesfromalldomainsstillactonsharedparameters.Whenteachersdiffersubstantiallyinresponsestyleanddomainexpertise,supervisionthatiseffectiveinisolationmaystillleadtopersistentgradientconflictsandcapabilitydegradationduringjointoptimization.Toaddressthischallenge,weproposeCross-TeacherGuardingforMOPD(RoadMOPD),whichintroducesnon-routedteachersasauxiliarycross-domaincriticsonthesamestudenttrajectoriesandcoordinatesmulti-teachersupervisionthroughtoken-levelcreditassignmentandgradientupdates.Specifically,cross-teacher-guidedcreditassignmentusesteacheragreementtostrengthentoken-levelcorrectionsjointlysupportedbytheteachersandattenuateconflictingsignals;preventivemulti-teachergradientprojectionusesauxiliarygradientsfromthesametrajectoriestoconstrainpotentiallyconflictingupdatesandadaptivelycontrolsprojectionstrengthtoretaintarget-domainlearningsignals.Inexperiments,RoadMOPDmitigatesdegradationinthedisadvantageddomain,gainsover5percentagepointsunderhighconflict,andimprovesoverallperformanceinlow-conflictandthree-domainsettings.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.13521

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.13521 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.13521 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.13521 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation

Hugging Face Daily Papers

MOPD-Router proposes a token-level routing framework for multi-teacher on-policy distillation, enabling the use of unlabeled data and cross-domain supervision without domain labels. Experiments demonstrate significant performance improvements over standard methods in both unlabeled and labeled scenarios.