Tag
D³-MOPD is a zero-overhead scheduler for multi-teacher distillation that dynamically adjusts domain sampling ratios based on per-domain reverse-KL trajectories, improving convergence efficiency and closing most of the student-to-teacher performance gap.
The paper introduces Open-MOPD, a framework that diagnoses and fixes capability imbalance in multi-teacher on-policy distillation by balancing token-level budgets, improving headroom recovery from 35.6% to 83.4% through dynamic allocation and reward refresh.
This paper identifies a failure mode in multi-teacher on-policy distillation for agentic language models where the student becomes over-reliant on tool-calling, and proposes Soft Clamp, a per-token divergence calibration method that reduces over-calling without sacrificing accuracy.
Agents-A1 is a 35B Mixture-of-Experts agentic model from InternScience that achieves competitive performance against frontier-scale systems like GPT-5.5 and DeepSeek-V4-pro using long-horizon trajectory scaling and multi-teacher multi-domain distillation.
MOPD proposes a multi-teacher on-policy distillation paradigm for LLM post-training, enabling efficient integration of multiple domain capabilities by distilling specialized RL teachers into a student model using its own rollouts. It outperforms existing methods like Mix-RL and Cascade RL, and has been deployed in industrial-scale models.