Tag
Poly-OPD is a framework for distilling complementary strengths from heterogeneous text-to-image flow models into a single compact flow-matching student, using pixel bridges and gradient-compatible adapters. It improves GenEval and DrawBench scores while consolidating multiple teacher capabilities.
This paper introduces AMRD, an adaptive multi-teacher relational distillation method for compressing large self-supervised speech emotion recognition models into lightweight student models for edge devices. It addresses teacher reliability variation and relational structure loss, showing improvements on IEMOCAP and CREMA-D datasets.
This paper diagnoses and proposes SoftClamp, a calibration method that reduces tool-call boundary drift in multi-teacher on-policy distillation for agentic language models, decreasing over-calling while maintaining accuracy.
This paper introduces a compete-then-collaborate framework where multiple frontier AI teachers (Claude, Codex-GPT, Grok, Gemini) are ranked by execution-based tests and then collaborate to build a verifiable curriculum. It finds that imitation (SFT) on teacher solutions degrades a competent coder student, while using the same curriculum for reinforcement learning with verifiable rewards (RLVR) improves performance, particularly on competition problems.
An article surveying how frontier AI models in 2026 use distillation techniques, covering off-policy, on-policy, and self-distillation stages, with examples from Gemma, DeepSeek, GLM, Nemotron, and Qwen3.
CoRD is a collaborative multi-teacher decoding framework that synthesizes reasoning trajectories through predictive perplexity scoring and beam search, enabling efficient distillation of large reasoning models with high-quality outputs and generalized performance.