capability-seesaw

Tag

Cards List
#capability-seesaw

PMOPD: Task Ordering, Cycling, and Parameter-Update Subspace Protection in Multi-Teacher On-Policy Distillation

Hugging Face Daily Papers ↗ · 2d ago Cached

PMOPD 提出一种几何感知的多教师在线策略蒸馏方法,通过构建任务子空间记忆并对梯度和优化器更新做投影来缓解能力跷跷板效应,并加入冲突探测与任务循环策略,在 Code/Reason/Math 任务上于 Qwen2.5-7B 和 Llama-3.1-8B 上分别平均提升 2.54 和 2.09 分。

0 favorites 0 likes
← Back to home

Submit Feedback