Self-Distillation with Verbal Regularization
Summary
论文提出VRSD（verbal regularization self-distillation），通过系统提示调节teacher对特权信息的使用，抑制on-policy自蒸馏中的捷径推理与熵坍缩，在11/12个模型-任务设置上平均提升Pass@1达3.9个百分点。
View Cached Full Text
Cached at: 10/14/26, 10:53 AM
Paper page - Self-Distillation with Verbal Regularization
Source: https://huggingface.co/papers/2610.16354 Published on Oct 12
·
Submitted byhttps://huggingface.co/jiazhengli
J Lion Oct 14
Abstract
On-policyself-distillation(OPSD)trainsalargelanguagemodelusingsupervisionfromateacherinstantiatedfromthesamemodelasthestudent.Theteacherisconditionedonprivilegedinformation(PI),suchasverifiedsolutions.Thestudent,whichdoesnotobservePI,learnsfromtheteacher’snext-tokendistributionsalongitsownsampledresponses.However,PIcanbiassupervisiontowardshortcutreasoningwithlimitedverificationorreconsideration,accompaniedbyasharpdropintheentropyoftheoutputdistribution.Toaddressthischallenge,weintroduceself-distillationwithverbalregularization(VRSD),whichusessystempromptstoregulatehowtheteacherusesPI,therebyencouragingrethinkingandexplorationtocounteractprematurecommitment.Theoretically,wemodelpromptingasattenuatingtheinfluenceofPIontheteacherdistributionandderiveconditionsforentropypreservation.Acrossthreemodelsandfourtasks,VRSDimprovesPass@1overOPSDin11of12model-tasksettings,by3.9percentagepointsonaverage,whilegenerallymaintaininghighertokenentropy.
View arXiv pageView PDFGitHub3Add to collection
Get this paper in your agent:
hf papers read 2610\.16354
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.16354 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.16354 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.16354 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement
UniEvo-VL 提出了一种用于多模态模型的在线自蒸馏训练方案，让单一模型同时充当教师与学生（教师条件于自生成的批评信息），以提升其图像生成能力，在 GenEval 上从 0.747 提升至 0.808。
Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation
Proposes NOPD, a self-distillation method that improves vision-language models without external supervision by leveraging prediction discrepancies between clean and corrupted inputs. Achieves significant gains on visual reasoning tasks, matching or exceeding RL and distillation from external models.
H^2SD: Hybrid Hindsight Self-Distillation
Introduces H^2SD, a hybrid hindsight self-distillation framework that improves RLVR by using the teacher model differently for successful and failed trajectories, achieving better reasoning performance.
Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information
Proposes Anti-Self-Distillation (AntiSD) which reverses the knowledge transfer direction in self-distillation to improve math reasoning efficiency and accuracy, achieving GRPO baseline accuracy in 2-10x fewer steps and up to 11.5 points higher final accuracy across models from 4B to 30B parameters.
Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning
The paper proposes EGRSD and CL-EGRSD, on-policy self-distillation methods that weight token-level supervision by teacher entropy to improve reasoning accuracy-length tradeoff in LLMs, evaluated on Qwen3-4B and Qwen3-8B.