self-distillation

Introduces U-OPSD, an unsupervised on-policy self-distillation method that uses internal consistency and majority-vote pseudo-solutions to improve LLMs without external supervision, matching or exceeding supervised methods on math benchmarks.

This paper introduces State-Matched Routing and Contextualized Self-Distillation (SMRC-SD), a method that selectively applies privileged trajectory guidance only when the student agent's current state matches the reference, improving multi-turn agent performance on ALFWorld and WebShop benchmarks.

iFAN is a training framework that improves mask transformers for segmentation by aligning query ranking with mask quality and distilling intermediate predictions to the final layer, yielding consistent gains across benchmarks.

This paper studies self-distillation with privileged information (PI) as a lone post-training objective for LLMs, reproducing reported gains on easy tasks but showing it fails on difficult reasoning tasks: per-token loss drops while validation accuracy stagnates or degrades. The authors trace the failure to PI bias, which pulls teacher targets toward a reference trajectory and trains students to be flatter and less decisive without improving reasoning.

AgentOPSD proposes a critic-free recursive method for turn-level credit assignment in agentic RL, reweighting outcomes via Bayesian belief updates in log-odds space. It achieves 89.1% success on ALFWorld with Qwen2.5-7B, outperforming GRPO and self-distillation baselines.

This paper introduces Rubrics as Privileged Information (RuPI), extending on-policy self-distillation to open-ended generation by conditioning the teacher on rubrics as soft privileged information. The method outperforms rubric-as-reward RL and reference-completion distillation across multiple LLMs and benchmarks like HealthBench.

Introduces OPD-V, a visual on-policy self-distillation paradigm for multimodal large language models that leverages positive and negative teachers to exploit modality balance as privileged information, improving reasoning performance across benchmarks while reducing training cost.

HelloWorld is a video world model that enables socially interactive characters, allowing users to prompt on-screen characters to respond via a single button press. It uses self-distillation and training-free cross-attention masking to naturalize interactions, and introduces HelloWorldBench for evaluation.

TurnSight introduces a turn-level hindsight self-distillation framework for tool-integrated reasoning, providing dense supervision via execution-conditioned hindsight and adaptive RL advantage modulation.

This paper introduces Stateful Knowledge Learning (SKL), a framework that shifts LLM agents from trajectory-level reflection to maintaining explicit state-anchored predictive knowledge, demonstrating improved generalization and policy-making on interactive environments like WebShop, ScienceWorld, and ChessPuzzles.

PCSD introduces a method for dense token-level supervision in agentic reinforcement learning by using persistent consistency of teacher-favoring signals, improving performance on ALFWorld and WebShop over baselines like GRPO and SDAR.

This paper introduces Dual-Anchored Policy Distillation (DAPD), a framework to address privilege illusion in on-policy self-distillation for language models. DAPD improves Qwen3-4B by +2.00 points on average across tasks, with gains persisting at larger scales.

RestoreKV introduces a learned restoration mechanism that complements query-agnostic KV cache eviction, recovering full-cache behavior under aggressive budgets by generating a compact context-conditioned restore cache via a single LoRA-adapted pass, improving performance on four long-context benchmarks.

This paper introduces CoMem, a method that exploits the depth-wise division of labor in LLMs to cache intermediate residual tensors and recompute only upper layers for retrieval, enabling bounded read compute and memory independent of stored-context length. Evaluated on Qwen3-8B, CoMem achieves strong long-context performance with significant memory savings and prefill speedups.

This paper introduces β-OPSD, a generalization of on-policy self-distillation that frames it as a policy optimization family with a controllable KL penalty. The method uses distillation to approximate expensive policy optimization, improving stability and reasoning performance on mathematical benchmarks.

Proposes NOPD, a self-distillation method that improves vision-language models without external supervision by leveraging prediction discrepancies between clean and corrupted inputs. Achieves significant gains on visual reasoning tasks, matching or exceeding RL and distillation from external models.

VCSD removes the need for external teachers, privileged answers, or visual evidence in on-policy self-distillation by using content-erased control images to produce contrastive signals, consistently outperforming existing methods on vision-language benchmarks.

PromptSD is a novel on-policy self-distillation method where the teacher differs from the student only by a learnable soft prompt, enabling multi-task knowledge absorption without weight drift. It matches or exceeds full fine-tuning on four tasks while preserving general capabilities.

Combining GRPO and OPSD, CriPO is a rubric-based reinforcement learning method from ByteDance and Zhejiang University that addresses unexplored and suppressed criteria via self-distillation, achieving better performance and compute efficiency.

Introduces CriPO, a method that enhances rubric-based reinforcement learning for LLMs via on-policy self-distillation, addressing unexplored and suppressed criteria. Experiments on medicine and science benchmarks show stronger final performance with roughly 2x fewer optimization steps.