Tag
The paper introduces Certified Anchor-Based Adaptation (CABA), a post-update gating framework that reserves held-out anchor observations to certify whether a proposed model update stays within a declared budget for changes to protected behavior, with theory-backed divergence guarantees and a UCI HAR proof-of-concept showing reduced forgetting at a retention–plasticity cost.
该论文从蒸馏目标本身推导 token 重要性（基于局部 KL 预算下的损失可降幅度，即 logit 梯度与 Fisher 矩阵构成的效用分数），并实例化为 Fisher Utility Weighting（FUW），在七个数学推理基准和三组 Qwen3 师生模型上相对均匀 OPD 提升 1.08–4.0 个宏平均百分点。
研究发现 on-policy 蒸馏（reverse KL）主要提升语言模型的组合推理能力，而几乎不传递新的事实知识；改用 forward KL 可恢复事实传递。该论文通过受控合成框架解耦蒸馏配方，揭示了技能与知识转移之间的不对称性。
提出 KL 正则化策略优化（KLPO）框架，通过在采样器处锚定 KL 正则项实现无 critic、无重要性权重的异步 RL 更新，仅需每 prompt 一次 rollout，并证明 SPPO、GPO、REBEL 和 BPO 均为其特例。
The paper introduces LLM-as-Jev, an architecture-preserving framework that extracts calibrated decision distributions from LLM next-token probabilities, and shows modern LLMs are already effective Jev-style decision models, with fine-tuning giving targeted gains on weaker models and specific tasks while KL anchors prevent conversational degradation.
This UCLA paper theoretically analyzes on-policy distillation (OPD) versus off-policy/SFT-style distillation, showing how forward and reverse KL objectives aggregate multiple teachers into arithmetic vs geometric mixtures, establishing logarithmic regret bounds, and explaining both the benefits and fragility of OPD (e.g., sensitivity to low-probability teachers and incorrect-prefix bias).
This paper analyzes why pointwise clipping of the forward-KL objective in on-policy self-distillation (OPSD) can backfire: it proves and empirically shows that clipped objectives can push token probabilities away from the teacher, causing persistent token repetitions in reasoning outputs.
This paper systematically studies the role of rollout policy in strong-to-weak distillation, finding that token-level KL direction and learning rate matter more than on-policy data, challenging the assumption that on-policy rollouts are inherently preferable.
This paper investigates whether KL divergence is necessary for on-policy distillation of large language models, showing that preserving update direction suffices and introducing Consensus Multi-Teacher On-Policy Distillation (C-MOPD) to enhance multi-teacher learning.
IncentRL presents a framework for preference-based reward shaping in reinforcement learning that uses KL divergence penalty to balance preference guidance with task performance, supported by theoretical analysis and experiments on MiniGrid benchmarks.
ByteShape released ShapeLearn GGUFs for Qwen 3.8 27B, achieving high accuracy on benchmarks, and discussed the role of KLD in quantization, with a related paper accepted at EMNLP 2026.
This paper proposes a principled communication gating mechanism for multi-agent reinforcement learning using KL divergence between agents' belief distributions, showing performance improvements and interpretability on benchmarks like Predator-Prey and MPE.
This paper introduces Fisher8, an output-layer gradient correction that uses Fisher geometry instead of Euclidean geometry to stabilize neural heteroscedastic regression, improving uncertainty calibration and likelihood-error tradeoffs.
A detailed benchmark comparing 16 quantizations of Qwen3.6 27B across GGUF, NVFP4, AWQ, AutoRound, and FP8 formats, measuring KL divergence from the unquantized reference. Weight-only GGUF quants generally offer the best quality-size tradeoffs, while vLLM quants vary substantially.
KLQ is a training-free LLM quantization method that allocates bits per direction based on measured KL divergence, outperforming existing training-free rotation-based methods on W4A4KV4-bit settings for models like Llama 3.2 1B and Qwen 2.5.
atomic.chat released 14 quantized GGUF builds of DeepSeek V4 Flash 0731, from lossless BF16 to 1-bit. They recommend AD-IQ2_M for 128GB hardware, which matches the original's token choice 83.6% of the time.
A developer built a testing harness that measures KL divergence per weight group during quantization, leading to three custom quantized builds of Qwen3.6-27B (Bedrock, Tightrope, Gambit) with optimized compression. Tool calling is identified as the first capability to degrade under quantization.
This paper establishes a rigorous quantitative connection between neural network score function approximation and the resulting distribution approximation in score-based diffusion models, proving that accurate score approximation leads to close distribution approximation in KL divergence, with an explicit bound.
Discusses the trade-offs of using Kullback-Leibler divergence in quantitative analysis, framing it as a Hamlet-like dilemma for quants.
This tweet lists 7 fundamental math ideas used daily by ML engineers, with brief explanations emphasizing the underlying derivations, such as why gradient descent moves in the steepest direction and why softmax plus cross-entropy yields a clean gradient.