Tag
This paper proposes SCALE (Selective Control of Adaptation via Local Entropy), an entropy-guided method that freezes the pretrained model and SFT delta while learning bounded token- and module-specific gates to suppress, reverse, or extrapolate learned features during supervised fine-tuning. The approach achieves strong math-reasoning and code-generation results on Qwen2.5-Math and Qwen3-4B-Base models, outperforming existing token-reweighting baselines.
This paper introduces Entropic Advantage Policy Optimization (EAPO), an entropy-guided credit assignment method for reinforcement learning in large language models, which asymmetrically treats success and failure to enhance exploration and achieve better performance on reasoning tasks.
Introduces a training-free Semantic-Aware Kernel Entropy (SAKE) guidance method for text diffusion models, using order-2 Rényi entropy over a kernel Gram matrix to balance fidelity and diversity during sampling. Experiments show improved Pareto frontier and multi-sample performance on reasoning-intensive tasks.
The paper proposes EGRSD and CL-EGRSD, on-policy self-distillation methods that weight token-level supervision by teacher entropy to improve reasoning accuracy-length tradeoff in LLMs, evaluated on Qwen3-4B and Qwen3-8B.