Tag
This paper introduces the Bridge-Garden Decomposition theory to explain why mixing hard and soft labels in LLM distillation reduces exposure bias, and develops hybrid supervision methods that outperform existing baselines while reducing training cost by 9.7×.
This paper introduces Motab, a new pipeline for LLM reasoning distillation that mitigates both off-policy and on-policy exposure biases by dynamically monitoring student generation and backtracking to safe states with teacher intervention, achieving ~3% average improvement.
This paper introduces SDFlow, a similarity-driven flow matching framework for time series generation that addresses exposure bias in autoregressive models. It achieves state-of-the-art performance and inference speedups by operating in the frozen VQ latent space with low-rank manifold decomposition.