loss-reweighting

This paper proposes LARE (Loss-Adaptive Reweighting under fixed Exposure), a method that adjusts domain loss weights from running loss statistics during LLM pretraining, achieving an average +7.84 pp improvement across five benchmarks for a 1.5B model over a matched data mixture.