Tag
A CMU-led paper introduces 'unified training,' showing that pretraining, midtraining, SFT, and RL can be collapsed into a single likelihood-based algorithm where RL objectives are treated as maximum-likelihood fitting to model-generated targets, outperforming staged training in several accuracy-compute and continual-learning experiments.
A paper unifying foundation-model training stages (pretraining, midtraining, SFT, RL) under a single likelihood-based algorithm that blends static data and RL generations, improving accuracy-compute tradeoff and capability retention.
This paper proves the Ho–Nguyen conjecture on the order of density cancellation in over-fitted Gaussian mixtures for all cluster sizes M≥2, deriving exact maximum-likelihood exponents and local minimax rates, and applies the coefficient framework to identifiability issues in mixtures of experts.
This paper connects score matching, maximum likelihood, and expectation-maximization in mixed linear regression, providing theoretical guarantees and analyzing gradient behavior under diffusion noise levels.
This paper identifies sufficient sample sizes for maximum-likelihood support recovery of sparse binary signals in high-SNR regimes, revealing an information-theoretic threshold and trade-offs between measurement sparsity and computational cost.