learned-optimization

Proposes Efficient Long-horizon Optimization (ELO) learning, a meta-training algorithm that reallocates compute to longer horizons and uses decoupled progressive expert supervision, improving learned optimizers' performance on long-unroll tasks and out-of-distribution generalization. ELO-Celo2 consistently outperforms AdamW and matches Muon on language modeling tasks.