Tag
This paper provides worst-case convergence analyses for Schedule-Free gradient descent and stochastic gradient descent in nonconvex optimization, establishing optimal rates and strict-saddle avoidance, thus theoretically justifying their empirical success.
This paper introduces SF-NorMuon, a schedule-free spectral optimizer that matches or exceeds tuned AdamW on language models up to 772M parameters, with theoretical guarantees for stationarity and long-horizon stability.