Tag
This paper establishes the first convergence guarantees for the plain vector-form Adam optimizer under heavy-tailed stochastic noise, showing it converges to stationary points but with suboptimal iteration complexity, and improves to optimal rates with a known domain radius.
OmniISR proposes a unified framework combining centralized and federated learning via intermediate supervision and regularization at hidden layers, offering theoretical convergence guarantees and reducing the CL–FL gap by 22.60%.
This paper studies nonconvex stochastic optimization under Blum-Gladyshev noise, where gradient variance grows with distance from initialization. It proves convergence guarantees for normalized SGD with momentum and a variance-reduced STORM method, achieving minimax optimal rates under certain conditions.
SMCEvolve introduces a principled framework for LLM-driven program evolution by reformulating it as sampling from a reward-tilted distribution using Sequential Monte Carlo. It provides convergence guarantees and outperforms existing methods across multiple scientific discovery benchmarks.