convergence-guarantees

OmniISR proposes a unified framework combining centralized and federated learning via intermediate supervision and regularization at hidden layers, offering theoretical convergence guarantees and reducing the CL–FL gap by 22.60%.

This paper studies nonconvex stochastic optimization under Blum-Gladyshev noise, where gradient variance grows with distance from initialization. It proves convergence guarantees for normalized SGD with momentum and a variance-reduced STORM method, achieving minimax optimal rates under certain conditions.

SMCEvolve introduces a principled framework for LLM-driven program evolution by reformulating it as sampling from a reward-tilted distribution using Sequential Monte Carlo. It provides convergence guarantees and outperforms existing methods across multiple scientific discovery benchmarks.