Tag
This paper shows that drawing fresh sketches at every iteration of under-constrained iterative ridge regression yields provably better convergence than reusing a single sketch, deriving residual-aware sampling rules and an oracle sampling distribution, with experiments including ridge probes on Qwen2.5 representations supporting the theory.
The paper proposes Finite-Library Input-Warped Bayesian Optimization (FLIWBO), which selects input warps from a finite library to adapt input geometry for improved optimization efficiency while maintaining high-probability convergence guarantees, outperforming standard GP-UCB in various benchmarks.
This paper establishes convergence guarantees for gradient descent on general feedforward neural networks of arbitrary width/depth, using a novel generalized Lipschitz smoothness condition that holds for common activations and mean-squared error, without special initialization or dataset requirements.
This paper establishes the first convergence guarantees for the plain vector-form Adam optimizer under heavy-tailed stochastic noise, showing it converges to stationary points but with suboptimal iteration complexity, and improves to optimal rates with a known domain radius.
OmniISR proposes a unified framework combining centralized and federated learning via intermediate supervision and regularization at hidden layers, offering theoretical convergence guarantees and reducing the CL–FL gap by 22.60%.
This paper studies nonconvex stochastic optimization under Blum-Gladyshev noise, where gradient variance grows with distance from initialization. It proves convergence guarantees for normalized SGD with momentum and a variance-reduced STORM method, achieving minimax optimal rates under certain conditions.
SMCEvolve introduces a principled framework for LLM-driven program evolution by reformulating it as sampling from a reward-tilted distribution using Sequential Monte Carlo. It provides convergence guarantees and outperforms existing methods across multiple scientific discovery benchmarks.