Tag
This paper proves lower bounds for smooth nonconvex zeroth-order optimization, showing that deterministic methods incur an extra factor of roughly d in query complexity compared to randomized methods under low curvature trace conditions, quantifying the price of determinism for tasks like LLM fine-tuning with function-value-only queries.
Qualcomm AI Research 展示了将 Combinatorial Multigrid 图粗化作为热启动与 Particle Polynomial Relaxation 非凸连续重构相结合，能在计算受限条件下更有效地逃离 MAX-CUT 的局部最优，在 Biq Mac 和 G-set 基准上取得接近最优的结果并大幅降低内存占用。
This paper proposes PROBE, a perturbed gradient algorithm for bilevel optimization with nonconvex lower levels, using second-order stationarity to achieve finite-time convergence and outperform state-of-the-art methods in experiments on LLM-based tasks and meta-learning.
This paper revisits median-of-means estimation from a deterministic optimization perspective and develops a family of block-Lp estimators for robust learning under heavy-tailed and adversarial corruption, showing that nonconvex methods can approach the trimmed oracle performance while remaining computationally tractable.
This paper establishes the first convergence guarantees for the plain vector-form Adam optimizer under heavy-tailed stochastic noise, showing it converges to stationary points but with suboptimal iteration complexity, and improves to optimal rates with a known domain radius.
This paper proposes an adaptive gradient descent method using one-sided Hölder regularity to control step sizes based on directional curvature rather than full gradient variation, providing convergence guarantees for nonconvex objectives and demonstrating empirical benefits.
This paper proves that online gradient descent achieves optimal √T regret for hidden-convex losses under a Hessian compatibility condition, resolving open questions in adversarial online learning. It also extends results to one-point bandit feedback with a T^{3/4} expected regret bound.
This paper proposes a robust subspace-constrained quadratic model for learning low-dimensional structures from high-dimensional data, accommodating heavy-tailed noise. A gradient-based algorithm with backtracking line search is developed, and experiments show improved robustness and reconstruction accuracy.
This paper studies nonconvex stochastic optimization under Blum-Gladyshev noise, where gradient variance grows with distance from initialization. It proves convergence guarantees for normalized SGD with momentum and a variance-reduced STORM method, achieving minimax optimal rates under certain conditions.
This paper introduces SHAPE, a structured adaptive port-Hamiltonian optimizer for fixed-budget nonconvex optimization that uses event-triggered mechanisms to balance descent, exploration, and budget allocation.