Tag
该论文确定了在重尾噪声（有界中心 p 阶矩，1<p≤2）下，平滑凸目标的裁剪 SGD 最后一次迭代的时间一致（time-uniform）最优收敛速率，证明了无需已知时间跨度的调度可达 O~(t^{-2(p-1)/(3p-1)})，且不存在更优的规则，回答了未知停止时间对收敛的代价问题。
This paper proposes robust, adaptive Langevin Monte Carlo methods (R-LMC, A-LMC, RA-LMC) that handle generalized smoothness and state-dependent heavy-tailed noise, establishing Fisher information convergence guarantees without dissipativity assumptions, and demonstrates effective Bayesian fine-tuning of LLMs and vision-language models.
论文研究了在未知重尾噪声统计下在线凸优化的区间动态遗憾界，提出一个不依赖 G、σ、p、区间长度等参数的学习器，并证明了包含噪声幂指数的通用遗憾上界及匹配的下界。
The paper analyzes why clipping is crucial in AdaGrad under generalized smoothness and heavy-tailed noise, proving it prevents anisotropic miscalibration and provides high-probability convergence guarantees.
This paper establishes the first convergence guarantees for the plain vector-form Adam optimizer under heavy-tailed stochastic noise, showing it converges to stationary points but with suboptimal iteration complexity, and improves to optimal rates with a known domain radius.
This paper provides the first comprehensive convergence analysis of vanilla SGD with momentum under heavy-tailed noise without gradient clipping or normalization, revealing inferior rates compared to clipped variants and supported by experiments on synthetic functions.