heavy-tailed-noise

This paper provides the first comprehensive convergence analysis of vanilla SGD with momentum under heavy-tailed noise without gradient clipping or normalization, revealing inferior rates compared to clipped variants and supported by experiments on synthetic functions.