Tag
Physicists have advanced fluid theory by developing a new approach based on symmetries, extending beyond the Navier-Stokes equations and incorporating microscopic particle behaviors.
This paper argues that the bulk of near-zero Hessian eigenvalues in neural networks arises from weakly broken symmetries of the network parametrization, showing that high-curvature directions are orthogonal to the symmetry subspace while the bulk lies within it.