layer-wise-updates

Introduces Depth-wise Gradient Augmentation, a general optimization paradigm that transforms block-wise optimizer updates along depth dimension. The method, Gradient Smoothing, improves optimization and generalization across diverse architectures including transformers and diffusion models.