Tag
Blog post by Sankalp detailing how he used Codex to achieve a 232x faster GPU kernel for QR decomposition in GPU Mode's contest, outlining his auto-research methodology.
This paper proposes a reparametrization of the preconditioner in Shampoo-based optimization methods (like KL-Shampoo and SOAP) to support BFloat16 storage and reduce computational overhead by updating only part of the basis via QR decomposition in a subspace, making these methods more memory- and time-efficient.