Tag
在 PyTorch Con 2026 上,Mark Saroufim 探讨了支撑更优优化器的线性代数内核,涵盖 QR、对称特征分解与 Cholesky,并从 GPU MODE 的线性代数竞赛中总结了如何通过暴露并行性、重叠独立阶段和合并更新来减少数据搬运。
A blog post detailing how the author used Codex to optimize a kernel in a GPU Mode contest, achieving a 232x speedup in QR decomposition and sharing learnings on auto-research.
Blog post by Sankalp detailing how he used Codex to achieve a 232x faster GPU kernel for QR decomposition in GPU Mode's contest, outlining his auto-research methodology.
This paper proposes a reparametrization of the preconditioner in Shampoo-based optimization methods (like KL-Shampoo and SOAP) to support BFloat16 storage and reduce computational overhead by updating only part of the basis via QR decomposition in a subspace, making these methods more memory- and time-efficient.