Tag
A technique to accelerate matrix multiplication when M and N are small but K is large, as encountered in MoE routers and small-batch decoding, by decomposing K and running partial GEMMs in parallel. The approach beats PyTorch Inductor on most shapes using a custom Triton kernel.
Flash-GMM introduces a fused Triton kernel for Gaussian Mixture Models that achieves 20x speedup and enables training on datasets 100x larger on a single GPU, making soft clustering a viable drop-in replacement for k-means in approximate nearest neighbor search.
This paper introduces Dynamic Ultrametric Attention, a framework where Transformers learn per-head block-sparse routing topologies during training, which are then offloaded to a custom Triton block-sparse kernel at inference time, achieving up to 28x speedup and 98.4% memory reduction over dense attention.