gemm

A systematic exploration of FP32 matrix multiplication optimization on AMD Zen 3, achieving 85.30 GFLOPS (63.5% of theoretical peak) using AVX2/FMA intrinsics, surpassing naive implementation by 56.5x and matching optimized libraries.

This blog post from PyTorch presents novel kernel fusion techniques for normalization ops like LayerNorm and RMSNorm, achieving significant speedups by reducing memory-IO overhead. Techniques include Lazy Pre-Norm and Multi-CTA Norm Fusion, hiding up to 90% of normalization latency when fused with GEMMs, and the FlashNormAttention algorithm achieving up to 35% kernel speedup.

Meta introduces techniques like Lazy Pre-Norm, Multi-CTA Norm Fusion, and FlashNormAttention to fuse normalization operations with GEMM and Attention kernels, hiding up to 90% of normalization latency on NVIDIA B200 hardware and achieving up to 35% latency reduction in attention blocks.

SpenseGPT introduces a practical one-shot pruning method for LLMs that enables both sparse and dense GEMMs during inference, improving efficiency.

Announces Part 2 of a profiling tutorial covering linear layer tracing, gemm epilogues, MLP tracing, and comparisons of torch compile vs Liger kernels, with a link to the full content.

RT-Lynx proposes using activation sparsity instead of weight sparsity to accelerate diffusion models, achieving up to 1.55× linear-layer speedup while maintaining generation quality, and is accepted at ICML 2026.

Introduces CODA, a GPU kernel abstraction that expresses Transformer operations as GEMM-plus-epilogue programs to reduce data movement, covering nearly all non-attention computation in a Transformer block.

CODA introduces a GPU kernel abstraction that rewrites transformer computations as GEMM-plus-epilogue programs, reducing memory-bound operations and improving efficiency in training.