@h100envy: PyTorch core engineer at Meta turned CUDA kernel writing into a sport in 13 minutes - better than $1500 GPU programming…
Summary
A PyTorch core engineer at Meta demonstrated a fast CUDA kernel optimization loop that outperforms expensive bootcamps, with the winning code merged into PyTorch via the KernelBot competition.
View Cached Full Text
Cached at: 06/30/26, 11:48 PM
PyTorch core engineer at Meta turned CUDA kernel writing into a sport in 13 minutes - better than $1500 GPU programming bootcamps.
profile the kernel -> find the bottleneck -> rewrite -> benchmark -> merge the winning code into PyTorch.
That loop is how the open community now beats hand-tuned vendor kernels.
GPU MODE community + KernelBot competition + winning kernel merged into the framework - that’s the stack.
Watch it, then steal the loop below.
Similar Articles
@PyTorch: Optimizing kernels can be time-consuming and require deep expert knowledge. KernelAgent is a multi-agent harness that c…
Kaiming Cheng of Meta will present KernelAgent, a hardware-guided multi-agent harness that optimizes GPU Triton kernels, at PyTorch Conference North America 2026; it achieved a 2.02x speedup over prior versions and 1.56x over default torch.compile across 100 KernelBench L1 tasks.
@PyTorch: Engineering teams from across the ecosystem are presenting major breakthroughs in torch.compile, custom kernel authorin…
The PyTorch Conference North America features engineering teams presenting major breakthroughs in torch.compile, custom kernel authoring, and scaling efficiency, with sessions led by experts from various organizations.
@PyTorch: PyTorch member Meta just open-sourced a GPU kernel that makes attention 2.3x faster on NVIDIA Blackwell. TLX Block Atte…
Meta open-sources TLX Block Attention, a warp-specialized Triton kernel that achieves 2.3x speedup for block-diagonal self-attention on NVIDIA Blackwell GPUs, with up to 3.5x speedup when fused with rotary embeddings.
Fast by Default: Compile-Time Heuristics for Helion Kernels (15 minute read)
PyTorch's Helion DSL now uses compile-time heuristics that analyze a kernel's IR and dataflow to pick fast block-size, warp, and stage configurations without slow autotuning, achieving near-autotuned speed — including 1.18x faster than hand-written Triton in Flash Linear Attention and 1.27x faster than vLLM's handwritten CUDA kernels on H100.
A hackable compiler to generate efficient fused GPU kernels for AI models [P]
The author presents a custom, hackable ML compiler written in Python that lowers LLMs to optimized CUDA kernels through a multi-stage IR pipeline, achieving performance competitive with or superior to PyTorch on specific operations. The article details the compiler's optimization passes, lowering rules, and CLI usage for generating efficient fused GPU kernels.