@Hikari_07_jp: Everyone, read this article! Jetha Chan is my ideal engineer. He disassembled the Attention kernel for the SM100 (data …
Summary
Jetha Chan disassembled the Attention kernel for the SM100 data center GPU, identified and fixed inefficiencies, resulting in significant performance improvements.
Similar Articles
@ekzhang1: me looking at people like this guy who write real gpu kernels :)
AI model Claude was used to write a FlashAttention forward kernel using the pyptx DSL, achieving near-parity performance with hand-tuned FlashAttention-4 on NVIDIA B200 hardware.
@damnang2: This article is fully open to everyone. In this piece, I cover two things. First, for readers without a technical backg…
An article explains how NVIDIA rack names are structured for non-technical readers, then analyzes SemiAnalysis's recent post to clarify implications about midplane yields.
@h100envy: CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $12…
A CMU PhD who developed the kernels now used by NVIDIA in TensorRT-LLM explains fast attention, covering fused CUDA kernels, FlashInfer, Triton, and paged-KV attention, enabling more tokens per second on the same GPU.
@hamzaelshafie: New in-depth blog post: "Dissecting ThunderKittens: Anatomy of a Compact DSL for High-Performance AI Kernels" This post…
A detailed blog post dissecting ThunderKittens, a compact DSL for high-performance AI kernels, including a bottom-up analysis of its abstractions and a benchmark implementing a non-causal attention prefill kernel that outperforms FlashAttention-2 by ~1.55x and matches FlashAttention-3.
@AMD: From bring-up to tuning, AMD and @OpenAI engineers are sharing insights to push performance further. Go behind the coll…
AMD and OpenAI engineers collaborate to share insights on performance optimization, featuring a behind-the-scenes look with OpenAI's VP of Compute Strategy, Sachin Katti.