@derangineer: the goats in the game
Summary
Charles Frye announces a blog post detailing contributions to FA4 internals, focusing on inference performance improvements that have been upstreamed.
View Cached Full Text
Cached at: 06/12/26, 06:54 AM
the goats in the game
Charles 🎉 Frye (@charles_irl): Last fall, we shared our deep dive on FA4 internals.
But we didn’t stop at grokking the kernel.
Since then, we’ve been developing improvements for inference performance and upstreaming them.
This blog post explains those contributions.
Similar Articles
@charles_irl: Last fall, we shared our deep dive on FA4 internals. But we didn't stop at grokking the kernel. Since then, we've been …
A blog post details contributions to FlashAttention-4 to improve its performance for large language model inference, especially for decode-heavy workloads, by adjusting parallelism strategies and supporting irregular memory accesses.
@charles_irl: On Friday, we released six new state-of-the-art drafters for accelerated inference. We also put out a blog post on why …
On Friday, we released six new state-of-the-art drafters for accelerated inference, along with a blog post on speculative decoding and a roofline model tool to estimate speedups.
@satvikgari: Wake up the goat posted
A progress update on reinventing Groq's LPU, with a redesigned vector execution module to better support overlap operations and self-attention.
@charles_irl: Rewriting parallelism is a big move and it'd be nice to make it even faster than we can do with CuTe DSL. FA4 is a very…
Discussion about rewriting parallelism to improve kernel performance using CuTe DSL and tile programming models for the FA4 (FlashAttention 4) kernel.
@googlegemma: “Agentic kernel optimization is the future of on-device inference” @xenovacom used Fable 5 to write kernels that pushed…
Xenova used Fable 5 to write optimized kernels achieving 255 tokens per second for Gemma 4 on WebGPU with M4, demonstrating agentic kernel optimization for on-device inference.