@charles_irl: ^That's a sample of CuTe DSL, which is used in, among others, the FlashAttention-4 kernel. Below is the sample CuTe ker…
Summary
A tweet showcasing a CuTe DSL kernel sample that uses layouts to express transposition, part of the FlashAttention-4 kernel.
Similar Articles
@charles_irl: The CuTe and CuTe DSL articles include minimal code snippets illustrating core principles and basic usage. These snippe…
The CuTe and CuTe DSL articles provide minimal code snippets with Modal Notebooks for hands-on learning.
@charles_irl: Rewriting parallelism is a big move and it'd be nice to make it even faster than we can do with CuTe DSL. FA4 is a very…
Discussion about rewriting parallelism to improve kernel performance using CuTe DSL and tile programming models for the FA4 (FlashAttention 4) kernel.
@charles_irl: New articles in the GPU Glossary for CuTe DSL, CUTLASS, and CuTe -- the tools used to write some of the highest-perform…
New articles in the GPU Glossary cover CuTe DSL, CUTLASS, and CuTe – tools for writing high-performance GPU kernels on data center GPUs, with examples in Python.
@PatrickToulme: This exercise makes me believe the future of DSLs and compilers is very much agentic. Programming languages and DSLs bu…
Claude Fable used the pyptx DSL to write a FlashAttention forward kernel for NVIDIA B200 that achieves near-parity performance with the hand-tuned CUTLASS kernel, demonstrating the potential for AI agents in compiler and DSL design.
@ekzhang1: me looking at people like this guy who write real gpu kernels :)
AI model Claude was used to write a FlashAttention forward kernel using the pyptx DSL, achieving near-parity performance with hand-tuned FlashAttention-4 on NVIDIA B200 hardware.