Tag
A hand-written GEMM kernel for NVIDIA Blackwell (B200), built with CUTLASS's CuTeDSL from raw primitives (tcgen05, TMA, mbarriers, TMEM), goes from 164 to 1401 TFLOP/s — about 97% of cuBLAS — with one key change doing most of the work and prefill vs decode workloads yielding different optimizations.
Discussion of the shift in GPU kernel engineering from C++ CuTe/CUTLASS to NVIDIA's Python-based CuTeDSL, questioning whether new engineers should learn legacy C++ templates or prioritize the emerging stack for LLM inference work.