@pavelsimo: i've solved all the attention problems on LeetGPU. putting them all in one place so they are easy to find: 1/13
Summary
A user announces they have solved all attention-related problems on LeetGPU and compiled them into a resource for easy access.
View Cached Full Text
Cached at: 09/07/26, 07:12 PM
i’ve solved all the attention problems on LeetGPU.
putting them all in one place so they are easy to find:
1/13
2/13 Softmax Attention
3/13 Attention with Linear Biases
4/13 Grouped Query Attention
5/13 Decaying Causal Attention
6/13 INT8 KV-Cache Attention
7/13 Softmax Attention Backward
8/13 Attention with Sinks
9/13 Multi-Head Attention
10/13 Multi-Head Cross-Attention
11/13 Causal Self-Attention
12/13 Linear Self-Attention
13/13 Sliding Window Self-Attention
thx my brother 😎
Similar Articles
@levidiamode: 157/365 of GPU Programming Another FlashAttention4 resource that's been really helpful for me is the talk @charles_irl …
A daily GPU programming thread highlights a talk by Charles_irl that reverse-engineers FlashAttention4 code before the paper release, praising the Modal team's deep code dissection and inferences about the forward pass.
@levidiamode: 158/365 of GPU Programming I think I understand the high level differences between the FlashAttention 2, 3 and 4 forwar…
The author documents their progress in learning GPU programming, focusing on understanding the high-level differences between FlashAttention 2, 3, and 4 forward passes, and lists several low-level concepts they need to explore further.
@pauliusztin_: I just found one of the most useful resources for understanding GPUs. No more jumping between random docs, PDFs, and fo…
Modal Labs has released an open-source, interlinked GPU glossary that consolidates fragmented NVIDIA documentation, CUDA details, and compiler flags into a single navigable resource for engineers optimizing LLM training and inference.
@Alacritic_Super: If you are serious about LLM inference, study FlashAttention. It's one of the most important optimizations behind moder…
A tweet recommending studying FlashAttention for LLM inference, highlighting its importance in optimizing GPU memory traffic and speeding up attention mechanisms, with links to the GitHub repository and papers for FlashAttention, FlashAttention-2, and FlashAttention-3.
@ekzhang1: me looking at people like this guy who write real gpu kernels :)
AI model Claude was used to write a FlashAttention forward kernel using the pyptx DSL, achieving near-parity performance with hand-tuned FlashAttention-4 on NVIDIA B200 hardware.