@levidiamode: 183/365 of GPU Programming This 4.5 hour lesson on CUDA + ThunderKittens by @bfspector (TK co-author, Stanford PhD stud…

X AI KOLs Timeline Tools
gpu-programming cuda thunderkittens educational kernels tutorial low-latency

Summary

A highly recommended 4.5-hour GPU programming lesson on CUDA and ThunderKittens by Ben Spector, offering an in-depth, behind-the-scenes look at kernel optimization.

183/365 of GPU Programming This 4.5 hour lesson on CUDA + ThunderKittens by @bfspector (TK co-author, Stanford PhD student) is one of the best educational videos on kernels out there (think @karpathy level video but solely focused on GPU stuff). It's so criminally underrated, I cannot believe this video only has 5k views. It's the highest density GPU programming lecture I've come across in my ~6 months of learning. For better or for worse, Ben talks (and thinks) at the speed of light. But thanks to @qamcintyre it's more of a dialogue between Ben who introduces concepts/ideas based on his experience creating TK and Quinn who intervenes from time to time with pointed questions that help ground the discussion in fundamentals. Regardless of whether you're just starting out with writing kernels or have some experience, it's an incredible resource because it gives you a behind-the scenes-look of someone who's spent years thinking about how to maximize what you can get out of the hardware (read his Hazy blog post "We Bought the Whole GPU, So We're Damn Well Going to Use the Whole GPU" for context). Even though I was sort of familiar with the topics of the lesson (e.g. GPU architecture and memory consistency), I still came away with pages of notes and questions around TMA accelerated pipelining, coscheduling instructions, pre vs post Hopper, levels of virtualization, cache lines, barriers, TK primitives, et cetera. As @suryaasub so aptly put it in the YT comments: "greatest video of all time".
Original Article
View Cached Full Text

Cached at: 07/06/26, 06:06 AM

183/365 of GPU Programming

This 4.5 hour lesson on CUDA + ThunderKittens by @bfspector (TK co-author, Stanford PhD student) is one of the best educational videos on kernels out there (think @karpathy level video but solely focused on GPU stuff). It’s so criminally underrated, I cannot believe this video only has 5k views.

It’s the highest density GPU programming lecture I’ve come across in my ~6 months of learning. For better or for worse, Ben talks (and thinks) at the speed of light. But thanks to @qamcintyre it’s more of a dialogue between Ben who introduces concepts/ideas based on his experience creating TK and Quinn who intervenes from time to time with pointed questions that help ground the discussion in fundamentals.

Regardless of whether you’re just starting out with writing kernels or have some experience, it’s an incredible resource because it gives you a behind-the scenes-look of someone who’s spent years thinking about how to maximize what you can get out of the hardware (read his Hazy blog post “We Bought the Whole GPU, So We’re Damn Well Going to Use the Whole GPU” for context). Even though I was sort of familiar with the topics of the lesson (e.g. GPU architecture and memory consistency), I still came away with pages of notes and questions around TMA accelerated pipelining, coscheduling instructions, pre vs post Hopper, levels of virtualization, cache lines, barriers, TK primitives, et cetera. As @suryaasub so aptly put it in the YT comments: “greatest video of all time”.

levi (@levidiamode): 182/365 of GPU Programming

Preparing my first submissions to the eigendecomp leaderboard today, and gotta say this GPU Mode challenge feels different already. The speed at which competitors have cleared sub 20ms and now 10ms even though we’re still more than a week out is quite

Similar Articles