@levidiamode: Day 138/365 of GPU Programming One of my favorite lectures I've watched this year is Stanford's CS336 lecture 7 on GPU …
Summary
A learner shares enthusiasm for Stanford CS336 lecture 7 on GPU parallelism, which covers fundamental operations and connects them to multi-GPU setups and parallelism techniques like tensor, data, and pipeline parallelism.
View Cached Full Text
Cached at: 05/22/26, 05:56 PM
Day 138/365 of GPU Programming
One of my favorite lectures I’ve watched this year is Stanford’s CS336 lecture 7 on GPU parallelism.
It builds up the foundations by teaching the fundamental operations like broadcasting, scattering, gathering, reducing, all-gathering and all-to-all from first principles and gives you an intuition for how these connect to the multi GPU setup in part 1 of the lecture.
It then connects these principles to tensor, data and pipeline parallelism in part 2.
The reason why I love the teaching methodology of this particular lecture is that it goes into the practical details right away and hands you the code for what this conceptually would look like if you were to see it in a codebase.
levi (@levidiamode): Day 137/365 of GPU Programming
Haven’t done a review day in a while, so taking some time looking at my notes over the last few days and rereading various papers I am still wrapping my head around.
The last week has been really useful in gaining an initial understanding of
Similar Articles
@levidiamode: Day 248/365 of GPU Programming The Cerebras CTO has some other great talks online. For example, one from two years ago …
A social media post highlighting talks by Cerebras CTO on GPU programming and AI hardware architecture, noting their low view counts despite being informative.
@levidiamode: 183/365 of GPU Programming This 4.5 hour lesson on CUDA + ThunderKittens by @bfspector (TK co-author, Stanford PhD stud…
A highly recommended 4.5-hour GPU programming lesson on CUDA and ThunderKittens by Ben Spector, offering an in-depth, behind-the-scenes look at kernel optimization.
@levidiamode: 157/365 of GPU Programming Another FlashAttention4 resource that's been really helpful for me is the talk @charles_irl …
A daily GPU programming thread highlights a talk by Charles_irl that reverse-engineers FlashAttention4 code before the paper release, praising the Modal team's deep code dissection and inferences about the forward pass.
@vivekgalatage: GPU Programming Fundamentals https://youtu.be/Cl2B_hmg4gA William Brandon, a performance engineer at Anthropic, outline…
A summary of William Brandon's (performance engineer at Anthropic) GPU programming fundamentals lecture, emphasizing that understanding the streaming multiprocessor (SM) structure of GPU hardware is key to predicting performance, rather than starting solely from the software abstraction of thread blocks/threads.
@Hesamation: Stanford dropped their latest course on Parallel Programming, GPU, and CUDA. 24 hours, 19 lessons. this is one of the h…
Stanford has released a free course on Parallel Programming, GPU, and CUDA, consisting of 24 hours of content across 19 lessons, covering GPU architecture, CUDA, and performance optimization.