@kazukifujii: Tech Blog Release Day5 This is the first installment of a blog series that explains CUDA Programming from the basics, w…
Summary
Kazuki Fujii announces the first installment of a blog series on CUDA Programming basics, written in an accessible way, essential for understanding FlashAttention and hardware-aware acceleration techniques.
View Cached Full Text
Cached at: 06/05/26, 01:15 PM
Tech Blog Release Day 5
This is the first installment of a blog series that explains CUDA Programming from the basics, which is essential for understanding or proposing FlashAttention or recent hardware-aware acceleration techniques. It’s a blog exceeding 30,000 characters, but I’ve written it in a very accessible way, so please take a look.
CUDA Programming Guide Part 1 | Kazuki Fujii
Kazuki Fujii (@kazukifujii):
Tech Blog Release Day 4.I gently explain how vLLM implements the essential weight sync function in the RLVR (reinforcement learning) era.
Inference Framework in the RLVR Era: Weight Syncing Edition | Kazuki Fujii
Similar Articles
@SzymonOzog_: Saturday reading: "What happens when you run a CUDA kernel" - very cool blogpost on the details about the CPU<->GPU com…
Tweet recommending a blog post that explains the CPU-GPU communication details required when launching a CUDA kernel.
@maxxfuu: Day 6/90 of Inference Engineering I wrote a CUDA kernel for 1D Convolution, just getting the reps in for writing unopti…
A developer shares their day 6 of inference engineering, writing a CUDA kernel for 1D convolution, explaining PagedAttention's memory efficiency, and providing an overview of GPU memory hierarchy (global, register, local, constant, shared).
@goyal__pramod: Software is evolving, so should you! These are the best blogs I read to understand GPUs and CUDA!
Tweet recommending a collection of blogs to understand GPUs and CUDA, encouraging developers to improve their skills.
@v0xium: If you are looking for an article going in details regarding basics of CUDA, please spend an hour reading this. Link to…
An updated beginner-friendly tutorial on CUDA programming, covering how to write a simple kernel to add arrays on the GPU.
@levidiamode: 158/365 of GPU Programming I think I understand the high level differences between the FlashAttention 2, 3 and 4 forwar…
The author documents their progress in learning GPU programming, focusing on understanding the high-level differences between FlashAttention 2, 3, and 4 forward passes, and lists several low-level concepts they need to explore further.