@reprompting: Naive CUDA softmax using shared memory reduction. Reduction seems to be a pretty straightforward concept.
Summary
A tweet sharing a naive CUDA softmax implementation using shared memory reduction, noting that reduction is straightforward.
View Cached Full Text
Cached at: 06/17/26, 06:02 PM
Naive CUDA softmax using shared memory reduction. Reduction seems to be a pretty straightforward concept. https://t.co/rJgj1omslJ
Similar Articles
CUDA Shared Memory Swizzling
The article explains CUDA shared memory swizzling techniques to optimize GPU memory access patterns, with code examples demonstrating performance improvements.
SignMuon: Communication-Efficient Distributed Muon Optimization
SignMuon is a 1-bit, matrix-aware optimizer for distributed training that combines signSGD's majority-vote sign aggregation with Muon's polar-step framework, achieving 32x bandwidth reduction over float32 while maintaining strong convergence and performance on benchmarks like CIFAR-10/ResNet-50 and nanoGPT.
@akshay_pachaar: https://x.com/akshay_pachaar/status/2087928032904523980
An educational thread explaining how GPUs work, focusing on the memory-compute asymmetry that dominates LLM serving performance, and demonstrating how techniques like quantization, speculative decoding, and continuous batching follow from that fundamental constraint.
@hardmaru: The human brain is incredibly efficient because it only activates the specific neurons needed for a thought. Modern LLM…
This paper introduces TwELL and Hybrid sparse formats with custom CUDA kernels to efficiently leverage unstructured sparsity in LLMs, achieving over 20% faster training and inference on H100 GPUs while reducing energy and memory usage.
Reflecting to optimise
A blog post discussing optimization techniques for constrained categorical probability distributions, using softmax reparameterization and log barrier methods, applied to protein binder design.