@ariG23498: It is profiling time! In Part 2 we cover: > trace a Linear layer > talk about mul + add vs linear > gemm epilogues (my …
Summary
Announces Part 2 of a profiling tutorial covering linear layer tracing, gemm epilogues, MLP tracing, and comparisons of torch compile vs Liger kernels, with a link to the full content.
View Cached Full Text
Cached at: 06/12/26, 06:56 AM
It is profiling time! ⏰
In Part 2 we cover: > trace a Linear layer > talk about mul + add vs linear > gemm epilogues (my favourite part) > trace MLP > torch compile vs liger kernels > use hugging face kernels
Bookmark this for a fun weekend read! 😎 https://t.co/ZRqcO3AF1K
Similar Articles
Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP
This blog post continues the profiling in PyTorch series, exploring nn.Linear, MLP blocks, and fusion techniques using Triton kernels to optimize performance.
Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler
A beginner-friendly guide to using PyTorch's torch.profiler for profiling and optimizing neural network operations, starting with matrix multiplication and bias addition. It explains how to read profiler traces and understand CPU/GPU interactions.
@RisingSayak: I realized that what I cannot profile, I cannot optimize. This is why I embarked on a little project in Diffusers, to t…
Sayak Paul describes a project to profile and optimize Diffusers pipelines using torch.compile, and announces a tutorial series by Ari G. on the topic.
@rasbt: A little talk on what we can learn from implementing LLM architectures from scratch in Python and PyTorch. And how I ap…
Sebastian Raschka discusses the value of implementing LLM architectures from scratch in Python/PyTorch, sharing his workflow for understanding new open-weight models by dissecting configs, coding, and layer-by-layer debugging.
I wrote a free 15-part series on LLM internals — real math, real tensor shapes, real hardware constraints. All grounded in Gemma 4 12B's actual config.
A comprehensive 15-part series covering LLM internals from tokenization to serving, grounded in Gemma 4 12B's actual config.