Tag
The Hundred-Page Language Models Book by Andriy Burkov offers a fast, code-first approach to building Transformer-based LLMs, with hands-on PyTorch examples, targeting readers comfortable with Python and math.
A tweet promoting a book containing 50 hands-on projects covering LLM topics like tokenization, embeddings, and attention, using Python and PyTorch.
Recommend 'The Little Book of Reinforcement Learning' as an introductory textbook for reinforcement learning. It's short but covers fundamentals to algorithms, and comes with PyTorch implementations and supplementary derivations, suitable for understanding core RL concepts.
The PyTorch Foundation is accepting nominations for the 2026 Contributor Awards until July 17, 2026, recognizing outstanding contributions across projects like PyTorch, vLLM, DeepSpeed, Ray, Helion, and Safetensors.
The Little Book of Reinforcement Learning, a concise introduction to RL with PyTorch implementations and supplementary material, has been shared on GitHub by Alexandre TL.
This blog post from PyTorch presents novel kernel fusion techniques for normalization ops like LayerNorm and RMSNorm, achieving significant speedups by reducing memory-IO overhead. Techniques include Lazy Pre-Norm and Multi-CTA Norm Fusion, hiding up to 90% of normalization latency when fused with GEMMs, and the FlashNormAttention algorithm achieving up to 35% kernel speedup.
This tutorial demonstrates how to profile various attention implementations in PyTorch, from naive attention to scaled dot-product attention with different backends, using the PyTorch profiler on an NVIDIA A100 GPU.
TorchCode is an open-source coding platform that turns manual deep learning operator implementation into a LeetCode-style experience. It includes 40 high-frequency interview questions, provides automated evaluation and hints, and supports one-click Docker deployment and online use via Hugging Face Spaces.
PyTorch 2.13 is released with major updates including FlexAttention on Apple Silicon (up to 12x speedup), CuTeDSL backend, nn.LinearCrossEntropyLoss for 4x memory reduction, new torchcomms communications backend, FSDP2 communication overlap, Python 3.15 wheel support, and broader platform support. A live Q&A is scheduled for July 22.
A tweet recommending The Annotated Transformer as a resource for software engineers to understand the foundational concepts of the Attention Is All You Need paper by building it in PyTorch.
NVIDIA BioNeMo Recipes provide step-by-step, parameter-efficient fine-tuning recipes for biological foundation models using LoRA, demonstrated on two case studies with ESM2-3B and Evo2-1B on a single workstation GPU.
AMD and Meta contributors ported PyTorch Monarch to AMD Instinct GPUs with ROCm, enabling fault-tolerant distributed training at scale. The blog details the engineering work and validation on large clusters.
TorchJD is a library for training models with multiple losses in PyTorch, implementing both scalarization and Jacobian descent methods. It has been accepted into the PyTorch ecosystem and aims to become the go-to library for multi-loss training.
PyTorch Foundation's expanding role in European AI sovereignty is highlighted, with projects like vLLM, DeepSpeed, Ray, and SafeTensors coming under its neutral governance.
This article provides an overview of the PyTorch ecosystem, highlighting libraries and tools such as PyTorch, LibTorch, ExecuTorch, TorchServe, and others for various AI/ML tasks.
PyTorch explains how its tests are dynamically generated at import time, affecting CI failure names, and offers a blog to help contributors understand the testing workflow.
OpenMythos is an open-source project that theoretically reproduces the Claude Mythos architecture, fully implementing the Recurrent-Depth Transformer (RDT), supporting MLA/GQA attention mechanisms and sparse MoE, providing preset configurations from 1B to 1T parameters, and is installable via pip.
The PyTorch Foundation supported the ExecuTorch Hackathon in San Francisco, where over 100 participants built real-time on-device AI applications using PyTorch and ExecuTorch on Snapdragon-powered Samsung Galaxy S25 Ultra devices. Winning projects included SafeScreen AI, SixthSense, and Toddle AI, showcasing local execution benefits for responsiveness, privacy, and offline capability.
A concise one-page cheat sheet covering PyTorch tensors, models, matrix multiplication, and autograd for deep learning training pipelines.
PyTorchCon China is co-located with KubeCon, CloudNativeCon, and OpenInfra Summit in Shanghai from September 7-9, 2026, featuring technical sessions and community collaboration. Registration discounts are available until July 28.