@PyTorch: PyTorch 2.12 introduces major updates across compilation, export, distributed training, and accelerator support. Highli…
Summary
PyTorch 2.12 release includes major updates to compilation, export, distributed training, and accelerator support, with up to 100x faster batched linalg.eigh on CUDA and new APIs like torch.accelerator.Graph.
Similar Articles
PyTorch 2.12 Release Highlights (7 minute read)
PyTorch 2.12 introduces significant performance improvements including up to 100x faster batched eigendecomposition on CUDA, a new device-agnostic torch.accelerator.Graph API, and support for Microscaling quantization in torch.export, continuing the framework's evolution into a unified production platform.
@PyTorch: PyTorch releases include thousands of changes, and our Release Live Q&A gives you direct access to the maintainers and …
PyTorch 2.12 release with major updates is highlighted, and a live Q&A event on May 20 with maintainers is announced to discuss compilation, distributed systems, quantization, and more.
@PyTorch: PyTorch 2.13 is here, with 3,328 commits from 526 contributors and updates across FlexAttention, CuTeDSL, nn.LinearCros…
PyTorch 2.13 is released with major updates including FlexAttention on Apple Silicon (up to 12x speedup), CuTeDSL backend, nn.LinearCrossEntropyLoss for 4x memory reduction, new torchcomms communications backend, FSDP2 communication overlap, Python 3.15 wheel support, and broader platform support. A live Q&A is scheduled for July 22.
PyTorch Distributed: Experiences on Accelerating Data Parallel Training
This paper details the design and optimization of PyTorch's distributed data parallel module, highlighting techniques like gradient bucketing and computation-communication overlap that enable near-linear scalability across 256 GPUs.
@PyTorch: vLLM and PyTorch worked together to fix a long-standing aarch64 install headache — as of PyTorch 2.11.0, pip install to…
PyTorch 2.11.0 now publishes CUDA-enabled aarch64 wheels to PyPI, fixing a long-standing installation issue for vLLM on NVIDIA Grace Hopper and Grace Blackwell systems, eliminating the need for custom index URLs and preventing silent CPU wheel replacements.