Tag
PyTorch announced a live Q&A webinar for the PyTorch 2.15 release, where Meta engineers will walk through new capabilities including the rewritten NCCL backend, NIXL one-sided transports, finer FSDP2/pipelining memory control, and faster NVFP4/FP8 decoding on NVIDIA Blackwell and Apple Silicon.
This article describes a method to train LoRA adapters using AsyncGRPOTrainer and sync them via Storage Buckets across separate Hugging Face Jobs, eliminating the need for NCCL communication.
NVIDIA shares debugging lessons from its Exemplar Cloud program, detailing how configuration issues in SMMU power management, NUMA placement, NCCL queue-pair concurrency, and hardware defects cause 8-12% training throughput gaps on AI clusters, and how to diagnose and fix them.