Tag
Tests NVLink on dual RTX 3090s for AI inference and training, finding significant speedups for tensor parallel prompt processing (30%) and FSDP training (3x), but minimal effect on token generation or layer split inference.
Explains the communication model for multi-GPU systems, covering the trade-off between latency and bandwidth, and compares MST and Ring algorithms for collective operations like broadcast.
A correction clarifies that the RTX Spark does not have 600GB/s bandwidth; that figure is actually the NvLink speed, as shown in Computex slides.
A user details their modding and benchmarking of an AMD Strix Halo system with dual RTX 3090 eGPUs and NVLink, finding improvements in LLM inference speed for dense models, especially with vLLM, and discusses power efficiency trade-offs.
A benchmark analysis of Qwen 3.6 27B MTP on 4x RTX 3090 GPUs, demonstrating that using NVLink for tensor parallelism yields significant throughput improvements (up to +53%) over PCIe configurations.