Tag
A user reflects on the complexity and fascination of running local LLMs, touching on hardware selection, quantization, and tensor parallelism.
A learner shares enthusiasm for Stanford CS336 lecture 7 on GPU parallelism, which covers fundamental operations and connects them to multi-GPU setups and parallelism techniques like tensor, data, and pipeline parallelism.
A fork of llama.cpp fixes the --split-mode tensor issue with quantized KV caches, achieving up to 40% speed improvement on dual GPU setups without quality loss.
At PyTorchCon Europe 2026, Edward Yang explains PyTorch's new pre-compilation support for distributed training and SPMD type system to help developers write correct tensor parallelism code, addressing common pitfalls in gradient correctness.
llama.cpp build b9095 introduces NCCL-free tensor parallelism for dual Blackwell PCIe GPUs, enabling efficient multi-GPU inference without relying on NCCL.