Tag
PyTorch's Helion kernel DSL is integrated into vLLM's linear backend, where a single autotuned GEMM implementation covers Standard GEMM, Split-K, and Swap-AB variants, outperforming CUTLASS and DeepGEMM backends on NVIDIA Hopper GPUs with over 10% throughput gains on some workloads.
The PyTorch team discusses the concept of PyTorch serving as both a reference language and an implementation language for deep learning, emphasizing its role in verifying optimized kernel implementations and the potential for LLMs to generate explicit forward-backward code.