Tag
This paper introduces Weaver, a lightweight autoregressive adapter that constructs proposal trees from factorized drafters' top-K marginals, achieving a 4.37x speedup over autoregressive decoding and outperforming the DFlash baseline by 24.7%.
This paper introduces TwELL and Hybrid sparse formats with custom CUDA kernels to efficiently leverage unstructured sparsity in LLMs, achieving over 20% faster training and inference on H100 GPUs while reducing energy and memory usage.
Atlas is a pure Rust LLM inference engine that delivers faster inference than vLLM and TensorRT-LLM by customizing CUDA kernels for each hardware × model × quantization combination.