For V100 Users: SGLang running Qwen+Dflash and Laguna

Reddit r/LocalLLaMA Tools

Summary

Modified SGLang to support Qwen and Laguna models on V100 GPUs using custom FlashAttention and Marlin kernels, achieving decent throughput on 4xV100 hardware.

Forked SGLang, wrote TeilLang FlashAttention for V100, used open-source marlin-v100, ungated flashinfer for sm70, made Dflash work for Qwen3.5/3.6 models, added Laguna S2.1 support, tried to make dflash work for Laguna(and no luck so far). ~4000-6000pp, ~100 tks tg(Qwen only). Running on my 4xV100 32GB NVLINK: https://preview.redd.it/wf4xcerbzqfh1.png?width=1356&format=png&auto=webp&s=3572c5d50a1813d63603b5424bc215631a6d253d https://preview.redd.it/vcb5d3tpzqfh1.png?width=1300&format=png&auto=webp&s=9c0e8b4fbf33265d2580792201b5eed5813d4e9d Repo: https://github.com/haohervchb/sglang-V100 Tilelang FA: https://github.com/haohervchb/Tilelang-FA-V100 Marlin-V100: https://github.com/zhinianqin/marlin_v100 There is a Docker image, so no building taking forever is needed.
Original Article

Similar Articles

Qwen3.8-Flash-Next NVFP4 Day-3 support for 4xV100

Reddit r/LocalLLaMA

RadixArk/Qwen3.8-Flash-Next-NVFP4 is now supported in SGLang-V100, enabling full context operation on 4xV100 GPUs with performance metrics showing high throughput and context handling up to 256k tokens.

366 t/s Qwen3.6 27B NVFP4 on v100s

Reddit r/LocalLLaMA

Introduces v100-skinny, a custom kernel library enabling fast NVFP4 inference on V100 (sm70) GPUs, achieving 366 t/s for Qwen3.6 27B in best-case extraction, with lower speeds for structured generation and code.