Make Volta Fast Again
Summary
The article introduces 1Cat-vLLM, a fork of vLLM optimized for NVIDIA V100 GPUs, and compares its performance with llama.cpp for serving large language models like Qwen3.6-35b.
Similar Articles
Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA
Tiny-vLLM is a high-performance LLM inference engine implemented in C++ and CUDA, offering features like continuous batching and PagedAttention, and serves as an educational resource.
vLLM v0.28.0
vLLM v0.28.0 is an updated release of the open-source library for fast and efficient LLM inference and serving, featuring enhancements like PagedAttention and broad hardware support.
@no_stp_on_snek: while everyone is talking about @SpaceXAI , @AnthropicAI , and @OpenAI updates (but where @GoogleAI?)... went and teste…
A detailed comparison of Unsloth's NVFP4 quantized model inference performance between vLLM and llama.cpp, highlighting prefill speed advantages for vLLM but decode and caching advantages for llama.cpp in single-stream agent workloads.
Qwen3.6 27B more dumb in vLLM compared to llama.cpp
A user reports that the Qwen3.6-27B model performs better and more reliably with llama.cpp than with vLLM, citing tool call errors and 'lobotomized' behavior in vLLM despite extensive configuration.
If you have a 3090, or other 30xx for local LLMs, I have something for you
The article presents a custom fork of llama.cpp optimized for NVIDIA Ampere architecture GPUs like the 3090, achieving over 90 tokens per second for 27B parameter models with large context windows, making local inference faster than API speeds.