Question: Why is prefill unbelievably faster in vLLM than other inference engines?

Reddit r/LocalLLaMA 09/01/26, 05:05 PM Tools

vllm inference-engine performance-benchmarking pre-fill ai-models gpu-acceleration llama-cpp

Summary

The user shares benchmark results showing vLLM's significantly faster prefill performance compared to llama.cpp and other engines, and questions the technical reasons behind this speed difference.

I only started using some vLLM forks recently in a 4 x 48GB 4090 system. DS4F - ~5000pp/180tg (DSpark) Qwen3.8 Flash next - ~7500pp/135tg (MTP) This is amazing, like having the API in my house. But it's also really hard to go back. It's weird that we never come close to prefill numbers like this in llama.cpp or ik_llama. The narrative is that vLLM is around the same speed for single requests, but that is clearly not true. There must some HUGE difference that constitutes an insurmountable obstacle to achieving such speeds in llama.cpp and many other inference engines. Does anyone know exactly what it is? edit: These results are from my benchmark script that actually times the response, not the vLLM log. And they are not cache hits. My benchmark script deliberately busts cache. Actual cache hits, which I also measure, are like 20k-100k+.

Original Article

Similar Articles

Is using vLLM actually worth it if you aren't serving the model to other people?

Reddit r/LocalLLaMA

A user discusses the trade-offs between using vLLM and llama.cpp for local, single-user inference on AMD hardware, questioning if vLLM's performance benefits justify the complexity in non-enterprise settings.

@_avichawla: Prefill & decode in LLM inference. Have you ever noticed that the first token from an LLM always takes a moment to appe…

X AI KOLs Timeline

Explains the two phases of LLM inference - prefill and decode - detailing how GPU bottlenecks shift from compute-bound during prefill to memory-bound during decode, and the importance of KV caching.

Prefill vs. decoding and local LLM ROI: is prefill underrated?

Reddit r/LocalLLaMA

An analysis comparing prefill and decoding phases in LLM inference, questioning whether prefill is underappreciated in terms of ROI for local LLM deployments.

Inside vLLM: Anatomy of a High-Throughput LLM Inference System (2025)

Hacker News Top

A deep dive into vLLM's architecture and components for high-throughput LLM inference, covering scheduling, paged attention, continuous batching, advanced features, scaling, serving, and benchmarking.

@no_stp_on_snek: while everyone is talking about @SpaceXAI , @AnthropicAI , and @OpenAI updates (but where @GoogleAI?)... went and teste…