Solved: LLM inference on Windows was 2–3x slower when the server window wasn't focused
Summary
LLM inference on Windows was 2–3x slower when the server window wasn't focused due to CPU scheduling delays; running the server detached/headless fixed the issue, maintaining consistent performance.
Similar Articles
@h100envy: Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference op…
An ex-vLLM core contributor explains how to reduce LLM inference cost by 10x using LMCache with KV cache offloading to CPU/SSD/remote storage, a technique used by production stacks like Bloomberg.
i finally switched from windows to linux and got a 30-50% boost in speed.
The user switched from Windows to Linux and replaced llamacpp with vllm, achieving a 30-50% speed improvement in AI inference.
WebLLM: high-performance in-browser LLM inference engine
WebLLM is a high-performance in-browser LLM inference engine that leverages WebGPU for hardware acceleration and is fully compatible with the OpenAI API, enabling local execution of open-source language models.
Inside vLLM: Anatomy of a High-Throughput LLM Inference System (2025)
A deep dive into vLLM's architecture and components for high-throughput LLM inference, covering scheduling, paged attention, continuous batching, advanced features, scaling, serving, and benchmarking.
@_avichawla: This is the strangest property of LLM inference: Your LLM's output depends on requests that aren't yours. The same prom…
A technical explainer on why LLM outputs can vary between identical requests: floating-point accumulation order changes with batch size on the GPU, causing tied tokens to flip and completions to diverge even at temperature 0. It highlights batch-invariance flags in vLLM, SGLang, and DeepSeek's DeepInfer, noting the throughput trade-offs for reproducible outputs.