I feel like I finally graduated.
Summary
The user switched from LM Studio to vLLM for serving Qwen 3.8 models, achieving 143 tokens per second and reduced GPU temperatures, referencing a helpful GitHub repository.
Similar Articles
Qwen3.6 27B more dumb in vLLM compared to llama.cpp
A user reports that the Qwen3.6-27B model performs better and more reliably with llama.cpp than with vLLM, citing tool call errors and 'lobotomized' behavior in vLLM despite extensive configuration.
Qwen3.8 27B C8 at 972 TG / 5,680 PP on 4x MI100 rig ($6.5k) using my new INT8 vLLM fork
A new vLLM fork introduces comprehensive INT8 optimization for Qwen3.8 27B on older AMD MI100 GPUs, achieving up to 972 tokens per second throughput with rigorous accuracy validation.
Qwen 3.6-35B-A3B with 977 tk/s prompt processing and 262k context window on Intel Arc B70 Pro
This article describes how to use the SYCL backend with llama.cpp to achieve over 60 tokens per second on the Qwen 3.6-35B-A3B model using an Intel Arc Pro B70 GPU, with the entire model and KV cache in VRAM.
Running Qwen 3.8 next on 16vram+32ram - A useful/fun post for the gpu poors
A Reddit user shares how they successfully ran the Qwen 3.8 Next MoE model on a system with 16GB VRAM and 32GB RAM using aggressive quantization and specific llama.cpp settings, achieving usable performance for large models on limited hardware.
Intel LLM-Scaler-vLLM PV 1.5 brings big improvements
Intel released LLM-Scaler-vLLM PV 1.5, a major update to its containerized vLLM stack for Arc Graphics hardware, jumping from vLLM 0.14 to 0.26, adding new Qwen and Gemma model support, FP8/INT4 quantization, and performance improvements.