Qwen 3.8 27B is faster than expected
Summary
A user reports that the Qwen 3.8 27B model achieves 50-60 tokens per second on dual 5060 TI cards, showing unexpected speed improvements over previous versions like Qwen 3.6.
Similar Articles
Qwen3.8-27B: slower tokens, faster and better results
Qwen3.8-27B is a new AI model that emphasizes wall-clock time over token speed, offering superior intelligence for local deployment on consumer hardware with 32 GB VRAM.
Qwen3.6 27B on a 5090, 6.4k sample tok/s distribution after tuning MTP/cache settings
Running Qwen3.6 27B on an RTX 5090, achieving 6.4k tokens per second after tuning MTP and cache settings, demonstrating optimization techniques for inference.
Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s): 50 tok/s with MTP
The article details an experiment achieving 50 tokens per second inference with Qwen3.8-27B at 256K context on a 24GB GPU using Multi-Token Prediction and custom optimizations.
What speed is everyone getting on Qwen3.6 27b?
User benchmarks Qwen3.6-27B-Q8_0 at ~13 tokens/sec on 3 mixed GPUs with 128k context via llama.cpp, asking if performance is typical.
Qwen 3.8 2.4T at 288k tokens/s on Nvidia GB300 NVL72
NVIDIA showcases the high-throughput performance of serving the Qwen3-8B 2.4T parameter model on GB300 NVL72 hardware, achieving over 4k tokens per second per GPU.