@rumgewieselt: Now its getting crazy ... 3x 1080 Ti (Pascal, 33GB VRAM) Qwen 3.6 27B MTP with 196K TurboQuant ~28-30 t/s consistently
Summary
A user demonstrates successful local inference of a 27B parameter Qwen model across three GTX 1080 Ti GPUs, achieving approximately 28-30 tokens per second using TurboQuant optimization.
View Cached Full Text
Cached at: 05/08/26, 01:32 PM
Now its getting crazy …
3x 1080 Ti (Pascal, 33GB VRAM)
Qwen 3.6 27B MTP with 196K TurboQuant ~28-30 t/s consistently https://t.co/asoDLIrAad
Similar Articles
Got MTP + TurboQuant running — Qwen3.6-27B -- 80+ t/s at 262K context on a single RTX 4090
Developer achieved 80+ t/s inference on Qwen3.6-27B with 262K context on a single RTX 4090 by combining MTP (Multi-Token Prediction) with TurboQuant's lossless KV cache compression, sharing their implementation fork and technical details.
I pushed Qwen3.8-27B to 99 tps single request and 1150 tps with a batch request on a RTX 3090
The author optimized the Qwen3.8-27B model inference on an RTX 3090 GPU, achieving up to 99 tokens per second for single requests and 1150 tps with batch processing through various quantization and optimization techniques, and released the updated code on GitHub.
Qwen 3.8 2.4T at 288k tokens/s on Nvidia GB300 NVL72
NVIDIA showcases the high-throughput performance of serving the Qwen3-8B 2.4T parameter model on GB300 NVL72 hardware, achieving over 4k tokens per second per GPU.
100$ worth of gpu runs qwen 3.8 27b at 7.39 t/s
A user demonstrates running the Qwen 27b AI model quantized to Q3_K_M on two RX 580 GPUs, achieving 7.39 tokens per second using old DDR3 hardware for under $100.
Qwen3.8-27B at 256K on a 24GB RTX PRO 4000 SFF (432 GB/s): 50 tok/s with MTP
The article details an experiment achieving 50 tokens per second inference with Qwen3.8-27B at 256K context on a 24GB GPU using Multi-Token Prediction and custom optimizations.