@danyurkin: i don't think i need cloud models anymore
Summary
A tweet demonstrates that Multi-Token Prediction (MTP) achieves significant speedups for Qwen models on dual RTX 5090 hardware, suggesting that local inference can now rival cloud-model performance.
View Cached Full Text
Cached at: 05/22/26, 09:45 AM
i don’t think i need cloud models anymore
atomic.chat (@atomic_chat_hq): MTP speedup Qwen by 2.5x in Atomic Chat
Dense vs MoE models on 2x RTX 5090 Qwen3.6 27B: 51 → 117 tps +137% Qwen3.6 35B-A3B: 218 → 267 tps +25%
MTP drafts several tokens ahead and verifies them in one pass. The speedup depends on memory moved per pass. Dense 27B reads all 27B
Similar Articles
Multi-Token Prediction (MTP) for Qwen on LLaMA.cpp + TurboQuant
Implemented Multi-Token Prediction for Qwen on LLaMA.cpp with TurboQuant, achieving a 40% performance boost and 90% acceptance rate, running locally on a MacBook Pro M5 Max.
Got MTP + TurboQuant running — Qwen3.6-27B -- 80+ t/s at 262K context on a single RTX 4090
Developer achieved 80+ t/s inference on Qwen3.6-27B with 262K context on a single RTX 4090 by combining MTP (Multi-Token Prediction) with TurboQuant's lossless KV cache compression, sharing their implementation fork and technical details.
@rumgewieselt: Now its getting crazy ... 3x 1080 Ti (Pascal, 33GB VRAM) Qwen 3.6 27B MTP with 196K TurboQuant ~28-30 t/s consistently
A user demonstrates successful local inference of a 27B parameter Qwen model across three GTX 1080 Ti GPUs, achieving approximately 28-30 tokens per second using TurboQuant optimization.
@ggerganov: llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance j…
llama.cpp adds Multi-Token Prediction (MTP) support for the Qwen3.6 family, delivering massive performance improvements for local AI inference on commodity hardware.
@davis7: @0xSero helped me setup local models properly and I uh, had no idea these things had gotten this good Are they frontier…
The author highlights the impressive capabilities of the open-source Qwen 3.6-27B model running locally on an RTX 5090, noting its strong performance on programming tasks and comparing it favorably to commercial models, despite the complexity of local deployment.