@TeksEdge: A localmaxxer hit ~381 tok/s on a SINGLE RTX 3090 with Qwen3.8-27B. This developer has turned a 24GB RTX 3090 into a mo…
Summary
A developer achieved up to 381 tok/s inference speed on a single RTX 3090 with the Qwen3.8-27B model using optimized techniques like DFlash2 and prefix caching, particularly effective for document-based tasks like RAG and coding assistants.
View Cached Full Text
Cached at: 08/21/26, 03:19 PM
A localmaxxer hit ~381 tok/s on a SINGLE RTX 3090 with Qwen3.8-27B.
This developer has turned a 24GB RTX 3090 into a monster Qwen inference box - w/some creativity.
Four days ago ~82 tok/s single-user
Then ~114 tok/s with optimized MTP ~138 tok/s with DFlash2 + lookup drafting
Now ~381 tok/s on ONE request
How? The recipe combines … Qwen3.8-27B 1× RTX 3090 24GB @ 250W heavily optimized vLLM DFlash2 speculative decoding lookup-augmented drafting prefix caching 16-token verification blocks quantized KV / heads / activations
DFlash2 normally proposes 7 tokens.
The developer realized the verification block doesn’t have to stop there.
If Qwen is answering from a document already sitting in the prompt, the system can fill the remaining draft positions using tokens found directly in that context.
So the target model can verify 16 tokens at once.
On a ~25K-token document reproduction task: Previous DFlash2 ~260 tok/s
Longer verification + context lookup ~382 tok/s
Acceptance: 15 of 16 tokens per verification step
That is where the crazy number comes from.
On ordinary real-world chat prompts, the same setup is around ~133 tok/s
Still extremely fast for a dense 27B model on an RTX 3090.
The 381 tok/s mode shines when the answer largely comes from material already in context so these are best use cases RAG / document Q&A Coding assistants applying edits Quoting or rewriting documents Extracting information from long prompts
And another optimization With prefix caching, a second question against the same 25K-token document reportedly goes from: 22.4 sec TTFT → 0.56 sec TTFT
Because the model doesn’t need to process the whole document again.
It’s specifically a mode for RAG front ends and coding agents.
Follow iamMess on Reddit or syv-ai on GitHub
Reddit: r/LocalLLaMA/comments/1vtup5s/ GitHub: /syv-ai/qwen38-27b-rtx3090
Similar Articles
@seclink: Just hit 134 tok/s with Qwen 3.5-27B Dense and 73 tok/s with the new Qwen 3.6-27B on a single RTX 3090. The 2026 open-source scene is moving at lightspeed…
A single RTX 3090 pushes 134 tok/s on the fresh 27B Qwen 3.5 Dense and 73 tok/s on Qwen 3.6-27B via fused kernels plus speculative decoding, with GGUF drops the same evening.
I pushed Qwen3.8-27B to 99 tps single request and 1150 tps with a batch request on a RTX 3090
The author optimized the Qwen3.8-27B model inference on an RTX 3090 GPU, achieving up to 99 tokens per second for single requests and 1150 tps with batch processing through various quantization and optimization techniques, and released the updated code on GitHub.
Got MTP + TurboQuant running — Qwen3.6-27B -- 80+ t/s at 262K context on a single RTX 4090
Developer achieved 80+ t/s inference on Qwen3.6-27B with 262K context on a single RTX 4090 by combining MTP (Multi-Token Prediction) with TurboQuant's lossless KV cache compression, sharing their implementation fork and technical details.
@ItsmeAjayKV: Update on 3090: Now with Qwen 3.6-35b-a3b moe (q6_k_xl). Crossed 90 t/s for the very first time, no MTP yet, prefill sp…
A user reports achieving over 90 tokens per second inference speed with Qwen 3.6-35b-a3b MoE model on an RTX 3090 using llama.cpp, with prefill speeds exceeding 1000 t/s, indicating practical local deployment of large language models on consumer hardware.
EXPERIMENT: Qwen3.8-2.4T-A95B running locally on an RTX 5090 + RTX 5060 Ti at ~0.80 tok/s
An experiment running the Qwen3.8-2.4T-A95B MoE model locally on dual consumer GPUs (RTX 5090 + 5060 Ti) with llama.cpp, achieving ~0.8 tok/s with MTP speculative decoding enabled.