@superalesha: Don't dare bury RTX 3090 until you read this! @UnslothAI shipped two new 4-bit quants of qwen3.6-35b this week. i spent…
Summary
A benchmark comparison of nvfp4, nvfp4-fast, and AWQ 4-bit quantizations of Qwen3.6-35B on RTX 3090s shows similar performance, with the MTP head trick boosting throughput by 41%.
View Cached Full Text
Cached at: 07/12/26, 12:56 PM
Don’t dare bury RTX 3090 until you read this!
@UnslothAI shipped two new 4-bit quants of qwen3.6-35b this week. i spent the night racing them against awq on my 4x3090 to find the fastest one.
nvfp4, nvfp4-fast and awq land within 17% on decode, ~3% on prefill, and within 3 points on gpqa, mmlu-pro and code. the format war is a draw. the 3090 has no fp4 units, so vllm runs every one as w4a16 on marlin. “fast” has nothing to run on.
then i flipped the mtp head that ships inside the checkpoint. 173 -> 246 tok/s , +41%, it accepted 95%
still beast
Similar Articles
Benchmark Qwen 3.6 27B MTP on 2x3090 NVLINK
A benchmark analysis of Qwen 3.6 27B MTP on 4x RTX 3090 GPUs, demonstrating that using NVLink for tensor parallelism yields significant throughput improvements (up to +53%) over PCIe configurations.
Qwen 3.6 27B Speculative Decoding Bench: Pushing ~100 TPS on a single RTX 3090
A detailed benchmark comparing speculative decoding engines for Qwen 3.6 27B on a single RTX 3090, showing ik_llama achieving ~100 tokens per second in code generation. Results include decode TPS, TTFT, VRAM usage, and context degradation across 5 engine variants.
Ternary Qwen3.6 27B Tested on 3090!
User tests ternary quantized Qwen3.6 27B on an RTX 3090, achieving 60 tk/s with two slots and 100k KV cache using 21GB VRAM, with good quality and stable tool calls.
I benchmarked Unsloth's Qwen3.6-27B NVFP4 on 1x/2x 5090s. MTP is great until it really isn't.
A detailed benchmark of Unsloth's Qwen3.6-27B NVFP4 model on RTX 5090 GPUs, showing MTP (multi-token prediction) gives large speedups for single requests at short context but becomes detrimental under batch concurrency or long contexts.
@TheAhmadOsman: Qwen 3.8 27B in NVFP4 would fit on a single RTX 5090 btw
The Qwen 3.8 27B model has been released in an NVFP4 quantized version, enabling it to run on a single RTX 5090 GPU with enhancements in coding, agentic tasks, and vision-language understanding.