Benchmarking the b9200 update of llama.cpp with optimized flags for Qwen 3.6 27B MTP on a single RTX 3090 shows significant performance gains, especially in prompt processing speed, for agentic workflows.
**UPDATED (POST b9200)** Okay the updated version using qwen 3.6 27B mtp gguf from unsloth, running it as the backend for the hermes agent. While dialing it in, I noticed that the currently recommended unsloth mtp flags actually bottleneck performance and tank draft acceptance rates for strict, multi-turn agentic workflows. Pairing a custom config with today's brand new llama.cpp b9200 release — which specifically fixes mtp memory traffic overhead — completely turns that around. Hardware/Software \* RTX 3090 (24GB VRAM) — currently undervolted to keep temps down \* Ryzen 7 5700G / 64GB \* Qwen3.6-27B-IQ4\_NL.gguf \* llama-server (b9200+ compiled from source, commit #23234) \* hermes agent (64K context) max to limit spillover Running the standard recommended mtp flags (--spec-draft-n-max 6 and --spec-draft-p-min 0.75) gave poor results for agentic loops. Generation speeds sat around 7–8 t/s, and the mtp draft acceptance rate hovered around 22–26%. Agent workflows are rigid. A 6-token lookahead frequently guesses the wrong punctuation, the main model rejects the draft, and the GPU throws out the math and recalculates — completely negating the mtp speed boost. Without explicitly declaring parallel slots, llama-server also defaults to 4, eating up memory bandwidth managing unused context slots. The fix and the b9200 boost For agent workflows on a 24GB card, limit to a single slot, drop the lookahead to 3, and remove the p-min threshold so it doesn't hesitate on rigid syntax. Combined with the b9200 release — which stops copying the full logits for every token in the batch during prompt processing — the optimized launch command looks like this: `.\build\bin\Release\llama-server.exe ^ -m D:\models\Qwen3.6-27B-IQ4_NL.gguf ^ --spec-type draft-mtp ^ --spec-draft-n-max 3 ^ --ctx-size 65536 ^ --parallel 1 ^ --flash-attn on ^ --cache-type-k q8_0 ^ --cache-type-v q8_0 ^ --port 8081` Results (Prior to the update vs. Post-b9200) Prior to the update (but with the optimized flags): \* Prompt processing sat around \~560 t/s. \* Token generation hit **17.06 t/s on short tasks and \~9.5 t/s during heavy** context reasoning loops. \* Draft acceptance rate climbed to **77%** (proving a shorter lookahead works better for strict formatting). After the b9200 update: \* Prompt processing jumped to **991+ t/s**. The memory traffic fix paired with --parallel 1 lets the 3090 tear through hermes' massive system prompts. \* Token generation hit a peak of **27.44 t/s** on short tasks and stabilized at a highly usable 13.69 t/s during heavy context loops where the agent is switching between tool calls and main memory. \* Draft acceptance rate maintained a **\~70%** on standard turns. When your VRAM bus isn't clogged by ghost parallel slots or 6-token lookahead rejections, an ***undervolted*** 3090 can still push nearly 30 t/s on a dense 27B model
A user benchmarks the MTP variant of Qwen3.6 27B against the normal version on a single RTX 3090 using llama.cpp, finding MTP offers up to 2.37x faster generation at long contexts (32k-64k) but with slower prefill and no concurrency support yet.
A benchmark analysis of Qwen 3.6 27B MTP on 4x RTX 3090 GPUs, demonstrating that using NVLink for tensor parallelism yields significant throughput improvements (up to +53%) over PCIe configurations.
The article benchmarks the Qwen3.6-27B model using Multi-Token Prediction (MTP) and tensor parallelism on dual Mi50 GPUs, demonstrating significant speedups via llama.cpp.
Discussion of performance tradeoffs when using the new MTP merge in llama.cpp to run Qwen 3.6 35B on dual 3090s, with users sharing token speeds and seeking optimal configurations.
A technical test of llama.cpp's new Multi-Token Prediction (MTP) support using Qwen3.6 models on an RTX 5090, comparing performance with and without MTP across different prompts and GGUF quantizations.