A user shares their llama.cpp server configuration for running Qwen 3.6 27B on an RTX 5090, achieving 80-100 t/s, and asks the community for alternative settings and tips.
I run the following on a 5090 and have been okay with its performance, it does most things somewhere 80-100 t/s, though that can slow down at full 262k context - more like 40 t/s at times. I use it primarily in appdev tasks. This just barely fits in the 5090, no vision, with very very little room to spare. The batch sizes (-b / -ub) I had at a much different number initially, but bracketing my coding tests settled there for the best performance with the best output. I see that a lot of people have this much higher, but it didn't seem to help here. Reasoning budget is another one that I have different than most people I see, I have it pretty high vs what people run that I've seen other places, but it seemed to be okay? It's hard to qualify this with evals but it seems to get lost more with lower values. Honestly I haven't really gone deeply into all of the settings, because tokens go brrrr and I'm pretty happy. But I wanted to see: Do you set any of this shit to something else? If so: why, motherfucker, what are you doing with it? Spill the beans not the tokens. Fuck spez. llama-server \ -m '/ai/models--unsloth--Qwen3.6-27B-MTP-GGUF/snapshots/5cb35eb3dcbf52dbce5f87dbc64df6aaffadcace/Qwen3.6-27B-Q6_K.gguf' \ -c 262144 \ -b 512 \ -ub 128 \ --gpu-layers all \ --spec-draft-ngl all \ --fit off \ --parallel 1 \ --flash-attn on \ --no-context-shift \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ --spec-type draft-mtp \ --spec-draft-n-max 3 \ --spec-draft-p-min 0.2 \ --cache-ram 16384 \ --ctx-checkpoints 32 \ --checkpoint-min-step 8192 \ --jinja \ --reasoning on \ --reasoning-budget 16384 \ --no-reasoning-preserve \ --load-mode none \ --no-mmproj \ --alias 'unsloth/Qwen3.6-27B-MTP-GGUF' \ --metrics \ --perf \ --host 0.0.0.0 \ --port 1234
A user shares optimized settings for running Qwen3.6 27B (Q8_0) on a dual GPU setup (RTX 4090 + RTX 3090) with llama.cpp, achieving 75-100 t/s and 1500 pp with 250k context.
The article compares llama.cpp backends for running Qwen 3.6 27B on an RTX 3090 24GB, finding ik_llama.cpp with IQ4_KS quantization yields the best performance (1261 tok/s prefill, 72.9 tok/s decode).
A user details their setup running Qwen 27B with llama.cpp on an RTX PRO 6000 Blackwell for local coding agents, compares performance to Claude models, and asks for help resolving frequent crashes and malformed response issues.
The author shares detailed tuning tips for running the Qwen3.6-35B-A3B MoE model on an 8GB RTX 3070 Ti with up to 262k context using llama.cpp, achieving 30+ tps, and notes a 25% speed boost when switching from Windows to Ubuntu Server.