Qwen3.6:35b UD Q4_K_M 80 tok/s on Nvidia P40

https://preview.redd.it/nrxkobib5hdh1.png?width=1147&format=png&auto=webp&s=a36a282635bcbb23c0c2bdffa855689eab5e76f9 I'm running Unsloth's Qwen3.6:35B UD Q4_K_M with a 100k context on a single NVIDIA P40 (24GB) using TheTom's TurboQuant fork of llama.cpp. I know this is only burst speed, but I thought it was impressive enough to share with the community. Current setup: TurboQuant enabled (stopped short of dropping K to Turbo2 to preserve agent quality) Reasoning disabled because it would consistently get stuck in a thinking loop Vision component not loaded, which saved about 300 MB of VRAM and let me fit the full 100k context on the P40 At this point I'm trying to squeeze every last bit of performance out of this old hardware without making the model unusable as an agent. If anyone knows of more "voodoo" optimizations for llama.cpp or the P40, I'd love to hear them! llama-server --host 0.0.0.0 --port 11436 --hf-repo unsloth/Qwen3.6-35B-A3B-MTP-GGUF --hf-file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf --ctx-size 100000 --parallel 1 --threads 12 --threads-batch 12 --batch-size 2048 --ubatch-size 512 --n-gpu-layers 99 --n-cpu-moe 0 --flash-attn on --no-mmap --jinja --alias qwen3.6:35b --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 --cache-type-k q8_0 --cache-type-v turbo2 --spec-type draft-mtp --spec-draft-n-max 2 --no-mmproj --reasoning off