GLM-5.2 UD-IQ1_M on llama.cpp — 5090 + 3090 Ti speed test (~ 579 t/s prefill @ 8k ctx, ~324 t/s prefill @ 57k ctx, ~10.6 t/s decode)

Reddit r/LocalLLaMA News
benchmark speed-test glm-5-2 llama-cpp inference gpu

Summary

Speed test results for GLM-5.2 running on llama.cpp with RTX 5090 and RTX 3090 Ti, showing prefill speeds up to 579 t/s at 8k context and decode at ~10.6 t/s.

Just sharing some speed test numbers for GLM-5.2 running on llama.cpp. Setup: Model: unsloth/GLM-5.2-GGUF, UD-IQ1_M quant GPUs: RTX 5090 + RTX 3090 Ti 186 GB DDR5 used Debian 13 CUDA 13.3 128k context, q8_0 KV cache Prefill (prompt processing): n_tokens tokens/s 8,201 579.75 16,393 522.28 24,585 468.21 32,777 422.61 40,969 384.43 49,161 351.90 57,353 324.48 Decode (generation): Holds steady around 10.6 t/s through 580+ decoded tokens. 9.37 t/s on 60k context. Start command: llama-server \ -m GLM-5.2-UD-IQ1_M.gguf \ -fa 1 \ --fit off \ --tensor-split 100,0 \ --override-tensor "blk\.[0-3]\.(ffn_(up|down|gate)_exps\.weight)=CUDA0,blk\.([4-9]|10])\.(ffn_(up|down|gate)_exps\.weight)=CUDA1,blk\.11\.(ffn_down_exps\.weight)=CUDA1" \ --main-gpu 0 \ --n-cpu-moe 99 \ --no-mmap \ --mlock \ --cpu-range 0-23 \ --cpu-range-batch 0-23 \ --ctx-size 131072 \ --parallel 1 \ --jinja --no-warmup --threads 24 --numa isolate \ --batch-size 8192 --ubatch-size 8192 --threads-batch 24 \ -cms 24000 \ -ctxcp 5 \ --cache-type-k q8_0 --cache-type-v q8_0 \ --alias glm.5.2 \ --host 0.0.0.0 --port 8080
Original Article

Similar Articles

Gemma 4 26B Hits 600 Tok/s on One RTX 5090

Reddit r/LocalLLaMA

A benchmark shows that using vLLM with DFlash speculative decoding boosts Gemma 4 26B inference to ~578 tokens per second on a single RTX 5090, achieving a 2.56x speedup over baseline.

GLM 5.2 on consumer hardware

Reddit r/LocalLLaMA

A user tested the unsloth quantized GLM-5.2 model on a high-end consumer-like system with dual RTX 5090, achieving 12 tokens per second.