GLM-5.2 UD-IQ1_M on llama.cpp — 5090 + 3090 Ti speed test (~ 579 t/s prefill @ 8k ctx, ~324 t/s prefill @ 57k ctx, ~10.6 t/s decode)

Reddit r/LocalLLaMA 06/22/26, 02:17 PM News

benchmark speed-test glm-5-2 llama-cpp inference gpu

Summary

Speed test results for GLM-5.2 running on llama.cpp with RTX 5090 and RTX 3090 Ti, showing prefill speeds up to 579 t/s at 8k context and decode at ~10.6 t/s.

Just sharing some speed test numbers for GLM-5.2 running on llama.cpp. Setup: Model: unsloth/GLM-5.2-GGUF, UD-IQ1_M quant GPUs: RTX 5090 + RTX 3090 Ti 186 GB DDR5 used Debian 13 CUDA 13.3 128k context, q8_0 KV cache Prefill (prompt processing): n_tokens tokens/s 8,201 579.75 16,393 522.28 24,585 468.21 32,777 422.61 40,969 384.43 49,161 351.90 57,353 324.48 Decode (generation): Holds steady around 10.6 t/s through 580+ decoded tokens. 9.37 t/s on 60k context. Start command: llama-server \ -m GLM-5.2-UD-IQ1_M.gguf \ -fa 1 \ --fit off \ --tensor-split 100,0 \ --override-tensor "blk\.[0-3]\.(ffn_(up|down|gate)_exps\.weight)=CUDA0,blk\.([4-9]|10])\.(ffn_(up|down|gate)_exps\.weight)=CUDA1,blk\.11\.(ffn_down_exps\.weight)=CUDA1" \ --main-gpu 0 \ --n-cpu-moe 99 \ --no-mmap \ --mlock \ --cpu-range 0-23 \ --cpu-range-batch 0-23 \ --ctx-size 131072 \ --parallel 1 \ --jinja --no-warmup --threads 24 --numa isolate \ --batch-size 8192 --ubatch-size 8192 --threads-batch 24 \ -cms 24000 \ -ctxcp 5 \ --cache-type-k q8_0 --cache-type-v q8_0 \ --alias glm.5.2 \ --host 0.0.0.0 --port 8080

Original Article

Similar Articles

GLM-5.2-Int4-Int8 on 8× GB10: ~1,200 t/s prefill, 33–54 t/s avg decode

Reddit r/LocalLLaMA

Describes deployment and benchmarking of the quantized GLM-5.2-Int4-Int8Mix model on an 8-node DGX Spark (GB10) cluster using a custom vLLM fork, achieving ~1,200 t/s prefill and ~35 t/s decode with MTP tool calling.

Comparing dual-GPU inference speed between llama.cpp row/tensor split and ik_llama graph split

Reddit r/LocalLLaMA

A user benchmarks dual-GPU inference speed on two RTX 3080 20GB using llama.cpp (row/tensor split) and ik_llama (graph split) with a Qwen3.6-27B GGUF model, comparing token generation and prompt processing speeds.

Dual 3090 setup: 400 pp t/s to 1600 pp t/s on Qwen 3.6 27B... with slightly lower tps.

Reddit r/LocalLLaMA

A user on a dual RTX 3090 setup discovers that llama.cpp's --split-mode tensor runs prompt processing on CPU (~400 t/s), while switching to --split-mode layer unlocks GPU prompt processing over 1600 t/s, with only a modest drop in token generation speed.

16x AMD MI50 32GB: GLM-5.2 Q4 at 12.2 tok/s with llama.cpp RPC

Reddit r/LocalLLaMA

Describes running the GLM-5.2 model with 4-bit quantization at 12.2 tokens per second on a cluster of 16 AMD MI50 GPUs using llama.cpp's RPC, achieving coherent long-form generation at 10.7k context.

@AlpinDale: GLM-5.2-FP8 running on 4 nodes of 4090-48GBs at around 28 tok/s decode. The nodes have an interconnect speed of 10 giga…

X AI KOLs Following

AlpinDale reports running GLM-5.2-FP8 on 4 nodes of RTX 4090 (48GB) achieving ~28 tok/s decode over 10Gbit ethernet, with plans to optimize using DSpark.

Similar Articles

GLM-5.2-Int4-Int8 on 8× GB10: ~1,200 t/s prefill, 33–54 t/s avg decode

Comparing dual-GPU inference speed between llama.cpp row/tensor split and ik_llama graph split

Dual 3090 setup: 400 pp t/s to 1600 pp t/s on Qwen 3.6 27B... with slightly lower tps.

16x AMD MI50 32GB: GLM-5.2 Q4 at 12.2 tok/s with llama.cpp RPC

@AlpinDale: GLM-5.2-FP8 running on 4 nodes of 4090-48GBs at around 28 tok/s decode. The nodes have an interconnect speed of 10 giga…

Submit Feedback