@AlpinDale: GLM-5.2-FP8 running on 4 nodes of 4090-48GBs at around 28 tok/s decode. The nodes have an interconnect speed of 10 giga…
Summary
AlpinDale reports running GLM-5.2-FP8 on 4 nodes of RTX 4090 (48GB) achieving ~28 tok/s decode over 10Gbit ethernet, with plans to optimize using DSpark.
View Cached Full Text
Cached at: 07/09/26, 05:39 PM
GLM-5.2-FP8 running on 4 nodes of 4090-48GBs at around 28 tok/s decode. The nodes have an interconnect speed of 10 gigabit (via ethernet), so this could be a lot better. I’m also working on getting DSpark running with this, which should hopefully help too. https://t.co/vinnhxbQsK
Similar Articles
@Tech2Wild: Running GLM-5.2 at home the FULL 744B, all 256 experts, UNPRUNED across 4× NVIDIA DGX Spark (GB10). 200K context · MTP …
A detailed recipe for running the unpruned GLM-5.2 model (744B parameters, 256 experts) across 4 NVIDIA DGX Spark nodes with 200K context, achieving up to 60.5 tok/s aggregate. Includes performance benchmarks, credits, and patches.
GLM-5.2 UD-IQ1_M on llama.cpp — 5090 + 3090 Ti speed test (~ 579 t/s prefill @ 8k ctx, ~324 t/s prefill @ 57k ctx, ~10.6 t/s decode)
Speed test results for GLM-5.2 running on llama.cpp with RTX 5090 and RTX 3090 Ti, showing prefill speeds up to 579 t/s at 8k context and decode at ~10.6 t/s.
Running GLM 5.2 on 4xGB10 with a 100G Switch, 330k ctx, ~25 t/s tg, ~650 t/s pp
This post details running GLM 5.2 on a 4xGB10 setup with a 100G switch, achieving ~25 tok/s decode and ~650 tok/s prefill at 330k context. It includes hardware costs, performance benchmarks with Depth Prefill, and notes on model pruning for longer context.
@aliez_ren: 这下真的舒服了，本地 70 tps 跑 GLM 5.2 几乎满血版！ https://huggingface.co/madeby561/GLM-5.2-MXFP8-NVFP4-NF3-Hybrid…
通过混合精度（MXFP8、NVFP4、NF3）量化，在4张96GB GPU上实现本地运行GLM-5.2（753B参数）几乎满血版，精度接近原始FP8，吞吐量达70 tps。
GLM5.2 @7tg on 4x3090 + 192GB on budget motherboard + cpu
Running GLM5.2 with 7 trillion tokens on a budget setup using 4x RTX 3090 GPUs and 192GB RAM.