Thinking about grabbing 4x Ascend GX10s
Summary
A user considers buying four Ascend GX10s to run GLM5.2, citing performance numbers like 400-500 tok/s prompt processing and ~15 tok/s output at 128k context, and plans for future open-source models.
Similar Articles
GLM 5.2 on 4x Sparks reasonable?
A user asks about the feasibility of running GLM-5.2 at 4-bit quantization on four Ascend GX10s or DGX Sparks, wondering about speed and memory for 100k context.
Running GLM 5.2 on 4xGB10 with a 100G Switch, 330k ctx, ~25 t/s tg, ~650 t/s pp
This post details running GLM 5.2 on a 4xGB10 setup with a 100G switch, achieving ~25 tok/s decode and ~650 tok/s prefill at 330k context. It includes hardware costs, performance benchmarks with Depth Prefill, and notes on model pruning for longer context.
I did some model hacks, and got GLM5.2 from about 2.5 tok/s to >50 tok/s on my GH200 system.
A detailed blog post describing how to dramatically speed up GLM-5.2 inference on a dual Grace Hopper system from 2.5 tok/s to over 50 tok/s by stopping model cross-module traffic and grafting an FP8 MTP head onto the INT4 base.
@AlpinDale: GLM-5.2-FP8 running on 4 nodes of 4090-48GBs at around 28 tok/s decode. The nodes have an interconnect speed of 10 giga…
AlpinDale reports running GLM-5.2-FP8 on 4 nodes of RTX 4090 (48GB) achieving ~28 tok/s decode over 10Gbit ethernet, with plans to optimize using DSpark.
GLM-5.2-Int4-Int8 on 8× GB10: ~1,200 t/s prefill, 33–54 t/s avg decode
Describes deployment and benchmarking of the quantized GLM-5.2-Int4-Int8Mix model on an 8-node DGX Spark (GB10) cluster using a custom vLLM fork, achieving ~1,200 t/s prefill and ~35 t/s decode with MTP tool calling.