@wafer_ai: BREAKING: these engineers figured out how to serve GLM 5.2 on @AMD MI355X at 2626 tok/s/node and 213 tok/s single strea…
Summary
Engineers successfully serve GLM 5.2 on AMD MI355X at 2626 tok/s per node and 213 tok/s single stream, achieving ~80% of B200 throughput at over 2x lower cost than Blackwell.
View Cached Full Text
Cached at: 07/04/26, 08:41 AM
🚨 BREAKING:
these engineers figured out how to serve GLM 5.2 on @AMD MI355X at 2626 tok/s/node and 213 tok/s single stream at over 2x lower cost than Blackwell
that’s ~80% of B200 throughput at over 2x lower cost
full write-up in reply to see how https://t.co/1wAickkQEm
BREAKING:
these engineers figured out how to serve GLM 5.2 on @AMD MI355X at 2626 tok/s/node and 213 tok/s single stream at over 2x lower cost than Blackwell
that’s ~80% of B200 throughput at over 2x lower cost
full write-up in reply to see how
this is how they did it:
yo that’s my dad you’re talking about
lollll
not as cool as you @tomgreenwald
thank you!
thank you!
we’re always on time
Similar Articles
Performance per dollar is getting faster and cheaper
Wafer demonstrates that AMD MI355X GPUs offer competitive inference performance for frontier models like GLM5.2 at significantly lower cost than NVIDIA Blackwell, achieving 80% of B200 throughput at under half the price, using MXFP4 quantization and sglang.
I did some model hacks, and got GLM5.2 from about 2.5 tok/s to >50 tok/s on my GH200 system.
A detailed blog post describing how to dramatically speed up GLM-5.2 inference on a dual Grace Hopper system from 2.5 tok/s to over 50 tok/s by stopping model cross-module traffic and grafting an FP8 MTP head onto the INT4 base.
LFM2.5 230M running in-browser at 1,400 tok/s using custom WebGPU kernels
LFM2.5 230M model achieves 1,400 tokens per second in-browser using custom WebGPU kernels, demonstrating efficient local inference.
GLM5.2 on 5x Pro 6000s and a 5090, an expensive journey
A report on running the GLM5.2 language model across 5 AMD Radeon Pro 6000 GPUs and an NVIDIA RTX 5090, detailing the high cost and technical challenges.
@aliez_ren: 这下真的舒服了,本地 70 tps 跑 GLM 5.2 几乎满血版! https://huggingface.co/madeby561/GLM-5.2-MXFP8-NVFP4-NF3-Hybrid…
通过混合精度(MXFP8、NVFP4、NF3)量化,在4张96GB GPU上实现本地运行GLM-5.2(753B参数)几乎满血版,精度接近原始FP8,吞吐量达70 tps。