GLM 5.2 on 4x Sparks reasonable?
Summary
A user asks about the feasibility of running GLM-5.2 at 4-bit quantization on four Ascend GX10s or DGX Sparks, wondering about speed and memory for 100k context.
Similar Articles
4-bit GLM-5.2 (753B MoE) on 4× DGX Spark: 70.8% on Terminal-Bench 2.1 vs 81.0% for the full model
Running a 4-bit quantized version of GLM-5.2 (753B MoE) on 4 DGX Spark machines achieves 70.8% on Terminal-Bench 2.1, compared to 81.0% from the full model.
@TheAhmadOsman: Luke Alonso has uploaded an NVFP4 of GLM 5.2 467GB, would fit on 4x DGX Sparks (~$20k)
Luke Alonso uploaded an NVFP4 quantized version of GLM 5.2 (467GB) that can fit on 4x DGX Sparks hardware, costing approximately $20k.
Follow-up: GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context
A bug in vLLM's speculative decoding configuration for GLM-5.2 NVFP4 on four DGX Sparks was fixed, resolving a performance tradeoff and achieving ~24 tok/s at 128K context with MTP4.
Thinking about grabbing 4x Ascend GX10s
A user considers buying four Ascend GX10s to run GLM5.2, citing performance numbers like 400-500 tok/s prompt processing and ~15 tok/s output at 128k context, and plans for future open-source models.
GLM-5.2-Int4-Int8 on 8× GB10: ~1,200 t/s prefill, 33–54 t/s avg decode
Describes deployment and benchmarking of the quantized GLM-5.2-Int4-Int8Mix model on an 8-node DGX Spark (GB10) cluster using a custom vLLM fork, achieving ~1,200 t/s prefill and ~35 t/s decode with MTP tool calling.