@tolak_eth: I wanted to share how we avoided spending roughly $160k/year to host GLM-5.2 with its full 1M context. When GLM-5.2 lau…
Summary
Phala avoided $160k/year hosting costs for GLM-5.2 with full 1M context by quantizing MoE experts to 4-bit and keeping critical parts in FP8/BF16, achieving the same benchmark results on a single 8×H200 node and releasing the optimized model GLM-5.2-W4AFP8 on Hugging Face.
View Cached Full Text
Cached at: 07/03/26, 06:40 PM
I wanted to share how we avoided spending roughly $160k/year to host GLM-5.2 with its full 1M context.
When GLM-5.2 launched, Phala was one of the launch partners of http://Z.ai. We tried to bring it up immediately on our existing 8×H200 setup. The model was exciting, but the deployment reality was less romantic: we could not fully open the 1M context window on that node. The obvious path was to move to a more expensive setup, likely Blackwell-class hardware. That was not a small cost decision.
This is where open source becomes powerful. Instead of treating the model as a fixed artifact, the team started asking whether we could make the memory budget work. They quantized the routed MoE experts to 4-bit, kept the important parts in FP8/BF16, and validated the result carefully. The outcome was GLM-5.2-W4AFP8: full 1M context on a single 8×H200 node, with benchmark results aligned with the FP8 baseline.
Until today, GLM-5.2-W4AFP8 on Hugging Face has already close to 20k downloads. I think that says something. Builders do not just want bigger context windows. They want models that are practical to run without turning every deployment into a hardware procurement problem.
It reminded me of Hackers & Painters: the beauty of open source is that users do not have to stop at “this is what was released.” They can reshape the tool until it fits the real world.
Download: https://huggingface.co/PhalaCloud/GLM-5.2-W4AFP8…
Similar Articles
@0xSero: We found a way to run GLM-5.2 with full context in vLLM without pruning. - top 32 experts NVFP4 - rest fp3 - intel auto…
A community researcher enabled running GLM-5.2 (753B parameters, all 256 experts) in vLLM without pruning via a hybrid quantization (NVFP4, NF3, MXFP8), fitting on 4×96GB GPUs with ~307k KV cache and near-FP8 accuracy.
@Tech2Wild: Running GLM-5.2 at home the FULL 744B, all 256 experts, UNPRUNED across 4× NVIDIA DGX Spark (GB10). 200K context · MTP …
A detailed recipe for running the unpruned GLM-5.2 model (744B parameters, 256 experts) across 4 NVIDIA DGX Spark nodes with 200K context, achieving up to 60.5 tok/s aggregate. Includes performance benchmarks, credits, and patches.
@0x_kaize: https://x.com/0x_kaize/status/2068775813785506091
A guide on avoiding rate limits and reducing costs when using the GLM 5.2 model, covering prompt batching, caching, free model alternatives, effort levels, context window management, and self-hosting.
Running GLM 5.2 on 4xGB10 with a 100G Switch, 330k ctx, ~25 t/s tg, ~650 t/s pp
This post details running GLM 5.2 on a 4xGB10 setup with a 100G switch, achieving ~25 tok/s decode and ~650 tok/s prefill at 330k context. It includes hardware costs, performance benchmarks with Depth Prefill, and notes on model pruning for longer context.
@HuggingPapers: NVIDIA just released an optimized GLM-5.2 on Hugging Face A 753B parameter MoE with 1M context, quantized to NVFP4 for …
NVIDIA released an optimized GLM-5.2 MoE model on Hugging Face with 753B parameters and 1M context, quantized to NVFP4 for Blackwell GPUs while nearly matching FP8 accuracy.