jlnsrk/GLM-5.2-colibri-int4
Summary
Pre-converted int4 quantized weights for the GLM-5.2 744B MoE model, designed to run on consumer hardware with ~25 GB RAM using the colibrì engine.
View Cached Full Text
Cached at: 07/14/26, 04:14 AM
jlnsrk/GLM-5.2-colibri-int4 · Hugging Face
Source: https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4
https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4#glm-52–colibr%C3%AC-int4-container-370-gbGLM-5.2 — colibrì int4 container (~370 GB)
Pre-converted weights for**colibrì**— the pure-C engine that runs GLM-5.2 (744B MoE) on a consumer machine with ~25 GB of RAM by streaming routed experts from disk.
This is the output of colibrì’scoli convert(convert\_fp8\_to\_int4\.py \-\-ebits 4 \-\-io\-bits 8, including the MTP head for native speculative decoding), uploaded so you don’t have to download the 756 GB FP8 checkpoint and spend a day converting it.
**⚠️ This is NOT a GGUF / AWQ / GPTQ / MLX model.**It is colibrì’s own container: for each quantized weight,name(U8, packed int4 nibbles) +name\.qs(F32 per-row scales), quantized with math bit-identical to the engine’s C kernels. It only works with the colibrì engine.
https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4#usageUsage
# get the engine
git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh
# download this repo to a FAST local disk (NVMe, ext4 — never a network/9p mount)
hf download jlnsrk/GLM-5.2-colibri-int4 --local-dir /nvme/glm52_i4
# chat (RAM budget, expert cache and MTP auto-detected)
COLI_MODEL=/nvme/glm52_i4 ./coli chat
Requirements: Linux (or WSL2), gcc + OpenMP, AVX2, ≥16 GB RAM, ~400 GB free NVMe.
https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4#whats-insideWhat’s inside
filecontentsout\-\*\.safetensorsdense weights (attention/MLA, shared experts, embeddings) + 21,504 routed experts, int4 per-row scales; router/norms kept F32MTP shardGLM-5.2’s multi-token-prediction head (layer 78) — enables lossless speculative decoding (~2 tok/forward)config\.json,tokenizer\*\.json,generation\_config\.jsoncopied from the base repo
Conversion: FP8 (e4m3, 128×128 block scales) → f32 → int4 withnp\.rintmatching the engine’slrintf— token-identical to converting locally.
https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4#provenance–licenseProvenance & license
Converted fromzai-org/GLM-5.2-FP8(MIT). This derivative is likewise MIT. Conversion performed with colibrì’s official converter, unmodified.
Similar Articles
@ErickSky: Forget about vLLM, llama.cpp, and expensive GPUs. [colibri] This runs GLM-5.2 (744B MoE) on ~25 GB of RAM with pure C a…
colibri is a pure C inference tool that runs the GLM-5.2 744B MoE model on ~25 GB RAM by streaming experts from disk, eliminating the need for expensive GPUs.
Show HN: Getting GLM 5.2 running on my slow computer
Colibrì is a pure C inference engine that runs the 744B GLM-5.2 MoE model on consumer hardware with ~25GB RAM by streaming experts from disk, achieving ~2.2-2.8 tokens/second with speculative decoding.
Colibri Hands-on: Running GLM 5.2 (744B) Locally without GPU
Colibri enables running the 744B-parameter GLM 5.2 model locally on CPU, making large-scale AI accessible without GPU hardware.
@GPTWare: Uhhhh WTF is this???
Colibri runs the 744B parameter GLM-5.2 MoE model on a laptop with 25GB RAM by activating only ~40B parameters per token and streaming experts from disk, all in a single 2,400-line C file with no GPU required.
4-bit GLM-5.2 (753B MoE) on 4× DGX Spark: 70.8% on Terminal-Bench 2.1 vs 81.0% for the full model
Running a 4-bit quantized version of GLM-5.2 (753B MoE) on 4 DGX Spark machines achieves 70.8% on Terminal-Bench 2.1, compared to 81.0% from the full model.