Colibri streaming for Hy3 (Run Hy3 on 10GB (V)RAM)
Summary
A port of Colibri streaming to work with Hy3, enabling the model to run on as little as 10GB of RAM/VRAM instead of the original 25GB.
Similar Articles
@ErickSky: Forget about vLLM, llama.cpp, and expensive GPUs. [colibri] This runs GLM-5.2 (744B MoE) on ~25 GB of RAM with pure C a…
colibri is a pure C inference tool that runs the GLM-5.2 744B MoE model on ~25 GB RAM by streaming experts from disk, eliminating the need for expensive GPUs.
Colibri Hands-on: Running GLM 5.2 (744B) Locally without GPU
Colibri enables running the 744B-parameter GLM 5.2 model locally on CPU, making large-scale AI accessible without GPU hardware.
@danveloper: Now everyone does it
Zane Chen demonstrates Colibri, which runs GLM-5.2 (744B MoE) on a laptop with 25GB RAM using pure C and CPU-only inference, by streaming experts from disk.
Hy3 1Bit 89-93 GB
Announcement of Hy3 1-bit quantized model with 89-93 GB memory footprint.
Show HN: Getting GLM 5.2 running on my slow computer
Colibrì is a pure C inference engine that runs the 744B GLM-5.2 MoE model on consumer hardware with ~25GB RAM by streaming experts from disk, achieving ~2.2-2.8 tokens/second with speculative decoding.