Tag
A port of Colibri streaming to work with Hy3, enabling the model to run on as little as 10GB of RAM/VRAM instead of the original 25GB.
Colibri runs the 744B parameter GLM-5.2 MoE model on a laptop with 25GB RAM by activating only ~40B parameters per token and streaming experts from disk, all in a single 2,400-line C file with no GPU required.
Colibri is an inference engine written in pure C, approximately 1300 lines of code, zero dependencies. It can run the 744-billion-parameter GLM-5.2 MoE model on a consumer machine with 25GB RAM, achieved by streaming loaded routing experts and efficient caching, no GPU or Python runtime needed.
Pre-converted int4 quantized weights for the GLM-5.2 744B MoE model, designed to run on consumer hardware with ~25 GB RAM using the colibrì engine.