@MaziyarPanahi: I finally got GLM-5.2 to work an entire 3-year patient chart that only Bonsai 27B was allowed to read 292 encounters li…
Summary
@MaziyarPanahi runs GLM-5.2 and Bonsai 27B locally on a Mac Studio using llama.cpp to process a 3-year patient chart, catching a dangerous drug interaction that was previously flagged but overlooked. The models operate entirely on-device under Apache-2.0, with Bonsai answering queries in ~2s and @PrismML claiming a 1-bit build fits an iPhone.
View Cached Full Text
Cached at: 07/16/26, 08:15 AM
I finally got GLM-5.2 to work an entire 3-year patient chart that only Bonsai 27B was allowed to read
292 encounters live inside Bonsai on my Mac Studio. llama.cpp, Metal, ternary, 7.2GB, Apache-2.0. The chart never leaves the machine.
GLM-5.2 can only ask questions. It asked three. Bonsai answered each in ~2s with 19,398 tokens still cached.
Then it caught the thing buried 17 months back: metformin + iodinated contrast at eGFR 39. Nephrology warned about it in 2025. The ED booked the CT anyway.
A 27B-class model used to need a datacentre. @PrismML say the 1-bit build is 3.9GB and fits an iPhone 17 Pro Max.
The orchestrator never touched the data. That’s the whole point.
What should it read next?
Similar Articles
@AlexFinn: I can't believe this is real I have GLM 5.2 running 100% locally on my Mac Studio. 2 bit quant. The results I'm getting…
A user reports running GLM 5.2 locally on a Mac Studio with 2-bit quantization, claiming it outperforms Opus 4.8 and enables free, private superintelligence for coding and agent tasks.
@pcuenq: GLM 5.2 has just been released Here it's already running with MLX on two Mac Studios (M3 Ultra). This is comparable to …
GLM 5.2, an open-weight AI model comparable to top closed models, has been released and is now running on MLX on two Mac Studios (M3 Ultra).
Show HN: Getting GLM 5.2 running on my slow computer
Colibrì is a pure C inference engine that runs the 744B GLM-5.2 MoE model on consumer hardware with ~25GB RAM by streaming experts from disk, achieving ~2.2-2.8 tokens/second with speculative decoding.
@UnslothAI: GLM-5.2 can now be run locally! The 2-bit model retains ~82% accuracy after we shrunk it from 1.51TB to 238GB (-84% siz…
UnslothAI announces GLM-5.2, Z.ai's strongest open model with 744B parameters, now runnable locally via dynamic GGUF quantization reducing size by ~84% to 239GB while retaining ~82% accuracy. It fits on 256GB Macs and supports long-context, reasoning, and agentic tasks.
@tolak_eth: I wanted to share how we avoided spending roughly $160k/year to host GLM-5.2 with its full 1M context. When GLM-5.2 lau…
Phala avoided $160k/year hosting costs for GLM-5.2 with full 1M context by quantizing MoE experts to 4-bit and keeping critical parts in FP8/BF16, achieving the same benchmark results on a single 8×H200 node and releasing the optimized model GLM-5.2-W4AFP8 on Hugging Face.