Tag
作者发布 Slipstream——一款针对 Apple Silicon 的 C++ Metal 推理引擎，支持原生 SSD 专家流式加载与投机草稿解码，可在 64GB Mac 上以 41–52 tok/s 运行 95.5 GiB 的 Qwen3.8-Flash-Next 模型，比 llama.cpp 快 1.76 倍，且 130k 长上下文下速度不衰减。
The author optimized the Qwen3.8-Flash-Next model to run on a 64GB Mac using expert streaming and other techniques, achieving ~27 tok/s by publishing a checkpoint and a llama.cpp fork.
Cherenkov is a new inference engine for Apple Silicon that enables efficient memory-constrained inference of large AI models like Qwen3.8-Flash-Next using predictive expert streaming and mixed-precision execution.
colibri is a pure C inference tool that runs the GLM-5.2 744B MoE model on ~25 GB RAM by streaming experts from disk, eliminating the need for expensive GPUs.