@linexjlin: K2.6 built a Zig LLM inference engine from scratch on Mac in 12h, pushing Qwen 3.5 0.8B from 15 tok/s to 193.1 tok/s
Summary
Developer wrote a Zig-based LLM inference engine from zero on macOS in 12 hours, boosting Qwen 3.5 0.8B throughput from 15 to 193 tokens per second.
View Cached Full Text
Cached at: 04/21/26, 10:00 AM
K2.6 spent 12 hours writing an LLM inference engine from scratch in Zig on a Mac, and tuned Qwen 3.5 0.8B inference from 15 tok/s to 193.1 tok/s.
Similar Articles
@sanbuphy: K2.6 successfully downloaded and deployed the Qwen3.5-0.8B model locally on a Mac, using the niche Zig language to implement and optimize inference, demonstrating the new model’s generalization ability. After 4,000+ tool calls and 12+ hours of continuous operation, K2.6 iterated 14 times…
K2.6 successfully downloaded and deployed the Qwen3.5-0.8B model locally on a Mac, using the niche Zig language to implement and optimize inference, demonstrating the new model’s generalization ability. After 4,000+ tool calls and 12+ hours of continuous operation, K2.6 iterated 14 times, boosting throughput from ~15 tokens/s to ~193 tokens/s, ultimately achieving 20% faster inference than LM Studio.
@nicekate8888: For the past twenty days, I've been obsessing over one thing — how to make Qwen3.6-27B run fast and well on my Mac. I started with Unsloth Q5, got 18 tok/s, and the fan was roaring. Then I switched to MLX 6bit + DFlash, hitting 22 tok/s, still not fast enough. Eventually I found MTPLX 4bit: 43 tok/s with good quality.
The user shares their experience optimizing Qwen3.6-27B inference speed on a Mac using different quantization methods (Unsloth Q5, MLX 6bit + DFlash, MTPLX 4bit), ultimately reaching 43 tok/s.
@akshay_pachaar: This inference engine runs LLMs 4x faster than llama.cpp: (Qwen3.5-9B at 92 tokens/s on 16GB Mac) I ran Qwen3.5 9B on a…
Benchmark thread showing Uzu, an open-source inference engine from Mirai, running Qwen3.5-9B at 92.1 tokens/s on a 16GB M5 MacBook Pro — about 4x faster than llama.cpp (22 tok/s) and MLX (25.1 tok/s) — by using tree-based speculative decoding with a Weaver drafter to break the memory-bandwidth ceiling of ordinary local decoding.
@unwind_ai_: What the hell China Kimi K2.6 wrote a local inference runtime in Zig, on a Mac, autonomously. ended up 20% faster than …
Kimi K2.6 autonomously wrote a Zig-based local inference runtime on Mac that is 20% faster than LM Studio after 14 iterations and 4,000+ tool calls, all open-sourced.
@LinusEkenstam: Quite a big deal. 1.35x faster inference than MLX-LM 1.23x faster at prefill having the hybrid option to pick from insi…
Perplexity has open-sourced Lily, a local inference engine optimized for Qwen3.6-35B-A3B on Apple Silicon, achieving 1.35x faster inference than MLX-LM.