A barebones CPU-only inference engine for Qwen 3, written from scratch in pure C
Summary
A minimal CPU-only inference engine for Qwen 3 models implemented from scratch in pure C.
Similar Articles
Running Qwen3.6 35b a3b on 8gb vram and 32gb ram ~190k context
The author shares a high-performance local inference configuration for running Qwen3.6 35B A3B on limited hardware (8GB VRAM, 32GB RAM) using a modified llama.cpp with TurboQuant support, achieving ~37-51 tok/sec with ~190k context.
First attempts at a CPU setup - MS-02 Intel 285hx, trying Qwen3, Qwen3.6 and Gemma4
Testing AI models Qwen3, Qwen3.6, and Gemma4 on a CPU setup using the Intel 285hx processor (MS-02).
Running Qwen3.5-122B on Mac Studio 96GB: Fixed 3 bugs that made long-context inference usable
Fixed three bugs in a qMLX fork for running Qwen3.5-122B on Mac Studio, reducing prefill time from minutes to sub-seconds for long-context inference; open-sourced the fork and benchmark script.
Built a from-scratch BitNet inference engine in pure C — 1.8× faster than bitnet.cpp on Xeon (36 tok/s), zero dependencies [BitNet & Bonsai CPU testers wanted]
Project Zero is a from-scratch C99 LLM inference engine that runs BitNet and Qwen Bonsai-27B on CPU with zero dependencies, achieving 1.8× speedup over bitnet.cpp on Xeon. The project seeks community benchmarks for both models.
Qwen 3.6 27B is the sweet spot for local development
Qwen 3.6 27B is praised as a powerful local AI model that outperforms expectations for general intelligence, suitable for practical tasks like code generation, and runs easily with llama.cpp.