Tag
A project demonstrates a 2.78-trillion-parameter model that can run on just 8GB of memory using a pure C99 inference engine with no BLAS, frameworks, or GPU reliance, trading speed for extreme memory efficiency (about 26.5 seconds/token at 8GB, 5.6 seconds/token with 128GB+).
A project claims to run Kimi K3, a 2.78T parameter MoE model, on CPU with 8GB RAM by streaming expert weights from NVMe SSD and using MXFP4 compression, trading speed for memory efficiency.
The author optimized a matmul kernel for BitNet's ternary models on CPU, achieving 29x speedup in isolation, but found that the model is memory-bound, resulting in only 6-10% end-to-end gain. The inference engine is available as open-source.
Project Zero is a from-scratch C99 LLM inference engine that runs BitNet and Qwen Bonsai-27B on CPU with zero dependencies, achieving 1.8× speedup over bitnet.cpp on Xeon. The project seeks community benchmarks for both models.
Z-Jail is a lightweight Linux sandbox written in C99 with seven defense layers, no external dependencies, and a tiny binary size (~130 KiB), designed for secure code execution in CI, CTF, and lightweight evaluation.
CParseC is a single-header C99 library for parser combinators inspired by Haskell's Parsec, offering zero-copy parsing, no hidden allocations, and SIMD-optimized combinators. It aims to provide a flexible, performant alternative to handwritten parsers and lex/yacc tools.