mistral.rs v0.9.0: up to 1.8x faster CPU decode than llama.cpp on x86 and ARM!
Summary
mistral.rs v0.9.0 delivers up to 1.8x faster CPU decoding than llama.cpp on x86 and ARM, with optimizations across all CPU calibers and full benchmark reproducibility.
Similar Articles
mistral.rs v0.8.2: up to 2.8x faster CUDA inference than llama.cpp on GB10, B200, and H100
Mistral.rs v0.8.2 delivers up to 2.8x faster CUDA inference than llama.cpp on GB10, B200, and H100 GPUs, with benchmarks on Gemma 4 models showing consistent performance improvements across quantization types.
@pupposandro: https://x.com/pupposandro/status/2054241934164492328
The article announces support for DFlash and PFlash speculative decoding in llama.cpp for AMD Strix Halo iGPUs, demonstrating significant speedups in inference performance using ROCm.
A llama.cpp PR makes Q2_0 3.0–3.6x faster on x86 CPUs, 8B decode goes 2.39 → 8.20 tok/s
A llama.cpp pull request adds an x86 VNNI implementation for Q2_0 × Q8_0 dot products, delivering 3.0–3.6x CPU-only speedups on Bonsai models with bit-for-bit kernel correctness and 99.2% token agreement.
llama.cpp MTP speculative simplified for July 2026 big wins on dense models, underwhelming on MoE
An analysis of native MTP speculative decoding in llama.cpp shows significant speedups (1.4x-2.2x) for dense models like Qwen3.6-27B, but underwhelming results on MoE architectures, where gains are minimal due to already low per-step overhead.
llama.cpp PR reports up to 169% faster quantized-KV decode at 118K context on Intel Battlemage from one SYCL kernel switch
A llama.cpp PR proposes switching quantized-KV decode from the VEC to TILE SYCL kernel on Intel Battlemage, reportedly delivering up to ~169% faster decode at 118K context. The PR is open with caveats and limited independent verification.