Tip: use this llama.cpp PR to improve PP on Intel ARC
Summary
A llama.cpp PR significantly improves prompt processing speed on Intel ARC GPUs, with benchmark showing speed increase from 245t/s to 462t/s on a B580. The improvement currently works for F16 KV quantization, with plans to support other quants.
Similar Articles
There's a new PR for llamacpp claiming to boost prompt processing with rocm by around 15%, also fixes a bug which makes Q2_K 28x faster
A new PR for llama.cpp boosts prompt processing on ROCm by ~15% and fixes a bug making Q2_K quantization 28x faster.
Llama.cpp PR 8% speed boost
A llama.cpp PR moves sampling from CPU to GPU, yielding 8% faster tokens on an RTX 5090 and ~4% on a Tesla P40 for Qwen3.6-35B inference.
llama.cpp PR reports up to 169% faster quantized-KV decode at 118K context on Intel Battlemage from one SYCL kernel switch
A llama.cpp PR proposes switching quantized-KV decode from the VEC to TILE SYCL kernel on Intel Battlemage, reportedly delivering up to ~169% faster decode at 118K context. The PR is open with caveats and limited independent verification.
Intel Arc Pro B70 llama.cpp benchmarks posted
Benchmark results for Intel Arc Pro B70 GPU running llama.cpp with SYCL on Qwen models show 63 tokens per second performance.
A llama.cpp PR makes Q2_0 3.0–3.6x faster on x86 CPUs, 8B decode goes 2.39 → 8.20 tok/s
A llama.cpp pull request adds an x86 VNNI implementation for Q2_0 × Q8_0 dot products, delivering 3.0–3.6x CPU-only speedups on Bonsai models with bit-for-bit kernel correctness and 99.2% token agreement.