AMD Loom Fun: Added Radeon 9700 support over the weekend. Benchmarked 48 models with an average 42% speedup over llama.cpp.

Reddit r/LocalLLaMA Tools

Summary

AMD Loom, an experimental inference stack with runtime, kernel compiler and server, added Radeon 9700 support and was benchmarked on 48 LLMs, showing an average 42% speedup over llama.cpp in prefill and decode.

On Friday I posted about AMD Loom, a new experimental inference stack that includes a runtime, kernel compiler, and inference server. People can use the entire thing as a vertical stack, or the compiler/runtime can be integrated into any other inference project (I'm working on Gufo right now, the team is also working on a llamacpp backend). I started with Strix Halo, where I ported the top-51 trending Q4_K_M LLMs from Hugging Face and got an average of 40% speedup over llama.cpp across decode and prefill metrics (picking the best results from ROCm or Vulkan backends). I'm back today with Radeon 9700 results. I took the same 51 LLMs, cut it down to the 48 that fit in 32 GB of VRAM, and got an average of 42% speedup. Some of the data is posted to the table below, and all data is available to browse on https://loom-fun.github.io/loom-dev/index.html?gpu=radeon-r9700 All kernels were generated via agent skills: loom-skills repo: https://github.com/loom-fun/loom-skills deterministic SDK (runtime, compiler, server): https://github.com/ROCm/hrx-system Model Loom pp32k (tok/s) llama.cpp pp32k (tok/s) Loom short decode (tok/s) llama.cpp short decode (tok/s) Flint 6,476 3,190 255 152 gemma-3-1b-it n/a n/a 400 322 gemma-4-12B-it 2,811 1,537 127 67.3 gemma-4-26B-A4B-it doesn't fit (32 GB) doesn't fit (32 GB) 151 72.2 gemma-4-31B-it doesn't fit (32 GB) doesn't fit (32 GB) 62.9 43.2 gemma-4-E2B-it 14,289 14,519 272 234 gemma-4-E4B-it 7,021 6,324 192 173 gpt-oss-20b 5,319 4,368 195 182 HyperCLOVAX-SEED-Text-Instruct-1.5B 11,653 8,078 362 344 Laguna-XS-2.1 doesn't fit (32 GB) doesn't fit (32 GB) 102 — LFM2.5-24B-A2B 10,099 5,479 244 197 LFM3-1.7B 20,062 13,614 372 261 LFM3-Omni-2B 20,220 13,435 368 299 Llama-3.2-3B-Instruct 5,687 3,908 228 211 Mellum2.1-12B-A2.5B-Thinking 8,822 5,486 224 149 MiniCPM-V-4.7-35B-A3B 5,717 3,540 135 113 Muse-Glimmer-30B 1,658 1,287 72.6 54.8 Nanbeige4.3-33B 946 666 29.4 28.7 NVIDIA-Nemotron-3.5-Bolt-E4B-4GB 6,932 3,656 225 143 Ornith-1.0-35B 5,628 3,320 152 127 Ornith-1.0-9B 4,886 3,391 100 88.0 Ornith-1.5-35B-A3B 5,586 3,136 145 78.8 Ornith-1.5-9B 4,767 3,156 151 90.5 Qwen2.5-0.5B-Instruct 26,022 17,812 646 529 Qwen2.5-1.5B-Instruct 12,546 8,908 360 331 Qwen2.5-32B-Instruct doesn't fit (32 GB) doesn't fit (32 GB) 30.0 29.2 Qwen2.5-7B-Instruct 3,810 2,679 118 116 Qwen2.5-VL-7B-Instruct 3,768 2,648 116 115 Qwen3-0.6B 13,020 8,631 506 480 Qwen3-1.7B 10,313 7,014 316 297 Qwen3-14B 1,944 1,167 65.0 62.6 Qwen3-30B-A3B-Instruct-2507 3,116 2,060 177 169 Qwen3-4B 3,770 2,826 231 130 Qwen3-4B-Instruct-2507 3,795 2,856 180 166 Qwen3-8B 3,076 2,287 163 94.7 Qwen3-Coder-30B-A3B-Instruct 2,921 2,107 157 54.8 Qwen3-VL-4B-Instruct 3,795 2,805 178 163 Qwen3-VL-8B-Instruct 3,085 2,265 112 103 Qwen3.5-0.8B 24,341 16,959 463 338 Qwen3.5-2B 16,990 12,165 318 236 Qwen3.5-35B-A3B 5,652 3,410 190 112 Qwen3.5-4B 7,030 4,897 200 123 Qwen3.5-9B 4,724 3,336 146 92.7 Qwen3.6-27B 1,503 1,164 60.1 38.9 Qwen3.6-35B-A3B 5,635 3,463 209 123 Qwen3.8-27B 1,592 1,026 73.5 55.3 Sage-A1.5 5,652 3,131 188 153 Spark-X2.5-4B 6,858 4,727 162 131 Spark-X2.6-4B 6,913 4,841 162 132
Original Article

Similar Articles

llama.cpp B9387 Significant AMD/ROCm PP Update

Reddit r/LocalLLaMA

llama.cpp version b9387 introduces MFMA support for AMD CDNA architecture (MI100, MI200, MI300 series), improving processing pipeline performance on datacenter AMD GPUs.