For Strix Halo - Official llama.cpp isn't ideal and how to highest possible throughput

Reddit r/LocalLLaMA Tools

Summary

The article discusses alternatives to the official llama.cpp for optimized performance on Strix Halo hardware, with specific forks achieving significantly higher throughput for AI inference tasks.

I've been making a lot of comments about optimal setup for Strix Halo (gfx1150) and from my observation, 90% of our community is using offcial llama.cpp for it, which is NOT optimized for Strix Halo at all, official llama.cpp is having extremely hard time to reach 50% hardware theory, wasting the silicon of this device. Here's alternatives that can bring the speed of Strix Halo to a totally different world, I will link to user's sastifaction comment to prove that the result is real: https://github.com/peonist-ai/halogen-flash-server - ~50t/s decode and 1200t/s prefill - 90% theory - Optimized for Strix Halo and Qwen 3.8 Flash Next (Q38FN - The best goto model for Strix Halo) only, if you've heard about Ninfer, this is exactly Ninfer for Strix Halo. https://github.com/myhacsint/llama.cpp/tree/production/strix-halo-qwen4exp-b10685 - Almost 60t/s decode and 600t/s prefill - 80% theory - This is an experiment fork for Q38FN, having very high decode and pretty good prefill https://github.com/halo-box/strix-llama.cpp - Almost 30t/s decode and 800t/s prefill - 75% theory - The latest commit pushed prefill through the roof, this is r/StrixHalo very first fork that has been updating nonstop with a Discord server with a lot of users. Note: Official llama.cpp running Qwen38FN at 2xt/s and 2xxt/s prefill - 50% theory. Hopefully this will be helpful to the Strix Halo users.
Original Article

Similar Articles

2x Strix Halo speed-up with an R9700

Reddit r/LocalLLaMA

A user shares how they achieved a 2x performance boost in AI inference by splitting a large MoE model between a Strix Halo APU and an R9700 GPU, detailing configurations and code modifications.

Is HIPfire worth it for Strix Halo?

Reddit r/LocalLLaMA

The article asks for community evaluations of HIPfire's performance and quality on AMD Strix Halo hardware, specifically regarding long context support compared to llama.cpp.

Strix Halo ROCm + MTP Notes (May 2026)

Reddit r/LocalLLaMA

Technical benchmark comparing ROCm and Vulkan backends for LLM inference on Strix Halo hardware after MTP merged into llama.cpp, revealing ROCm suffers severe performance drops at full context while Vulkan remains stable.