GLM and I created a llama.cpp fork optimized for AMD GFX906 (Mi50, Mi60, Radeon VII, GCN HIP) - Machine Learning, LLMs, & AI
Summary
A fork of llama.cpp has been created and optimized for AMD GFX906 GPUs, improving performance on Mi50, Mi60, Radeon VII, and GCN HIP for machine learning and LLM applications.
Similar Articles
AMD Users: Have you tried the llamma.cpp AMD-Ecosystem branch? Up to 2x PP Speed
The article discusses an AMD-specific branch of llama.cpp that significantly boosts prompt processing speed for AMD users, with up to 2x faster performance on dense models using ROCm/Hip, though with some trade-offs in other metrics.
llama.cpp B9387 Significant AMD/ROCm PP Update
llama.cpp version b9387 introduces MFMA support for AMD CDNA architecture (MI100, MI200, MI300 series), improving processing pipeline performance on datacenter AMD GPUs.
llama.cpp expert-pool fork for Qwen 3.8 Flash next IQ4 + 16Gb VRAM tested on MI50 gfx906 with parameters
An unofficial fork of llama.cpp introduces a persistent expert pool for MoE models, optimized to reduce expert re-copies over PCIe on 16GB AMD gfx906 GPUs, thereby improving decode throughput for large context lengths.
ggml-cuda: hip: add missing AMD GCN MMQ config by thelittlefireman · Pull Request #27841 · ggml-org/llama.cpp - PP improvements for RDNA2(MI50, MI60)
This pull request adds missing AMD GCN MMQ configuration to ggml-cuda for HIP, enhancing prefill performance for RDNA2 GPUs such as MI50 and MI60 in the llama.cpp inference library.
If you have a 3090, or other 30xx for local LLMs, I have something for you
The article presents a custom fork of llama.cpp optimized for NVIDIA Ampere architecture GPUs like the 3090, achieving over 90 tokens per second for 27B parameter models with large context windows, making local inference faster than API speeds.