I made a custom llama.cpp build optimized for 7900xtx (one or two). for qwen 3.8 next and 27B. includes optimizations for PciE x4 and tensor parallel. read inside! (no AI slop)

I found a lot of room on the table for these cards so I decided to make a specialized build to squeeze all I could. first The results: qwen 3.8 next Q3_K_XL: 920tk/s pp8192 (2 cards, ram offload), 24/27 tk/s on prose, 40+ tk/s on code with MTP but without MoE expert cache (which IS included if yuo want, read below) qwen 3.8 27B Q8_0: 1600 tk/s pp8192, 60/65 tk/s prose, 100+ tk/s code, tensor parallel. this is measured with ONE CARD BEHIND the chipset on X4. with cards on a good PciE x8 on cpu I think more is reachable! let me know. qwen 3.6 27B Q4_K_M: (single card) --> this was not the optimization target but I did a test with MTP, PP8192 1020tk/s; prose about 58/60 tk/s ; code 75/80 tk/s --> Dflash probably here could push much faster, I think above 100tk/s My objecives: fast prompt processing on 3.8 Next to make it actually usable for code enable and optimize tensor parallel on two cards where 1 is behind chipset, for max speed on qwen 27B Q8_0 this build includes stuff like: Data compression for the PciE transmission. data between cards is compressed to Q8_0 to save bandwidth (optional) P2P enabled also for cards sitting benhind the chipset (custom HIP allreduce path), so you can use tensor parallel even on setups like ... mine all the fixes and features from RDNA_BOOST including --adaptive-mtp, so it automatically adapts MTP n-max based on acceptance A LOT of AMD speed tunings and overhauls which are NOT upstream already, kernel tweaks etc... good stuff. many are labelled for RDNA3.5 but they DO work on RDNA3. MoE expert cache if you want to use it. personally I don't like It because i much prefer fast prompt processing. but hey it's there. latest PRs from llama.cpp that are not yet upstream, which speed up various things, like --lazy-mode on-direct to massively speed up Ngram table reads (and thus, PP) DFLASH2 support on tensor parallel (!) For a complete list check the Readme. Here it is: https://github.com/nasone32/llama.cpp-RDNA3-7900xtx-opt notes: don't use Q8_K_XL because it's slower, for the 27B model this is heavily optimized for INT8 calculations. feel free to tweak the context, 200k f16 should be reachable on 2 cards, compressing KV to q8_0 is fine but slower. the custom HIP allreduce works for 2 cards, if you have 3/4 cards, compile with RCCL as usual and skip the allreduce=internal flag, should work fine but untested. This is tested on UBUNTU 24 and rocm 7.14; if your system is different or encounter problems use a LLM to solve them, because I WILL NOT offer support nor update this build, these things hopefully will be merged and this frankenstein can die peacefully :) enjoy EDIT: Summary of most impacting patches: PR / change Area PP / Prefill TG / Decode AMD #39 MoE MMQ sizing RDNA3 +14.32% Flash +5.38% Flash AMD #63 compacted MoE tiling RDNA3 +4.39% Flash +0.86% Flash AMD #52 + qwen4exp port channels-major GDN +5.93% +7.21% #28213 QSA sparse-attention decode +1.42% Flash +1.17% QSA d8192 #28313 TOP_K ROCm wave32/hybrid -6.45% Flash +11.82% Flash #27861 GPU MoE expert cache — +19.95% #28136 + on-direct/mmap lazy PLE/load path +58.88% Flash -1.52%