Tag
一位用户分享了他基于 Strix Halo 128GB 主机加解锁版 CMP 170HX eGPU 的本地 LLM 推理方案,运行 Qwen 3.8 Flash Next 与双路 Qwen 27B 子代理,聚合吞吐达到约 1700 tok/s 预填充和 100 tok/s 生成,成本约 3000 美元,已完全取代其云端订阅用于日常 DevOps 工作流。
An update on the LoRA over GGUF series showing that Qwen3.8-Flash-Next (125B-A6B + 51B engram) can now be LoRA-trained in 40 GiB VRAM without CPU offloading, achieving 200 token/s on Strix Halo, with initial GGUF support merged in transformers 5.18 and further kernel optimizations (GGTensile) on the horizon.
Gufo is a vertical local inference engine optimized for AMD Strix Halo (Ryzen AI MAX+ 395 with Radeon 8060S, up to 128 GiB unified memory), with a new Windows pre-package release supporting Qwen3.8, DeepSeek V4 Flash, ASR/TTS, image and video models at high throughput benchmarks.
Gufo is an open-source local inference engine optimized for AMD Strix Halo (Ryzen AI MAX+ 395, 128 GiB unified memory), achieving 3,095 tok/s prefill and up to 190 tok/s decode on a Q6dense quantized Qwen3.6-35B-A3B model, with benchmarks for Qwen3.8 and DeepSeek V4 Flash.
The Asus ROG Flow Z13 gaming tablet with 64GB RAM and AMD Ryzen AI Max Plus 395 is discounted to $2,100 from $2,999.99 at Best Buy, offering solid gaming performance in a portable form factor.
The article announces support for DFlash and PFlash speculative decoding in llama.cpp for AMD Strix Halo iGPUs, demonstrating significant speedups in inference performance using ROCm.
A user seeks recommendations on choosing between AMD Strix Halo and Nvidia DGX Spark hardware for setting up a local network-accessible LLM server.