Tag
A developer built Red Curtain to run Fish Audio's Drama 3 text-to-speech model (35B MoE, ~3B active) on a 16 GB Radeon RX 9070 despite its official 48 GB NVIDIA requirement, offloading FP8 experts to system RAM and caching active ones on the GPU. The tool ships with a web UI supporting voice cloning and multi-speaker scripts, and the author shares tuning tips and a known first-line silence issue.
A user benchmarks Qwen-3.8 Flash-Next in Q4 quantization running on Deadcode's Radiance inference engine, achieving 176 t/s decode and 6,450 t/s prefill on a custom PC with 2xR9700 GPUs and 64GB DDR5, using the BetterBench throughput tool.
ROCmFPX's gated delta net prefill kernel was optimized to process 16-token chunks via RDNA3 WMMA instead of one token at a time, cutting latency from 772µs to 610µs per 512-token call and improving prompt-processing throughput by ~2-4% across configurations, with Strix Halo testing ongoing.
ROCmFPX 更新了融合 HC norm 的匹配器，使其能识别 qwen4exp 将 REPEAT/MUL/ADD 合并为单一 op 的组合，从而将 HC 内核每 2048 token 耗时从 416 ms 降至 175 ms。相关修复还解决了导致 llama-perplexity 崩溃的 xn 别名问题，并在 BF16 拷贝路径上移除了变慢的 Q8_0 MMB down proj。
ROCmFPX 解决了预填充阶段 MMQ 阻塞问题，通过寄存器路径将 Q4_0_ROCMI4 权重反量化为 BF16 WMMA GEMM，在 Qwen3.8 Flash Next HC-Q8 上实现 ub 512 约 13%、ub 1024 约 23% 的吞吐提升，稀疏专家处理时间从约 2.3 秒降至 1.8 秒。
ROCmFPX 修复了在 AMD Strix Halo (gfx1151, ROCm 7.2.1) 上统一内存与 HIP 图捕获导致 top_k 中 hipMemcpy2DAsync 未执行、使 Qwen3.8 Flash Next logits 静默变为垃圾的问题，修复后 KLD 从约 8 降至 0.005，top-1 准确率恢复到 97% 以上且基准速度不受影响。
GNU Guix 正式将 AMD 的 HIP 与 ROCm 生态作为一等公民支持，使在 Guix 中进行 AMD GPU 计算栈的功能化打包与部署更加顺畅。
LemonSeed Studio 0.5.8 brings on-device LLM inference to iPadOS via an eGPU (AMD R9700 in a Thunderbolt enclosure), running Qwen3.8-27B Q4 at ~159 tok/s with DFlash2 speculative decoding across iPadOS, macOS and Linux using the unified LemonSeed Engine.
The author notes that developers are already moving away from vLLM and llama.cpp to build custom inference engines from scratch for AMD GPUs, because general-purpose engines are only tapping about 200 GB of a 600 GB memory bandwidth budget. As AI-assisted coding becomes widespread, projects like these will proliferate, ushering in the most egalitarian era in software development.
A user shares and asks for feedback on running Qwen3.8-Flash-Next in EXL3 5.05bpw on a single Radeon RX 9700 (gfx12) with mixed GPU/CPU expert offloading, reporting ~863 t/s prefill and ~35 t/s decode at 230k context.
Appreciation post for tcclaviger adding expert RAM offloading support to vLLM, enabling running large AI models like DeepSeek-V4-Flash-Vision-Exp on local setups with multiple GPUs.
BeeLlama v0.4.7 is released, offering gigabytes of VRAM savings through independent drafter ubatch sizing for MTP and DFlash, along with major ROCm and CUDA performance enhancements for AI inference.
This article demonstrates a setup where an AMD Radeon AI Pro R9700 GPU is used on a MacBook Pro via Thunderbolt 5 with a custom driver and LemonSeed-Engine for running LLMs like Qwen3.6-3.8.
Optimized performance for running the Qwen3.8 27b NVFP4 model on a single AMD Radeon R9700 GPU, achieving up to 153 tokens per second in decode and 3,619 tokens per second in prefill with improved concurrency.
The article introduces AMDKernelVault, an open dataset and training framework for AMD GPU kernel optimization, featuring large-scale HIP and Triton kernels and agent-driven pipelines for generating and validating kernels.
This tool provides a reproducible Windows setup to run CUDA-targeted applications on AMD GPUs using ZLUDA and AMD HIP/ROCm, validated with RX 9060 XT and LibTorch workloads.
This pull request adds missing AMD GCN MMQ configuration to ggml-cuda for HIP, enhancing prefill performance for RDNA2 GPUs such as MI50 and MI60 in the llama.cpp inference library.
VoxGen is a new lightweight native inference engine for VoxCPM2 models, optimized for AMD cards using Rust and Vulkan compute to enhance performance and remove Python/PyTorch dependencies.
The article describes the optimization of AMD MI350X GPUs for running the Qwen3.6-35B-A3B LLM, achieving high output token throughput and open-sourcing the kernel to improve performance.
The user fixed crashes when running large AI models on an AMD GPU in headless Linux by disabling runtime power management, preventing model weights from being dumped into limited RAM and causing OOM errors.