rocm

Tag

Cards List
#rocm

Power Limit test R9700 100w-300w llama.cpp

Reddit r/LocalLLaMA ↗ · 3d ago

A user tested how far the AMD Radeon R9700 Pro's power limit can be lowered (100W-300W) while running llama.cpp with ROCm, finding performance stays near-linear down to 180W and throughput only drops below 140W, allowing much quieter fan operation.

0 favorites 0 likes
#rocm

@AIatAMD: Can @PyTorch + ROCm run an LLM? Start with the stack: model, framework, ROCm software, driver, OS and supported AMD har…

X AI KOLs Following ↗ · 5d ago Cached

AMD publishes an AI Playbook guiding developers through running LLMs like gpt-oss-20b and qwen3.5-4B locally with PyTorch and ROCm, including memory configuration on Windows and Linux and building a document summarizer.

0 favorites 0 likes
#rocm

AMD Loom Fun: Added Radeon 9700 support over the weekend. Benchmarked 48 models with an average 42% speedup over llama.cpp.

Reddit r/LocalLLaMA ↗ · 6d ago

AMD Loom, an experimental inference stack with runtime, kernel compiler and server, added Radeon 9700 support and was benchmarked on 48 LLMs, showing an average 42% speedup over llama.cpp in prefill and decode.

0 favorites 0 likes
#rocm

Qwen3.8-Flash-Next (local) vs Claude

Reddit r/LocalLLaMA ↗ · 2026-11-03

A developer compares running Qwen3.8-Flash-Next locally (UD-Q4_K_XL on llama-server with ROCm) against Claude Opus in Claude Code for reviewing a real Swift commit: Qwen produced 8 findings with zero false positives and caught 2 issues Claude missed, but missed 5 others, making it a solid private first-pass reviewer rather than a final authority.

0 favorites 0 likes
#rocm

AMD's ROCm HRX Loom v0.1 released for targeting AMD CPUs, GPUs & NPUs

Reddit r/artificial ↗ · 2026-10-29 Cached

AMD released the ROCm HRX Loom SDK v0.1, a specialized compiler and IR stack that targets AMD CPUs, GPUs, and NPUs, with support for AMDGPU HSA, SPIR-V/Vulkan, and emerging x86/WebAssembly backends.

0 favorites 0 likes
#rocm

Qwen 27B (IQ4_XS, Q8/Q4 KV) at full 262k context on a 16GB RX 9070 XT: 80+ t/s decode with Adaptive KV Streaming

Reddit r/LocalLLaMA ↗ · 2026-10-17

作者将 adaptive KV streaming 从 CUDA 移植到 ROCm/HIP,并结合 RDNA 4 内核优化,使 Qwen3-27B (IQ4_XS) 能在单张 16GB RX 9070 XT 上以完整 262k 上下文运行,MTP 解码达到 80+ t/s。项目包含大量内核级优化与功耗调优结果,已发布到 GitHub 分支。

0 favorites 0 likes
#rocm

Very impressed with LFM2.5-24-A2B in Pi Agent

Reddit r/LocalLLaMA ↗ · 2026-10-15

一位用户分享了对新发布的 LiquidAI LFM2.5-24B-A2B 模型的实测体验,称其在 llama.cpp + ROCm 上可达约 95 tps,Q8 量化占 30GB 显存并支持 128k 上下文,在 Pi Agent 工作流中表现稳定可靠。

0 favorites 0 likes
#rocm

strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs quality

Reddit r/LocalLLaMA ↗ · 2026-10-12

rdna35-perf branch of strix-llama.cpp 为 Strix Halo (Radeon 8060S) 带来新的 RDNA3.5 flash attention 与 GEMM 内核优化,prompt processing 比 Vulkan 快 21–54%,在 128k 长上下文下比原版 ROCm 最快达 2.7 倍,且输出质量与原版 ROCm 基本一致。

0 favorites 0 likes
#rocm

@no_stp_on_snek: Spent the last few days helping @Italianclownz speed up ROCmFPX, his llama.cpp fork, on a borrowed Strix Halo (Ryzen AI…

X AI KOLs Following ↗ · 2026-10-11 Cached

Developer helped speed up ROCmFPX, a llama.cpp fork, on AMD Strix Halo (Ryzen AI Max+ 395), achieving major prefill and decode speedups with 16 merged PRs and improved quantization accuracy.

0 favorites 0 likes
#rocm

@no_stp_on_snek: The draft vocabulary was opt-in because it only allowed one sequence. It kept one candidate set per model. More than on…

X AI KOLs Timeline ↗ · 2026-10-11 Cached

ROCmFPX 的 llama.cpp 下游项目更新了投机解码的草稿词表实现,改为按序列存储候选集并默认启用(16384 行),将候选行数从 248K 降至 16K,令牌生成速度提升约 11.4%(35.14/35.31 → 39.18/39.32 tok/s)。

0 favorites 0 likes
#rocm

@no_stp_on_snek: MTP drafts were reading 1.27 GB of BF16 LM head just to guess the next token. 248K x 2560, one-row GEMV, every draft st…

X AI KOLs Following ↗ · 2026-10-11 Cached

MTP 投机解码的草稿 LM head 原本每步都要读取 1.27 GB 的 BF16 权重,现改为 Q4_0 量化草稿头(验证与正常解码仍用 BF16),实现约 10.6% 的 token/s 提升,且 greedy 输出 md5 完全一致。相关改动已在 ROCmFPX 的 PR #49 中提交。

0 favorites 0 likes
#rocm

@no_stp_on_snek: The fused HC norm was throwing away a quantize it had already paid for. It writes the 10240-wide stream, then MMQ reads…

X AI KOLs Timeline ↗ · 2026-10-10 Cached

ROCmFPX 的融合 HC norm 内核通过在寄存器中直接写出 Q8_1 副本,省去了 MMQ 重复量化 335MB 数据的开销,在 2.3%–3.6% 的 pp8192 性能提升下保持位级一致。

0 favorites 0 likes
#rocm

@no_stp_on_snek: Gated delta net prefill was still walking tokens one by one. RDNA3 now does 16-token chunks on WMMA. State stays in the…

X AI KOLs Timeline ↗ · 2026-10-10 Cached

ROCmFPX's gated delta net prefill kernel was optimized to process 16-token chunks via RDNA3 WMMA instead of one token at a time, cutting latency from 772µs to 610µs per 512-token call and improving prompt-processing throughput by ~2-4% across configurations, with Strix Halo testing ongoing.

0 favorites 0 likes
#rocm

@no_stp_on_snek: dsv4_hc_post dropped the fused HC norm in ROCmFPX. The old matcher wanted REPEAT / MUL / ADD. qwen4exp builds that comb…

X AI KOLs Following ↗ · 2026-10-10 Cached

ROCmFPX 更新了融合 HC norm 的匹配器,使其能识别 qwen4exp 将 REPEAT/MUL/ADD 合并为单一 op 的组合,从而将 HC 内核每 2048 token 耗时从 416 ms 降至 175 ms。相关修复还解决了导致 llama-perplexity 崩溃的 xn 别名问题,并在 BF16 拷贝路径上移除了变慢的 Q8_0 MMB down proj。

0 favorites 0 likes
#rocm

@no_stp_on_snek: ROCmI4 prefill was stuck on MMQ in ROCmFPX. On Qwen3.8 Flash Next HC-Q8 every expert and every dense projection except …

X AI KOLs Following ↗ · 2026-10-09 Cached

ROCmFPX 解决了预填充阶段 MMQ 阻塞问题,通过寄存器路径将 Q4_0_ROCMI4 权重反量化为 BF16 WMMA GEMM,在 Qwen3.8 Flash Next HC-Q8 上实现 ub 512 约 13%、ub 1024 约 23% 的吞吐提升,稀疏专家处理时间从约 2.3 秒降至 1.8 秒。

0 favorites 0 likes
#rocm

@no_stp_on_snek: Silent garbage prefill on ROCmFPX, fixed. Unified memory + HIP graphs on Strix Halo (gfx1151, ROCm 7.2.1) was capturing…

X AI KOLs Following ↗ · 2026-10-09 Cached

ROCmFPX 修复了在 AMD Strix Halo (gfx1151, ROCm 7.2.1) 上统一内存与 HIP 图捕获导致 top_k 中 hipMemcpy2DAsync 未执行、使 Qwen3.8 Flash Next logits 静默变为垃圾的问题,修复后 KLD 从约 8 降至 0.005,top-1 准确率恢复到 97% 以上且基准速度不受影响。

0 favorites 0 likes
#rocm

HIP and ROCm as first-class citizens in Guix

Lobsters Hottest ↗ · 2026-10-09

GNU Guix 正式将 AMD 的 HIP 与 ROCm 生态作为一等公民支持,使在 Guix 中进行 AMD GPU 计算栈的功能化打包与部署更加顺畅。

0 favorites 0 likes
#rocm

ROCm 10.1: Breaking the Data-Movement Bottleneck

Reddit r/LocalLLaMA ↗ · 2026-10-08 Cached

AMD 发布 ROCm 10.1,重点突破数据搬运瓶颈,通过 AMD Infinity Storage (hipFile)、HIP 运行时的 NUMA 感知内存分配以及 ROCm CLI 和 AMD Skills 等开发者工具提升 GPU 数据传输与训练推理效率。

0 favorites 0 likes
#rocm

Two ~300B MoE models, each on ONE 128 GB mini PC (AMD Strix Halo): GLM-5.3-Flash at ~580 tok/s prefill, MiMo-V2.6-Flash up to 44 tok/s decode. EXL3 weights + open ROCm engine

Reddit r/LocalLLaMA ↗ · 2026-10-03

Yamz Labs released Kyojin, a ROCm inference engine built on ExLlamaV3 for AMD Strix Halo, enabling two ~300B-class MoE models (GLM-5.3-Flash and MiMo-V2.6-Flash) to each fit and run on a single 128 GB mini PC, with EXL3 quantized weights hitting up to 580 tok/s prefill and 44 tok/s speculative decode.

0 favorites 0 likes
#rocm

AMD Ryzen AI Developer Platform OS updated with ROCm 10.0, Linux 7.2

Reddit r/artificial ↗ · 2026-10-02

AMD's Ryzen AI Developer Platform OS has been updated to ROCm 10.0 with Linux 7.2, advancing its open software stack for AI development on Ryzen AI hardware.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback