strix-halo

The memory shortage has driven up prices across every device capable of running large models locally, rendering all the numbers in old buying guides obsolete. This piece compares the trade-offs between Strix Halo mini PCs, the Mac Studio M5 Max, and the DGX Spark in terms of decode/prefill speeds and pricing, arguing that MoE models are the whole point of owning a 128 GB machine.

A community discussion asking whether AMD Strix Halo systems (GMKtec EVO-X2, Framework Desktop) are the closest thing to an ideal local LLM box, weighing unified memory against GPU, NPU, and FPGA options for running 20B-32B models at high-quality quants.

A user recommends a GitHub project for faster inference of Qwen 3.8 Flash Next on Strix Halo, showing benchmark speeds that significantly outperform llama.cpp.

The article questions whether now is a good time to purchase Strix Halo hardware considering the upcoming release of 495+ and RAM crisis, and requests recommendations for quiet boxes to run local LLMs like Qwen models.

Updated size chart for Strix Halo mini PCs, final for model 395 before the release of model 495.

The author shares their experience running local AI models on a Framework desktop with Strix Halo and 128GB unified memory, preferring Qwen models for coding, and asks for recommendations on better hardware utilization.

The article discusses alternatives to the official llama.cpp for optimized performance on Strix Halo hardware, with specific forks achieving significantly higher throughput for AI inference tasks.

The article presents benchmark results for DeepSeek V4 Flash 0731 on Strix Halo hardware, showing performance with different draft models and n_max settings, concluding that n_max=3 offers the best speed balance.

The author tested the Qwen3.8-27B Q8_0 AI model on a ROG Flow Z13 with Ryzen AI Max+ 395, achieving impressive local inference performance in generating a flight simulator using Lemonade Server and llama.cpp with speculative decoding.

Tweet reports that Ling 3.0 Flash on AMD Strix Halo is significantly faster than Qwen-122b using ROCm-optimized formats, but notes tool calls are broken in certain harnesses.

This article compares the energy cost of running AI models on Strix Halo (max $0.48/day) versus Nvidia A6000, highlighting power efficiency and versatility.

This article explains that for RAG applications, prefill speed matters more than decode speed, and discusses why AMD's Strix Halo APU struggles with interactive use cases.

A setup guide for using a custom Docker/Podman toolbox with ROCm/RCCL RDMA support to cluster two AMD Strix Halo nodes, enabling vLLM with tensor parallelism across 256GB unified memory.

This article evaluates the performance of running GLM 5.2 (IQ2M quantized version) on Dual Strix Halo (256GB VRAM). The generation speed is only about 7 tokens/s, and coding tasks take twice as long as DeepSeek V4 Flash. Its cost-performance ratio is far inferior to other models, so it is not recommended for use with this hardware configuration.

The NPU on AMD Strix Halo devices is now usable for AI inference, enabling hybrid mode that combines NPU and iGPU for faster prompt processing. Tools like Lemonade and AMD's ROCm software make this possible.

Testing DeepSeek v4 Flash on the AMD Ryzen AI Max+ 395 with 128GB RAM achieves ~15 TPS for a 284B MoE model (13B active) locally, costing $3,000 versus $25,000+ for a datacenter setup, highlighting the feasibility of running large models on consumer hardware.

Comparison of four large language models (≤120B parameters) on deep context performance using Strix Halo hardware. Nemotron Super excels in prompt processing speed at deep context depths compared to GPT-OSS and Qwen models.

xdna-top is a terminal monitor that shows both NPU and iGPU activity on Ryzen AI Max/Strix Halo systems, providing an honest view of NPU counter deltas instead of fake utilization percentages.

Antirez is consolidating community contributions from DwarfStar to improve Strix Halo support, with final QA and merge expected soon.

Noticing price increases for DGX Spark and Strix Halo hardware, suggesting rising demand for local AI computing.