A user seeks advice on building a budget home AI server with 32-48GB VRAM, debating between AMD RX 9060 XT and Nvidia RTX 5060 Ti GPUs, and whether to use AM5 or used EPYC platforms for local LLM inference and large MoE model offloading.
I’m planning a dedicated home AI server, mainly for local LLM inference, agents/tool use, Docker services, and eventually larger MoE models with CPU offload. My plan is to start with 2x 16GB GPUs = 32GB VRAM, but I want to build the platform from day one knowing that I’ll almost certainly add a third identical GPU later for 48GB total VRAM. I’m in Brazil, so pricing is a bit weird. Converting roughly to USD, these are the deals I’m currently seeing: RX 9060 XT 16GB: around $490 each new/sealed on the used market 2x = ~$980 3x = ~$1,475 RTX 5060 Ti 16GB: around $710 each new/sealed 2x = ~$1,420 3x = ~$2,125 So going AMD saves me roughly $650 on the final 48GB setup, which is significant. The NVIDIA option is obviously more attractive from the software side because of CUDA, wider framework support, NVFP4, etc. The AMD option is mostly tempting because 48GB of relatively new RDNA4 VRAM for ~$1.5k sounds very hard to ignore. The rest of the AM5 build I’m considering is roughly: - Ryzen 9 9900X: ~$430 new or a used Ryzen 9 7900 for around ~$300 - ASUS ProArt X870E-Creator: ~$790 new, possibly ~$600 used - 128GB DDR5 2x64GB: ~$1,020 - 96GB 2x48GB alternative: ~$785 - Good 1000-1200W PSU: ~$200-240 - 1TB NVMe Gen4: ~$150 - Large case + cooler + fans: ~$150-200 The expensive part is the motherboard/RAM platform rather than the GPUs. The ProArt is attractive because with three GPUs I’d get roughly: - GPU 1: PCIe 5.0 x8 - GPU 2: PCIe 5.0 x8 - GPU 3: PCIe 4.0 x4 through chipset For llama.cpp layer splitting, I assume that should be reasonably usable, but I’m more concerned about the third x4 link if I want tensor parallelism or vLLM. My other big goal is eventually experimenting with very large MoE models, including DeepSeek-class models, where a lot of expert weights could remain in system RAM while attention / active layers are GPU-offloaded. That’s why I’m leaning toward 128GB RAM rather than 64/96GB. However, this has made me wonder whether AM5 is actually the wrong platform. I’m also considering a used EPYC 7002 / SP3 setup, something like an EPYC 7302P + Supermicro H12SSL-class board + 128GB ECC DDR4. That would give me: - 128 PCIe lanes - 3 GPUs without awkward x8/x8/x4 lane sharing - 8-channel memory - much higher system RAM bandwidth - cheap used ECC RDIMMs The downsides would obviously be older CPU architecture, worse single-thread performance, higher idle consumption, and a more server-like/less convenient platform. So my main questions are: 1. At ~$490 vs ~$710 per 16GB GPU, would you pick 3x RX 9060 XT or 3x RTX 5060 Ti for a dedicated Linux inference server? 2. How mature is multi-GPU ROCm on RDNA4 right now in practice, especially with llama.cpp and vLLM? 3. Is PCIe 4.0 x4 for the third GPU actually a major issue for LLM inference, or mostly a concern for tensor parallel workloads? 4. For large MoE models with CPU offload, would you rather have 128GB dual-channel DDR5 on AM5 or 128GB 8-channel DDR4 on EPYC? 5. Is 128GB system RAM worth it here, or would you just buy 96GB and put the savings elsewhere? 6. Am I overbuilding the CPU/motherboard side for 3x relatively inexpensive 16GB GPUs? 7. Any better budget platform for 3 GPUs + lots of RAM bandwidth that I should be looking at? I’m not chasing maximum benchmark numbers. The goal is a cost-efficient always-on local AI server that can run 27B/35B-class models comfortably now and leave room for much larger quantized/MoE models later. Would especially love to hear from anyone actually running 2-3x 9060 XT, 5060 Ti, or a used EPYC multi-GPU setup.
A user seeks advice on choosing between a modded RTX 4090 48GB, dual AMD Radeon AI Pro R9700, or dual Intel Arc Pro B70 for running local coding LLMs, highlighting trade-offs in price, VRAM, software ecosystem, and inference speed.
A detailed analysis on whether to run AI models locally or via API, covering hardware options like RTX 5090, RTX PRO 6000, and DGX Spark, with emphasis on memory vs bandwidth trade-offs, cost considerations, and privacy needs.
This post discusses budget GPU options (Radeon VII vs two P100s) for LLM inference with an RX 6800, focusing on VRAM vs speed tradeoffs for MoE models.
A user seeks advice on purchasing a failover inference server under $150K to serve 300 people, discussing options like used H100s, RTX Pro 6000, and DGX Station for running 122b AWQ models with vLLM.