Tag
A user is evaluating whether to add a 4060 Ti 16GB GPU to a multi-GPU setup with 3060s for AI model parallelism and gaming, weighing the benefits of extra VRAM against potential memory bandwidth limitations.
The article provides a guide comparing GPUs based on cost per gigabyte and bandwidth to help optimize script performance.
A user is considering upgrading from a 5060ti 16GB to either 1x32GB V100 or 2x16GB V100 for better performance with the Qwen 3.8 model using llama.cpp, and asking for other options in a similar price range.
The user is comparing GPU options like R9700, Mi210, and 4080S to achieve 128GB VRAM for running multiple AI models in parallel, considering factors like cost, performance, and compatibility.
A user seeks advice on building a budget home AI server with 32-48GB VRAM, debating between AMD RX 9060 XT and Nvidia RTX 5060 Ti GPUs, and whether to use AM5 or used EPYC platforms for local LLM inference and large MoE model offloading.
A user shares benchmark results comparing 6x AMD MI50 (96GB) vs 6x NVIDIA P40 (144GB) running MiniMax M2.7 REAP 139B Q3_K_L model, showing P40 faster in prompt processing but MI50 faster in token generation.
Comparison between AMD RX 9070 XT and NVIDIA RTX 5070 Ti graphics cards, likely covering performance, features, and value.
The author compares various GPUs for LLM inference, critiquing common benchmarks and emphasizing the importance of prefill performance over generation speed, offering recommendations for different budgets and use cases.
The author ran 55 inference benchmark runs across Strix Halo, RTX 3090, and RTX 5070 with multiple backends, revealing that memory bandwidth dominates decode speed, the RTX 5070 beats the 3090 on small models, and reasoning models appear ~5x slower due to hidden reasoning content.
A user reports that llama.cpp with ROCm consumes significantly more VRAM for the KV cache than the Vulkan backend, despite identical model and settings, prompting investigation into potential causes.