A community-curated list of budget and legacy GPUs (NVIDIA GTX/Maxwell/Pascal, AMD Radeon and Instinct MI-series) that can run LLM inference through the Vulkan backend on Linux, bypassing the latest CUDA/ROCm support requirements. The AMD Instinct MI50 16GB at around $150 is highlighted as the top value pick.
Help with this list. Give me your recommendation on "not supported anymore" GPUs. Looking for budget and Vulkan friendly options. Most of the GPU are not supported by latest CUDA / ROCm. Often with some witchcraft magic they are able to run with native backend. I prefer the simplicity offered by running Vulkan backend. I'll successfully ran GTX 1080Ti, P102-100, and MI50 on a single system thanks for Vulkan and Linux. Gemini helped with data gathering. Here is the filtered table including only NVIDIA GeForce GTX series GPUs with a memory bandwidth of 256 GB/s or greater and at least 8 GB of VRAM: GPU Model Total VRAM Memory Bandwidth Bus Width Memory Type GeForce GTX 1070 8 GB 256.3 GB/s 256-bit GDDR5 GeForce GTX 1070 Ti 8 GB 256.3 GB/s 256-bit GDDR5 GeForce GTX 1080 8 GB 320.3 GB/s 256-bit GDDR5X GeForce GTX Titan X (Maxwell) 12 GB 336.5 GB/s 384-bit GDDR5 GeForce GTX Titan X (Pascal) 12 GB 480.0 GB/s 384-bit GDDR5X GeForce GTX 1080 Ti 11 GB 484.4 GB/s 352-bit GDDR5X GeForce GTX Titan Xp 12 GB 547.7 GB/s 384-bit GDDR5X The table below lists the specifications for the specialized datacenter, enterprise, and crypto-mining NVIDIA cards you mentioned, applying your rule of maintaining a memory bandwidth greater than or equal to 256 GB/s and filtering for 8 GB or more of VRAM. All five models successfully qualify: GPU Model Total VRAM Memory Bandwidth Bus Width Memory Type Focus/Architecture NVIDIA P104-100 8 GB 320.3 GB/s 256-bit GDDR5X Mining (Pascal) Tesla M40 12 GB / 24 GB 288.4 GB/s 384-bit GDDR5 Datacenter (Maxwell) Tesla P40 24 GB 347.1 GB/s 384-bit GDDR5 Datacenter/AI (Pascal) NVIDIA P102-100 10 GB 400.0 GB/s 320-bit GDDR5X Mining (Pascal) NVIDIA CMP 50HX 10 GB 560.0 GB/s 320-bit GDDR6 Mining (Turing) Here is the updated AMD Radeon desktop GPU table with all RX 6000 and RX 7000 series models removed, while still filtering for a minimum of 8 GB VRAM and 256 GB/s memory bandwidth: GPU Model Total VRAM Memory Bandwidth Bus Width Memory Type Radeon RX 480 (8 GB) 8 GB 256.0 GB/s 256-bit GDDR5 Radeon RX 580 (8 GB) 8 GB 256.0 GB/s 256-bit GDDR5 Radeon RX 590 8 GB 256.0 GB/s 256-bit GDDR5 Radeon R9 390 8 GB 384.0 GB/s 512-bit GDDR5 Radeon R9 390X 8 GB 384.0 GB/s 512-bit GDDR5 Radeon RX Vega 56 8 GB 410.0 GB/s 2048-bit HBM2 Radeon RX 5700 8 GB 448.0 GB/s 256-bit GDDR6 Radeon RX 5700 XT 8 GB 448.0 GB/s 256-bit GDDR6 Radeon RX Vega 64 8 GB 483.8 GB/s 2048-bit HBM2 Radeon VII 16 GB 1,024.0 GB/s 4096-bit HBM2 Note: MI50 and the Radeon VII, Radeon Pro VII share same firmware. GPU Model Total VRAM Memory Bandwidth Bus Width Memory Type Focus / Architecture Radeon Instinct MI25 16 GB 484.0 GB/s 2048-bit HBM2 Machine Learning (Vega 10) Radeon Instinct MI50 16 GB / 32 GB 1,024.0 GB/s 4096-bit HBM2 Datacenter AI (Vega 20) Top Contender: AMD Instinct MI50 16GB. Current used market on MI50 16GB is around $150.
A community-compiled ranking of 76 GPUs benchmarked with llama.cpp's Vulkan backend for local LLM inference, sorted by price-to-performance on the used market. Older wide-bus and HBM2 cards like the P102-100, MI50, and Radeon VII dominate value, while modern mid-range cards are bottlenecked by narrow memory buses.
This post discusses budget GPU options (Radeon VII vs two P100s) for LLM inference with an RX 6800, focusing on VRAM vs speed tradeoffs for MoE models.
The article discusses the search for cost-effective GPU options like Tesla P40 and V100, noting that prices often increase after popular videos, and questions current feasible alternatives.
A comparison of AI inference frameworks ROCm, Vulkan, and vLLM running on dual AMD Radeon 9700 GPUs, likely benchmarking performance for large language models.
A user with limited VRAM hardware reviews the Laguna XS 2.1 model, finding it performant and better than alternatives like Qwen and Gemma for their use case, though not as polished.