Modded RTX 4090 48GB vs Radeon AI Pro R9700 vs Arc Pro B70 for local coding LLMs?

Reddit r/LocalLLaMA News
hardware-comparison gpu llm-inference local-ai cuda rocm coding-assistant

Summary

A user seeks advice on choosing between a modded RTX 4090 48GB, dual AMD Radeon AI Pro R9700, or dual Intel Arc Pro B70 for running local coding LLMs, highlighting trade-offs in price, VRAM, software ecosystem, and inference speed.

Building a personal rig mainly for running coding LLMs locally (inference,maybe light fine-tuning). Already have the motherboard/rest of the platform sorted — just deciding on the GPU. Three options I keep coming back to: Modded RTX 4090 48GB (Chinese clamshell mod) — I have an eBay offer at $3,500. 48GB GDDR6X, full AD102, ~1TB/s bandwidth, and obviously CUDA. The catch: third-party firmware, no real warranty, blower cooler, and general "is this thing reliable long-term" nerves. 2x AMD Radeon AI Pro R9700 32GB — RDNA4, 640 GB/s, PCIe 5.0, official card with a warranty, ~$1,300. ROCm is maturing but not CUDA. 2x Intel Arc Pro B70 32GB — Battlemage, 608 GB/s, 367 TOPS INT8, $949 MSRP (street ~$1,080). Cheapest, newest, but oneAPI/OpenVINO and driver maturity are the question marks. No FP4 support. Anyone running any of these for a similar workload — how's the real-world experience, especially CUDA-vs-ROCm-vs-oneAPI friction for coding stacks? I am looking at a decent speed around 30-40 tps I already have a dgx spark which runs fine but I am not happy with the speed at I cannot seem to go beyond 20 tps.
Original Article

Similar Articles

4xR9700, 2xMi210 or 4x4080S 32G

Reddit r/LocalLLaMA

The user is comparing GPU options like R9700, Mi210, and 4080S to achieve 128GB VRAM for running multiple AI models in parallel, considering factors like cost, performance, and compatibility.

Nvidia RTX 3090 vs Intel Arc Pro B70 llama.cpp Benchmarks

Reddit r/LocalLLaMA

Community benchmark shows Intel Arc Pro B70 averages ~71% slower prompt processing and ~54% slower token generation than RTX 3090 under llama.cpp, with SYCL backend sometimes beating Vulkan on the same card.

ROCm vs Vulkan vs vLLM on Dual R9700's

Reddit r/LocalLLaMA

A comparison of AI inference frameworks ROCm, Vulkan, and vLLM running on dual AMD Radeon 9700 GPUs, likely benchmarking performance for large language models.