Going from single GPU to dual GPU is nice but not in the way I expected
Summary
The author shares that upgrading from single 24GB GPU to dual 24GB GPUs improved LLM performance not through larger models but through parallelism, using a main orchestrator agent with smaller subagents, increasing overall throughput for coding tasks.
Similar Articles
@analogalok: Stop blindly trusting the default multi GPU settings for your Local LLMs. You are literally leaving 25% performance on …
Benchmark results comparing layer vs. tensor parallelism in llama.cpp for dual GPU setups: layer mode is 25% faster for prefill (RAG pipelines), while tensor mode is 16% faster for decode (interactive chat).
@no_stp_on_snek: good hardware find for anyone running local: two GPUs as independent instances beat the same two lashed together with t…
A hardware tip for running local AI models: using two GPUs as independent instances is faster than tying them together with tensor-parallel over PCIe, which was 23% slower than a single card alone. Tensor parallel is only beneficial for models too large for one GPU.
@analogalok: I can't afford a $2,000 GPU is officially a dead excuse. yesterday I showed you how to unlock an enterprise grade 16GB …
A guide on using Kaggle's free dual Tesla T4 GPUs (32GB VRAM) to run large LLMs with massive context windows, covering multi-GPU parallelism strategies in llama.cpp.
Dual GPU llama.cpp speedup
A fork of llama.cpp fixes the --split-mode tensor issue with quantized KV caches, achieving up to 40% speed improvement on dual GPU setups without quality loss.
Weird to get near linear scaling by adding another GPU?
A user reports near-linear performance scaling when adding a second RTX 3090 for inference with a Qwen model, achieving roughly 1.8x decode TPS improvement without NVLink.