@Snixtp: More efficiency tests on a single 3090 TL;DR: - I tested 8 local LLMs on a single RTX 3090, power limit from 100W to 45…
Summary
The article presents benchmark results for 8 local LLMs on an RTX 3090, showing that power efficiency peaks around 225W, with diminishing returns at maximum power.
Similar Articles
Finding the 4x 3090 Sweet Spot
A user shares power limit testing on a 4x RTX 3090 setup running Qwen3.6-27B with vLLM, finding 220W as the sweet spot for peak efficiency with minimal throughput loss.
Scrambling to max StrixHalo (+NVLink dual eGPU 3090 mod)
A user details their modding and benchmarking of an AMD Strix Halo system with dual RTX 3090 eGPUs and NVLink, finding improvements in LLM inference speed for dense models, especially with vLLM, and discusses power efficiency trade-offs.
Dual 3090 setup: 400 pp t/s to 1600 pp t/s on Qwen 3.6 27B... with slightly lower tps.
A user on a dual RTX 3090 setup discovers that llama.cpp's --split-mode tensor runs prompt processing on CPU (~400 t/s), while switching to --split-mode layer unlocks GPU prompt processing over 1600 t/s, with only a modest drop in token generation speed.
@ItsmeAjayKV: So last 7 day analysis of local LLM's i ran on my home-lab. MoE models now rule my 3090. Top used Model: @thinkymachine…
A user shares 7-day stats of local LLMs running on a home-lab, noting MoE models dominate their RTX 3090, with Inkling-small and Ling-3.0-flash performing well, and plans to open-source their dashboard.
[Benchmark] 5090RTX: Promt Parsing, Token Generation and Power Level
A user benchmarks the Nvidia 5090 RTX GPU for LLM inference using llama.cpp, measuring prompt processing and token generation at various power levels, finding that prompt processing is more sensitive to power limits than token generation, and noting differences from the 4090 RTX.