Tag
gpu-unlock 1.0.0 发布候选版是一个 Linux 内核模块,通过在 NVIDIA 驱动绑定前解锁 RTX 3090 的 BF16 全速档位,免费实现约 15% 的 prompt processing 与 5% 的 token generation 提速,无需修改驱动或 UEFI。
一位开发者分享了他用二手零件组装的双 RTX 3090 本地 LLM 主机,并询问为何多 GPU 推理性能提升不及预期,以及如何定位瓶颈和优化配置。
RTX 3090 Founders Edition owners running local LLMs may be cooking their VRAM: benchmarks show stock fan/power curves let GDDR6X hit 112°C within seconds of token generation, far above the 96°C continuous rating. The poster recommends capping the card at 270W with fans at 100% (or 240W at 70% fan) at roughly a 30% throughput cost, and notes undervolting doesn't help because Nvidia only applies clock offsets above 300W.
User @loktar00 shares that RTX 3090s can be run at much lower power limits (down to ~120W) than the usual 210-240W range, saving significant heat and power during inference workloads.
A user with a local AI server running one RTX 3090 asks for advice on utilizing a second card effectively as a single user.
In July 2023, the user built a high-end computer with 8x RTX 3090s and an Epyc 7004 CPU to run Llama 2 70B locally, expressing confidence in the future of open-source AI.
An experiment demonstrated that a quantized Qwen 3.8 27B model, running locally on a single RTX 3090 GPU, autonomously pursued optimizing CUDA inference for over three weeks, producing functional kernels and benchmarks while maintaining coherent long-term goal-following.
A user runs a local LLM on an RTX 3090 to create a basic FPS zombie game and plans to improve its graphics overnight.
A user built a game to test the performance limits of the Qwen3.8 27b AI model using an overclocked RTX 3090, and plans to release it as open-source on GitHub for community contribution.
The user describes a hardware configuration with dual RTX 3090 GPUs and an AMD EPYC CPU for running the Qwen3-Flash-Next model using llama.cpp, achieving 38 tokens per second in single-stream inference, and seeks advice on whether to add a third GPU or upgrade the CPU to improve performance, especially for running multiple parallel agents.
Running the DeepSeek-V4-Flash-Vision-Exp 285B MoE model on 10-12x RTX 3090 GPUs achieves over 60-120 tok/s decode speeds with vision and tool support, fully documented for reproducibility.
The post questions why INT8 W8A8 models are not more common among LLM users despite the RTX 3090's native INT8 support, based on hardware usage data from Hugging Face.
Running the Qwen 3.8 27b model with vision on a single RTX 3090 GPU enables 150k context length, and integration with DeepSeek harness allows for plugin development, though a search engine plugin caused a system crash.
A performance report and guide for running the Qwen3.8-27B model on dual RTX 3090 GPUs using vLLM, achieving 308 tokens per second with 32 concurrent streams, with a GitHub repository for local deployment configurations.
A developer achieved up to 381 tok/s inference speed on a single RTX 3090 with the Qwen3.8-27B model using optimized techniques like DFlash2 and prefix caching, particularly effective for document-based tasks like RAG and coding assistants.
A user shares satisfaction with using the Qwen3.8-27B AI model on RTX 3090 GPUs, comparing it favorably to GLM and describing the creation of a bossfight scene.
The author optimized the Qwen3.8-27B model inference on an RTX 3090 GPU, achieving up to 99 tokens per second for single requests and 1150 tps with batch processing through various quantization and optimization techniques, and released the updated code on GitHub.
User reports that Muse Glimmer, a 30B model, fits on a single RTX 3090 with full 256k context using Q4_K_XL quantization and DFlash, achieving 64-124 tok/s and perfect long-context retrieval, unlike comparable models.
A user on a dual RTX 3090 setup discovers that llama.cpp's --split-mode tensor runs prompt processing on CPU (~400 t/s), while switching to --split-mode layer unlocks GPU prompt processing over 1600 t/s, with only a modest drop in token generation speed.
A developer benchmarks Qwen3.6-35B-A3B Q6 on an RTX 3090, showing that offloading eight MoE expert layers to CPU and increasing batch sizes improves prompt processing by 2.36× (564→1330 tok/s) with no decode speed regression, using evolutionary search to find the tuning config.