vram

Tweet highlights running the Qwen 3.8 27B model locally on an RTX 5090 system with 32GB VRAM, achieving 115 tokens/sec, and notes the official BF16 checkpoint is 55.6GB.

A user outlines a step-by-step plan to achieve 200GB of VRAM by combining multiple NVIDIA GPUs in a custom PC build, addressing purchase, installation, and power management.

The author discusses the low adoption of the qwen 3.8 27b model based on download counts and estimates that very few users have the high-VRAM GPUs needed for productive local LLM development.

A hands-on benchmark of Nvidia CMP170HX mining cards repurposed as 64GB VRAM AI inference accelerators, showing they can run large local LLMs like DeepSeek V4-Flash and gpt-oss-120B at useful speeds, with caveats around Ampere-class throughput and PCIe Gen2 x4 connectivity.

Discussion about running LLMs on 12GB VRAM, noting current focus on dense models like Muse Glimmer 30B and Qwen 3.8 27B, and questioning whether upgrading to 24GB VRAM is needed.

Leaked specs suggest Nvidia's rumored 50-series Super refresh will increase VRAM on several GPUs using new 3GB GDDR7 modules, making them more attractive for local LLM use, though pricing remains a concern.

User reports that Muse Glimmer, a 30B model, fits on a single RTX 3090 with full 256k context using Q4_K_XL quantization and DFlash, achieving 64-124 tok/s and perfect long-context retrieval, unlike comparable models.

A user reports successfully running a 1M-token context on a single RTX 3090 using a Qwen-based 35B A3B model (17GB VRAM) with KVarN 4-bit KV-cache quantization in a BeeLlama.cpp fork, extracting 7 needles from different parts of the text.

User asks for performance numbers on Deepseek V4 Flash running via Colibri, focusing on high VRAM setups, long context prefill, and token generation speed for agentic workloads.

The author observes that Nvidia's desktop 70-class GPUs have stayed at 12GB VRAM across two generations, and suggests Nvidia may be intentionally limiting memory to preserve demand for higher-margin AI-focused hardware.

Daniel Han of Unsloth validates that Qwen3.8-27B will run in only 17GB VRAM, making it accessible for local inference.

Poolside released Laguna S 2.1, and the user acquired 3x AMD V620 GPUs totaling 96 GB VRAM.

A guide on training a generative AI model for kick drum sounds using an old Linux desktop with only 6GB VRAM, making AI audio generation accessible with limited hardware.

A technique to remove 423 GB from GLM-5.2 (753B weights) without quantization or retraining, achieving bit-exact compression by keeping weights compressed in VRAM.

Demonstrates achieving 20GB VRAM and 448GB/s bandwidth for around $100 using two NVIDIA P102-100 cards, running a llama.cpp server with a Qwen model and supporting 3 concurrent users with large context.

A user shares detailed VRAM usage measurements for running Qwen 3.6 27b on a 24GB GPU, showing context window sizes and headroom, and notes that a used RTX 3090 performs identically to newer cards.

The user asks for recommendations on the best local AI models for different VRAM classes (8-16GB, 24-32GB, 128GB), mentioning Gemma4, Qwen, and DeepSeek variants, as they work on improving local model support in Hermes.

Applications like Discord, Steam, and Telegram reserve VRAM even when minimized, consuming 1GB+ collectively; users working with LLMs should close these apps or disable hardware acceleration to free up VRAM.

Open-source build guides for a personal AI computer with up to 384GB VRAM, supporting configurations from home to on-prem business. Includes bill of materials, assembly photos, and software setup for running open-source AI models locally.

A developer with 64GB VRAM shares their preference for an unsloth version of Qwen 3.5 122b-a10b for coding and asks the community for their recommendations.