Tag
A user shares satisfaction with using the Qwen3.8-27B AI model on RTX 3090 GPUs, comparing it favorably to GLM and describing the creation of a bossfight scene.
The author optimized the Qwen3.8-27B model inference on an RTX 3090 GPU, achieving up to 99 tokens per second for single requests and 1150 tps with batch processing through various quantization and optimization techniques, and released the updated code on GitHub.
User reports that Muse Glimmer, a 30B model, fits on a single RTX 3090 with full 256k context using Q4_K_XL quantization and DFlash, achieving 64-124 tok/s and perfect long-context retrieval, unlike comparable models.
A user on a dual RTX 3090 setup discovers that llama.cpp's --split-mode tensor runs prompt processing on CPU (~400 t/s), while switching to --split-mode layer unlocks GPU prompt processing over 1600 t/s, with only a modest drop in token generation speed.
A developer benchmarks Qwen3.6-35B-A3B Q6 on an RTX 3090, showing that offloading eight MoE expert layers to CPU and increasing batch sizes improves prompt processing by 2.36× (564→1330 tok/s) with no decode speed regression, using evolutionary search to find the tuning config.
Tests et réglages détaillés pour optimiser DeepSeek-V4-Flash-0731 en GGUF sur une RTX 3090, atteignant ~15 tok/s à 128K de contexte grâce à différentes quantifications et paramètres de chargement.
A reply celebrates Unsloth AI's upcoming Qwen3.8-27B model, which will run on 17GB RAM/VRAM setups, and notes TurboQuant+ is already integrated into many inference engines — great news for RTX 3090 users.
A four-line fix reportedly boosts DeepSeek-V4-Flash prefill speed from 127 to 312 t/s at 10K context and 91 to 283 t/s at 40K, with decode unchanged. The patch is shared via the club-3090 project for serving LLMs on RTX 3090s.
This tweet discusses rising hardware prices, including used RTX 3090s, and argues that frontier intelligence is not limited to data centers, making a case for being bullish on local/open-source AI.
A quantized GGUF version of the Kimi K3 model is now available, optimized for running on RTX 3090 GPUs.
A detailed guide on running the quantized NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B model on two RTX 3090s using vLLM with full 262K context, achieving high inference speeds without CPU offloading.
A benchmark comparison of nvfp4, nvfp4-fast, and AWQ 4-bit quantizations of Qwen3.6-35B on RTX 3090s shows similar performance, with the MTP head trick boosting throughput by 41%.
User runs local benchmarks comparing Qwen3.6 27b, Gemma4 26B, and Ornith1.0 35B on an RTX 3090 using inspect-ai. Results show Qwen leading in knowledge and coding, while Ornith is competitive in grounding and recall.
A detailed benchmark comparing speculative decoding engines for Qwen 3.6 27B on a single RTX 3090, showing ik_llama achieving ~100 tokens per second in code generation. Results include decode TPS, TTFT, VRAM usage, and context degradation across 5 engine variants.
Announces Ornith-1.0-35B, a coding fine-tune of Qwen3.6-35B-A3B that slightly outperforms the base model on aider benchmarks. Also promotes the club-3090 repository for running LLMs on RTX 3090s.
A detailed guide on running the Qwen3.6-35B-A3B APEX model on an RTX 3090, comparing two llama.cpp forks and quantization methods for optimal speed and quality.
A user reports achieving over 90 tokens per second inference speed with Qwen 3.6-35b-a3b MoE model on an RTX 3090 using llama.cpp, with prefill speeds exceeding 1000 t/s, indicating practical local deployment of large language models on consumer hardware.
Discusses the cheapest hardware options for running Qwen 3.6 models, comparing RTX 3090 and Tesla V100 GPUs, and provides a detailed cost breakdown for a system at around $2000.
A setup using RTX 5080 and RTX 3090 GPUs achieves 80 tokens per second on the Qwen 3.6 27B Q8 model.
club-3090 adds experimental FP8 support for Qwen3.6-27B, enabling dual RTX 3090 rigs to run the model with performance similar to unquantized BF16.