Tag
This article compares the performance of NVIDIA DGX Spark and a modified RTX 4090 in locally deploying the Qwen3.8-27B and Ling-3.0-flash models, providing benchmark data and purchase recommendations.
The author has updated their NInfer fork with rk2v4-e8 quantization for the KV cache, enabling up to 250-350K tokens context on a single RTX 4090 without system RAM spill and with optimizations for faster generation speeds.
A user shares benchmark results for running large language models (Qwen 27B-122B) on a dual-GPU setup with RTX 4090 and RTX 5060 Ti, achieving high token generation speeds (e.g., 206 t/s on 35B-A3B, 37-41 t/s on 122B). The post includes setup details and a link to a GitHub repo with scripts and raw data.
A user shares their experience running the DeepSeek v4 Flash model with a 24GB GPU and DDR5 RAM, including performance numbers and tips for optimization.
We're the first to run the full GLM-5.2 (753B FP8) on RTX 4090s by porting sparse-attention kernels to Ada GPUs, enabling frontier open-weights model on commodity hardware.
A user seeks community feedback on purchasing a modded RTX 4090 with 48GB VRAM from GpuWorld.eu, asking for trustworthy sources and alternative sellers like Taobao.
A Reddit user expresses curiosity about modded Chinese GPUs (e.g., 48GB RTX 4090) and seeks information on performance, reliability, and sourcing, proposing to form a research group.
Developer achieved 80+ t/s inference on Qwen3.6-27B with 262K context on a single RTX 4090 by combining MTP (Multi-Token Prediction) with TurboQuant's lossless KV cache compression, sharing their implementation fork and technical details.