NVIDIA Puzzle-75B-A9B NVFP4 at 132 t/s on 3×3090 — Why is this size category a desert otherwise?
Summary
NVIDIA's Puzzle-75B-A9B model achieves 132 tokens per second using NVFP4 quantization on three RTX 3090 GPUs, raising discussion about the lack of competition in this model size category.
Similar Articles
NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B on 2x3090s
A detailed guide on running the quantized NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B model on two RTX 3090s using vLLM with full 262K context, achieving high inference speeds without CPU offloading.
nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4
NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B, a compressed version of Nemotron-3-Super with improved inference efficiency and strong benchmark performance.
nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16 · Hugging Face
NVIDIA releases Nemotron-Labs-3-Puzzle-75B-A9B, a compressed hybrid MoE LLM derived from Nemotron-3-Super, achieving approximately 2× higher server throughput and improved concurrency while maintaining strong accuracy across reasoning, coding, and long-context benchmarks.
@rumgewieselt: Now its getting crazy ... 3x 1080 Ti (Pascal, 33GB VRAM) Qwen 3.6 27B MTP with 196K TurboQuant ~28-30 t/s consistently
A user demonstrates successful local inference of a 27B parameter Qwen model across three GTX 1080 Ti GPUs, achieving approximately 28-30 tokens per second using TurboQuant optimization.
@superalesha: Don't dare bury RTX 3090 until you read this! @UnslothAI shipped two new 4-bit quants of qwen3.6-35b this week. i spent…
A benchmark comparison of nvfp4, nvfp4-fast, and AWQ 4-bit quantizations of Qwen3.6-35B on RTX 3090s shows similar performance, with the MTP head trick boosting throughput by 41%.