Tag
This post describes two configuration flags that increase the official Ling-3.0-flash INT4 inference speed from 20.8 to 38.7 tok/s on a single DGX Spark, while warning about the need for a specific vLLM fork and noting tradeoffs with long-context performance.
The article compares three quantized variants of Qwen3.6-27B (NVFP4 from Unsloth and Nvidia, int4-AutoRound from Intel) and requests benchmarks and hallucination data from the community.
Reports on testing the RTX Pro 4500 GPU with Oculink, focusing on INT4 Autoround and NVFP4 W4A4 quantized models using PrismaQuant.
Pre-converted int4 quantized weights for the GLM-5.2 744B MoE model, designed to run on consumer hardware with ~25 GB RAM using the colibrì engine.
User shares an optimized recipe for running Qwen 3.5 122B Int4 on a single DGX Spark with vLLM, achieving over 40 tokens per second. They invite others to try and further optimize it.
Reddit post compares quantized Qwen3.6-27B variants (INT4, NVFP4, BF16-INT4) showing trade-offs between memory size and accuracy for different use-cases.