4-bit

A benchmark comparison of nvfp4, nvfp4-fast, and AWQ 4-bit quantizations of Qwen3.6-35B on RTX 3090s shows similar performance, with the MTP head trick boosting throughput by 41%.

Running a 4-bit quantized version of GLM-5.2 (753B MoE) on 4 DGX Spark machines achieves 70.8% on Terminal-Bench 2.1, compared to 81.0% from the full model.

FourTune proposes a fully 4-bit quantization framework (W4A4G4) for efficient post-training of diffusion models, using a triple-branch hybrid pipeline and custom fused kernels to reduce memory by 2.25× and increase throughput by 2.27× on 12B FLUX.1-dev without quality loss.

NVIDIA's 4-bit quantized Qwen3.6-27B model (NVFP4) is reported to be near-lossless, maintaining full-size quality at a quarter the size.

NVIDIA's 4-bit quantized Qwen3.6-27B (NVFP4) is found to be near-lossless compared to the full bf16 model, with behavioral differences being minor and random rather than systematic, making it a practical drop-in replacement.

GGUF quantizations of DeepSeek V4 Flash in 2-bit, 3-bit, and 4-bit precisions, made available on Hugging Face for local inference with tools like llama.cpp and Ollama.

A detailed guide on building and understanding the TD4 4-bit DIY CPU kit from Aliexpress, covering soldering, schematics, and operation principles.

The article questions the validity of vendor benchmarks for Alpie Core 32B, a 4-bit reasoning coding model optimized for low VRAM and agent workflows, noting a lack of independent benchmark replication.

UnslothAI announces that its 4-bit Qwen3.6 MTP GGUF model can search over 70 websites from a single prompt, running locally on 20GB RAM via Unsloth Studio. The update adds automatic MTP and speculative decoding support.

Cyankiwi introduced an updated version of their AWQ 4-bit quantization method that jointly optimizes scales and quantization ranges, achieving lower KL divergence than existing methods on Llama-3 models.