low-precision

QUASAR is a quantization-aware training method that uses loss-aware reconstruction to lower the loss floor, improving low-bit model performance in large language models with significant accuracy gains at 2-4 bits.

Introduces CurveFP, a closed-product codebook family that distributes quantized magnitudes across interleaved logarithmic curves, enabling exact sign XOR and integer-index updates. It achieves FP8-class behavior with seven-bit inference and improves perplexity on 7B-9B models.

This paper presents HiFloat4 format and Rollout-ResQ for end-to-end FP4 reinforcement learning post-training of LLMs, achieving a 1.1% accuracy gap to BF16 on Qwen2.5 models.

This paper studies quantization of recursive reasoning models where weight-tied blocks are reused, finding that per-tensor 4-bit quantization causes catastrophic drift but per-block scaling (e.g., MXInt4) restores accuracy, with deeper architectures being more sensitive.

This article presents a recipe for low-precision (NVFP4) RL training that balances throughput and stability, addressing issues from forward and backward pass quantization errors.

A detailed analysis of how NVIDIA GPU programming evolved from Volta to Blackwell, highlighting the shift from synchronous thread models to asynchronous dataflow and the challenges of feeding Tensor Cores. The article discusses new hardware features like TMA, TMEM, and tcgen05 MMA, and shows how modern kernels like FlashAttention-3 and FlashMLA exploit these changes for higher utilization.

dMX is a differentiable mixed-precision quantization framework that learns optimal floating-point bit-width assignments per layer for LLMs, targeting the MXFP family of formats defined by the OCP standard. It uses continuous optimization with temperature-based annealing and a budget-aware regularization term, consistently outperforming KL-divergence heuristics on Llama, Qwen3, and SmolLM2 models.

Mix-Quant proposes a phase-aware quantization framework for agentic LLMs, using NVFP4 quantization for the prefilling stage to accelerate computation while preserving BF16 precision for decoding to maintain accuracy. The method achieves up to 3x speedup in prefilling with minimal performance degradation on agentic benchmarks.

A page from Modal's LLM Engineer's Almanac that provides an interactive explorer for understanding low-precision floating-point formats like bf16 and fp4.