mxfp4

NVIDIA 的这篇技术博客介绍了在 DGX Spark 上加速 Ling-3.0-flash MoE 模型推理的三种优化技术：Humming MXFP4 MoE 内核、LM head 的在线 FP8 量化，以及基于 NEXTN 的多 token 预测投机解码，并分析了组合配置下的集成问题。

论文提出开源训练配方 ALTO，用于在 MXFP4 微缩放格式下训练 20B 参数 MoE 模型 GPT-OSS-20B，通过混合块量化、随机 Hadamard 变换、随机舍入与权重去振荡方案，将 FP4 与 BF16 的验证损失差距缩小至 0.007，收敛开销从 +20% 降至 +5%，并报告了若干无效尝试的负面结果。

Testing a 1.56TB Mixture-of-Experts model on a 6GB RTX 4050 laptop, requiring patched memory streaming with NVMe to achieve 0.106 tokens/s decode speed.

A user runs Kimi K3, a 2.8T-parameter open-weight MoE model, on 80 RTX 5090 GPUs using only GDDR7 and Ethernet—no HBM—achieving 20 tok/s single stream, the first frontier model to run on consumer hardware.

A user questions why bartowski claims DeepSeek-V4-Flash is in MXFP4 format when the original model page lists BF16, F32, and other tensor types, not MXFP4.

Discusses the largest dense model that can be loaded in 128 GB RAM using MXFP4 quantization.

A tweet thread introduces a visualizer for micro-scaling/block quant formats like NVFP4 and MXFP4, explaining how these low-precision floats work and their use in LLM inference to reduce memory bandwidth demands.

A technical comparison between nvfp4 and mxfp4 formats, highlighting that nvfp4 uses an additional tensor-wise scale factor to overcome fp4's range limit, allowing more precision in block-wise scale factors.

DealignAI releases CRACK-abliterated and MXFP4/MXFP8 quantized versions of Qwen3.6-27B and 35B models, preserving MTP for faster speculative decoding on Apple Silicon.

This paper decomposes MXFP4 quantization error into three additive components—scale bias, deadzone truncation, and grid noise—and proposes targeted corrections that recover BF16 accuracy to within 0.7 pp on Qwen2.5-3B and 3.0 pp on Qwen3-30B-A3B-Base for LLM reinforcement learning post-training.