@zcbenz: nvfp4 vs mxfp4 is not just different choices of block size and scale format, nvfp4 uses an additional tensor-wise scale…
Summary
A technical comparison between nvfp4 and mxfp4 formats, highlighting that nvfp4 uses an additional tensor-wise scale factor to overcome fp4's range limit, allowing more precision in block-wise scale factors.
View Cached Full Text
Cached at: 06/17/26, 04:02 PM
nvfp4 vs mxfp4 is not just different choices of block size and scale format, nvfp4 uses an additional tensor-wise scale factor to overcome the range limit of fp4, and thus can use more precisions for block-wise scale factors. https://t.co/9d1hvNBWhO
Similar Articles
@RayFernando1337: “The selected runtime uses NVFP4 weights for maximum performance. From the original FP8 weights, we performed an in-hou…
Discusses using NVFP4 4-bit floating point weights for maximum performance, achieved via in-house quantization from FP8 using NVIDIA ModelOpt, highlighting the data format's dual scale factors for high dynamic range.
@AaronWeiHuang: Our new blog looks at how FP4 is moving beyond compression into a practical primitive for training and inference across…
NVIDIA's blog details how FP4, with the NVFP4 format and Blackwell hardware, has evolved from a compression trick to a practical primitive for training and inference across LLMs and diffusion models, achieving near 16-bit accuracy.
@hotschmoe: After reading this post, I decided to get nvfp4 running on my Intel arc b70s just to see, after 12 hours it's running a…
A user successfully ran nvfp4 quantization on Intel Arc B70s GPUs, achieving nearly double speed and higher accuracy compared to their best int4 configuration, challenging hardware-specific format assumptions.
@ArkadiiBessonov: Three main ways to do FP8 in LLM pretraining — and they differ in mainly one thing: how the scale is attached. per-tens…
Explains three main approaches to FP8 scaling in LLM pretraining—per-tensor, blockwise, and MXFP8—focusing on how the scale is attached, and derives tile geometries from the constraint that scale must remain constant along the matmul's contracted dimension.
Any idea why bartowski claims DeepSeek-V4-Flash is MXFP4?
A user questions why bartowski claims DeepSeek-V4-Flash is in MXFP4 format when the original model page lists BF16, F32, and other tensor types, not MXFP4.