nvfp4

A Hugging Face repo publishes a claimed leaked mid-pretraining checkpoint of Anthropic's unreleased ~14T-parameter Mythos-class model codenamed "numbat" (referenced in a Claude Code source leak), including 772 safetensors shards (~7.72 TB), a full reference implementation, and training logs at step 1.83M of ~1.96M.

对 Qwen3.8-Flash-Next NVFP4 模型在单张 RTX 5090 上的三个推理引擎（basalt、capy-flash、ninfer-ext）进行全 262k 上下文基准测试，basalt 在所有上下文长度和两种解码模式下均领先（prefill 8.1k-9.2k tok/s，greedy decode 176-265 tok/s），解码排名与 MTP 草稿接受率一致。

ISTA-DASLab 发布了 Qwen3.8-27B 的 NVFP4 量化版本，通过从 SSD 流式读取的 prefiller 模型在低 VRAM 下恢复低比特量化精度，且在超过约 4k 的上下文长度下无明显速度损失，需要 Nvidia Blackwell GPU。

Basalt inference tool adds native NVFP4 support and faster prefill for Qwen3.8 Flash-Next on a single RTX 5090, delivering up to 479 tok/s decode and 8,582 tok/s prefill with near-Q8 quality retention.

capy-flash 是基于 ninfer-tp2 进化的本地推理引擎，可在两张 16GB 消费级 RTX 5070 Ti 上以 NVFP4 全精度量化运行 Qwen3.8-Flash-Next 125B-A6B（MoE 专家卸载至内存），生成速度约 100 tok/s，超过 Strata。作者还发现并修复了张量并行下双重 BF16 舍入导致的误差放大问题，改用 FP16 传输部分和即可获得与单 GPU 相当的数值精度。

QUASAR 释放了针对 Qwen3.5-4B、Gemma 4 E4B/12B 和 Muse Glimmer 30B 的 4-bit QAT 检查点（NVFP4 与 Q4_0 GGUF 格式），通过量化感知蒸馏在 KL 散度和下游基准上超越 Google 与 NVIDIA 的官方 4-bit 版本。代码、论文与项目页现已公开。

用户在 RTX 5090 单卡上使用 strata-nvfp4 引擎（Strata 的 NVFP4 分支）运行 Qwen3.8-Flash-Next 125B-A6B 混合 MoE 模型，分享了「小马挑战」（生成 Three.js 低多边形奔跑小马的 HTML）的性能数据：87 步、12.4 万输出 token、平均 103 t/s 解码速度、99% 缓存命中率。

NVIDIA released NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4, an NVFP4-quantized vision-language model with 120B total/12B active parameters, Mamba-2 hybrid MoE architecture, a C-RADIO vision encoder, up to 1M-token context, and native multi-token prediction for speculative decoding, licensed under OpenMDW-1.1 for commercial and non-commercial use.

发布 autotrust/JEV-35B-NVFP4，将 JEV-35B 的 10,240 个路由专家 MLP 量化为 NVIDIA NVFP4 格式，模型体积从 71.9 GB 降至 25.6 GB，同时在 Decision Index、视觉和机器人任务上仅保留极小的精度损失。

AutoTrust AI 发布了开源权重模型 autotrust/JEV-35B，一个基于 Qwen3.5-35B-A3B 构建的 System 1 决策模型，单次前向可处理多达 256 个选项，在 Decision Index 0.3 公开测试集上取得 59.73 分，优于 JEV-27B-VL，并提供可在单张 32–48 GB GPU 上运行的 NVFP4 量化版本。

Red Hat AI released an NVFP4-quantized checkpoint of Qwen3.8-Flash-Next on Hugging Face, with MoE experts quantized to FP4 while keeping the rest in BF16, ready for vLLM inference.

autotrust/GEV-26B-Decide-NVFP4 is an NVFP4 4-bit quantized release of GEV-26B-Decide that reduces weights from 49.5 GB to 18 GB (−66% GPU memory) by quantizing only the 3,840 routed expert MLPs, with benchmark results within run-to-run noise of the bf16 model and support for native W4A4 inference on Blackwell GPUs via vLLM.

SoloQ提出一种免校准的量化框架，将权重和激活映射到归一化旋转基底，实现数据无关的4比特量化，在扩散语言模型上保持精度并最多降低2.61倍峰值内存、加速2.24倍推理。

A developer showcases a fully local parkour simulation built over the weekend using GLM 5.3 Flash running on 2x DGX Sparks with a vLLM TP2 recipe, reporting ~1500 t/s prefill and ~40 t/s decode at 100k context, and praising the model's coding, vision, and 3D understanding. He shares the vLLM NVFP4 configuration repo for others to reproduce the setup.

A user benchmarks the new Ninfer inference backend on a Blackwell MaxQ GPU, reporting 130-180 t/s on qwen3.8-27b-nvfp4 with minimal slowdown under parallel decoding — claiming vLLM-level concurrency and far faster throughput than llama-server, even after installing it on Windows 10.

Optimized performance for running the Qwen3.8 27b NVFP4 model on a single AMD Radeon R9700 GPU, achieving up to 153 tokens per second in decode and 3,619 tokens per second in prefill with improved concurrency.

This paper explains why fully quantizing the recurrent Gated DeltaNet layers in hybrid LLMs to 4-bit NVFP4 preserves accuracy across long-context benchmarks, detailing mechanisms like outlier localization and robust delta-rule dynamics.

OCGQuant proposes an outlier-companion grouping method for NVFP4 quantization to reduce activation block errors, enhancing LLM inference efficiency. Experiments on Llama3 and Qwen3 demonstrate its superior performance over existing post-training quantization techniques.

NVIDIA researchers will present at PyTorch Conference North America 2026 on using NVFP4 pretraining recipes to accelerate large-scale LLM training while maintaining quality comparable to BF16, with integration into PyTorch tools like TorchAO and TorchTitan.

RadixArk/Qwen3.8-Flash-Next-NVFP4 is now supported in SGLang-V100, enabling full context operation on 4xV100 GPUs with performance metrics showing high throughput and context handling up to 256k tokens.