Tag
@NVIDIARTXSpark shares three techniques to accelerate Mixture-of-Experts (MoE) inference on the NVIDIA DGX Spark, covering optimized kernels, FP8 quantization, and multi-token prediction to boost tokens per second.
NVIDIA 的这篇技术博客介绍了在 DGX Spark 上加速 Ling-3.0-flash MoE 模型推理的三种优化技术:Humming MXFP4 MoE 内核、LM head 的在线 FP8 量化,以及基于 NEXTN 的多 token 预测投机解码,并分析了组合配置下的集成问题。
dealignai 发布 DeepSeek V4.1 Flash 1017 的「完全无审查」版本,通过移除拒答行为实现,同时保留视觉、视频、工具调用、推理档位和 1M token 上下文等原始能力,采用原生 FP8 (e4m3) 权重与 FP4 MoE 专家,无模板越狱或系统提示捷径。
The paper proposes Calibrated Clipping to stabilize FP8 quantization in reinforcement learning for LLMs by aligning clipping bounds with high-precision distributions, eliminating entropy surges and restoring performance.
dealignai released a cybersecurity-focused weight-modified variant of the 753B-parameter GLM-5.3-FP8 model that reduces refusals for offensive-security tasks such as red-teaming, exploit development, and malware analysis while preserving base capabilities and running natively on Hopper GPUs via stock vLLM.
The article releases FP8-quantized weights for the Qwen3.8-Flash-Next model, introducing architectural innovations like Hybrid Attention with QSA, Gated Residual, and N-gram Embedding to improve efficiency in large language models.
This article presents an uncensored version of the Qwen3.8 27B AI model, which has been modified to remove safety refusals and quantized to FP8 for efficiency, intended for research purposes.
A modified version of Qwen3.8-27B with safety refusal removed and FP8 quantization, designed for research in AI safety and interpretability.
Qwen releases FP8-quantized weights for Qwen3.8-2.4T-A95B, a 2.4T-parameter MoE model with 95B activated parameters, claiming Qwen-Max-class capability in an open release with strong coding, agentic, and long-context performance.
Quantized 27B Qwen3.6 model achieves 200 tok/s peak (136 avg) with 256k context and 10 agents on a single 49W GB10 GPU using Dflash+DDTree optimizations.
Alibaba releases Qwen3.6-27B-FP8, a 27B FP8-quantized model with strong agentic coding and reasoning benchmarks, now available on Hugging Face.