Tag
A new EXL3 quant of Qwen3.8-27B by yeasah achieves the lowest KLD (0.0002) among competitors while requiring only 18.8 GiB of VRAM for weights, outperforming quants from Unsloth and turboderp.
作者分享使用 DeepSeek V4.1 Flash（EXL3 2.9 bpw，2x DGX Spark，TP2）作为实现代理、配合独立审查代理开发真实功能的实践经验，指出其报告有时会超出实际证据。
Qwen3.8-Flash-Next is released in a 1.66bpw EXL3 quantization that fits entirely within a 36GB RTX 5090 with no offloaded experts, delivering stable 200-300 t/s inference without reasoning loops.
A user shares and asks for feedback on running Qwen3.8-Flash-Next in EXL3 5.05bpw on a single Radeon RX 9700 (gfx12) with mixed GPU/CPU expert offloading, reporting ~863 t/s prefill and ~35 t/s decode at 230k context.
A community EXL3 quantization of GLM-5.3-UNCENSORED (a 753B-parameter MoE weight-edit of Z.ai's GLM-5.3) at 3.0 bpw, 273 GiB, with fidelity benchmarks, tau2-bench agentic evals, and TabbyAPI serving configs including an MTP speculative-draft mode.
A head-to-head benchmark of two community inference stacks running the 320B GLM-5.3-Flash on a pair of NVIDIA DGX Sparks, comparing Entrpi's EXL3 setup against TensorFold with DFlash2 drafters across prose, code, and prefill workloads.
ninfer-ext is an extended fork of NInfer that adds support for larger Qwen models, faster speculative decoding, agent serving features, and EXL3 quantization for Qwen3.8-27B on NVIDIA RTX 5090.
DeepSeek-V4.1-Flash has been quantized to 4.75 bpw EXL3 for deployment on 4× DGX Spark, optimizing memory usage and enabling efficient local inference with plans for validation and further optimization.
User @ViC305 successfully runs DeepSeek-V4-Flash-Vision with EXL3 MixedK and DSpark speculative decoding on a single DGX Spark, achieving improved performance and fixing technical issues for multimodal AI deployment.
The author shares their experience running a 30B parameter model with EXL3 quantization on a 12GB VRAM GPU, achieving efficient performance and speed for coding and agent tasks.
A user asks for a verdict on whether EXL3 quantization is advancing compared to NFP4 in AI models.
The article highlights the GLM-5.3-Flash EXL3-3.0bpw AI model with an inference speed of 193.8 tokens per second, attributed to multiple contributors.
A new tool enables converting and running EXL3 quantized models on Apple Silicon Macs, matching or nearly matching RTX conversion quality, making high-fidelity quants more accessible.