Tag
The author shares their experience running a 30B parameter model with EXL3 quantization on a 12GB VRAM GPU, achieving efficient performance and speed for coding and agent tasks.
A user asks for a verdict on whether EXL3 quantization is advancing compared to NFP4 in AI models.
The article highlights the GLM-5.3-Flash EXL3-3.0bpw AI model with an inference speed of 193.8 tokens per second, attributed to multiple contributors.
A new tool enables converting and running EXL3 quantized models on Apple Silicon Macs, matching or nearly matching RTX conversion quality, making high-fidelity quants more accessible.