Tag
The author shares findings from testing quantization impacts on local LLMs for agentic use, revealing that many quants are statistically indistinguishable, MoEs are less affected than dense models, and significant degradation occurs below Q4 quantization.