Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them

Hugging Face Daily Papers Papers
llm-quantization model-audit gguf low-bit-quantization humanEval model-quality

Summary

研究审计了 Ollama 与 Hugging Face 上的量化 GGUF 文件，发现部分低比特量化模型因未校准的量化配方而功能失效，发布了 quantcheck 工具并建议在发布前进行筛查与校准。

Developers download quantized large language models (LLMs) from public registries to run them locally. None of the registries we examined documented functional testing of their files. We tested all 305 eligible code-capable GGUF tags (the llama.cpp format) across 15 model lines in the official Ollama library, plus 22 popular Hugging Face community files. A 15-task code screen identified suspects, which we retested on all 164 HumanEval+ tasks. Seven official tags (2.3%) solved at most six of them. All were low-bit quantizations of models with at most four billion parameters. We rebuilt five suspect files from the original weights. All five registry files matched our uncalibrated rebuilds tensor for tensor, so none was corrupted. Exact reproducibility did not imply function. With the weights, toolchain and tensor mix held fixed, adding an importance matrix raised three Qwen2.5-Coder-3B builds from zero to 76-113 solved tasks. The same uncalibrated recipe left the smaller Qwen2.5-Coder-1.5B functional, showing that the recipe alone does not predict failure. None of 66 sampled tags that passed the screen was non-functional. We release the audit data and the quantcheck tool and recommend screening low-bit tags before publication, then calibrating or withholding failures.
Original Article
View Cached Full Text

Cached at: 10/30/26, 07:12 PM

Paper page - Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them

Source: https://huggingface.co/papers/2609.05881

Abstract

Quantized small language model files in public registries can be non-functional due to uncalibrated quantization recipes, and screening plus calibration is recommended before release.

Developers downloadquantized large language models(LLMs) from public registries to run them locally. None of the registries we examined documented functional testing of their files. We tested all 305 eligible code-capableGGUFtags (thellama.cppformat) across 15 model lines in the officialOllamalibrary, plus 22 popularHugging Facecommunity files. A 15-task code screen identified suspects, which we retested on all 164HumanEval+tasks. Seven official tags (2.3%) solved at most six of them. All werelow-bit quantizations of models with at most four billion parameters. We rebuilt five suspect files from the original weights. All five registry files matched our uncalibrated rebuildstensorfortensor, so none was corrupted. Exact reproducibility did not imply function. With the weights, toolchain andtensormix held fixed, adding animportance matrixraised threeQwen2.5-Coder-3B builds from zero to 76-113 solved tasks. The same uncalibrated recipe left the smallerQwen2.5-Coder-1.5B functional, showing that the recipe alone does not predict failure. None of 66 sampled tags that passed the screen was non-functional. We release the audit data and the quantcheck tool and recommend screening low-bit tags before publication, then calibrating or withholding failures.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2609\.05881

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.05881 in a model README.md to link it from this page.

Datasets citing this paper1

#### aditiPatodiya/quantcheck-audit Viewer• Updated1 day ago • 9.66k

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.05881 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

AtomicChat/LFM3-1.7B-Uncensored-GGUF

Hugging Face Models Trending

AtomicChat 发布了基于 LiquidAI/LFM3-1.7B 的安全消融版 GGUF 模型，通过投影秩-1 LoRA 移除有害提示的拒答（61.7%降至0%），同时在 MMLU 上无精度损失，并提供 AD-Q6_K 和 AD-Q4_K_M 量化版本。

abenzerps/Qwen-Image-2.1-Uncensored-Quantized

Hugging Face Models Trending

Abenzerps 发布了 Qwen-Image-2.1 的去审查量化版本，提供 GGUF、Safetensors 和 MLX 格式，支持在 ComfyUI 中进行本地图像生成，Q4_K_M 为推荐量化档位。

moar QAT stuff and hairy ticks

Reddit r/LocalLLaMA

The author releases improved GGUF quantized versions of Gemma 4 models (12B and 31B) using a more accurate quantization-aware training process that achieves lower KLD and higher same-top percentage than stock quantizations.