Unsloth Gemma 4 QAT MTP assistant models now available

Reddit r/LocalLLaMA Models
unsloth gemma-4 qat mtp gguf hugging-face quantized-model

Summary

Unsloth released Gemma 4 QAT MTP assistant models as GGUF files on Hugging Face, available in q8_0 and larger quantization formats.

They're both available as q8_0 models named `mtp-gemma-4-*.gguf` on the root of the directory and in both q8_0 and larger quants within an `MTP` folder. - https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF/tree/main - https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF/tree/main - https://huggingface.co/unsloth/gemma-4-31B-it-qat-GGUF/tree/main - https://huggingface.co/unsloth/gemma-4-E2B-it-qat-GGUF/tree/main - https://huggingface.co/unsloth/gemma-4-E2B-it-qat-mobile-GGUF/tree/main - https://huggingface.co/unsloth/gemma-4-E4B-it-qat-GGUF/tree/main - https://huggingface.co/unsloth/gemma-4-E4B-it-qat-mobile-GGUF/tree/main
Original Article

Similar Articles

unsloth/gemma-4-12B-it-qat-GGUF

Hugging Face Models Trending

Unsloth releases GGUF quantized versions of Google DeepMind's Gemma 4 models, optimized with Quantization-Aware Training (QAT) to reduce memory requirements while preserving quality, supporting multiple formats and sizes for diverse deployment.

google/gemma-4-12B-it-qat-q4_0-gguf

Hugging Face Models Trending

Google DeepMind releases Gemma 4 models optimized with Quantization-Aware Training (QAT) in multiple formats including GGUF, enabling high quality with reduced memory requirements.

moar QAT stuff and hairy ticks

Reddit r/LocalLLaMA

The author releases improved GGUF quantized versions of Gemma 4 models (12B and 31B) using a more accurate quantization-aware training process that achieves lower KLD and higher same-top percentage than stock quantizations.