Release of Gemma4-12B-QAT Uncensored Balanced, a fine-tuned uncensored model with a multi-token-prediction draft head for ~60% faster speculative decoding, optimized for llama.cpp and offering vision support.
First of all, I'm stoked to announce we are almost at 20 million downloads on HF! (counted only on my own account, no duplicates/quants/finetunes/etc) and almost 5000 members on Discord! https://huggingface.co/HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced GenRM Defeated! 0/465 refusals*. Balanced = a light reasoning preamble on the absolute edgiest stuff before delivering the full answer. No personality changes/alterations or any of that. This is the ORIGINAL Gemma4-12B-QAT, just uncensored. An Aggressive variant is not required for this release. As always with my Balanced releases, a handful of edge-case prompts can deflect on the first try but follow through on a re-ask (on extreme, non-RP scenarios). If you hit one Balanced won't get past, feel free to join the Discord and let me know the prompt so I can work on it in a future release. This is the recommended default as 99%+ of users will be happy here. Best for creative writing, RP, emotional intelligence. Normally I'd also say "agentic coding/tool use," but in my in-depth testing Qwen3.6 has been net superior on those. From my own testing: there is no looping, sampling stays stable across re-runs, long-context coherence holds. NEW — ~60% faster with MTP: this release ships a multi-token-prediction (MTP) draft head for speculative decoding. Roughly 60% faster generation with identical output (the model verifies every drafted token which is pure speed, zero quality cost). In llama.cpp: -md mtp-gemma-4-12B-it.gguf --spec-type draft-mtp. (MTP draft courtesy of the Unsloth team — thanks!) Heads up: I tested it only through llama.cpp To disable thinking: edit the jinja template or pass {"enable_thinking": false} as a chat-template kwarg. What's included: - Q4_K_M (text) - mmproj (vision support) - MTP draft head (speculative decoding) Why only Q4_K_M? Gemma 4 is quantization-aware-trained for ~4-bit, so Q4_K_M is the quality sweet spot — higher-precision quants are just bigger, not better, on a QAT model. Quick specs: - 12B dense (no MoE) - 48 layers, hybrid attention: 5× sliding-window (1024) + 1× full global, repeating - Hidden 3840, head_dim 256 SWA / 512 full, 16 query heads, 8 KV heads (sliding) / 1 KV head (global) - 262K native context - p-RoPE - Multimodal (text + image via mmproj) Sampling params (specifically made for this release, make sure to use these): temp=0.6, top_k=64, top_p=0.9, min_p=0.05, repeat_penalty=1.1 Notes: - Use the --jinja flag with llama.cpp - Place images before text in prompts for vision - Multi-GPU + LM Studio: Gemma 4 can crash under LM Studio's tensor-split mode — use a single GPU (or layer-split) All my models: HuggingFace — HauhauCS The Discord link is in the HF repo — updates, roadmap, projects, learn or just chat. As always, hope everyone enjoys the release! * = Tested with both automated and manual refusal benchmarks/prompts which resulted in none found. Based on Discord feedback I may further update the release.
Release of an uncensored version of Gemma4-12B with quantization-aware training (QAT) and a multi-token-prediction head for ~60% faster generation via speculative decoding. The model removes refusals while maintaining quality, and is available on Hugging Face.
Super Gemma 4 26B Uncensored GGUF v2 is a community fine-tuned model offering uncensored responses with zero refusals, improved speed, and fixed tool-calling, optimized for local inference on llama.cpp and vLLM.
llama.cpp is an open-source C/C++ library for efficient LLM inference on various hardware, supporting multiple quantization formats and GPU backends. This README details its features, installation, and recent updates including Hugging Face cache migration and multimodal support.
Google's Gemma 4 12B QAT model achieves 120 tok/s on a 12GB GPU using Multi-Token Prediction (MTP) with llama.cpp. A step-by-step guide and benchmark comparison without MTP show a 2x speedup.
Google DeepMind released Gemma 4 MTP drafters for the Gemma 4 family, enabling significant decoding speedups via speculative decoding while maintaining exact generation quality for low-latency applications.