Extracted MTP tensor GGUFs - smaller donor models for grafting.

Reddit r/LocalLLaMA Tools

Summary

The author provides extracted GGUF files containing only MTP tensors for Qwen3.6 models, allowing users to graft tensors with a significantly reduced download size compared to full model files.

The [script](https://gist.github.com/buzz/1c439684d5e3f36492ae9f64ef7e3f67) to graft MTP tensors requires a full GGUF model file. I felt that was a bit hefty, so I asked local Gemma to write something to just extract what's required. The results are two faux GGUFs weighing in at just 900MB ([35A3B](https://huggingface.co/IHaveNoClueAndIMustPost/Qwen3.6-35A3B-MTP-TENSORS-ONLY)) and 450MB ([27B](https://huggingface.co/IHaveNoClueAndIMustPost/Qwen3.6-27b-MTP-TENSORS-ONLY)), containing only the tensors and fully compatible with the script. A lot quicker to download compared to the original 38GB and 29GB models for those who just want to convert their existing library or save some bandwidth. Testing was done using SHA256 hashes, comparing the models made with these mini-GGUFs to those using the full models (identical results), along with some brief chats. Credits: [am17an](https://huggingface.co/am17an) for the original GGUFs, and [buzz](https://gist.github.com/buzz) for the original script. Disclaimers: The MTP implementation isn't finalized. These models might break or become obsolete at any time. Do not delete the original models in case there are updates to the conversion process. Testing was only done on the two models I use myself; other variants might not work well/at all. Also, 100% clueless vibecoding with a Q4_1 model.
Original Article

Similar Articles

havenoammo/Qwen3.6-27B-MTP-UD-GGUF

Hugging Face Models Trending

This Hugging Face repository provides GGUF files for Qwen3.6-27B with Multi-Token Prediction (MTP) layers grafted onto Unsloth UD XL quantizations. It includes instructions for building llama.cpp with MTP support to enable speculative decoding.

unsloth/Qwen3.6-27B-MTP-GGUF

Hugging Face Models Trending

Unsloth has released GGUF weights for the Qwen3.6-27B model, featuring Multi-Token Prediction (MTP) for faster generation and enhanced agentic coding capabilities.

MTP on Unsloth

Reddit r/LocalLLaMA

Unsloth releases GGUF-quantized versions of Qwen3.6 models with Multi Token Prediction (MTP) support.

Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF

Hugging Face Models Trending

A GGUF quantized version of the Qwopus3.6-27B-Coder-MTP model is released on Hugging Face, optimized for local inference and compatible with Transformers, vLLM, SGLang, and Unsloth Studio.