bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF · Hugging Face

Source: https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF

GGUF /llama.cppquantizations ofbottlecapai/ThinkingCap-Qwen3.6-27B— capability of Qwen3.6-27B with50% lessthinking tokens on average, achieved by finetuningQwen3.6-27B (Qwen Team, 2026)with online reinforcement learning while preserving the original answer quality and style.

➡️ Full model description, evaluation results (multi-seed, statistically tested), recommended sampling params, and citation: see the main model card atbottlecapai/ThinkingCap-Qwen3.6-27B.

https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF#about-gguf-and-quantizationAbout GGUF and quantization

GGUFis a single-file model format for running LLMs locally with llama.cpp and compatible runtimes (Ollama, LM Studio, …). The quantized variants below store weights at reduced precision — e.g. ≈4.7 bits per weight forQ4\_K\_Minstead of the 16-bitf16source — cutting download size and memory severalfold at a small, measured quality cost.

https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF#filesFiles

FileQuantSizeThinkingCap\-Qwen3\.6\-27B\-Q4\_K\_M\.ggufQ4_K_M15.7 GBThinkingCap\-Qwen3\.6\-27B\-Q8\_0\.ggufQ8_027.1 GBThinkingCap\-Qwen3\.6\-27B\-f16\.gguff1650.9 GBmmproj\-ThinkingCap\-Qwen3\.6\-27B\-f16\.ggufmmproj (vision)0.9 GB f16is the unquantized source;Q8\_0is near-lossless;Q4\_K\_Mis the recommended size/quality balance for most local setups.

https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF#usage-llamacppUsage (llama.cpp)

# pull a specific quant straight from the Hub and chat
llama-cli -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M -p "Hi"

# or download one file and run it
huggingface-cli download bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf --local-dir .
llama-cli -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf -p "Hi"

https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF#speculative-decoding-mtpSpeculative decoding (MTP)

llama.cpp can run MTP (multi-token-prediction) self-speculative decoding on these GGUFs for a decode speed-up — no separate draft model needed. Add\-\-spec\-type draft\-mtpwhen serving:

llama-server -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M --spec-type draft-mtp

Set the draft length with\-\-spec\-draft\-n\-max(e.g.4). Requires a recent llama.cpp build with MTP support.

https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF#vision-image-inputVision (image input)

ThinkingCap is avision-language model. Image input needs the multimodal projectormmproj\-ThinkingCap\-Qwen3\.6\-27B\-f16\.gguf(in this repo) loaded alongside a text GGUF — the singlef16mmproj pairs with any of the quants above.

• **LM Studio / Jan / Ollama, …:**download themmproj\-\*\.gguffrom this repo; LM Studio auto-detects it and enables the image (🖼️) button.
• llama.cpp CLI:

huggingface-cli download bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF \
  ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf --local-dir .
llama-mtmd-cli -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf \
  --mmproj mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf --image photo.jpg -p "Describe this image."

• **llama-server:**add\-\-mmproj mmproj\-ThinkingCap\-Qwen3\.6\-27B\-f16\.ggufto expose an OpenAI-compatible vision endpoint.

https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF#expected-performanceExpected performance

From our internal serving-validation harness (llama.cpp, single-stream, temperature 0) on a fastN=100/datasetsubset of MMLU-Pro (reasoning) and RealWorldQA (vision) — a quickquant-parity + decode-speedcheck,notthe headline accuracy evals (for the multi-seed, statistically-tested results see themain model card).

Our three quants (f16/Q8\_0/Q4\_K\_M) stay within subset noise off16on accuracy, and MTP self-speculative decoding (\-\-spec\-type draft\-mtp,n=4) accepts ≈3.75 tokens per verify step — a ≈1.4–1.7× per-token decode speed-up on top of the finetune’s ≈50% token savings.Q4\_K\_M+ MTP (bold) is the recommended local config. For reference we also list**unsloth’s Dynamic GGUFs of thebasemodel**(UD\-\*): same llama.cpp path, but base-model quants — so they match base accuracy and reason ≈2× longer (none of the finetune’s token savings).

median tokens= median completion length;task s= median tokens ÷ single-stream tok/s (real per-request time);speedupis vs the unquantized base model in standard decoding.

MMLU-Pro (reasoning)

configaccmedian tokenstok/stask sspeedupaccept_len (n=4)Qwen3.6-27B base · standard0.85189057.432.91.00×—f16 · standard0.8988450.417.51.88×—f16 · MTP0.8887086.710.03.28×3.78Q8_0 · standard0.8889057.215.62.12×—Q8_0 · MTP0.8685699.48.63.82×3.77Q4_K_M · standard0.8681461.813.22.50×—Q4_K_M · MTP0.8584889.29.5****3.46×****3.74unsloth UD-Q8_K_XL (base) · standard0.85189654.534.80.95×—unsloth UD-Q8_K_XL (base) · MTP0.86192598.219.61.68×3.74unsloth UD-Q4_K_XL (base) · standard0.84197662.131.81.03×—unsloth UD-Q4_K_XL (base) · MTP0.83192887.122.11.49×3.72 RealWorldQA (vision)

configaccmedian tokenstok/stask sspeedupaccept_len (n=4)Qwen3.6-27B base · standard0.7455657.49.71.00×—f16 · standard0.7927150.45.41.80×—f16 · MTP0.7927186.73.13.10×3.78Q8_0 · standard0.7927057.24.72.05×—Q8_0 · MTP0.7827399.42.73.53×3.77Q4_K_M · standard0.7828361.84.62.11×—Q4_K_M · MTP0.7827489.23.1****3.15×****3.74unsloth UD-Q8_K_XL (base) · standard0.6853054.59.71.00×—unsloth UD-Q8_K_XL (base) · MTP0.6955098.25.61.73×3.74unsloth UD-Q4_K_XL (base) · standard0.6565562.110.50.92×—unsloth UD-Q4_K_XL (base) · MTP0.7056487.16.51.49×3.72