empero-ai/Qwen3.8-27B-Ridge-GGUF · Hugging Face

Source: https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#qwen38-27b-ridge-37bpwQwen3.8-27B-Ridge-3.7bpw

Developed byEmpero

A Gated-DeltaNet-aware mixed GGUF of official**Qwen/Qwen3.8-27B**(1d4bf0f2) forllama.cpp, Ollama, LM Studio, jan, KoboldCpp, and other stock GGUF runtimes.

This is a quantization of the Qwen3.8-27B checkpoint. Ridge is a probed mix of types written for this architecture: 64 layers = 16 ×\(3 × GatedDeltaNet → FFN \+ 1 × GatedAttn → FFN\). GenericIQ2\_XSand UD-IQ2 do not treat GDN state (ssm\_alpha/ssm\_beta) or the GDN mixers as first-class. We fixed that.

Nothing was stripped to make the file fit. The native MTP draft head (blk\.64/nextn) stays in the GGUF. Vision is a separate BF16mmproj.

This card is about choosing the file and running it. The official capability writeup lives on the**base model card**.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#filesFiles

The repository isQwen3\.8\-27B\-Ridge\-GGUF. Use the exact filenames below when downloading or passing\-m.

FileQuantSizeNotesQwen3\.8\-27B\-Ridge\-3\.7bpw\.ggufRidge mix,3.69 bpw11.73 GiB / 12.59 GBthis release— text + native MTPmmproj\-Qwen3\.8\-27B\-BF16\.ggufBF160.87 GiB / 0.93 GBvision encoder + projector;required for images If you only want text, download the Ridge GGUF. Add themmprojfor image input.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#what-fits-on-a-gpuWhat fits on a GPU?

These are practicalweight-size-based estimates, not a VRAM benchmark. They assume a modest context and leave room for runtime and the KV cache. Image input adds the 0.87 GiBmmproj. The native 262k window and the 1M YaRN extension — make KV the dominant cost and may need offload regardless of weight quant.

Measured:Qwen3\.8\-27B\-Ridge\-3\.7bpw\.gguffully offloaded to a singleRTX PRO 6000 Blackwell (96 GB)runs at~54 tok/s generation, ~130 tok/s prompt(llama.cpp CUDA,\-ngl 99, short smoke). One data point on one card, not a sweep — but a 27B at 11.7 GiB is comfortably interactive on a 16–24 GB card at modest context.

FileApproximate hardware guidance at modest contextRidge-3.7bpwThe practical 16 GB starting point; 24 GB is comfortable once you add KV and (optionally) the mmproj.+ mmprojAdd ~1 GiB. Still a 24 GB card for everyday use.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#recipeRecipe

Qwen3.8 is a hybrid: three Gated-DeltaNet layers for every full-attention layer. GDN state is disproportionately sensitive to low-bit quantization, so Ridge holds that path high and spends the saved bits by dropping mid-stack FFN.

**The Gated-DeltaNet state path is Q8_0.**Mixers are Q4_K, not IQ2. That is the difference between this file and a flat 2-bit dump of the same model.

Built with llama.cppadb55e5, CUDA, importance matrix on 80 × 512-token chunks (\-\-process\-output, wikitext + code). MTP tensors are unused during calibration and havenoimatrix — IQ2/IQ3 onblk\.64will abort, so the draft head stays Q6_K.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#measuredMeasured

Same box, same calibration file,llama\-perplexity, 80 chunks,\-c 512 \-b 512. BF16 is our convert of the same official checkpoint.

CandidateSizeBPWWiki-style PPLvs BF16BF16 GGUF (this convert)50.89 GiB16.007.15 ± 0.12—Ridge-3.7bpw11.73 GiB3.697.82 ± 0.14+9.3 %

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#comparisonComparison

Published Hugging Face file sizes as of 2026-08-15. PPL is filled only where we measured the file ourselves.

FilePublisherSizeNominal bandPPL vs this BF16BF16this convert50.89 GiB16 bpw7.15UD\-IQ2\_XXSunsloth8.39 GiB~2.1 bpwnot measured here(Unsloth quotes 82.5 % top-1 vs BF16)UD\-IQ2\_Munsloth9.61 GiB~2.4 bpwnot measuredIQ2\_XXSbartowski8.75 GiB~2.2 bpwnot measuredQ3\_K\_Sunsloth11.71 GiB~3.1 bpwnot measuredRidge-3.7bpwempero-ai11.73 GiB3.69 bpw7.82 (+9 %)**IQ3\_XXSbartowski11.76 GiB~2.9 bpwnot measuredUD\-Q3\_K\_XLunsloth12.52 GiB~3.4 bpwnot measured

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#quick-startQuick start

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#llamacpp-llama-clillama.cpp (llama\-cli)

Sampling from the official Qwen3.8 card. Thinking is on by default.

# thinking
llama-cli \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  -ngl 99 -n 16384 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain the design tradeoffs in a Gated-DeltaNet hybrid model."

# instruct (thinking off)
llama-cli \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  -ngl 99 --reasoning off \
  --temp 0.7 --top-p 0.80 --top-k 20 --presence-penalty 1.5 \
  -p "Say hello in one short sentence."

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#llamacpp-llama-serverllama.cpp (llama\-server)

llama-server \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  -c 16384 --port 8080

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#ollamaOllama

ollama run hf.co/empero-ai/Qwen3.8-27B-Ridge-GGUF

Or a local Modelfile:

FROM ./Qwen3.8-27B-Ridge-3.7bpw.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20

ollama create qwen38-ridge -f Modelfile
ollama run qwen38-ridge

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#lm-studio–jan–koboldcppLM Studio / jan / KoboldCpp

DownloadQwen3\.8\-27B\-Ridge\-3\.7bpw\.ggufand load it. Preserve the embedded Qwen3.8 chat template if the runtime asks you to select one.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#llamacpp-with-mtp-draft-speculationllama.cpp with MTP draft speculation

The Ridge GGUF keeps the native MTP head. Use a recent llama.cpp build that supports\-\-spec\-type draft\-mtp:

llama-server \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  -c 16384 --port 8080

If your runtime does not support MTP, the file still runs as a normal 27B — you just will not get the draft speedup.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#vision-image-inputVision (image input)

Download the text GGUF andmmproj\-Qwen3\.8\-27B\-BF16\.gguf.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#llamacpp-llama-mtmd-clillama.cpp (llama\-mtmd\-cli)

llama-mtmd-cli \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  --image ./photo.jpg \
  -p "Describe this image in detail." \
  --temp 0.7 --top-p 0.80 --top-k 20 \
  -c 16384

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#llamacpp-serverllama.cpp server

llama-server \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  -c 16384 --port 8080

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#samplingSampling

Qwen3.8 is a hybrid thinking model. Responses open with a<think\>…</think\>block unless thinking is disabled.

Modetemperaturetop_ptop_kpresence_penaltyThinking (default)1.00.95200.0Instruct (thinking off)0.70.80201.5 Use the runtime chat/completions path rather than hand-rolling a different prompt format. The embedded template is Qwen3.8’s, including tool-use (<tool\_call\>…</tool\_call\>).

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#long-contextLong context

Native context is262,144tokens, extensible to1,000,000with YaRN. Set\-cto what you actually need — the KV cache, not the 11.7 GiB weights, is what blows up a 16–24 GB card at long context.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#limitationsLimitations

• Not lossless.+9 % wiki-style PPL vs our BF16 convert
• **Context costs memory.**Weight size is only part of the hardware budget.
• **MTP is runtime-dependent.**The head is in the file; the speedup needs a runtime that knowsdraft\-mtp.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#stay-in-the-loopStay in the loop

Sign up for the Empero newsletter at**empero.org**for releases, evals, and research notes.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#support–donateSupport / Donate

If this model helped you, consider supporting the project:

• BTC:bc1qx6zepu6sfkvshgdmc4ewu6pk6rpadvpgffpp7v
• LTC:ltc1qv2mefzps2vtjcpwfx8xxdrpplrcvltswm68r7x
• XMR:42Dbm5xg5Nq26fdyzfEU7KBnAJfhi7Cvz5J2ex5CzHXkfKuNEJzYCcmJ1GTbgjFZ5MBx72sdG1G9239Cd6rsZfv4QeDkYJY

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#provenance–licensingProvenance & licensing

Quantization of**Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. Weights areApache-2.0**, inherited from the Qwen base, shared as-is.

https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF#acknowledgementsAcknowledgements

• Developed and released byEmpero
• Base model:Qwen3.8-27B(Alibaba Qwen team)
• GGUF quantization:llama.cpp(ggml-org)