kaitchup/Qwen3.8-27B-SLQ-GGUF · Hugging Face

Source: https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF This isQwen/Qwen3.8-27B, compressed with a**customized version of SLQ (Statistically-Lossless Quantization)**and assembled as GGUF. The search combines existing quantized donor weights and chooses a source separately for 498 complete matrices, including embeddings and the output head.

Choose one of three models:

• **EAR targets:**98%, 98.5%, and 99%, selected using the BF16 reference’s top 10 tokens.
• **Sizes:**decimal GB (1 GB = 1,000,000,000 bytes). Active text weights include fixed text tensors and exclude MTP, metadata and alignment.
• **Embeddings / output head:**Q4_K / Q8_0 for EAR98; Q4_K / BF16 for EAR98.5; Q6_K / BF16 for EAR99.
• Each file retains 351,008,768 bytes of quantized MTP tensors from the UD-Q5_K_XL donor. MTP was disabled during evaluation. No vision tensors or multimodal projector are included.
• Made by:The Kaitchup.
• License:Apache 2.0.

The filename suffixesQ5_K_S,Q5_K_M, andQ6_Kare Hub compatibility and dominant-precision-family labels for thesecustom SLQ allocations.

Method:SLQ paper·SLQ repository.

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#candidate-sourcesCandidate sources

The search consideredUD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL, Q4_0, Q8_0, and BF16fromunsloth/Qwen3.8-27B-GGUF, plusIQ3_SfromISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF. Identical tensor payload alternatives were canonicalized. The final three models select no GSQ donor tensors; its candidates remained eligible during search.

Credit goes to the Qwen team for the base model, Unsloth for the GGUF donors, ISTA-DASLab for GSQ/RCO, and Michael Helcig, Eldar Kurtic and Dan Alistarh for SLQ. The base and donor repositories declare Apache 2.0 licensing; the full license is includedbelow. Original copyright and attribution rights remain with their authors.

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#precision-by-layerPrecision by layer

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#ear98EAR98

SVG

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#ear985EAR98.5

SVG

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#ear99EAR99

SVG

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#ear-comparisonEAR comparison

**EAR means Expected Acceptance Rate.**It measures the overlap between the original BF16 model’s next-token probabilities and those of the compared model. For EAR@K, both distributions are separately renormalized over the BF16 reference’s K most likely tokens. Full-vocabulary EAR uses all 248,320 token IDs. An EAR of 0.99 means 99% average probability overlap on that measured support; it does not mean 99% benchmark accuracy or identical generated answers.

Zoomable SVG

SLQ’s “statistically lossless” objective concerns approximate fidelity, not identical numerical outputs. These releases meet their stated measured EAR@10 targets. The same calibration corpus guided model selection, so these values are calibration fidelity rather than an independent held-out benchmark. Statistical equivalence on downstream tasks has not been established for these releases.

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#instructions-llamacppInstructions: llama.cpp

Use a llama.cpp build with Qwen3.8 architecture support. Native text scoring and standalone GGUF parity were validated on an NVIDIA GH200 with llama.cpp commit45451c366d4256f94fbc8845645e31a6b9155054. This validation did not cover image/video inference or speculative decoding.

Install the Hugging Face CLI, then download one file:

pip install -U huggingface_hub
hf download kaitchup/Qwen3.8-27B-SLQ-GGUF \
  Qwen3.8-27B-SLQ-EAR98-Q5_K_S.gguf --local-dir .

Start an OpenAI-compatible text server with the downloaded model:

llama-server \
  --model Qwen3.8-27B-SLQ-EAR98-Q5_K_S.gguf \
  --n-gpu-layers 99 \
  --ctx-size 8192 \
  --port 8080

To use EAR98.5 or EAR99, download the corresponding file from the table and change\-\-model.

Send a chat request:

curl http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Is Earth smaller than Venus?"}],
    "max_tokens": 256
  }'

https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF#supportSupport

If these models are useful,support the workorsubscribe to The Kaitchup.