We made the preview publicly available: https://huggingface.co/trymirai/Qwen3.8-27B-S-experimental… Qwen3.8 27B that fits in 8.5Gb Works with an experimental build of uzu (our inference engine) on Apple Silicon, and in vllm on Nvidia See the model card for instructions

trymirai/Qwen3.8-27B-S-experimental · Hugging Face

Source: https://huggingface.co/trymirai/Qwen3.8-27B-S-experimental Qwen3.8-27B quantized with the Mirai S codec: 2.4 bits per weight, 8.45 GB on disk.

**This is a work in progress and the numbers here are not final.**Work on both runtimes is unfinished. Treat anything measured with it as provisional.

On Apple silicon it runs on a prebuilt uzu runtime that ships in this repo. On NVIDIA it runs on stock vLLM with a small plugin that also ships in this repo.

https://huggingface.co/trymirai/Qwen3.8-27B-S-experimental#apple-silicon–uzuApple silicon — uzu

You need an Apple silicon Mac on macOS 26.4 or newer with 24 GB of memory or more. Theuzu/folder holds a prebuilt runtime, so there is nothing to build:

hf download trymirai/Qwen3.8-27B-S-experimental --exclude "vllm/*" --local-dir qwen3.8-s
chmod +x qwen3.8-s/uzu/uzu
qwen3.8-s/uzu/uzu server --model qwen3.8-s

That starts an OpenAI-compatible endpoint on port 8000:

curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"qwen3.8-s","messages":[{"role":"user","content":"Capital of Australia? One word."}],"enable_thinking":false,"max_tokens":8}'

To chat in the terminal instead, leave outserver. With the server running,python3 qwen3\.8\-s/speedcheck\.pymeasures your Mac in under a minute; on an M5 Pro it prints 52 tokens/s decoding code (17 on prose), 440 tokens/s prefill, and 0.4 s to start answering a follow-up in a 7k-token chat. Follow-ups stay fast because uzu keeps the conversation cached between turns. More inuzu/README.md.

To build uzu yourself instead, install Rust withrustup, clone uzu’sryan/qtip\-s\-agentbranch, and runcargo run \-\-release \-p cli \-\- server \-\-model <model dir\>inside it. If the build stops because the Metal toolchain is missing, runxcodebuild \-downloadComponent MetalToolchainand try again.

Thinking is on by default. Turn it off per request with"enable\_thinking": false, or server-wide by starting the server withUZU\_SERVER\_THINKING=0. Things to know:

• uzu’s levels arelow/medium/high/xhigh.
• WithUZU\_SERVER\_THINKING=0, sendenable\_thinkingtogether withreasoning\_effort;reasoning\_efforton its own returns 400.
• Streaming replies carry the thinking asreasoning\_contentdeltas, then the answer ascontent, the way vLLM’s reasoning parser does.

https://huggingface.co/trymirai/Qwen3.8-27B-S-experimental#nvidia–vllmNVIDIA — vLLM

Thevllm/folder (9.3 GB download) holds a vLLM-ready copy of this checkpoint and themirai\_splugin, which runs the compressed weights directly on the GPU: 8.2 GiB of GPU memory, so a 12 GB card runs it and a 16 GB card has room for 54k tokens of context with speculative decoding. On an RTX 3090 it answers one chat at 86 (prose) to 141 (code) tokens/s with speculative decoding, 135 tokens/s while reasoning, serves 620 tokens/s across 64 parallel requests, and has room for about 170k tokens of context. Install and serve commands, including one for drivers older than 580, are invllm/README.md.

https://huggingface.co/trymirai/Qwen3.8-27B-S-experimental#known-issuesKnown issues

• **Both runtimes:**a tightmax\_tokenswith thinking on can spend the whole budget inside the trace and return an empty answer. Give reasoning requests at least 4000 tokens.

Artur Chakhvadze (@norpadon): We may have cooked a 2-bit version of Qwen3.8 27B which actually works, is not deep fried to benchmaxx, and fits in 8.5GB

We have not yet finished evaluating it, and it’ll take some time to properly ship it, but if someone wants to try and share feedback please dm me