We made GLM-5.3-Flash run 3.3x faster locally! Local GGUF inference is now 1.6–3.4× faster with optimized decoding and bonus multi-token prediction. Run 3-bit on 128GB setups via Unsloth Desktop or llama.cpp. Guide: https://unsloth.ai/docs/models/glm-5.3-flash#faster-inference-and-mtp-support… GGUF: https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF…

GLM-5.3-Flash: How to Run Locally | Unsloth Documentation

Source: https://unsloth.ai/docs/models/glm-5.3-flash For the complete documentation index, seellms.txt. This page is also available asMarkdown.

1. Models

GLM-5.3-Flash: How to Run Locally

Run the new GLM-5.3-Flash aka ox-alpha model by Z.ai.

GLM-5.3-Flash, also known asox\-alpha, is Z.ai’s new 320B parameter (18B active) multimodal open model whichoutperformsGLM-5.2. GLM-5.3-Flash is the smaller version ofGLM-5.3and rivalsClaude Opus 4.8on coding and agentic benchmarks. You can now run the 1-bit model locally on 102GB RAM/VRAM or 3-bit on 128GB setups via llama.cpp orUnsloth. Thank you Z.ai for day-zero access.

Unsloth dynamic1-bit(93GB) GGUFs retains71% of top-1% accuracywhilst being85% smallervs BF16 (642GB). Dynamic 3-bit is 76% smaller and retains 87% accuracy.

​​Run GLM-5.3-Flash GuideDownload Unsloth

GLM-5.3-Flash was trained on 30T tokens and is built on a newly trained base model. Its hybrid sparse and linear attention architecture lowers long-context serving costs without sacrificing accuracy.

You can now run the model directly inUnsloth Desktop.

⚙️Usage Guidehttps://unsloth.ai/docs/models/glm-5.3-flash#usage-guide

GLM-5.3-Flash Requirements:https://unsloth.ai/docs/models/glm-5.3-flash#glm-5.3-flash-requirements

The smallest 1-bit quant works on 100GB RAM while 3-bit works on 128GB devices like a Mac or NVIDIA DGX Spark.Table: Hardware requirements(units = total memory: RAM + VRAM, or unified memory)

Recommended Settingshttps://unsloth.ai/docs/models/glm-5.3-flash#recommended-settings

GLM-5.3-Flash has3 thinking modes: Low, High, and Max. Use Max Thinking for complicated tasks. InUnsloth, you can easily select Low, High, or Max Thinking with a toggle in the chat area.

Use these settings for most use cases:

Default Settings (Most Tasks)

• Maximum context window:1,048,576.

Changing reasoning efforthttps://unsloth.ai/docs/models/glm-5.3-flash#changing-reasoning-effort

GLM-5.3-Flash uses Max reasoning by default. It also supports reasoning efforts wherereasoning\_effortcan be “low”, “high”, or “max”.

Faster Inference & MTP Supporthttps://unsloth.ai/docs/models/glm-5.3-flash#faster-inference-and-mtp-support

As of Sep 4, we’ve added several improvements and optimizations to our day-zerollama.cpp PR. We implemented faster decoding pat plus bonus MTP support, enabling up to3.3× faster inferenceat long context lengths!

Everything works out of the box inUnsloth Desktop, simply update to the latest version if needed. No additional modules or MTP files are required. Alternatively, you can follow ourllama.cppguide.

Using GLM-5.3-Flash UD-IQ1_S on 1xB200 and disregarding MTP first, we get:

Then once we add MTP, we see even larger payoffs especially for longer contexts. However we should stop at around n=2 as more draft tokens makes inference slower.

For shorter context lengths, we still see speedups, albeit up to 1.6x faster.

📈 Quantization Analysishttps://unsloth.ai/docs/models/glm-5.3-flash#quantization-analysis

We quantized GLM-5.3-Flash down to UD-IQ1_S 1bit (93.09GB) and it retains 71% of top-1% accuracy whilst being 85% smaller vs BF16 (641.64GB)

Dynamic 2-bit UD-Q2_K_XL is 109GB, is 83% smaller and retains 78% of accuracy. Dynamic 3-bit UD-IQ3_XXS is 120GB, is 81% smaller and retains 82% of accuracy. Dynamic 4-bit UD-Q4_K_XL is 200GB, is 69% smaller and retains 93% of accuracy.

Run GLM-5.3-Flash (Ox-Alpha) Locallyhttps://unsloth.ai/docs/models/glm-5.3-flash#run-glm-5.3-flash-ox-alpha-locally

You can now run GLM-5.3-Flash (Ox-Alpha) in Unsloth Desktop and llama.cpp with ourspecific PR. We are using 3-bitUD\-IQ3\_XXSin our demos as it fits on 128GB devices. Feel free to change quantization type.

Run in Unsloth DesktopRun in llama.cpp

🦥 Run GLM-5.3-Flash in Unslothhttps://unsloth.ai/docs/models/glm-5.3-flash#run-glm-5.3-flash-in-unsloth

GLM-5.3-Flash can now run inUnsloth Desktop, an open-source UI app for local AI.Unsloth automatically offloads to RAM and detects multiGPU setups. With Unsloth Desktop, you can run models locally onMacOS, Windows, Linux and:

• Search, download,run GGUFs, MLX and safetensor models
• Fast CPU + GPU inference via MLX and llama.cpp

Search and download GLM-5.3-Flashhttps://unsloth.ai/docs/models/glm-5.3-flash#search-and-download-glm-5.3-flash

Go toUnsloth Chator Model hub and search for GLM-5.3-Flash in the search bar and download your desired model and quant.

Run GLM-5.3-Flashhttps://unsloth.ai/docs/models/glm-5.3-flash#run-glm-5.3-flash

Inference parameters should be auto-set when using Unsloth, however you can still change it manually. You can also edit the context length, chat template and other settings.

For more information, you can view ourUnsloth inference guide. 1-bit running below:

Serve GLM-5.3-Flash with Unsloth APIhttps://unsloth.ai/docs/models/glm-5.3-flash#serve-glm-5.3-flash-with-unsloth-api

You can useunsloth runcommand and serve GLM-5.3-Flash via an API usingllama\-serverruntime flags, including context sizing, GPU layers, threading, sampling, networking, and tool configuration. For more info see ourAPI docsorunsloth start.

Unsloth is now readyhttps://unsloth.ai/docs/models/glm-5.3-flash#unsloth-is-now-ready

You can also do many other things with GLM-5.3-Flash via Unsloth Desktop like:

• **Train models:**Fine-tune text, diffusion,embedding, and more
• **Generate media:**Create and trainimages, video,TTSlocally

🦙Run GLM-5.3-Flash in llama.cpphttps://unsloth.ai/docs/models/glm-5.3-flash#run-glm-5.3-flash-in-llama.cpp

We need to use our specific llama.cpp PRhere. You can follow the build instructions below as well. Change\-DGGML\_CUDA=ONto\-DGGML\_CUDA=OFFif you don’t have a GPU or just want CPU inference.For Apple Mac / Metal devices, set\-DGGML\_CUDA=OFFthen continue as usual - Metal support is on by default.

To run the model, you can do:

Then to run it:

ReplaceUD\-IQ3\_XXSwith your preferred quant, such asIQ2\_XXSfor 2-bit once uploaded.

📊 Benchmarkshttps://unsloth.ai/docs/models/glm-5.3-flash#benchmarks

CharXiv Reasoning

w/ Tools

Last updated2 minutes ago

• ⚙️Usage Guide
• Recommended Settings
• Faster Inference & MTP Support
• 📈 Quantization Analysis
• Run GLM-5.3-Flash (Ox-Alpha) Locally
• 🦥 Run GLM-5.3-Flash in Unsloth
• 🦙Run GLM-5.3-Flash in llama.cpp
• 📊 Benchmarks

curl -fsSL https://unsloth.ai/install.sh | sh

irm https://unsloth.ai/install.ps1 | iex

unsloth run --model unsloth/GLM-5.3-Flash-GGUF:UD-IQ3_XXS

apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone --branch glm5next/upstream https://github.com/unslothai/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

pip install -U "huggingface_hub[cli]"
hf download unsloth/GLM-5.3-Flash-GGUF \
    --local-dir unsloth/GLM-5.3-Flash-GGUF \
    --include "*UD-IQ3_XXS*" # Use "*IQ2_XXS*" for 2-bit

./llama.cpp/llama-cli \
    --model unsloth/GLM-5.3-Flash-GGUF/UD-IQ3_XXS/GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf \
    --temp 1.0 \
    --top-p 0.95 \
    --chat-template-kwargs '{"reasoning_effort":"max"}'

Z.ai (@Zai_org): Introducing GLM-5.3-Flash

• Leading capabilities at a highly competitive price
• Natively multimodal with a 1M-token context window
• A 320B-A18B model released under the MIT License
• Previously previewed as Ox Alpha, running entirely on Chinese AI chips

Blog: