I want to setup and build an unstoppable local AI (full guide):

If you’re able to setup GLM 5.2 then you’re able to finally have access to run cheaper agentic loops, save money on Claude/Codex sessions, hold massive codebases, automate the boring tasks, test local agents, and more.

• It is the strongest open sourced model available right now.

• It beats GPT-5.5 on several coding benchmarks.

• It still trails Claude Opus 4.8 on most raw scores (not all).

All three of those are true at the same time, but the benefits of GLM 5.2 is how cheap it is to utilise whilst being comparatively close to the other frontier models that exist right now.

So if you want a self-owned, repo-scale, agentic coding engine, cheap but powerful model, then read on.

The benchmarks:

shoutout to GPT for its image creation skills for these lil graphics in my article.

shoutout to GPT for its image creation skills for these lil graphics in my article.

So what is it good at?

GLM-5.2 is a coding-and-agentics-first foundation model. It is built for long-horizon, task-driven work, not conversational chat. Z.ai’s own framing is a model that “stably sustains long-horizon work” across “long, messy coding-agent trajectories.”

Put simply, it’s fucking great at:

• Agentic, repo-scale coding.

• Frontend and design.

• Long-context engineering.

• Maths and reasoning.

What it’s not so good at:

• Abstract reasoning without tools.

• Vision and audio.

• Verbosity and speed.

• Creative writing.

It trails Claude Opus 4.8 on most shared coding benchmarks, with gaps ranging from under 1 point to around 21 points depending on the benchmark. Against GPT-5.5 it is mixed: it wins on some coding tasks and loses on others.

But, that’s not the point really. This is comparatively very god damn good and changes the game completely when it comes to function and cost.

Oh, what’s it cost?

cheap brev.

cheap brev.

if this is you looking at the table above, just know, it’s VERY CHEAP.

if this is you looking at the table above, just know, it’s VERY CHEAP.

Alright then hoeem, so how can I set this up?

You have a few options, so I have provided a guide for each one in this section, it’s cloud, or it’s local and self-hosted, I’ll start by sharing the cloud version:

Path A: Cloud

This is what almost everyone should do first.

Option 1: GLM Coding Plan subscription (best value for interactive coding)

Tiers: Lite from $18/mo. Pro and Max are higher (around $72 and $160/mo base per secondary sources, so verify on the live subscribe page). Discounts run 10% monthly, 20% quarterly, 30% yearly, which brings Lite to about $12.60/mo on the annual plan.

Limits are prompt-based per 5-hour rolling cycle: Lite around 80, Pro around 400, Max around 1,600 prompts. Weekly caps are Lite 400 / Pro 2,000 / Max 8,000. One “prompt” triggers roughly 15 to 20 model calls.

MCP monthly quotas (web search and reader): Lite 100, Pro 1,000, Max 4,000.

Quota multiplier: GLM-5.2 deducts 3x during peak hours (14:00 to 18:00 UTC+8) and 2x off-peak. A limited-time promo makes it 1x off-peak through the end of September 2026.

(I think this is information overkill, oh well, I want this guide to be a guide that covers it all, but basically you go and subscribe to z.ai/subscribe. All its tiers support GLM 5.2. The plan is restricted to officially supported tools (Claude Code, Cline, OpenCode, Roo Code, Kilo Code, OpenClaw, Cursor, Crush, Goose, and others). SDK or unsupported-tool access can be throttled.

Claude Code setup (exact steps)

Get a Z.ai API key: log in at z.ai/chat, open the profile menu, go to API Keys, and create a new key (you can re-copy it later).

Then either run the automated helper (Coding Tool Helper / npx @z_ai/coding-helper) or edit ~/.claude/settings.json:

Run claude, verify with /status (it should show GLM-5.2), and switch effort with /effort (xhigh/max/ultracode map to Max, low/medium/high map to High).

Three details that matter:

The [1m] suffix selects the 1M-context variant. API_TIMEOUT_MS=3000000 (50 minutes) is mandatory so long calls are not killed. CLAUDE_CODE_AUTO_COMPACT_WINDOW=1000000 lets Claude Code use the full window before summarising. CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 appears in Z.ai’s own config example and suppresses background calls that the gateway does not expect.

Verify routing with /status. That is the official check and it should show GLM-5.2. A self-identification question (“What model are you?”) works only as a rough sanity check, not a definitive one, because models often misreport themselves through system prompts or compatibility wrappers.

A handy shell alias (set ZAI_API_KEY first):

Other coding tools

Base URLs:

Anthropic-compatible: https://api.z.ai/api/anthropic (for Claude Code and Anthropic-SDK harnesses) OpenAI-compatible coding endpoint: https://api.z.ai/api/coding/paas/v4 (for Cline, OpenCode, Roo Code, Kilo Code, Goose, Crush, OpenClaw) General programmatic endpoint: https://api.z.ai/api/paas/v4

Model string: glm-5.2, or glm-5.2[1m] for full context. For OpenAI-compatible CLIs:

OpenClaw users can manually add a glm-5.2 model object (contextWindow 1000000, maxTokens 131072) to ~/.openclaw/openclaw.json, set it as agents.defaults.model.primary, and run openclaw gateway restart.

Option 2: Pay-as-you-go API

Create an account at z.ai/chat, generate a key, and use the endpoints above. Rates: $1.40/M input, $0.26/M cached, $4.40/M output (cached-input storage is currently a limited-time free promo).

Caching matters. Keep the stable prefix at the front of the prompt and you cut repeated-context input by around 81%.

Worked example: a 100K-input, 20K-output coding turn costs roughly $0.14 plus $0.088, so about $0.23. With 80K of that input served from cache it drops to around $0.13.

Also available via OpenRouter (z-ai/glm-5.2, around 1.20/4.10 blended across providers, with Balanced/Nitro/Exacto routing) and Requesty (zai/glm-5.2, base_url https://router.requesty.ai/v1).

Option 3: Ollama cloud-routed

This runs on Ollama’s US NVIDIA Blackwell GPUs with a large hosted context window and requires an Ollama cloud subscription. The :cloud tag is hosted, not local. Local GGUF use is possible through Unsloth’s Hugging Face routes (for example hf.co/unsloth/GLM-5.2-GGUF), but do not confuse that with Ollama’s official hosted glm-5.2:cloud model.

You can launch agents against it, for example ollama launch hermes –model glm-5.2:cloud, or hit the local daemon (http://localhost:11434/v1) with model glm-5.2:cloud.

Path B: Genuinely Local and Self-Hosted

Hardware reality check

Full BF16 weights are around 1.5 TB. FP8 is around 756 GB on disk. The reference production deployment is 8x H200 or 8x H20 (141 GB each) for single-node FP8, or 8x B200 (180 GB each) for the full 1M context.

That translates to: FUCKING EXPENSIVE.

But there’s other options available and I’ll explain that here (be aware this is going to get fucking nerdy, there’s no other way to give the detail required without it getting super nerdy, so if you need this broken down further just copy and paste everything below and get a model of choice to break it down into digestible chunks).

Quantisation, the realistic local path:

Unsloth shipped day-zero dynamic GGUF quants (unsloth/GLM-5.2-GGUF):

Dynamic 2-bit (UD-IQ2_M): around 239 GB. Fits a 256 GB unified-memory Mac, or 1x 24 GB GPU plus 256 GB RAM with MoE offloading. Retains around 82% top-1 accuracy (KLD-tested). The sweet spot for most people.

Dynamic 1-bit: around 217 GB (fits around 223 GB RAM), around 76% accuracy.

4-bit / 5-bit (dynamic): sizes range roughly from 365 GB to 467 GB depending on quant. UD-Q4_K_M is around 373 GB; UD-Q4_K_XL is around 467 GB; UD-Q5_K_XL sits above that. Unsloth describes these as generally lossless by KLD testing, so use them for out-of-distribution work.

8-bit: around 810 GB RAM.

Reference formats: BF16 (zai-org/GLM-5.2) and FP8 (zai-org/GLM-5.2-FP8, around 756 GB, the popular self-host choice. FP8 led BF16 downloads roughly 8 to 1 in the first days).

MoE means quantisation errors are diluted across inactive experts, so 2-bit stays surprisingly usable for coding.

Performance expectations:

On consumer hardware with 2-bit quants, expect around 3 to 9 tokens per second (community-reported on M3/M4 Ultra Mac Studio, 256 GB). An H200 with Q2_K_XL has been reported at around 8.7 tok/s. Treat both as anecdotal until you benchmark your own box. Fine for batch and agentic coding, not for snappy chat.

Running the full 1M context locally is impractical on consumer gear. You need the multi-GPU data-centre config plus KV-cache quantisation.

Inference engines and commands

llama.cpp (Mac/PC, GGUF). Build with CUDA (cmake -B build -DGGML_CUDA=ON) or Metal (automatic on Mac). Download a quant:

Serve it:

This exposes an OpenAI-compatible endpoint at localhost:8080. Use KV-cache quantisation (q4_1/q5_1) to stretch context around 3x.

vLLM (FP8, 8 GPUs, v0.23+):

SGLang (v0.5.13+):

(add –tp-size 8 for multi-GPU)

Also supported: xLLM (v0.10+, for Huawei Ascend), Transformers (v0.5.12+), KTransformers (v0.5.12+).

LM Studio: GUI. Search “Unsloth GLM-5.2-GGUF,” one-click download, built-in OpenAI-compatible local server. Best for non-CLI users and easy quant-swapping.

Driving an agentic workflow fully locally

Once llama.cpp, vLLM, SGLang, or LM Studio exposes an OpenAI-compatible endpoint (for example http://localhost:8080/v1), point Aider, Cline, Hermes, or Claude Code (via an OpenAI-compatible bridge or router) at it. Set base_url to your local server and model to your alias.

That gives you a fully self-owned agentic coding loop. No cloud dependency. No data leaving your machine.

BUT… what does running local actually cost?

This is the worst time in years to buy hardware for local LLMs. The 2026 AI-driven DRAM shortage has inflated exactly the components GLM-5.2 leans on hardest, memory and VRAM, and the shortage is forecast to persist until at least Q4 2027, with no real price relief expected before 2028.

The model is memory-bound. Even the smallest usable quant (around 239 GB) must live entirely in RAM or VRAM, so the cost story is really a memory-price story.

The broad shortage picture is well-documented. The specific dollar figures below are mid-2026 spot-market estimates that move fast and vary by region and configuration. Check live retail and Apple pricing before you spend.

Tier 1: Mac Studio (the realistic “one box”), around $6,000, if you can get one

A 256 GB M3 Ultra is the cleanest consumer fit, thanks to unified memory: the CPU and GPU share one large pool, so a single box holds a model that would otherwise need a rack of GPUs.

In March 2026 Apple raised the 96GB to 256GB upgrade from $1,600 to $2,000 and discontinued the 512 GB option entirely because of the shortage. A 256 GB M3 Ultra now lands somewhere around $6,000 all-in (a US ballpark that depends on configuration and country), with availability reported as tight.

Old 512 GB units, which ran the bigger near-lossless quants comfortably, now trade on the second-hand market at a steep premium. Power draw stays low, an estimated 200 to 300W. There is a dedicated Mac Studio walkthrough below.

Tier 2: GPU plus system-RAM offload build, around $4,000 to $8,000, and slower than the Mac

The “budget” route (1x 24 GB GPU plus 256 GB RAM, MoE offload via llama.cpp) is sabotaged by the same shortage. A 256 GB DDR5 kit has climbed to roughly the price of a high-end GPU, and the RTX 5090 (a $1,999 MSRP card) has been selling well above list. Both figures are volatile spot-market prices, so check current retail before budgeting.

Add CPU, board, PSU, and storage and you reach $4,000 to $8,000 for something that runs slower than the Mac. Used RTX 3090 (24 GB) builds are cheaper, but even used cards are elevated.

Tier 3: Run it properly (FP8, full speed, long context), $250,000+ to buy

I AM NOT GOING TO RUN IT PROPERLY LMAO.

The reference config, 8x H200-class GPUs (141 GB each, around $30k to $40k apiece), is around $250,000 to $320,000 for the cards alone, and $300k+ with chassis, networking, and power. This is a data-centre purchase, not a home build, and it carries four-figure annual power-and-cooling bills.

But you can always rent this for the task at hand!

Price table of the different options:

Can you run it on a Mac Studio? Yes

The Mac Studio is the best consumer machine for GLM-5.2, precisely because unified memory lets one box hold a model that would otherwise need multiple GPUs.

On a 256 GB Studio you can buy today, the 1-bit quant runs comfortably and 2-bit is a tight squeeze (macOS needs part of the pool, and you need headroom for the KV/context cache). Expect around 3 to 9 tokens per second: fine for fire-and-forget agentic coding, frustrating for chat.

Step 1. Install LM Studio (easiest, GUI, one-click downloads plus a built-in OpenAI-compatible server) or llama.cpp(more control, Metal is automatic on Mac).

Step 2. Download unsloth/GLM-5.2-GGUF and pick the 1-bit (safe) or 2-bit (tight) quant. CLI:

Step 3. Raise the macOS GPU wired-memory limit so the model can claim most of the pool, for example:

(around 240 GB. Tune to your build, leaving the OS around 16 GB.)

Warning: advanced, Mac-only tuning. This changes GPU wired-memory behaviour and can destabilise the machine if you set it too aggressively. Use it only if you understand the trade-off, leave the OS real headroom, and be ready to revert it on reboot if anything misbehaves.

Step 4. Serve it. In LM Studio, load the model and start the local server. Or with llama.cpp:

(use KV-cache quant to stretch context.)

Step 5. Point your agentic tool at it. Set base_url to http://localhost:8080/v1 (or LM Studio’s port) and the model to your alias in Claude Code (via OpenAI bridge), Cline, or Aider for a fully offline coding loop.

Prompting and optimisation (model-specific)

Treat it as an agent, not a chatbot. GLM-5.2 is post-trained for goal-driven, tool-using, long-horizon execution. Give it a clear objective, the relevant context, success criteria, and constraints. Skip the conversational back-and-forth.

Effort switching. Use Max effort for hard or important coding tasks (deeper reasoning, more stable multi-step performance). Use High for simpler or faster work. As a rough heuristic rather than a measured constant, High roughly halves output tokens for only a few points of capability loss, which makes it a real cost and latency lever. In Claude Code use /effort. Via API set reasoning_effort (“high”, or unset / anything-else for Max), and enable_thinking=false for cheap, fast completions.

Context loading for the 1M window. Load all relevant files, specs, and tests upfront rather than chunking. Keep the stable content (system prompt, tool defs, repo map, unchanged files) at the front so prompt caching applies (around 81% discount on the cached prefix), and put the variable request at the end.

One caveat that matters: long-context retrieval quality at 800K+ tokens is not yet independently verified. Treat 1M as a ceiling, not a guarantee, and test it on your own repo.

Optimal prompt structure. Task in one sentence, plus context (files, environment, prior decisions), plus explicit success criteria, plus constraints (libraries, style, what not to change). Pair it with the autonomous-run or plan-mode features in your harness for long agentic sessions.

Integration tips. MCP servers (Vision MCP via GLM-4.6V, Web Search/Reader MCP) work through the Coding Plan and inside Claude Code and Cline.

Watch one known failure mode: on long agentic loops the Anthropic bridge occasionally drops nested tool-result content. The symptom is the model repeating a tool call instead of acknowledging it. When that happens, fall back to the OpenAI-compatible /api/coding/paas/v4 endpoint. Set generous timeouts everywhere. For multi-file repo-scale work, lean on the large window plus caching rather than manual chunking, and route short or simple queries to a cheaper, faster model.

GLM 5.2 makes looping a real option for people that don’t have unlimited budgets, the future looks incredibly bright right now.

Recommendations:

1. You just want to code today, minimal setup. Subscribe to GLM Coding Plan Lite ($18/mo), wire it into Claude Code with the env block above, set /effort max, and work in the off-peak window (1x quota promo through September 2026). Threshold to upgrade: if you regularly hit the around 80 prompts per 5-hour Lite cap, or open multiple terminals, or run sub-agents, move to Pro (around 400 prompts per 5 hours).

2. You are building a product or running programmatic usage. Use pay-as-you-go API (1.40/0.26/$4.40) with aggressive prefix caching, or OpenRouter/Requesty for managed failover. Route only long-repo and long-horizon tasks to GLM-5.2 and send short queries to a cheaper model. So if you cannot predict daily volume within 2x, metered API plus caching beats the subscription.

3. You have a data-sovereignty or no-China-routing requirement, or very high volume. Self-host. With 8x H200-class GPUs, run FP8 via vLLM or SGLang for the full experience including longer context. If not, run Unsloth 2-bit GGUF on a 256 GB Mac Studio or a 24 GB GPU plus 256 GB RAM box via llama.cpp, accepting around 3 to 9 tok/s for batch and agentic work. Threshold to self-host: sustained heavy volume, strict compliance, or air-gap needs. Otherwise the Coding Plan is cheaper and far less effort. (See “What Local Actually Costs” above for the full breakdown.)

4. You need vision, creative writing, or the fastest and cheapest reasoning. Do not use GLM-5.2. It is text-only, verbose, and a coding specialist. Use a multimodal or general model for those jobs.

5. Signals that would change these recommendations. Independent confirmation of 800K+ context retrieval quality. Community FP4 quants dropping the production rig from 8x H100 to 4x H100. And third-party SWE-bench Verified, LiveCodeBench, or Aider polyglot results, still missing from the public set. Any of these landing strengthens the self-host case and the “trust the benchmarks” case.

P.S. People will tell you GLM 5.2 is overrated, but it’s completely changed the game for people that want to run agentic loops who don’t have silly money, you can run this on the cloud and send it.

The future is indeed very bright. Wow. Wow. Wow.

I’m going to be writing a lot more guides on substack soon.

Please come and join me on: sevenc.substack.com

Let’s fuck (no homo).