@lxfater: Two old V100s can also run the 321 billion parameter GLM-5.3-Flash locally!! This open-source project is called Project…

X AI KOLs Timeline Tools
local-llm open-source inference llm-serving quantization gpu-offloading

Summary

开源项目 Project Maya 基于 Strata 引擎并适配 GLM，让两张 V100 32GB 显卡加约 32GB 内存和 NVMe SSD 即可本地运行 3210 亿参数的 GLM-5.3-Flash，生成速度可达 40 tokens/s，还内置 Web 聊天、图像理解和 OpenAI/Anthropic API 兼容。

Two old V100s can also run the 321 billion parameter GLM-5.3-Flash locally!! This open-source project is called Project Maya. A few days ago, I wrote that with 12GB VRAM + 64GB RAM, you can run Qwen's 100-billion-parameter model using Strata. This time, Maya is also based on Strata, but with the engine modified to adapt to GLM. Author's actual test: 2 V100 32GB + about 32GB RAM + NVMe SSD, generation speed up to 40 tokens/s. The principle is still to keep commonly used model experts in VRAM, distribute the others to RAM and SSD, and load them in as needed. Defaults to using the about 90GB Maya-S quantized model. Comes with web chat, image understanding, and compatibility with OpenAI / Anthropic API, so you can connect programming agents, all running locally. Currently supports Linux; Windows / WSL2 not supported yet. If you have this hardware setup and want to run GLM locally, check it out:
Original Article
View Cached Full Text

Cached at: 10/08/26, 02:18 PM

Two old V100s can also run the 321 billion parameter GLM-5.3-Flash locally!!

This open-source project is called Project Maya.

A few days ago, I wrote that with 12GB VRAM + 64GB RAM, you can run Qwen’s 100-billion-parameter model using Strata. This time, Maya is also based on Strata, but with the engine modified to adapt to GLM.

Author’s actual test: 2 V100 32GB + about 32GB RAM + NVMe SSD, generation speed up to 40 tokens/s.

The principle is still to keep commonly used model experts in VRAM, distribute the others to RAM and SSD, and load them in as needed.

Defaults to using the about 90GB Maya-S quantized model.

Comes with web chat, image understanding, and compatibility with OpenAI / Anthropic API, so you can connect programming agents, all running locally.

Currently supports Linux; Windows / WSL2 not supported yet.

If you have this hardware setup and want to run GLM locally, check it out:

Similar Articles

"Strata" for GLM5.3 Flash is here for some! Project Maya

Reddit r/LocalLLaMA

Project Maya 是一个开源工具，可在本地 GPU 上运行 321B 参数的 GLM-5.3-Flash 模型，通过分层存储（GPU/RAM/NVMe）和自动调优实现单卡 30+ tokens/s 的推理速度，并提供 OpenAI/Anthropic 兼容 API。

@lxfater: With 64GB RAM, the 100-billion-parameter model is already deployed. We've got the hammer — now where's the nail?

X AI KOLs Timeline

A 12GB GPU with 64GB of system RAM is enough to run a 100-billion-parameter large model: the open-source project Strata can run quantized versions of Qwen3.8-Flash-Next — about 94 tokens/s with the Q2_0 quantization and about 53 tokens/s with IQ3_S on an RTX 5070. The author remarks that the hammer is ready; the question is where the nail (i.e., the application scenario) is.

@yibie: Recommend this project—a single person wrote an inference engine in pure C, making the 744-billion-parameter GLM-5.2 run on a consumer machine with 25GB RAM. No GPU, no BLAS, no Python runtime—about 1300 lines of C. The core insight is simple: MoE…

X AI KOLs Timeline

Colibri is an inference engine written in pure C, approximately 1300 lines of code, zero dependencies. It can run the 744-billion-parameter GLM-5.2 MoE model on a consumer machine with 25GB RAM, achieved by streaming loaded routing experts and efficient caching, no GPU or Python runtime needed.