@AlmustyFX: This is the kind of local AI test that actually matters. A 7.9B MoE running at 152 tok/s on an M4 Pro with 64GB unified…
Summary
A 7.9B MoE model runs at 152 tokens per second on an M4 Pro with 64GB unified memory, enabling offline processing of sensitive contract data and demonstrating the practical use of local AI.
View Cached Full Text
Cached at: 08/29/26, 04:07 PM
This is the kind of local AI test that actually matters.
A 7.9B MoE running at 152 tok/s on an M4 Pro with 64GB unified memory, while handling sensitive contract data completely offline.
No cloud. No data leaving the machine. Just a lightweight model doing useful work in under a second.
Local AI is becoming less of a demo and more of a real tool.
洛一可🍥 (@luoyike2003): 手痒把 Ling-3.0-tiny 的 MLX 4bit 拉到本机跑了一圈，把流程、选型理由和踩的坑一起记下来
【机器】Apple M4 Pro，64G 统一内存，macOS 26.5.1
【为什么挑 4bit 这一版】
Ling-3.0-tiny 是 7.9B 总参、每 token 只激活 1.3B 的稀疏 MoE：128 个路由专家取 top-8 加 1 个共享专家，24 层，注意力是
Similar Articles
I measured it: swapping a dense 27B for a 30B-A3B MoE changes the local concurrency ceiling (same rig as prior test, one variable change)
The author tests and compares the concurrency performance of dense versus MoE AI models on a MacBook Pro, finding that MoE models scale significantly better due to lower memory bandwidth usage per token.
@analogalok: Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what yo…
Alok demonstrates running Gemma 4 26B MoE on 8GB VRAM using Unsloth's QAT quant and the -cmoe flag in llama.cpp, achieving 20 tokens/sec with 250k context, marking a major milestone for budget local AI.
@ciruai: Testing DeepSeek v4 Flash on the AMD Ryzen AI Max+ 395 Strix Halo with 128GB RAM. Getting ~15 TPS over a decently long …
Testing DeepSeek v4 Flash on the AMD Ryzen AI Max+ 395 with 128GB RAM achieves ~15 TPS for a 284B MoE model (13B active) locally, costing $3,000 versus $25,000+ for a datacenter setup, highlighting the feasibility of running large models on consumer hardware.
@analogalok: I just got Gemma 4 26B A4B MoE model running fully locally with Hermes agent on an 8GB RTX 4060 and it's now backtestin…
A developer demonstrates running Gemma 4 26B MoE model locally on an 8GB RTX 4060 with Hermes agent to fully automate backtesting of trading strategies, highlighting the growing capability of local LLMs as autonomous agents.
@TheAhmadOsman: HOLYYYY 27B model under 6GBs and 4GBs Local AI will be the default P.S. We are gonna get this optimized in ODS by @Osma…
Ternary Bonsai 27B, a large language model, is demonstrated running locally on an NVIDIA RTX 5090 GPU, requiring under 6GB of memory and enabling end-to-end agentic workflows on consumer hardware.