@aehyok: Share an open-source project FreeToken, a local inference engine specifically for running ultra-large Mixture-of-Experts (MoE) models on consumer-grade computers. Qwen3.6 35B → 8GB RTX 4060 laptop @ 39 tok/s DeepSeek-V4-Flash 284B → R…
Summary
FreeToken is an open-source local inference engine designed to run large Mixture-of-Experts (MoE) models on consumer-grade computers, offering significantly faster performance than alternatives like Ollama with easy installation and native GUI.
View Cached Full Text
Cached at: 08/24/26, 03:44 AM
| Download | Paper | Developer Slack | Community Discord | Community WeChat |
Similar Articles
Wow, this project lets your 4060 run large models??? Researchers from UC Berkeley, MIT, and UT Austin have just open-sourced FreeToken: an inference framework specifically designed for running ultra-large MoE models on personal computers. According to the official paper, several sets of data have been produced...
FreeToken is an open-source inference framework developed by researchers from UC Berkeley, MIT, and UT Austin, specifically designed for running ultra-large MoE models on personal computers, achieving efficient inference by dynamically allocating GPU, CPU, and memory resources.
@AISuperDomain: Stop buying multi-GPU workstations to run large models! Open-source inference engine FreeToken integrates CPU, GPU, and memory: 8GB VRAM slim laptops run 35B MoE, home single-GPU gaming laptops handle 290B+! Completely solves the VRAM capacity issue, open-source and free: #AI #LLM …
Open-source inference engine FreeToken integrates CPU, GPU, and memory, enabling consumer hardware like 8GB VRAM laptops to run 35B MoE models, and home single-GPU gaming laptops to run 290B+ models, completely solving the VRAM limitation.
FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution
FreeToken is an edge-native serving system that dynamically maps computation and model state onto heterogeneous local hardware to run large open-weight models on personal machines, enabling efficient execution of models up to 753B on a single GPU.
FreeToken: Efficient Edge-Native MoE Serving (24 minute read)
FreeToken is an edge-native Mixture of Experts serving system that efficiently runs large open-weight models on consumer hardware by adaptively managing resources across heterogeneous edge devices.
I thought I’d done something extraordinary by running massive models on standard smartphones but
Creator of bigedgeonmoe open-source codebase enables running massive MoE models (up to 120B parameters) on mobile devices and consumer PCs, achieving 6 tokens/s for Qwen 35B on a mid-range phone.