JetBrains Mellum 2.1 12B A2.5B MoE is a very good model.
Summary
JetBrains Mellum 2.1 12B A2.5B MoE 被评价为一款表现优秀的编码模型，能在 M4 Max 上以 130k 全上下文跑出 60 TPS，在 9070XT 上达到 140 TPS，且可在 8GB/12GB 显卡上流畅运行。
Similar Articles
Mellum 2 12B A2.5B
JetBrains released Mellum 2 12B A2.5B, a coding-focused small MoE model with reasoning performance comparable to Qwen 3.5 9B but weaker in other tasks.
Mellum2 Technical Report
Mellum 2 is a 12B-parameter open-weight MoE language model by JetBrains with 2.5B active parameters, specialized in software engineering tasks and optimized for efficient inference on commodity GPUs.
JetBrains's Mellum 2 (49 minute read)
JetBrains releases Mellum 2, a 12B-parameter open-weight Mixture-of-Experts language model specialized in software engineering, with competitive performance in code generation, reasoning, and tool use, available under Apache 2.0.
JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP · Hugging Face
JetBrains released Mellum2.1-12B-A2.5B-Thinking-MTP, a 12B MoE (2.5B active) thinking coding model with a multi-token prediction head for speculative decoding in vLLM. The update focuses mainly on reinforcement learning post-training in real repository environments to improve agentic software engineering tasks.
24+ tok/s from ~30B MoE models on an old GTX 1080 (8 GB VRAM, 128k context)
A developer demonstrates running MoE models like Qwen 3.6 35B-A3B and Gemma 4 26B-A4B at 24+ tok/s on an old GTX 1080 (8GB VRAM) with 128k context using llama.cpp with MoE offloading and TurboQuant KV cache quantization, revealing optimization tricks for Gemma's MTP speculative decoding.