Underrated Muse Glimmer

Reddit r/LocalLLaMA News

Summary

Benchmarking results reveal that Muse Glimmer surprisingly outperforms qwen3.8 in implicit knowledge tests, indicating smaller models can achieve competitive performance with RAG enhancements.

Benchmarked qwen3.8 xhigh, medium and muse glimmer. Xhigh effort mode with qwen3.8 took almost 30hrs. (And still failed on 16 cases because of the 32K output token limit) Medium effort mode and muse glimmer were 3-4 hours each. But I'm actually surprised by the muse glimmer results, they came better than the qwen. These benchmarks are on implicit knowledge of the model, which is a bit unfair to smaller models, but throw in a RAG and I'm sure they get on par with frontier models. I have taken the result of claude models directly from embedeval repo by ecro.
Original Article

Similar Articles

Introducing Muse Glimmer

Simon Willison's Blog

Meta introduces Muse Glimmer, a new 30B open-weights model under Apache 2.0, optimized for agentic task completion, reliable tool use, and multi-step reasoning. Simon Willison tests it locally with LM Studio and llm-coding-agent.