LEMoE is an open-source, stateless Mixture of Experts (MoE) router that acts as an API proxy to route prompts to specialized LLMs, featuring cascading contextual routing and silent self-correction.
Hi everyone, I wanted to share a project I’ve been working on called LEMoE (Light Easy Mix of Experts). **The Backstory & Why I Built It**: I’ve always been fascinated by the Mixture of Experts (MoE) architecture, but I wanted to take the concept further and use it in a more extended way. I felt that most existing solutions were either too heavy, baked into specific model weights, or lacked advanced routing logic. I wanted a flexible, external routing layer that could orchestrate different specialized APIs (Ollama, OpenAI, etc.) with more practical, production-ready features. **What it does & How it works**: LEMoE acts as an API proxy (fully compatible with OpenAI and Ollama clients). You configure different "experts" (LLMs specialized in coding, writing, reasoning, etc.) via JSON. When a prompt comes in, it routes it to the best expert. But I wanted to add some smart features that make it stand out: * **Cascading Contextual Routing:** Most API routers only evaluate the very last prompt, which breaks down when a user says something ambiguous like "make it shorter". LEMoE statelessly evaluates the last 2-3 messages in the conversation history to maintain topic continuity, cascading down only if confidence is low. * **Silent Self-Correction**: If one of your backend experts fails (API timeout, server down, etc.), LEMoE silently and instantly redirects the request to a fallback expert. The end user never sees an error, and it’s logged server-side for the admin. * **Completely Stateless**: It doesn't require databases, complex sessions, or heavy RAM usage. Everything is handled on the fly using standard API message arrays. How it compares to competitors: Unlike native MoE models (which require massive VRAM and dedicated hardware to load multiple experts), LEMoE lets you run lightweight local models (or mix them with external APIs) on standard hardware. Compared to simple API routers, LEMoE handles multi-turn conversation context for routing and offers built-in silent error failovers out of the box. **Current State & License:** The project is actively developed. It's ready to use, but since it’s in active development, there might still be some bugs. I would absolutely love it if you guys could test it out and give me some feedback, suggestions, or feature requests! It is completely free and open-source for personal/non-commercial use. **Links:** GitHub Repository: [https://github.com/lemoelink/LeMoE](https://github.com/lemoelink/LeMoE) Documentation (EN): [https://docs.lemoe.link/en/](https://docs.lemoe.link/en/) Official Website: [https://lemoe.link/](https://lemoe.link/)
dMoE proposes block-level expert routing for diffusion LLMs, reducing the number of uniquely activated experts from 69.5 to 14.6 while retaining 99.11% performance and achieving 76-80% memory reduction with 1.14-1.66× speedup.
Mix-MoE proposes a mixed Mixture-of-Experts framework with specialized expert groups and Fourier-transform-enhanced routing to mitigate parameter interference in multilingual machine translation, achieving significant improvements over baselines.
StickyMoE proposes a differentiable routing consistency loss that encourages adjacent tokens to activate the same experts in MoE models, reducing expert-swapping overhead and cache misses during inference on edge devices by up to 3.92× while improving perplexity.
This paper proposes Multi-level Context Fusion MOE (MCF-MOE), a framework that improves routing consistency in Mixture-of-Experts models by integrating cross-layer semantic aggregation and local token-level interactions, outperforming strong baselines on language modeling and understanding benchmarks.
This paper extends refusal steering (activation-based jailbreaking) to Mixture-of-Experts LLMs, finding that MoE routing patterns do not inhibit steering, and proposes expert-aware methods that can suppress refusal behavior based on a single expert's output.