I built a local-first hybrid router for AI Agent Skills (sub-20ms, zero tokens, runs on CPU) [P]

Reddit r/MachineLearning Tools

Summary

Routed is an open-source, local-first hybrid router for AI agent skills that runs offline on CPU using a combination of dense embeddings, BM25, and exact matching to route prompts in under 20ms without API calls. The tool supports major agentic environments including Cursor, Claude Code, and MCP servers, and just released v1.1.0 with multilingual support and an MCP server mode.

If you use agentic workflows with custom skills or rules (Cursor rules, Claude Code slash commands, OpenCode, etc.), you have probably run into the routing trade-off: Stuff every skill definition into the system prompt (destroys your context window and degrades instruction-following). Use an LLM router turn to classify the user prompt (costs money, wastes 1,000+ tokens, and adds 2+ seconds of network latency). To solve this, I built Routed; an open-source, local-first hybrid router for agent skills that runs 100% offline on your CPU. GitHub: https://github.com/bshea-1/Routed License: MIT https://i.redd.it/5ca68gffgxnh1.gif How it Works Under The Hood Routed indexes your installed skill directories and evaluates prompts through a 4-part hybrid scoring pipeline: * Dense Vector Embeddings (60%): Runs quantized ONNX models (Arctic Embed S / MiniLM) locally on CPU. * Lexical BM25 (25%): Okapi BM25 for strict keyword relevance. * Exact / Alias Match (10%): Direct command and alias matching. * Metadata (5%): Recency and usage heuristics. The entire lookup completes in under 20ms without sending a single byte of prompt data over the wire. Supported Environments Routed auto-detects and injects adapters into: Cursor, Claude Code, LM Studio, Ollama, Antigravity IDE, Windsurf, OpenCode, Continue, Codex, and I just dropped support for MCP Servers!! And although v1.0 dropped last night, I just shipped v1.1.0 with two major additions based on early feedback: Model Context Protocol (MCP) Server (routed mcp): Instead of loading 20+ tool schemas into your GPU's context window, your local model only sees a single route_skill tool. Routed executes on CPU, selects the exact skill needed, and injects only that schema on demand. Native Multilingual Understanding: The embedding pipeline now natively understands input across 100+ languages (German, Spanish, French, Japanese, etc.) and automatically decomposes compound nouns (like German Speicherleck), mapping prompts directly to the correct skill without language tags or manual translation. Usage Inside your agent chat, you can just use /route to trigger the best skill(s) dynamically. It also handles compound intents (e.g., matching multiple skills when a prompt asks for two distinct tasks). Pre-built binaries are available on GitHub Releases (macOS .pkg, Linux .deb, Windows .exe), or you can build it from source via Node. Check it out and let me know what you think or if there are other environments you would like added!
Original Article

Similar Articles

Show HN: Open-source model routing for coding agents at Astra-level performance

Hacker News Top

Weave Router 2.0 is an open-source model router for coding agents that intelligently switches between LLMs, matching GPT-6 Astra's pass rates on Terminal Bench 4.0 and SWE Atlas while cutting costs to ~52% and speeding tasks 2.2–2.5x via an HMM-based architecture, larger training data, and smarter cache-eviction calculations.

Switchcraft: AI Model Router for Agentic Tool Calling

arXiv cs.AI

This paper introduces Switchcraft, the first AI model router specifically optimized for agentic tool calling to reduce inference costs. By using a lightweight DistilBERT classifier, it achieves significant cost savings while maintaining high accuracy in tool-use tasks.

🚀 AgentRouter Just Got Even More Powerful!

Reddit r/AI_Agents

AgentRouter is a developer platform that provides a unified API to access and switch between multiple AI models, such as GPT and Claude variants, for tasks like coding, reasoning, and automation.