$16 refactor, 400 steps, 95% routed to open MoE
Summary
A developer built a routing layer on vLLM to route simple agent steps to a cheap open-source MoE model (21B active) and hard steps to Opus, reducing costs to $15.60 for a 400-step refactor with 93.4% success rate.
Similar Articles
Split my agent into a cheap router model and a premium synthesis model, bill dropped about 75%
A developer splits their AI agent's LLM calls into a cheap router model (GPT-OSS 120B) for tool-picking and a premium model (gpt-5.4) for synthesis, cutting costs by ~78% while maintaining output quality.
Routing agent work across 4 LLM tiers: orchestrator, advisor, deep reasoning, premier
The author shares a practical 4-tier LLM routing stack for agent work, where a fast orchestrator handles most requests and only escalates to expensive models when deep reasoning is required, significantly improving cost and interactivity.
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
This paper presents a method for compiling repeated standard operating procedure steps into validated, versioned tools before deployment, replacing inference-time code generation. In a fulfillment center alarm-triage system, this approach reduces p50 latency by 42% and end-to-end error rate by up to 53%.
6 weeks daily-driving an open-source desktop agent shell with a 3-model split (Haiku triager → Sonnet reviewer → Opus executor). Real cost numbers + what broke.
A 6-week real-world experiment using an open-source desktop agent shell with a three-model split (Haiku triager, Sonnet reviewer, Opus executor) reports a 64% cost reduction and details failure modes like context bloat and runaway sub-agents.
I built LOLM: a lower-cost LLM agent with live control decisions and sealed run receipts
The builder of LOLM announces a hybrid Transformer-SSM language model and agent system from Qira, featuring a controller for live decisions, run receipts, a CLI, coding sandbox, MCP support, and lower-cost hosted access.