@DeRonin_: i built a system to run LLMs without limits... my Token Router it's why Claude Code runs 10+ hours a day and my API spe…
Summary
The author describes a token routing system to optimize LLM usage, reducing API costs by up to 70% through techniques like cost-based routing, prompt caching, and output discipline.
View Cached Full Text
Cached at: 08/23/26, 09:41 PM
i built a system to run LLMs without limits… my Token Router
it’s why Claude Code runs 10+ hours a day and my API spend dropped ~70%
one layer between me and every model, pricing each call BEFORE it happens
here’s how to build the same tokens saver:
- route by cost-of-failure, not by task
every request gets classified first: what does a wrong answer cost here? boilerplate → local model. production work → cheap workhorse. architecture decisions → frontier. the router picks, not you
- make your prompts cache-eligible
providers give a 90% discount on cached input and almost nobody collects it.. because the cache breaks on a SINGLE changed byte. keep your prefix stable (tools → system → context, same order, same bytes) and put anything volatile (timestamps, fresh data) at the END. yes, your AI bill partially depends on byte order. nobody tells you this
- block redundant reads
agents re-read the same files constantly. a simple pre-tool hook that says “you already saw this, here’s the summary” kills the dumbest spend in every agentic workflow
- feed skeletons, not files
parse your codebase into a map of functions and calls (tree-sitter does this free). the model queries the skeleton and only opens files it actually needs.. up to 49x fewer tokens on big repos
- isolate the dirty work
sub-agents read the heavy stuff and return only conclusions. your main context stays clean.. think of it as the CEO’s inbox: reports come in, raw dumps don’t
- discipline the output side
output tokens cost ~5x input and everyone ignores them. demand diffs instead of full rewrites, terse mode by default, hard stop sequences. 30-50% cut from one instruction
- run overnight work in the batch lane
every provider gives ~50% off for non-urgent jobs. your nightly agents are paying rush-hour prices for work nobody reads until morning
the killer part: none of this is a tool you buy. it’s ~200 lines of routing rules and hooks you write once, and every session after that is discounted forever
same philosophy that runs my whole agency: spend intelligence like money, because it is
your AI bill isn’t the price of intelligence. it’s the price of lazy routing
P.S. this is the surface layer
the full router also does semantic caching (similar question = cached answer, zero cost) and compresses prompts with a small local model before the big one ever sees them
show me your interest and i’ll drop the full build guide with all configs
sharing everything for free
Similar Articles
TokenRouter: Efficient Serving System for Token-Level LLM Routing
TokenRouter 是一个面向 token 级 LLM 路由的高效推理服务系统,采用「请求中心编程、模型中心执行」架构与延迟批调度策略,实现较现有系统 2.01–64.15 倍的解码吞吐提升。
LLM Routing is not the problem to solve; token efficiency is
The article argues that model routing isn't the real problem to solve—token efficiency is. It advocates for a holistic closed-loop approach combining cheaper defaults, preference-aware routing, and better caching (e.g., Coinbase's 5%→60% cache hit improvement) to maximize useful intelligence per dollar.
@DeRonin_: i ran Fable 5 the whole day and still haven't touched my limits why? i stopped paying surgeon rates for small talk here…
A user shares a detailed workflow strategy for efficiently using multiple AI models (Fable, Opus, Codex, DeepSeek, GLM, Qwen, Kimi) by delegating tasks based on cost and capability, using a single CLAUDE.md routing table, and avoiding small talk to reduce token usage.
@DeRonin_: 800M free tokens a month, every major LLM, open source this guy literally made you to forget about any limits repo: htt…
FreeLLMAPI is an open-source tool that aggregates free tiers from 11 major LLM providers into a single OpenAI-compatible endpoint, routing requests and managing rate limits to deliver ~1B+ tokens per month. It simplifies access to multiple free models through one local server.
@DeRonin_: https://x.com/DeRonin_/status/2054235707791778034
A practical guide on reducing AI coding expenses by 80% through smarter token management, including multi-model routing, prompt caching, and context discipline, rather than simply switching to cheaper models.