@mattshumer_: Every time a VC sends me an auto-router company to diligence, I send something like this back. If caching wasn't a thin…
Summary
The post critiques auto-routing companies in AI, emphasizing that frequent model switching invalidates prompt caches, resulting in higher token costs and inefficiencies.
View Cached Full Text
Cached at: 08/26/26, 11:22 AM
Every time a VC sends me an auto-router company to diligence, I send something like this back.
If caching wasn’t a thing, auto-routing would make much more sense.
But… it is a thing.
Teknium 🪽 (@Teknium): Just FYI. If you are switching models in a session all the time - you are doing it wrong.
Every time you switch, your entire prompt cache is invalidated on the new model you switch to, and you have to repay the full input tokens price for all of it.
Stop doing this unless those
Similar Articles
@DeRonin_: https://x.com/DeRonin_/status/2054235707791778034
A practical guide on reducing AI coding expenses by 80% through smarter token management, including multi-model routing, prompt caching, and context discipline, rather than simply switching to cheaper models.
How Caching Saved Us Hundreds of Dollars in AI Costs Every Month
The article describes how building an intelligent caching gateway (Hawiyat Composer) saved significant AI API costs by eliminating repeated token waste through exact-match caching, semantic caching, model routing, and local routing.
@avyvar: Token-maxxing is getting out of hand. Most AI apps send every request to the biggest model, even when a smaller model w…
The tweet criticizes AI apps for overusing large models and introduces Dari Router, a tool designed to route requests to appropriate model sizes for efficiency.
LLM Routing is not the problem to solve; token efficiency is
The article argues that model routing isn't the real problem to solve—token efficiency is. It advocates for a holistic closed-loop approach combining cheaper defaults, preference-aware routing, and better caching (e.g., Coinbase's 5%→60% cache hit improvement) to maximize useful intelligence per dollar.
@mckaywrigley: bullish model routers. same perf at lower cost is obvious. but there are massive gains to be had by creating "smoother"…
A tweet highlights the promise of model routers and 'model melding', referencing the launch of Not Diamond Code, an intelligent model router for coding agents that cuts costs by 20-65%.