An AI consultant reflects on how teams optimize LLM costs without questioning whether the task needs a model at all, and advocates measuring cost per successful outcome rather than per token.
I run a small AI consultancy and after 8 years of building software, a meeting from last month is still bugging me. A client walked me through a dashboard showing their monthly LLM spend down nearly 60% after an optimization sprint...Cheaper models here, caching there, prompts trimmed to the bone. It was good engineering and I said so. Then I asked what the system was now cheaper at doing and the room was quiet for a sec. No one knew because nobody was tracking that. No judgment tbh. I have done the exact same thing. The bill gets uncomfortable and you pull the levers you know how to pull and you feel like a responsible adult doing it. Which you are, sort of. It's real engineering. It just skips the question of whether the task should be going to a language model at all. So we audited their pipelines. The first workflow we opened was an LLM reading incoming support emails to extract the order number which was billed per thousand messages. The order number was sitting in the subject line, same format every time because their own system put it there.Tbh this could have been a one liner or a regex at most. The deeper we dug, the more we found the model covering for something broken upstream like a free text field that should have been a dropdown years ago. That kind of work is a tax on a process no one fixed and the model is just how the tax gets collected. Here's what I think is underneath it. Everyone measures cost per token because the dashboard hands you that no. for free but the cost per outcome you have to assemble yourself, out of support tickets and somebody's lost afternoon so it mostly never happens... Which is a shame, because the arithmetic is brutal once you write it down. A cheap model that's wrong 15 times in 100 isn't cheap if a human quietly fixes those 15. The correction time sits in a different budget line and never gets anywhere near the API bill. Meanwhile the pricier model that's wrong 3 times in 100 can end up the cheaper system overall and you would never learn it from the token dashboard. Before your next cost sprint try this instead. Take your 3 most expensive AI workflows and write down the full cost per successful outcome including human correction minutes. Then ask, for each one, whether a rule or a form change (or honestly a person) would get the same result for less. The biggest saving of the quarter for this client wasn't a model swap. It was: one dropdown + deleting the workflow that had existed to clean up after the old free text field. I'm not anti-LLM. They're how I make a living but I have started collecting stories where AI was the most expensive possible way to solve a problem so do tell me yours. Mine was the order number reader until a friend mentioned a company using GPT to check whether form fields were empty and I refuse to believe that's the ceiling.
This article argues that common LLM cost advice focusing on token reduction is too shallow, and that the more impactful strategy in production is to route different workflow steps to different models rather than using a single default model.
The author discusses strategies for AI cost optimization by emphasizing outcome-focused approaches, such as request tagging, budget reservation, and preventing waste from retries and loops.
The article discusses the challenges of accurately attributing costs to AI agents beyond LLM spend, including tools and models for measurement in multi-agent environments.
The article argues that the real challenge in AI isn't just building smarter models but making them cost-efficient at scale, highlighting the importance of reducing token usage, improving speed, and optimizing infrastructure.