Token prices dropped significantly, yet many enterprises exceed AI budgets. A unit of inference cost $60 per million tokens in 2020, now just pennies. The discrepancy suggests inefficiencies despite lower token costs.

Why your AI bill went up even though token prices are falling

Token prices have been falling roughly 10x a year. A unit of inference that cost $60 per million tokens in 2020 goes for pennies today. And yet almost every enterprise I talk to is over budget on AI. I saw a report the other day that said 73% of enterprises blew past their AI cost projections this year.

Ramp’s data explains this pretty well: business token consumption grew about 1,000% from January 2025 to April 2026, while spend grew about 500%. Prices are collapsing, and invoices are exploding at the same time, because volume is growing twice as fast as prices are falling.

We’ve seen this happen organically at Decagon, both in our own internal AI usage and in the agents we deploy. We partner with a lot of large regulated institutions (banks, telecom, healthcare payors, etc) – here is a look at the (unscaled) token usage in one of those agents.

Token usage has gone up consistently for a single conversation. I mention the regulated clients because the accuracy of our agents is critical when they have conversations with consumers. For example, there are some rules (e.g. ‘don’t give financial advice’) that can’t be broken.

The reason this chart is going up is that you can maximize accuracy by running more model calls. You can have a supervisor model checking the answers of the main model. You can have models running in parallel to check if there’s a bad actor. As you optimize for quality (rather than cost), token usage will naturally increase.

If your AI bill went up this year, here’s what actually happened.

The first thing is that the unit of consumption changed underneath you. In 2024, “using AI” mostly meant a chatbot: one prompt, one response, done. In 2026, it means an agent. An agent doesn’t answer a request, it works a request. It retrieves context, reasons about it, calls a tool, checks the result, retries if it failed, and loops until the task is done. A single user task can trigger dozens of model calls. Your budget assumed one call per interaction. The architecture you actually deployed makes twenty.

The second thing is that each of those calls got fatter. RAG pipelines stuff retrieved documents into every prompt. Conversation histories get replayed on every turn. Reasoning models think out loud in tokens you pay for. Context per call has inflated several-fold, so even where call counts held steady, tokens per call didn’t. Because context windows have increased so much, it’s rarely worth the effort to over-optimize what goes in there. It’s easier to just have the model see more.

The third thing is a quiet one: background inference. Monitoring agents, document watchers, compliance checks that run against every event whether or not a human asked for anything. It’s easy for your team to set up always-on jobs. One of our more popular products is Watchtower, an AI that constantly monitors your live conversations to watch for certain things (e.g. mentions of a bug you’re trying to solve, measure sentiment, etc). This was simply not possible with earlier models.

Applications (like Decagon) will usually not pass on the cost of the extra tokens, but most of your token spend will come from the frontier labs themselves.

Notice that none of these are waste, exactly. It’s basically Jevons paradox: when inference was expensive, you wrote one careful prompt and took the answer. When it got cheap, it became rational to build systems that burn tokens extravagantly, because the extravagant version actually works better. Cheap tokens didn’t lower your bill. They made a more expensive way of computing viable, and everyone adopted it in the same year.

And there’s another factor that’s just getting started: the prices you’ve been paying weren’t real. While token prices fell ~80% over the past eighteen months, the compute underneath them got more expensive. H100 rental rates bottomed in October 2025 and then reversed — one-year contract rates jumped 40% by March, on-demand capacity sold out across the market, and essentially every GPU coming online through September is already booked. Nvidia announced rental price increases.

So what do you actually do about it? One answer is to match the model to the maturity of the use case, which I wrote about recently.

Another option is to be simply more careful about what your team is using tokens for. You can set limits and unlock them if the use cases are truly valuable. Tesla, for example, reportedly caps each employee’s spending on third-party AI tools at $200 per week; exceeding that requires manager approval.

Anyways, the core takeaway is that cost per token is a vanity metric now. It’s the one number that’s reliably improving, but it tells you nothing about your bill. The number that matters is cost per completed task: per ticket resolved, per document processed, per workflow finished. Denominate your AI spend in outcomes and the paradox dissolves. Tokens are cheap and getting cheaper. What you’re actually buying is finished work, and the companies that price, measure, and architect around that are the ones whose bills make sense.