How I easily cut my input token burn ~90% on long agent runs

Reddit r/AI_Agents News

Summary

The author shares a practical tip to reduce input token costs by ~90% on long agent runs using prompt caching: placing unchanged text (system prompt, tool definitions, context) at the start of every prompt to leverage cached prefixes from LLM providers.

My open-source agent harness SmithersBot lets me run Claude Code and Codex unattended for hours at a time. I'm on the $100 Claude plan and the $20 OpenAI one, so I kept hitting my 5 hour usage caps and went looking for ways to cut my token burn. The easiest win was caching. All you have to do is put the text that never changes (system prompt, tool definitions, context) at the very start of every prompt, in the same order every time. Why it works is LLMs run on next token prediction and if the start of your prompt is identical to one the model already processed, it can use that state as a checkpoint and start predicting from there rather than needing to predict every token up to that point again. That's why all model providers price cached input at a 90% discount. If you resend a prompt you've used before and add the new information at the end, the repeated part costs 90% less than if you'd put that same new information at the front and broken the already seen ordering. For one chat this barely matters but for an agent like mine firing hundreds of calls against the same context, the usage really adds up. Order it once and every call after that rides the cached prefix. How are you handling this? Curious on what other ways people are reducing their token burn so they don't hit usage caps.
Original Article

Similar Articles

Prompt Caching In Agents

Lobsters Hottest

The article explains how prompt caching works in large language model agents, covering KV cache mechanics, prefill and decode phases, and the impact on latency, cost, and agent design.