Testmu eval cost jumped 3x after we added 4 tools to our agent. Anyone optimize this?
Summary
A user reports that the evaluation cost for their AI agent tripled after adding four tools, seeking optimization advice.
Similar Articles
When I finally instrumented my agents' tool calls, the cost breakdown surprised me. A few lessons.
The author shares lessons from instrumenting AI agent tool calls, revealing that tools like web_search can account for ~50% of spend, and highlighting the importance of tracking p95 latency and attributing costs per workflow or customer to avoid surprises.
do agents get worse to maintain as you add more tools?
Explores the potential challenges of maintaining AI agents as the number of integrated tools increases, questioning scalability and complexity.
Gave an agent 30 tools. It got worse at using the 3 that mattered.
The author observed that adding more tools to an AI agent decreased its accuracy in selecting the correct tool due to increased classification complexity, and found that using multiple smaller agents with narrower toolsets improved reliability.
@svpino: Our token costs tripled over a few weeks. We only had a few agents running, and nothing was broken, but costs went 3x w…
The author explains why AI agent token costs tripled due to enhanced agent activities, highlights the importance of observability for managing autonomous systems, and promotes a free observability engineering masterclass by Honeycomb and Liz Fong-Jones.
How do you Mapout AI workflows when one suddenly costs 2× more than usual?
The article discusses common causes of cost spikes in AI workflows, such as retries, repeated tool calls, long-running workflows, and growing context, and asks how teams investigate such issues.