@AstroHanRay: We ran an actual A/B benchmark test for active tool pruning, comparing 121 Terminal Bench tasks: - Performance: no regression (even slight improvement +2.48pp) - Token consumption: reduced by 41.7…
Summary
A/B testing for agent active tool pruning shows: across 121 Terminal Bench tasks, performance slightly improves (+2.48 percentage points), token consumption reduces by 41.7%, and cost decreases by 31.6%.
View Cached Full Text
Cached at: 06/28/26, 10:07 PM
We conducted an A/B benchmark test on active tool pruning, comparing 121 Terminal Bench tasks:
- Performance: No regression (slight improvement of +2.48pp)
- Token usage: Reduced by 41.7%
- Cost: Reduced by 31.6%
kabikabi (@jakevin7): There’s an unspoken default assumption in building agents: the tool result is important, and the model must read the full text before continuing inference.
Recently, I discovered this assumption might be wrong.
https://t.co/3wF7yrD3ES — star welcome
In maka, we aggressively prune tool results
Similar Articles
@yibie: Recommends this hardcore real-world test. An engineer tracked his coding agent session for a week and found that only 0.67% of tokens were spent on actual tasks—the remaining 99% all went to moving tool directories, skill descriptions, and system prompts. Work-to-overhead ratio 1:1…
An engineer tracked his coding agent's token usage over a week, finding that only 0.67% of tokens were spent on actual tasks, with 99% consumed by tool directories, skill descriptions, and system prompts. He provides optimization strategies, including shell output filtering which saved 46.9% of tokens.
TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents
TUA-Bench is a comprehensive benchmark for evaluating general-purpose terminal-use agents across diverse digital activities and specialized workflows, revealing significant performance gaps among current frontier agents.
Cut my agent’s tokens by 72% (11.9k ➝ 3.3k per task). Here’s exactly what I changed, with numbers
A developer shares a detailed case study on reducing an AI agent's token consumption by 72% through system prompt reduction, tighter retrieval, tool output pruning, and other techniques, with minimal impact on success rate.
I benchmarked my browser agent against Browser Use on a live site (150 verified runs, same model). Sending page diffs instead of full re-renders cut token growth by 37%.
A developer benchmarks Rote, a memory manager for browser agents that sends page diffs instead of full re-renders, showing a 37% reduction in token growth compared to Browser Use, but with trade-offs on short tasks.
GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows
GTA-2 introduces a hierarchical benchmark for evaluating general tool agents across atomic tool-use and open-ended workflows, revealing a significant capability cliff where frontier models achieve only 14.39% success on complex tasks despite reasonable atomic performance.