@AstroHanRay: We ran an actual A/B benchmark test for active tool pruning, comparing 121 Terminal Bench tasks: - Performance: no regression (even slight improvement +2.48pp) - Token consumption: reduced by 41.7…

X AI KOLs Following Tools
tool-pruning agent benchmark token-reduction cost-optimization

Summary

A/B testing for agent active tool pruning shows: across 121 Terminal Bench tasks, performance slightly improves (+2.48 percentage points), token consumption reduces by 41.7%, and cost decreases by 31.6%.

We ran an actual A/B benchmark test for active tool pruning, comparing 121 Terminal Bench tasks: - Performance: no regression (even slight improvement +2.48pp) - Token consumption: reduced by 41.7% - Cost: reduced by 31.6%
Original Article
View Cached Full Text

Cached at: 06/28/26, 10:07 PM

We conducted an A/B benchmark test on active tool pruning, comparing 121 Terminal Bench tasks:

  • Performance: No regression (slight improvement of +2.48pp)
  • Token usage: Reduced by 41.7%
  • Cost: Reduced by 31.6%

kabikabi (@jakevin7): There’s an unspoken default assumption in building agents: the tool result is important, and the model must read the full text before continuing inference.

Recently, I discovered this assumption might be wrong.

https://t.co/3wF7yrD3ES — star welcome

In maka, we aggressively prune tool results

Similar Articles

@yibie: Recommends this hardcore real-world test. An engineer tracked his coding agent session for a week and found that only 0.67% of tokens were spent on actual tasks—the remaining 99% all went to moving tool directories, skill descriptions, and system prompts. Work-to-overhead ratio 1:1…

X AI KOLs Timeline

An engineer tracked his coding agent's token usage over a week, finding that only 0.67% of tokens were spent on actual tasks, with 99% consumed by tool directories, skill descriptions, and system prompts. He provides optimization strategies, including shell output filtering which saved 46.9% of tokens.

Tested 15 local models for agent/tool use.. Bonsai 27B was last!

Reddit r/LocalLLaMA

评测了15个本地运行的模型在 Toolery agent/工具使用基准上的表现，Qwen3.8-27B 以71.8%排名第一，而1-bit量化压缩到约4GB的 Bonsai 27B 垫底（50.5%），主要失败原因是工具调用次数超限（budget_violated）而非选错工具。

@AndyL5cc: Tencent Hunyuan is going all out this time! The latest open-source paper, Terminal-RSI, boosts a 27B model's terminal task success rate from 51.7% to 86.5%! The core idea is simple: AI summarizes its own successful experiences → retrains → keeps optimizing, in two loops! No need to just scale parameters—just...

X AI KOLs Timeline

Tencent Hunyuan's open-source paper Terminal-RSI proposes a recursive self-improvement framework, alternating between recursive self-rewriting and policy refinement training, raising a 27B terminal agent's task success rate on Terminal-Bench 2.1 from 51.7% to 86.52%.