@rohanpaul_ai: New Microsoft paper on Automated harness optimization for agents. Most harness auto-tuners focus on how to patch prompt…

X AI KOLs Timeline Papers
ai-agents harness-optimization reinforcement-learning active-learning benchmark microsoft-research

Summary

Microsoft researchers propose ActiveSaddler, a method that selects training tasks for agent harness auto-tuners based on unresolved failure patterns, boosting pass rates by 4.4 points on GAIA2 and 7.5 on Terminal-Bench 2.0 while cutting tuning costs sharply.

New Microsoft paper on Automated harness optimization for agents. Most harness auto-tuners focus on how to patch prompts and tools, but which tasks produce the feedback also changes how good the final harness gets. But you will get stronger AI agents when you pick training tasks based on which failures are still unfixed, so stop feeding them a fixed task list. ActiveSaddler tracks failure patterns and works on the one most worth fixing, or tries unseen tasks to find new ones. On the same optimizer, ActiveSaddler raised test pass rates by 4.4 points on GAIA2 and 7.5 points on Terminal-Bench 2.0. Reaching 58.5% GAIA2 dev accuracy cost $298, versus $1,360 with a fixed order. If you auto-tune an agent, aim your run budget at the failures that are still open.
Original Article
View Cached Full Text

Cached at: 10/06/26, 01:40 AM

New Microsoft paper on Automated harness optimization for agents.

Most harness auto-tuners focus on how to patch prompts and tools, but which tasks produce the feedback also changes how good the final harness gets.

But you will get stronger AI agents when you pick training tasks based on which failures are still unfixed, so stop feeding them a fixed task list.

ActiveSaddler tracks failure patterns and works on the one most worth fixing, or tries unseen tasks to find new ones.

On the same optimizer, ActiveSaddler raised test pass rates by 4.4 points on GAIA2 and 7.5 points on Terminal-Bench 2.0. Reaching 58.5% GAIA2 dev accuracy cost $298, versus $1,360 with a fixed order.

If you auto-tune an agent, aim your run budget at the failures that are still open.

Similar Articles

Automatic Harness Optimization (GitHub Repo)

TLDR AI

AutoSaddler is a Microsoft tool that automatically optimizes LLM-agent harnesses by diagnosing execution traces and applying structured updates to prompts, tools, and middleware, demonstrating significant benchmark improvements.

Claude Code improved my agent harness by 40% overnight

Reddit r/AI_Agents

The author introduces 'Autoharness', a tool that uses Claude Code to autonomously optimize agent harnesses by iterating on prompts and hyperparameters. This resulted in a 40% performance increase on the tau2-airline benchmark.

@dair_ai: Great paper from Meta on agent harness optimization. Meta-Harness-style search uses one development set and one proposa…

X AI KOLs Timeline

Meta (with Duke and UC Davis) proposes a Mixture of Self-Improving Branches framework for agent harness optimization, splitting the single-trajectory Meta-Harness search into adaptive branches with evolving development subsets and proposal policies, plus a router that selects the best branch per input, achieving up to +34.8% relative gains on Olympiad-level math, +11.6% on Terminal-Bench 2.0, and +3.8% on SWE-bench Lite.