@rohanpaul_ai: This paper shows that agent performance depends less on prompts alone and more on the harness around them. “Agent intel…
Summary
This paper argues that AI agent performance depends more on the harness (control layer) than on prompts alone, proposing natural-language agent harnesses to make design choices inspectable and portable.
View Cached Full Text
Cached at: 05/23/26, 02:07 PM
This paper shows that agent performance depends less on prompts alone and more on the harness around them.
“Agent intelligence” is becoming partly a systems problem. The problem is that many AI agents look like 1 model, but their real behavior comes from surrounding code that controls planning, tools, memory, retries, checking, and stopping.
A model may reason well in one step, but long tasks fail in messier places: state disappears, verification drifts, tools return partial evidence, and the agent forgets which intermediate artifact actually matters.
Natural-Language Agent Harnesses try to make that control layer visible.
Instead of burying the logic in controller code, they express the stages, roles, contracts, state rules, failure modes, and stopping conditions in structured natural language that a shared runtime can execute.
The claim is not that natural language should replace code, but that the important design choices around an agent should become inspectable, portable, and testable instead of hiding inside one framework’s habits.
On SWE-bench, heavier harnessing changed behavior dramatically, with more calls, tools, delegation, and runtime, but it did not produce a simple win curve; sometimes added structure helped, and sometimes it pushed the agent away from the shortest benchmark-aligned repair.
A harness is not magic scaffolding around a model; it is a set of bets about where reliability comes from.
Paper Link – arxiv. org/abs/2603.25723
Paper Title: “Natural-Language Agent Harnesses”
Similar Articles
@rohanpaul_ai: This Meta + Stanford + Illinois survey paper argues that AI agents work better when code becomes their main working lay…
This survey paper from Meta, Stanford, and Illinois argues that AI agents perform better when code is used as their primary working layer, treating code as the environment for reasoning, action, and modeling. The authors introduce the concept of an 'agent harness' encompassing tools, memory, sandboxes, and feedback loops.
@rohanpaul_ai: New Microsoft paper on Automated harness optimization for agents. Most harness auto-tuners focus on how to patch prompt…
Microsoft researchers propose ActiveSaddler, a method that selects training tasks for agent harness auto-tuners based on unresolved failure patterns, boosting pass rates by 4.4 points on GAIA2 and 7.5 on Terminal-Bench 2.0 while cutting tuning costs sharply.
@mardehaym: To truly understand AI agents, you need to understand the harness. And it's not the model. I went deep on how a working…
The article emphasizes that in AI agents, the harness—comprising tools, context, controls, and workflows—is more critical than the model for achieving reliable, safe, and traceable outcomes.
The Evolution of the Agent Harness (10 minute read)
The article discusses how the simultaneous evolution of AI models and harnesses has led to significant improvements in agent capabilities, shifting the harness's role to focus on human attention interfaces.
@omarsar0: // Self-Harness: Harnesses That Improve Themselves // (bookmark this one) Most of the agent scaffolds we rely on today …
This paper introduces Self-Harness, a new paradigm where LLM-based agents iteratively improve their own operating harness—prompts, tools, and control flow—without human engineers or stronger external agents, achieving significant performance gains across multiple models.