@rohanpaul_ai: New John Hopkins and Carnegie Mellon University Paper Shows that a small model can learn to improve an agent's harness …
Summary
A Johns Hopkins and Carnegie Mellon paper shows a 4B editor model can learn to rewrite an agent's harness code from failure reports, with edits transferring to unseen tasks and outperforming its 35B teacher on average edit scores.
View Cached Full Text
Cached at: 10/06/26, 09:41 AM
New John Hopkins and Carnegie Mellon University Paper Shows that a small model can learn to improve an agent’s harness code from run results, and the skill transfers to new tasks.
A small model trained to rewrite an agent’s harness code from failure reports can adapt agents to new tasks, so let it tune your harness instead of doing it by hand.
The harness is the code that decides what the model sees and which tools it calls. The editor reads the harness and what failed, then writes a code change, rewarded by how well the new harness scores.
On 21 unseen reasoning task types, a 4B editor’s average edit score rose from 0.32 to 0.62, above its 35B teacher. A separate editor trained on HotpotQA kept improving harnesses on 2 other QA benchmarks.
Run it for several rounds per task, since repeated edits beat single fixes.
Similar Articles
@omarsar0: Banger paper from CMU on harness learning. (bookmark it) Also, pay attention to this important new AI engineering skill…
CMU paper introduces "harness learning," where an RL-trained proposer model revises an agent's executable harness from execution feedback without changing solver weights, achieving generalizable test-time adaptation across unseen tasks with a 4B proposer outperforming its 35B teacher.
@omarsar0: New paper from Microsoft and colleagues on evolving agent harnesses. It's a really cool idea to evolve a harness from p…
Microsoft and UCSB researchers introduce ScholarEvolve, a framework that evolves agent harnesses by mining published AI research for improvement strategies, achieving substantial gains on AppWorld and Tau2-Bench with fixed models.
@rohanpaul_ai: New Microsoft paper on Automated harness optimization for agents. Most harness auto-tuners focus on how to patch prompt…
Microsoft researchers propose ActiveSaddler, a method that selects training tasks for agent harness auto-tuners based on unresolved failure patterns, boosting pass rates by 4.4 points on GAIA2 and 7.5 on Terminal-Bench 2.0 while cutting tuning costs sharply.
@rohanpaul_ai: Brilliant new paper from Meta, CMU and other labs. Shows that coding agents improve faster by manufacturing their own s…
A new paper from Meta, CMU, and other labs presents Self-play SWE-RL, a method where coding agents train themselves by manufacturing and fixing bugs in real codebases, achieving significant gains on SWE-bench benchmarks without relying on human-written tasks.
Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
The paper introduces an on-policy expert-correction pipeline to co-evolve harnesses and models, enabling weaker AI models to improve by correcting failing turns without full imitation, preserving compatibility and enhancing performance on enterprise agent tasks.