@Xudong07452910: Agent Self-Modifying Harness: The hardest part isn't actually "how to modify it." It's about how to know whether this m…

Agent Self-Modifying Harness: The hardest part isn't actually "how to modify it." It's about how to know whether this modification has truly improved things when there's no standard answer. The Google team's Harness² paper specifically studies the self-evolution of Harness in open-world environments. It designs a two-layer recursion. The first layer targets the current task: First propose modifications to the Harness, then have the new and old versions execute under identical conditions, and judge which changes are worth keeping based on behavioral differences. In other words, it doesn't rely on a ground-truth verifier and can use real execution results to guide modifications. The second layer is even more interesting: The process of modifying the Harness each time is also summarized as experience, recording what problems were encountered, what modifications were used, why they were effective, and then using that to guide future new tasks. So what it accumulates isn't just a better Harness, but also the practice of "how to modify Harness" itself. On WorkBuddy-Bench, JobBench, and the Legal Agent Benchmark, a single round of improvement can steadily boost GLM-5 and Gemini 3.5 Flash, with maximum gains of 11.5, 4.5, and 11.8 percentage points respectively; continuing to increase the recursion budget can further improve performance. I'm more focused on the second layer of recursion. It's not hard to tweak an Agent better in a single instance; what truly approaches long-term self-evolution is when the system starts accumulating experience on "how it tuned itself well in the past." Harness²'s step forward is to make the improvement mechanism itself begin to have memory. 📎 arxiv: