Dynamic Harness Search: Building Multi-Agent Systems Per-Query via Prediction

Hugging Face Daily Papers Papers
multi-agent-systems llm-agents monte-carlo-tree-search prompt-optimization dynamic-workflow prediction-models

Summary

The paper introduces SHIFT, a framework that builds multi-agent harnesses per-query using a learned local LLM architect and value function combined with Monte Carlo tree search, achieving ~80% mean accuracy across 9,193 tasks and outperforming 17 baselines while reducing execution token costs by 32% in its cheaper mode.

Agent harnesses specify the roles, instructions, tools, and communication structure used to solve a task, and the right harness depends on the query. Because the value of each design choice is observable only through execution, tailoring a harness to each query has required either executing alternatives at inference time or costly manual design. We introduce SHIFT, which moves execution out of the per-query search loop. A local LLM architect learns a policy over harness-building actions from search, and a value function that predicts, from measured executions, a utility balancing accuracy against execution cost. For each query, Monte Carlo tree search uses these predictions to construct a harness. Across 9,193 tasks in six benchmarks, from math to document and general-assistant tasks, with a Gemini 3.5 Flash executor, SHIFT attains the highest mean accuracy, about 80%, outperforming 17 baselines that span prompting, prompt optimization, and workflow search, and exceeding the strongest baseline by 7.2 percentage points. A cheaper mode of SHIFT also attains a higher mean accuracy than every baseline while using 32% fewer execution tokens than the strongest baseline. We further show that choosing structure, instructions, and tools jointly beats choosing only instructions or only tools by up to 9.1 percentage points, and that learned value selection identifies more accurate harnesses with lower execution cost from candidate pools.
Original Article
View Cached Full Text

Cached at: 10/06/26, 04:52 AM

Paper page - Dynamic Harness Search: Building Multi-Agent Systems Per-Query via Prediction

Source: https://huggingface.co/papers/2610.04137

Abstract

Agentharnessesspecifytheroles,instructions,tools,andcommunicationstructureusedtosolveatask,andtherightharnessdependsonthequery.Becausethevalueofeachdesignchoiceisobservableonlythroughexecution,tailoringaharnesstoeachqueryhasrequiredeitherexecutingalternativesatinferencetimeorcostlymanualdesign.WeintroduceSHIFT,whichmovesexecutionoutoftheper-querysearchloop.AlocalLLMarchitectlearnsapolicyoverharness-buildingactionsfromsearch,andavaluefunctionthatpredicts,frommeasuredexecutions,autilitybalancingaccuracyagainstexecutioncost.Foreachquery,MonteCarlotreesearchusesthesepredictionstoconstructaharness.Across9,193tasksinsixbenchmarks,frommathtodocumentandgeneral-assistanttasks,withaGemini3.5Flashexecutor,SHIFTattainsthehighestmeanaccuracy,about80%,outperforming17baselinesthatspanprompting,promptoptimization,andworkflowsearch,andexceedingthestrongestbaselineby7.2percentagepoints.AcheapermodeofSHIFTalsoattainsahighermeanaccuracythaneverybaselinewhileusing32%fewerexecutiontokensthanthestrongestbaseline.Wefurthershowthatchoosingstructure,instructions,andtoolsjointlybeatschoosingonlyinstructionsoronlytoolsbyupto9.1percentagepoints,andthatlearnedvalueselectionidentifiesmoreaccurateharnesseswithlowerexecutioncostfromcandidatepools.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.04137

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.04137 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.04137 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.04137 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

@dair_ai: Great paper from Meta on agent harness optimization. Meta-Harness-style search uses one development set and one proposa…

X AI KOLs Timeline

Meta (with Duke and UC Davis) proposes a Mixture of Self-Improving Branches framework for agent harness optimization, splitting the single-trajectory Meta-Harness search into adaptive branches with evolving development subsets and proposal policies, plus a router that selects the best branch per input, achieving up to +34.8% relative gains on Olympiad-level math, +11.6% on Terminal-Bench 2.0, and +3.8% on SWE-bench Lite.

Harness-G: A Graph-Structured Harness for Search Agents

Hugging Face Daily Papers

This paper introduces Harness-G, a graph-structured retrieval framework that reformulates free-form query generation as finite action selection to reduce retrieval aliasing in RL-powered search agents. Across six QA benchmarks, Harness-G outperforms the strongest baseline Graph-R1 by 10.74 points at 1.5B and 3.98 points at 3B scale.

HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry

Hugging Face Daily Papers

HarnessX is a foundry for composable, adaptive, and evolvable AI agent harnesses that uses compositional primitives and trace-driven evolution to improve agent performance. Across five benchmarks, it achieves an average gain of +14.5% (up to +44.0%), demonstrating that runtime interface evolution is a complementary lever to model scaling.