Harness-G: A Graph-Structured Harness for Search Agents

Hugging Face Daily Papers Papers
reinforcement-learning search-agents retrieval graph-structured credit-assignment

Summary

This paper introduces Harness-G, a graph-structured retrieval framework that reformulates free-form query generation as finite action selection to reduce retrieval aliasing in RL-powered search agents. Across six QA benchmarks, Harness-G outperforms the strongest baseline Graph-R1 by 10.74 points at 1.5B and 3.98 points at 3B scale.

Reinforcement learning (RL) search agents commonly model retrieval as free-form natural-language query generation and optimize multi-turn interactions using final-answer rewards. Current studies mainly improve training with denser or more structured credit signals, but rarely examine whether retrieval is properly formulated at the policy-environment interface. We observe pronounced retrieval aliasing during Search-R1 training: rollouts for the same question continue to generate distinct query strings, yet their accumulated evidence sets increasingly overlap. We call this phenomenon retrieval-equivalence collapse; in this regime, trajectories approach utility equivalence with respect to retrieval decisions, leaving within-group returns with little effective retrieval contrast. To address this problem, we propose Harness-G, a graph-structured retrieval framework that redesigns this interface. It reformulates free-form query generation as finite action selection: the policy selects an evidence sentence or entity, or chooses to answer, while the environment constructs the menu, tracks retrieval state, and validates and executes each choice. This interface reduces linguistic aliasing and makes same-state alternatives directly comparable. Building on this interface, we introduce Structured Non-myopic Credit (SNC), which uses a frozen answer scorer to compare the selected action with its alternatives and assigns downstream gains to the earlier actions that enabled them. Across six QA benchmarks, Harness-G achieves the highest average F1 at both evaluated model scales, outperforming the strongest baseline, Graph-R1, by 10.74 points at 1.5B and 3.98 points at 3B.
Original Article
View Cached Full Text

Cached at: 07/31/26, 05:53 AM

Paper page - Harness-G: A Graph-Structured Harness for Search Agents

Source: https://huggingface.co/papers/2607.27652

Abstract

Reinforcementlearning(RL)searchagentscommonlymodelretrievalasfree-formnatural-languagequerygenerationandoptimizemulti-turninteractionsusingfinal-answerrewards.Currentstudiesmainlyimprovetrainingwithdenserormorestructuredcreditsignals,butrarelyexaminewhetherretrievalisproperlyformulatedatthepolicy-environmentinterface.WeobservepronouncedretrievalaliasingduringSearch-R1training:rolloutsforthesamequestioncontinuetogeneratedistinctquerystrings,yettheiraccumulatedevidencesetsincreasinglyoverlap.Wecallthisphenomenonretrieval-equivalencecollapse;inthisregime,trajectoriesapproachutilityequivalencewithrespecttoretrievaldecisions,leavingwithin-groupreturnswithlittleeffectiveretrievalcontrast.Toaddressthisproblem,weproposeHarness-G,agraph-structuredretrievalframeworkthatredesignsthisinterface.Itreformulatesfree-formquerygenerationasfiniteactionselection:thepolicyselectsanevidencesentenceorentity,orchoosestoanswer,whiletheenvironmentconstructsthemenu,tracksretrievalstate,andvalidatesandexecuteseachchoice.Thisinterfacereduceslinguisticaliasingandmakessame-statealternativesdirectlycomparable.Buildingonthisinterface,weintroduceStructuredNon-myopicCredit(SNC),whichusesafrozenanswerscorertocomparetheselectedactionwithitsalternativesandassignsdownstreamgainstotheearlieractionsthatenabledthem.AcrosssixQAbenchmarks,Harness-GachievesthehighestaverageF1atbothevaluatedmodelscales,outperformingthestrongestbaseline,Graph-R1,by10.74pointsat1.5Band3.98pointsat3B.

View arXiv pageView PDFGitHub2Add to collection

Get this paper in your agent:

hf papers read 2607\.27652

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.27652 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.27652 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.27652 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable

arXiv cs.AI

The Harness Handbook is a behavior-centric representation synthesized from agent harness codebases using static program analysis and LLM assistance, helping developers and coding agents locate code implementing specific behaviors. It introduces Behavior-Guided Progressive Disclosure (BGPD) to guide agents from high-level descriptions to relevant implementation details, improving localization accuracy and edit-plan quality.