GraphEcho: Structural Redundancy and Evidence Provenance in LLM Graph Agents

arXiv cs.AI Papers

Summary

GraphEcho is a benchmark that evaluates LLM graph agents on distinguishing structural redundancy from distinct evidence provenance, revealing gaps between efficient exploration and effective evidence use.

arXiv:2609.17695v1 Announce Type: new Abstract: A large language model (LLM) agent can follow more graph paths without acquiring more independent evidence. GraphEcho tests whether agents mistake these repeated encounters for additional corroboration. The benchmark varies path counts and evidential origins while holding evidence content fixed, and evaluates both judgments and active exploration. Controlled synthetic experiments reveal model-dependent judgment shifts, but redundant supporting paths increase the share of repeated walks across all evaluated frozen agents. Provenance-aware post-training (PAPT) reduces revisits and improves synthetic accuracy, yet covers fewer distinct sources. On scientific claims, it continues to reduce repetition while accuracy declines. These findings expose a gap between efficient exploration and effective evidence use: an agent can learn to stop repeating itself while overlooking information it needs. GraphEcho provides a controlled way to evaluate both what graph agents conclude and whether their exploration reaches distinct evidential sources.
Original Article
View Cached Full Text

Cached at: 09/17/26, 09:22 AM

# Structural Redundancy and Evidence Provenance in LLM Graph Agents
Source: [https://arxiv.org/html/2609.17695](https://arxiv.org/html/2609.17695)
Sikun Wang††thanks:Equal contributionYixi Zhou11footnotemark:1Affiliation:Hong Kong Baptist UniversityEmail:[jc19546883@gmail\.com](mailto:[email protected])Lei FanAffiliation:University of MichiganEmail:[leifanus@gmail\.com](mailto:[email protected])Fan Zhang††thanks:Corresponding authorAffiliation:The University of TokyoEmail:[zhang\-fan@g\.ecc\.u\-tokyo\.ac\.jp](mailto:[email protected])

###### Abstract

A large language model \(LLM\) agent can follow more graph paths without acquiring more independent evidence\. GraphEcho tests whether agents mistake these repeated encounters for additional corroboration\. The benchmark varies path counts and evidential origins while holding evidence content fixed, and evaluates both judgments and active exploration\. Controlled synthetic experiments reveal model\-dependent judgment shifts, but redundant supporting paths increase the share of repeated walks across all evaluated frozen agents\. Provenance\-aware post\-training \(PAPT\) reduces revisits and improves synthetic accuracy, yet covers fewer distinct sources\. On scientific claims, it continues to reduce repetition while accuracy declines\. These findings expose a gap between efficient exploration and effective evidence use: an agent can learn to stop repeating itself while overlooking information it needs\. GraphEcho provides a controlled way to evaluate both what graph agents conclude and whether their exploration reaches distinct evidential sources\.

## 1One observation can yield many paths

A graph agent can encounter one experimental observation through a result node, a conclusion node, and a table entry\. These paths share one evidential origin\. Counting them as independent corroboration would change the agent’s assessment without adding evidence\. Repeated retrieval can also consume its exploration budget\.

Retrieval\-augmented reasoning connects evidence selection with intermediate decisions\([Trivedi et al\., 2023](https://arxiv.org/html/2609.17695#bib.bib9);[Asai et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib13)\), while graph retrieval links information across passages\([Gutiérrez et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib15)\)\. Recent work examines repeated retrieval contexts\([Ross et al\., 2026](https://arxiv.org/html/2609.17695#bib.bib4)\), learned graph exploration\([Liu et al\., 2026](https://arxiv.org/html/2609.17695#bib.bib2)\), and provenance\-aware evidence selection\([Deng et al\., 2026](https://arxiv.org/html/2609.17695#bib.bib3)\)\. These directions motivate a controlled test of how structural representations affect both judgment and evidence acquisition\.

GraphEcho measures whether an agent distinguishes structural multiplicity from provenance multiplicity\. Our central comparison holds the claim and five path templates fixed while changing whether the paths originate from one study or five separate studies\. A second comparison expands one path into five paths from the same study\. Together, these interventions separate sensitivity to additional representations from sensitivity to additional evidential origins \(Figure[1](https://arxiv.org/html/2609.17695#S1.F1)\)\.

Figure 1:Path count and provenance count define separate interventions\. Each route contracts a three\-hop path to the same claim; the five\-route comparison preserves representation types and evidence wording\.This comparison requires control over evidence content, source strength, and graph presentation\. Different path lengths could change reasoning difficulty, while different evidence statements could add information\. We therefore match path lengths and evidence wording in the synthetic source\-count intervention\.

Our contributions establish three findings:

- •Controlled judgment\.GraphEcho\-Syn separates redundant paths from distinct origins across 800 fictional claims; static responses differ in direction across models\.
- •Active acquisition\.Redundant supporting paths raise echo walk rates in all four frozen agents, while accuracy changes in both directions\.
- •Training tradeoffs\.Provenance\-aware post\-training \(PAPT\) raises synthetic agent accuracy by 41\.5 percentage points over frozen Qwen3\-4B, but lowers source recall and SciFact accuracy\. Cheap controls and reward ablations distinguish reduced repetition from improved evidence use\.

## 2Related Work

#### Evidence conflict and redundancy\.

Conflicting passages can change answers without a corresponding reduction in model confidence\([Chen et al\., 2022](https://arxiv.org/html/2609.17695#bib.bib7)\)\.[Tan et al\. \(2024\)](https://arxiv.org/html/2609.17695#bib.bib10)find that models favor generated over retrieved contexts under knowledge conflict, while[Yoran et al\. \(2024\)](https://arxiv.org/html/2609.17695#bib.bib14)improve tolerance to irrelevant passages through training\. Prior work studies context position\([Liu et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib17)\)and retrieval redundancy\([Ross et al\., 2026](https://arxiv.org/html/2609.17695#bib.bib4)\)\. GraphEcho isolates a different source of distortion: multiple graph paths can repeat one evidential origin\. Matching evidence wording and path templates separates structural repetition from source count\.

#### Attribution and graph retrieval\.

ALCE evaluates answer correctness and citation quality separately\([Gao et al\., 2023](https://arxiv.org/html/2609.17695#bib.bib8)\)\. HippoRAG combines knowledge graphs with Personalized PageRank for multi\-hop retrieval\([Gutiérrez et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib15)\); HippoRAG 2 extends this approach with passage integration\([Gutiérrez et al\., 2025](https://arxiv.org/html/2609.17695#bib.bib16)\)\. PAGE\-RAG tracks provenance when selecting graph evidence\([Deng et al\., 2026](https://arxiv.org/html/2609.17695#bib.bib3)\)\. These methods motivate testing whether accessible source information suffices to prevent duplicate corroboration\. GraphEcho varies provenance assignments while holding evidence content fixed, and compares visible source identifiers with preprocessing that collapses repeated evidence\.

#### Adaptive evidence acquisition\.

ReAct interleaves reasoning with actions\([Yao et al\., 2023](https://arxiv.org/html/2609.17695#bib.bib12)\), and IRCoT alternates retrieval with intermediate reasoning steps\([Trivedi et al\., 2023](https://arxiv.org/html/2609.17695#bib.bib9)\)\. Adaptive\-RAG selects retrieval strategies according to question complexity\([Jeong et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib11)\), while Self\-RAG learns when to retrieve and how to assess retrieved passages and generated answers\([Asai et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib13)\)\. Call Neighbours Yourself \(CNY\) trains graph\-neighbor selection through destination\-conditioned on\-policy self\-distillation\([Liu et al\., 2026](https://arxiv.org/html/2609.17695#bib.bib2)\)\. GraphEcho examines whether acquisition reaches distinct origins within a fixed action budget\. PAPT uses group\-relative policy optimization\([Shao et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib5)\)with rewards for judgments, source counting, and distinct evidence acquisition\. Evaluating accuracy together with repeated walks and source recall distinguishes fewer revisits from broader evidence coverage\.

## 3Separate paths from evidential origins

### 3\.1Provenance defines the intervention

An evidence provenance atomeeidentifies an underlying evidential origin\. For a graph pathpp, the mappingπ⁡\(p\)=e\\pi\(p\)=erecords that origin\. Given a set of pathsPP, structural and provenance multiplicity are

Npath\\displaystyle N\_\{\\mathrm\{path\}\}=\|P\|,\\displaystyle=\|P\|,\(1\)Nprov\\displaystyle N\_\{\\mathrm\{prov\}\}=\|\{π⁡\(p\):p∈P\}\|\.\\displaystyle=\|\\\{\\pi\(p\):p\\in P\\\}\|\.\(2\)We writessSkkP for a condition withssdistinct origins andkkpaths\. Redundant\-path interventions changeNpathN\_\{\\mathrm\{path\}\}while preserving the evidence atoms and their content\. Source\-count interventions changeNprovN\_\{\\mathrm\{prov\}\}at a fixed path count\.

### 3\.2Synthetic graphs control provenance

GraphEcho\-Syn contains 800 claims across biomedicine, materials, agriculture, and economics\. Fictional entity names separate claim content from familiar factual associations\. Each claim has a candidate pool of five supporting atoms and five refuting atoms\. The generator assigns every atom a distinct source identifier and preserves both source and atom identifiers through graph construction\.

All atoms have the same relative reliability weight, 1\.0\. This weight expresses equal strength within the benchmark; it does not make an observation infallible\. Supporting atoms report a change in the claim’s direction, and refuting atoms report the opposite change\. The gold verdict follows the balance of distinct origins selected for the displayed condition: SUPPORT for a supporting majority, REFUTE for a refuting majority, and MIXED for a tie\. This label describes the evidence balance under the benchmark convention\.

Each atom admits up to five three\-hop paths through different representation types\. The primary source\-count comparison uses identical evidence wording and relation templates for 1S5P and 5S5P\. Only the origin assignments change\. Across variants of the same claim, the generator also preserves claim wording\. Token counts remain an explicit diagnostic because tokenization can treat source identifiers differently\.

The split contains 500 training, 100 development, and 200 test claims\. Every variant of a base claim stays in that claim’s split\. Entity names also remain disjoint across splits\. The initial 100\-claim pilot uses development claims, preserving the test split for the subsequent evaluation\.

Table 1:Conflict interventions vary path counts and provenance counts separately\.
### 3\.3Conflict exposes false corroboration

Table[1](https://arxiv.org/html/2609.17695#S3.T1)specifies five conflict conditions\. C1 and C2 contain the same two opposing atoms\. C2 adds four representations of the supporting atom, so an increased SUPPORT rate measures a decision shift under redundant evidence\. C3 supplies five distinct supporting origins\. C4 reverses the provenance majority, and C2R checks whether redundant refuting paths produce a corresponding shift\.

### 3\.4SciFact tests document provenance

SciFact pairs scientific claims with annotated evidence abstracts and rationale sentences\([Wadden et al\., 2020](https://arxiv.org/html/2609.17695#bib.bib1)\)\. We use its labeled development split for evaluation because the official test labels are unavailable\. The conversion retains 188 claims with gold evidence and produces 584 graph instances\. Ten eligible claims have multiple evidence documents\. Claims without gold evidence do not enter this gold\-rationale experiment\.

For each document, the converter preserves the annotated rationale sets and collects their sentences into one evidence record\. Redundant routes repeat that record through different graph representations\. Document identifiers provide observable source provenance in this experiment\. They do not establish statistical independence between scientific studies\. PAPT uses synthetic supervision only, leaving SciFact as a transfer evaluation\.

## 4Acquire distinct evidence within a budget

### 4\.1Observe graph actions

The agent environment exposes a current node, visited nodes, available neighbors, short previews, and the remaining walk budget\. A walk action reveals the selected neighbor’s full content\. A stop action returns the final judgment\. The primary budget permits six walk actions; additional runs use budgets of three, five, and eight\. Each three\-hop provenance path connects a source, a representation, an observation, and the claim\. Explicit shared\-claim edges connect observation nodes, permitting direct movement to another candidate observation\. Every action follows an edge; the trace records connector movements separately from repeated evidence acquisition\.

The agentic comparison uses balanced provenance, redundant supporting routes, and distinct supporting origins\. Neighbor previews withhold unseen evidence content and its gold stance\. Provenance metadata may remain visible under the corresponding mitigation condition\. An atom’s first evidence\-bearing visit supplies its complete synthetic observation, making later visits to equivalent representations redundant under this controlled setup\.

### 4\.2Compare cheap mitigations

Four inference controls test whether training adds value beyond metadata, instructions, and preprocessing\. M0 displays source nodes without atom identifiers\. M1 adds provenance identifiers; M2 also instructs the model to count distinct origins when assessing corroboration\. M3 groups equivalent paths while preserving their multiplicity\.

### 4\.3Train a provenance\-aware policy

PAPT combines a verdict reward with source counting and evidence acquisition:

R=\\displaystyle R=\{\}Rverdict\+0\.4​Rcount\\displaystyle R\_\{\\mathrm\{verdict\}\}\+0\.4R\_\{\\mathrm\{count\}\}\+0\.4​Rcoverage−0\.4​Recho−0\.05​Rcost\.\\displaystyle\+0\.4R\_\{\\mathrm\{coverage\}\}\-0\.4R\_\{\\mathrm\{echo\}\}\-0\.05R\_\{\\mathrm\{cost\}\}\.\(3\)The verdict reward equals\+1\+1for a correct final judgment and−1\-1otherwise\. The count reward measures agreement with gold supporting and refuting provenance counts\. Coverage rewards acquisition of distinct useful origins; the echo penalty measures repeated provenance acquisition\. The cost penalty divides the walk count by the budget\. With the counts defined in Section[5](https://arxiv.org/html/2609.17695#S5), coverage equalsU/max⁡\(1,min⁡\(B,D\)\)U/\\max\(1,\\min\(B,D\)\), echo equalsE/max⁡\(1,W\)E/\\max\(1,W\), and cost equalsW/BW/B\. LetΔ\\Deltasum the absolute support and refute count errors over acquired origins\. The count reward equalsmax⁡\(0,1−Δ/max⁡\(1,U\)\)\\max\(0,1\-\\Delta/\\max\(1,U\)\)for a valid final output and zero for an invalid output\.

The training protocol applies group\-relative policy optimization \(GRPO\)\([Shao et al\., 2024](https://arxiv.org/html/2609.17695#bib.bib5)\)to Qwen3\-4B with groups of four trajectories\. We apply low\-rank adaptation \(LoRA\)\([Hu et al\., 2022](https://arxiv.org/html/2609.17695#bib.bib6)\)with rank 16 and scaling parameterα=32\\alpha=32\. Each training claim contributes one conflict instance, giving 100 examples per condition across C1, C2, C3, C4, and C2R\. Training uses these 500 examples for one epoch with random seed 42\.

All reward variants share the selected training instances and optimization settings\. Ablations remove the echo reward, remove the coverage reward, or retain only the verdict reward\. Checkpoint selection maximizes mean full\-objective reward on development claims among the final three retained checkpoints\. Using the same selection objective across variants makes their development scores comparable\. Test labels and SciFact supervision never enter checkpoint selection\.

## 5Measure judgment and acquisition separately

### 5\.1Models and outputs

The frozen evaluation covers Qwen3\-1\.7B, Qwen3\-4B, and Qwen3\-8B\.111Official model cards:[Qwen3\-1\.7B](https://huggingface.co/Qwen/Qwen3-1.7B),[Qwen3\-4B](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507), and[Qwen3\-8B](https://huggingface.co/Qwen/Qwen3-8B)\.The 1\.7B and 8B models use non\-thinking generation\. The primary open\-model setting uses greedy decoding\. GPT\-5\-mini222OpenAI,[Introducing GPT\-5](https://openai.com/index/introducing-gpt-5/)\.provides a supplementary API baseline with its native sampling settings, which differ from temperature\-zero decoding\.

Outputs contain a verdict, confidence, supporting and refuting provenance counts, cited source identifiers, and a short rationale\. Confidence means subjective claim probability, not confidence in the selected verdict\. Parsing failures count as incorrect verdicts; confidence estimates use valid pairs with reported sample counts\.

### 5\.2Static effects use paired claims

LetC⁡\(s​S​k​P\)C\(s\\mathrm\{S\}k\\mathrm\{P\}\)denote confidence for the corresponding graph condition\. We measure path multiplicity and independent\-source effects as

PMEk\\displaystyle\\mathrm\{PME\}\_\{k\}=C⁡\(1​S​k​P\)−C⁡\(1​S​1​P\),\\displaystyle=C\(1\\mathrm\{S\}k\\mathrm\{P\}\)\-C\(1\\mathrm\{S\}1\\mathrm\{P\}\),\(4\)ISEk\\displaystyle\\mathrm\{ISE\}\_\{k\}=C⁡\(k​S​k​P\)−C⁡\(1​S​k​P\)\.\\displaystyle=C\(k\\mathrm\{S\}k\\mathrm\{P\}\)\-C\(1\\mathrm\{S\}k\\mathrm\{P\}\)\.\(5\)The primary comparison usesk=5k=5, withk=3k=3as an additional condition\. For one\-sided supporting evidence, positive PME indicates increased confidence from redundant paths\. ISE measures the response to distinct sources at a fixed path count\. We report both effects separately; we report their ratio only when ISE exceeds 0\.5 percentage points to avoid an unstable denominator\.

Evidence count inflation subtracts the true provenance count from the model’s predicted count\. Topology flip rate measures verdict changes between 1S1P and 1S5P\. Conflict bias compares the SUPPORT rate and claim confidence in C2 against C1\. The mirrored C2R condition checks the direction of the corresponding refuting\-path effect\.

### 5\.3Acquisition measures expose repeated visits

LetWWcount all walk attempts,VVcount valid visits to evidence\-bearing nodes, andUUcount distinct acquired origins\. Every selected atom bears on the claim and counts as useful, including refuting atoms\. LetDDcount graph\-reachable origins, and letEEcount valid visits whose origin appeared in an earlier visit\. We compute

UPC\\displaystyle\\mathrm\{UPC\}=U/V,\\displaystyle=U/V,EWR\\displaystyle\\mathrm\{EWR\}=E/W,\\displaystyle=E/W,\(6\)USR\\displaystyle\\mathrm\{USR\}=U/D,\\displaystyle=U/D,WE\\displaystyle\\mathrm\{WE\}=U/W\.\\displaystyle=U/W\.\(7\)These ratios measure unique provenance coverage \(UPC\), echo walk rate \(EWR\), unique source recall \(USR\), and walk efficiency \(WE\)\. They equal zero for zero denominators\. Connector and invalid actions enterWW;VVcounts only evidence\-bearing visits\. Appendix[D](https://arxiv.org/html/2609.17695#A4)separates connector and invalid\-action counts\. Shared\-claim edges in the primary graph make each origin accessible in one acquisition action, allowingmin⁡\(B,D\)\\min\(B,D\)distinct origins within budgetBB\.

Agent source\-count error compares predicted counts with the distinct origins acquired\. Final verdict accuracy uses the full graph’s evidence\-balance label\. This distinction separates errors about observed provenance from incomplete evidence acquisition\.

### 5\.4Mitigation must preserve source sensitivity

A reduction in redundant\-path sensitivity is insufficient if a model also stops responding to distinct sources\. The mitigation comparison therefore reports PME and ISE together with verdict accuracy and source\-count error\. Agentic evaluation adds coverage, repeated walks, and cost\. SciFact reports SUPPORT and REFUTE strata separately\. Its stance\-aligned PME multiplies the raw probability change by\+1\+1for supporting evidence and−1\-1for refuting evidence\. Document\-count effects use only eligible multi\-document claims and compare matched path counts; we call these document\-provenance effects \(DPE\)\. These comparisons test transfer without scientific\-data supervision\.

### 5\.5Uncertainty respects repeated measurements

The main evaluation randomizes path order across five seeds\. We average repeated measurements within each base claim before resampling claims\. Tables show point estimates with bracketed lower and upper bounds of 95% confidence intervals, using 10,000 claim\-bootstrap resamples\. For binary verdict comparisons, the analysis also computes exact McNemar tests using one pair per claim at the lowest presentation seed and sample\. The static sampling sensitivity analysis uses temperature 0\.7 and five outputs per instance at presentation seed zero\. Primary synthetic comparisons use five presentation seeds; comparisons with inference\-time controls use their common seed zero\. SciFact agent evaluation uses seed zero\. Tables state the applicable pairing scope\.

Appendix[D](https://arxiv.org/html/2609.17695#A4)reports valid\-final and zero\-walk counts alongside acquisition measures\. A post hoc accuracy diagnostic retains only matched condition, seed, and sample pairs with valid finals on both sides\. Selection changes which observations enter this comparison and cannot isolate a causal reasoning effect\. Primary accuracy includes all received outputs, counting malformed finals as incorrect\.

## 6Redundancy changes judgment and acquisition

### 6\.1Static effects depend on the model

Table 2:Static synthetic test evaluation\. Probabilities and flip rates use percent; PME and ISE use percentage points\. Brackets show 95% paired base\-claim bootstrap intervals\. Each model uses 200 test claims and five presentation seeds\. Count inflation refers to 1S5P\.Static responses differ across models \(Table[2](https://arxiv.org/html/2609.17695#S6.T2)\)\. Qwen3\-4B responds more to redundant paths than to additional origins, whereas Qwen3\-1\.7B barely changes either probability\. Qwen3\-8B and GPT\-5\-mini lower probability under redundant paths and raise it under distinct origins\. Source counting reveals another distinction: at 1S5P, Qwen3\-8B overcounts by four origins, while Qwen3\-4B undercounts by one\. The same intervention therefore produces different judgment and counting errors across models\.

Conflict conditions separate categorical and probability responses\. Qwen3\-1\.7B shifts strongly toward SUPPORT when redundant supporting paths enter C2\. Qwen3\-8B and GPT\-5\-mini keep their SUPPORT rates unchanged, although their claim probabilities rise\. Repetition can therefore shift expressed confidence while leaving the categorical decision unchanged\. Path sensitivity, source counting, and categorical judgment therefore capture different failures\. Appendix[B](https://arxiv.org/html/2609.17695#A2)reports the complete conflict contrasts and confidence curves\.

### 6\.2Redundant paths raise echo walk rates

Table 3:Redundant support raises echo walk rates in all four frozen agents\. C2\-minus\-C1 differences use percentage points, 200 claims and five presentation seeds; brackets show 95% paired claim\-bootstrap intervals\. Accuracy counts malformed received outputs as incorrect\.All four frozen agents increase their echo walk rate when C2 replaces C1 \(Table[3](https://arxiv.org/html/2609.17695#S6.T3)\)\. The paired increases range from 5\.80 to 38\.84 percentage points, and all four intervals exclude zero\. Accuracy changes in both directions: Qwen3\-4B and GPT\-5\-mini lose accuracy, while the other models gain it\. Echo walk rate therefore provides a more consistent response to redundant topology than accuracy degradation\.

Output validity complicates the accuracy comparison for Qwen3\-1\.7B\. Valid finals rise from 39\.1% in C1 to 90\.5% in C2\. Restricting the analysis to pairs with valid finals reverses the estimated accuracy change, but its interval spans zero\. This selected subset covers 168 of 200 claims \(Appendix[D](https://arxiv.org/html/2609.17695#A4)\)\. The primary accuracy gain thus depends on which outputs enter the analysis\.

### 6\.3PAPT learns shorter, less repetitive walks

Table 4:Qwen3\-4B agentic evaluation with budget six\. All methods use presentation seed zero\. We average C1, C2, C3, and C2R within each claim before computing estimates and bootstrap intervals\. Rates use percent; count error uses acquired evidence\. Brackets show 95% base\-claim bootstrap intervals\.Table 5:PAPT trades repeated acquisition against coverage and transfer accuracy\. Entries show paired differences in percentage points with 95% claim\-bootstrap intervals\. Synthetic comparisons use 200 claims and C1/C2/C3/C2R: five seeds for M1 and reward ablations, common seed zero for M2/M3\. SciFact uses 188 claims, three one\-document conditions and seed zero\.PAPT improves synthetic agent accuracy over frozen Qwen3\-4B by 41\.5 percentage points across five presentation seeds \(Table[5](https://arxiv.org/html/2609.17695#S6.T5)\)\. It also reduces repeated acquisition, cutting echo walk rate by 36\.9 points while using about three fewer walks\. Table[4](https://arxiv.org/html/2609.17695#S6.T4)reports the inference controls on their common seed zero\. In the balanced conflict in Figure[2](https://arxiv.org/html/2609.17695#S6.F2), PAPT visits each origin once and stops\.

Figure 2:PAPT stops after two distinct origins in this C2 example\. The matched seed\-zero pair lies nearest the median EWR change; both policies return the correct MIXED verdict\.The shorter trajectories also leave more evidence unseen\. C3 drives the recall loss: PAPT reaches about one\-third of the six available origins, whereas the frozen policy reaches more than four\-fifths\. PAPT averages roughly two walks, leaving most origins unvisited even though each origin is reachable in one action\. These results link reduced repetition to earlier stopping, which also leaves evidence unseen\.

Provenance collapse provides a strong cheaper control\. On common seed zero, PAPT exceeds M3 accuracy by about four percentage points, but revisits more evidence and reaches fewer distinct origins\. M3 changes the action space by grouping equivalent evidence, so this comparison assesses the complete preprocessing intervention\. The static results show a further distinction: collapse nearly eliminates redundant\-path sensitivity while retaining a response to distinct sources\. PAPT still responds more to redundant paths than to additional origins\. Appendix[C](https://arxiv.org/html/2609.17695#A3)reports the full static mitigation and quality estimates\.

### 6\.4Transfer reduces echoes but loses accuracy

SciFact exposes a quality cost that the synthetic aggregate conceals\. PAPT reduces echo walk rate by 36\.0 percentage points relative to frozen Qwen3\-4B, yet loses 3\.6 points in accuracy \(Table[5](https://arxiv.org/html/2609.17695#S6.T5)\)\. Against provenance collapse, PAPT attains higher accuracy but repeats more evidence\. Thus neither intervention dominates both outcomes, and fewer revisits alone provide an incomplete measure of transfer\.

All three common SciFact agent conditions contain one document origin\. Every compared Qwen3\-4B policy reaches that origin, placing source recall at its ceiling\. This test measures repeated acquisition and final judgment on scientific rationales; it provides no multi\-source acquisition test\. The static document\-count analysis uses ten eligible multi\-document claims and yields correspondingly wide intervals \(Appendix[C](https://arxiv.org/html/2609.17695#A3)\)\.

At this recall ceiling, fewer revisits change repeated exposure without expanding the acquired origins\. The accuracy loss therefore calls for evaluating how the policy uses scientific evidence\.

### 6\.5Reward ablations expose competing effects

The full reward improves synthetic accuracy over verdict\-only training by about 16 percentage points, while reducing both echo rate and source recall \(Table[5](https://arxiv.org/html/2609.17695#S6.T5)\)\. Removing the echo term also lowers accuracy relative to full PAPT\. Removing coverage yields higher recall than full PAPT, despite receiving no coverage reward\. These outcomes show why reward components require joint evaluation: changing one term alters the resulting acquisition policy across several measures\.

On SciFact, the full reward offers no clear accuracy advantage over verdict\-only training: the paired interval includes zero\. The evidence thus supports a synthetic accuracy benefit, while leaving that benefit uncertain in transfer\. All variants share a training seed and development selection objective; these comparisons characterize the resulting policies under that shared training seed\.

PAPT retains positive accuracy differences over the frozen agent under budgets 3, 5, and 8, ring topology, and variable path lengths \(Appendix[E](https://arxiv.org/html/2609.17695#A5)\)\. Echo rate falls in all five settings\. Source recall increases slightly at budget three and falls in the remaining settings\. The acquisition tradeoff therefore extends beyond the primary topology\.

### 6\.6Stopping depends on unobserved sources

The C2 and C3 constructions expose a stopping ambiguity\. After acquiring one supporting and one refuting origin, an agent has observed balanced evidence in either condition\. That pair completes C2’s two\-origin graph, whereas C3 still contains four unseen supporting origins\. The full\-graph verdict is therefore MIXED in C2 and SUPPORT in C3\. A stopping rule based only on the balance of acquired evidence would treat these states alike\. Source recall identifies the difference because its denominator includes the distinct origins that remain available\. This comparison explains why correct source counting over observed evidence can coexist with an incorrect final judgment\.

Provenance collapse and policy training intervene at different points in this decision\. Collapse removes equivalent choices before exploration, while PAPT selects among the original graph’s neighbors and can end its trajectory early\. Their accuracy differences can therefore reflect both evidence selection and stopping behavior\. A follow\-up comparison could give both policies the same set of distinct origins before eliciting their verdicts\. Matching the acquired evidence would separate differences in judgment from differences in evidence access\. Complementary tests could vary the stopping budget on a fixed graph to examine which origins each policy prioritizes\.

## 7Conclusion and Future Work

GraphEcho separated structural repetition from provenance through matched graph interventions\. Frozen agents showed heterogeneous judgment effects and consistently higher echo walk rates under redundant support\. PAPT improved synthetic accuracy and reduced repeated walks, but lowered source recall and SciFact accuracy\. Evaluation must measure both repeated and unobserved evidence\. Future work should test multi\-source acquisition on scientific graphs across training seeds\.

Future scientific benchmarks could track shared datasets and experiments across papers\. These annotations would help test whether agents recognize common empirical origins across documents\. They would also support comparisons between evidence grouping and learned exploration when document boundaries provide an incomplete account of source dependence\.

## Limitations of the Work

Synthetic source independence is a controlled benchmark assumption\. Equal\-weight evidence balance defines the synthetic verdict labels, while real scientific evidence varies in strength and dependence\. Repeating identical evidence text controls information content but covers a limited form of graph extraction\. SciFact’s document identifiers provide an observable provenance proxy, and its gold\-rationale setting omits retrieval errors\. Self\-reported probabilities measure model responses without establishing probabilistic calibration\. The agentic results concern the implemented graph topologies, previews, and action costs\. Shorter trajectories can improve echo rate while leaving distinct evidence unobserved\.

Training uses one random seed\. Claim\-bootstrap intervals quantify evaluation variation conditional on the fitted policy and omit variation across training runs\. The reward ablations therefore compare the resulting policies under a shared training seed\.

## Declaration on Generative AI

The authors used generative AI to polish the language of the manuscript\. The authors retain full responsibility for its content\.

## References

- Asaiet al\.\(2024\)A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. HajishirziSelf\-RAG: learning to retrieve, generate, and critique through self\-reflection\.InProceedings of the Twelfth International Conference on Learning Representations,ICLR ’24,Vienna, Austria\.External Links:[Link](https://openreview.net/forum?id=hSyW5go0v8)Cited by:[§1](https://arxiv.org/html/2609.17695#S1.p2.1),[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px3.p1.1)\.
- Chenet al\.\(2022\)H\. Chen, M\. Zhang, and E\. ChoiRich knowledge sources bring complex knowledge conflicts: recalibrating models to reflect conflicting evidence\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,EMNLP ’22,Abu Dhabi, United Arab Emirates,pp\. 2292–2307\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.146),[Link](https://aclanthology.org/2022.emnlp-main.146/)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px1.p1.1)\.
- Denget al\.\(2026\)H\. Deng, X\. Li, H\. Qin, and R\. LiPAGE\-RAG: provenance\-aware graph evidence promotion for fixed\-budget multi\-hop retrieval\-augmented generation\.ArXiv preprintarXiv:2608\.29753\.External Links:[Link](https://arxiv.org/abs/2608.29753)Cited by:[§1](https://arxiv.org/html/2609.17695#S1.p2.1),[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px2.p1.1)\.
- Gaoet al\.\(2023\)T\. Gao, H\. Yen, J\. Yu, and D\. ChenEnabling large language models to generate text with citations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,EMNLP ’23,Singapore,pp\. 6465–6488\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.398),[Link](https://aclanthology.org/2023.emnlp-main.398/)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px2.p1.1)\.
- Gutiérrezet al\.\(2024\)B\. J\. Gutiérrez, Y\. Shu, Y\. Gu, M\. Yasunaga, and Y\. SuHippoRAG: neurobiologically inspired long\-term memory for large language models\.InAdvances in Neural Information Processing Systems 37,NeurIPS ’24,Vancouver, Canada,pp\. 59532–59569\.External Links:[Document](https://dx.doi.org/10.52202/079017-1902),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/6ddc001d07ca4f319af96a3024f6dbd1-Abstract-Conference.html)Cited by:[§1](https://arxiv.org/html/2609.17695#S1.p2.1),[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px2.p1.1)\.
- Gutiérrezet al\.\(2025\)B\. J\. Gutiérrez, Y\. Shu, W\. Qi, S\. Zhou, and Y\. SuFrom RAG to memory: non\-parametric continual learning for large language models\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,Vancouver, Canada,pp\. 21497–21515\.External Links:[Link](https://proceedings.mlr.press/v267/gutierrez25a.html)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px2.p1.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InProceedings of the Tenth International Conference on Learning Representations,ICLR ’22,Virtual\.External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§4\.3](https://arxiv.org/html/2609.17695#S4.SS3.p2.1)\.
- Jeonget al\.\(2024\)S\. Jeong, J\. Baek, S\. Cho, S\. J\. Hwang, and J\. ParkAdaptive\-RAG: learning to adapt retrieval\-augmented large language models through question complexity\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),NAACL ’24,Mexico City, Mexico,pp\. 7036–7050\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.389),[Link](https://aclanthology.org/2024.naacl-long.389/)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2024\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638),[Link](https://aclanthology.org/2024.tacl-1.9/)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2026\)Y\. Liu, B\. Luo, Y\. Tang, R\. Qiu, and Z\. HuangCall Neighbours Yourself: graph walks with destination\-conditioned on\-policy self\-distillation\.ArXiv preprintarXiv:2608\.29588\.External Links:[Link](https://arxiv.org/abs/2608.29588)Cited by:[§1](https://arxiv.org/html/2609.17695#S1.p2.1),[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px3.p1.1)\.
- Rosset al\.\(2026\)J\. J\. Ross, B\. Koopman, A\. van der Vegt, and G\. ZucconHow retriever redundancy and diversity impact RAG effectiveness\.ArXiv preprintarXiv:2608\.13956\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2608.13956),[Link](https://arxiv.org/abs/2608.13956)Cited by:[§1](https://arxiv.org/html/2609.17695#S1.p2.1),[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px1.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.ArXiv preprintarXiv:2402\.03300\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2402.03300),[Link](https://arxiv.org/abs/2402.03300)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2609.17695#S4.SS3.p2.1)\.
- Tanet al\.\(2024\)H\. Tan, F\. Sun, W\. Yang, Y\. Wang, Q\. Cao, and X\. ChengBlinded by generated contexts: how language models merge generated and retrieved contexts when knowledge conflicts?\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),ACL ’24,Bangkok, Thailand,pp\. 6207–6227\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.337),[Link](https://aclanthology.org/2024.acl-long.337/)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px1.p1.1)\.
- Trivediet al\.\(2023\)H\. Trivedi, N\. Balasubramanian, T\. Khot, and A\. SabharwalInterleaving retrieval with chain\-of\-thought reasoning for knowledge\-intensive multi\-step questions\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),ACL ’23,Toronto, Canada,pp\. 10014–10037\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.557),[Link](https://aclanthology.org/2023.acl-long.557/)Cited by:[§1](https://arxiv.org/html/2609.17695#S1.p2.1),[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px3.p1.1)\.
- Waddenet al\.\(2020\)D\. Wadden, S\. Lin, K\. Lo, L\. L\. Wang, M\. van Zuylen, A\. Cohan, and H\. HajishirziFact or fiction: verifying scientific claims\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,EMNLP ’20,Online,pp\. 7534–7550\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.609),[Link](https://aclanthology.org/2020.emnlp-main.609/)Cited by:[§3\.4](https://arxiv.org/html/2609.17695#S3.SS4.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InProceedings of the Eleventh International Conference on Learning Representations,ICLR ’23,Kigali, Rwanda\.External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px3.p1.1)\.
- Yoranet al\.\(2024\)O\. Yoran, T\. Wolfson, O\. Ram, and J\. BerantMaking retrieval\-augmented language models robust to irrelevant context\.InProceedings of the Twelfth International Conference on Learning Representations,ICLR ’24,Vienna, Austria\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/8011b23e1dc3f57e1b6211ccad498919-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2609.17695#S2.SS0.SSS0.Px1.p1.1)\.

## Appendix AData construction checks

GraphEcho\-Syn keeps claims and entity names disjoint across training, development, and test splits\. Construction checks verify atom counts, path counts, path lengths, and matched evidence content across interventions\. The SciFact conversion preserves the original document identifiers and rationale sets, grouping each document’s rationale sentences into one evidential origin\. Additional representations retain the atom’s source identity and preserve the evidence\-balance label\.

## Appendix BStatic and conflict responses

Table[6](https://arxiv.org/html/2609.17695#A6.T6)reports the paired conflict contrasts\. Figure[3](https://arxiv.org/html/2609.17695#A6.F3)shows all one\-sided supporting conditions\. The source\-count intervention preserves path templates and evidence wording\. The comparison therefore attributes response differences to the provenance assignment within this construction, while retaining model\-specific directions of change\. An unchanged SUPPORT rate can coexist with a probability shift toward redundant evidence\.

## Appendix CStatic mitigation and task quality

Tables[7](https://arxiv.org/html/2609.17695#A6.T7)and[8](https://arxiv.org/html/2609.17695#A6.T8)use common presentation seed zero for every method\. The synthetic setting contains 200 claims and ten conditions\. SciFact contains 188 claims and 584 instances; only ten claims contribute to document\-count effects\. We average each claim’s available conditions before pooling task\-quality estimates\. Document\-count effects compare the available two\-to\-five\-document set against one document at a matched path count\. A stratum containing one claim provides a descriptive observation without an inferential interval\. Within\-claim averaging gives each claim equal weight despite different numbers of available variants\.

## Appendix DAgent validity and condition\-level behavior

Table[9](https://arxiv.org/html/2609.17695#A6.T9)reports exact operational counts; all five policies attempt at least one walk in every received output\. Table[10](https://arxiv.org/html/2609.17695#A6.T10)conditions on final\-output validity and therefore describes a selected subset\. Figure[4](https://arxiv.org/html/2609.17695#A6.F4)retains all received outputs for accuracy and shows each conflict condition separately\. Malformed responses count as incorrect judgments in the primary evaluation\. Invalid actions enter the walk denominator; only valid repeated evidence visits enter the echo numerator\.

## Appendix ETraining and sensitivity settings

All four policies use the same 500 balanced training instances, one epoch, seed 42, learning rate10−510^\{\-5\}, group size four, LoRA rank 16 and alpha 32\. Training samples actions with temperature 0\.7 and top\-pp0\.9\. Training uses a zero Kullback–Leibler \(KL\) coefficient and gradient clipping at norm one\. Checkpoint selection compares the policies at training steps 450, 475, and 500\. Each candidate receives the same 400\-instance development evaluation on C1/C2/C3/C2R\. Development full\-objective reward selects the test policy\. This shared selection criterion evaluates every reward ablation against the same acquisition objective\.

Figure[5](https://arxiv.org/html/2609.17695#A6.F5)shows that full\-reward and verdict\-only training attain different echo\-rate and source\-recall trajectories\. The curves describe optimization conditional on one training seed\.

Table[11](https://arxiv.org/html/2609.17695#A6.T11)pairs the frozen and trained agents under alternate budgets and graph constructions\. Ring topology changes adjacency, and the variable\-hop construction cycles through two to six hops while preserving evidence and gold labels\. All three frozen Qwen models and full PAPT also complete static sampling and variable\-hop evaluations\. These tests keep training fixed and evaluate sensitivity conditional on the fitted policies\.

## Appendix FSupplementary SciFact baselines

Table[12](https://arxiv.org/html/2609.17695#A6.T12)compares the frozen models on scientific rationales and reports the supplementary GPT\-5\-mini controls\. The static and agent columns measure different outcomes and use the stated presentation\-seed scopes\. The agent columns use 1S5P, where all routes share one document origin\. Positive static PME indicates movement toward the gold evidence stance under repetition\. Since this agent condition contains one origin, its echo rate describes repeated exposure within a document\.

Table 6:Conflict effects for C2 minus C1 in percentage points, with 95% paired base\-claim bootstrap intervals\.Figure 3:Probability responses separate redundant paths from distinct origins\. Same\-origin curves keep one source; distinct\-origin curves match source count to path count\. Error bars show 95% claim\-bootstrap intervals over 200 claims and five seeds\.Table 7:Qwen3\-4B static mitigation and transfer effects in percentage points\. All methods use presentation seed zero\. The dagger denotes stance alignment\. Brackets show 95% base\-claim bootstrap intervals; n/a denotes an unscheduled control\.Table 8:Task quality accompanies redundancy sensitivity for Qwen3\-4B\. All methods use presentation seed zero\. Accuracy uses percent\. DPE compares each available document set against one document at matched path counts, pooling the eligible two\-to\-five\-document claims\. Brackets show 95% base\-claim bootstrap intervals\.Table 9:Operational counts over 4,000 synthetic agent outputs per policy \(200 claims, four conditions and five seeds\)\. Counts describe received outputs; a valid output format does not guarantee a correct verdict\.Table 10:Final\-output validity changes the population available for a diagnostic\. Each condition contains 1,000 outputs\. The last column gives C2\-minus\-C1 accuracy among matched pairs with valid finals on both sides, with 95% claim\-bootstrap intervals\.Figure 4:Echo\-rich conditions affect acquisition and accuracy differently\. Frozen agents use 200 claims, five seeds and budget six; error bars show 95% claim\-bootstrap intervals\.Table 11:PAPT\-minus\-frozen agent differences across acquisition settings\. Each comparison uses 200 claims, C1/C2/C3/C2R and seed zero\. Entries show percentage points and 95% paired claim\-bootstrap intervals\. Ring and variable\-hop evaluations use budget six\.Figure 5:Reward variants produce different acquisition behavior during training\. Curves use trailing ten\-step averages and the full reward for comparison across objectives; the traces describe one training seed\.Table 12:SciFact frozen baselines and supplementary API controls\. Static PME aligns the redundant\-path effect with gold evidence stance; M1 uses five presentation seeds and M2/M3 use seed zero\. Agent columns show the 1S5P condition at seed zero\. Agent estimates use 188 claims\. Static PME uses 188 claims except GPT\-5\-mini M3, which retains 187 claims with valid paired outputs\. Brackets show 95% claim\-bootstrap intervals\.

Similar Articles

Towards Security-Auditable LLM Agents: A Unified Graph Representation

arXiv cs.AI

This paper introduces Agent-BOM, a unified graph representation for security auditing in LLM-based agentic systems. It addresses the semantic gap in post-hoc auditing by modeling static capabilities and dynamic runtime states to detect complex attack chains like memory poisoning and tool misuse.

Graph Engineering (GitHub Repo)

TLDR AI

A curated collection of research papers, benchmarks, and open-source projects on Graph Engineering in the era of LLM Agents, accompanying an arXiv survey paper to advance research from individual to system intelligence.