SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning

# SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning
Source: [https://arxiv.org/html/2608.22132](https://arxiv.org/html/2608.22132)
Zhaohan MengAffiliation:School of Computing Science, University of GlasgowAffiliation:Brigham and Women’s Hospital, Harvard Medical School, Harvard UniversityZaiqiao MengAffiliation:School of Computing Science, University of GlasgowAffiliation:Language Technology Lab, University of CambridgeHao XuAffiliation:Brigham and Women’s Hospital, Harvard Medical School, Harvard UniversityKe YuanAffiliation:School of Cancer Sciences, University of GlasgowAffiliation:Cancer Research UK Scotland Institutez\.meng\.3@research\.gla\.ac\.ukIadh OunisAffiliation:School of Computing Science, University of Glasgow

###### Abstract

Biomedical multi\-hop question answering \(QA\) requires models to connect evidence across intermediate entities such as diseases, drugs, proteins, and phenotypes\. Existing agents typically rely on static retrieval workflows or coarse\-grained prompt rewriting, which can lead to instruction drift when reasoning procedures need to be updated\. We proposeSSE\-Bio, a structured self\-evolving agent with an agentic retrieval policy for multi\-hop biomedical reasoning\. Instead of globally rewriting agent instructions, SSE\-Bio maintains a structured state, selectively retrieves knowledge triplets and prior templates through a trainable proxy policy, and improves its reasoning memory through fine\-grained template editing\. To optimise retrieval decisions, we introduce a proxy\-training strategy based on group relative policy optimization, where the proxy is improved through decision\-contrastive groups over alternative retrieval choices\. Experiments on three biomedical multi\-hop QA benchmarks show that SSE\-Bio consistently outperforms existing baselines, achieving an improvement of 6\.56 absolute points over the strongest self\-evolving baseline on BioHopR\.111The source code for SSE\-Bio is publicly available at[https://github\.com/ZhaohanM/SSE\-Bio](https://github.com/ZhaohanM/SSE-Bio)\.

## 1Introduction

Biomedical multi\-hop question answering \(QA\) task is important for evidence\-grounded biomedical discovery, such as disease understanding and drug repurposing[35](https://arxiv.org/html/2608.22132#bib.bib4)\. It requires models to answer questions by reasoning across multiple entities, such as diseases, drugs, and proteins, connected through complex biomedical relations[30](https://arxiv.org/html/2608.22132#bib.bib10);[19](https://arxiv.org/html/2608.22132#bib.bib20);[31](https://arxiv.org/html/2608.22132#bib.bib5)\. For example, a disease\-oriented biomedical question may require a system to first identify the bridge protein associated with the disease and then infer the drug linked to that protein\. It is challenging because biomedical evidence is highly specialised and relationally structured, so errors in resolving the intermediate bridge entity can mislead the subsequent reasoning chain, while in multi\-answer settings the system must avoid prematurely stopping after finding only a subset of valid answers[16](https://arxiv.org/html/2608.22132#bib.bib30);[4](https://arxiv.org/html/2608.22132#bib.bib27);[20](https://arxiv.org/html/2608.22132#bib.bib24)\.

Recent retrieval\-augmented generation \(RAG\) models have improved the performance of multi\-hop biomedical QA by incorporating external knowledge into the reasoning process[10](https://arxiv.org/html/2608.22132#bib.bib31);[2](https://arxiv.org/html/2608.22132#bib.bib21);[24](https://arxiv.org/html/2608.22132#bib.bib7)\. However, without explicit verification and feedback over the reasoning path, they may leave intermediate bridge entities unresolved, causing the reasoning chain to drift and potentially returning incomplete answers in multi\-answer scenarios\. Interactive multi\-agent systems make this process more flexible by enabling step\-wise planning, execution, and feedback, while self\-evolving agents further introduce long\-term memory to adapt reasoning strategies across questions[27](https://arxiv.org/html/2608.22132#bib.bib2);[8](https://arxiv.org/html/2608.22132#bib.bib16)\. Yet these agentic systems still face two limitations\. First, they often rely on static retrieval workflows rather than explicitly learned retrieval control, which can lead to sub\-optimal performance[11](https://arxiv.org/html/2608.22132#bib.bib35)\. Second, their memory updates are often coarse\-grained, typically through prompt\-level rewriting, so that a local failure may alter the entire reasoning scaffold[13](https://arxiv.org/html/2608.22132#bib.bib14);[6](https://arxiv.org/html/2608.22132#bib.bib15)\. These limitations motivate a more controllable framework that combines explicit retrieval control with fine\-grained memory evolution\.

To address these limitations, we proposeSSE\-Bio, a structured self\-evolving agent with an agentic retrieval policy for biomedical multi\-hop QA\. As shown in Fig\.[1](https://arxiv.org/html/2608.22132#S1.F1), SSE\-Bio combines structured reasoning memory with explicit retrieval management in a unified agentic framework\. During inference, theManagerconverts the structured state into a query\-specific plan, which theDevagent executes with the retrieved triplets, while theCriticprovides structured feedback for iterative refinement\. TheManagermaintains two forms of memory: a structured state that captures the current reasoning status and serves as the agent’s short\-term memory, and a long\-term memory of reusable templates distilled from prior successful cases\. Crucially, instead of evolving memory through global rewriting, SSE\-Bio updates templates through fine\-grained editing, enabling local and auditable memory evolution that is designed to mitigate hallucination and instruction\-drift risks from unconstrained template rewriting\. In addition, SSE\-Bio introduces a trainableProxymodel to manage retrieval explicitly by deciding whether knowledge triplets and/or prior templates should be retrieved at each reasoning step\. Knowledge triplets are beneficial in the biomedical domain since they organize complex biomedical information into a clear structure, giving the Manager structured bridge entities and relation paths to inspect during multi\-hop verification\. In this way, SSE\-Bio supports both dynamic evolution of reasoning memory over time and adaptive, selective retrieval from the current reasoning state, rather than relying on static retrieval pipelines or coarse\-grained agent rewriting\.

To optimise the retrieval behaviour of theProxy, we adopt a two\-stage training strategy\. We first use supervised fine\-tuning \(SFT\) to initialise the Proxy with retrieval decision labels that indicate whetherknowledge tripletsand/orprior templatesshould be retrieved under a given structured reasoning state\. These labels are constructed by comparing alternative retrieval branches and selecting the decisions that lead to better downstream reasoning outcomes\. We then further refine the Proxy with Group Relative Policy Optimization \(GRPO\)[25](https://arxiv.org/html/2608.22132#bib.bib36)\. In particular, we introducedecision\-contrastive trajectory generation, which expands alternative retrieval actions from the same structured reasoning state into contrastive rollout branches\. The optimisation is driven by an answer\-grounded reward that combines final answer correctness and evidence\-supported reasoning behaviour, allowing GRPO to favour retrieval decisions that are both effective and well grounded\. We evaluate SSE\-Bio on three biomedical reasoning benchmarks, namely BioHopR, MedHop, and Humanity’s Last Exam: Biomedicine, and show that it consistently outperforms strong baselines\. Our contributions are summarised as follows:

- •We proposeSSE\-Bio, a structured self\-evolving agent for multi\-hop biomedical reasoning, which unifies short\-term structured state tracking, long\-term template memory, and explicit retrieval management within a single agentic framework\.
- •We introduce afine\-grained template evolutionmechanism that updates retrieved templates locally from successful trajectories, reducing unnecessary changes from the whole\-template rewriting used in existing self\-evolving agents\.
- •We design aGRPO\-based optimisation strategyfor the Proxy, which couples decision\-contrastive trajectory generation with answer\-grounded reward signals to compare retrieval branches under matched reasoning contexts and improve retrieval decisions through grounded supervision\.

![Refer to caption](https://arxiv.org/html/2608.22132v1/SSE-Bio.png)Figure 1:Left:TheManagermaintains a structured state and template memory\. Based on the current state, theProxydecides whether to retrieve knowledge triplets and/or prior templates\. TheManagerthen constructs a plan for theDevagent, while theCriticreturns structured feedback for iterative refinement\.Right:Decision\-contrastive trajectory generation explores alternative retrieval actions from the same state, and GRPO updates only theProxywith an answer\-grounded reward combining correctness and evidence support\.
## 2Related Work

#### Biomedical RAG Models\.

They improve biomedical QA by incorporating external knowledge into the reasoning process[29](https://arxiv.org/html/2608.22132#bib.bib3);[5](https://arxiv.org/html/2608.22132#bib.bib22)\. In particular, i\-MedRAG[36](https://arxiv.org/html/2608.22132#bib.bib17)strengthens biomedical QA by retrieving medical evidence to support answer generation\. Similarly, MedGraphRAG[34](https://arxiv.org/html/2608.22132#bib.bib19)organises medical knowledge into graph structure so that retrieval can better reflect biomedical relations\. Building on this direction, KRAGEN[17](https://arxiv.org/html/2608.22132#bib.bib18)combines knowledge graphs with retrieval\-augmented reasoning for biomedical problem solving\. More recently, AMG\-RAG[24](https://arxiv.org/html/2608.22132#bib.bib7)further extends this line through agentic medical graph retrieval for evidence\-grounded reasoning\. Together, these methods show that external knowledge can improve biomedical multi\-hop QA, but they still lack explicit verification and feedback over the evolving reasoning path, making them vulnerable to incorrect bridge\-entity resolution and incomplete answers in multi\-answer settings\.

#### Biomedical Multi\-agent Systems\.

They improve complex biomedical reasoning by enabling step\-wise planning, execution, and feedback through specialised agent roles[1](https://arxiv.org/html/2608.22132#bib.bib26);[4](https://arxiv.org/html/2608.22132#bib.bib27)\. MDAgents[14](https://arxiv.org/html/2608.22132#bib.bib28)improves medical reasoning through collaborative specialist agents\. KGARevion[28](https://arxiv.org/html/2608.22132#bib.bib23)introduces a knowledge\-graph\-based agent for knowledge\-intensive biomedical QA\. Biomni[11](https://arxiv.org/html/2608.22132#bib.bib35)further extends this direction through broader biomedical tool use\. Beyond such multi\-agent coordination, self\-evolving systems aim to improve reasoning across questions by accumulating experience from previous trajectories\. STELLA[13](https://arxiv.org/html/2608.22132#bib.bib14)is a self\-evolving biomedical agent that improves over time through an evolving Template Library and a dynamic Tool Ocean\. Although it shows that long\-term adaptation can improve biomedical reasoning, it provides limited control over when and what evidence should be retrieved, and its self\-evolution updates templates by rewriting them from newly successful cases\. In contrast, SSE\-Bio explicitly couples state\-aware retrieval control with structured long\-term memory that evolves through fine\-grained local editing, making both evidence use and memory evolution more bounded and controllable\.

## 3Methodology

We formulate SSE\-Bio as an iterative agentic process in Sec\.[3\.1](https://arxiv.org/html/2608.22132#S3.SS1), then introduce its structured memory in Sec\.[3\.2](https://arxiv.org/html/2608.22132#S3.SS2), agentic retrieval policy in Sec\.[3\.3](https://arxiv.org/html/2608.22132#S3.SS3), andProxypolicy training in Sec\.[3\.4](https://arxiv.org/html/2608.22132#S3.SS4)\.

### 3\.1Agentic Formulation

We formulate biomedical multi\-hop reasoning in SSE\-Bio as an iterative agentic process involving four components: aManager, aProxy, aDevagent, and aCritic\. Given a biomedical questionqq, the goal of the agent is to resolve intermediate entities, verify the reasoning chain, and produce the answer\. At reasoning roundtt, theManagermaintains a structured stateutu\_\{t\}as the short\-term memory of the current question\. Conditioned onutu\_\{t\}, theProxyselects a retrieval actionata\_\{t\}, which determines whether external knowledge should be introduced at the current step\. The selected evidence is returned to theManager, which combines it withutu\_\{t\}to construct a query\-specific planπt\\pi\_\{t\}for theDevagent\. SSE\-Bio also maintains a long\-term memoryℰ\\mathcal\{E\}of reusable templates distilled from prior successful trajectories\.

TheDevagent executes the current plan and produces a reasoning trajectoryτt\\tau\_\{t\}and answery^t\\hat\{y\}\_\{t\}\. TheCritic, without access to the gold answer, assesses whether the trajectory and answer are coherent and sufficiently supported, and returns structured feedbackftf\_\{t\}for refinement\. If the current reasoning fails, theManagerupdatesutu\_\{t\}and replans the next round\. If it succeeds, SSE\-Bio updatesℰ\\mathcal\{E\}through fine\-grained template editing or new\-template distillation\. Thus, SSE\-Bio evolves through short\-term structured state tracking within each question and long\-term memory evolution across questions\.

### 3\.2Structured Memory

#### Short\-term Memory\.

At each reasoning step, theManagermaintains a structured stateutu\_\{t\}as the short\-term memory of the current question\. Concretely,task typedenotes the relation pattern of the current question,information gapspecifies the missing evidence needed for the next reasoning step,feedbackrecords the Critic’s structured comments on the current trajectory, andretrieval statesummarises what external evidence has already been retrieved\. This state compactly represents the current reasoning state and makes retrieval decisions, planning, and refinement all grounded in the same explicit structure rather than scattered across free\-form intermediate text\. The structured state is updated across rounds to reflect reasoning progress and the structured feedback returned by theCritic\. An example is provided in Fig\.[3](https://arxiv.org/html/2608.22132#A1.F3)\.

#### Long\-term Memory Evolution\.

Beyond the current question, SSE\-Bio maintains a long\-term memory of reusable templates distilled from prior successful cases\. Each template stores reusable reasoning experience through a fixed set of fields:task typeidentifies the question pattern to which the template applies,reasoning flowrecords a reusable high\-level solution procedure,verification criteriaspecifies what conditions must be checked before returning an answer,tool\-use policyspecifies what external tools can be used to support the reasoning process, andfailure warninghighlights common mistakes to avoid\. These templates are retrieved as planning priors for new questions\. If the trajectory is aligned with a retrieved template, theManagerperforms fine\-grained template editing, updating only the local fields that require refinement\. Otherwise, SSE\-Bio performs new\-template distillation, adding a new reusable template to memory\. This design enables controllable memory evolution and constrains the kinds of broad template changes that can lead to instruction drift under coarse\-grained rewriting\. An example is provided in Fig\.[4](https://arxiv.org/html/2608.22132#A1.F4)\.

### 3\.3Agentic Retrieval Policy

TheProxyis responsible for explicitly controlling whether external knowledge should be introduced at each reasoning round\. At reasoning roundtt, the Proxy selects a retrieval action:

at∼πθ\(⋅∣ut\),at∈𝒜=2\{KG,TPL\},a\_\{t\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid u\_\{t\}\),\\qquad a\_\{t\}\\in\\mathcal\{A\}=2^\{\\\{\\mathrm\{KG\},\\mathrm\{TPL\}\\\}\},\(1\)whereKG\\mathrm\{KG\}andTPL\\mathrm\{TPL\}denote knowledge\-triplet retrieval and template retrieval, respectively\. This yields four actions: retrieving neither source, retrieving only knowledge triplets, retrieving only templates, or retrieving both\. WhenKG∈at\\mathrm\{KG\}\\in a\_\{t\}, SSE\-Bio uses theinformation gapfield inutu\_\{t\}as the retrieval query, encodes it together with candidate biomedical triplets using BiCA[26](https://arxiv.org/html/2608.22132#bib.bib11), and returns the top\-KKtriplets after schema and duplicate filtering\. WhenTPL∈at\\mathrm\{TPL\}\\in a\_\{t\}, SSE\-Bio uses thetask typeencoded inutu\_\{t\}as the retrieval query, applies the same BiCA\-based retrieval procedure to the long\-term memoryℰ\\mathcal\{E\}, and returns the top\-KKcompatible template\. In this way, knowledge\-triplet retrieval provides step\-specific factual support, while template retrieval provides a reusable reasoning prior\. Appendix[A\.2](https://arxiv.org/html/2608.22132#A1.SS2)and[A\.6](https://arxiv.org/html/2608.22132#A1.SS6)report the memory schema and the retrieval budgets\.

### 3\.4Training of the Proxy Policy

TheProxyis the only trainable component in SSE\-Bio; the Manager, Dev, Critic, retrievers, and reasoning environment remain fixed\. The goal of training is to learn a retrieval policyπθ​\(at∣ut\)\\pi\_\{\\theta\}\(a\_\{t\}\\mid u\_\{t\}\)that makes better retrieval decisions under the current structured reasoning state\.

#### SFT Initialisation\.

We first use SFT[32](https://arxiv.org/html/2608.22132#bib.bib9);[12](https://arxiv.org/html/2608.22132#bib.bib8)to initialise the Proxy with retrieval decision pseudo\-labels\. For a given structured stateutu\_\{t\}, we compare alternative retrieval branches and assign as pseudo\-label the action whose rollout achieves the highest reward defined below\. This stage provides a stable initialisation for the policy before reinforcement learning\.

#### Decision Contrastive Trajectory Generation\.

We then refine the Proxy with GRPO[25](https://arxiv.org/html/2608.22132#bib.bib36);[22](https://arxiv.org/html/2608.22132#bib.bib6)\. SSE\-Bio constructsdecision\-contrastive trajectory groupsby expanding alternative retrieval actions from the same reasoning state, forming matched rollout branches as shown in Fig\.[1](https://arxiv.org/html/2608.22132#S1.F1)\. Branching is applied only in early reasoning rounds\. Partial trajectories are pruned if they violate the required relation schema, leave the current information gap unresolved, or produce an unsupported bridge entity under the required schema; the remaining branches are ranked by intermediate answer\-grounded reward signals, and only the top\-KKare retained for further rollout\. This keeps exploration focused while avoiding exponential growth\. Appendix[A\.7](https://arxiv.org/html/2608.22132#A1.SS7)provides the full pruning criteria and rollout\-group statistics\.

#### Reward Shaping\.

The Proxy is trained with an answer\-grounded composite reward:

R⁡\(τ\)=λo​Rout​\(τ\)\+λb​Rbeh​\(τ\),R\(\\tau\)=\\lambda\_\{o\}R\_\{\\mathrm\{out\}\}\(\\tau\)\+\\lambda\_\{b\}R\_\{\\mathrm\{beh\}\}\(\\tau\),\(2\)whereλo\+λb=1\\lambda\_\{o\}\+\\lambda\_\{b\}=1\.

Outcome reward\.The first term rewards final answer correctness:

Rout​\(τ\)=𝟏​\{corr⁡\(τ\)\},R\_\{\\mathrm\{out\}\}\(\\tau\)=\\mathbf\{1\}\\\{\\mathrm\{corr\}\(\\tau\)\\\},\(3\)wherecorr⁡\(τ\)\\mathrm\{corr\}\(\\tau\)indicates whether the final answer generated by trajectoryτ\\taumatches the gold answer of the current training question\.

Behavioural reward\.Final answer correctness alone is too sparse to distinguish trajectories with similar outcomes but different grounding quality\. Therefore, we introduce a behavioural reward that measures whether key reasoning steps are supported by the retrieved evidence:

Rbeh​\(τ\)=1\|𝒮τ\|​∑s∈𝒮τ𝟏​\{s​is evidence\-supported\},R\_\{\\mathrm\{beh\}\}\(\\tau\)=\\frac\{1\}\{\|\\mathcal\{S\}\_\{\\tau\}\|\}\\sum\_\{s\\in\\mathcal\{S\}\_\{\\tau\}\}\\mathbf\{1\}\\\{s\\ \\text\{is evidence\-supported\}\\\},\(4\)where𝒮τ\\mathcal\{S\}\_\{\\tau\}denotes key steps such as bridge resolution and answer verification\. A step is counted as supported only when it is grounded by a retrieved triplet or is consistent with the verification criteria specified by the retrieved template: bridge resolution requires a retrieved query–bridge relation, while answer verification requires retrieved or template\-supported bridge–answer evidence\. This term rewards grounded use of evidence rather than additional retrieval calls, with full support rules in Appendix table[12](https://arxiv.org/html/2608.22132#A1.T12)\.

#### Group\-relative Optimisation\.

For a retained group\{τi\}i=1K\\\{\\tau\_\{i\}\\\}\_\{i=1\}^\{K\}with rewards\{ri\}i=1K\\\{r\_\{i\}\\\}\_\{i=1\}^\{K\}, we compute the group\-relative advantage as follows:

Ai=ri−μ⁡\(𝐫\)σ⁡\(𝐫\)\+ϵ,A\_\{i\}=\\frac\{r\_\{i\}\-\\mu\(\\mathbf\{r\}\)\}\{\\sigma\(\\mathbf\{r\}\)\+\\epsilon\},\(5\)where𝐫\\mathbf\{r\}denotes the rewards within the same trajectory group\. Let\(ui,ai\)\(u\_\{i\},a\_\{i\}\)denote the Proxy state and retrieval action associated with trajectoryτi\\tau\_\{i\}\. Since the Proxy outputs discrete retrieval actions rather than tokens, we apply GRPO at the action level and compare the current policy with the old sampling policy:

ρi​\(θ\)=πθ​\(ai∣ui\)πθold​\(ai∣ui\)\.\\rho\_\{i\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(a\_\{i\}\\mid u\_\{i\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(a\_\{i\}\\mid u\_\{i\}\)\}\.\(6\)Withρ¯i=clip⁡\(ρi,1−ϵc,1\+ϵc\)\\bar\{\\rho\}\_\{i\}=\\mathrm\{clip\}\(\\rho\_\{i\},1\-\\epsilon\_\{c\},1\+\\epsilon\_\{c\}\), the Proxy is trained by maximising the action\-level GRPO objective:

𝒥GRPO=𝔼i​\[min⁡\(ρi​Ai,ρ¯i​Ai\)−βKL​DKLi\],\\mathcal\{J\}\_\{\\mathrm\{GRPO\}\}=\\mathbb\{E\}\_\{i\}\\\!\\left\[\\min\(\\rho\_\{i\}A\_\{i\},\\bar\{\\rho\}\_\{i\}A\_\{i\}\)\-\\beta\_\{\\mathrm\{KL\}\}D\_\{\\mathrm\{KL\}\}^\{i\}\\right\],\(7\)whereDKLiD\_\{\\mathrm\{KL\}\}^\{i\}regularises the retrieval\-action distribution against the frozen SFT reference policy\. Appendix[A\.7](https://arxiv.org/html/2608.22132#A1.SS7)reports the pruning rules and rollout statistics used for this optimisation\.

FamilyMethodSingle AnswerMulti AnswerPrecH1PrecH2Bothcor↑\\uparrowBothwr↓\\downarrowPrecH1PrecH2Bothcor↑\\uparrowBothwr↓\\downarrowLLMLlama\-3\.1\-8B0\.130\.040\.0099\.830\.070\.010\.0099\.92LLMLlama\-3\.1\-70B26\.419\.454\.9469\.0619\.826\.943\.2476\.51LLMGPT\-4o32\.8614\.597\.8460\.4325\.4810\.715\.2569\.03Medical LLMHuatuoGPT\-70B0\.150\.000\.0099\.840\.090\.000\.0099\.91Medical LLMHuatuoGPT\-8B0\.210\.030\.0099\.760\.110\.010\.0099\.88Medical LLMUltraMedical\-8B13\.775\.192\.2983\.309\.443\.291\.3688\.60RAG Modeli\-MedRAG25\.207\.484\.4371\.7516\.865\.732\.4979\.90RAG ModelAMG\-RAG28\.138\.945\.6168\.4719\.266\.623\.3877\.21Biomedical AgentKGARevion30\.8410\.626\.7865\.3221\.477\.824\.4175\.12Biomedical AgentDoctorAgent\-RL34\.8612\.748\.1260\.5226\.189\.365\.7270\.18Biomedical AgentMedAgents36\.7413\.628\.8958\.5328\.0510\.126\.1868\.01Biomedical AgentBiomni39\.1215\.3010\.4556\.0330\.6411\.417\.2865\.23Self\-evolving AgentSTELLA38\.9414\.899\.9656\.1330\.1210\.986\.9165\.81Self\-evolving AgentSSE\-Bio47\.21†21\.38†16\.52†47\.93†38\.36†16\.24†11\.73†57\.13†Table 1:Performance \(%\) comparison onBioHopR\. PrecH1and PrecH2evaluate the linked 1\-hop and 2\-hop questions, while Bothcorand Bothwrmeasure whether both questions in a pair are jointly correct or jointly wrong\.†indicates a significant improvement over STELLA \(paired t\-test,p<0\.05p<0\.05\)\.![Refer to caption](https://arxiv.org/html/2608.22132v1/SSE-Bio_Figure2.png)Figure 2:\(1\)Zero\-shot accuracy \(%\) comparison with agentic baselines on HLE\.\(2\)Zero\-shot accuracy \(%\) comparison with agentic baselines on MedHop\.\(3\)Performance \(%\) of different proxy backbones on BioHopR\.

## 4Experimental Setup

Datasets\.We evaluate SSE\-Bio on three biomedical reasoning benchmarks\. BioHopR[15](https://arxiv.org/html/2608.22132#bib.bib32)contains 7\.63K paired instances constructed from complex biomedical relations, and we use a 7:3 train/test split, yielding approximately 5\.34K training pairs and 2\.29K test pairs\. Each instance consists of a linked 1\-hop and 2\-hop question pair, and the benchmark supports both single\-answer and multi\-answer evaluation\. We further assess cross\-benchmark generalisation in a zero\-shot setting on MedHop[33](https://arxiv.org/html/2608.22132#bib.bib33)and Humanity’s Last Exam: Biomedicine \(HLE\)[23](https://arxiv.org/html/2608.22132#bib.bib34)\. Appendix[A\.8](https://arxiv.org/html/2608.22132#A1.SS8)provides further details\.

Baselines\.We compare SSE\-Bio against representative baselines from five categories: general\-purpose LLMs[9](https://arxiv.org/html/2608.22132#bib.bib25);[21](https://arxiv.org/html/2608.22132#bib.bib12), medical LLMs[3](https://arxiv.org/html/2608.22132#bib.bib13), retrieval\-augmented generation[36](https://arxiv.org/html/2608.22132#bib.bib17);[24](https://arxiv.org/html/2608.22132#bib.bib7)methods, agent\-based systems[28](https://arxiv.org/html/2608.22132#bib.bib23);[7](https://arxiv.org/html/2608.22132#bib.bib29);[14](https://arxiv.org/html/2608.22132#bib.bib28);[11](https://arxiv.org/html/2608.22132#bib.bib35), and self\-evolving agents[13](https://arxiv.org/html/2608.22132#bib.bib14), which together cover non\-agentic, retrieval\-augmented, and self\-evolving reasoning settings\. For a fair comparison, all agentic baselines use the same backbone configuration as SSE\-Bio: Gemini\-2\.5\-Pro for Manager/Critic\-style roles and Claude\-4\.5\-Sonnet for execution\. Appendix[A\.10](https://arxiv.org/html/2608.22132#A1.SS10)provides details of each baseline\.

Evaluation\.For BioHopR, we follow the benchmark protocol and evaluate both single\-answer and multi\-answer instances using four metrics:PrecH1,PrecH2,Bothcor, andBothwr\. PrecH1and PrecH2measure precision on the linked 1\-hop and 2\-hop questions, while Bothcorand Bothwrmeasure the proportions of paired instances where both questions are answered correctly or incorrectly\. Correctness is computed by embedding\-based precision with cosine\-similarity matching between predictions and gold answers\. Higher values are better for PrecH1, PrecH2, and Bothcor, whereas lower values are better for Bothwr\. For MedHop and HLE, we report zero\-shot test accuracy\. Appendix[A\.9](https://arxiv.org/html/2608.22132#A1.SS9)provides further details\.

Implementation details\.We fix Gemini\-2\.5\-Pro as the Manager and Critic and Claude\-4\.5\-Sonnet as the Dev agent, so performance differences can be attributed to retrieval\-policy learning rather than changes in the reasoning backbones\. SSE\-Bio is trained on BioHopR and then transferred directly to MedHop and HLE for zero\-shot evaluation\. During evaluation, the template memory is frozen, and no instance from the BioHopR test split, MedHop, or HLE is used to create or edit templates\. Additional details are provided in Appendix[A\.5](https://arxiv.org/html/2608.22132#A1.SS5)\.

Model / SettingBackboneSingle AnswerMulti AnswerPrecH1PrecH2Bothcor↑\\uparrowBothwr↓\\downarrowPrecH1PrecH2Bothcor↑\\uparrowBothwr↓\\downarrowEffect of Structured State Tracking and Fine\-grained Template EvolutionFree\-form state \+ rewritingQwen\-72B41\.5816\.9311\.7253\.2132\.9412\.588\.0462\.52Structured state \+ rewritingQwen\-72B44\.1218\.9213\.8650\.8235\.0714\.189\.6660\.41Free\-form state \+ editingQwen\-72B45\.0319\.4714\.5250\.0235\.8814\.8610\.2859\.54Structured state \+ editingQwen\-72B47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13Component Ablationw/o proxyQwen\-72B39\.7415\.8810\.6255\.0030\.8111\.927\.0564\.32w/o tripletsQwen\-72B43\.9118\.6413\.6451\.0934\.7214\.029\.4160\.67w/o prior templateQwen\-72B44\.6319\.0214\.2650\.6135\.4514\.439\.9560\.07Effect of Training and Boundary AnalysisSSE\-Bio \(SFT\)Qwen\-8B44\.6719\.1114\.1050\.3235\.6814\.529\.8459\.64SSE\-Bio \(SFT\)Qwen\-72B45\.7219\.8815\.0849\.4836\.8115\.1910\.6258\.62SSE\-Bio \(GRPO\)Qwen\-8B46\.0820\.6415\.6248\.9037\.2415\.7111\.0858\.13SSE\-Bio \(GRPO\)Qwen\-72B47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13SSE\-BioQwen\-72B47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13\+ Oracle TripletsQwen\-72B47\.8622\.3117\.3447\.1739\.0217\.0412\.3656\.30\+ Oracle TemplateQwen\-72B47\.6321\.9817\.0247\.4138\.7516\.8112\.1456\.58\+ Oracle Triplets & TemplateQwen\-72B48\.3422\.9418\.0646\.7839\.6317\.6413\.0555\.78

Table 2:Ablation study and boundary analysis onBioHopR\.Relation TypeSTELLASSE\-BioSingle AnswerMulti AnswerSingle AnswerMulti AnswerPrecH1PrecH2PrecH1PrecH2PrecH1PrecH2PrecH1PrecH2Same Query and BridgeDisease:Drug:Phenotype48\.2722\.8338\.6410\.5859\.12†34\.26†47\.03†16\.82†Disease:Drug:Protein46\.7510\.3637\.424\.3752\.18†12\.41†41\.02†5\.06†Drug:Disease:Phenotype50\.1820\.6733\.158\.7458\.74†27\.35†37\.91†11\.96†Drug:Disease:Protein48\.8611\.9431\.833\.8950\.67†13\.18†33\.24†4\.52†Same Query and TargetDisease:Phenotype:Drug27\.7416\.1315\.686\.4730\.12†18\.42†17\.08†7\.31†Disease:Protein:Drug32\.9611\.3814\.257\.5441\.76†18\.93†19\.34†12\.84†Drug:Phenotype:Disease25\.978\.0413\.764\.4827\.18†9\.07†14\.91†5\.18†Drug:Protein:Disease30\.4317\.3520\.877\.4938\.84†25\.96†26\.72†11\.83†Phenotype:Disease:Drug10\.3713\.588\.2611\.2711\.54†15\.06†8\.97†15\.47†Phenotype:Drug:Disease23\.956\.5414\.784\.9631\.86†11\.74†19\.52†8\.36†Protein:Disease:Drug34\.287\.4615\.844\.1736\.11†8\.53†16\.92†4\.82†Protein:Drug:Disease29\.165\.3821\.854\.6937\.92†9\.86†28\.04†7\.74†

Table 3:Relation\-type breakdown onBioHopR\.†indicates a significant improvement \(paired t\-test,p<0\.05p<0\.05\)\.
## 5Results and Analysis

In this section, we answer five Research Questions \(RQ\) about SSE\-Bio: overall performance \(Sec\.[5\.1](https://arxiv.org/html/2608.22132#S5.SS1)\), the effect of the Proxy \(Sec\.[5\.2](https://arxiv.org/html/2608.22132#S5.SS2)\), structured self\-evolution \(Sec\.[5\.3](https://arxiv.org/html/2608.22132#S5.SS3)\), retrieval headroom \(Sec\.[5\.4](https://arxiv.org/html/2608.22132#S5.SS4)\), and robustness across biomedical relation types \(Sec\.[5\.5](https://arxiv.org/html/2608.22132#S5.SS5)\)\.

### 5\.1RQ1: How does SSE\-Bio compare with baselines?

We compare SSE\-Bio with strong baselines in Table[1](https://arxiv.org/html/2608.22132#S3.T1)and evaluate its cross\-benchmark transfer in Fig\.[2](https://arxiv.org/html/2608.22132#S3.F2)\. Table[1](https://arxiv.org/html/2608.22132#S3.T1)shows that SSE\-Bio achieves the best performance across all four BioHopR metrics under both single\-answer and multi\-answer settings\. In the single\-answer setting, SSE\-Bio significantly outperforms STELLA, on all four metrics \(paired t\-test,p<0\.05p<0\.05\), improving from 38\.94 to 47\.21 on PrecH1, from 14\.89 to 21\.38 on PrecH2, and from 9\.96 to 16\.52 on Bothcor, while reducing Bothwrfrom 56\.13 to 47\.93\. This shows that its gains extend beyond isolated hop\-level prediction to paired multi\-hop correctness\.

In the more challenging multi\-answer setting, SSE\-Bio again ranks first on all four metrics, reaching 38\.36 on PrecH1, 16\.24 on PrecH2, and 11\.73 on Bothcor, showing that its advantage is preserved under the harder answer\-coverage requirement\. Compared with i\-MedRAG and Biomni, SSE\-Bio further improves single\-answer Bothcorby 12\.09 and 6\.07 points, indicating that state\-aware retrieval and controllable memory evolution are more effective than fixed retrieval or generic biomedical agents\. Fig\.[2](https://arxiv.org/html/2608.22132#S3.F2)further shows that SSE\-Bio transfers strongly to MedHop and reaches 28\.4% accuracy on HLE\. These results show that SSE\-Bio consistently outperforms strong baselines while also exhibiting generalisable reasoning ability\. Token, latency, and sensitivity analyses are in Appendices[A\.4](https://arxiv.org/html/2608.22132#A1.SS4)and[A\.3](https://arxiv.org/html/2608.22132#A1.SS3)\.

### 5\.2RQ2: How does the Proxy affect SSE\-Bio?

We evaluate whether the Proxy is necessary, whether it improves over fixed retrieval schedules, and whether GRPO further improves retrieval policy learning\. In Table[2](https://arxiv.org/html/2608.22132#S4.T2), the w/o Proxy setting, which retrieves neither knowledge triplets nor prior templates, reduces single\-answer Bothcorfrom 16\.52 to 10\.62 and increases Bothwrfrom 47\.93 to 55\.00, indicating that retrieval control is important\. Appendix[A\.13](https://arxiv.org/html/2608.22132#A1.SS13)further shows that the learned Proxy outperforms fixed retrieval schedules under the same backbone and retrieval budget, confirming that the gain is not simply due to retrieving more context\. The training block in Table[2](https://arxiv.org/html/2608.22132#S4.T2)shows that GRPO improves over SFT\. With the Qwen\-72B Proxy, single\-answer Bothcorrises from 15\.08 to 16\.52 and Bothwrfalls from 49\.48 to 47\.93, with the same trend in the multi\-answer setting\. Appendix[A\.13](https://arxiv.org/html/2608.22132#A1.SS13)further shows that larger Proxy backbones yield stronger performance\. Overall, the Proxy acts as an effective trainable control point for state\-dependent retrieval decisions\.

### 5\.3RQ3: What are the effects of structured self\-evolution?

We evaluate structured self\-evolution by comparing free\-form and structured variants of short\-term state tracking, together with coarse\-grained rewriting and fine\-grained template editing for long\-term memory evolution, in Table[2](https://arxiv.org/html/2608.22132#S4.T2)\. Replacing both structured state and fine\-grained editing with free\-form state and coarse\-grained rewriting reduces single\-answer Bothcorfrom 16\.52 to 11\.72 and increases Bothwrfrom 47\.93 to 53\.21\. Introducing only structured state or only fine\-grained editing recovers part of this gap, yielding Bothcorvalues of 13\.86 and 14\.52, respectively, but neither matches the full structured design\. This pattern suggests that structured state tracking helps maintain an explicit reasoning status for retrieval and refinement, while fine\-grained template editing updates reusable memory locally rather than rewriting the whole reasoning scaffold\. Together, they provide a more controlled form of self\-evolution\. Appendix[A\.13](https://arxiv.org/html/2608.22132#A1.SS13)further supports this interpretation by showing that fine\-grained editing changes fewer unrelated fields and introduces fewer unsupported or contradictory instructions than coarse rewriting\.

### 5\.4RQ4: How much headroom remains under stronger retrieval?

We evaluate the remaining headroom of SSE\-Bio from two perspectives in Table[2](https://arxiv.org/html/2608.22132#S4.T2): theComponent Ablationblock tests whether triplets and prior templates each contribute to performance, and theBoundary Analysisblock estimates how much further the system could improve with stronger evidence and template selection\. In the component ablation block, removing triplets or prior templates reduces single\-answer Bothcorfrom 16\.52 to 13\.64 and 14\.26, respectively, showing that factual evidence and reusable reasoning priors are complementary rather than redundant\. In the boundary analysis block of Table[2](https://arxiv.org/html/2608.22132#S4.T2), oracle triplets improve single\-answer Bothcorto 17\.34, oracle templates improve it to 17\.02, and combining both reaches 18\.06 while reducing Bothwrto 46\.78\. The multi\-answer setting follows the same pattern, with combined oracle retrieval increasing Bothcorfrom 11\.73 to 13\.05 and reducing Bothwrfrom 57\.13 to 55\.78\. These results show that SSE\-Bio already learns useful retrieval behaviour, but still has measurable headroom under stronger evidence and template retrieval\. Appendix[A\.12](https://arxiv.org/html/2608.22132#A1.SS12)further evaluates robustness under injected noisy triplets\.

### 5\.5RQ5: How does SSE\-Bio perform across different relation types?

To assess whether SSE\-Bio’s gains generalise across biomedical reasoning patterns, we report a relation\-type breakdown on BioHopR and compare SSE\-Bio with STELLA\. As shown in Table[3](https://arxiv.org/html/2608.22132#S4.T3), SSE\-Bio improves over STELLA across all listed relation types, suggesting that its advantage is not confined to a specific query or bridge configuration\. The gains are especially clear for relations involving stronger second\-hop reasoning\. For example, Disease:Drug:Phenotype improves from 22\.83 to 34\.26 on single\-answer PrecH2and from 10\.58 to 16\.82 on multi\-answer PrecH2\. By contrast, Drug:Disease:Protein shows more modest gains, from 11\.94 to 13\.18 and from 3\.89 to 4\.52, indicating that some relation structures remain challenging\. Multi\-answer scores are consistently lower than single\-answer scores, reflecting the stricter requirement of retrieving multiple correct answers\. Overall, SSE\-Bio provides consistent relation\-level improvements, with larger gains on relation types requiring effective bridge tracking and second\-hop reasoning\. Appendix[A\.15](https://arxiv.org/html/2608.22132#A1.SS15)provides a concrete failure case showing how relation\-specific retrieval and local verification\-field editing correct an unsupported bridge\-entity error\.

## 6Conclusions

We presented SSE\-Bio, a structured self\-evolving agent for biomedical multi\-hop reasoning that combines state\-aware retrieval control through a trainable proxy with controllable memory evolution through structured templates and fine\-grained editing\. This design enables adaptive retrieval and reasoning without fixed retrieval pipelines or unconstrained self\-rewriting\. Experiments on three benchmarks show that SSE\-Bio consistently outperforms strong baselines\. On BioHopR, SSE\-Bio improves over the self\-evolving baseline, STELLA, by 6\.56 points on single\-answer Bothcor, while also achieving the best performance under multi\-answer settings\. Ablation and relation\-type analyses further show that these gains are supported by effective retrieval control and structured self\-evolution\. Overall, our results suggest that combining structured retrieval control with fine\-grained self\-evolution is a promising step towards more controllable biomedical reasoning agents\.

## Limitations

We identify the following limitations of our work\. \(1\) SSE\-Bio relies on retrieved biomedical triplets and prior templates to support multi\-hop reasoning\. Although our retrieval design is effective, the oracle analysis shows that stronger evidence and template matching can still improve downstream performance\. In future work, we plan to explore more advanced biomedical retrieval and template matching strategies\. \(2\) SSE\-Bio depends on long\-term template memory to support structured self\-evolution\. While fine\-grained editing makes memory updates more controllable than unconstrained rewriting, memory growth may still introduce redundant or outdated templates over time\. Future work should therefore investigate memory consolidation, pruning, and provenance tracking\. \(3\) Our proxy analysis shows that stronger proxy backbones consistently improve retrieval control\. Although SSE\-Bio already performs well with the current proxy settings, it remains an open question how far proxy scaling can further improve performance\. We leave the exploration of larger and more efficient proxy architectures to future work\. \(4\) SSE\-Bio trains only the Proxy while relying on fixed commercial Manager, Dev, and Critic agents, so reproduction may be affected by prompt details and API behaviour\. Moreover, SFT labels and GRPO rewards are induced from rollouts within the same fixed agent environment; future work should test whether the learned retrieval policy transfers across different agent backbones\.

## Acknowledgements

The authors acknowledge the use of resources provided by the Isambard\-AI National AI Research Resource \(AIRR\)[18](https://arxiv.org/html/2608.22132#bib.bib1)\. Isambard\-AI is operated by the University of Bristol and is funded by the UK Government’s Department for Science, Innovation and Technology \(DSIT\) via UK Research and Innovation; and the Science and Technology Facilities Council \[ST/AIRR/I\-A\-I/1023\]\.

K\.Y\. acknowledges support from Cancer Research UK \(EDDPGM\-Nov21/100001, DRCMDP\-Nov23/100010 and core funding to the CRUK Scotland Institute \(A31287\)\), BBSRC BB/V016067/1, Prostate Cancer UK MA\-TIA22\-001 and EU Horizon 2020 grant ID: 101016851\.

## References

- Almansooriet al\.\(2025\)M\. Almansoori, K\. Kumar, and H\. CholakkalSelf\-evolving multi\-agent simulations for realistic clinical interactions\.arXiv preprint arXiv:2503\.22678\.Cited by:[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px2.p1.1)\.
- Caoet al\.\(2024\)J\. Cao, J\. Fang, Z\. Meng, and S\. LiangKnowledge graph embedding: a survey from the perspective of representation spaces\.ACM Computing Surveys56\(6\),pp\. 1–42\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p2.1)\.
- Chenet al\.\(2024\)J\. Chen, Z\. Cai, K\. Ji, X\. Wang, W\. Liu, R\. Wang, J\. Hou, and B\. WangHuatuogpt\-o1, towards medical complex reasoning with llms\.arXiv preprint arXiv:2412\.18925\.Cited by:[2nd item](https://arxiv.org/html/2608.22132#A1.I2.i2.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Chenet al\.\(2025\)K\. Chen, X\. Li, T\. Yang, H\. Wang, W\. Dong, and Y\. GaoMdteamgpt: a self\-evolving llm\-based multi\-agent framework for multi\-disciplinary team medical consultation\.arXiv preprint arXiv:2503\.13856\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px2.p1.1)\.
- Fanget al\.\(2025\)J\. Fang, Z\. Meng, and C\. MacdonaldKirag: knowledge\-driven iterative retriever for enhancing retrieval\-augmented generation\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 18969–18985\.Cited by:[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px1.p1.1)\.
- Fenget al\.\(2026a\)X\. Feng, X\. Song, L\. Li, G\. Liu, and J\. ShaoSEARL: joint optimization of policy and tool graph memory for self\-evolving agents\.arXiv preprint arXiv:2604\.07791\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p2.1)\.
- Fenget al\.\(2026b\)Y\. Feng, J\. Wang, L\. Zhou, Z\. Lei, and Y\. LiDoctoragent\-rl: a multi\-agent collaborative reinforcement learning system for multi\-turn clinical dialogue\.InICASSP 2026\-2026 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 16952–16956\.Cited by:[4th item](https://arxiv.org/html/2608.22132#A1.I2.i4.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Gaoet al\.\(2025\)H\. Gao, J\. Geng, W\. Hua, M\. Hu, X\. Juan, H\. Liu, S\. Liu, J\. Qiu, X\. Qi, Y\. Wu,et al\.A survey of self\-evolving agents: on path to artificial super intelligence\.arXiv preprint arXiv:2507\.210461\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p2.1)\.
- Grattafioriet al\.\(2024\)A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[1st item](https://arxiv.org/html/2608.22132#A1.I2.i1.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Heet al\.\(2024\)X\. He, Y\. Tian, Y\. Sun, N\. Chawla, T\. Laurent, Y\. LeCun, X\. Bresson, and B\. HooiG\-retriever: retrieval\-augmented generation for textual graph understanding and question answering\.Advances in Neural Information Processing Systems37,pp\. 132876–132907\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p2.1)\.
- Huanget al\.\(2025a\)K\. Huang, S\. Zhang, H\. Wang, Y\. Qu, Y\. Lu, Y\. Roohani, R\. Li, L\. Qiu, G\. Li, J\. Zhang,et al\.Biomni: a general\-purpose biomedical ai agent\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.1101/2025.05.30.656746),[Link](https://doi.org/10.1101/2025.05.30.656746)Cited by:[4th item](https://arxiv.org/html/2608.22132#A1.I2.i4.p1.1),[§1](https://arxiv.org/html/2608.22132#S1.p2.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Huanget al\.\(2025b\)Z\. Huang, W\. Sun, J\. Zhao, and K\. LiuImprove rule retrieval and reasoning with self\-induction and relevance reestimate\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 5473–5488\.Cited by:[§3\.4](https://arxiv.org/html/2608.22132#S3.SS4.SSS0.Px1.p1.1)\.
- Jinet al\.\(2025\)R\. Jin, Z\. Zhang, M\. Wang, and L\. CongStella: self\-evolving llm agent for biomedical research\.arXiv preprint arXiv:2507\.02004\.Cited by:[5th item](https://arxiv.org/html/2608.22132#A1.I2.i5.p1.1),[§1](https://arxiv.org/html/2608.22132#S1.p2.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Kimet al\.\(2024\)Y\. Kim, C\. Park, H\. Jeong, Y\. S\. Chan, X\. Xu, D\. McDuff, H\. Lee, M\. Ghassemi, C\. Breazeal, and H\. W\. ParkMdagents: an adaptive collaboration of llms for medical decision\-making\.Advances in Neural Information Processing Systems37,pp\. 79410–79452\.Cited by:[4th item](https://arxiv.org/html/2608.22132#A1.I2.i4.p1.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Kimet al\.\(2025\)Y\. Kim, Y\. Abdulle, and H\. WuBioHopR: a benchmark for multi\-hop, multi\-answer reasoning in biomedical domain\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 12894–12908\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.668),[Link](https://aclanthology.org/2025.findings-acl.668/)Cited by:[1st item](https://arxiv.org/html/2608.22132#A1.I1.i1.p1.1),[§A\.9](https://arxiv.org/html/2608.22132#A1.SS9.p2.1),[§4](https://arxiv.org/html/2608.22132#S4.p1.1)\.
- Li and Du \(2023\)R\. Li and X\. DuLeveraging structured information for explainable multi\-hop question answering and reasoning\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 6779–6789\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1)\.
- Matsumotoet al\.\(2024\)N\. Matsumoto, J\. Moran, H\. Choi, M\. E\. Hernandez, M\. Venkatesan, P\. Wang, and J\. H\. MooreKRAGEN: a knowledge graph\-enhanced rag framework for biomedical problem solving using large language models\.Bioinformatics40\(6\),pp\. btae353\.Cited by:[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px1.p1.1)\.
- McIntosh\-Smithet al\.\(2024\)S\. McIntosh\-Smith, S\. Alam, and C\. WoodsIsambard\-ai: a leadership\-class supercomputer optimised specifically for artificial intelligence\.InProceedings of the Cray User Group,pp\. 44–54\.Cited by:[Acknowledgements](https://arxiv.org/html/2608.22132#Sx2.p1.1)\.
- Menget al\.\(2024\)Z\. Meng, S\. Liu, S\. Liang, B\. Jani, and Z\. MengHeterogeneous biomedical entity representation learning for gene–disease association prediction\.Briefings in Bioinformatics25\(5\),pp\. bbae380\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1)\.
- Menget al\.\(2026\)Z\. Meng, Z\. Meng, S\. Liu, and I\. OunisTriplet\-structured knowledge integration for interactive medical question answering\.In4th Workshop on Towards Knowledgeable Foundation Models at ACL 2026,Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1)\.
- OpenAI \(2024\)OpenAIGPT\-4o system card\.Note:[https://openai\.com/index/gpt\-4o\-system\-card/](https://openai.com/index/gpt-4o-system-card/)Accessed: 2026\-05\-14Cited by:[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Parket al\.\(2025\)C\. Park, S\. Han, X\. Guo, A\. E\. Ozdaglar, K\. Zhang, and J\. KimMaporl: multi\-agent post\-co\-training for collaborative large language models with reinforcement learning\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 30215–30248\.Cited by:[§3\.4](https://arxiv.org/html/2608.22132#S3.SS4.SSS0.Px2.p1.1)\.
- Phanet al\.\(2025\)L\. Phan, A\. Gatti, Z\. Han, N\. Li, J\. Hu, H\. Zhang, S\. Shi, M\. Choi, A\. Agrawal, A\. Chopra,et al\.Humanity’s last exam\.External Links:2501\.14249,[Link](https://arxiv.org/abs/2501.14249)Cited by:[3rd item](https://arxiv.org/html/2608.22132#A1.I1.i3.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p1.1)\.
- Rezaeiet al\.\(2025\)M\. R\. Rezaei, R\. S\. Fard, J\. L\. Parker, R\. G\. Krishnan, and M\. LankaranyAgentic medical knowledge graphs enhance medical question answering: bridging the gap between LLMs and evolving medical knowledge\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 12682–12701\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.679/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.679),ISBN 979\-8\-89176\-335\-7Cited by:[3rd item](https://arxiv.org/html/2608.22132#A1.I2.i3.p1.1),[§1](https://arxiv.org/html/2608.22132#S1.p2.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.External Links:[Link](https://arxiv.org/abs/2402.03300)Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p4.1),[§3\.4](https://arxiv.org/html/2608.22132#S3.SS4.SSS0.Px2.p1.1)\.
- Sinhaet al\.\(2026\)A\. Sinha, R\. Balaji, N\. P\. Bhatt,et al\.BiCA: effective biomedical dense retrieval with citation\-aware hard negatives\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 33010–33018\.Cited by:[§A\.6](https://arxiv.org/html/2608.22132#A1.SS6.p1.1),[§3\.3](https://arxiv.org/html/2608.22132#S3.SS3.p1.2)\.
- Srivastavaet al\.\(2025\)G\. Srivastava, Z\. Bi, M\. Lu, and X\. WangDEBATE, train, evolve: self\-evolution of language model reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 32752–32798\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p2.1)\.
- Suet al\.\(2024\)X\. Su, Y\. Wang, S\. Gao, X\. Liu, V\. Giunchiglia, D\. Clevert, and M\. ZitnikKGARevion: an ai agent for knowledge\-intensive biomedical qa\.The Fourteenth International Conference on Learning Representations\.Cited by:[4th item](https://arxiv.org/html/2608.22132#A1.I2.i4.p1.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.
- Tranet al\.\(2025\)H\. Tran, Z\. Yao, Z\. Yang, J\. Wang, Y\. Zhang, S\. Han, F\. Ouyang, and H\. YuRARE: retrieval\-augmented reasoning enhancement for large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 18305–18330\.Cited by:[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px1.p1.1)\.
- Valizadeh and Parde \(2022\)M\. Valizadeh and N\. PardeThe ai doctor is in: a survey of task\-oriented dialogue systems for healthcare applications\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 6638–6660\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1)\.
- Wanget al\.\(2025\)Y\. Wang, B\. Wang, R\. Mercer, F\. Rudzicz, S\. S\. Roy, P\. Ren, Z\. Chen, and X\. WangTrustworthy medical question answering: an evaluation\-centric survey\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 27477–27490\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, M\. Bosma, V\. Zhao, K\. Guu, A\. W\. Yu, B\. Lester, N\. Du, A\. M\. Dai, and Q\. V\. LeFinetuned language models are zero\-shot learners\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=gEZrGCozdqR)Cited by:[§3\.4](https://arxiv.org/html/2608.22132#S3.SS4.SSS0.Px1.p1.1)\.
- Welblet al\.\(2018\)J\. Welbl, P\. Stenetorp, and S\. RiedelConstructing datasets for multi\-hop reading comprehension across documents\.Transactions of the Association for Computational Linguistics6,pp\. 287–302\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00021),[Link](https://aclanthology.org/Q18-1021/)Cited by:[2nd item](https://arxiv.org/html/2608.22132#A1.I1.i2.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p1.1)\.
- Wuet al\.\(2025\)J\. Wu, J\. Zhu, Y\. Qi, J\. Chen, M\. Xu, F\. Menolascina, Y\. Jin, and V\. GrauMedical graph RAG: evidence\-based medical large language model via graph retrieval\-augmented generation\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 28443–28467\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1381),ISBN 979\-8\-89176\-251\-0Cited by:[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px1.p1.1)\.
- Xianget al\.\(2025\)C\. Xiang, T\. Ma, X\. Fu, Y\. Liu, B\. Song, and X\. ZengFrom knowledge to treatment: large language model assisted biomedical concept representation for drug repurposing\.arXiv preprint arXiv:2510\.12181\.Cited by:[§1](https://arxiv.org/html/2608.22132#S1.p1.1)\.
- Xionget al\.\(2024\)G\. Xiong, Q\. Jin, X\. Wang, M\. Zhang, Z\. Lu, and A\. ZhangImproving retrieval\-augmented generation in medicine with iterative follow\-up questions\.InBiocomputing 2025: Proceedings of the Pacific Symposium,pp\. 199–214\.Cited by:[3rd item](https://arxiv.org/html/2608.22132#A1.I2.i3.p1.1),[§2](https://arxiv.org/html/2608.22132#S2.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2608.22132#S4.p2.1)\.

## Appendix AAppendix

### A\.1Computing Hardware

All experiments were conducted on a compute node equipped with sixNVIDIA GH200 Grace Hopper superchips\. This hardware provided sufficient memory and computational capacity for the training and inference settings used in this work\.

### A\.2Examples of Structured Memory

To make the structured memory design in Sec\.[3\.2](https://arxiv.org/html/2608.22132#S3.SS2)more concrete, we provide the field schema and two illustrative examples\. Table[4](https://arxiv.org/html/2608.22132#A1.T4)defines the fields used in SSE\-Bio’s short\-term structured state and long\-term template memory\. The short\-term state captures the current reasoning status for retrieval control and replanning, while the long\-term template stores reusable reasoning priors that can be retrieved and locally edited for future questions\. Figures[3](https://arxiv.org/html/2608.22132#A1.F3)and[4](https://arxiv.org/html/2608.22132#A1.F4)instantiate these definitions with concrete examples of a structured state and a fine\-grained template update\.

Figure[3](https://arxiv.org/html/2608.22132#A1.F3)shows a representative structured state used as short\-term memory during iterative reasoning\. It illustrates how theManagerorganises the task type, information gap, Critic feedback, and retrieval state into a compact representation for retrieval control and next\-step planning\.

Figure[4](https://arxiv.org/html/2608.22132#A1.F4)shows an example of long\-term template memory and its fine\-grained update after a successful trajectory\. The example highlights how SSE\-Bio keeps the task type fixed while locally editing fields such as the reasoning flow, verification criteria, tool\-use policy, and failure warning, rather than rewriting the entire template\.

Example: structured stateTask typeDisease–Protein–Drug \(multi answer\)Information gapThe disease is known, but the bridge proteins and the candidate drugs must still be verified exhaustively\.FeedbackThe Critic requires explicit disease–protein and protein–drug support before returning the final drug set\.Retrieval stateKnowledge triplets retrieved for disease–protein; one compatible template retrieved for multi\-step verification\.Figure 3:Example of the structured state used as short\-term memory for retrieval control and planning\.Example: Long\-term Template Memory and Fine\-grained UpdateTask typeDisease–Protein–Drug \(multi answer\)Reasoning flowIdentify candidate proteins associated with the disease; retrieve candidate drugs linked to those proteins…Verification criteriaEnsure each predicted drug is linked to at least one protein associated with the disease…Tool\-use policyPython for aggregation or deduplication…Failure warning \(before\)Avoid returning a partial drug set after verifying only one plausible disease\-associated protein\.Failure warning \(after\)Avoid finalising the answer before checking whether alternative disease\-associated proteins change the verified drug set\.Figure 4:Example of long\-term template memory and its fine\-grained update after a successful case\. The grey row shows the original field implicated by the failure, and the blue row shows the updated field\. Other fields remain unchanged\.MemoryFieldRoleStateutu\_\{t\}Task typetemplate retrieval queryStateutu\_\{t\}Information gaptriplet retrieval queryStateutu\_\{t\}FeedbackCritic guidance for refinementStateutu\_\{t\}Retrieval stateretrieved evidence summaryTemplatexxTask typetemplate retrieval keyTemplatexxReasoning flowreusable reasoning stepsTemplatexxVerification criteriaanswer\-checking requirementsTemplatexxTool\-use policyexternal tools available for supporting the reasoning processTemplatexxFailure warningcommon failure patternTable 4:Field schema of the structured state and template memory in SSE\-Bio\.
### A\.3Prompt and API Sensitivity

To assess whether SSE\-Bio’s gains depend on brittle prompt wording or a specific commercial API endpoint, we conduct two controlled sensitivity analyses on the BioHopR test split\.

#### Prompt sensitivity\.

We vary the system prompts of the Manager, Dev, and Critic agents across three settings: no system prompt \(Empty\), a generic prompt \(“You are a helpful biomedical assistant\.”,Basic\), and the role\-specific prompts used in SSE\-Bio\. All other components, including the user prompts, structured input and output schemas, retrieval action space, template format, stopping criterion, answer format, model APIs, and trained Proxy, are kept fixed\. As shown in Table[5](https://arxiv.org/html/2608.22132#A1.T5), replacing the role\-specific system prompts with Empty or Basic prompts causes only small drops, suggesting that SSE\-Bio is not driven by brittle system\-prompt wording\.

System PromptSingle AnswerMulti AnswerPrecH1PrecH2BothcorBothwrPrecH1PrecH2BothcorBothwrEmpty46\.3820\.7116\.0848\.6237\.6415\.6111\.2158\.04Basic46\.7220\.9616\.2748\.3137\.9115\.8711\.4457\.68SSE\-Bio47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13Table 5:Prompt\-sensitivity analysis on BioHopR\. Replacing role\-specific system prompts causes only small drops, indicating that SSE\-Bio is not driven by brittle prompt wording\.
#### API sensitivity\.

We replace the Manager and Critic backbones while keeping the Dev agent, trained Proxy, retriever, template memory, prompts, and evaluation protocol fixed\. As shown in Table[6](https://arxiv.org/html/2608.22132#A1.T6), alternative Manager/Critic APIs lead to moderate but not catastrophic degradation\.

Manager / CriticDevSingle AnswerMulti AnswerPrecH1PrecH2BothcorBothwrPrecH1PrecH2BothcorBothwrGemini\-2\.5\-ProClaude\-4\.5\-Sonnet47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13Gemini\-2\.5\-FlashClaude\-4\.5\-Sonnet45\.9620\.1415\.8649\.0837\.0215\.1211\.0458\.36GPT\-5\.5Claude\-4\.5\-Sonnet46\.5420\.8316\.1148\.5737\.6815\.7411\.3857\.91GPT\-5\.5\-MiniClaude\-4\.5\-Sonnet45\.3119\.6215\.4249\.7436\.4414\.8310\.7258\.92Table 6:API\-sensitivity analysis on BioHopR\. Alternative Manager/Critic APIs lead to moderate but not catastrophic degradation\.Together, these analyses indicate that SSE\-Bio is reasonably stable under both prompt and API variations\.

### A\.4Efficiency Details

Table[7](https://arxiv.org/html/2608.22132#A1.T7)compares SSE\-Bio with the strongest agentic baselines on the BioHopR test split in terms of total token usage and single\-answer Bothcor\. SSE\-Bio achieves the best paired\-correctness performance, reaching 16\.52 on Bothcor, compared with 10\.45 for Biomni and 9\.96 for STELLA\. This gain, however, comes with higher token usage: SSE\-Bio uses 28\.3M tokens, compared with 23\.6M for Biomni and 21\.2M for STELLA\. These results indicate that SSE\-Bio improves effectiveness substantially, while incurring additional token cost relative to the compared baselines\.

ModelTokens \(M\)BothcorBiomni23\.610\.45STELLA21\.29\.96SSE\-Bio28\.316\.52Table 7:Efficiency comparison on the BioHopR test split\.Beyond token usage, Table[8](https://arxiv.org/html/2608.22132#A1.T8)reports end\-to\-end per\-example inference cost, including all Manager, Dev, Critic, retrieval, and local Proxy operations, and Table[9](https://arxiv.org/html/2608.22132#A1.T9)reports the one\-time Proxy training cost\.

MethodTokens/ Ex\.LLM Calls/ Ex\.Latency/ Ex\.API Cost/ Ex\.Biomni10\.3K5\.818\.4s$0\.039STELLA9\.3K4\.916\.7s$0\.035SSE\-Bio12\.4K6\.722\.1s$0\.047Table 8:Per\-example inference cost on BioHopR\. SSE\-Bio adds moderate overhead relative to STELLA \(3\.1K tokens, 1\.8 calls, 5\.4s, and approximately $0\.012 per example\) while substantially improving paired correctness\. The local Proxy adds no commercial API calls; its latency is included in the end\-to\-end measurement\.StageTokensData / RolloutsGPU\-hProxy SFT41\.6M12\.6K labelled pairs42Proxy GRPO132\.4M38\.5K rollouts96Table 9:One\-time Proxy training cost \(138 GPU\-hours in total\), not incurred during test\-time inference\.![Refer to caption](https://arxiv.org/html/2608.22132v1/triplet_retrieval_budget.png)Figure 5:Effect of the triplet retrieval budget on BioHopR\.![Refer to caption](https://arxiv.org/html/2608.22132v1/template_retrieval_budget.png)Figure 6:Effect of the template retrieval budget on BioHopR\.
### A\.5Implementation Details

SSE\-Bio uses a four\-agent architecture consisting of a Manager, a Proxy, a Dev agent, and a Critic\. The Manager and Critic are implemented with Gemini\-2\.5\-Pro, and the Dev agent uses Claude\-4\.5\-Sonnet\. These fixed commercial agents execute candidate rollouts and provide critic feedback for both SFT data construction and GRPO trajectory groups\. The Proxy is the only trainable component and is instantiated with open\-weight backbones, including Llama\-3\.1\-8B/70B and Qwen\-2\.5\-7B/72B\. For compactness, result tables abbreviate Qwen\-2\.5\-7B/72B as Qwen\-8B/72B\. We perform full\-parameter fine\-tuning for all Proxy backbones, including Qwen\-2\.5\-72B, using the hardware described in Appendix[A\.1](https://arxiv.org/html/2608.22132#A1.SS1)\. Unless otherwise stated, reported SSE\-Bio results use the Qwen\-2\.5\-72B Proxy\.

#### Potential risks and intended use\.

Although SSE\-Bio is evaluated only on benchmark biomedical reasoning tasks, incorrect answers or unsupported reasoning could be harmful if the system were used directly in biomedical or clinical decision\-making\. SSE\-Bio should therefore be treated as a research system for evidence\-grounded reasoning evaluation rather than as a tool for clinical diagnosis, treatment recommendation, or patient\-facing deployment\.

#### Artifact licences and intended use\.

We use publicly available datasets and models in accordance with their stated research\-use terms and licences\. BioHopR, MedHop, Humanity’s Last Exam: Biomedicine, BiCA, Llama, Qwen, and the compared baseline systems are used only for research evaluation\. Commercial LLM APIs, including Gemini\-2\.5\-Pro and Claude\-4\.5\-Sonnet, are used according to their provider terms of service\. The released SSE\-Bio code will be distributed for research use\.

HyperparameterValueProxy backboneQwen\-2\.5\-72BSFT learning rate1×10−51\\times 10^\{\-5\}SFT batch size64SFT max length2048SFT \# epochs1GRPO learning rate5×10−75\\times 10^\{\-7\}GRPO KL coefficient0\.002GRPO batch size1024GRPO max length2048GRPO LR schedulerCosineReward weights\(λo,λb\)\(\\lambda\_\{o\},\\lambda\_\{b\}\)\(0\.8,0\.2\)\(0\.8,\\,0\.2\)Retained trajectory capKK8Triplet retrieval budgetkKGk\_\{\\mathrm\{KG\}\}5Template retrieval budgetkTPLk\_\{\\mathrm\{TPL\}\}1Template edit thresholdθtpl\\theta\_\{\\mathrm\{tpl\}\}0\.75BioHopR matching thresholdτ\\tau0\.9Table 10:Key training and retrieval hyperparameters of SSE\-Bio\.

### A\.6Agentic Retrieval

For KG retrieval, SSE\-Bio uses theinformation gapfield in the current stateutu\_\{t\}as the query\. Candidate biomedical triplets are linearised as head–relation–tail records with entity aliases, encoded with BiCA[26](https://arxiv.org/html/2608.22132#bib.bib11), and ranked by cosine similarity\. SSE\-Bio then removes schema\-incompatible results and duplicate entity pairs, retaining the top\-55triplets for the Manager\. For template retrieval, SSE\-Bio uses thetask typefield inutu\_\{t\}as the query and retrieves the top\-1 compatible template fromℰ\\mathcal\{E\}\. Compatibility is determined by dense BiCA similarity plus symbolic schema and answer\-cardinality checks:

stpl​\(ut,x\)=0\.60​ctpl\+0\.25​mschema\+0\.15​mans,s\_\{\\mathrm\{tpl\}\}\(u\_\{t\},x\)=0\.60\\,c\_\{\\mathrm\{tpl\}\}\+0\.25\\,m\_\{\\mathrm\{schema\}\}\+0\.15\\,m\_\{\\mathrm\{ans\}\},\(8\)wherectpl=cos⁡\(ϕ⁡\(ψ⁡\(ut\)\),ϕ⁡\(ψ⁡\(x\)\)\)c\_\{\\mathrm\{tpl\}\}=\\cos\(\\phi\(\\psi\(u\_\{t\}\)\),\\phi\(\\psi\(x\)\)\),ψ⁡\(⋅\)\\psi\(\\cdot\)extracts the task\-type descriptor,mschemam\_\{\\mathrm\{schema\}\}indicates relation\-schema compatibility, andmansm\_\{\\mathrm\{ans\}\}indicates whether the template has the same answer cardinality\. We conduct a threshold\-sensitivity analysis overθtpl∈\{0\.65,0\.70,0\.75,0\.80\}\\theta\_\{\\mathrm\{tpl\}\}\\in\\\{0\.65,0\.70,0\.75,0\.80\\\}using training rollouts, and setθtpl=0\.75\\theta\_\{\\mathrm\{tpl\}\}=0\.75as a conservative threshold to avoid merging trajectories with only superficial task overlap\. A successful trajectory is aligned with the top retrieved template only if the task type matches andstpl≥θtpls\_\{\\mathrm\{tpl\}\}\\geq\\theta\_\{\\mathrm\{tpl\}\}; otherwise SSE\-Bio distils a new template\. Fine\-grained editing changes only thereasoning flow,verification criteria,tool\-use policy, and/orfailure warningfields identified by the Critic, while keeping the task type fixed\.

We further study the sensitivity of SSE\-Bio to the retrieval budgets for biomedical triplets and prior templates using training rollouts\. As shown in Fig\.[5](https://arxiv.org/html/2608.22132#A1.F5), performance peaks when retrievingkKG=5k\_\{\\mathrm\{KG\}\}=5triplets, while larger triplet budgets introduce more redundant evidence and slightly reduce single\-answer Bothcor\. As shown in Fig\.[6](https://arxiv.org/html/2608.22132#A1.F6), retrieving a single template performs best, with performance remaining relatively stable across larger budgets\. These results support the use of a small retrieval budget in SSE\-Bio\. For biomedical triplets, retrieving a moderate number of facts provides sufficient evidence without overloading the reasoning context\. For templates, a single compatible template is most effective, which is consistent with the role of template memory as a planning prior rather than an evidence pool\.

StatisticValueInterpretationInitial persistent templates0The memory starts empty; no held\-out BioHopR, MedHop, or HLE examples are used to seed templates\.Accepted post\-success updates2\.47KTraining\-only updates accepted after successful trajectories and critic validation\.New\-template distillations612Updates whose task type does not align with an existing template aboveθtpl\\theta\_\{\\mathrm\{tpl\}\}\.Fine\-grained edit operations1\.86KUpdates applied to aligned templates without changing the task type\.Final deduplicated templates512Compact memory checkpoint used for all held\-out evaluations\.Covered task types24Twelve BioHopR relation schemas under single\-answer and multi\-answer variants\.Templates per task type21\.3 / 19\.0Mean / median number of templates per covered task type\.Edited templates37473\.0% of final templates receive at least one fine\-grained edit\.Fields touched per edit1\.42Average number of non\-task\-type fields modified per accepted edit\.Reasoning\-flow edits31\.8%Edits that revise bridge\-resolution or answer\-finalisation steps\.Verification\-criteria edits28\.5%Edits that strengthen hop\-level grounding or answer\-coverage checks\.Tool\-use\-policy edits22\.4%Edits that change when triplet retrieval, template retrieval, or entity normalisation is invoked\.Failure\-warning edits17\.3%Edits that add local failure triggers and recovery actions\.Table 11:Template\-memory construction and evolution statistics on the BioHopR training split\. Counts are rounded after deduplication and memory validation\.
### A\.7Proxy Policy Training

For GRPO training, each retained trajectoryτ\\taureceives the reward

R⁡\(τ\)=0\.80​Rout​\(τ\)\+0\.20​Rbeh​\(τ\),R\(\\tau\)=0\.80R\_\{\\mathrm\{out\}\}\(\\tau\)\+0\.20R\_\{\\mathrm\{beh\}\}\(\\tau\),\(9\)whereRout​\(τ\)∈\{0,1\}R\_\{\\mathrm\{out\}\}\(\\tau\)\\in\\\{0,1\\\}is a binary outcome reward awarded only at the conclusion of a trajectory\. For BioHopR, it is 1 only when the linked 1\-hop and 2\-hop answers both satisfy the benchmark answer\-matching criterion; otherwise it is 0\. The behavioural termRbeh​\(τ\)∈\[0,1\]R\_\{\\mathrm\{beh\}\}\(\\tau\)\\in\[0,1\]measures the fraction of required reasoning steps whose bridge or answer claim is explicitly supported by selected triplets or by a retrieved template step accepted by the Critic\. This design keeps the main training signal aligned with final QA correctness while using evidence support only as a dense tie\-breaker among trajectories with similar outcomes\.

Step typeSupported ifNot supported ifBridge resolutionA retrieved triplet or accepted template step links the query entity to the proposed bridge with the required relation type\.The bridge is introduced only from parametric generation or by a triplet with an incompatible schema\.Answer verificationThe final answer is linked to the resolved bridge by a retrieved triplet, or satisfies the retrieved template’s verification criteria\.The answer is plausible but lacks explicit bridge\-to\-answer evidence\.Multi\-answer coverageEvery returned entity has support, and the state records that candidate expansion has been exhausted within the retrieval budget\.The trajectory stops after finding only one supported entity when the task type requires a set\.No\-retrieval branchThe existing retrieval state already contains the evidence needed by the current plan\.The branch avoids retrieval while the information gap remains unresolved\.Table 12:Evidence\-support rules used forRbehR\_\{\\mathrm\{beh\}\}\. These rules operationalise the behavioural reward in Sec\.[3\.4](https://arxiv.org/html/2608.22132#S3.SS4)without rewarding retrieval volume by itself\.SFT labels are constructed from the same answer\-supervised rollout pool used for GRPO group construction\. For each retained structured state, we compare the four retrieval actions by the composite reward above and choose the highest\-reward action as the pseudo\-label\. If multiple actions obtain the same outcome reward, we break ties by evidence support and then by retrieval parsimony, preferring fewer sources when the additional source does not improve support\. GRPO uses all retained branches in the decision\-contrastive trajectory group rather than only the best branch\.

StatisticValueInterpretationBioHopR training pairs5\.34KTraining split used for trajectory construction and persistent template\-memory updates\.Candidate rollouts before pruning82\.4KMean 15\.4 rollouts per pair, close to the42=164^\{2\}=16upper bound from expanding the first two retrieval decisions\.Valid parsed rollouts76\.8K93\.2% of generated rollouts have valid state, action, evidence, and answer fields\.Evidence\-bearing non\-duplicate rollouts49\.7KRemaining rollouts after removing duplicate state\-action paths and branches with empty selected evidence\.Retained GRPO rollouts38\.5KMean group sizeK=7\.21K=7\.21after theK≤8K\\leq 8cap; median group size is 8\.0\.Discarded or pruned rollouts43\.9K53\.3% of generated rollouts are removed by parsing, duplicate/evidence filters, or the group\-size cap\.Gold\-answer successful retained rollouts17\.3KRetained rollouts whose final paired answer satisfies the BioHopR answer\-matching criterion\.SFT\-labelled state\-action pairs12\.6KMean 2\.36 labelled Proxy decisions per training pair; states without a successful answer\-supervised rollout are excluded from SFT labels\.GRPO state\-action pairs38\.5KOne retained state\-action pair is associated with each rollout branch in the decision\-contrastive trajectory groups\.SFT label: triplets \+ templates43\.8%Most labelled decisions require both factual evidence and a reusable reasoning prior\.SFT label: triplets only33\.6%Used when the current state mainly lacks biomedical bridge evidence\.SFT label: templates only15\.1%Used when planning structure is useful but extra triplets are not needed\.SFT label: neither source7\.5%Used when the current state already contains enough supported evidence to proceed\.Table 13:Construction statistics for SFT pseudo\-labels and GRPO trajectory groups on the BioHopR training split\. Counts are rounded; each training pair seeds one decision\-contrastive rollout trajectory generation\.
### A\.8Datasets

We evaluate SSE\-Bio on three biomedical reasoning benchmarks:

- •BioHopR[15](https://arxiv.org/html/2608.22132#bib.bib32)is a biomedical benchmark for multi\-hop, multi\-answer reasoning over structured knowledge graphs\. Built from PrimeKG, it contains linked 1\-hop and 2\-hop questions designed to reflect one\-to\-many and many\-to\-many biomedical relations\. The dataset contains 7\.63K paired instances, and we use a 7:3 train/test split, corresponding to approximately 5\.34K training pairs and 2\.29K test pairs\.
- •MedHop[33](https://arxiv.org/html/2608.22132#bib.bib33)is a cross\-document biomedical question answering dataset that requires models to combine evidence distributed across multiple documents in order to infer the correct answer\. It was originally introduced to study multi\-hop reading comprehension in the biomedical domain\.
- •Humanity’s Last Exam: Biomedicine[23](https://arxiv.org/html/2608.22132#bib.bib34)is a challenging subset of Humanity’s Last Exam, a frontier\-level benchmark designed to test closed\-ended expert reasoning on difficult academic questions\. HLE contains multiple\-choice and short\-answer questions with unambiguous, verifiable solutions, and the Biomedicine subset provides a demanding zero\-shot setting for biomedical reasoning\.

SSE\-Bio is trained only on the BioHopR training split; the BioHopR test split, MedHop, and HLE: Biomedicine are never used for proxy tuning or template\-memory construction\.

### A\.9Evaluation

For BioHopR, we reportPrecH1,PrecH2,Bothcor, andBothwr\. PrecH1and PrecH2evaluate performance on the linked 1\-hop and 2\-hop questions, respectively, while Bothcorand Bothwrmeasure the proportions of paired instances for which both questions are correct or both are wrong\.

Following BioHopR[15](https://arxiv.org/html/2608.22132#bib.bib32), answer matching first applies exact normalised entity matching\. Unmatched predictions are then evaluated with embedding\-based precision\. Letppdenote the embedding of a predicted response and\{ai\}i=1n\\\{a\_\{i\}\\\}\_\{i=1\}^\{n\}denote the embeddings of the gold answers\. The cosine similarity betweenppand a gold answer embeddingaia\_\{i\}is defined as

cos⁡\(p,ai\)=p⋅ai‖p‖​‖ai‖\.\\cos\(p,a\_\{i\}\)=\\frac\{p\\cdot a\_\{i\}\}\{\\\|p\\\|\\,\\\|a\_\{i\}\\\|\}\.\(10\)A predicted response is counted as correct if

maxi∈\{1,…,n\}⁡cos⁡\(p,ai\)\>τ,\\max\_\{i\\in\\\{1,\\dots,n\\\}\}\\cos\(p,a\_\{i\}\)\>\\tau,\(11\)whereτ\\tauis the cosine threshold specified in Table[10](https://arxiv.org/html/2608.22132#A1.T10)\. Precision is then computed as

Prec=\|True​Positives\|\|Predicted​Responses\|\.\\mathrm\{Prec\}=\\frac\{\|\\mathrm\{True\\ Positives\}\|\}\{\|\\mathrm\{Predicted\\ Responses\}\|\}\.\(12\)
For MedHop and HLE: Biomedicine, we report accuracy\.

### A\.10Baselines

The main experiments compare SSE\-Bio with five baseline groups: general\-purpose LLMs, medical LLMs, retrieval\-augmented generation systems, biomedical agent systems, and self\-evolving agents\. General\-purpose and medical LLM baselines use the same question format and answer normalisation as SSE\-Bio, but do not use SSE\-Bio’s structured memory or retrieval control\. RAG baselines retrieve biomedical evidence before generation, but do not train a state\-conditioned retrieval controller\. Biomedical agent systems decompose the task or invoke biomedical tools, while self\-evolving baselines update templates from previous trajectories\. Biomni serves as the strongest biomedical agent baseline, and STELLA serves as the strongest self\-evolving agent baseline\.

- •General\-purpose LLMs:Llama\-3\.1\-8B, Llama\-3\.1\-70B and GPT\-4o\. These models serve as strong general\-purpose language\-model baselines without biomedical specialisation[9](https://arxiv.org/html/2608.22132#bib.bib25)\.
- •Medical LLMs:HuatuoGPT\-o1\-70B, HuatuoGPT\-o1\-8B, and UltraMedical\-8B\. These models are domain\-adapted medical or biomedical language models designed for specialised biomedical reasoning[3](https://arxiv.org/html/2608.22132#bib.bib13)\.
- •Retrieval\-augmented generation methods:i\-MedRAG and AMG\-RAG\. These methods combine biomedical question answering with external knowledge retrieval during inference[36](https://arxiv.org/html/2608.22132#bib.bib17);[24](https://arxiv.org/html/2608.22132#bib.bib7)\.
- •Biomedical agent systems:KGARevion, DoctorAgent\-RL, MedAgents, and Biomni\. These systems use agentic decomposition, collaboration, or external biomedical tools to improve medical and biomedical reasoning[28](https://arxiv.org/html/2608.22132#bib.bib23);[7](https://arxiv.org/html/2608.22132#bib.bib29);[14](https://arxiv.org/html/2608.22132#bib.bib28);[11](https://arxiv.org/html/2608.22132#bib.bib35)\.
- •Self\-evolving agents:STELLA\. This model improves reasoning behaviour over time through memory\- or workflow\-based evolution, and provides the most direct comparison for SSE\-Bio[13](https://arxiv.org/html/2608.22132#bib.bib14)\.

Retrieval PolicySingle AnswerMulti AnswerPrecH1PrecH2BothcorBothwrPrecH1PrecH2BothcorBothwrRetrieve neither39\.7415\.8810\.6255\.0030\.8111\.927\.0564\.32Retrieve KG only42\.3617\.4112\.4352\.8433\.1813\.068\.5262\.37Retrieve template only41\.5816\.9611\.8653\.4632\.4712\.648\.0762\.98Retrieve KG \+ template43\.2718\.1213\.1851\.7634\.0213\.589\.0261\.31Learned Proxy47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13Table 14:Comparison of fixed retrieval strategies on BioHopR\. The fixed strategies use the same Manager, Dev, Critic, retriever, and retrieval budget as SSE\-Bio, but replace the trainable Proxy with a fixed retrieval decision at each reasoning round\. “Retrieve neither” corresponds to the w/o proxy setting in Table[2](https://arxiv.org/html/2608.22132#S4.T2)\.SettingSingle AnswerMulti AnswerPrecH1PrecH2BothcorBothwrPrecH1PrecH2BothcorBothwrTop\-5 retrieved triplets47\.2121\.3816\.5247\.9338\.3616\.2411\.7357\.13\+ 2 random noisy triplets46\.3220\.7616\.0248\.6437\.5815\.6811\.3157\.86Table 15:Noise\-injection analysis on BioHopR\. Adding two irrelevant triplets reduces single\-/multi\-answer Bothcorby only 0\.50/0\.42 points, indicating reasonable robustness under this controlled perturbation\.
### A\.11Proxy Training and Memory Construction Statistics

Table[11](https://arxiv.org/html/2608.22132#A1.T11)summarizes the construction and evolution of the persistent template memory\. This memory starts empty and is updated from successful BioHopR training trajectories only after validation by reviewers\. Most accepted updates consist of fine\-tuning aligned templates, while new template distillation is employed when no compatible templates are found\. After deduplication, the final memory remains compact and covers variants of all BioHopR task types, with adjustments distributed across the inference process, validation criteria, tool usage strategies, and failure warnings\. These statistics support the view that SSE\-Bio evolves its memory through local field\-level updates rather than global prompt\-level rewrites\.

Table[13](https://arxiv.org/html/2608.22132#A1.T13)reports the construction statistics for the SFT pseudo\-labels and GRPO trajectory sets generated on the BioHopR training set\. Each training pair provides initial conditions for decision\-contrastive rolling generation by expanding alternative retrieval actions in the matched inference state\. After syntactic analysis, repetition/evidence filtering, and trajectory pruning, the retained rolling results provide both state\-action pairs with SFT labels and GRPO trajectory sets for agent optimization\. The resulting SFT label distribution covers all four retrieval operations: simultaneous retrieval of triples and templates, retrieval of triples only, retrieval of templates only, and no retrieval from either source\. This indicates that the agent model was not trained to follow a simplified, fixed retrieval strategy, but rather was supervised to make diverse retrieval decisions\.

### A\.12Robustness to Noisy Retrieval

To directly test robustness to noisy evidence, whenever the Proxy selects triplet retrieval, we preserve the top\-five retrieved triplets and inject two random irrelevant triplets\. Table[15](https://arxiv.org/html/2608.22132#A1.T15)shows that this perturbation reduces single\-/multi\-answer Bothcorby only 0\.50/0\.42 points: SSE\-Bio is affected by noisy evidence, but performance does not collapse, indicating reasonable robustness under this controlled perturbation\.

### A\.13Additional Analysis of Retrieval Control and Memory Evolution

#### Does the Proxy outperform fixed retrieval schedules?

Table[14](https://arxiv.org/html/2608.22132#A1.T14)compares the learned Proxy with fixed retrieval policies under the same agent backbone and retrieval budget\. Retrieving neither source corresponds to removing the Proxy and external retrieval, which substantially reduces performance\. Fixed retrieval from a single source improves over this setting, showing that both knowledge triplets and prior templates are useful\. However, always retrieving both sources remains clearly below the learned Proxy, with single\-answer Bothcordecreasing from 16\.52 to 13\.18 and multi\-answer Bothcordecreasing from 11\.73 to 9\.02\. These results indicate that SSE\-Bio’s improvement is not simply due to retrieving more context; rather, training the Proxy enables state\-dependent source selection that better matches the current reasoning state\.

MetricCoarserewritingFine\-grainededitingChanged fields / update↓\\downarrow3\.281\.46Unchanged\-field preservation↑\\uparrow64\.088\.0Unsupported new instruction↓\\downarrow12\.04\.0Contradictory instruction↓\\downarrow7\.02\.0Table 16:Template\-update quality analysis on 100 randomly sampled memory updates\. Rates are percentages except for changed fields per update\.
#### Does fine\-grained editing make memory evolution more local?

For the template\-update quality analysis, we randomly sample 100 memory updates and compare coarse rewriting with fine\-grained editing under the same template schema\.Changed fields per updatecounts the average number of template fields modified by each update\.Unchanged\-field preservation ratemeasures the proportion of fields that should remain unchanged and are indeed preserved\.Unsupported new instruction ratemeasures the proportion of updates that introduce at least one new instruction not supported by the successful trajectory, retrieved evidence, or Critic feedback\.Contradictory instruction ratemeasures the proportion of updates that introduce at least one instruction conflicting with the original template, task schema, or verification criteria\. All rates are computed at the update level over the 100 sampled updates\.

Table[16](https://arxiv.org/html/2608.22132#A1.T16)shows that fine\-grained editing modifies fewer fields per update than coarse rewriting, reducing the average number of changed fields from 3\.28 to 1\.46\. It also preserves unrelated fields more often, improving the unchanged\-field preservation rate from 64\.0% to 88\.0%\. Beyond locality, fine\-grained editing introduces fewer unsupported new instructions and contradictory instructions, with rates decreasing from 12\.0% to 4\.0% and from 7\.0% to 2\.0%, respectively\. These results support a more precise version of our claim: fine\-grained editing makes template evolution more localised and easier to audit\. We therefore do not claim that editing eliminates hallucinations or instruction drift; rather, the inspected cases suggest that it reduces unsupported or inconsistent template updates\.

### A\.14Error Analysis

We conduct a manual audit of failed HLE: Biomedicine cases and categorise the primary error source of each failure in Table[17](https://arxiv.org/html/2608.22132#A1.T17)\.

Error SourceShare \(%\)Dev reasoning error31\.2Triplet retrieval failure24\.6Wrong Proxy action13\.1Answer\-format / coverage error11\.4Template mismatch10\.8Critic feedback error8\.9Table 17:Distribution of primary error sources over failed HLE: Biomedicine cases\.Dev reasoning errors \(31\.2%\), where relevant evidence is available but the Dev agent makes an incorrect inference, and triplet retrieval failures \(24\.6%\), where a key entity or relation is missing from the retrieved evidence, are the two largest remaining sources\. This indicates that better retrieval alone will not resolve the remaining errors; stronger evidence\-conditioned reasoning and verification are also needed\.

### A\.15Case Study: Failure Analysis against STELLA

ItemSTELLASSE\-BioQuestion patternDrug→\\rightarrowGene/Protein→\\rightarrowDiseaseDrug→\\rightarrowGene/Protein→\\rightarrowDiseaseQuestionName a disease that is related to a gene/protein that is associated with drug \(2S\)\-2\-\{\[HYDROXY\(4\-IODOBENZYL\)PHOSPHORYL\]METHYL\}PENTANEDIOIC ACID\.Gold bridgeFOLH1Gold answer\{neurotic disorder, colorectal neoplasm, prostate cancer, schizophrenia, dysthymic disorder, colorectal cancer, familial prostate carcinoma, colorectal carcinoma, unipolar depression, anxiety disorder, prostate carcinoma\}Predicted bridgeARFOLH1Retrieved evidenceRetrieves disease\-associated context around prostate cancer and androgen\-receptor signalling, but does not establish that the query drug is associated with AR\.Retrieves relation\-specific triplets supporting both hops:\(\(2S\)\-2\-PENTANEDIOIC ACID, associated with, FOLH1\)and\(FOLH1, related to, prostate cancer\)\.Memory/template behaviourCoarse rewriting updates multiple fields, including reasoning flow and tool\-use policy, and shifts the template towards a broad disease\-association heuristic\.Local editing preserves the task type and reasoning flow, and modifies only the verification\-criteria field to require both the drug–gene/protein and gene/protein–disease links to be evidence\-supported\.Final answerProstate carcinomaProstate cancerFailure/success reasonThe answer is semantically close to a gold disease, but the trajectory is not valid because the selected bridge AR is not supported as the gene/protein associated with the query drug\.The answer is verified through the required two\-hop path: the query drug is associated with FOLH1, and FOLH1 is related to prostate cancer\.Table 18:A BioHopR case study comparing STELLA and SSE\-Bio\.Table[18](https://arxiv.org/html/2608.22132#A1.T18)illustrates a typical BioHopR case featuring a “drug→\\rightarrowgene/protein→\\rightarrowdisease” relationship pattern\. The golden intermediate entity is FOLH1, and the answer should be validated via a two\-hop path from the query drug to FOLH1, and then from FOLH1 to a specific disease\. The disease returned by STELLA is semantically close to the gold answer set, but its reasoning trail relies on AR as an intermediate bridge and does not verify whether the query drug is associated with that bridge\. This means that even if the final disease appears reasonable, the reasoning trail is still considered invalid\. In contrast, SSE\-Bio retrieves relationship\-specific triples, preserves the task type and reasoning process, and performs only local edits to the verification criteria fields\. This requires that the final answer be supported by both the “drug\-gene/protein” and “gene/protein\-disease” links, enabling SSE\-Bio to return a valid answer via the FOLH1 bridge\.