基于指南增强的多智能体框架实现 schema-as-code 生物医学命名实体识别

# A Guideline-Augmented Multi-Agent Framework for Schema-as-Code Biomedical Named Entity Recognition
Source: [https://arxiv.org/html/2610.02970](https://arxiv.org/html/2610.02970)
Songtao Li1, Yijia Zhang1, Shidi Zhang1, Jianyuan Yuan2, Fengyu Zhang3, Hongfei Lin4Affiliation:1Dalian Maritime University2Beijing Institute of Technology\{devin\_lee, zhangyijia, zhangshidi\}@dlmu\.edu\.cnjianyuany@bit\.edu\.cn3Northeastern University4Dalian University of Technologyfengyu\.felix@gmail\.comhflin@dlut\.edu\.cn

###### Abstract

Large language models \(LLMs\) have shown promising potential for biomedical named entity recognition \(BioNER\) through instruction following and in\-context learning\. However, existing LLM\-based BioNER methods still face two key limitations\. First, retrieved demonstrations and external biomedical knowledge provide limited support for dataset\-specific annotation semantics, leaving entity boundaries, type scopes, and annotation conventions ambiguous\. Second, free\-form generation lacks sufficient structural control, often leading to invalid formats, hallucinated mentions, duplicated entities, and boundary errors\. To address these limitations, we propose GAMA, a guideline\-augmented multi\-agent framework for schema\-as\-code BioNER\. GAMA first induces candidate annotation rules from labeled training instances and verifies them against annotated data to construct reliable dataset\-specific guideline memory\. Guided by these verified rules, a planning component generates ranked span\-type hypotheses with rationales, and a coding component converts them into schema\-constrained entity objects\. A verification module then checks span grounding, type validity, and structural compliance, and performs dual\-loop refinement to correct invalid or low\-confidence predictions\. Experiments on five widely used BioNER datasets with multiple LLM backbones show that GAMA consistently outperforms strong LLM\-based baselines\. Ablation and parameter analyses further verify the effectiveness of the proposed components\.

###### Index Terms:

biomedical named entity recognition, large language models, in\-context learning, multi\-agent

## IIntroduction

Biomedical named entity recognition \(BioNER\) aims to identify domain\-specific entity mentions, such as diseases, chemicals, genes, proteins, and anatomical entities\[[1](https://arxiv.org/html/2610.02970#bib.bib1)\]\. As a core task in biomedical information extraction, BioNER supports many downstream applications, including biomedical relation extraction\[[2](https://arxiv.org/html/2610.02970#bib.bib2)\]and biomedical event extraction\[[3](https://arxiv.org/html/2610.02970#bib.bib3)\]\. Supervised neural models and domain\-specific pretrained language models such as BioBERT\[[4](https://arxiv.org/html/2610.02970#bib.bib4)\]and PubMedBERT\[[5](https://arxiv.org/html/2610.02970#bib.bib5)\]have achieved substantial progress\. Nevertheless, these approaches require large\-scale manually annotated corpora, which are costly due to the need for both linguistic expertise and domain knowledge\.

Large language models \(LLMs\) offer a promising alternative, as they can perform downstream tasks using instructions and a few in\-context examples without additional training\[[6](https://arxiv.org/html/2610.02970#bib.bib6)\]\. Recent studies have mainly explored LLM\-based BioNER from two perspectives: \(i\) retrieving similar annotated examples as in\-context demonstrations to provide instance\-level evidence\[[7](https://arxiv.org/html/2610.02970#bib.bib7),[8](https://arxiv.org/html/2610.02970#bib.bib8)\], and \(ii\) incorporating external biomedical knowledge, such as entity semantics or UMLS information, to improve domain understanding\[[9](https://arxiv.org/html/2610.02970#bib.bib9),[10](https://arxiv.org/html/2610.02970#bib.bib10)\]\. Despite their effectiveness, these approaches still face two major limitations\. First, they provide limited support for dataset\-specific annotation semantics\. Retrieved examples usually reflect local annotation decisions, while external knowledge mainly provides general biomedical information\. As a result, dataset\-specific entity boundaries, type scopes, and annotation conventions remain ambiguous, especially when entity definitions and annotation standards vary across datasets\. Second, structural control remains insufficient\. Since BioNER requires each predicted entity to be grounded in the input sentence, assigned to a predefined type, and returned in a valid format, free\-form LLM generation may produce invalid outputs, duplicated entities, hallucinated mentions, or boundary errors, which ordinary prompting cannot fully prevent\.

To address these limitations, we present GAMA, a guideline\-augmented multi\-agent framework for schema\-as\-code BioNER\. Rather than prompting a single LLM to directly produce entity predictions, GAMA decomposes the extraction process into five specialized components: Annotation Rule Induction, Rule Verification, Planning, Coding, and Verification with Dual\-Loop Refinement, each responsible for a distinct stage of the task\. GAMA is built upon three core principles: \(1\)Explicit Guideline Modeling, where the Annotation Rule Induction Agent summarizes dataset\-specific annotation rules from training instances, and the Rule Verification Agent filters unreliable or overly instance\-specific rules to construct explicit and reusable guideline memory; \(2\)Multi\-Agent Decomposition, in which the Planning Agent generates mention\-type hypotheses with explanatory rationales, ranks them according to their consistency with the input sentence, the predefined label schema, and the verified guideline rules, and the Coding Agent converts the most reliable hypotheses into schema\-constrained code\-style entity objects; \(3\)Schema\-as\-Code Verification, where each predicted entity object is validated for span grounding, type correctness, and structural validity, while a dual\-loop refinement procedure corrects local object\-level errors or revisits lower\-confidence span–type hypotheses when necessary\.

Our main contributions are summarized as follows:

- •We introduce a guideline modeling mechanism that induces and verifies dataset\-specific annotation rules, providing explicit dataset\-level guidance for entity boundaries, type scopes, and annotation conventions\.
- •We propose GAMA, a guideline\-augmented multi\-agent framework that decomposes BioNER into rule induction, rule verification, guideline\-guided planning, schema\-constrained coding, and prediction verification\.
- •We develop a schema\-as\-code verification mechanism with dual\-loop refinement, which checks span grounding, type validity, and structural compliance to reduce invalid, hallucinated, and inconsistent predictions\.
- •We conduct experiments on five widely used BioNER datasets with multiple LLM backbones\. The results show that GAMA consistently outperforms strong LLM\-based baselines, and ablation studies further verify the effectiveness of its key components\.

## IIRelated Work

### II\-ABioNER with In\-Context Learning

BioNER remains challenging due to domain\-specific terminology, complex entity boundaries, and annotation ambiguity\. Traditional supervised approaches often require large\-scale annotated corpora, which are expensive and time\-consuming to construct in the biomedical domain\. To reduce such annotation dependence, recent studies have explored in\-context learning \(ICL\) with LLMs, where models perform entity extraction by conditioning on natural language instructions and a few contextual examples without updating model parameters\. Early work focused on improving the content of prompts to enhance LLMs’ understanding of the task\. For instance, Hu et al\.\[[11](https://arxiv.org/html/2610.02970#bib.bib11)\]designed a clinical NER prompt framework by incorporating annotation guidelines, error\-analysis\-based instructions, and few\-shot examples, while Ge et al\.\[[7](https://arxiv.org/html/2610.02970#bib.bib7)\]further investigated structured static prompts that integrate dataset descriptions, high\-frequency entities, UMLS knowledge, and error feedback, demonstrating that careful prompt construction can substantially improve few\-shot BioNER performance\. Beyond prompt content, other studies aimed to strengthen LLMs’ domain awareness and reasoning capabilities\. Bian et al\.\[[9](https://arxiv.org/html/2610.02970#bib.bib9)\]reformulated BioNER as a two\-step process of span extraction followed by entity type prediction, augmented with UMLS\-based external knowledge to support accurate type determination\. Similarly, Li et al\.\[[12](https://arxiv.org/html/2610.02970#bib.bib12)\]introduced the RT framework, which leverages retrieved demonstrations combined with Chain\-of\-Thought reasoning, guiding LLMs to gradually recognize entities with higher accuracy\. These works highlight that effective BioNER often requires domain\-specific knowledge and reasoning beyond simple text generation\. More recently, retrieval\-augmented demonstration selection has been explored to improve example selection in ICL\. EICL\[[8](https://arxiv.org/html/2610.02970#bib.bib8)\]introduces an entity\-aware retrieval strategy that combines semantic similarity with domain\-specific terminology overlap to select more relevant in\-context examples for BioNER\.

### II\-BMulti\-Agent Systems in Information Extraction

Recent advances in large language models have promoted the use of multi\-agent collaboration for information extraction, where different agents are assigned specialized roles to improve structured prediction through cooperation, refinement, and verification\. Wang et al\.\[[13](https://arxiv.org/html/2610.02970#bib.bib13)\]proposed CMAS, which employs a self\-annotator, a type\-related feature extractor, a demonstration discriminator, and an overall predictor to construct and filter self\-annotated demonstrations for zero\-shot named entity recognition\. Lu et al\.\[[14](https://arxiv.org/html/2610.02970#bib.bib14)\]introduced CROSSAGENTIE, a cross\-type and cross\-task multi\-agent framework for zero\-shot information extraction, where multiple agents collaborate to improve structured prediction across named entity recognition and relation extraction\. Zhang et al\.\[[15](https://arxiv.org/html/2610.02970#bib.bib15)\]developed a generation\-evaluation multi\-agent framework, where a generation agent synthesizes event data and an evaluation agent assesses semantic consistency to provide reward signals for iterative optimization\. These studies indicate that role specialization, agent collaboration, and verification\-oriented refinement are effective for improving the reliability of LLM\-based structured extraction\.

![Refer to caption](https://arxiv.org/html/2610.02970v1/fig/main_model.png)Fig\. 1:Overview of the proposed GAMA framework\. The framework instantiates a guideline\-augmented multi\-agent paradigm for schema\-as\-code biomedical named entity recognition\. It consists of five stages: rule induction from training annotations, guideline memory construction, guideline\-guided mention planning, schema\-as\-code entity generation, and dual\-loop verification refinement for producing final structured entities\.

## IIIMethods

We propose GAMA, a guideline\-augmented multi\-agent framework for schema\-as\-code BioNER\. GAMA reformulates entity extraction as a schema\-constrained programming process with explicit guideline modeling and iterative verification\. The framework consists of five components: \(1\) an Annotation Rule Induction Agent that induces dataset\-specific annotation rules from training data, \(2\) a Rule Verification Agent that validates and filters induced rules, \(3\) a Planning Agent that generates ranked mention\-type hypotheses, \(4\) a Coding Agent that converts these hypotheses into executable entity objects, and \(5\) a Verification and Dual\-Loop Refinement module that performs prediction\-level validation and iterative correction\. The overall workflow is illustrated in Figure[1](https://arxiv.org/html/2610.02970#S2.F1)\.

### III\-AAnnotation Rule Induction

BioNER is strongly affected by dataset\-specific annotation criteria\. Different datasets may exhibit variations in entity boundary conventions, type granularity, and annotation scope, even for semantically similar biomedical mentions\. Directly using input–label pairs provides only implicit supervision and fails to explicitly expose annotation semantics\.

To address this limitation, we introduce an Annotation Rule Induction Agent, which transforms instance\-level annotations into explicit candidate annotation rules\. LetD\(s\)=\{\(xi,yi\)\}i=1M\(s\)D^\{\(s\)\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{M^\{\(s\)\}\}denote the annotated training set of datasetss, wherexix\_\{i\}is a sentence andyi=\{\(ej,tj\)\}j=1Niy\_\{i\}=\\\{\(e\_\{j\},t\_\{j\}\)\\\}\_\{j=1\}^\{N\_\{i\}\}is its corresponding entity annotation set\. Each\(ej,tj\)\(e\_\{j\},t\_\{j\}\)consists of an entity mentioneje\_\{j\}and its typetj∈𝒯\(s\)t\_\{j\}\\in\\mathcal\{T\}^\{\(s\)\}\.

For each training instance\(xi,yi\)\(x\_\{i\},y\_\{i\}\), the rule induction agent generates a set of candidate rules:

ℛi\(s\)=ℳrule​\(Prule​\(xi,yi\)\),\\mathcal\{R\}\_\{i\}^\{\(s\)\}=\\mathcal\{M\}\_\{\\mathrm\{rule\}\}\\big\(P\_\{\\mathrm\{rule\}\}\(x\_\{i\},y\_\{i\}\)\\big\),\(1\)whereℳrule\\mathcal\{M\}\_\{\\mathrm\{rule\}\}denotes the LLM\-based rule generator andPruleP\_\{\\mathrm\{rule\}\}is the prompting function\.

The global candidate rule pool is defined as:

ℛ\(s\)=⋃i=1M\(s\)ℛi\(s\)\.\\mathcal\{R\}^\{\(s\)\}=\\bigcup\_\{i=1\}^\{M^\{\(s\)\}\}\\mathcal\{R\}\_\{i\}^\{\(s\)\}\.\(2\)
These rules are subsequently passed to the Rule Verification Agent for validation and filtering\.

### III\-BRule Verification and Guideline Memory Construction

Not all induced rules are reliable or consistent with dataset\-specific annotation standards\. Therefore, we introduce a Rule Verification Agent to filter candidate rules before constructing the guideline memory\. The key intuition is that a useful annotation rule should help recover gold entities from the training data when it is applied as an explicit constraint\.

For each candidate rulerk∈ℛ\(s\)r\_\{k\}\\in\\mathcal\{R\}^\{\(s\)\}, the verification agent applies the rule to training instances and generates rule\-guided predictions:

y^i,k\(s\)=ℳver​\(Pver​\(xi,rk\)\),\\hat\{y\}\_\{i,k\}^\{\(s\)\}=\\mathcal\{M\}\_\{\\mathrm\{ver\}\}\\big\(P\_\{\\mathrm\{ver\}\}\(x\_\{i\},r\_\{k\}\)\\big\),\(3\)whereℳver\\mathcal\{M\}\_\{\\mathrm\{ver\}\}denotes the verification LLM andPverP\_\{\\mathrm\{ver\}\}is the verification prompt\. The predictiony^i,k\(s\)\\hat\{y\}\_\{i,k\}^\{\(s\)\}is then compared with the gold annotation setyiy\_\{i\}\. A rule is regarded as supported by an instance if it helps recover at least one gold entity with the correct mention boundary and entity type\.

For each rule, we compute a support score by counting how many training instances support it\. Rules with no support are discarded\. The remaining rules are grouped by their associated entity types and ranked according to their support scores\. For each entity typett, we retain the Top\-NNverified rules to construct the dataset\-specific guideline memory:

𝒢t\(s\)=Top−N⁡\(ℛtver,\(s\)\),𝒢\(s\)=\{𝒢t\(s\)\}t∈𝒯\(s\),\\mathcal\{G\}\_\{t\}^\{\(s\)\}=\\operatorname\{Top\-N\}\(\\mathcal\{R\}\_\{t\}^\{\\mathrm\{ver\},\(s\)\}\),\\quad\\mathcal\{G\}^\{\(s\)\}=\\\{\\mathcal\{G\}\_\{t\}^\{\(s\)\}\\\}\_\{t\\in\\mathcal\{T\}^\{\(s\)\}\},\(4\)whereℛtver,\(s\)\\mathcal\{R\}\_\{t\}^\{\\mathrm\{ver\},\(s\)\}denotes the verified rules associated with entity typett\.

The resulting guideline memory𝒢\(s\)\\mathcal\{G\}^\{\(s\)\}provides explicit and structured annotation constraints for downstream Planning, Coding, and Verification modules\. By filtering rules through training\-data consistency, the Rule Verification Agent reduces noisy or overly instance\-specific rules without introducing additional trainable parameters\.

### III\-CGuideline\-Guided Mention Planning

Given an input biomedical sentencexx, the verified guideline memory𝒢\(s\)\\mathcal\{G\}^\{\(s\)\}constructed by the Rule Verification Agent, and the entity schema𝒮\(s\)\\mathcal\{S\}^\{\(s\)\}, the Planning Agent generates a ranked set of mention\-type hypotheses:

𝒫=ℳplan​\(Pplan​\(x,𝒢\(s\),𝒮\(s\)\)\),\\mathcal\{P\}=\\mathcal\{M\}\_\{\\mathrm\{plan\}\}\\big\(P\_\{\\mathrm\{plan\}\}\(x,\\mathcal\{G\}^\{\(s\)\},\\mathcal\{S\}^\{\(s\)\}\)\\big\),\(5\)whereℳplan\\mathcal\{M\}\_\{\\mathrm\{plan\}\}denotes the LLM used for planning,PplanP\_\{\\mathrm\{plan\}\}is the planning prompt, and𝒫\\mathcal\{P\}is the generated planning set\. The schema𝒮\(s\)\\mathcal\{S\}^\{\(s\)\}specifies the valid entity type set𝒯\(s\)\\mathcal\{T\}^\{\(s\)\}and the output constraints for datasetss\.

The planning set is defined as:

𝒫=\{\(mi,ti,βi,ρi\)\}i=1K,\\mathcal\{P\}=\\\{\(m\_\{i\},t\_\{i\},\\beta\_\{i\},\\rho\_\{i\}\)\\\}\_\{i=1\}^\{K\},\(6\)wheremim\_\{i\}denotes a candidate biomedical entity mention,ti∈𝒯\(s\)t\_\{i\}\\in\\mathcal\{T\}^\{\(s\)\}denotes its predicted entity type,βi∈\[0,1\]\\beta\_\{i\}\\in\[0,1\]is a confidence score used for ranking, andρi\\rho\_\{i\}is a natural\-language rationale grounded in the dataset\-specific guideline memory𝒢\(s\)\\mathcal\{G\}^\{\(s\)\}\.

The scoreβi\\beta\_\{i\}is used to prioritize candidate mention\-type hypotheses during subsequent Coding and Verification, especially when overlapping spans or type conflicts occur\. The rationaleρi\\rho\_\{i\}links the mentionmim\_\{i\}to relevant annotation rules in𝒢\(s\)\\mathcal\{G\}^\{\(s\)\}, making the type assignment explicit and providing interpretable evidence for the downstream agents\.

### III\-DSchema\-as\-Code Entity Construction

Existing BioNER methods often produce outputs in free\-form text, tagging sequences, or loosely structured JSON objects\. Such representations may lead to format inconsistencies, missing fields, or invalid entity\-type assignments, making them difficult to integrate into a multi\-stage verification workflow\. To address this issue, we introduce the Coding Agent, which converts the ranked mention\-type hypotheses from the Planning Agent into schema\-constrained entity objects\.

Each candidate entity is represented using a minimal BioNER schema, as illustrated in Figure[2](https://arxiv.org/html/2610.02970#S3.F2):

![Refer to caption](https://arxiv.org/html/2610.02970v1/fig/entity_code.png)Fig\. 2:Code\-based template for BioNER entities\.spandenotes the extracted biomedical mention, andentity\_typedenotes its predicted category\.Given the planning set𝒫=\{\(mi,ti,βi,ρi\)\}i=1K\\mathcal\{P\}=\\\{\(m\_\{i\},t\_\{i\},\\beta\_\{i\},\\rho\_\{i\}\)\\\}\_\{i=1\}^\{K\}, the Coding Agent instantiates each mention\-type hypothesis as a schema\-constrained entity object:

ci=EntityObject\(mi,ti\),i=1,…,K\.c\_\{i\}=\\operatorname\{EntityObject\}\(m\_\{i\},t\_\{i\}\),\\quad i=1,\\ldots,K\.\(7\)The complete set of generated entity objects is denoted as𝒞=\{ci\}i=1K\\mathcal\{C\}=\\\{c\_\{i\}\\\}\_\{i=1\}^\{K\}\. Here,cic\_\{i\}corresponds to theii\-th planning hypothesis, wheremim\_\{i\}fills thespanfield andtit\_\{i\}fills theentity\_typefield under the dataset\-specific schema𝒮\(s\)\\mathcal\{S\}^\{\(s\)\}\. The fieldentity\_typeis constrained by the dataset\-specific type set𝒯\(s\)\\mathcal\{T\}^\{\(s\)\}\.

The ranking scoreβi\\beta\_\{i\}and rationaleρi\\rho\_\{i\}are preserved as auxiliary planning information for later verification and conflict handling\. Thus, the Coding Agent standardizes each mention\-type hypothesis into a schema\-aware entity object while maintaining the planning context needed by the subsequent Verification and Dual\-Loop Refinement module\.

### III\-EVerification and Dual\-Loop Refinement

After the Coding Agent generates schema\-constrained entity objects, GAMA applies a prediction\-level verification and refinement mechanism to ensure that the outputs are both text\-grounded and schema\-compliant\. Each generated object is checked before being included in the final prediction set\.

Given the input sentencexx, the dataset\-specific schema𝒮\(s\)\\mathcal\{S\}^\{\(s\)\}, and the generated entity objects𝒞=\{ci\}i=1K\\mathcal\{C\}=\\\{c\_\{i\}\\\}\_\{i=1\}^\{K\}, each objectcic\_\{i\}is verified along three dimensions: span grounding, type validity, and structural compliance\. Span grounding checks whether the predicted mention appears inxx, type validity checks whether the predicted type belongs to𝒯\(s\)\\mathcal\{T\}^\{\(s\)\}, and structural compliance checks whether the object follows the schema𝒮\(s\)\\mathcal\{S\}^\{\(s\)\}\. Formally, this prediction\-level verification is defined as:

\(Vi,ϵi\)=𝒱pred​\(ci,x,𝒮\(s\)\),\(V\_\{i\},\\epsilon\_\{i\}\)=\\mathcal\{V\}\_\{\\mathrm\{pred\}\}\(c\_\{i\},x,\\mathcal\{S\}^\{\(s\)\}\),\(8\)whereViV\_\{i\}indicates whethercic\_\{i\}passes the verification checks, andϵi\\epsilon\_\{i\}denotes diagnostic feedback used for refinement\.

To improve robustness, GAMA employs a dual\-loop refinement mechanism\. The outer loop follows the ranked hypotheses produced by the Planning Agent and prioritizes candidates with higherβi\\beta\_\{i\}\. The inner loop uses the diagnostic feedbackϵi\\epsilon\_\{i\}to revise and re\-check the corresponding entity object until it passes verification or reaches the maximum number of refinement attempts\. Only verified objects are included in the final prediction set:

𝒴\(s\)\(x\)=\{\(mi,ti\)∣Vi=1,ci=EntityObject\(mi,ti\)\}\.\\mathcal\{Y\}^\{\(s\)\}\(x\)=\\\{\(m\_\{i\},t\_\{i\}\)\\mid V\_\{i\}=1,\\ c\_\{i\}=\\operatorname\{EntityObject\}\(m\_\{i\},t\_\{i\}\)\\\}\.\(9\)
This mechanism enables GAMA to correct object\-level errors while preserving the hypothesis ranking produced by the Planning Agent, thereby producing text\-grounded and schema\-compliant entity predictions\.

TABLE I:Statistics of the BioNER datasets used in our experiments\.DatasetLabelsTrainTestEntitiesNCBI154329426881BC2GM112632506524583BC4CHEMD131770424084310GENIA516692185450509AnatEM15861383013701

## IVExperiments

### IV\-ADatasets

We use five public BioNER datasets in our experiments: NCBI Disease\[[16](https://arxiv.org/html/2610.02970#bib.bib16)\], BC2GM\[[17](https://arxiv.org/html/2610.02970#bib.bib17)\], BC4CHEMD\[[18](https://arxiv.org/html/2610.02970#bib.bib18)\], GENIA\[[19](https://arxiv.org/html/2610.02970#bib.bib19)\], and AnatEM\[[20](https://arxiv.org/html/2610.02970#bib.bib20)\]\. NCBI Disease focuses on disease entity recognition, BC2GM mainly contains gene and protein mentions, BC4CHEMD is designed for chemical entity recognition, GENIA includes multiple biomedical entity types, and AnatEM provides annotations for anatomical entities\. Together, these datasets cover a broad range of biomedical entity categories and are widely used for evaluating BioNER systems\. Table[I](https://arxiv.org/html/2610.02970#S3.T1)lists the detailed statistics\.

### IV\-BBaselines

We compare our method with four representative LLM\-based NER baselines, all of which perform inference without fine\-tuning LLMs on task\-specific data\.

- •GPT\-NER\[[21](https://arxiv.org/html/2610.02970#bib.bib21)\]transforms NER into a text generation task by asking LLMs to mark entity spans with special tokens, followed by self\-verification to mitigate hallucinated predictions\.
- •CodeIE\[[22](https://arxiv.org/html/2610.02970#bib.bib22)\]converts information extraction as code generation, using Code\-LLMs to generate structured predictions in Python\-style lists and dictionaries under few\-shot settings\.
- •CMAS\[[13](https://arxiv.org/html/2610.02970#bib.bib13)\]performs zero\-shot NER with a cooperative multi\-agent framework that integrates self\-annotation, type\-related feature extraction, demonstration discrimination, and final prediction\.
- •EICL\[[8](https://arxiv.org/html/2610.02970#bib.bib8)\]selects demonstrations for BioNER through entity\-aware in\-context learning, combining sentence\-level semantic similarity, entity\-centric representations, and domain\-specific vocabulary overlap\.

TABLE II:Performance comparison of different methods across five BioNER datasets under different LLMs\. The best results for each LLM\-dataset combination are highlighted inbold\.LLMMethodNCBIBC2GMBC4CHEMDGENIAAnatEMLlama3\-8BCodeIE68\.2459\.9468\.2557\.2570\.29GPT\-NER62\.8952\.9765\.2148\.2864\.98CMAS61\.2153\.6465\.5452\.7264\.75EICL73\.5663\.4176\.2960\.3175\.26GAMA74\.6266\.3576\.7462\.4575\.95Llama3\-70BCodeIE73\.2565\.2372\.4163\.5175\.15GPT\-NER68\.2659\.6470\.6257\.1569\.24CMAS65\.8958\.2969\.9556\.9168\.89EICL78\.5469\.6580\.5969\.8479\.61GAMA79\.2670\.6981\.6871\.5980\.65
### IV\-CBase LLMs

We evaluate our method and all baselines on top of multiple representative instruction\-tuned LLMs to ensure a comprehensive comparison across model families, parameter scales, and closed\-source/open\-source settings\. The evaluated models include Llama3\-8B\-Instruct and Llama3\-70B\-Instruct from the Llama3 family\[[23](https://arxiv.org/html/2610.02970#bib.bib23)\], Qwen2\.5\-14B\-Instruct and Qwen2\.5\-72B\-Instruct from the Qwen2\.5 family\[[24](https://arxiv.org/html/2610.02970#bib.bib24)\], and two proprietary OpenAI models, GPT\-3\.5\-turbo and GPT\-4o\[[25](https://arxiv.org/html/2610.02970#bib.bib25)\]\.

### IV\-DImplementation Details

Our framework is built upon instruction\-tuned LLM backbones without task\-specific fine\-tuning\. For fair comparison, the same annotated training split is used whenever labeled information is required: baselines select in\-context examples from it, while GAMA uses it only to induce and verify dataset\-specific guideline memory\. All components are driven by natural\-language prompts and communicate through structured intermediate outputs, enabling multi\-agent collaboration without updating model parameters\. Unless otherwise specified, the same LLM backbone is used for all components\. For guideline memory construction, the top\-nnverified annotation rules are retained for each entity type, wheren=10n=10\. The Planning Agent generatesk=3k=3ranked candidate hypotheses, and the maximum number of inner\-loop refinement iterations is set tot=3t=3\. All experiments use greedy decoding withtemperature=0\\mathrm\{temperature\}=0\. We report the average results over three independent runs\.

### IV\-EEvaluation Metrics

Following previous work\[[8](https://arxiv.org/html/2610.02970#bib.bib8)\], we adopt the Micro\-F1 score as the primary evaluation metric for all methods\. We use exact\-match evaluation, where an extracted entity is counted as correct only if its span boundary and entity type are both identical to the gold annotation\. Micro\-F1 provides an entity\-level measure that balances precision and recall over all entity categories, enabling fair comparison with existing NER methods\.

## VEXPERIMENTAL RESULTS

### V\-AMain Results

Table[II](https://arxiv.org/html/2610.02970#S4.T2)summarizes the main results under Llama3\-8B and Llama3\-70B backbones\. GAMA consistently achieves the best Micro\-F1 scores across all five BioNER datasets under both backbone settings, demonstrating the overall effectiveness of the proposed guideline\-augmented multi\-agent framework\. Compared with conventional prompting\-based methods such as GPT\-NER and CMAS, GAMA obtains substantially higher performance, indicating that directly prompting LLMs with natural\-language instructions is insufficient for reliable biomedical entity extraction\. In addition, GAMA also outperforms the strongest retrieval\-based baseline, EICL, showing that explicit guideline modeling and schema\-constrained verification provide complementary benefits beyond demonstration selection\.

The improvements are particularly noticeable on datasets that require more careful boundary and type decisions\. Under Llama3\-8B, GAMA improves over EICL by 2\.94 points on BC2GM and 2\.14 points on GENIA\. Under Llama3\-70B, GAMA further improves over EICL by 1\.75 points on GENIA and maintains consistent gains on BC2GM, BC4CHEMD, and AnatEM\. These results suggest that dataset\-specific guideline memory helps the model better capture annotation conventions, while the planning and verification stages reduce boundary ambiguity and invalid predictions\. The advantage on GENIA is especially meaningful because it contains multiple entity types, making type assignment more challenging than single\-label datasets\.

When scaling the backbone from Llama3\-8B to Llama3\-70B, all methods generally achieve better performance, but GAMA remains the best\-performing method across all datasets\. This indicates that the proposed framework is not tied to a specific model scale and can effectively cooperate with stronger LLM backbones\. Overall, the main results verify that GAMA improves BioNER performance through the combined effects of guideline\-aware reasoning, schema\-as\-code entity construction, and iterative verification\.

TABLE III:Performance comparison of different methods under multiple LLM backbones \(Qwen2\.5\-14B, Qwen2\.5\-72B, GPT\-3\.5\-turbo, and GPT\-4o\)\. The best results for each LLM group are highlighted inbold\.LLMMethodNCBIBC2GMBC4CHEMDGENIAAnatEMQwen2\.5\-14BCodeIE68\.4760\.1368\.8957\.5470\.71GPT\-NER63\.0853\.2565\.5448\.6265\.19CMAS61\.7653\.9465\.9353\.0965\.06EICL73\.9663\.8876\.7360\.6275\.65GAMA75\.1266\.9177\.3962\.9476\.42Qwen2\.5\-72BCodeIE73\.0665\.4872\.1363\.2674\.98GPT\-NER67\.9459\.3770\.2556\.8867\.93CMAS65\.6158\.0269\.6656\.6468\.52EICL78\.2069\.8280\.2869\.5679\.34GAMA78\.9170\.8781\.3371\.2180\.36GPT\-3\.5\-turboCodeIE72\.3964\.6671\.2762\.8374\.23GPT\-NER67\.5258\.5169\.9656\.2467\.95CMAS65\.2857\.4569\.0356\.1868\.01EICL78\.0568\.8880\.0169\.1378\.92GAMA78\.8270\.1781\.2570\.9880\.17GPT\-4oCodeIE73\.0165\.4272\.1663\.3875\.04GPT\-NER68\.1259\.3470\.6257\.1968\.68CMAS65\.7558\.1469\.7956\.7268\.89EICL78\.7770\.7280\.5969\.9379\.83GAMA79\.7170\.7982\.0871\.9380\.89
### V\-BGeneralization across LLMs

The results in Table[III](https://arxiv.org/html/2610.02970#S5.T3)demonstrate that the proposed method exhibits strong cross\-model generalization ability across both open\-source and proprietary LLMs\. Specifically, our framework consistently ranks first on all datasets and all backbone models, verifying that the performance improvements are not tied to a specific LLM family\. From the perspective of backbone scaling, a clear positive correlation can be observed between model capability and final BioNER performance\. As the backbone evolves from Qwen2\.5\-14B to Qwen2\.5\-72B and further to GPT\-4o, nearly all methods show performance improvements; however, our method benefits the most from stronger LLM reasoning capacity\. For instance, on NCBI, the proposed framework improves from 75\.12 under Qwen2\.5\-14B to 78\.91 under Qwen2\.5\-72B and further to 79\.71 under GPT\-4o\. Similar trends are consistently observed on BC2GM and AnatEM\. Compared with traditional prompt\-based baselines such as GPT\-NER and CMAS, the proposed framework achieves substantially larger margins, suggesting that structured code\-oriented reasoning is more effective than conventional natural\-language prompting for biomedical entity extraction\. Moreover, even compared with the strongest baseline EICL, our approach still achieves stable gains across nearly all datasets and backbone models, demonstrating the effectiveness of the proposed collaborative reasoning and verification mechanism\. Notably, the improvements become more pronounced on datasets involving complex biomedical terminology and long\-span entities, such as BC4CHEMD and AnatEM\. This indicates that the proposed framework can better capture structured semantic dependencies and reduce boundary ambiguity through iterative multi\-agent refinement\.

### V\-CAblation Study

Table[IV](https://arxiv.org/html/2610.02970#S5.T4)evaluates five key design choices of GAMA across two LLM backbones and three BioNER datasets\. All ablated variants consistently underperform the full model, with similar trends observed for Llama3\-70B and GPT\-4o, demonstrating the robustness of the proposed design\. Removing the entire guideline modeling pathway, including Rule Induction and Rule Verification, decreases F1 by 3\.53–4\.89 points across the evaluated settings\. This confirms the importance of incorporating dataset\-specific annotation knowledge into the extraction process\. When only Rule Verification is removed, performance decreases by 2\.79–3\.96 points\. Moreover, the gains of using unverified rules over using no guideline modeling are relatively limited and dataset\-dependent\. These results indicate that rule induction alone may introduce noisy or overly instance\-specific guidance, while verification is important for constructing a reliable guideline memory\. Removing Planning Rationales produces the largest average degradation among the evaluated variants, with F1 reductions ranging from 2\.96 to 6\.09 points\. This demonstrates that explicitly relating candidate mentions and entity types to the induced annotation guidelines helps organize the prediction process and improves boundary and type decisions\. Notably, this variant retains candidate generation and removes only the rationale\-based reasoning mechanism\. Removing Schema\-as\-Code results in consistent reductions of 1\.45–2\.87 F1 points, showing that schema\-constrained representations provide additional benefits over unconstrained entity generation\. Disabling the Verification Loop causes larger drops of 2\.20–4\.15 points, particularly on BC4CHEMD under both backbones\. This finding is consistent with the role of iterative verification and correction in identifying inconsistent predictions and improving the reliability of complex biomedical entity extraction\. Overall, the results demonstrate that GAMA benefits from the complementary interaction of verified guideline modeling, rationale\-guided planning, schema\-constrained representation, and iterative refinement\.

TABLE IV:Ablation studies on two different LLMs, evaluated on three BioNER datasets\.Llama3\-70BAblation SettingNCBIBC2GMBC4CHEMDGAMA \(full model\)79\.2670\.6981\.68w/o Guideline Modeling75\.4966\.2777\.42w/o Rule Verification76\.2166\.9877\.95w/o Planning Rationales74\.1766\.3776\.43w/o Schema\-as\-Code77\.6368\.9879\.92w/o Verification Loop76\.5467\.5977\.53GPT\-4oAblation SettingNCBIBC2GMBC4CHEMDGAMA \(full model\)79\.7170\.7982\.08w/o Guideline Modeling75\.2167\.2677\.19w/o Rule Verification76\.9267\.0178\.12w/o Planning Rationales73\.6267\.8376\.32w/o Schema\-as\-Code78\.2669\.0879\.21w/o Verification Loop77\.5168\.1877\.96
### V\-DEffect of Top\-N Rules

To investigate the effect of top\-nnrules on biomedical entity recognition, we evaluate differentnnsettings under two LLM backbones\. As shown in Table[V](https://arxiv.org/html/2610.02970#S5.T5), model performance first improves and then gradually saturates asnnincreases\. This phenomenon indicates that incorporating additional rules initially enhances the model’s understanding of biomedical annotation guidelines and entity semantics\. For example, under Llama3\-70B, the NCBI score improves from 77\.26 atn=5n=5to 79\.26 atn=10n=10, demonstrating that a moderate number of rules effectively strengthens semantic disambiguation and boundary prediction\. However, excessively large rule sets do not continuously improve performance\. Whenn=30n=30, both models exhibit noticeable performance declines on several datasets\. We attribute this phenomenon to two main factors\. First, larger rule sets significantly increase the prompt context length, which may weaken the model’s ability to focus on the most important annotation constraints\. Second, introducing low\-frequency or weakly relevant rules may dilute the influence of high\-quality core rules and increase semantic noise during reasoning\. In addition, the smaller Llama3\-8B model reaches saturation earlier than Llama3\-70B, indicating that smaller LLMs are more sensitive to long and complex guideline contexts\. In contrast, larger models demonstrate stronger capabilities in handling larger rule collections and maintaining stable reasoning performance\.

TABLE V:Top\-N performance of Llama3 models on three BioNER datasets\.ModelTop\-NNCBIBC2GMBC4CHEMDLlama3\-8B573\.2665\.2173\.281074\.6266\.3576\.742074\.2166\.1777\.123073\.5265\.6475\.94Llama3\-70B577\.2669\.6180\.261079\.2670\.6981\.682078\.6970\.9281\.213077\.4269\.2680\.36
### V\-EImpact ofkkandtt

To further evaluate the proposed iterative reasoning framework, we analyze two key hyperparameters: the number of generated hypotheseskkand the maximum refinement iterationstt\. Table[VI](https://arxiv.org/html/2610.02970#S5.T6)summarizes the results on three biomedical NER datasets\. Several observations can be made\. First, increasingkkimproves performance in most settings, indicating that multiple candidate hypotheses help the model explore diverse semantic interpretations of biomedical entities\. This is particularly beneficial for datasets with complex entity structures and ambiguous terminology, such as BC4CHEMD\. Second, increasingttalso leads to noticeable gains, demonstrating the effectiveness of iterative refinement and self\-correction\. Additional refinement iterations allow the framework to repair inconsistent predictions and improve entity boundary accuracy\. Nevertheless, the gains gradually saturate when bothkkandttbecome excessively large\. Although\(k=5,t=5\)\(k=5,t=5\)achieves competitive performance, it does not consistently outperform\(k=3,t=3\)\(k=3,t=3\)\. This suggests that overly large hypothesis pools may introduce noisy low\-confidence candidates, while excessive refinement iterations can produce redundant corrections and reduce prediction stability\. Overall, these findings show that the proposed framework achieves optimal performance through a balanced trade\-off between candidate exploration and iterative verification, with\(k=3,t=3\)\(k=3,t=3\)providing the most effective configuration\.

TABLE VI:Experimental results of differentkkandttsettings under the Llama3\-70B backbone on three BioNER datasets\.kkttNCBIBC2GMBC4CHEMD1174\.6565\.3471\.421375\.2666\.2572\.253178\.4569\.1281\.153379\.2670\.6981\.685378\.9269\.5281\.315579\.2370\.7281\.57

## VIConclusions

In this paper, we presented GAMA, a guideline\-augmented multi\-agent framework for schema\-as\-code BioNER\. GAMA addresses two key limitations of existing LLM\-based BioNER methods: the lack of explicit modeling for dataset\-specific annotation semantics and the weak structural control of free\-form generation\. By inducing and verifying annotation guidelines from training instances, decomposing extraction into planning and coding stages, and validating schema\-as\-code entity objects through dual\-loop refinement, GAMA produces more grounded and structurally valid biomedical entity predictions\. Experiments on five public BioNER datasets and multiple LLM backbones show that GAMA consistently outperforms strong LLM\-based baselines\. Ablation and parameter analyses further verify the effectiveness of each key component\. These results highlight the importance of combining dataset\-level annotation guidance with explicit structural constraints for reliable LLM\-based BioNER\. In future work, we will explore more efficient guideline selection and extend the framework to broader biomedical information extraction tasks\.

## References

- \[1\]B\. Song, F\. Li, Y\. Liu, and X\. Zeng, “Deep learning methods for biomedical named entity recognition: a survey and qualitative comparison,”*Briefings in Bioinformatics*, vol\. 22, no\. 6, p\. bbab282, 2021\.
- \[2\]Y\. Hu, Y\. Chen, R\. Huang, Y\. Qin, and Q\. Zheng, “A hierarchical convolutional model for biomedical relation extraction,”*Information Processing & Management*, vol\. 61, no\. 1, p\. 103560, 2024\.
- \[3\]A\. Ramponi, R\. Van Der Goot, R\. Lombardo, and B\. Plank, “Biomedical event extraction as sequence labeling,” in*Proceedings of the 2020 conference on empirical methods in natural language processing \(emnlp\)*, 2020, pp\. 5357–5367\.
- \[4\]J\. Lee, W\. Yoon, S\. Kim, D\. Kim, S\. Kim, C\. H\. So, and J\. Kang, “Biobert: a pre\-trained biomedical language representation model for biomedical text mining,”*Bioinformatics*, vol\. 36, no\. 4, pp\. 1234–1240, 2020\.
- \[5\]Y\. Gu, R\. Tinn, H\. Cheng, M\. Lucas, N\. Usuyama, X\. Liu, T\. Naumann, J\. Gao, and H\. Poon, “Domain\-specific language model pretraining for biomedical natural language processing,”*ACM Transactions on Computing for Healthcare \(HEALTH\)*, vol\. 3, no\. 1, pp\. 1–23, 2021\.
- \[6\]T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell*et al\.*, “Language models are few\-shot learners,”*Advances in neural information processing systems*, vol\. 33, pp\. 1877–1901, 2020\.
- \[7\]Y\. Ge, S\. Das, Y\. Guo, and A\. Sarker, “Retrieval augmented generation based dynamic prompting for few\-shot biomedical named entity recognition using large language models,”*Research Square*, pp\. rs–3, 2025\.
- \[8\]Z\. Liu and N\. Yao, “Eicl: Entity\-aware in\-context learning for bioner via hybrid semantic\-terminological retrieval,” in*2025 IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\)*\. IEEE, 2025, pp\. 1–6\.
- \[9\]J\. Bian, J\. Zheng, Y\. Zhang, and S\. Zhu, “Inspire the large language model by external knowledge on biomedical named entity recognition,”*arXiv preprint arXiv:2309\.12278*, 2023\.
- \[10\]W\. Zhou, “Biomedical nested ner with large language model and umls heuristics,”*arXiv preprint arXiv:2407\.05480*, 2024\.
- \[11\]Y\. Hu, Q\. Chen, J\. Du, X\. Peng, V\. K\. Keloth, X\. Zuo, Y\. Zhou, Z\. Li, X\. Jiang, Z\. Lu*et al\.*, “Improving large language models for clinical named entity recognition via prompt engineering,”*Journal of the American Medical Informatics Association*, vol\. 31, no\. 9, pp\. 1812–1820, 2024\.
- \[12\]M\. Li, H\. Zhou, H\. Yang, and R\. Zhang, “Rt: a retrieving and chain\-of\-thought framework for few\-shot medical named entity recognition,”*Journal of the American Medical Informatics Association*, vol\. 31, no\. 9, pp\. 1929–1938, 2024\.
- \[13\]Z\. Wang, Z\. Zhao, Y\. Lyu, Z\. Chen, M\. de Rijke, and Z\. Ren, “A cooperative multi\-agent framework for zero\-shot named entity recognition,” in*Proceedings of the ACM on Web Conference 2025*, 2025, pp\. 4183–4195\.
- \[14\]M\. Lu, Y\. Xie, Z\. Bi, S\. Cao, and X\. Wang, “Crossagentie: Cross\-type and cross\-task multi\-agent llm collaboration for zero\-shot information extraction,” in*Findings of the Association for Computational Linguistics: ACL 2025*, 2025, pp\. 13 953–13 977\.
- \[15\]G\. Zhang, H\. Zhang, Y\. Han, Y\. Fan, Y\. Shao, H\. Tan, and R\. Li, “Learning to generate and extract: A multi\-agent collaboration framework for zero\-shot document\-level event arguments extraction,” in*Proceedings of the AAAI Conference on Artificial Intelligence*, vol\. 40, no\. 41, 2026, pp\. 34 665–34 673\.
- \[16\]R\. I\. Doğan, R\. Leaman, and Z\. Lu, “Ncbi disease corpus: a resource for disease name recognition and concept normalization,”*Journal of biomedical informatics*, vol\. 47, pp\. 1–10, 2014\.
- \[17\]L\. Smith, L\. K\. Tanabe, R\. J\. N\. Ando, C\.\-J\. Kuo, I\.\-F\. Chung, C\.\-N\. Hsu, Y\.\-S\. Lin, R\. Klinger, C\. M\. Friedrich, K\. Ganchev*et al\.*, “Overview of biocreative ii gene mention recognition,”*Genome biology*, vol\. 9, no\. Suppl 2, p\. S2, 2008\.
- \[18\]M\. Krallinger, O\. Rabal, F\. Leitner, M\. Vazquez, D\. Salgado, Z\. Lu, R\. Leaman, Y\. Lu, D\. Ji, D\. M\. Lowe*et al\.*, “The chemdner corpus of chemicals and drugs and its annotation principles,”*Journal of cheminformatics*, vol\. 7, no\. Suppl 1, p\. S2, 2015\.
- \[19\]J\.\-D\. Kim, T\. Ohta, Y\. Tateisi, and J\. Tsujii, “Genia corpus—a semantically annotated corpus for bio\-textmining,”*Bioinformatics*, vol\. 19, no\. suppl\_1, pp\. i180–i182, 2003\.
- \[20\]S\. Pyysalo and S\. Ananiadou, “Anatomical entity mention recognition at literature scale,”*Bioinformatics*, vol\. 30, no\. 6, pp\. 868–875, 2014\.
- \[21\]S\. Wang, X\. Sun, X\. Li, R\. Ouyang, F\. Wu, T\. Zhang, J\. Li, G\. Wang, and C\. Guo, “Gpt\-ner: Named entity recognition via large language models,” in*Findings of the association for computational linguistics: NAACL 2025*, 2025, pp\. 4257–4275\.
- \[22\]P\. Li, T\. Sun, Q\. Tang, H\. Yan, Y\. Wu, X\.\-J\. Huang, and X\. Qiu, “Codeie: Large code generation models are better few\-shot information extractors,” in*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, 2023, pp\. 15 339–15 353\.
- \[23\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan*et al\.*, “The llama 3 herd of models,”*arXiv preprint arXiv:2407\.21783*, 2024\.
- \[24\]A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu, “Qwen2\.5 technical report,”*CoRR*, vol\. abs/2412\.15115, 2024\. \[Online\]\. Available:[https://doi\.org/10\.48550/arXiv\.2412\.15115](https://doi.org/10.48550/arXiv.2412.15115)
- \[25\]A\. Hurst, A\. Lerer, A\. P\. Goucher, A\. Perelman, A\. Ramesh, A\. Clark, A\. Ostrow, A\. Welihinda, A\. Hayes, A\. Radford*et al\.*, “Gpt\-4o system card,”*arXiv preprint arXiv:2410\.21276*, 2024\.