PiPMRE: A Pipeline Based on Language Model for Medical Relation Extraction

# PiPMRE: A Pipeline Based on Language Model for Medical Relation Extraction
Source: [https://arxiv.org/html/2609.02896](https://arxiv.org/html/2609.02896)
Jiaxin Duan \(duanjx@stu\.pku\.edu\.cn\) School of Software and Microelectronics, Peking University, Beijing, China Fengyu Lu \(fengyul@stu\.pku\.edu\.cn\) School of Software and Microelectronics, Peking University, Beijing, China Junfei Liu \(liujunfei@pku\.edu\.cn\) School of Software and Microelectronics, Peking University, Beijing, China

###### Abstract

Medical relation extraction \(MRE\) is commonly known for extracting entities and their relations jointly from a medical text, which has attracted considerable attention in recent years\. Previous studies treat MRE as a sequence tagging task, which results in either a challenging design of the tagging schema or a failed extraction of multiple relations \- due to intricate relationships among medical entities\. In this work, we review the task from the linguistic perspective and propose a novel pipeline framework, PiPMRE, developed on language models to enhance MRE performance\. Specifically, PiPMRE consists of a relation generator and a relation filter\. Given a text, the generator first yields multiple relational triplets, and then the filter scores each triplet and retains only those that pass the borderline as the final results\. Implementing PiPMRE requires no tagging schema; instead, we use a simple template to reformulate the input text while ensuring entities and relations are generated in contextual order\. Extensive experimental results on two public datasets demonstrate the advancement of PiPMRE\. It surpasses the previous state\-of\-the\-art by an average of 5\.6 recall points and 4\.4 accuracy points\. PiPMRE’s superiorities are also demonstrated in few\-shot settings\.

Keywords:Medical Relation Extraction; Information Extraction; Pre\-trained Language Models;

9999footnotetext:Preprint: This work is accepted by theProceedings of the 47th Annual Conference of the Cognitive Science Society\.## Introduction

Medical relation extraction \(MRE\) aims to jointly extract paired entities and their relations from unstructured medical text intorelation tripletsformatted like<<subject entity, relation, object entity\>\>\(?, ?, ?\)\. It is a foundation step in building knowledge\-intensive applications, such as medical dialog systems\(?, ?\), medical knowledge graph completion\(?, ?\), medical question answering\(?, ?\), etc\., and has attracted increasing attention in recent years\.

Existing approaches for MRE are mainly categorized into two groups, treating the task as a sequential or a sequence\-to\-sequence \(Seq2Seq\) tagging problem\. As illustrated in Fig\.[1](https://arxiv.org/html/2609.02896#Sx1.F1), sequential tagging approaches\(?, ?, ?, ?, ?\) put major effort into the design of tagging schema\. In the fundamental work\(?, ?\), researchers imitate the BIEO notations used in named entity recognition \(NER\)\(?, ?\) and propose BIEO\-R\-SO tagging, where labels indicate theBegin/Inner/End token of aSubject/Objective entity in an inter\-entitiesRelation\. Subsequently,? \(?\) leverage groups of binary sequences to tag overlapped relations, and? \(?\) introduce a binary tree to tag tree\-form relational structures in the medical text\. Since sequential tagging is a long\-term topic in the machine learning community, this line of methods benefits from reusing mature technologies\. However, mapping model\-predicted tags to relational triplets is complicated, and how to tag an overwhelming number of terminologies and intricate relations in the medical domain remains an open question\. Seq2Seq approaches\(?, ?, ?\) instead learn to generate a linearized relation triplet conditioned on the given text, where subject, object, and relation are ordered by their contextual positions\. They have advanced since giving up cumbersome tagging labor, but they are limited in multi\-relation extraction due to the undetermined length of the target sequence\(?, ?\)\.

![Refer to caption](https://arxiv.org/html/2609.02896v1/x1.png)Figure 1:Comparison of MRE approaches in different styles\. Row four shows the tagging schema proposed in\(?, ?\), row six shows the linearized relation triple in\(?, ?\), and the last row shows an example of triple textualization\. PC: pathological classification\.In the open domain, recently proposed approaches focus on pre\-trained language models \(PLMs\)\(?, ?, ?\) and convert relation extraction \(RE\) into a fundamental linguistic problem to facilitate evoking the parametric knowledge of PLM\.? \(?\) covert RE into masked language modeling,? \(?\) convert RE into text\-denoising, and? \(?, ?\) convert RE into the generation of structured extraction language\. However, despite their advantages in a short\-sentence context, few previous studies have explored their potential in more complicated scenarios, especially in MRE\.

In this paper, we propose PiPMRE, a novel PLM pipeline to improve current MRE performance\. Specifically, PiPMRE consists of a relation generator and a relation filter\. The generator is powered by a Transformer pre\-trained with Seq2Seq text\-infilling\(?, ?\)\. To match its pre\-training task, we rewrite an MRE instance, including the text and relational triplets, into a corrupted text and let the generator restore the corrupted words about entities and relations into another formatted text\. When the pipeline works, the generator outputs a cluster of such texts via beam searching\(?, ?\), and then the filter scores and compares each text with a borderline\. The ones with a score above the borderline are accepted and mapped back to the final relation triplets\. Therefore, PiPMRE can not only extract an unknown number of relations into a fixed\-length text but also utilize semantic information discarded in most previous studies\.

Beyond structuralism, we learn the generator in two continuous stages, which first injects medical knowledge into the model with incremental cross\-domain pre\-training and further tunes it for our reformulated MRE by preference optimization\(?, ?\)\. We also follow contrastive learning\(?, ?\) and train the filter with pairwise margin loss, ensuring that it understands the meaning of the borderline score\. Through the generate\-filter workflow and individual component learning, PiPMRE can adaptively decide multiple relation triplets for a given medical text without designing complex tagging schema or concerning about triplets overlapping\.

Our main contributions are as follows:

- •We propose a novel pipeline approach for MRE, which exhibits advantages over the previous methods on multi\-relation extraction, as well as knowledge and semantic awareness\.
- •We propose learning paradigms tailored to the generator and filter of PiPMRE, respectively\.
- •Extensive experimental results on two public MRE datasets show that PiPMRE significantly outperforms the previous methods in both full\-data and few\-shot settings\. We also conduct ablation studies to test the key factors that affect PiPMRE’s performance\.

## Methodology

### Task Formulation

Given an MRE dataset𝒟=\{𝒳,𝒯,ℰ\}\\mathcal\{D\}=\\\{\\mathcal\{X\},\\mathcal\{T\},\\mathcal\{E\}\\\}, where theii\-th instance is a text \(or sentence\)xi∈𝒳x\_\{i\}\\in\\mathcal\{X\}containingnin\_\{i\}medical entitiese1,e2,⋯,eni∈ℰe\_\{1\},e\_\{2\},\\cdots,e\_\{n\_\{i\}\}\\in\\mathcal\{E\}, and thejj\-th pair of entitiesejs,ejoe^\{s\}\_\{j\},e^\{o\}\_\{j\}hasmjm\_\{j\}relationsr1,r2,⋯,rmj∈𝒯r\_\{1\},r\_\{2\},\\cdots,r\_\{m\_\{j\}\}\\in\\mathcal\{T\}, the task of MRE is to extract all triplet<ejs,rk,ejo\>,i∈\(0,\|𝒟\|\),j∈\(0,ni\),k∈\(0,mj\)<e^\{s\}\_\{j\},r\_\{k\},e^\{o\}\_\{j\}\>,i\\in\(0,\|\\mathcal\{D\}\|\),j\\in\(0,n\_\{i\}\),k\\in\(0,m\_\{j\}\)fromxix\_\{i\}\. Mathematically, this equals modeling the summed conditional probability:

∑i=1\|𝒟\|∑j=1ni∑k=1mjP\(<ejs,rk,ejo\>\|xi\)\.\\sum\_\{i=1\}^\{\|\\mathcal\{D\}\|\}\\sum\_\{j=1\}^\{n\_\{i\}\}\\sum\_\{k=1\}^\{m\_\{j\}\}\\mathrm\{P\}\(<e^\{s\}\_\{j\},r\_\{k\},e^\{o\}\_\{j\}\>\|x\_\{i\}\)\.\(1\)
In solving this problem, we do not enumerate all possible triples<ejs,rk,ejo\><e^\{s\}\_\{j\},r\_\{k\},e^\{o\}\_\{j\}\>like work\(?, ?\), which is trivial and memory\-consuming; also, we give up linearizing a triplet to a word sequence\(?, ?\) because its length varies on different instances\. Instead, we textualize an instance into natural language and then corrupt the resulting instance to convert MRE as a text\-infilling problem easily tackled with PLM\.

Triplet Textualization\.We use a simple language schema to express the relation triplet involved in MRE instances, where the entity type is also considered:

> The \{To\} \{Obj\} is the \{Rel\} of the \{Ts\} \{Sub\}\.

In this template,\{S​u​b\},\{O​b​j\}\\\{Sub\\\},\\\{Obj\\\}are subject and object entities,\{T​s\},\{T​o\}\\\{Ts\\\},\\\{To\\\}are the types of subject and object, respectively, and\{R​e​l\}\\\{Rel\\\}is their relation, e\.g\.,

> <<Congenital Hydrocephalus, pathological classification, Cerebrospinal fluid circulation\>\>→\\rightarrowThe disease cerebrospinal fluid circulation is the pathological classification of the disease congenital Hydrocephalus\.

Instance Reformat\.We then append the textualized triplet to the instance text\. Consequently, an MRE instance is written in a coherent language context, easily understood by humans and language models, e\.g\.,

> Most congenital hydrocephalus is non\-communicable hydrocephalus caused by obstruction of cerebrospinal fluid circulation\. The circulation of cerebrospinal fluid in the disease is the pathological classification of congenital hydrocephalus disease\.

Instance Corruption\.Finally, we make corruptions in the reformatted instance\. Following\(?, ?\), we replace the token spans located in the entities, entity types, and relation slots with distinct sentinel tokens while keeping the remaining content integral\. A processed instance like the below schema:

> \{Text\}\. The \[TO\] \[OBJ\] is the \[R\] of the \[TS\] \[SUB\]

where \[OBJ\], \[SUB\], \[TO\], \[TS\], and \[R\] are sentinel tokens\.

On the above foundations, we approach MRE as Seq2Seq text\-infilling, which conditions on a corrupted instancex~\\tilde\{x\}to predict a group of missed slotsyyaccording to the sentinel tokens indication\.yyis a structured text consisting of five slots with their contextual order inx~\\tilde\{x\}:

> \[TO\] \{To\} \[OBJ\] \{Obj\} \[R\] \{Rel\} \[TS\] \{Ts\} \[SUB\] \{Sub\}

![Refer to caption](https://arxiv.org/html/2609.02896v1/x2.png)Figure 2:The overview of our PiPMRE\.\{⋅\}\\\{\\cdot\\\}locates the slot to be filled, \[⋯\\cdots\] is a special token, and \[E\] is added to the text end\.
### Relation Generator

We start the PiPMRE generator from T5\(?, ?\) \- a Transformer model pre\-trained with Seq2Seq text\-infilling and has shown significant strength on extractive linguistic tasks\. Additionally, we perform an incremental cross\-domain pre\-training \(ICPT\) to inject medical knowledge into the model and then fine\-tune it toward our task requirements\.

Incremental pre\-training\.Given an entityeein the medical knowledge graph𝒢\\mathcal\{G\}, itsneighborsconsists of entities linkedeeby a direct relation andindirect neighborsℰ¯i\\bar\{\\mathcal\{E\}\}\_\{i\}linkeeby 2\-skip relations\. A corpus entailing medical knowledge is easily created on this assumption\. For eachei∈𝒢e\_\{i\}\\in\\mathcal\{G\}, we ask Llama3\.1\(?, ?\) to make a sentence conditioned on one of its neighborse¯i\\bar\{e\}\_\{i\}and one of its indirect neighborse~i\\tilde\{e\}\_\{i\}by instructing it with the following prompt:

> Considering thee¯i\\bar\{e\}\_\{i\}isr1r\_\{1\}ofeie\_\{i\}ande~i\\tilde\{e\}\_\{i\}isr2r\_\{2\}ofeie\_\{i\}, please make a sentence usingeie\_\{i\},e¯i\\bar\{e\}\_\{i\}ande~i\\tilde\{e\}\_\{i\}\.

We mask entity tokens in the text by probability 0\.8 while other tokens by 0\.2\. Then, we train the generator to restore the original text at the target end\.

In this work, we build our pre\-training corpus with CMeKG†††http://cmekg\.pcl\.ac\.cn/, which widely covers 6,310 diseases, 19,853 drugs, 1,237 diagnosis and treatment technologies, and more than 30 common relationships\. The final corpus contains 269,930 sentences, and the supervised cross\-entropy loss is used during the training process\.

Fine\-tuning\.During fine\-tuning, the biggest challenge is to preserve pre\-training knowledge\. Thanks to prompt tuning\(?, ?\) technology, which fine\-tunes a PLM while freezing all pre\-trained parameters, this problem is alleviated to a great extent\. Noting the generator intakes a corrupted instancex~\\tilde\{x\}and aims to output the target textyycontaining the slots missed inx~\\tilde\{x\}and joined by sentinel tokens\. It models the following probabilistic:

P​\(y\|x~;θ\)=∏t=1\|y\|P​\(yt\|x~,y<t;θ\)P​\(yt\|x~,y<t;θ\)=D​e​c​\(E​n​c​\(x~;θ\),y<t;θ\),\\begin\{aligned\} \\mathrm\{P\}\(y\|\\tilde\{x\};\\theta\)&=\\prod\_\{t=1\}^\{\|y\|\}\\mathrm\{P\}\(y\_\{t\}\|\\tilde\{x\},y\_\{<t\};\\theta\)\\\\ \\mathrm\{P\}\(y\_\{t\}\|\\tilde\{x\},y\_\{<t\};\\theta\)&=Dec\(Enc\(\\tilde\{x\};\\theta\),y\_\{<t\};\\theta\)\\end\{aligned\},\(2\)whereθ\\thetadenotes the generator’s parameters, andE​n​c,D​e​cEnc,Decare the generator’s encoder and decoder, respectively\. Following\(?, ?\), we insertnncontinuous soft tokens𝐜=c1,c2,⋯,cn\\mathbf\{c\}=c\_\{1\},c\_\{2\},\\cdots,c\_\{n\}at the beginning ofx~\\tilde\{x\}, and then Eq\.[2](https://arxiv.org/html/2609.02896#Sx2.E2)converts to:

P​\(y\|x~;θ\)→P​\(y\|c1:n,x~;θ,ϕ\),\\begin\{array\}\[\]\{c\}\\mathrm\{P\}\(y\|\\tilde\{x\};\\theta\)\\to\\mathrm\{P\}\(y\|c\_\{1:n\},\\tilde\{x\};\\theta,\\phi\)\\\\ \\end\{array\},\(3\)where

P​\(yt\|c1:n,x~,y<t;θ,ϕ\)=D​e​c​\(E​n​c​\(\[𝐜;x~\];θ\),y<t;θ,ϕ\),\\mathrm\{P\}\(y\_\{t\}\|c\_\{1:n\},\\tilde\{x\},y\_\{<t\};\\theta,\\phi\)=Dec\(Enc\(\[\\mathbf\{c\};\\tilde\{x\}\];\\theta\),y\_\{<t\};\\theta,\\phi\),\(4\)ϕ\\phidenotes the embeddings of𝐜\\mathbf\{c\}, which are learnable, and\[⋅;⋅\]\[\\cdot;\\cdot\]means texts connection\.

Besides, to meet the intricate medical scenario, where more than one relation triplet may be extracted from a text, we introduce an additional objective, which aims to maximize the probability of textsy\+∈𝒯x\+y^\{\+\}\\in\\mathcal\{T\}^\{\+\}\_\{x\}established by gold relation triplets while minimizing the onesy−∈𝒯x−y^\{\-\}\\in\\mathcal\{T\}^\{\-\}\_\{x\}built on fake triplets:

maxϕ⁡𝔼y\+,y−​\[P​\(y\+\|𝐜,x~;θ,ϕ\)−P​\(y−\|𝐜,x~;θ,ϕ\)\]\.\\max\_\{\\phi\}\\mathbb\{E\}\_\{y^\{\+\},y^\{\-\}\}\[\\mathrm\{P\}\(y^\{\+\}\|\\mathbf\{c\},\\tilde\{x\};\\theta,\\phi\)\-\\mathrm\{P\}\(y^\{\-\}\|\\mathbf\{c\},\\tilde\{x\};\\theta,\\phi\)\]\.\(5\)
After incremental pre\-training, we follow direct preference optimization \(DPO\)\(?, ?\) and continuously train the model to minimize the lossℒd​p​o​\(ϕ\)\\mathcal\{L\}\_\{dpo\}\(\\phi\):

−𝔼y\+,y−​\[log⁡σ​\(β​log⁡πϕ∗​\(y\+∣x~\)πθ∗​\(y\+∣x~\)−β​log⁡πϕ∗​\(y−∣x~\)πθ∗​\(y−∣x~\)\)\],\-\\underset\{y^\{\+\},y^\{\-\}\}\{\\mathbb\{E\}\}\[\\log\\sigma\\left\(\\beta\\log\\frac\{\\pi\_\{\\phi^\{\*\}\}\\left\(y^\{\+\}\\mid\\tilde\{x\}\\right\)\}\{\\pi\_\{\\theta^\{\*\}\}\\left\(y^\{\+\}\\mid\\tilde\{x\}\\right\)\}\-\\beta\\log\\frac\{\\pi\_\{\\phi^\{\*\}\}\\left\(y^\{\-\}\\mid\\tilde\{x\}\\right\)\}\{\\pi\_\{\\theta^\{\*\}\}\\left\(y^\{\-\}\\mid\\tilde\{x\}\\right\)\}\\right\)\],\(6\)whereπϕ\(⋅\|x~\)\\pi\_\{\\phi\}\(\\cdot\|\\tilde\{x\}\)is an abbr\. ofP\(⋅\|𝐜,x~;θ,ϕ\)\\mathrm\{P\}\(\\cdot\|\\mathbf\{c\},\\tilde\{x\};\\theta,\\phi\),θ∗\\theta^\{\*\}denotes a copy of model parameters learned with supervised fine\-tuning, andϕ∗\\phi^\{\*\}meansϕ\\phiupdated after back\-propagation\. The next section discusses the construction of fault target texts𝒯x−\\mathcal\{T\}\_\{x\}^\{\-\}\.

### Relation Filter

The PiPMRE filter is built upon BERT\(?, ?\) and estimates whether the generator infers correct triplet elements from a corrupted instance\. Formally, given an instance textxx, and the texty∗∼gϕ​\(x~\)y^\{\*\}\\sim g\_\{\\phi\}\(\\tilde\{x\}\)sampled from the generator outputsgϕ​\(⋅\)g\_\{\\phi\}\(\\cdot\), the function of the filter isf​\(x,y∗\)→s,s∈\(0,1\)f\(x,y^\{\*\}\)\\to s,s\\in\(0,1\)\. Like what has performed on the generator, we embedxxand the slots iny∗y^\{\*\}into a language template to match the filter’s pre\-training:

> \{Text\}\. The \{Obj\} \[M\] \(is or isn’t\) the \{Rel\} of the \{Sub\}\.

It is seen that we abandon the entity\-type slots iny∗y^\{\*\}, which are proven negligible in our preliminary experiments, and require the filter to predict the mask token \[M\] \-isorisn’t\. Practically, the filter models the probabilityP\(⋅\|T\[x,y∗\]\)\\mathrm\{P\}\(\\cdot\|T\[x,y^\{\*\}\]\), whereT​\[⋅,⋅\]T\[\\cdot,\\cdot\]means template infilling, rather than making an absolute binary choice between the two given words\. To bridge this gap, we sharpen the filter’s output distribution:

P^​\(vi\|T​\[x,y∗\]\)=exp⁡\(v¯i/τ\)∑kNexp⁡\(v¯k/τ\),\\mathrm\{\\hat\{P\}\}\(v\_\{i\}\|T\[x,y^\{\*\}\]\)=\\frac\{\\exp\\left\(\\bar\{v\}\_\{i\}/\\tau\\right\)\}\{\\sum\_\{k\}^\{N\}\\exp\\left\(\\bar\{v\}\_\{k\}/\\tau\\right\)\},\(7\)whereτ<1\\tau<1is a temperature andv¯i\\bar\{v\}\_\{i\}is the output of the filter’s second last layer corresponding to the wordviv\_\{i\}\. Furthermore, we letf​\(x,y∗\)=P^​\(i​s\|T​\[x,y∗\]\)f\(x,y^\{\*\}\)=\\mathrm\{\\hat\{P\}\}\(is\|T\[x,y^\{\*\}\]\)quantify the confidence of using the entity and relation slots\{S​u​b\}∗,\{R​e​l\}∗,\{O​b​j\}∗\\\{Sub\\\}^\{\*\},\\\{Rel\\\}^\{\*\},\\\{Obj\\\}^\{\*\}iny∗y^\{\*\}to compose a correct relation triplet that belongs toxx\.

Based on the above mentions, we learn the filter to provide reasonable ratings, depending on the validness of the generator output\. Feeding in an instancexx, the generator’s output text is either made up of elements in gold relation triplet or not\. We collect the former into𝒯x\+\\mathcal\{T\}\_\{x\}^\{\+\}and the later𝒯x−\\mathcal\{T\}\_\{x\}^\{\-\}\. Intuitively, a negative caseyi−∈𝒯x−y^\{\-\}\_\{i\}\\in\\mathcal\{T\}\_\{x\}^\{\-\}is easily fabricated by replacing one or more slots inyi\+∈𝒯x\+y^\{\+\}\_\{i\}\\in\\mathcal\{T\}\_\{x\}^\{\+\}with random tokens or peer tokens stick with entity/relation/entity type\. We train the filter with a cloze prompt tuning approach, i\.e\., inserting soft tokenst1,t2,⋯,tmt\_\{1\},t\_\{2\},\\cdots,t\_\{m\}\(with learnable embeddingsψ\\psi\) after the \{Text\} slot in the filter input text, also, a pair\-wise margin lossℒc​t​l​\(ψ\)\\mathcal\{L\}\_\{ctl\}\(\\psi\)is used inspired by contrastive learning\(?, ?\):

∑i=1\|𝒯x\+\|max⁡\(ζ−fψ​\(x,yi\+\),0\)\+max⁡\(fψ​\(x,yi−\)−ζ,0\)\.\\sum\_\{i=1\}^\{\|\\mathcal\{T\}\_\{x\}^\{\+\}\|\}\\max\(\\zeta\-f\_\{\\psi\}\(x,y^\{\+\}\_\{i\}\),0\)\+\\max\(f\_\{\\psi\}\(x,y^\{\-\}\_\{i\}\)\-\\zeta,0\)\.\(8\)Notably,𝒯x−\\mathcal\{T\}\_\{x\}^\{\-\}is dynamically derived from𝒯x\+\\mathcal\{T\}\_\{x\}^\{\+\}during training, depending on the instancexx, therefore, the two sets have an identical size\.ζ\\zetais a borderline parameter\.

### Pipeline Inference

The inference of PiPMRE is described in Fig\.[2](https://arxiv.org/html/2609.02896#Sx2.F2)\. Starting from a templated instance, the generator first produces a cluster of candidate outputs, then, the filter scores and compares each with a borderlineζ\\zetato decide its correctness\.

Modulated Decoding\.Due to the randomness of autoregressive generation, each token in the generated target text is conditionally sampled from the generator vocabulary, causing a risk of yielding invalid slots that fail to match any relation triplet\. To address this problem, we draw ideas of\(?, ?, ?\) and propose amodulated decodingmechanism\.

Given a corrupted instancex~∗\\tilde\{x\}^\{\*\}to the encoder and a starting token \[TO\] to the decoder, the generator adopts beam search \(BS\)\(?, ?\) to sampleKKcandidate target textsy1∗,y2∗,⋯,yK∗y^\{\*\}\_\{1\},y^\{\*\}\_\{2\},\\cdots,y^\{\*\}\_\{K\}\. In this process, the searching space of BS is constrained by a Trie\(?, ?\), where the possible next token in a branch can only be one of the children of the last visited node, depending on the pre\-given sets𝒯\\mathcal\{T\}andℰ\\mathcal\{E\}\. Taking the instance in Fig[2](https://arxiv.org/html/2609.02896#Sx2.F2), once the model has generated an incomplete phrase\[TO\] disease \[OBJ\], the next token must start an entity of the type disease; also, the next token of the phrase\[TO\] disease \[OBJ\] Cerebrospinal fluid \[R\] pathogenesis \[TS\]must bediseasebecause only a disease can be the object ofpathogenesis\. This strategy offers higher efficiency than searching on the entire vocabulary and ensures the validness of model outputs\(?, ?\)\.

Scoring and Filtering\.Finally, the filter scores each of the generator’s outputs:

si=fψ​\(x,yi∗\),i=1,⋯,K\.s\_\{i\}=f\_\{\\psi\}\(x,y^\{\*\}\_\{i\}\),i=1,\\cdots,K\.\(9\)Eachsis\_\{i\}is compared with the borderlineζ\\zeta\. Ifsi\>ζs\_\{i\}\>\\zeta, the triplet<\{Sub\}i∗,\{Rel\}i∗,\{Obj\}i∗\><\\\{Sub\\\}\_\{i\}^\{\*\},\\\{Rel\\\}\_\{i\}^\{\*\},\\\{Obj\\\}\_\{i\}^\{\*\}\>, is deemed established for the instancex∗x^\{\*\}\(\{⋅\}i∗\\\{\\cdot\\\}\_\{i\}^\{\*\}is the slot inyi∗y\_\{i\}^\{\*\}\); otherwise, it is abandoned\.

Table 1:Datasets Statistics\.Dataset\#Instance\#Relations\#RPITrainTestTrainTestTrainTestCHIP14,3393,58543,66010,6263\.042\.95CMeIE14,3393,58564,83516,3091\.144\.55

## Experiments

### Datasets

CMeIE\(?, ?\) andCHIP\(?, ?\) are used to evaluate our method\. Table[1](https://arxiv.org/html/2609.02896#Sx2.T1)reports their statistics\.

### Comparison Methods

We compare our PiPMRE with nine advanced RE methods scattered in three types\.Sequential tagging methods:NovelTagging\(?, ?\) proposes a BIEO\-R\-SO tagging schema for relation extraction\.CASREL\(?, ?\) extends the tagging label from a single numerical sequence to groups of binary sequences to cope with overlapped relations\.TP\-Linker\(?, ?\) extracts entities and relations simultaneously using a novel handshaking tagging strategy\.BiTT\(?, ?\) develop binary tree\-based tagging schema to capture tree\-like relation structure in medical texts\.Seq2Seq methods:GenIE\(?, ?\) generates linearized relation triplet from a given sentence, where special tokens<<sub\>\>,<<rel\>\>, and<<obj\>\>demarcate the start of a subject, relation, and object, respectively, and<<et\>\>demarcates the object end\.E\-REBEL\(?, ?\) uses<<triplet\>\>to mark the start of a relation triplet and<<sub\>\>and<<obj\>\>to split subject, object, and their relation\.PLM\-based methods:FPC\(?, ?\) transforms RE into MLM and fine\-tunes a BERT model with curriculum\-guided prompting\.TANL\(?, ?\) frames RE as a translation task based on augmented natural language\.GenPT\(?, ?\) converts RE to a Seq2Seq text\-denoising task, recovering the corrupted entities and their relation at the target end\.

Table 2:Evaluation results on CHIP test set\.Underlineresults are the previous best\.Boldresults are the best\.MethodBase ModelPrecisionRecallF1Sequential Tagging MethodsNovelTaggingBERT\-base74\.671\.071\.1CASRELBERT\-base80\.171\.871\.8TP\-LinkerBERT\-base83\.168\.577\.4BiTTBERT\-base82\.173\.284\.4Seq2Seq MethodsGenIEBART\-large82\.173\.280\.4E\-REBELREBEL\-large81\.468\.380\.9PLM\-based MethodsFPCRoBARTa\-large83\.168\.577\.4TANLT5\-large82\.474\.078\.0GenPTBART\-large89\.284\.087\.8PiPMRET5\-large89\.786\.188\.9
### Implementation Details

In our experiments, we implement the PiPMRE generator with T5\-large‡‡‡https://huggingface\.co/IDEA\-CCNL/Randeng\-T5\-784Mand implement the filter with BERT\-base§§§https://huggingface\.co/google\-bert/bert\-base\-chinese\. As for hyperparameters, we setγ=0\.2\\gamma=0\.2,β=0\.6\\beta=0\.6,τ=0\.2\\tau=0\.2andζ=0\.45\\zeta=0\.45\. Besides, we set the length of soft tokens used in prompt tuning to 20 for both the generator and filter\. When beam search\(?, ?\) is used, the beam widthKKis set to 16\. Our codes are built with Pytorch¶¶¶https://pytorch\.organd Huggingface Transformer∥∥∥https://huggingface\.co/modelslibraries, and 8/4 NVIDIA RTX 4090 GPUs are used to run model training/inference\. We incrementally pre\-train the PiPMRE generator for 10K steps on our collected corpus, the batch size is 32, and an AdamW\(?, ?\) optimizer with a learning rate 1e\-4 is used\. During fine\-tuning, we train the PiPMRE’s two components for at most 10 epochs\. We bind the AdamW optimizer with a linear learning schedule, where the initial learning rate is 2\.5e\-5 for the generator and 1e\-4 for the filter\. It warms up during the first 10% training steps and decays to 0 gradually in the subsequent steps\.

Following previous works, we take micro precision \(%\), recall \(%\), and F1 score \(%\) as the main metrics for model evaluation\. Besides, our experiments also compare the foundation model \(and its scale\) used in each method \(the tagger network in tagging methods, the Seq2Seq model used in Seq2Seq methods, and the language model used in PLM\-based methods\)\.

Table 3:Evaluation results on CMeIE test set\. Results with†\{\{\\dagger\}\}are reported in the original or previous papers, otherwise from our reproduction\.MethodParametersPrecisionRecallF1Sequential Tagging MethodsNovelTagging110M51\.4†17\.1†25\.6†CASREL110M53\.5†28\.2†37\.0†TP\-Linker110M52\.327\.738\.5BiTT110M55\.6†45\.5†50\.1†Seq2Seq MethodsGenIE406M51\.727\.738\.5E\-REBEL770M51\.319\.935\.9PLM\-based MethodsFPC355M52\.639\.344\.5TANL770M53\.333\.946\.0GenPT406M55\.643\.149\.6PiPMRE700M56\.746\.650\.3
### Main Results

Table[2](https://arxiv.org/html/2609.02896#Sx3.T2)and Table[3](https://arxiv.org/html/2609.02896#Sx3.T3)present the comprehensive evaluation results of our PiPMRE and the comparison methods\. On both datasets, BiTT and GenIE exhibited individually the best performance in their group\. GenPT outperformed them, and our PiPMRE further surpassed GenPT by 1\.1 points of F1 on CHIP and 0\.7 points of F1 on CMeIE\. We note that GenPT converts MRE as a Seq2Seq text\-denoising problem, similar to ours\. To some extent, this suggests the compatibility between the two tasks, indicating a promising direction for future studies\. On the other hand, the methods mentioned in our experiments can also be grouped according to their base model\. Taking the BERT group as an example, the success of BiTT indicated the importance of exploiting the semantic structure of medical text\. With BART\-largeas the backbone, GenPT showed significant superiorities over GenIE\. GenPT performed a task transformation that exactly meets the BART pre\-training purpose, i\.e\., text\-denoising\. On the contrary, GenIE only uses BART’s function of Seq2Seq generation, without matching pre\-training and downstream tasks or utilizing the model’s linguistic ability\. Therefore, PLM\-based approaches present an adequate idea that directs pre\-trained Transformers for RE\-like tasks\. Finally, our PiPMRE performed better than TANL\. Consider that T5 is pre\-trained with text\-infilling while TANL converts RE to a machine translation task\. This comparison highlights again the consistency of pre\-training and downstream tasks\.

Table 4:Evaluation results on CHIP and CMeIE test instances with single relation\.MethodCHIPCMeIEH@1H@3H@1H@3CASREL70\.1698\.8476\.2498\.47BiTT80\.4199\.0581\.2899\.82GenIE74\.8497\.9479\.3098\.03GenPT86\.8699\.0185\.6499\.34PiPMRE91\.0899\.6592\.1499\.39\- w/oDPO89\.1899\.0189\.4899\.31Table 5:Evaluation results on CHIP and CMeIE test instances with multiple relations\. F1 scores are reported\.MethodCHIPCMeIENEOEPOSEONEOEPOSEOCASREL81\.481\.180\.147\.145\.846\.1BiTT82\.783\.783\.647\.749\.450\.3GenIE79\.867\.174\.836\.342\.343\.1GenPT78\.667\.775\.437\.541\.842\.4PiPMRE85\.788\.087\.350\.049\.450\.2#### Single\-relation MRE

To make the evaluation of single\-relation MRE fair for various methods, we introduce a novel metric \- H@M\. Given an instance associated with a relation tripletrr, we can sample top\-MMcandidate triplets𝒞M\\mathcal\{C\}\_\{M\}from a RE model’s outputs according to estimated likelihood, and:

H​@​M=𝕀​\(r⊂ℛM\),H@M=\\mathbb\{I\}\(r\\subset\\mathcal\{R\}\_\{M\}\),\(10\)where𝕀​\(⋅\)\\mathbb\{I\}\(\\cdot\)is 1 if the condition in\(⋅\)\(\\cdot\)holds, otherwise 0\. Methods in Table[4](https://arxiv.org/html/2609.02896#Sx3.T4)performed closely evaluated by H@3\. However, when the metric converts to H@1, our PiPMRE showed obvious advantages\. We attribute this to the fine\-tuning of the PiPMRE generator, which enables the model to produce a more promising triplet by a higher probability\. The last row of Table[4](https://arxiv.org/html/2609.02896#Sx3.T4)proved our view\.

#### Multi\-relation MRE

? \(?, ?\) recognize multi\-relation RE into three scenarios: with a pair of entities overlapped \(EPO\), single entity overlapped \(SEO\), and no entity overlapped \(NEO\)\. Table[5](https://arxiv.org/html/2609.02896#Sx3.T5)reports the evaluation results of multi\-relation RE\*\*\*\*\*\*The CHIP/CMeIE test set contains 40/19 EOP instances and 2,054/3,000 SEO instances, respectively\.\. GenIE and GenPT showed weakness across all scenarios\. Although we set the maximum length of the output sequence to 256 tokens, they tend to generate incomplete sequences or invalid ones that fail to recover relation triplets due to therepetition problem\. In contrast, sequential tagging approaches showed more promising results\. However, in most cases, our PiPMRE surpassed the previous state\-of\-the\-art BiTT by a large margin, one reason for its advanced comprehensive performance\.

Table 6:Evaluation results of PiPMRE variants on CMeIE test set\. The declined scores are reported\. CTL: contrastive learning\.△\\triangle: we keep the original setting of this component\.GeneratorFilterPrecisionRecallF1\- w/oICPT△\\triangle↑\\uparrow1\.8↑\\uparrow1\.5↑\\uparrow0\.9\- w/oDPO△\\triangle↓\\downarrow2\.9↓\\downarrow1\.6↓\\downarrow2\.1△\\triangle\- w/oCTL↓\\downarrow3\.7↓\\downarrow4\.5↓\\downarrow3\.8

### Further Analysis

#### Ablation Study

To further analyze the importance of each technology used in building PiPMRE, we consider five variants of PiPMRE and compare their performance in Table[5](https://arxiv.org/html/2609.02896#Sx3.T5)\. When the filter is kept, the lack of DPO during the generator fine\-tuning process impacts the PiPMRE performance most, followed by the absence of incremental pre\-training\. Once we abandon contrastive learning for the filter, we assign label 1/0 to a textyi\+y\_\{i\}^\{\+\}/yi−y\_\{i\}^\{\-\}and supervised train the model with a cross\-entropy loss\. It greatly harms the PiPMRE performance because of inadequate scoring conducted on the generator outputs\.

#### Few\-shot Learning

We also test our method in few\-shot settings to follow previous studies\. We train PiPMRE \(its generator and filter\) and S3AAL\(?, ?\) on 1%∼\\sim50% training samples \(S3AAL learns a support set, especially for few\-shot MRE\)\. Notably, since prompt tuning \(PT\) is famous for advancement in few\-shot tasks, the PiPMRE variant that gives up PT in both components’ learning process \(referred to as PiPMRE\-FPFT\) is also considered\. The comparison results are displayed in Fig\.[3](https://arxiv.org/html/2609.02896#Sx3.F3), evaluated by the micro F1 score\. The original PiPMRE outperformed the varients without prompt tuning, no matter how many training samples were used\. On the other hand, PiPMRE performed close to S3AAL with less than 10% training samples, and the gap between the two methods increases with the increased number of training samples\. This is because S3AAL adapts only for few\-shot scenarios, and the support set it learned shows limited advantages given enough training data\.

![Refer to caption](https://arxiv.org/html/2609.02896v1/x3.png)\(a\)Few\-shot MRE on CHIP\.
![Refer to caption](https://arxiv.org/html/2609.02896v1/x4.png)\(b\)Few\-shot MRE on CMeIE\.

Figure 3:Few\-shot performance on CHIP \(a\) and CMeIE \(b\)\.

## Conclusion

This paper introduces a novel two\-stage pipeline for MRE, named PiPMRE\. PiPMRE adopts a Seq2Seq PLM to generate formatted texts containing grouped elements of relational triplet and a filter to validate the generation results\. In contrast to traditional approaches that carry on sequence\-based tagging, PiPMRE abandons the design of complicated tagging schema while adequately tackling medical texts containing multiple relation triplets\. Experimental results in diverse full\-data/few\-shot and single\-/multiple\-relation settings demonstrate the robustness of PiPMRE\.

## References