LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction

# LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction
Source: [https://arxiv.org/html/2607.23420](https://arxiv.org/html/2607.23420)
Xiao You1Tianwei Yan2Zixu Shan1Longyu Du1Shan Zhao1 1Hefei University of Technology, Hefei, China 2Chongqing Jiaotong University, Chongqing, China

###### Abstract

Large language models show strong promise for information extraction \(IE\), but existing reflection\-based correction methods are often misaligned with structured extraction outputs\. Free\-form self\-reflection can flag an error, yet it rarely identifies whether the failure is a missing span, wrong label, boundary mismatch, invalid relation type, or reversed argument order\. We introduce LA\-RL \(Label\-Aware Reflective Reinforcement Learning\), an outcome\-supervised framework that guides IE self\-correction with task\-grounded diagnostic labels\. A single backbone first predicts an extraction, diagnoses task\-specific error labels, and then revises its output conditioned on the diagnosis\. Training starts from diagnostic data labeled by an annotation model for cold\-start supervised fine\-tuning and proceeds through two GRPO stages that reward final extraction quality, format validity, and first\-pass correctness, without a process reward model\. Experiments on named entity recognition, relation extraction, and event extraction show consistent same\-backbone gains over SFT, including 6\.83 average F1 on SciER relation extraction, about 20 F1 on out\-of\-distribution relation extraction, and 14\.80 trigger F1 plus 17\.50 argument F1 on DuEE1\.0\. Ablations show that reflection structure is task\-sensitive: stronger constraints benefit relation extraction, whereas named entity recognition needs less restrictive correction under domain shift\.

LA\-RL: Label\-Aware Self\-Reflection for Reinforcement Learning in Information Extraction

Xiao You1Tianwei Yan2Zixu Shan1Longyu Du1Shan Zhao11Hefei University of Technology, Hefei, China2Chongqing Jiaotong University, Chongqing, China

## 1Introduction

Large language models have made rapid progress on open\-ended reasoning,from chain\-of\-thought prompting to reinforcement learning with verifiable rewards,yet a quieter failure persists: they still struggle to reliably extract structured facts from text\. When asked to identify entities, relations, or events in a sentence, even capable models produce systematic, recurring errors: missing spans, wrong labels, boundary mismatches, and reversed arguments\. Unlike free\-form generation, where an imperfect answer can still be useful, information extraction \(IE\) demands outputs that are simultaneously faithful to the input text and compliant with a predefined schema\. This dual constraint makes IE a revealing stress test for structured generation, and a natural setting in which to ask: can a model not only detect that its extraction is wrong, but identify which kind of error it has made, and use that diagnosis to correct itself?

![Refer to caption](https://arxiv.org/html/2607.23420v1/x1.png)Figure 1:Relation extraction example illustrating why free\-form reflection can be insufficient for IE\. A standard LLM output contains relation\-type, boundary, argument\-order, and missing\-relation errors; enabling free\-form thinking fixes only part of the output\. By contrast,LA\-RLexplicitly diagnoses these errors with task\-grounded labels and uses them to guide targeted revision\.Reflection and self\-correction have been widely studied as mechanisms for improving LLM outputs\. Prompting\-based methods encourage the model to produce free\-form rationales before answering\(Weiet al\.,[2022](https://arxiv.org/html/2607.23420#bib.bib169); Shinn and others,[2023](https://arxiv.org/html/2607.23420#bib.bib5); Madaan and others,[2023](https://arxiv.org/html/2607.23420#bib.bib4)\), while training\-based approaches use outcome or process rewards to cultivate reasoning behaviors via reinforcement learning\(Lightmanet al\.,[2023](https://arxiv.org/html/2607.23420#bib.bib6); Jaechet al\.,[2024](https://arxiv.org/html/2607.23420#bib.bib166); El\-Kishkyet al\.,[2025](https://arxiv.org/html/2607.23420#bib.bib165)\)\. Despite their success in mathematical reasoning and code generation, these methods share a common assumption: that reflection is best expressed as unconstrained natural language\. For structured IE, this assumption is limiting\. A free\-form rationale may note that "something is wrong" with an extracted triple, but it rarely pinpoints whether the failure is a missing relation, a wrong entity boundary, or a reversed argument order\. Without this diagnostic precision, the model is left to guess at a repair, and the correction step becomes unreliable\.

IE offers a distinctive opportunity to move beyond free\-form reflection\. Because its outputs follow explicit schemas, IE errors are not arbitrary but systematic and nameable, as shown in\(Weiet al\.,[2023](https://arxiv.org/html/2607.23420#bib.bib71); Xuet al\.,[2023](https://arxiv.org/html/2607.23420#bib.bib47)\)\. Figure[1](https://arxiv.org/html/2607.23420#S1.F1)illustrates this concretely\. A standard LLM produces an invalid relation type \(“from”\), truncates“Benjamin Raich”to“Benjamin,”reverses the arguments in \[“Italy”,“nationality”,“Giorgio Rocca”\], and misses two nationality relations entirely\. Enabling free\-form chain\-of\-thought corrects the boundary error but leaves the reversed relation untouched\. The deeper problem is structural: the model senses its output is imperfect, yet lacks a mechanism for targeting which error to fix—an invalid type, a reversed argument, a boundary mismatch, or a missing relation\. Diagnostic labels, by contrast, make the repair target explicit: fix the relation type, swap the arguments, expand the entity boundary, add the missing relations\. This observation motivates the central question of our work:Can IE self\-correction be organized as label\-guided diagnosis and revision, rather than as free\-form text generation?

We answer this question withLA\-RL\(Label\-AwareReflectiveReinforcementLearning\), a framework that structures reflection through a discrete, task\-grounded diagnostic state\. Rather than generating open\-ended rationales,LA\-RLinserts a diagnostic step between first\-pass extraction and final revision: the model first predicts an extraction, diagnoses one or more error labels from a predefined task\-specific set \(e\.g\.,Missing Relation,Wrong Entity Boundary,Reversed Entities\), and then revises its output conditioned on that diagnosis\. Training proceeds in three stages: \(1\) cold\-start supervised fine\-tuning on diagnostic data constructed with an annotation model that labels errors made by a base extractor; \(2\) a reflection\-oriented GRPO stage that rewards final extraction quality and format validity; and \(3\) a direct\-inference enhancement stage that additionally rewards first\-pass quality, encouraging the model to solve correctly as early as possible while preserving the diagnostic correction pathway\.LA\-RLdoes not train a separate process reward model; all rewards are outcome\-based and computed directly from extraction metrics\.

We evaluateLA\-RLon SciER\(Zhanget al\.,[2024](https://arxiv.org/html/2607.23420#bib.bib1)\), OOD RE datasets, and DuEE1\.0\(Liet al\.,[2020](https://arxiv.org/html/2607.23420#bib.bib2)\)using Qwen2\.5\-7B\-Instruct\.LA\-RLconsistently outperforms same\-backbone baselines: \+6\.83 F1 on SciER RE,∼\\sim\+20 F1 on OOD RE, and \+14\.80/\+17\.50 trigger/argument F1 on DuEE1\.0\. Ablations over Generic Reflection, Label\-only Correction, andLA\-RLshow that reflection structure is task\-sensitive: RE improves monotonically with stronger constraints, while NER gains no further benefit over label\-only correction under domain shift\. Reflection structure should thus be matched to task error complexity, not applied uniformly\.

#### Contributions\.

- •LA\-RL, a label\-aware reflective RL framework for structured IE self\-correction\. Rather than relying on free\-form rationales, LA\-RL inserts a discrete diagnostic state between first\-pass extraction and final revision, trained end\-to\-end with outcome\-supervised GRPO\.
- •A two\-stage RL training paradigm\.Beyond the conventional SFT cold\-start followed by a single direct RL phase, LA\-RL decomposes reinforcement learning into reflection\-oriented optimization and direct\-inference enhancement, showing that staged RL yields stable and cumulative improvements\.
- •Strong empirical gains\.Under the same Qwen2\.5\-7B\-Instruct backbone, LA\-RL achieves \+6\.83 F1 on SciER RE, \+∼\\sim20 F1 on OOD RE, and \+14\.80/17\.50 trigger/argument F1 on DuEE1\.0, outperforming SFT, GRPO, and R2GRPO\.

## 2Task Formulation

Given an input sentenceX=\(x1,x2,⋯,xl\)X=\(x\_\{1\},x\_\{2\},\\cdots,x\_\{l\}\), the goal of information extraction is to produce a structured outputYYthat captures task\-relevant facts expressed in the text\. Depending on the task,YYmay denote entity spans and types, relations among entities, or event structures composed of triggers and arguments\.

For named entity recognition \(NER\), the target output is a set of typed spansYNER=\{\(e,t\)\}Y\_\{\\text\{NER\}\}=\\\{\(e,t\)\\\}, whereeedenotes an entity span in the sentence andt∈𝒯t\\in\\mathcal\{T\}is its entity type\. For relation extraction \(RE\), the target output is a set of structured tuplesYRE=\{\(e1,r,e2\)\}Y\_\{\\text\{RE\}\}=\\\{\(e\_\{1\},r,e\_\{2\}\)\\\}, wherer∈ℛr\\in\\mathcal\{R\}is a relation type ande1,e2e\_\{1\},e\_\{2\}are its arguments\. For event extraction \(EE\), the target output is a set of event structuresYEE=\{\(g,c,A\)\}Y\_\{\\text\{EE\}\}=\\\{\(g,c,A\)\\\}, whereggis an event trigger,c∈𝒞c\\in\\mathcal\{C\}is an event type, andA=\{\(ai,ρi\)\}i=1mA=\\\{\(a\_\{i\},\\rho\_\{i\}\)\\\}\_\{i=1\}^\{m\}contains argument spansaia\_\{i\}with their rolesρi\\rho\_\{i\}\.

We formulate all these IE tasks as constrained text generation\. Given a task instructionℐ\\mathcal\{I\}and an input sentenceXX, the model generates a task\-specific structured outputYYin a predefined format\. Under this formulation, different IE tasks share the same high\-level inference process, while differing only in their output schema and evaluation criteria\.

This unified formulation supports LA\-RL: since structured outputs can be checked for fine\-grained errors, task\-specific diagnostic labels can organize reflection\. In our experiments, NER, RE, and EE differ in output formats, error labels, and reward signals\.

## 3Label\-Aware Self\-Reflection for Reinforcement Learning

LA\-RL instantiates structured reflection by inserting a task\-grounded diagnostic state between the first\-pass extraction and the revised extraction\. The diagnostic state is a discrete set of error labels, such as missing entities, wrong boundaries, missing relations, or reversed arguments, rather than an unrestricted natural\-language rationale\. We train this behavior in two phases: first constructing label\-annotated error data for cold\-start SFT, and then applying two\-stage outcome\-supervised reinforcement learning\. The diagnostic labels can be interpreted as a structured reflection space, but they are used here as an intermediate representation inside a standard sequence\-generation policy rather than as a new RL problem formulation\.

![Refer to caption](https://arxiv.org/html/2607.23420v1/x2.png)Figure 2:Overview of LA\-RL\. The first\-pass extraction is followed by a task\-grounded diagnostic state, which structures the reflection step before final revision\.### 3\.1Stage 1: Diagnostic Data Construction

As summarized in Algorithm[1](https://arxiv.org/html/2607.23420#alg1), to construct diagnostic supervision, we use a two\-step inference pipeline for structured IE tasks\. In the first step, we formulate standardized instruction\-style prompts for the target task and apply Qwen2\.5\-7B\-Instruct to perform direct inference on raw texts, obtaining initial structured predictions\. As the model is not yet adapted to the task, the generated outputs contain many erroneous instances\. These errors provide the raw material for defining task\-grounded diagnostic states\.

In the second step, we feed each input instance, its gold output, and the corresponding initial prediction into a larger language model, DeepSeek\-V3\. The annotator compares the initial prediction against the gold output and returns one or more labels from a predefined error set in a structured format\. Because a single prediction may contain multiple structural errors simultaneously, we allow the annotator to output an error\-label set rather than forcing a single dominant label\. The resulting dataset consists of input instances, gold outputs, initial predictions, and diagnostic label sets, which supports cold\-start learning of the diagnosis\-to\-revision pattern\.

Algorithm 1Diagnostic Error\-Label Generation1:Training set

𝒟train=\{\(xi,yi\)\}i=1N\\mathcal\{D\}\_\{\\mathrm\{train\}\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}
2:Base IE model

ℳbase\\mathcal\{M\}\_\{\\mathrm\{base\}\}, annotator model

ℳanno\\mathcal\{M\}\_\{\\mathrm\{anno\}\}
3:Error type set

ℰ\\mathcal\{E\}
4:Diagnostic training set

𝒟diag\\mathcal\{D\}\_\{\\mathrm\{diag\}\}
5:Initialize

𝒟diag←∅\\mathcal\{D\}\_\{\\mathrm\{diag\}\}\\leftarrow\\emptyset
6:foreach

\(xi,yi\)∈𝒟train\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{D\}\_\{\\mathrm\{train\}\}do

7:

y^i←ℳbase​\(xi\)\\hat\{y\}\_\{i\}\\leftarrow\\mathcal\{M\}\_\{\\mathrm\{base\}\}\(x\_\{i\}\)⊳\\trianglerightInitial prediction

8:if

y^i≠yi\\hat\{y\}\_\{i\}\\neq y\_\{i\}then

9:

ℰi←ℳanno​\(xi,yi,y^i,ℰ\)\\mathcal\{E\}\_\{i\}\\leftarrow\\mathcal\{M\}\_\{\\mathrm\{anno\}\}\(x\_\{i\},y\_\{i\},\\hat\{y\}\_\{i\},\\mathcal\{E\}\)⊳\\trianglerightLabel set

10:

𝒟diag←𝒟diag∪\{\(xi,yi,y^i,ℰi\)\}\\mathcal\{D\}\_\{\\mathrm\{diag\}\}\\leftarrow\\mathcal\{D\}\_\{\\mathrm\{diag\}\}\\cup\\\{\(x\_\{i\},y\_\{i\},\\hat\{y\}\_\{i\},\\mathcal\{E\}\_\{i\}\)\\\}
11:endif

12:endforreturn

𝒟diag\\mathcal\{D\}\_\{\\mathrm\{diag\}\}

### 3\.2Stage 2: Outcome\-Supervised Reflective RL

After constructing diagnostic supervision, we optimize the policy in two sequential stages that correspond to the ability\-oriented blocks in Figure[2](https://arxiv.org/html/2607.23420#S3.F2)\. The first stage improves the model’sReflective Ability: the model learns to diagnose extraction errors and use diagnostic labels to revise the final answer\. The second stage improves the model’sIntegrated Ability: the model learns to combine stronger first\-pass extraction with the same diagnostic correction pathway\. Both stages use outcome rewards computed from extraction quality and format validity; no process reward model is trained\.

#### Stage I: Improving Reflective Ability\.

We initialize LA\-RL from an SFT cold start trained on the diagnostic data\. This initialization alleviates reward sparsity and gives the model a basic ability to map extraction errors to one or more diagnostic labels and revise its own predictions under explicit supervision\. Starting from this checkpoint, we perform GRPO training with a reward that emphasizes the quality of the final structured answer after diagnostic revision\. In this stage, the reward is evaluated exclusively at the final answer: the first reasoning output and the intermediate diagnostic labels are not directly rewarded\. Specifically, letrnerr\_\{\\mathrm\{ner\}\}andrrer\_\{\\mathrm\{re\}\}denote the NER and RE F1 scores of the final prediction, and letrfmtr\_\{\\mathrm\{fmt\}\}denote the format reward\. We first define the task reward as

rtask=λner​rner\+λre​rre,r\_\{\\mathrm\{task\}\}=\\lambda\_\{\\mathrm\{ner\}\}r\_\{\\mathrm\{ner\}\}\+\\lambda\_\{\\mathrm\{re\}\}r\_\{\\mathrm\{re\}\},\(1\)where\(λner,λre\)=\(1\.0,2\.0\)\(\\lambda\_\{\\mathrm\{ner\}\},\\lambda\_\{\\mathrm\{re\}\}\)=\(1\.0,2\.0\)\. The Stage\-I reward is then

rI=rtask\+λfmt​rfmt,r\_\{\\mathrm\{I\}\}=r\_\{\\mathrm\{task\}\}\+\\lambda\_\{\\mathrm\{fmt\}\}r\_\{\\mathrm\{fmt\}\},\(2\)whereλfmt=0\.5\\lambda\_\{\\mathrm\{fmt\}\}=0\.5\. This stage primarily trains the model to use diagnostic reflection to improve the final structured output\.

#### Stage II: Improving Integrated Ability\.

We then continue RL from the Stage\-I checkpoint and augment the reward to explicitly value the quality of the first reasoning output, i\.e\., the first\-pass extraction before diagnostic revision\. The final\-answer reward from Stage I is retained, and an additional first\-pass reward is added\. Letrre\(1\)r\_\{\\mathrm\{re\}\}^\{\(1\)\}denote the RE F1 score of the first\-pass output\. The Stage\-II reward is defined as

rII=rI\+λfirst​rre\(1\),r\_\{\\mathrm\{II\}\}=r\_\{\\mathrm\{I\}\}\+\\lambda\_\{\\mathrm\{first\}\}r\_\{\\mathrm\{re\}\}^\{\(1\)\},\(3\)whereλfirst=1\.0\\lambda\_\{\\mathrm\{first\}\}=1\.0\. Thus, unlike Stage I, the reward is evaluated at both the first reasoning output and the final answer\. This integrated reward encourages the model to solve the task correctly as early as possible, while still preserving the diagnostic reflection step when the initial prediction is imperfect\.

#### Optimization Objective\.

We optimize both stages with Group Relative Policy Optimization \(GRPO\)\(Shaoet al\.,[2024](https://arxiv.org/html/2607.23420#bib.bib144)\), which avoids training a separate value model and fits our outcome\-supervised reward design\. For each queryxx, we sample a group of outputs\{o1,…,oG\}\\\{o\_\{1\},\\ldots,o\_\{G\}\\\}from the old policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}and compute the corresponding rewards according to Eq\. \([2](https://arxiv.org/html/2607.23420#S3.E2)\) or Eq\. \([3](https://arxiv.org/html/2607.23420#S3.E3)\)\. Letρi=πθ​\(oi∣x\)πθold​\(oi∣x\)\\rho\_\{i\}=\\frac\{\\pi\_\{\\theta\}\(o\_\{i\}\\mid x\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(o\_\{i\}\\mid x\)\}denote the policy ratio\. The policy is updated by maximizing

𝒥​\(θ\)\\displaystyle\\mathcal\{J\}\(\\theta\)=1G∑i=1G\[min\(ρiAi,clip\(ρi,1−ϵ,1\+ϵ\)Ai\)\\displaystyle=\\frac\{1\}\{G\}\\sum\_\{i=1\}^\{G\}\\Bigl\[\\min\\\!\\bigl\(\\rho\_\{i\}A\_\{i\},\\mathrm\{clip\}\(\\rho\_\{i\},1\-\\epsilon,1\+\\epsilon\)A\_\{i\}\\bigr\)−βDKL\(πθ∥πref\)\],\\displaystyle\\qquad\\qquad\{\}\-\\beta D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\pi\_\{\\theta\}\\,\\\|\\,\\pi\_\{\\mathrm\{ref\}\}\\right\)\\Bigr\],\(4\)whereAiA\_\{i\}is the normalized group\-relative advantage:

Ai=ri−mean​\(\{r1,…,rG\}\)std​\(\{r1,…,rG\}\)\.A\_\{i\}=\\frac\{r\_\{i\}\-\\mathrm\{mean\}\(\\\{r\_\{1\},\\ldots,r\_\{G\}\\\}\)\}\{\\mathrm\{std\}\(\\\{r\_\{1\},\\ldots,r\_\{G\}\\\}\)\}\.\(5\)A KL regularizer keeps the updated policy close to the reference model from the previous stage, which stabilizes training while allowing the policy to gradually internalize the diagnostic reflection behavior\.

![Refer to caption](https://arxiv.org/html/2607.23420v1/figs/GRPO.png)Figure 3:Group Relative Policy Optimization \(GRPO\) process for optimizing the diagnostic reflective policy with outcome rewards\.

## 4Experiments

Table 1:Test F1 scores of different baselines on SciER and OOD setting\. “Rel” and “Rel\+” represent relation extraction under boundary\-level and strict evaluation, respectively\. The model trained by R2GRPO is used as the starting base model for LA\-RL training, and the R2GRPO results are obtained from our reproduction\.SciEROODMethodsNERRelRel\+NERRelRel\+Supervised BaselinesPUREZhong and Chen \([2020](https://arxiv.org/html/2607.23420#bib.bib100)\)81\.6053\.2752\.6771\.9950\.4449\.46PL\-MarkerYeet al\.\([2021](https://arxiv.org/html/2607.23420#bib.bib102)\)83\.3160\.0659\.2473\.9359\.0256\.68HGEREYanet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib101)\)86\.8562\.3261\.1081\.3261\.3158\.32Zero\-Shot LLMs\-based BaselinesQwen2\.5\-7B\-Instruct42\.4212\.0711\.0945\.1914\.8314\.53DeepSeek\-V355\.8122\.6321\.9457\.9820\.9220\.22DeepSeek\-R171\.2128\.7428\.6167\.0228\.0427\.44Kimi\-K2\-Thinking63\.1226\.5826\.3566\.6934\.2233\.91Kimi\-K256\.2123\.2223\.0765\.6930\.0329\.66Qwen3\-Max64\.9126\.2926\.1465\.9232\.6132\.64Qwen3\-Max\-Thinking65\.3328\.0527\.8665\.6131\.8331\.67Gemini\-2\.573\.3332\.0131\.6764\.6126\.3226\.07Fine\-tuned LLMsGRPO70\.8737\.0234\.3859\.9527\.2723\.99R2GRPOLiet al\.\([2025](https://arxiv.org/html/2607.23420#bib.bib99)\)82\.8161\.4960\.3066\.7038\.8037\.76OursSFT82\.9759\.5857\.5665\.4136\.8135\.44LA\-RLI84\.2564\.6263\.0868\.8746\.6245\.86LA\-RLII86\.9166\.2664\.5375\.3456\.6856\.03### 4\.1Datasets

Our evaluation spans multiple representative information extraction settings\. For the joint named entity recognition \(NER\) and relation extraction \(RE\) setting, we conduct the primary evaluation on the SciER benchmark and its associated out\-of\-distribution \(OOD\) datasetsZhanget al\.\([2024](https://arxiv.org/html/2607.23420#bib.bib1)\)\. SciER is a widely used scientific IE benchmark comprising 106 research papers annotated with approximately 25k entity mentions and 12k relational instances\. To further examine the applicability of the proposed framework beyond the primary NER–RE setting, we additionally evaluate it on DuEE1\.0Liet al\.\([2020](https://arxiv.org/html/2607.23420#bib.bib2)\), a large\-scale Chinese event extraction benchmark constructed from real\-world scenarios and comprising approximately 13k instances annotated with 16k events and 33k arguments\. Detailed dataset statistics are reported in Tables[5](https://arxiv.org/html/2607.23420#A1.T5)and[6](https://arxiv.org/html/2607.23420#A1.T6)\.

### 4\.2Experiment Setup

Training SettingsBase ModelAll our fine\-tuning experiments are conducted by adapting the Qwen2\.5\-7B\-Instruct modelYanget al\.\([2024](https://arxiv.org/html/2607.23420#bib.bib164)\)\.

For the NER/RE experiments, all training settings use Qwen2\.5\-7B\-Instruct as the backbone\. GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2607.23420#bib.bib144)\)denotes direct reinforcement learning with task\-level rewards, and R2GRPO\(Liet al\.,[2025](https://arxiv.org/html/2607.23420#bib.bib99)\)denotes our reproduced Relevance and Rule\-Induction Group Relative Policy Optimization baseline\. For our methods, SFT is obtained by performing cold\-start SFT on the reproduced R2GRPO\-trained model; LA\-RL is then initialized from this cold\-started model: LA\-RL \(Stage I\) is the intermediate model after the first reinforcement learning stage, and LA\-RL \(Stage II\) is the final model after the second stage\. For the EE experiments, LA\-RL is trained directly from Qwen2\.5\-7B\-Instruct rather than from the R2GRPO\-trained model\. Following the R2GRPO setup, the RL stages use 1,000 samples selected from the training set for reinforcement learning; the EE experiments follow the same protocol and also use 1,000 sampled training instances\.

The system prompts used for joint NER/RE and event extraction \(EE\) training follow a unified reflection format with task\-specific diagnostic labels\. We provide the full prompts in Appendix[D](https://arxiv.org/html/2607.23420#A4)\(Figures[4](https://arxiv.org/html/2607.23420#A4.F4)and[5](https://arxiv.org/html/2607.23420#A4.F5)\)\.

Implementation DetailsDetailed hardware, LoRA, SFT/RL hyperparameters, and inference settings are provided in Appendix[B](https://arxiv.org/html/2607.23420#A2)\.

Evaluation MetricsFor Named Entity Recognition \(NER\) and Relation Extraction \(Rel and Rel\+\), following R2GRPOLiet al\.\([2025](https://arxiv.org/html/2607.23420#bib.bib99)\), we report the standard micro F1\-score\. NER: An entity is correct if its span and type match a gold entity\. Rel: A relation is correct if the types and spans of both entities and the relation type match a gold relation\. Rel\+: It further requires the entity type is correct in the triples\. For event extraction \(EE\), we report precision, recall, and F1 for trigger identification \(Trig\-I\), trigger classification \(Trig\-C\), argument identification \(Arg\-I\), and argument classification \(Arg\-C\)\. Trig\-I evaluates trigger span matching, while Trig\-C additionally requires the event type to be correct\. Arg\-I evaluates argument span identification, and Arg\-C additionally requires the predicted argument role to match the gold annotation\.

### 4\.3Results and Analysis

#### Main results\.

Table[1](https://arxiv.org/html/2607.23420#S4.T1)reports F1 scores for both NER and RE on the SciER test set and the OOD setting, with RE evaluated under both boundary\-level \(Rel\) and strict \(Rel\+\) settings\. We use this table primarily to compare training strategies under the same Qwen2\.5\-7B\-Instruct backbone, while also listing supervised and zero\-shot systems for context\.

Within the same\-backbone fine\-tuned setting, LA\-RL improves over SFT from 59\.58/57\.56 to 66\.26/64\.53 on SciER Rel/Rel\+, and from 36\.81/35\.44 to 56\.68/56\.03 on OOD Rel/Rel\+\. It also improves over the pure GRPO baseline trained directly from the base model, which supports the practical role of cold\-start SFT before outcome\-supervised RL\. Compared with Stage I, Stage II further improves relation extraction, suggesting that adding first\-pass quality to the outcome reward helps the model preserve direct extraction ability while still using reflection when needed\. The substantially larger OOD gain \(\+19\.87 Rel F1\) relative to the in\-distribution gain \(\+6\.68 Rel F1\) suggests that label\-aware diagnostic reflection provides stronger generalization benefits under domain shift, where SFT alone is more prone to overfitting to the training distribution\. On NER, LA\-RL Stage II also improves over SFT by \+3\.94 F1 on SciER, a gain that is more modest than on RE—a pattern we examine further in the ablation where NER shows a different sensitivity to reflection structure than RE\.

LA\-RL is competitive with strong supervised systems on SciER and substantially stronger than zero\-shot prompting baselines in this setting, but our central conclusion comes from controlled same\-backbone comparisons and the structured\-reflection ablation in Table[2](https://arxiv.org/html/2607.23420#S4.T2)\.

Table 2:Ablation on structured reflection designs\. Generic Reflection removes explicit error\-type labels and performs free\-form second\-pass correction\. Label\-only Correction uses explicit error labels but removes the full reflective diagnosis process\. All variants are evaluated after the second training stage\.SciEROODMethodsNERRelRel\+NERRelRel\+SFT82\.9759\.5857\.5665\.4136\.8135\.44Generic Reflection83\.6862\.7261\.6975\.2656\.2055\.76Label\-only Correction83\.6664\.3062\.9276\.2155\.7855\.33LA\-RL86\.9166\.2664\.5375\.3456\.6856\.03
#### Ablation on Structured Reflection\.

Table[2](https://arxiv.org/html/2607.23420#S4.T2)is the core evidence for the reflection\-space structure hypothesis\. The variants form a spectrum\. Generic Reflection keeps a second\-pass correction step but removes explicit diagnostic labels\. Label\-only Correction exposes error labels but does not require the full diagnosis\-to\-revision trajectory\. LA\-RL uses both: the model identifies task\-specific diagnostic labels and conditions the final revision on them during training\.

The results generally follow this spectrum\. On SciER, SFT obtains 59\.58/57\.56 on Rel/Rel\+, Generic Reflection improves to 62\.72/61\.69, Label\-only Correction further improves to 64\.30/62\.92, and LA\-RL reaches 66\.26/64\.53\. On OOD relation extraction, Generic Reflection already brings a large gain over SFT, and LA\-RL gives the best Rel/Rel\+ scores among the reflection variants\. The only exception is OOD NER, where Label\-only Correction is slightly higher than LA\-RL\. We therefore avoid claiming that more structure is uniformly better across every metric; instead, the ablation supports a more modest conclusion that relation extraction improves most consistently when free\-form reflection is replaced by a task\-grounded diagnosis\-to\-revision path\.

Generic Reflection resembles ordinary self\-correction: the second pass may help, but the reflection is not anchored to a specific IE failure mode\. Label\-only Correction shows that diagnostic labels are useful, but labels alone do not fully capture the benefit\. LA\-RL performs best on the main relation metrics because it couples the diagnostic state with the revision step under outcome\-supervised RL\. In this sense, the label set acts as a lightweight way to structure the reflection space, not as a process reward model or a claim about optimal reasoning traces\.

Table 3:Error\-label distribution on the SciER test and OOD relation extraction settings\.Error LabelSciEROODCountRatioCountRatioMissing relation50236\.86%13038\.24%Wrong entity/relation types41830\.69%9327\.35%Wrong entity boundary17512\.85%4112\.06%Undefined relation00\.00%10\.29%Correct entities, incorrect relation25818\.94%7221\.18%Reversed entities90\.66%30\.88%

#### Error Analysis\.

Table[3](https://arxiv.org/html/2607.23420#S4.T3)summarizes the diagnostic error\-label distribution on the SciER test and OOD relation extraction settings\. Missing relations are the most frequent errors in both settings, accounting for 36\.86% on SciER test and 38\.24% on OOD, followed by wrong entity/relation types\. Correct entities with incorrect relations also form a substantial portion of errors, while undefined relations and reversed entities are rare\. This distribution suggests that relation extraction failures are dominated by missing relational facts and schema\-alignment mistakes, motivating an explicit label\-aware diagnosis step before revision\.

Table 4:Event extraction results on DuEE1\.0 across different training stages\. Qwen2\.5\-7B\-Instruct denotes zero\-shot inference without task\-specific training\. GRPO denotes pure GRPO training directly from the base model, without SFT cold\-start initialization\. LA\-RL \(Stage I\) and LA\-RL \(Stage II\) refer to the models obtained after the first and second reinforcement learning stages, respectively\. Best results are shown in bold\.Trig\-ITrig\-CArg\-IArg\-CMethodPRF1PRF1PRF1PRF1Qwen2\.5\-7B\-Instruct52\.1733\.7140\.9545\.7529\.5635\.9121\.9916\.5118\.8619\.4914\.5816\.68SFT65\.3954\.8059\.6359\.9750\.2554\.6834\.4230\.0432\.0831\.6227\.5929\.47GRPO63\.6261\.1962\.3858\.3756\.1457\.2333\.1035\.9034\.4530\.6033\.1931\.84LA\-RLI66\.3061\.5863\.8660\.6956\.3758\.4535\.2536\.0135\.6333\.0333\.6833\.35LA\-RLII76\.7672\.2474\.4374\.4370\.0572\.1847\.6851\.6349\.5846\.4350\.2748\.27

#### Generalization to Event Extraction\.

To test whether the same recipe can be instantiated beyond relation extraction, we further evaluate on the DuEE1\.0 event extraction benchmark\. Event extraction has a different output structure, involving triggers and argument\-role assignments, but it still admits task\-specific diagnostic labels\. As shown in Table[4](https://arxiv.org/html/2607.23420#S4.T4), SFT substantially improves over zero\-shot Qwen2\.5\-7B\-Instruct, and LA\-RL further improves over SFT across trigger identification, trigger classification, argument identification, and argument classification\. Stage II gives the best F1 on all four metric groups\. We interpret these results as evidence that the diagnostic\-state design is not limited to one relation extraction table, while noting that a broader set of IE tasks would be needed to fully characterize its scope\.

Additional qualitative case studies, including a reflection\-based correction example and examples where the first\-stage prediction is already correct, are provided in Appendix[F](https://arxiv.org/html/2607.23420#A6)\.

## 5Related Work

#### Generative Information Extraction

Information extraction has traditionally used supervised NER and RE models, such as LSTM\-CRF, dependency\-tree\-based methods, and cross\-task architectures, to capture local, syntactic, and interactive signalsLampleet al\.\([2016](https://arxiv.org/html/2607.23420#bib.bib86)\); Tanget al\.\([2018](https://arxiv.org/html/2607.23420#bib.bib87)\); Zhaoet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib9)\)\. Recent generative Transformer\-based approaches instead formulate IE as text generation, with systems such as Crop, MCL\-NER, and zero\-shot ChatGPT\-based extraction showing the promise of LLMsYanget al\.\([2022](https://arxiv.org/html/2607.23420#bib.bib34)\); Moet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib77)\); Weiet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib71)\)\. Nevertheless, generative IE remains limited when applied directly without task\-specific fine\-tuning\.

#### Error Samples and Error Labels

Error samples provide useful supervision because model\-generated mistakes often expose systematic weaknesses\. Early representation learning and pretraining showed that learning from incorrect or contrasting alternatives can sharpen model discriminationMikolovet al\.\([2013](https://arxiv.org/html/2607.23420#bib.bib21)\); Devlinet al\.\([2019](https://arxiv.org/html/2607.23420#bib.bib22)\)\. Recent work extends this idea by categorizing LLM hallucinations, integrating error analysis into reasoning, and using related signals in NER, RE, and consistency\-based negative\-sample selectionSimhiet al\.\([2024](https://arxiv.org/html/2607.23420#bib.bib23)\); Luet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib24)\); Wanget al\.\([2025](https://arxiv.org/html/2607.23420#bib.bib8)\); Yanet al\.\([2025](https://arxiv.org/html/2607.23420#bib.bib7)\); Moet al\.\([2024](https://arxiv.org/html/2607.23420#bib.bib3)\)\. Building on these studies, we model recurring IE errors as task\-grounded labels, enabling labeled error samples to guide reflection and improve task\-specific extraction\.

#### RL in LLM

Reinforcement learning \(RL\) is a key paradigm for aligning large language models \(LLMs\)\. RLHF established the supervised fine\-tuning, reward modeling, and PPO pipeline, but remains costly and unstable due to its reliance on human annotations and reward optimizationOuyanget al\.\([2022](https://arxiv.org/html/2607.23420#bib.bib15)\); Afzaliet al\.\([2024](https://arxiv.org/html/2607.23420#bib.bib16)\)\. Recent variants reduce these costs through AI\-generated preferences, direct preference optimization, group\-based policy updates, or rule\-verifiable rewardsLeeet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib17)\); Rafailovet al\.\([2023](https://arxiv.org/html/2607.23420#bib.bib18)\); Dinget al\.\([2025](https://arxiv.org/html/2607.23420#bib.bib19)\); Guoet al\.\([2025](https://arxiv.org/html/2607.23420#bib.bib20)\)\.

## 6Conclusion

This paper studies how to structure reflection for outcome\-supervised information extraction\. Instead of unconstrained long rationales, LA\-RL places a task\-grounded discrete diagnostic state between first\-pass extraction and final revision, combining annotation\-model diagnostic labels, cold\-start SFT, and two\-stage GRPO with outcome rewards without a separate process reward model\.

Experiments show consistent same\-backbone gains over SFT and pure GRPO\. Ablations indicate that relation extraction benefits from structured diagnosis\-to\-revision rather than free\-form self\-correction, and DuEE1\.0 results show the same recipe can extend to event extraction, though broader task coverage is needed for stronger generalization claims\. Overall, reflection\-space structure is an important design dimension for reflective LLM training\.

## 7Limitations

Despite its contributions, this study has the following limitations: \(1\) although we evaluate LA\-RL on representative IE tasks including NER, RE, and EE, broader settings such as document\-level extraction, low\-resource IE, and multilingual IE remain underexplored; \(2\) LA\-RL currently relies on manually defined diagnostic labels, and future work can explore automatic methods for generating diverse sets of error\-label categories to facilitate transfer across different IE tasks; \(3\) our experiments validate LA\-RL using only a 7B\-parameter backbone model, and whether the method remains similarly effective when scaled to larger backbone models requires further exploration\.

## References

- Aligning visual contrastive learning models via preference optimization\.arXiv preprint arXiv:2411\.08923\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px3.p1.1)\.
- J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova \(2019\)Bert: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\),pp\. 4171–4186\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- F\. Ding, B\. Wang, Z\. Zeng, and Y\. Wang \(2025\)Multi\-layer grpo: enhancing reasoning and self\-correction in large language models\.arXiv preprint arXiv:2506\.04746\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px3.p1.1)\.
- A\. El\-Kishky, A\. Wei, A\. Saraiva, B\. Minaiev, D\. Selsam, D\. Dohan, F\. Song, H\. Lightman, I\. Clavera, J\. Pachocki,et al\.\(2025\)Competitive programming with large reasoning models\.arXiv preprint arXiv:2502\.06807\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p2.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px3.p1.1)\.
- A\. Jaech, A\. Kalai, A\. Lerer, A\. Richardson, A\. El\-Kishky, A\. Low, A\. Helyar, A\. Madry, A\. Beutel, A\. Carney,et al\.\(2024\)Openai o1 system card\.arXiv preprint arXiv:2412\.16720\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p2.1)\.
- G\. Lample, M\. Ballesteros, S\. Subramanian, K\. Kawakami, and C\. Dyer \(2016\)Neural architectures for named entity recognition\.InNAACL HLT 2016,pp\. 260–270\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px1.p1.1)\.
- H\. Lee, S\. Phatale, H\. Mansoor, K\. R\. Lu, T\. Mesnard, J\. Ferret, C\. Bishop, E\. Hall, V\. Carbune, and A\. Rastogi \(2023\)Rlaif: scaling reinforcement learning from human feedback with ai feedback\.Preprint\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px3.p1.1)\.
- R\. Li, S\. Di, Y\. Liu, C\. Jing, Y\. Qiu, and L\. Chen \(2025\)Beyond path selection: better llms for scientific information extraction with mimicsft and relevance and rule\-induced\(r2\)grpo\.External Links:2505\.22068,[Link](https://arxiv.org/abs/2505.22068)Cited by:[§4\.2](https://arxiv.org/html/2607.23420#S4.SS2.p2.4),[§4\.2](https://arxiv.org/html/2607.23420#S4.SS2.p5.1),[Table 1](https://arxiv.org/html/2607.23420#S4.T1.5.1.1)\.
- X\. Li, F\. Li, L\. Pan, Y\. Chen, W\. Peng, Q\. Wang, Y\. Lyu, and Y\. Zhu \(2020\)DuEE1\.0: a large\-scale dataset for chinese event extraction in real\-world scenarios\.InCCF International Conference on Natural Language Processing and Chinese Computing,pp\. 534–545\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p5.1),[§4\.1](https://arxiv.org/html/2607.23420#S4.SS1.p1.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2023\)Let’s verify step by step\.arXiv preprint arXiv:2305\.20050\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p2.1)\.
- Q\. Lu, B\. Qiu, L\. Ding, L\. Xie, and D\. Tao \(2023\)Error analysis prompting enables human\-like translation evaluation in large language models: a case study on chatgpt\.Preprints\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- A\. Madaanet al\.\(2023\)Self\-refine: iterative refinement with self\-feedback\.arXiv preprint arXiv:2303\.17651\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p2.1)\.
- T\. Mikolov, I\. Sutskever, K\. Chen, G\. S\. Corrado, and J\. Dean \(2013\)Distributed representations of words and phrases and their compositionality\.Advances in neural information processing systems26\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- Y\. Mo, J\. Liu, J\. Yang, Q\. Wang, S\. Zhang, J\. Wang, and Z\. Li \(2024\)C\-icl: contrastive in\-context learning for information extraction\.arXiv preprint arXiv:2402\.11254\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- Y\. Mo, J\. Yang, J\. Liu, Q\. Wang, R\. Chen, J\. Wang, and Z\. Li \(2023\)MCL\-ner: cross\-lingual named entity recognition via multi\-view contrastive learning\.CoRRabs/2308\.09073\.External Links:[Link](https://doi.org/10.48550/arXiv.2308.09073),[Document](https://dx.doi.org/10.48550/ARXIV.2308.09073),2308\.09073Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px1.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px3.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px3.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§3\.2](https://arxiv.org/html/2607.23420#S3.SS2.SSS0.Px3.p1.4),[§4\.2](https://arxiv.org/html/2607.23420#S4.SS2.p2.4)\.
- N\. Shinnet al\.\(2023\)Reflexion: language agents with verbal reinforcement learning\.arXiv preprint arXiv:2303\.11366\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p2.1)\.
- A\. Simhi, J\. Herzig, I\. Szpektor, and Y\. Belinkov \(2024\)Distinguishing ignorance from error in llm hallucinations\.arXiv preprint arXiv:2410\.22071\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- B\. Tang, J\. Hu, X\. Wang, and Q\. Chen \(2018\)Recognizing continuous and discontinuous adverse drug reaction mentions from social media using lstm\-crf\.Wireless Communications and Mobile Computing2018\.Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px1.p1.1)\.
- C\. Wang, S\. Zhao, T\. Yan, S\. Song, W\. Ma, K\. Liu, and M\. Wang \(2025\)Hierarchical label\-enhanced contrastive learning for chinese ner\.IEEE Transactions on Neural Networks and Learning Systems\(\),pp\. 1–11\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2025.3528416)Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p2.1)\.
- X\. Wei, X\. Cui, N\. Cheng, X\. Wang, X\. Zhang, S\. Huang, P\. Xie, J\. Xu, Y\. Chen, M\. Zhang, Y\. Jiang, and W\. Han \(2023\)Zero\-shot information extraction via chatting with chatgpt\.CoRRabs/2302\.10205\.External Links:[Link](https://doi.org/10.48550/arXiv.2302.10205),[Document](https://dx.doi.org/10.48550/ARXIV.2302.10205),2302\.10205Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p3.1),[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px1.p1.1)\.
- D\. Xu, W\. Chen, W\. Peng, C\. Zhang, T\. Xu, X\. Zhao, X\. Wu, Y\. Zheng, and E\. Chen \(2023\)Large language models for generative information extraction: A survey\.CoRRabs/2312\.17617\.External Links:[Link](https://doi.org/10.48550/arXiv.2312.17617),[Document](https://dx.doi.org/10.48550/ARXIV.2312.17617),2312\.17617Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p3.1)\.
- T\. Yan, S\. Zhao, M\. Hu, M\. Wang, X\. Zhang, Z\. Luo, and M\. Wang \(2025\)HCL: a hierarchical contrastive learning framework for zero\-shot relation extraction\.IEEE Transactions on Neural Networks and Learning Systems36\(3\),pp\. 5694–5705\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2024.3379527)Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px2.p1.1)\.
- Z\. Yan, S\. Yang, W\. Liu, and K\. Tu \(2023\)Joint entity and relation extraction with span pruning and hypergraph neural networks\.arXiv preprint arXiv:2310\.17238\.Cited by:[Table 1](https://arxiv.org/html/2607.23420#S4.T1.7.9.1)\.
- A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.\(2024\)Qwen2\. 5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§4\.2](https://arxiv.org/html/2607.23420#S4.SS2.p1.1)\.
- J\. Yang, S\. Huang, S\. Ma, Y\. Yin, L\. Dong, D\. Zhang, H\. Guo, Z\. Li, and F\. Wei \(2022\)CROP: zero\-shot cross\-lingual named entity recognition with multilingual labeled sequence translation\.InFindings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7\-11, 2022,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),pp\. 486–496\.External Links:[Link](https://doi.org/10.18653/v1/2022.findings-emnlp.34),[Document](https://dx.doi.org/10.18653/V1/2022.FINDINGS-EMNLP.34)Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px1.p1.1)\.
- D\. Ye, Y\. Lin, P\. Li, and M\. Sun \(2021\)Packed levitated marker for entity and relation extraction\.arXiv preprint arXiv:2109\.06067\.Cited by:[Table 1](https://arxiv.org/html/2607.23420#S4.T1.7.8.1)\.
- Q\. Zhang, Z\. Chen, H\. Pan, C\. Caragea, L\. J\. Latecki, and E\. Dragut \(2024\)SciER: an entity and relation extraction dataset for datasets, methods, and tasks in scientific documents\.arXiv preprint arXiv:2410\.21155\.Cited by:[§1](https://arxiv.org/html/2607.23420#S1.p5.1),[§4\.1](https://arxiv.org/html/2607.23420#S4.SS1.p1.1)\.
- S\. Zhao, M\. Hu, Z\. Cai, and F\. Liu \(2023\)Dynamic modeling cross\-modal interactions in two\-phase prediction for entity\-relation extraction\.IEEE Transactions on Neural Networks and Learning Systems34\(3\),pp\. 1122–1131\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2021.3104971)Cited by:[§5](https://arxiv.org/html/2607.23420#S5.SS0.SSS0.Px1.p1.1)\.
- Z\. Zhong and D\. Chen \(2020\)A frustratingly easy approach for joint entity and relation extraction\.arXiv preprint arXiv:2010\.12812\.Cited by:[Table 1](https://arxiv.org/html/2607.23420#S4.T1.7.7.1)\.

## Appendix ADataset Statistics

For relation extraction, we use the SciER benchmark as the primary dataset together with several out\-of\-distribution test sets; their detailed entity and relation distributions are summarized in Table[5](https://arxiv.org/html/2607.23420#A1.T5)\.

Table 5:Statistics of the SciER benchmark and OOD test sets used for relation extractionEntity/Relation TypeTrainDevSciER TestOOD TestTotalEntity TypesMethod1142415491890101815881DATASET3220269370833942TASK33974166881944695Total1804122342948129524518Relation TypesPART\-OF18652143041112494USED\-FOR23983435461673454EVALUATED\-WITH86378131491121SYNONYM\-OF88076170891215COMPARE\-WITH875175114541218SUBCLASS\-OF697114176731060BENCHMARK\-FOR551648528728SUBTASK\-OF21031659315TRAINED\-WITH40437352478Total87431132162658212083For event extraction, we use the DuEE1\.0 benchmark; its detailed split statistics are summarized in Table[6](https://arxiv.org/html/2607.23420#A1.T6)\.

Table 6:Statistics of the DuEE1\.0 benchmark used for event extraction\. Since the test sets for DuEE1\.0 are not open\-sourced, we use the valid sets as our evaluation data\.MetricTrainDevTotalInstances11908149213400Events13860178315643Arguments28905368232587
## Appendix BImplementation Experiment Details

All experiments are conducted using the PyTorch deep learning framework on a single NVIDIA A800 GPU\. To ensure stable training and decoding, we carefully select the experimental settings and hyperparameters\. The inference\-stage hyperparameters used throughout all experiments are summarized in Table[7](https://arxiv.org/html/2607.23420#A2.T7), the supervised fine\-tuning \(SFT\) hyperparameters are summarized in Table[8](https://arxiv.org/html/2607.23420#A2.T8), and the reinforcement learning \(RL\) hyperparameters are summarized in Table[9](https://arxiv.org/html/2607.23420#A2.T9)\.

Table 7:Inference\-stage hyperparameters used throughout all experiments\.Inference\-stage parameterValueMax sequence length8192Top\-pp1\.0Top\-kk1Temperature0Table 8:SFT\-stage hyperparameters used in LA\-RL training\.SFT\-stage parameterValuelearning\_rate1×10−41\\times 10^\{\-4\}optimadamw\_8bitweight\_decay0\.01lr\_scheduler\_typelinearnum\_train\_epochs2per\_device\_train\_batch\_size1gradient\_accumulation\_steps8lora\_rank16lora\_alpha32Table 9:RL\-stage hyperparameters used in LA\-RL training\.RL\-stage parameterValuelearning\_rate5×10−65\\times 10^\{\-6\}adam\_beta10\.9adam\_beta20\.99weight\_decay0\.1warmup\_ratio0\.1temperature1\.0lr\_scheduler\_typecosineoptimadamw\_8bitnum\_train\_epochs2max\_grad\_norm0\.1lora\_rank64lora\_alpha128per\_device\_train\_batch\_size16num\_generations16gradient\_accumulation\_steps1For the inference stage, we use a maximum sequence length of 8192 with deterministic decoding, setting top\-ppto 1\.0, top\-kkto 1, and temperature to 0 throughout all experiments in this paper\. This deterministic setup reduces sampling noise and helps ensure stable and reproducible evaluation\.

For the SFT stage, we fine\-tune the model for two epochs using 8\-bit AdamW with a linear learning\-rate scheduler, a learning rate of1×10−41\\times 10^\{\-4\}, and weight decay of 0\.01\. We use LoRA adapters with rank 16 and scaling factor 32, set the per\-device training batch size to 1, and accumulate gradients for 8 steps before each optimizer update\.

For the RL stage, we optimize the policy for two epochs using 8\-bit AdamW with cosine learning\-rate decay, a warmup ratio of 0\.1, a sampling temperature of 1\.0, and LoRA adapters with rank 64 and scaling factor 128\. We use a per\-device batch size of 16, sample 16 generations for each prompt during policy optimization, and set the gradient accumulation steps to 1\. The standalone GRPO baseline in the main paper uses the same RL\-stage hyperparameters, but it is trained directly from Qwen2\.5\-7B\-Instruct and does not use the SFT checkpoint as a cold start\.

For event extraction, we adopt the same two\-stage RL design as in the NER/RE setting, but use event\-specific evaluation signals\. Letrtrigr\_\{\\mathrm\{trig\}\}andrargr\_\{\\mathrm\{arg\}\}denote the trigger and argument F1 scores of the final prediction, respectively, and letrfmtr\_\{\\mathrm\{fmt\}\}denote the format reward\. We first define the EE task reward as

rtaskEE=λtrig​rtrig\+λarg​rarg,r\_\{\\mathrm\{task\}\}^\{\\mathrm\{EE\}\}=\\lambda\_\{\\mathrm\{trig\}\}r\_\{\\mathrm\{trig\}\}\+\\lambda\_\{\\mathrm\{arg\}\}r\_\{\\mathrm\{arg\}\},\(6\)where\(λtrig,λarg\)=\(1\.0,2\.0\)\(\\lambda\_\{\\mathrm\{trig\}\},\\lambda\_\{\\mathrm\{arg\}\}\)=\(1\.0,2\.0\), since argument prediction is the more difficult and structurally informative part of EE\.

The Stage\-I reward is then

rIEE=rtaskEE\+λfmt​rfmt,r\_\{\\mathrm\{I\}\}^\{\\mathrm\{EE\}\}=r\_\{\\mathrm\{task\}\}^\{\\mathrm\{EE\}\}\+\\lambda\_\{\\mathrm\{fmt\}\}r\_\{\\mathrm\{fmt\}\},\(7\)whereλfmt=0\.5\\lambda\_\{\\mathrm\{fmt\}\}=0\.5\.

In Stage II, we further encourage stronger direct inference by adding the first\-pass argument F1\. Letrarg\(1\)r\_\{\\mathrm\{arg\}\}^\{\(1\)\}denote the argument F1 score of the first\-pass output\. The Stage\-II reward is defined as

rIIEE=rIEE\+λfirst​rarg\(1\),r\_\{\\mathrm\{II\}\}^\{\\mathrm\{EE\}\}=r\_\{\\mathrm\{I\}\}^\{\\mathrm\{EE\}\}\+\\lambda\_\{\\mathrm\{first\}\}r\_\{\\mathrm\{arg\}\}^\{\(1\)\},\(8\)whereλfirst=1\.0\\lambda\_\{\\mathrm\{first\}\}=1\.0\.

## Appendix CSupplementary Experiments

#### Zero\-shot Evaluation on Unseen RE Datasets\.

We include a small zero\-shot evaluation in which the model is trained on SciER and directly evaluated, without target\-domain fine\-tuning, on CoNLL04 and NYT10\. Table[10](https://arxiv.org/html/2607.23420#A3.T10)shows that LA\-RL outperforms the Qwen2\.5\-7B\-Instruct baseline across both datasets\. We treat this result as a supporting observation that the learned reflection behavior can transfer to some unseen RE datasets, while leaving broader cross\-domain generalization to future work\.

Table 10:Zero\-shot relation extraction performance on unseen datasets\. Best results are shown in bold\.ModelCoNLL04NYT10Qwen2\.5\-7B\-Instruct0\.33790\.1159LA\-RL0\.35500\.1347

## Appendix DPrompt

We provide the system prompts used for task\-specific reflective training\. Figure[4](https://arxiv.org/html/2607.23420#A4.F4)shows the prompt for joint named entity recognition \(NER\) and relation extraction \(RE\), and Figure[5](https://arxiv.org/html/2607.23420#A4.F5)shows the corresponding prompt for event extraction \(EE\)\.

System PromptRespond in the following format:
<think\>
Provide step\-by\-step reasoning to solve the task based on the given instructions and sentence\. Because the joint NER and RE extraction task is difficult, you need to try one derivation in your thinking and output the result in JSON format as\{"ner": \[\[entity, entity type\], \.\.\.\], "rel": \[\[subject, relation, object\], \.\.\.\]\}\. The reasoning process must include a reflection step: if an error occurs, categorize it under one or more common error labels for the current task and revise your reasoning accordingly\. For NER, use \[Missing entities\], \[Wrong entity types\], \[Wrong entity boundary\], \[Undefined entity types\], or \[Spurious entities\]\. For RE, use \[Missing relation\], \[Wrong entity/relation types\], \[Wrong entity boundary\], \[Undefined relation\], \[Correct entities, incorrect relation\], or \[Reversed entities\]\. If no error is found, explicitly return an empty label set \{\}\.
</think\>
<answer\>
Provide the final answer in JSON format as specified in the instruction\.
</answer\>Figure 4:System prompt used in LA\-RL training for joint named entity recognition and relation extraction with task\-specific NER and RE error labels\.System PromptRespond in the following format:
<think\>
Provide step\-by\-step reasoning to solve the task based on the given instructions and sentence\. Because the EE task is difficult, you need to try one derivation in your thinking and output the result\. The reasoning process must include a reflection step: if an error occurs, categorize it under one of the common error labels \(\[Correct trigger, incorrect arguments\], \[Missing trigger\], \[Spurious trigger\], \[Wrong trigger boundary\], \[Wrong event type\], \[Missing argument\], \[Spurious argument\], \[Wrong argument boundary\], \[Wrong argument role\], \[Argument attached to wrong event\]\) and revise your reasoning accordingly\. If no error is found, explicitly return an empty label set \{\}\.
</think\>
<answer\>
Provide the final answer in JSON format as specified in the instruction\.
</answer\>Figure 5:System prompt used in LA\-RL training for event extraction with task\-specific EE error labels\.
## Appendix ELabel Definition Explanation

![Refer to caption](https://arxiv.org/html/2607.23420v1/x3.png)Figure 6:Relation extraction error label set used for multi\-label diagnosis\. The RE setting uses a six\-label set\.Figure[6](https://arxiv.org/html/2607.23420#A5.F6)illustrates the relation extraction error label set and the corresponding definitions used for multi\-label diagnosis in our method\. For the Relation Extraction \(RE\) task, we use a six\-label set for diagnosing common error sources:Missing relation,Wrong entity/relation types,Wrong entity boundary,Undefined relation,Correct entities, incorrect relation, andReversed entities\. When multiple structural errors co\-occur in a single prediction, the diagnosis may contain more than one label from this set\. Each label in the set is immediately followed by an explanation of its definition\.

Table[11](https://arxiv.org/html/2607.23420#A5.T11)reports the empirical distribution of individual label assignments in the RE multi\-label diagnosis dataset\.Missing relationis the most frequently assigned label, accounting for more than half of all label assignments\.

Table 11:Distribution of individual label assignments in the RE multi\-label diagnosis dataset\. Ratios are rounded to two decimals\.RE error labelCountRatioMissing relation697550\.83%Wrong entity/relation types314022\.88%Wrong entity boundary156011\.37%Correct entities, incorrect relation12959\.44%Undefined relation4603\.35%Reversed entities2932\.14%For the Event Extraction \(EE\) task, Table[12](https://arxiv.org/html/2607.23420#A5.T12)summarizes the ten\-label set used for multi\-label diagnosis of trigger\- and argument\-level errors\.

Table 12:Event extraction error label set used in our multi\-label diagnosis framework\.EE error labelDefinitionMissing triggerA gold event trigger is not predicted by the model\.Spurious triggerThe model predicts a trigger that does not correspond to any gold event mention\.Wrong trigger boundaryThe trigger is detected, but its span boundary is incorrect\.Wrong event typeThe trigger span is matched, but the assigned event type is incorrect\.Correct trigger, incorrect argumentsThe trigger prediction is correct, but the associated argument set is wrong\.Missing argumentA gold argument of an event is omitted\.Spurious argumentThe model predicts an extra argument that is not supported by the gold annotation\.Wrong argument boundaryThe argument role is predicted on the wrong span boundary\.Wrong argument roleThe argument span is identified, but its semantic role is assigned incorrectly\.Argument attached to wrong eventAn argument is predicted for an incorrect trigger or linked to the wrong event instance\.

## Appendix FCase Study

In this appendix, we provide additional qualitative case studies for LA\-RL, covering both reflection\-based correction and cases in which the first\-stage prediction is already correct\.

### F\.1Reflection\-Based Correction

Figure[7](https://arxiv.org/html/2607.23420#A6.F7)presents a representative example in which the initial prediction contains structured errors that are corrected during reflection\.

![Refer to caption](https://arxiv.org/html/2607.23420v1/x4.png)Figure 7:A representative case in which label\-aware reflection corrects missing relations and entity\-boundary errors\.We use the following input sentence:

“We first evaluate the robustness of LSTM, BERT, and BERT NOPT on binary sentiment analysis using the Yelp dataset \(Zhang et al\., 2015\)\.”

This sentence contains multiple entity types, including methods \(LSTM, BERT, BERT NOPT\), a task \(binary sentiment analysis\), and a dataset \(Yelp\), together with several implicit semantic relations\.

In the first\-stage inference, the model produces the following initial reasoning result:

“the result of the reasoning may be: \[\[’LSTM’, ’Used\-For’, ’binary sentiment analysis’\], \[’BERT’, ’Used\-For’, ’binary sentiment analysis’\], \[’BERT NOPT’, ’Used\-For’, ’binary sentiment analysis’\], \[’Yelp dataset’, ’Benchmark\-For’, ’binary sentiment analysis’\]\]”

Although the model correctly identifies major entities and some relations, this output exhibits two common errors\. First, it fails to extract the implicit Evaluated\-With relations between methods and the dataset, corresponding to a Missing relation error\. Second, it uses “Yelp dataset” instead of the canonical entity “Yelp,” resulting in a Wrong entity boundary error\.

In the second stage, the model performs label\-aware reflection by explicitly diagnosing and categorizing these errors, as indicated by:

“I find that the reasoning may contain errors… ’Missing relation’: …, ’Wrong entity boundary’: …”

Guided by the identified error types, the model revises its reasoning and generates the final structured output:

“ner”: \[\["LSTM", "Method"\], \["BERT", "Method"\], \["BERT NOPT", "Method"\], \["binary sentiment analysis", "Task"\], \["Yelp", "Dataset"\]\]

“rel”: \[\["LSTM", "Used\-For", "binary sentiment analysis"\], \["BERT", "Used\-For", "binary sentiment analysis"\], \["BERT NOPT", "Used\-For", "binary sentiment analysis"\], \["Yelp", "Benchmark\-For", "binary sentiment analysis"\], \["LSTM", "Evaluated\-With", "Yelp"\], \["BERT", "Evaluated\-With", "Yelp"\], \["BERT NOPT", "Evaluated\-With", "Yelp"\]\]

Compared with the initial inference, the final result recovers all missing Evaluated\-With relations and normalizes the dataset entity boundary\. This example illustrates how the two\-stage label\-aware reflective mechanism enables effective self\-correction and leads to more complete and consistent extraction results\.

### F\.2Correct First\-Stage Inference

Figure[8](https://arxiv.org/html/2607.23420#A6.F8)shows an additional example in which the first\-stage direct inference is already correct\.

![Refer to caption](https://arxiv.org/html/2607.23420v1/x5.png)Figure 8:An example of correct direct inference in the first stage\.This case study analyzes a scenario in which the proposed two\-stage label\-aware reflective framework produces a correct extraction result during the first\-stage direct inference, demonstrating that the reflection mechanism does not introduce unnecessary modifications when the initial reasoning is accurate\.

The input sentence is:

“Here we see that constraints imposed by GS \- EC make it superior than GS \- GR in terms of retrieving words that are coherent with the context\.”

This sentence describes a comparative relationship between two methods, GS \- EC and GS \- GR, explicitly indicating their relative performance\.

During the first\-stage direct inference, the model generates the following reasoning result:

“the result of the reasoning may be: \[\[’GS \- EC’, ’Compare\-With’, ’GS \- GR’\]\]”

The model correctly identifies both entities as Method types and assigns the appropriate semantic relation Compare\-With\. Importantly, the subsequent error categorization step reports no detected errors across all predefined categories, including Missing relation, Wrong entity boundary, and Incorrect relation type, indicating high confidence in the initial prediction\.

Based on this assessment, the model preserves the original reasoning and outputs the final structured result as:

“ner”: \[\["GS \- EC", "Method"\], \["GS \- GR", "Method"\]\]

“rel”: \[\["GS \- EC", "Compare\-With", "GS \- GR"\]\]

This case highlights an important property of the proposed framework: the reflection stage acts as a conditional correction mechanism rather than an obligatory revision step\. When the direct inference result is already consistent with label definitions and relation semantics, the model maintains the original output without introducing spurious changes\.

Overall, this example demonstrates that the two\-stage label\-aware reflective reinforcement learning framework is both corrective and conservative\. It not only enables effective self\-correction in the presence of errors but also preserves correct predictions, ensuring robustness and stability in end\-to\-end information extraction tasks\.