DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models

# DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models
Source: [https://arxiv.org/html/2607.26891](https://arxiv.org/html/2607.26891)
###### Abstract

Sequence labeling is a fine\-grained information extraction task, yet existing large language model\-based approaches suffer from insufficient domain alignment and low inference efficiency\. To address these issues, we propose DIRECT, a framework that addresses these issues through training\-time optimization and inference\-time rectification\. Specifically, DIRECT performs Direct Preference Optimization \(DPO\) after supervised fine\-tuning to strengthen task alignment with human preferences, and introduces a controlled decoding process that enforces fixed output formats and restricts predictions to candidate sets\. To further improve efficiency, a template\-filling mechanism requires the model to generate only label tokens while reusing prefixed content through the KV Cache, thus reducing redundant computation\. Experimental results on eight datasets demonstrate that DIRECT achieves significant improvements in both performance and efficiency compared to existing methods\.

Index Terms—Large Language Models, Few\-shot Learning, Sequence Labeling

## 1Introduction

Sequence labeling is a fine\-grained information extraction \(IE\) task that covers sub\-tasks such as named entity recognition \(NER\), word segmentation, and part\-of\-speech \(POS\) tagging\. It is crucial for other downstream natural language processing \(NLP\) tasks, such as co\-reference resolution\[[1](https://arxiv.org/html/2607.26891#bib.bib2)\], knowledge graph\[[5](https://arxiv.org/html/2607.26891#bib.bib3)\], and question generation\[[15](https://arxiv.org/html/2607.26891#bib.bib4)\]\.

Currently, large language models \(LLMs\), endowed with extensive knowledge bases and powerful language modeling capabilities, have been widely applied across a broad range of tasks\[[12](https://arxiv.org/html/2607.26891#bib.bib23),[19](https://arxiv.org/html/2607.26891#bib.bib24),[22](https://arxiv.org/html/2607.26891#bib.bib25)\]\. In the context of IE under low\-resource scenarios, recent studies have primarily focused on fine\-tuning open\-source LLMs to enhance their performance, which can be broadly categorized into two approaches: \(1\) Multi\-task generative frameworks: for example, UIE\[[9](https://arxiv.org/html/2607.26891#bib.bib20)\]resolves the challenge of heterogeneous task formats and limited transferability by unifying all IE tasks into a text\-to\-structure generative framework; USM\[[8](https://arxiv.org/html/2607.26891#bib.bib21)\]addresses the limitations of generative unification in terms of discriminative accuracy and cross\-task sharing by reformulating IE as a semantic matching problem; and InstructUIE\[[18](https://arxiv.org/html/2607.26891#bib.bib5)\]further eliminates the need for task\-specific fine\-tuning in UIE by introducing an integrated framework for universal IE\. \(2\) Auxiliary information\-driven fine\-tuning: for example, GNER\[[2](https://arxiv.org/html/2607.26891#bib.bib7)\]alleviates boundary ambiguity in generative NER by introducing negative instances during training; GoLLIE\[[16](https://arxiv.org/html/2607.26891#bib.bib6)\]improves model consistency in zero\-shot IE by fine\-tuning LLMs to explicitly follow detailed annotation guidelines; and C\-ICL\[[10](https://arxiv.org/html/2607.26891#bib.bib8)\]enhances few\-shot learning performance by incorporating both correct examples and constructed hard negatives into in\-context demonstrations for contrastive learning\.

Despite the progress achieved by the aforementioned methods, several challenges remain:\(1\) Insufficient domain\-specific alignment\.In strict sequence labeling tasks, existing approaches rely solely on specifying task formats and output structures through instructions during fine\-tuning, which cannot guarantee that the model outputs strictly align with the expected results\.\(2\) Low inference efficiency\.Most existing studies focus on training and fine\-tuning LLMs, while still adopting the original autoregressive process during inference, leading to inefficiency in model decoding\.

![Refer to caption](https://arxiv.org/html/2607.26891v1/x1.png)Fig\. 1:The overall architecture of DIRECT\.To address the aforementioned challenges, we propose an innovative approach namedDPO\-basedInferenceRECTification \(DIRECT\)\.To enhance the domain\-specific alignment of LLMs,we perform Direct Preference Optimization \(DPO\) training after supervised fine\-tuning \(SFT\), thereby improving the model’s understanding of sequence labeling tasks and aligning its behavior more closely with human preferences\. In addition, during inference, we intervene in the generation process by enforcing a fixed output format and restricting the generated content to a predefined candidate set\. This dual mechanism of training and inference substantially improves the alignment of LLMs in domain\-specific tasks\.To improve inference efficiency, we introduce a template\-filling mechanism in which the model generates only the label components to be predicted, while the remaining parts are prefilled through the template with full reuse of the KV Cache\. This approach avoids redundant generation and significantly enhances decoding efficiency\.

Our contributions can be summarized as follows:

\(1\)Domain\-adaptive alignment optimization\.By incorporating DPO during the training phase and introducing controlled interventions in the generation process of LLMs during inference, our approach enables LLMs to better understand sequence labeling tasks and align more effectively with human preferences\. \(2\)Significant improvement in inference efficiency\.A template\-filling mechanism requires the model to generate only label tokens, while the remaining parts are completed by the template with full KV Cache reuse, yielding up to9×\\timesfaster inferencecompared with state\-of\-the\-art methods\. \(3\)State\-of\-the\-art performance\.We conduct extensive experiments on NER and POS tasks, and the results demonstrate that DIRECT achieves superior performance in sequence labeling tasks under low\-resource settings\.

## 2METHODOLOGY

### 2\.1Task Definition

Given a token sequence of lengthnn,X=\{x1,x2,…,xn\}X=\\\{x\_\{1\},x\_\{2\},\\dots,x\_\{n\}\\\}, the objective of sequence labeling is to map it into a label sequence of the same length,Y=\{y1,y2,…,yn\}Y=\\\{y\_\{1\},y\_\{2\},\\dots,y\_\{n\}\\\}\. In this study, we formulate both NER and POS tagging as token\-level classification tasks, where the expected model output is expressed asO=\{x1​\(y1\),x2​\(y2\),⋯,xn​\(yn\)\}O=\\\{x\_\{1\}\(y\_\{1\}\),x\_\{2\}\(y\_\{2\}\),\\cdots,x\_\{n\}\(y\_\{n\}\)\\\}\. Figure[1](https://arxiv.org/html/2607.26891#S1.F1)illustrates the overall framework of our proposed DIRECT method\.

Table 1:Overall performance evaluated using F1\-scores, withboldvalues indicating the best results andunderlinedvalues representing the second\-best\. The models in parentheses denote different backbones\.

### 2\.2Precision\-Oriented Model Optimization

To enhance the ability of LLMs to capture the characteristics of sequence labeling tasks and generate outputs that more closely align with the expected results, we apply DPO training after SFT, guided by a refined data selection strategy\. Specifically, the SFT\-trained LLM is first used to generate candidate answers for each inputXX, from which we construct preference pairs\(X,O\+,O−\)\(X,O^\{\+\},O^\{\-\}\), whereO\+O^\{\+\}denotes the preferred answer andO−O^\{\-\}denotes the less\-preferred one\. To maximize entropy gain during training, we adopt an offline sampling strategy that integrates the BLEU score\[[13](https://arxiv.org/html/2607.26891#bib.bib16)\]with an F1\-based template matching method for entity extraction\. Among the sampled responses, those with the highest BLEU score but the lowest F1 score are selected as less\-preferred responses, while the ground\-truth answer is designated as the preferred response\.

After constructing the preference pairs\(X,O\+,O−\)\(X,O^\{\+\},O^\{\-\}\), both the reference modelπref\\pi\_\{\\mathrm\{ref\}\}and the trainable discriminator modelπθ\\pi\_\{\\theta\}are initialized with parameters obtained from SFT\. The parameters of the reference model remain fixed, while only the discriminator is updated during the DPO stage\. For each preference triplet, we compute the log\-probability differences under the discriminator and the reference model, respectively:

Δθ\\displaystyle\\Delta\_\{\\theta\}=log⁡πθ​\(O\+∣X\)−log⁡πθ​\(O−∣X\),\\displaystyle=\\log\\pi\_\{\\theta\}\(O^\{\+\}\\mid X\)\-\\log\\pi\_\{\\theta\}\(O^\{\-\}\\mid X\),\(1\)Δref\\displaystyle\\Delta\_\{\\mathrm\{ref\}\}=log⁡πref​\(O\+∣X\)−log⁡πref​\(O−∣X\)\.\\displaystyle=\\log\\pi\_\{\\mathrm\{ref\}\}\(O^\{\+\}\\mid X\)\-\\log\\pi\_\{\\mathrm\{ref\}\}\(O^\{\-\}\\mid X\)\.\(2\)The objective of DPO is to maximize the log\-probability gain of the preferred output under the discriminator model relative to the reference model, which is typically achieved by minimizing a binary classification likelihood loss:

ℒDPO=−log⁡σ​\(β​\(Δθ−Δref\)\),\\mathcal\{L\}\_\{\\mathrm\{DPO\}\}=\-\\log\\sigma\\\!\\Big\(\\beta\\big\(\\Delta\_\{\\theta\}\-\\Delta\_\{\\mathrm\{ref\}\}\\big\)\\Big\),\(3\)whereσ\\sigmadenotes the sigmoid function, andβ\\betais a scaling hyperparameter\.

### 2\.3Inference Rectification

During the inference stage, we incorporate a precise intervention mechanism into the generation process of LLMs\. This mechanism ensures that the output strictly adheres to a predefined format while also improving inference efficiency through the integration of Key\-Value \(KV\) Cache and prefilling techniques\. Specifically, we first tokenize the complete set of possible labelsL=\{l1,l2,⋯,lK\}L=\\\{l\_\{1\},l\_\{2\},\\cdots,l\_\{K\}\\\}and standardize them into token sequences of fixed lengthTT, denoted asS=\{s1,s2,⋯,sK\}S=\\\{s\_\{1\},s\_\{2\},\\cdots,s\_\{K\}\\\}\. The core of our method is an iterative procedure that predicts a label for each input wordxix\_\{i\}by efficiently managing the context’s KV Cache\. The process begins with context prefilling\. Before predicting the label for a given wordxix\_\{i\}, we process the preceding sequence to obtain the contextCt−2C\_\{t\-2\}\. A prefilling operation is then performed to compute and store the KV Cache for all tokens inCt−2=\{h1,h2​⋯,ht−2\}C\_\{t\-2\}=\\left\\\{h\_\{1\},h\_\{2\}\\cdots,h\_\{t\-2\}\\right\\\}in a single pass, thereby avoiding redundant attention computations for this known prefix in subsequent steps\.

Next, we construct the initial context for predicting the token ofsis\_\{i\}asCi=Ct−2⊕h⟨LBR⟩⊕h\[t:t\+T\]C\_\{i\}=C\_\{t\-2\}\\oplus h\_\{\\text\{\\textlangle LBR\\textrangle\}\}\\oplus h\_\{\[t:t\+T\]\}, where⊕\\oplusdenotes sequence concatenation and ⟨LBR⟩ represents the token sequence for the left parenthesis “\(”\. During computation, it is only necessary to incrementally compute the key\-values for ⟨LBR⟩ andsis\_\{i\}and append them to the existing KV Cache ofCt−2C\_\{t\-2\}\. To generate the label tokenui,j∈siu\_\{i,j\}\\in s\_\{i\}, the model produces the label of lengthTTin an auto\-regressive, token\-by\-token manner\. At each step, the model takes as input the KV Cache corresponding to the contextCt−1\+j−1C\_\{t\-1\+j\-1\}and computes the probability distribution for the next token, which corresponds to the conditional probabilitypθ​\(ui,j∣Ct−1\+j−1\)p\_\{\\theta\}\(u\_\{i,j\}\\mid C\_\{t\-1\+j\-1\}\)\. To ensure the validity of the generated sequence, we impose strict constraints on the decoding space\. The model is restricted to selecting tokens from a predefined candidate setV=\{u1,u2,⋯,um\}V=\\\{u\_\{1\},u\_\{2\},\\cdots,u\_\{m\}\\\}, whereui,j∈si⊆Vtu\_\{i,j\}\\in s\_\{i\}\\subseteq V\_\{t\}\. The final prediction is obtained by choosing the token with the highest probability:

ui,j=arg⁡maxui,j∈V⁡pθ​\(ui,j∣Ct−1\+j−1\)\.u\_\{i,j\}=\\arg\\max\_\{u\_\{i,j\}\\in V\}p\_\{\\theta\}\(u\_\{i,j\}\\mid C\_\{t\-1\+j\-1\}\)\.\(4\)After selecting the tokenssis\_\{i\}, we update the context toCi=Ct\+TC\_\{i\}=C\_\{t\+T\}and incrementally update its corresponding key\-values\. At this stage, both the subsequent right parenthesis ⟨RBR⟩ and the next input wordxi\+1x\_\{i\+1\}are already determined\. To prepare for predicting the label ofxi\+1x\_\{i\+1\}, we continue the prefilling process for the subsequent known content based on the current KV Cache\. In this way, the reuse of the existing KV Cache enables seamless iteration while avoiding redundant computations for known sequences\. By combining this constrained generation strategy with KV Cache prefilling, we not only enforce the generation of syntactically valid labels but also maximize computational efficiency\. Multiple attention operations are consolidated into a single prefill pass for known prefixes, which yields a substantial improvement in inference speed while preserving output quality\.

## 3Experiments

### 3\.1Experimental Setup

DatasetsWe evaluate the performance of our model on six NER datasets and two POS datasets\. The NER datasets include four Chinese datasets Weibo\[[14](https://arxiv.org/html/2607.26891#bib.bib9)\], Youku\[[6](https://arxiv.org/html/2607.26891#bib.bib10)\], Taobao\[[6](https://arxiv.org/html/2607.26891#bib.bib10)\], and Resume\[[23](https://arxiv.org/html/2607.26891#bib.bib11)\], as well as two English datasets MIT\-Movie\[[7](https://arxiv.org/html/2607.26891#bib.bib12)\]and CoNLL03\[[17](https://arxiv.org/html/2607.26891#bib.bib13)\]\. The POS datasets include two Chinese datasets, UD\[[11](https://arxiv.org/html/2607.26891#bib.bib15)\]and CTB6\[[21](https://arxiv.org/html/2607.26891#bib.bib14)\]\. To assess the model’s performance in low\-resource scenarios, we randomly sample a subset of𝒦\\mathcal\{K\}samples from the training set for training, respectively, where𝒦\\mathcal\{K\}is set to 250, 500, and 1000\.

BaselinesTo evaluate our model’s performance on low\-resource sequence labeling tasks, we compare it with the following baselines\. \(1\)InstructUIE\[[18](https://arxiv.org/html/2607.26891#bib.bib5)\], which is based on Flan\-T5\-xxl\. \(2\)GoLLIE\[[16](https://arxiv.org/html/2607.26891#bib.bib6)\], which is based on Code\-LLaMA\-7B\. \(3\)GNER\[[2](https://arxiv.org/html/2607.26891#bib.bib7)\], which is based on LLaMA\-7B\.

Training DetailsTo better evaluate the effectiveness of our approach, we employ two LLMs, LLaMA\-3\.1\-8B\-Instruct\[[3](https://arxiv.org/html/2607.26891#bib.bib18)\]and GLM\-4\-9B\-Chat\[[4](https://arxiv.org/html/2607.26891#bib.bib17)\], as our backbones for comparison\. For DPO training, we set the hyperparameterβ\\betato 0\.1\. In addition, since the Flan\-T5 model used in InstructUIE is limited in its ability to encode Chinese, we adopt the mT5\-xxl\[[20](https://arxiv.org/html/2607.26891#bib.bib19)\]model from the T5 family for experiments on Chinese datasets, while retaining Flan\-T5\-xxl for English datasets\.

### 3\.2Main Results

We evaluate the performance of different methods using the F1\-score, and the results are presented in Table[1](https://arxiv.org/html/2607.26891#S2.T1)\. The experiments show that when DIRECT adopts GLM4\-9B\-Chat as the backbone model, it consistently achieves the best performance across virtually all NER and POS datasets under different values of𝒦\\mathcal\{K\}\. When DIRECT employs LLaMA\-3\.1\-8B\-Instruct as the backbone, it generally attains the second\-best results across datasets\. Specifically, under different𝒦\\mathcal\{K\}settings, the average F1\-score of DIRECT improves by 0\.98%, 5\.22%, 4\.25%, 2\.55%, 0\.53%, 14\.72%, 0\.37%, and 0\.82% on the Youku, Taobao, Weibo, Resume, CoNLL03, MIT\-Movie, UD, and CTB6 datasets, respectively\. Notably, on the MIT\-Movie dataset, compared to other baseline models, our method achieves substantial improvements, with F1\-scores increased by 13\.69%, 14\.78%, and 15\.69% under𝒦=250\\mathcal\{K\}=250,𝒦=500\\mathcal\{K\}=500, and𝒦=1000\\mathcal\{K\}=1000, respectively\. We attribute these performance gains to the incorporation of preference pairs for DPO optimization during training, as well as controlled interventions in the generation process during inference, which enable the model outputs to better align with the expected results\.

Table 2:Ablation study on different backbone LLMs with𝒦=1000\\mathcal\{K\}=1000\. The best results are highlighted inbold\.

### 3\.3Ablation Study

To further validate the effectiveness of each component and strategy in DIRECT, we conducted ablation experiments under the setting of𝒦=1000\\mathcal\{K\}=1000across all datasets and backbone LLMs, with the results summarized in Table[2](https://arxiv.org/html/2607.26891#S3.T2)\. \(1\) “w/o DPO” denotes the removal of the DPO strategy during training\. The results show that this modification consistently leads to performance degradation across all datasets and backbone LLMs\. This is mainly because, without the DPO strategy, the model cannot learn effective contrastive signals from preference pairs, making it difficult to establish decision boundaries that are well aligned with the requirements of sequence labeling tasks, ultimately resulting in degraded performance\. \(2\) “w/ SFT” indicates performing only SFT on the backbone model without incorporating additional DPO training or inference intervention\. The results reveal that this modification leads to further performance decline\. Notably, when the backbone LLM is LLaMA\-3\.1\-8B\-Instruct, the F1\-scores of DIRECT on the Taobao and MIT\-Movie datasets drop by 11\.00% and 20\.21%, respectively\. These findings demonstrate that, through DPO training and inference correction, our method achieves significant performance improvements over the original backbone LLMs\. DPO training enables the model to better align with preference signals, while inference correction effectively reduces redundancy and uncertainty in the generation process, thereby delivering superior generalization and alignment performance across diverse datasets and scenarios\.

![Refer to caption](https://arxiv.org/html/2607.26891v1/x2.png)Fig\. 2:Inference efficiency comparison of different methods, where DIRECT adopts LLaMA\-3\.1\-8B\-Instruct as the backbone\.
### 3\.4Inference Time Analysis

To validate the efficiency of DIRECT in the inference stage, we compare its inference speed against three representative baseline models, with the results shown in Figure[2](https://arxiv.org/html/2607.26891#S3.F2)\. To ensure fairness and maintain consistency with the settings of other methods, we fix the batch size to 1 and conduct experiments on a single NVIDIA L40 GPU\. The experimental data are sampled from 10 sentences in the CTB6 dataset, with an average length of 192 tokens\. The results demonstrate that DIRECT significantly outperforms all baselines in inference speed, requiring only 32\.86 s to complete inference on all sentences, which is about one\-tenth of the strong baseline GNER\. This efficiency gain primarily stems from the template\-filling mechanism adopted during inference, where the LLMs only need to generate the predicted label tokens, while the remaining parts are automatically filled by the template\. This reduces redundant generation and substantially accelerates the inference process\. These findings highlight that DIRECT not only achieves strong performance but also demonstrates superior practical deployability\.

![Refer to caption](https://arxiv.org/html/2607.26891v1/x3.png)Fig\. 3:Case study with DIRECT and baselines, where DIRECT adopts LLaMA\-3\.1\-8B\-Instruct as the backbone\.
### 3\.5Case Study

To provide a more intuitive comparison of the advantages of DIRECT over other methods, we conduct case studies in Figure[3](https://arxiv.org/html/2607.26891#S3.F3)\. In case 1 and 2, the other three baseline models all fail to correctly identify the “PLOT” entity or the “MISC” entity\. At the same time, these baselines generally suffer from issues of missing outputs or redundant outputs\. For example, in case 1, the output of GNER omits the word “the” and its corresponding label, as well as the label for the final word “?”; GoLLIE produces an output where “span=‘superhero’ ” does not conform to its predefined formatting rules; and InstructUIE fails to provide the corresponding entity content after the “GENRE” label\. In contrast, DIRECT not only identifies and classifies all entities completely and accurately in both cases but also strictly adheres to the expected output format, thereby demonstrating stronger robustness and controllability\.

## 4Conclusion

In this paper, we propose DIRECT, a DPO\-based inference rectification method\. To enhance the alignment capability of LLMs in domain adaptation, we perform DPO optimization with preference pairs during training and intervene in the generation process during inference, ensuring that the outputs of LLMs are better aligned with human preferences and the requirements of sequence labeling tasks\. Furthermore, to improve inference efficiency, we introduce a template\-filling mechanism that substantially reduces redundancy in generation\. Experimental results demonstrate that DIRECT achieves significant improvements in both performance and efficiency compared with previous methods\.

## References

- \[1\]\(2016\)Improving coreference resolution by learning entity\-level distributed representations\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 643–653\.Note:[https://doi\.org/10\.18653/v1/P16\-1061](https://doi.org/10.18653/v1/P16-1061)Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p1.1)\.
- \[2\]Y\. Ding, J\. Li, P\. Wang, Z\. Tang, B\. Yan, and M\. Zhang\(2024\)Rethinking negative instances for generative named entity recognition\.arXiv preprint arXiv:2402\.16602\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1),[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p2.1)\.
- \[3\]A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv e\-prints,pp\. arXiv–2407\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p3.1)\.
- \[4\]T\. GLM, A\. Zeng, B\. Xu, B\. Wang, C\. Zhang, D\. Yin, D\. Zhang, D\. Rojas, G\. Feng, H\. Zhao,et al\.\(2024\)Chatglm: a family of large language models from glm\-130b to glm\-4 all tools\.arXiv preprint arXiv:2406\.12793\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p3.1)\.
- \[5\]S\. Ji, S\. Pan, E\. Cambria, P\. Marttinen, and S\. Y\. Philip\(2021\)A survey on knowledge graphs: representation, acquisition, and applications\.IEEE transactions on neural networks and learning systems33\(2\),pp\. 494–514\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p1.1)\.
- \[6\]Z\. Jie, P\. Xie, W\. Lu, R\. Ding, and L\. Li\(2019\)Better modeling of incomplete annotations for named entity recognition\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 729–734\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.
- \[7\]J\. Liu, P\. Pasupat, S\. Cyphers, and J\. Glass\(2013\)Asgard: a portable architecture for multilingual dialogue systems\.In2013 IEEE International Conference on Acoustics, Speech and Signal Processing,pp\. 8386–8390\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.
- \[8\]J\. Lou, Y\. Lu, D\. Dai, W\. Jia, H\. Lin, X\. Han, L\. Sun, and H\. Wu\(2023\)Universal information extraction as unified semantic matching\.InProceedings of the AAAI conference on Artificial Intelligence,Vol\.37,pp\. 13318–13326\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1)\.
- \[9\]Y\. Lu, Q\. Liu, D\. Dai, X\. Xiao, H\. Lin, X\. Han, L\. Sun, and H\. Wu\(2022\)Unified structure generation for universal information extraction\.arXiv preprint arXiv:2203\.12277\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1)\.
- \[10\]Y\. Mo, J\. Liu, J\. Yang, Q\. Wang, S\. Zhang, J\. Wang, and Z\. Li\(2024\)C\-icl: contrastive in\-context learning for information extraction\.arXiv preprint arXiv:2402\.11254\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1)\.
- \[11\]J\. Nivre, M\. De Marneffe, F\. Ginter, Y\. Goldberg, J\. Hajic, C\. D\. Manning, R\. McDonald, S\. Petrov, S\. Pyysalo, N\. Silveira,et al\.\(2016\)Universal dependencies v1: a multilingual treebank collection\.InProceedings of the Tenth International Conference on Language Resources and Evaluation \(LREC’16\),pp\. 1659–1666\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.
- \[12\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1)\.
- \[13\]K\. Papineni, S\. Roukos, T\. Ward, and W\. Zhu\(2002\)Bleu: a method for automatic evaluation of machine translation\.InProceedings of the 40th annual meeting of the Association for Computational Linguistics,pp\. 311–318\.Cited by:[§2\.2](https://arxiv.org/html/2607.26891#S2.SS2.p1.4)\.
- \[14\]N\. Peng and M\. Dredze\(2015\)Named entity recognition for chinese social media with jointly trained embeddings\.InProceedings of the 2015 conference on empirical methods in natural language processing,pp\. 548–554\.Note:[https://doi\.org/10\.18653/v1/D15\-1064](https://doi.org/10.18653/v1/D15-1064)Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.
- \[15\]G\. Pergola, E\. Kochkina, L\. Gui, M\. Liakata, and Y\. He\(2021\)Boosting low\-resource biomedical qa via entity\-aware masking strategies\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,pp\. 1977–1985\.Note:[https://doi\.org/10\.18653/v1/2021\.eacl\-main\.169](https://doi.org/10.18653/v1/2021.eacl-main.169)Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p1.1)\.
- \[16\]O\. Sainz, I\. García\-Ferrero, R\. Agerri, O\. L\. de Lacalle, G\. Rigau, and E\. Agirre\(2023\)Gollie: annotation guidelines improve zero\-shot information\-extraction\.arXiv preprint arXiv:2310\.03668\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1),[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p2.1)\.
- \[17\]E\. F\. Sang and F\. De Meulder\(2003\)Introduction to the conll\-2003 shared task: language\-independent named entity recognition\.arXiv preprint cs/0306050\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.
- \[18\]X\. Wang, W\. Zhou, C\. Zu, H\. Xia, T\. Chen, Y\. Zhang, R\. Zheng, J\. Ye, Q\. Zhang, T\. Gui,et al\.\(2023\)Instructuie: multi\-task instruction tuning for unified information extraction\.arXiv preprint arXiv:2304\.08085\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1),[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p2.1)\.
- \[19\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1)\.
- \[20\]L\. Xue, N\. Constant, A\. Roberts, M\. Kale, R\. Al\-Rfou, A\. Siddhant, A\. Barua, and C\. Raffel\(2020\)MT5: a massively multilingual pre\-trained text\-to\-text transformer\.arXiv preprint arXiv:2010\.11934\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p3.1)\.
- \[21\]N\. Xue, F\. Xia, F\. Chiou, and M\. Palmer\(2005\)The penn chinese treebank: phrase structure annotation of a large corpus\.Natural language engineering11\(2\),pp\. 207–238\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.
- \[22\]R\. Yang, B\. Yang, S\. Ouyang, T\. She, A\. Feng, Y\. Jiang, F\. Lecue, J\. Lu, and I\. Li\(2024\)Graphusion: leveraging large language models for scientific knowledge graph fusion and construction in nlp education\.arXiv preprint arXiv:2407\.10794\.Cited by:[§1](https://arxiv.org/html/2607.26891#S1.p2.1)\.
- \[23\]Y\. Zhang and J\. Yang\(2018\)Chinese ner using lattice lstm\.arXiv preprint arXiv:1805\.02023\.Cited by:[§3\.1](https://arxiv.org/html/2607.26891#S3.SS1.p1.2)\.