FSE: Continual Learning for Named Entity Recognition by Fast-Slow Experts

# FSE: Continual Learning for Named Entity Recognition by Fast-Slow Experts
Source: [https://arxiv.org/html/2607.22075](https://arxiv.org/html/2607.22075)
###### Abstract

Continual Learning for Named Entity Recognition \(CLNER\) enable models to incrementally learn new entity types without forgetting previously acquired ones\. However, existing methods suffer from catastrophic forgetting and insufficient exploitation of shared information across tasks\. This paper proposesFSE, aFast\-SlowExperts enhanced span\-based NER model for CLNER\. The shared fast expert learns token\-level links to efficiently filter out unlikely spans, while the task\-specific slow expert performs span classification only on the remaining candidates\. It stabilizes learning by promoting knowledge sharing across tasks and maintains plasticity by reducing learning burden at each task\. A length\-decay negative sampling strategy to mitigate span imbalance is also introduced\. Extensive experiments on OntoNotes and FewNERD synthestic datasets demonstrate that FSE achieves state\-of\-the\-art performance in CLNER scenarios, with effectiveness of each component, empirical evidence of faster convergence and expected functionality of both experts\.

###### keywords:

Continual Learning, Named Entity Recognition, Fast\-Slow Experts

††journal:Pattern Recognition Letters\\affiliation

\[label1\]organization=School of Computer Science and Technology, Harbin Institute of Technology, city=Shenzhen, state=Guangdong, country=China

## 1Introduction

Continual learning is pivotal for advancing evolvable artificial general intelligence, enabling models to acquire new knowledge from streaming data\[[1](https://arxiv.org/html/2607.22075#bib.bib1)\]\. In this field, continual learning for named entity recognition \(CLNER\) is key to developing real\-time or personal application that requires dynamic adaptation to changing scenarios\. NER aims to extract entities from unstructured text, CLNER further enables the incremental learning of new entity types over time without forgetting previously learned ones\[[2](https://arxiv.org/html/2607.22075#bib.bib2)\]\. For example, after learning to detect medical terminology by training on medical text, the model can still accurately identify previously learned entity types like persons and locations, without requiring retraining from scratch\.

CLNER always encounters catastrophic forgetting\[[3](https://arxiv.org/html/2607.22075#bib.bib3),[4](https://arxiv.org/html/2607.22075#bib.bib4)\], where models trained on new tasks tends to rapidly disrupt previously learned ones\. To mitigate this, standard NER models often incorporate anti\-forgetting techniques such as knowledge distillation or model expansion\. Meanwhile, these efforts also witness the paradigm shift of NER from sequence tagging to span\-based methods\[[5](https://arxiv.org/html/2607.22075#bib.bib5)\]\. Here we focus on span\-based methods, which enumerates and classify all possible text spans, providing richer representations and often yielding superior performance\. Notably, recent large language models \(LLMs\) show promise in CLNER owing to their strong generalization capabilities\. However, empirical studies\[[6](https://arxiv.org/html/2607.22075#bib.bib6)\]reveal that LLMs still underperform their smaller, NER\-specific Bert\-based counterparts\. Moreover, The prohibitive computational demands and challenges in dynamic updates of large parameters also hinder its practical deployment\.

![Refer to caption](https://arxiv.org/html/2607.22075v1/x1.png)Figure 1:Compared to learning massive spans in traditional methods, the fast expert in FSE efficiently learns a few links to filter out most spans, which reduces the learning burden on the slow expert that ultimately detects entities\.Hence, span\-based CLNER methods remain preferable, but we argue that there still exist unsolved challenges from two aspects:1\) First,task interference may lead to conflicting parameter updates during continual learning\. Thus, models capable of explicitly modeling task\-shared information can facilitate stability in this process\. However, existing span\-based methods, which only share the Bert base encoder, fail to explore additional high\-level information specific to NER that could be shared between different tasks\. For example, when a span text ”\[PER\] at” \(a person entity but followed by a preposition\) has already been learned as a non\-entity span in previous tasks, this pattern is also unlikely to be the entity in subsequent tasks\. Such knowledge could be shared\.2\) Second, model’s plasticity gradually diminishes as learning progresses\[[7](https://arxiv.org/html/2607.22075#bib.bib7)\], making the acquisition of new entity types increasingly difficult\. Thus, reducing the learning burden for each task would benefit continual learning\. For example, given a currently learned non\-entity span ”\[PER\] at”, model should readily identify the extended span ”\[PER\] at the” as a non\-entity as well\.

Here we propose that a properly designed ”links” between adjacent tokens can address both considerations\. Specifically, we introduce aFast\-SlowExperts enhanced span\-based model \(FSE\), where the fast expert \(shared across tasks\) models these links, while the slow experts \(task\-specific\) performs standard span classification at each task\. As shown in Fig[1](https://arxiv.org/html/2607.22075#S1.F1), given a sentence withnntokens, traditional span\-based model would laboriously enumerate all0\.5​\(n2−n\)0\.5\(n^\{2\}\-n\)spans to detect entities for each task\. Instead, our fast expert first learns the relatively fewer links \(n−1n\\\!\-\\\!1\) between adjacent tokens\. We define that links within an entity span are all strong, while non\-entity spans must contain at least one weak link\. Notably, this link mechanism is merely a designed inductive bias imposed on the model, the specific link strengths are still automatically learned by the model itself during training\. Upon this design, we expect that once the fast expert learn a weak links \(e\.g, between ”\[PER\] at”, massive spans cover this link can be filtered out rapidly\. Then the remaining fewer spans are focused by the slow expert\. This substantially alleviated the span learning burden of slow expert, and the sharable mechanism of fast experts also benefit the newcome slow experts in subsequent tasks\.

In implementation, dual experts are jointly optimized to avoid error propagation\. First, the fast expert applies softmin pooling to aggregate the learned link scores within a certain span into this span’s ”fast score”, ensuring the span\-level fast score tends to be low if any weak link is detected\. Next, The fast score is fused with the slow expert’s output score for each span, ensuring that an entity is recognized only when the fast expert confidently identifies strong links and the slow expert confirms it as a valid entity\. Spans without meeting both conditions would be rejected\. During continual learning, we use knowledge distillation to distill the fused outputs of dual experts in previous tasks for transferring prior knowledge\. Moreover, benefiting from fast expert’s efficiency in filtering out spans, fewer negative spans are needed for explicit learning\. This makes negative sampling, a technique aiming to mitigate the positive\-negative span imbalance inherent in span\-based models, particularly suitable\. We propose a length\-decay negative sampling strategy that adjusts the sampling probability based on span length, applying it to both current task learning and old task distillation\.

Extensive experiments on synthesized datasets from OntoNotes and FewNERD show the effectiveness of FSE, achieving SoTA performance among various baseline including LLMs methods\. The contributions of this paper include:

- •We identify the potential of adjacent token links to promote the stability and plasticity in span\-based CNLER model\.
- •We introduce jointly optimized fast\-slow experts to enhance standard span\-based CLNER model, where the fast expert utilizes links to guide the span modeling of slow expert\. This reduces task\-specific learning difficulty while enabling effective inter\-task knowledge sharing\.
- •Extensive experiments including LLM\-based competitor demonstrate superior performance of our FSE in CLNER scenarios\.

## 2Related Work

Named Entity Recognitionconstitutes a cornerstone of information extraction in natural language processing \(NLP\), aiming to recognize predefined entity types from unstructured text\. NER is traditionally formalized as sequence tagging by adopting CRF after neural representations\[[8](https://arxiv.org/html/2607.22075#bib.bib8),[9](https://arxiv.org/html/2607.22075#bib.bib9)\]\. In contrast, span\-based methods enumerate all possible spans within the text and directly classify them without tagging scheme \(BIO\) conversion and accommodate nested entities\[[10](https://arxiv.org/html/2607.22075#bib.bib10),[11](https://arxiv.org/html/2607.22075#bib.bib11)\]\. Besides, Machine Reading Comprehension \(MRC\)\-based methods use pointer networks to locate entity positions in the input text\[[12](https://arxiv.org/html/2607.22075#bib.bib12),[13](https://arxiv.org/html/2607.22075#bib.bib13)\]\. Beyond these extraction\-based paradigm, generation\-based approaches such as Seq2Seq\-based directly generate entity mentions from the given input\[[14](https://arxiv.org/html/2607.22075#bib.bib14),[15](https://arxiv.org/html/2607.22075#bib.bib15)\]\. Recent LLM\-based NER methods are also generative, exhibiting remarkable generalization capabilities in handling both seen and unseen entity types\. However, smaller models remain valuable due to their efficiency and domain\-specific adaptability\.

Continual Learning NERtackles the evolving scenarios where new entity types emerge sequentially\. This progressive shift in data distribution violates the i\.i\.d\. assumption, leading to catastrophic forgetting of previously learned entity types\. CLNER aim to balance the retention of past learning with the acquisition of new information, also known as the stability\-plasticity dilemma\. Monaikul\[[2](https://arxiv.org/html/2607.22075#bib.bib2)\]first framed CLNER as class\-incremental continual learning and introduce sequence tagging\-based AddNER and ExtendNER\. The training data for each sequential task are annotated only with the the entity types currently being learned, and the model is evaluated on data that requires predicting all accumulated learned entity types\. AddNER uses an individual head for each new task and ExtendNER uses a single unified head shared across all tasks, both incorporating knowledge distillation\. Then, L&R\[[16](https://arxiv.org/html/2607.22075#bib.bib16)\]and ExtendNER\+DLD\[[17](https://arxiv.org/html/2607.22075#bib.bib17)\]both build upon ExtendNER to further improve its performance\. SpanKL\[[11](https://arxiv.org/html/2607.22075#bib.bib11)\]first uses span\-based modeling and binary classification in CLNER to alleviate conflicts in non\-entity types across different tasks\. SKD\-NER\[[18](https://arxiv.org/html/2607.22075#bib.bib18)\]further uses reinforcement learning upon it\. However, all these methods share information solely through the underlying BERT parameters and neglect the potential to leverage other types of information shared across tasks\. In contrast, in CV task such as continual semantic segmentation, architecture decomposition methods have explored decomposing the model into task\-specific and task\-sharing components, where the task\-sharing part supports reconciling old and new knowledge simultaneously, with the task\-specific part adapting to new tasks\[[19](https://arxiv.org/html/2607.22075#bib.bib19)\]\. For example, Representation Compensation Networks\[[20](https://arxiv.org/html/2607.22075#bib.bib20)\]uses structural re\-parameterization to decouple CNN modules into two parallel branches for shared and task\-specific information\. LAG\[[21](https://arxiv.org/html/2607.22075#bib.bib21)\]disentangles semantic features under channel\-wise and spatial\-level to simultaneously reconcile knowledge inheritance and new\-task learning, where semantic\-invariant knowledge is modeled as abstract prototypes shared across tasks\. To the best of our knowledge, we are the first to explore shareable information specific to NER under CL setting, especially span\-based methods, to improve overall performance across tasks\.

![Refer to caption](https://arxiv.org/html/2607.22075v1/x2.png)Figure 2:Illustration of FSE: The task\-shared fast expert for links modeling outputs fast score\. The task\-specific slow expert for span modeling outputs span score\. Both scores are finally fused\. Previous tasks and current tasks are optimized by KL loss and BCE loss, respectively\. Negative sampling apply in both optimizations\.
## 3Method

In this section, we first formulate CLNER, and then introduce the base architecture of FSE, including slow expert, fast expert and their collaboration mechanism\. Finally, we describe the training techniques employed\.

### 3\.1CLNER Formulation

CLNER is formally defined as: An NER model is sequentially trained on a stream of tasks𝒯1,𝒯2,…,𝒯L\\mathcal\{T\}\_\{1\},\\mathcal\{T\}\_\{2\},\\dots,\\mathcal\{T\}\_\{L\}\. Each task𝒯l\\mathcal\{T\}\_\{l\}introduces a new set of entity typesℰl=\{el1,el2,…\}\\mathcal\{E\}\_\{l\}=\\\{e\_\{l\}^\{1\},e\_\{l\}^\{2\},\\dots\\\}\. Dataset for𝒯l\\mathcal\{T\}\_\{l\}only annotates its corresponding entity typesℰl\\mathcal\{E\}\_\{l\}\. Typically, during training on task𝒯l\\mathcal\{T\}\_\{l\}, the model cannot access data from previous tasks𝒯1,…,𝒯l−1\\mathcal\{T\}\_\{1\},\\dots,\\mathcal\{T\}\_\{l\-1\}\.

The learning process begins by training the initial modelℳ1\\mathcal\{M\}\_\{1\}on𝒯1\\mathcal\{T\}\_\{1\}to recognize entity types inℰ1\\mathcal\{E\}\_\{1\}\. For each subsequent stepl\>1l\>1, the modelℳl\\mathcal\{M\}\_\{l\}is incrementally trained on the data of𝒯l\\mathcal\{T\}\_\{l\}based onℳl−1\\mathcal\{M\}\_\{l\-1\}, aiming to recognize all previously learned entity types⋃i=1lℰi\\bigcup\_\{i=1\}^\{l\}\\mathcal\{E\}\_\{i\}\.

### 3\.2Model Architecture

Fig\.[2](https://arxiv.org/html/2607.22075#S2.F2)shows the overall architecture of FSE, including contextual encoder for token learning, task\-shared fast expert for link learning and task\-specific slow expert for span learning\. Negative sampling applies in both KL and BCE optimizations\.

#### 3\.2\.1Contextual Encoder

Given input sentenceXXwithnntokens\[x1,x2,…,xn\]\[x\_\{1\},x\_\{2\},\.\.\.,x\_\{n\}\], we define spansi​js\_\{ij\}as continuous tokens fromxix\_\{i\}toxjx\_\{j\}, where1≤i≤j≤n1\\\!\\leq\\\!i\\leq\\\!j\\leq\\\!n\. The contextual encoder capture tokens dependencies and generates contextualized representations for each token\. We employ the widely\-used Bert\-base PLMs as our contextual encoder, formulated as:

𝐇=𝙴𝚗𝚌𝚘𝚍𝚎𝚛​\(X\)\\mathbf\{H\}=\\mathtt\{Encoder\}\(X\)\(1\)where𝐇=\[𝐡1,𝐡2,…,𝐡n\]∈ℝn×dh\\mathbf\{H\}=\[\\mathbf\{h\}\_\{1\},\\mathbf\{h\}\_\{2\},\\dots,\\mathbf\{h\}\_\{n\}\]\\\!\\in\\\!\\mathbb\{R\}^\{n\\times d^\{h\}\}represents the contextualized representations vector for each token,dhd^\{h\}is hidden size\.

#### 3\.2\.2Slow Expert

The slow expert models the representation of each span to facilitate span classification\. We follow SpanKL to use a simple scaled dot\-product interaction between the ”start” and ”end” feature space of token\. This means the slow expert is boundary\-aware and responsible for detecting span boundaries, enabling the fast expert to focus on the information within the spans\. Specifically, for each entity types, we use two distinct single\-layer feedforward network \(FFN\) to yield the start and end representations of tokens, respectively\. Then span representationhsi​jh^\{s\_\{ij\}\}is computed as:

𝐡si​j\\displaystyle\\mathbf\{h\}^\{s\_\{ij\}\}=\[h1si​j,h2si​j,…​hKsi​j\]∈ℝK\\displaystyle=\[h^\{s\_\{ij\}\}\_\{1\},h^\{s\_\{ij\}\}\_\{2\},\\dots h^\{s\_\{ij\}\}\_\{K\}\]\\in\\mathbb\{R\}^\{K\}\(2\)hksi​j\\displaystyle h^\{s\_\{ij\}\}\_\{k\}=𝙵𝙵𝙽kstart​\(𝐡i\)⊺⋅𝙵𝙵𝙽kend​\(𝐡j\)×\(do\)−0\.5\\displaystyle=\\mathtt\{FFN\}^\{\\text\{start\}\}\_\{k\}\(\\mathbf\{h\}\_\{i\}\)^\{\\intercal\}\\cdot\\mathtt\{FFN\}^\{\\text\{end\}\}\_\{k\}\(\\mathbf\{h\}\_\{j\}\)\\times\(d^\{o\}\)^\{\-0\.5\}wherekkdenote the k\-th entity type among the total K types currently being learned\. All the𝙵𝙵𝙽start\\mathtt\{FFN\}^\{\\text\{start\}\}and𝙵𝙵𝙽end\\mathtt\{FFN\}^\{\\text\{end\}\}relative to each entity type have the same output dimensiondod^\{o\}\. There are totally2×K2\\times Kdistinct FFNs\. This is similar to multi\-head attention mechanism by treating start, end and entity type as Query, Key and Head and enables seamless addition of FFNs initialized for the newcome tasks\.

#### 3\.2\.3Fast Expert

Letαi∈ℝ\\alpha\_\{i\}\\in\\mathbb\{R\}denote the link score, which quantifies the link strength between adjacent tokensxix\_\{i\}andxi\+1x\_\{i\+1\}in the sequenceXX, structured as:x1,α1¯,x2,α2¯,x3,…,xn−1,αn−1¯,xnx\_\{1\},\\underline\{\\alpha\_\{1\}\},x\_\{2\},\\underline\{\\alpha\_\{2\}\},x\_\{3\},\.\.\.,x\_\{n\-1\},\\underline\{\\alpha\_\{n\-1\}\},x\_\{n\}, where higher score indicates stronger link\. There are totaln−1n\\\!\-\\\!1link scores within sentence of lengthnnand each link score is dynamically computed by the fast expert as:

ai=𝙵𝙵𝙽linkstart​\(𝐡i\)⊺⋅𝙵𝙵𝙽linkend​\(𝐡i\+1\)×\(dl\)−0\.5\\displaystyle a\_\{i\}=\\mathtt\{FFN\}^\{\\text\{start\}\}\_\{\\text\{link\}\}\(\\mathbf\{h\}\_\{i\}\)^\{\\intercal\}\\cdot\\mathtt\{FFN\}^\{\\text\{end\}\}\_\{\\text\{link\}\}\(\\mathbf\{h\}\_\{i\+1\}\)\\times\(d^\{l\}\)^\{\-0\.5\}\(3\)where𝙵𝙵𝙽linkstart\\mathtt\{FFN\}^\{\\text\{start\}\}\_\{\\text\{link\}\}and𝙵𝙵𝙽linkend\\mathtt\{FFN\}^\{\\text\{end\}\}\_\{\\text\{link\}\}are two distinct FFNs modeling the link’s start and end feature space respectively\.dld^\{l\}is output dimension\. This process is similar to span modeling in Equ\.[2](https://arxiv.org/html/2607.22075#S3.E2), allowing it to be treated as an ”additional entity types” modeling step, which facilitates parallel computation in practice\. Notably,𝙵𝙵𝙽linkstart\\mathtt\{FFN\}^\{\\text\{start\}\}\_\{\\text\{link\}\}and𝙵𝙵𝙽linkend\\mathtt\{FFN\}^\{\\text\{end\}\}\_\{\\text\{link\}\}of fast expert are shared across tasks\.

#### 3\.2\.4Span\-Level Fast Score

To propagate link information to each span, we need to broadcast the obtained 1D token\-level link scores into 2D span\-level scores \(termed as fast scoresff\)\. As shown in Fig\.[3](https://arxiv.org/html/2607.22075#S3.F3), we aggregate all link scores within a certain span and pooling them into a single relative fast score\. Given our design objective that a span should be rejected if it contains even a single weak link \(i,e,\. the weakest link acts as an upper bound\), we introducesoftmin\-pooling, a stable and smooth version of min\-pooling, to mitigate potential gradient sparsity and instability issues\. The softmin is implemented via an inverted logsumexp operator as:

fsi​j\\displaystyle f^\{s\_\{ij\}\}=Pooling​\(\{ai,…,aj−1\}\)=−τ⋅log​∑a∈𝒜exp⁡\(−a/τ\)\\displaystyle=\\text\{Pooling\}\(\\\{a\_\{i\},\\dots,a\_\{j\-1\}\\\}\)=\-\\tau\\cdot\\log\\sum\_\{a\\in\\mathcal\{A\}\}\\exp\(\-a/\\tau\)\(4\)where𝒜=\{ai,…,aj−1\}\\mathcal\{A\}\\\!=\\\!\\\{a\_\{i\},\\dots,a\_\{j\-1\}\\\}is the set of link scores within spansi​js\_\{ij\}, temperatureτ\\taucontrols the sharpness of the pooling and is set to0\.10\.1\. Notably, we also address the challenge of batch\-level paralleled implementation for softmin\-pooling by heuristically using thelogcumsumexpoperator in PyTorch\.

![Refer to caption](https://arxiv.org/html/2607.22075v1/x3.png)Figure 3:Using softmin\-pooling to propagate link scores into span\-level fast scores from a matrix perspective\.
#### 3\.2\.5Dual Experts Fusion

Given both span\-level scores derived from the slow expert:𝐡si​j∈ℝK\\mathbf\{h\}^\{s\_\{ij\}\}\\\!\\in\\\!\\mathbb\{R\}^\{K\}\(containing K scalar scores regarding each entity type\) and from the fast expert:fsi​jf^\{s\_\{ij\}\}, we fuse them into final score for classification\. We proactively apply the sigmoid activation to both scores into0∼10\{\\sim\}1\. As expected, recognition of entity requires consensus from both experts, where a low score from either expert leads to rejection\. This fusion requirement can be satisfied by Harmonic Mean\. Fused score𝐡~si​j\\mathbf\{\\tilde\{h\}\}^\{s\_\{ij\}\}is computed as:

hksi​j≔σ​\(hksi​j\),fsi​j≔σ​\(fsi​j\)\\displaystyle h\_\{k\}^\{s\_\{ij\}\}\\coloneqq\\sigma\(h\_\{k\}^\{s\_\{ij\}\}\),\\quad f^\{s\_\{ij\}\}\\coloneqq\\sigma\(f^\{s\_\{ij\}\}\)\(5\)𝐡~si​j=Fuse​\(𝐡si​j,fsi​j\)=\[fuse​\(h1si​j,fsi​j\),…,fuse​\(hKsi​j,fsi​j\)\]\\displaystyle\\mathbf\{\\tilde\{h\}\}^\{s\_\{ij\}\}=\\\!\\textbf\{Fuse\}\(\\mathbf\{h\}^\{s\_\{ij\}\},f^\{s\_\{ij\}\}\)=\\\!\[\\text\{fuse\}\(h\_\{1\}^\{s\_\{ij\}\},f^\{s\_\{ij\}\}\),\\dots,\\text\{fuse\}\(h\_\{K\}^\{s\_\{ij\}\},f^\{s\_\{ij\}\}\)\]\(6\)fuse​\(x,y\)=2​x​y/\(x\+y\)\\displaystyle\\text\{fuse\}\(x,y\)=2xy/\(x\+y\)\(7\)whereσ​\(⋅\)\\sigma\(\\cdot\)is sigmoid function normalizing both scores, then Fuse\(\) function implements a broadcastable Harmonic Mean operation, ensuring that the link information contributes to all entity types\. Notably, spans of length 1 lack relative link score so their fused scores directly adopt the slow expert scores\.

We employ Binary Cross\-Entropy loss function for coherent optimization\[[11](https://arxiv.org/html/2607.22075#bib.bib11)\], which independently determines whether a span belongs to a specific entity type\. Notably, Eq\.[5](https://arxiv.org/html/2607.22075#S3.E5)ensures that the final fused score𝐡~si​j\\mathbf\{\\tilde\{h\}\}^\{s\_\{ij\}\}is probabilities, we denote here asp^i​jk\\hat\{p\}\_\{ij\}^\{k\}\. Then the loss is computed as:

ℒb​c​e=−∑i,j,k\[pi​jk​log⁡p^i​jk\+\(1−pi​jk\)​log⁡\(1−p^i​jk\)\]\\displaystyle\\mathcal\{L\}\_\{bce\}=\-\\sum\_\{i,j,k\}\\left\[p\_\{ij\}^\{k\}\\log\\hat\{p\}\_\{ij\}^\{k\}\+\(1\-p\_\{ij\}^\{k\}\)\\log\(1\-\\hat\{p\}\_\{ij\}^\{k\}\)\\right\]\(8\)wherepi​jkp\_\{ij\}^\{k\}is the one\-hot label for spansi​js\_\{ij\}being k\-th entity type\.

#### 3\.2\.6Knowledge Retention

We use knowledge distillation \(KD\) to transfer knowledge from both experts learned in previous tasks by distilling their fused scores\. Specifically, before training on a new task, we perform a one\-shot prediction on the new task’s data to generate pseudo labels \(i\.e\., distilled labels\) of all previously learned entity types\. Than the distilled labels are used to optimize the predictions for old tasks in current step via a Bernoulli KL\-divergence loss:

ℒk​d=∑i,j,k\[p¯i​jk​log⁡\(p¯i​jk/p^i​jk\)\+\(1−p¯i​jk\)​log⁡\(\(1−p¯i​jk\)/\(1−p^i​jk\)\)\]\\displaystyle\\mathcal\{L\}\_\{kd\}=\\sum\_\{i,j,k\}\\left\[\\bar\{p\}\_\{ij\}^\{k\}\\log\(\\bar\{p\}\_\{ij\}^\{k\}/\\hat\{p\}\_\{ij\}^\{k\}\)\+\(1\-\\bar\{p\}\_\{ij\}^\{k\}\)\\log\(\(1\-\\bar\{p\}\_\{ij\}^\{k\}\)/\(1\-\\hat\{p\}\_\{ij\}^\{k\}\)\)\\right\]\(9\)wherep¯i​jk\\bar\{p\}\_\{ij\}^\{k\}is the distilled label generated by spansi​js\_\{ij\}for the k\-th entity type from old tasks\. Here,p^i​jk\\hat\{p\}\_\{ij\}^\{k\}is fused score similar to that in Equ\.[8](https://arxiv.org/html/2607.22075#S3.E8)but specially for the old entity types\.

#### 3\.2\.7Length\-Decay Negative Sampling

In span\-based models, positive spans \(entities\) are significantly outnumbered by negative spans\. This severe imbalance leads to biased optimization during standard training\. Negative sampling aims to address the imbalance by randomly removing a portion of negative spans during learning, but it may result in insufficient learning or the omission of hard negative spans\. Opportunely, fast expert can provide shortcut information to reject negative spans, enabling lossless integration of negative sampling\. It also mitigates the risk of overly optimizing weak links\.

Given the empirical observation that longer spans are less likely to be entities, we design a length\-decay negative sampling strategy for our loss computation\. As shown in Fig\.[4](https://arxiv.org/html/2607.22075#S3.F4), negative sampling probabilitiesPnegP\_\{\\text\{neg\}\}corresponding to span lengthℓ\\ellis design as:

Pneg​\(ℓ\)=η⋅\(0\.95⋅σ​\(−0\.8⋅ℓ\+11\)\+0\.05\)P\_\{\\text\{neg\}\}\(\\ell\)=\\eta\\cdot\\left\(0\.95\\cdot\\sigma\(\-0\.8\\cdot\\ell\+11\)\+0\.05\\right\)\(10\)where the base sampling ratioη\\etais set to 1 in experiments, 0\.95 and 0\.05 are used to scale the domain of the probability curve to the range \[0\.05, 1\], ensuring that even overly long spans have a non\-zero probability of being sampled\. 0\.8 controls the decay slope and 11 controls the shift of sigmoid, i\.e\., the length threshold at which the sampling probability begins to decline\. These values and their approximate ranges were tested in preliminary experiments\. The current values performed best, but other approximate settings also yielded comparable results owing to the robustness derived from the probabilistic nature\. We apply negative sampling when computing bothℒb​c​e\\mathcal\{L\}\_\{bce\}andℒk​d\\mathcal\{L\}\_\{kd\}losses, obtainingℒb​c​en​s\\mathcal\{L\}\_\{bce\}^\{ns\}andℒk​dn​s\\mathcal\{L\}\_\{kd\}^\{ns\}, respectively\. Noted that sampling only apply to negative spans\. The spans of entities and the spans within entities are forcibly kept\. In KD, we treat span’s distilled labels with values below 0\.5 as negative spans\. Finally, the training loss is the weighted sum as:

ℒ=α⋅ℒb​c​en​s\+β⋅ℒk​dn​s\\mathcal\{L\}=\\alpha\\cdot\\mathcal\{L\}\_\{bce\}^\{ns\}\+\\beta\\cdot\\mathcal\{L\}\_\{kd\}^\{ns\}\(11\)where the weight is set toα=β=1\\alpha\\\!=\\\!\\beta\\\!=\\\!1in experiments\.

![Refer to caption](https://arxiv.org/html/2607.22075v1/x4.png)Figure 4:The Curve of designed negative sampling probability against the length of the negative span\.

## 4Experiments

In this section, we first introduce the datasets and metrics, followed by the training details and the selected baselines\. We then present the overall performance comparisons and conduct an ablation study\. Finally, we analyze the learning efficiency and provide case visualizations\.

### 4\.1Datasets

In CLNER, evaluating models using datasets synthesized from traditional NER benchmarks become common practice\[[2](https://arxiv.org/html/2607.22075#bib.bib2),[11](https://arxiv.org/html/2607.22075#bib.bib11)\], where the original dataset is partitioned into distinct segments, each serving as an individual continual learning task\. Following\[[11](https://arxiv.org/html/2607.22075#bib.bib11)\], we randomly split training data and retain annotations only for entity types designated for the current learning task\. The full test data is preserved for evaluation, but only entity types learned up to the current task are visible for testing\. This setup ensures the inclusion of unannotated entities from previously learned or future types, as well as purely negative sentences that contain no entities, thereby better aligning with realistic CLNER scenario\. We use two source dataset:

OntoNotes5\.0\-EN\[[22](https://arxiv.org/html/2607.22075#bib.bib22)\], annotated with 18 entity types, is converted by selecting 6 types to ensure adequate training samples per task and each task learn single entity type:Organization\(ORG\),Person\(PER\),Geo Political Entity\(GPE\),Date\(DATE\),Cardinal\(CARD\),Nationalities/Religious/Political Groups\(NORP\)\.

FewNERD\[[23](https://arxiv.org/html/2607.22075#bib.bib23)\], hierarchically annotated with 8 coarse\-grained and 66 fine\-grained entity types, is converted by assigning coarse\-grained types for each task, i\.e\., each task learns multiple \(6∼\\sim12\) fine\-grained types\. Coarse\-grained types areLocation\(LOC\),Person\(PER\),Organization\(ORG\),Other\(OTH\),Product\(PROD\),Building\(BUID\),Art\(ART\),Event\(EVET\)\.

### 4\.2Metrics

For each task, model is incrementally trained on its training set and evaluated on its test set using the best checkpoint based on its development set\. To factor out task order impact, we follow previous works\[[2](https://arxiv.org/html/2607.22075#bib.bib2),[11](https://arxiv.org/html/2607.22075#bib.bib11)\]to use 6 permutations for OntoNotes and 4 permutations for FewNERD\. Results are averaged across all permutations and the standard deviations are typically reported at the final step, when all permutations have cumulatively learned the same complete set of entities\. We report Macro\-F1 across all entity types learned up to each incremental step\. Macro\-F1 of the final step is compared between models\. Notably, FewNERD contains multiple types per task, we compute Micro\-F1 for this fine\-grained types within tasks due to severe imbalance, while still maintaining Macro\-F1 for the final coarse\-grained types\. For all span\-based models, we only keep the entity with the highest predicted score and discard the overlapping others when predicting overlapped spans\.

### 4\.3Implementation Details

For contextual encoder, we use PLMbert\-base\-cased\[[24](https://arxiv.org/html/2607.22075#bib.bib24)\]from HuggingFace followed by a BiLSTM with an output hidden size of 400 and subsequently apply 0\.2 dropout rate\. \(i\.e\.,dh=400d^\{h\}\\\!=\\\!400\)\. We setdo=dl=50d^\{o\}\\\!=\\\!d^\{l\}\\\!=\\\!50on FFNs to enable simultaneous modeling of fast and slow expert as treating link computation as an additional head in span modeling\. We fine\-tune all parameters using AdamW optimizer\[[25](https://arxiv.org/html/2607.22075#bib.bib25)\]by setting learning rate1​e−51e^\{\-5\}for PLM bert and1​e−31e^\{\-3\}for the remaining, with schedule of warmup at first 200 steps followed by a cosine decay\. Sentences are tokenized by PLM and truncated to a maximum length of 512 tokens\. We aggregate tokens within each word by using mean\-pooling of BERT output to represent the final word\-level output\. We maintained a consistent batch size of 32 and identical hyperparameters across all datasets, except for training 10 epochs on OntoNotes and 5 epochs on FewNERD\.

### 4\.4Baselines

We compare FSE with the following representative baselines:SeqFTsequentially fine\-tunes the model on new data without any anti\-forgetting strategy based on the conventional BERT sequence tagging, providing the lower\-bound\.AddNERandExtendNERboth incorporate KD under the sequence labeling framework, by using single and multiple heads for learning new tasks, respectively\.L&Rimproves ExtendNER by reviewing and generating synthetic samples to ensure the presence of old entity types\.ExtendNER\+DLDequips ExtendNER with decomposed logits distillation \(DLD\) to strengthen discriminative ability in distillation\.SpanKLadopts span\-based paradigm with binary classification and Bernoulli KD improving the coherence of CLNER\.SKD\-NERbuilds upon SpanKL with using reinforcement learning to optimize KD process\. ForLLM, we select GPT\-5 \(𝚐𝚙𝚝​\-​𝟻​\-​𝟸𝟶𝟸𝟻​\-​𝟶𝟾​\-​𝟶𝟻\\mathtt\{gpt\}\\text\{\-\}\\mathtt\{5\}\\text\{\-\}\\mathtt\{2025\}\\text\{\-\}\\mathtt\{08\}\\text\{\-\}\\mathtt\{05\}\)\[[26](https://arxiv.org/html/2607.22075#bib.bib26)\]and Llama3\.1\-8B\[[27](https://arxiv.org/html/2607.22075#bib.bib27)\]for comparison\. GTP\-5 is directly prompted at each step to recognize the entity types intended to learn so far, while Llama is finetuned using LoRA on the same synthesized CL datasets\. We also evaluate our method under a non\-CL standard training regime, which uses all available training data up to the current step while retaining all annotations of entity types learned so far, as the upper\-bound performance\.

Table 1:Macro\-F1 scores \(%\) of different methods at each incremental step onOntoNotesunder continual learning\.MethodIncremental StepsStep1Step2Step3Step4Step5S​t​e​p​6Step6SeqFT\[[24](https://arxiv.org/html/2607.22075#bib.bib24)\]82\.4656\.7339\.3829\.528\.0723\.64±4​\.​223\.64\_\{\\pm 4\\textbf\{\.\}2\}AddNER\[[2](https://arxiv.org/html/2607.22075#bib.bib2)\]82\.5283\.9084\.6685\.0285\.4885\.03±\.​1885\.03\_\{\\pm\\textbf\{\.\}18\}ExtendNER\[[2](https://arxiv.org/html/2607.22075#bib.bib2)\]82\.7983\.5484\.4884\.6785\.1284\.96±\.​1584\.96\_\{\\pm\\textbf\{\.\}15\}L&R\[[16](https://arxiv.org/html/2607.22075#bib.bib16)\]84\.0282\.4482\.0281\.8883\.2284\.42±\.​2084\.42\_\{\\pm\\textbf\{\.\}20\}ExtendNER\+DLD\[[17](https://arxiv.org/html/2607.22075#bib.bib17)\]83\.9785\.5085\.9486\.4586\.5886\.85±\.​1186\.85\_\{\\pm\\textbf\{\.\}11\}SpanKL\[[11](https://arxiv.org/html/2607.22075#bib.bib11)\]85\.6087\.9288\.2288\.7689\.0288\.98±\.​1088\.98\_\{\\pm\\textbf\{\.\}10\}SKD\-NER\[[18](https://arxiv.org/html/2607.22075#bib.bib18)\]85\.4987\.9888\.3088\.6789\.1188\.92±\.​1388\.92\_\{\\pm\\textbf\{\.\}13\}LLMsGPT\-5\[[26](https://arxiv.org/html/2607.22075#bib.bib26)\]61\.2169\.5373\.8869\.7665\.3167\.2367\.23Llama3\.1\(Finetuned\)78\.3065\.1536\.6225\.179\.1014\.74±1​\.​114\.74\_\{\\pm 1\\textbf\{\.\}1\}Llama3\.1\(Finetuned\)m78\.3076\.6784\.9381\.7375\.2574\.58±\.​8974\.58\_\{\\pm\\textbf\{\.\}89\}FSE \(Ours\)85\.7088\.9388\.8188\.9889\.0289\.23±\.​09\\textbf\{89\.23\}\_\{\\pm\\textbf\{\.\}09\}standard non\-CL\(upper\-bound\)85\.7089\.2389\.2789\.6989\.7590\.07±\.​1390\.07\_\{\\pm\\textbf\{\.\}13\}

Table 2:Macro\-F1 scores \(%\) of different methods at each incremental step onFewNERDunder continual learning\.MethodIncremental StepsStep1Step2Step3Step4Step5Step6Step7S​t​e​p​8Step8SeqFT\[[24](https://arxiv.org/html/2607.22075#bib.bib24)\]63\.8737\.0021\.6114\.6912\.3411\.1210\.797\.21±2​\.​87\.21\_\{\\pm 2\\textbf\{\.\}8\}AddNER\[[2](https://arxiv.org/html/2607.22075#bib.bib2)\]64\.0161\.3260\.5459\.4358\.7459\.3260\.4159\.32±\.​1359\.32\_\{\\pm\\textbf\{\.\}13\}ExtendNER\[[2](https://arxiv.org/html/2607.22075#bib.bib2)\]64\.0659\.0257\.0555\.7255\.4655\.9656\.8556\.16±\.​0856\.16\_\{\\pm\\textbf\{\.\}08\}L&R\[[16](https://arxiv.org/html/2607.22075#bib.bib16)\]64\.0659\.7858\.0755\.8955\.4556\.1057\.2157\.02±\.​1157\.02\_\{\\pm\\textbf\{\.\}11\}ExtendNER\+DLD\[[17](https://arxiv.org/html/2607.22075#bib.bib17)\]64\.0660\.1257\.8356\.2256\.6357\.1758\.0258\.82±\.​1058\.82\_\{\\pm\\textbf\{\.\}10\}SpanKL\[[11](https://arxiv.org/html/2607.22075#bib.bib11)\]67\.8164\.1663\.6262\.3161\.6762\.1763\.2462\.15±\.​0962\.15\_\{\\pm\\textbf\{\.\}09\}SKD\-NER\[[18](https://arxiv.org/html/2607.22075#bib.bib18)\]67\.6864\.5563\.9062\.7861\.8862\.8463\.2062\.42±\.​1162\.42\_\{\\pm\\textbf\{\.\}11\}LLMsGPT\-5\[[26](https://arxiv.org/html/2607.22075#bib.bib26)\]59\.0458\.0353\.8953\.1051\.9549\.7249\.4847\.9247\.92Llama3\.1\(Finetuned\)73\.5839\.0834\.2532\.8617\.4017\.1315\.0413\.03±\.​8413\.03\_\{\\pm\\textbf\{\.\}84\}Llama3\.1\(Finetuned\)m73\.5851\.1446\.2039\.6238\.6139\.8130\.0635\.57±\.​7735\.57\_\{\\pm\\textbf\{\.\}77\}FSE \(Ours\)66\.6162\.4362\.3962\.6462\.2763\.7864\.2063\.69±\.​08\\textbf\{63\.69\}\_\{\\pm\\textbf\{\.\}08\}standard non\-CL\(upper\-bound\)66\.6163\.5363\.7764\.5664\.8966\.6567\.1166\.83±\.​1166\.83\_\{\\pm\\textbf\{\.\}11\}

Table 3:Performance of final step on two datasets evaluated by removing different components of our FSE\.MethodDatasetTraining Throughput\(steps/second\)OntoNotesFewNERDFSE \(Ours\)89\.23±\.​09\{\}\_\{\\pm\\textbf\{\.\}09\}63\.69±\.​08\{\}\_\{\\pm\\textbf\{\.\}08\}≈11\.70\\approx 11\.70w/osharedFastExpert89\.02±\.​08\{\}\_\{\\pm\\textbf\{\.\}08\}63\.43±\.​06\{\}\_\{\\pm\\textbf\{\.\}06\}≈11\.19\\approx 11\.19\(i\.e\., w/o shared information\)w/oFastExpert88\.77±\.​14\{\}\_\{\\pm\\textbf\{\.\}14\}62\.95±\.​11\{\}\_\{\\pm\\textbf\{\.\}11\}≈12\.56\\approx 12\.56w/oNegativeSampling89\.00±\.​12\{\}\_\{\\pm\\textbf\{\.\}12\}63\.46±\.​09\{\}\_\{\\pm\\textbf\{\.\}09\}≈9\.36\\approx 9\.36replace Fast Expert with hard pruning\(pruning span with length ¿ 10\)86\.67±\.​14\{\}\_\{\\pm\\textbf\{\.\}14\}57\.46±\.​12\{\}\_\{\\pm\\textbf\{\.\}12\}≈12\.84\\approx 12\.84

![Refer to caption](https://arxiv.org/html/2607.22075v1/x5.png)Figure 5:The loss curves for the initial 100 steps of each task on both datasets show the convergence speed\. The red line is FSE while blue line is FSE without fast expert\. A faster decrease in loss indicates a lighter learning burden and more efficient continual learning for each task\.
### 4\.5Overall Performance

We report the task\-wise performance after sequentially learning each tasks on two synthesized datasets of different models\. The final step’s metrics best represent the overall performance\.

OntoNotes’s results are shown in Tab\.[1](https://arxiv.org/html/2607.22075#S4.T1), where FSE achieves the highest score at the final step6, demonstrating the strongest recognition capability for all accumulated entity types\.FewNERD’s results are shown in Tab\.[2](https://arxiv.org/html/2607.22075#S4.T2), where FSE consistently maintains SoTA performance even in this challenging scenario typically learning more than 6 entity types per incremental step\. The improvement of FSE is more pronounced here, indicating the potential of shared fast expert to learn sizable entity types\.

LLMperformance on both datasets reveals that neither GPT\-5 nor Llama3\.1 achieve optimal results, primarily due to inherent limitations of prompt\-based method: As the number of predefined entity types filled in the instruction increases, it become challenging to accurately follow instruction\. Also, LLMs may not fully comprehend the predefined entity type names, and the uncontrollable LLM\-generated outputs occasionally lead to parsing failure, both causing incorrect extractions\. This is evidenced by the poorer results on FewNERD which contains total 66 entity types\. Notably, original result of finetuned Llama encounters train\-test inconsistency since the instruction are filled with single task entity types in training but involve multiple tasks in prediction\. We thereby perform multiple predictions \(results withm\), each focusing on a single task entities, and then aggregate them\. This finally improve the performance but still remain suboptimal\.

![Refer to caption](https://arxiv.org/html/2607.22075v1/x6.png)Figure 6:Values probed from the span matrix view of a sampled test sentence on FewNERD, with coarse\-grained types \(ORG,PER\), using FSE final step model\. \(a\) Loose span scores from span modeling by slow expert, showing reduced learning burden\. \(b\) Span\-level fast scores derived from link scores by fast expert, are shared up to the last step\. \(Hollow boxes denote spans of length 1\.\) \(c\) Fused scores between both experts, where fast expert can refine the ’arbitrary’ results from the slow expert to accurately recognize entities\.
### 4\.6Ablation Study

We conduct ablation study to verify the effectiveness of each design in FSE\. The ablated variants includes:w/o shared FEremoves the sharing mechanism of the fast expert across tasks, meaning each task independently initializes a new fast expert, which can exclusively evaluate the effect of using shared information\.w/o FEremoves the entire fast expert module, retaining only the slow expert for span modeling\.w/o NSremoves the negative sampling strategy\. We report the final step result as the overall performance\.

As shown in Tab\.[3](https://arxiv.org/html/2607.22075#S4.T3), removing different components of FSE results in a decline in overall performance\. Non\-shared fast expert leads to a performance drop, which indicates that the sharing mechanism of the fast expert is useful as it allows each task to leverage the previously learned fast expert, utilizing the shared information across tasks\. Removing the entire fast expert module results in a more significant performance degradation, demonstrating the advantage of fast expert in reducing the learning burden to comprehensively enhance model performance\. Results without negative sampling strategy also confirm its contribution to FSE\. The training throughput on H800 when using different components are also reported, showing that fast expert does not significantly slow down the hardware training speed due to its lightweight architecture and the parallel implementation of fast score propagation\. We also evaluate a hard pruning variant that replaces the neural\-learning based link and fast score of the fast expert with a heuristically set length threshold to prune overly long spans\. The threshold is set to 10 to consider avoiding unnecessary loss of long entities appearing in both corpora\. The results show that this rigid strategy reduces performance, as it lacks the flexibility in handling long entities and the plasticity in continual learning\.

### 4\.7In\-Depth Analysis

Learning Efficiency\.FSE is expected to reduce the learning burden for each entity type by rapidly soft filtering out the majority of candidate spans using neural\-based fast scores\. Here, we plot the loss curves during the initial 100 training steps of each task to verify whether FSE improves the convergence rate, where a faster convergence rate implies easier learning\. Fig\.[5](https://arxiv.org/html/2607.22075#S4.F5)shows the loss curves for each training tasks on OntoNotes and FewNERD, respectively\. Compared to removing the fast expert, FSE accelerates learning per task, as reflected in faster loss convergence\. It becomes more pronounced after the first task, indicating that the learned fast expert is successfully shared to promotes the newcome tasks\. Notably, the counterpart \(i\.e\., w/o fast expert, having non\-weak performance in the ablation study\) essentially represents existing competitive span\-based method \(e\.g\., SpanKL, with the same span modeling architecture\)\. The comparison results thereby reveal a defect in current strong span\-based approaches\. Reducing the learning difficulty of each task enable models to better handle the incoming tasks, which explains the improvement during continual learning\. Moreover, the learning efficiency can help save training time by allowing fewer training epochs in practice\.

Visualization\.To intuitively verify the functionality of both experts, we probe the values in the span matrix for visualization\. The span matrix is defined as the scores of each span arranged in a matrix perspective\. Here, we consider three matrices underlying FSE: \(a\) the original span score matrix derived from slow experts, \(b\) the fast score matrix derived from the fast expert, and \(c\) the resulting fused score between them\. Noted that we aggregate the span scores of all entity types by max\-pooling for simultaneous display\. Fig\.[6](https://arxiv.org/html/2607.22075#S4.F6)visualizes a sample randomly selected from test set of FewNERD\.

Evidently, the presence of considerable number of high scores in \(a\) indicate that the slow expert can afford some mistake in span modeling, as these raw scores will ultimately be correctly deactivated by the fast score of fast expert\. This robust span modeling allows the slow expert to focus only on the spans remained in link\-enabled area \(the high scores area in \(b\)\), thereby enhancing its learning capacity\.

## 5Conclusion

This paper propose a fast\-slow experts enhanced span\-based model FSE for CLNER\. The fast expert mitigates task interference by modeling shared link information between adjacent tokens to filter improbable spans across tasks, improving stability\. The slow expert efficiently focuses on the remaining spans to reduce learning burden, improving plasticity\. The tailored length\-decay negative sampling strategy applied to both experts handles the span imbalance issue\. Comprehensive experiments on synthetic datasets derived from real\-world corpora OntoNotes and FewNERD confirm that FSE achieves superior performance compared to baseline including LLM\-based competitor\. The ablation studies validating the contribution of each component\. The loss curve verifies the accelerated convergence per task, and the visualized results validate the expected functionality of both experts\. Future work will explore more sophisticated link modeling mechanisms and extend this framework to broader information extraction tasks under continual learning\.

## References

- Wang et al\. \[2024\]L\. Wang, X\. Zhang, H\. Su, J\. Zhu,A comprehensive survey of continual learning: Theory, method and application,IEEE transactions on pattern analysis and machine intelligence 46 \(2024\) 5362–5383\.
- Monaikul et al\. \[2021\]N\. Monaikul, G\. Castellucci, S\. Filice, O\. Rokhlenko,Continual learning for named entity recognition,in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2021, pp\. 13570–13577\.
- Benkő \[2024\]B\. Benkő,Example forgetting and rehearsal in continual learning,Pattern Recognit\. Lett\. 179 \(2024\) 65–72\.
- Zhou and Song \[2024\]D\. Zhou, Y\. Song,Pnsp: Overcoming catastrophic forgetting using primary null space projection in continual learning,Pattern Recognit\. Lett\. 179 \(2024\) 137–143\.
- Seow et al\. \[2025\]W\. L\. Seow, I\. Chaturvedi, A\. Hogarth, R\. Mao, E\. Cambria,A review of named entity recognition: from learning methods to modelling paradigms and tasks,Artif\. Intell\. Rev\. 58 \(2025\) 315\.
- Liu et al\. \[2025\]H\. Liu, X\. Xin, W\. Peng, J\. Song, J\. Sun,Concept\-driven knowledge distillation and pseudo label generation for continual named entity recognition,Expert Systems with Applications 270 \(2025\) 126546\.
- Dohare et al\. \[2024\]S\. Dohare, J\. F\. Hernandez\-Garcia, Q\. Lan, P\. Rahman, A\. R\. Mahmood, R\. S\. Sutton,Loss of plasticity in deep continual learning,Nature 632 \(2024\) 768 – 774\.
- Lample et al\. \[2016\]G\. Lample, M\. Ballesteros, S\. Subramanian, K\. Kawakami, C\. Dyer,Neural architectures for named entity recognition,in: Proceedings of NAACL\-HLT, 2016, pp\. 260–270\.
- Straková et al\. \[2019\]J\. Straková, M\. Straka, J\. Hajič,Neural architectures for nested ner through linearization,arXiv preprint arXiv:1908\.06926 \(2019\)\.
- Yu et al\. \[2020\]J\. Yu, B\. Bohnet, M\. Poesio,Named entity recognition as dependency parsing,in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020\.
- Zhang and Chen \[2023\]Y\. Zhang, Q\. Chen,A neural span\-based continual named entity recognition model,in: Proceedings of the AAAI conference on artificial intelligence, volume 37, 2023, pp\. 13993–14001\.
- Li et al\. \[2020\]X\. Li, J\. Feng, Y\. Meng, Q\. Han, F\. Wu, J\. Li,A unified MRC framework for named entity recognition,in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020\.
- Fei and Xu \[2023\]Y\. Fei, X\. Xu,Gfmrc: A machine reading comprehension model for named entity recognition,Pattern Recognition Letters 172 \(2023\) 97–105\.
- Yan et al\. \[2021\]H\. Yan, T\. Gui, J\. Dai, Q\. Guo, Z\. Zhang, X\. Qiu,A unified generative framework for various ner subtasks,in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, 2021\.
- Lu et al\. \[2022\]Y\. Lu, Q\. Liu, D\. Dai, X\. Xiao, H\. Lin, X\. Han, L\. Sun, H\. Wu,Unified structure generation for universal information extraction,in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022, pp\. 5755–5772\.
- Xia et al\. \[2022\]Y\. Xia, Q\. Wang, Y\. Lyu, Y\. Zhu, W\. Wu, S\. Li, D\. Dai,Learn and review: Enhancing continual named entity recognition via reviewing synthetic samples,in: Findings of the association for computational linguistics: ACL 2022, 2022, pp\. 2291–2300\.
- Zhang et al\. \[2023\]D\. Zhang, Y\. Yu, F\. Chen, X\. Chen,Decomposing logits distillation for incremental named entity recognition,in: Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, 2023, pp\. 1919–1923\.
- Chen and He \[2023\]Y\. Chen, L\. He,Skd\-ner: Continual named entity recognition via span\-based knowledge distillation with reinforcement learning,in: Proceedings of the 2023 conference on empirical methods in natural language processing, 2023, pp\. 6689–6700\.
- Yuan and Zhao \[2024\]B\. Yuan, D\. Zhao,A survey on continual semantic segmentation: Theory, challenge, method and application,IEEE Transactions on Pattern Analysis and Machine Intelligence 46 \(2024\) 10891–10910\.
- Zhang et al\. \[2022\]C\.\-B\. Zhang, J\.\-W\. Xiao, X\. Liu, Y\.\-C\. Chen, M\.\-M\. Cheng,Representation compensation networks for continual semantic segmentation,in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp\. 7053–7064\.
- Yuan et al\. \[2024\]B\. Yuan, D\. Zhao, Z\. Shi,Learning at a glance: Towards interpretable data\-limited continual semantic segmentation via semantic\-invariance modelling,IEEE Transactions on Pattern Analysis and Machine Intelligence 46 \(2024\) 7909–7923\.
- Weischedel et al\. \[2013\]R\. Weischedel, M\. Palmer, M\. Marcus, E\. Hovy, S\. Pradhan, L\. Ramshaw, N\. Xue, A\. Taylor, J\. Kaufman, M\. Franchini, M\. El\-Bachouti, R\. Belvin, A\. Houston, Ontonotes Release 5\.0, 2013\.
- Ding et al\. \[2021\]N\. Ding, G\. Xu, Y\. Chen, X\. Wang, X\. Han, P\. Xie, H\.\-T\. Zheng, Z\. Liu,Few\-nerd: A few\-shot named entity recognition dataset,in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, 2021, pp\. 3198–3213\.
- Devlin et al\. \[2019\]J\. Devlin, M\.\-W\. Chang, K\. Lee, K\. Toutanova,Bert: Pre\-training of deep bidirectional transformers for language understanding,in: North American Chapter of the Association for Computational Linguistics, 2019, pp\. 4171–4186\.
- Loshchilov and Hutter \[2017\]I\. Loshchilov, F\. Hutter,Decoupled weight decay regularization,in: International Conference on Learning Representations, 2017\.
- OpenAI \[2025\]OpenAI, Gpt\-5, 2025\. URL:[https://openai\.com/zh\-Hans\-CN/gpt\-5/](https://openai.com/zh-Hans-CN/gpt-5/)\.
- Grattafiori et al\. \[2024\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, et al\.,The llama 3 herd of models,arXiv preprint arXiv:2407\.21783 \(2024\)\.