视觉类增量学习方法能迁移到蛋白质领域吗？冻结蛋白质编码器的诊断性基准评测

# Do Vision CIL Methods Transfer to Protein? A Diagnostic Benchmark of Frozen Protein Encoders
Source: [https://arxiv.org/html/2610.18879](https://arxiv.org/html/2610.18879)
\\workshoptitle

Continual Learning in the Era of Foundation Models and Embodied Agents

Ziyuan GaoAffiliation:Department of Computer Science and TechnologyAffiliation:University of CambridgeEmail:[zg344@cam\.ac\.uk](mailto:)

###### Abstract

Protein function annotation operates over an expanding label space, making class\-incremental learning \(CIL\) a natural framework for incorporating newly discovered functions without full retraining\. Yet existing CIL methods have been developed largely in vision and have not been systematically evaluated over frozen protein representations\. We conduct a controlled benchmark of nine representative CIL methods across five sequence\-, structure\-, and sequence–structure\-based protein encoders and two enzyme function datasets at different scales, while keeping the encoders frozen and evaluation settings consistent across representations\. We find that no method transfers reliably across encoders, and that these failures persist across alternative phase constructions\. Diagnostic analyses reveal that this instability arises from weak gradient\-based stability signals, substantial heterogeneity in embedding geometry, and covariance assumptions whose appropriate capacity changes with both dataset scale and encoder\. These findings suggest that the representation itself should be treated as a central component of continual\-learning design rather than a passive feature extractor\. More generally, reliable protein CIL will require encoder\-aware methods that adapt their statistical assumptions and capacity to the geometry of the representation on which they operate\. Code is available at https://github\.com/zygao930/ProteinCIL\.

## 1Introduction

Protein function annotation operates over a label space that continues to grow\. Taking Enzyme Commission \(EC\) numbers as an example, the Swiss\-Prot database expanded from roughly 4,700 EC numbers in 2018 to over 5,000 in 2024\[[1](https://arxiv.org/html/2610.18879#bib.bib24)\]\. A deployed annotation model should therefore absorb newly discovered enzyme classes without retraining on the full historical database whenever the label space changes\[[2](https://arxiv.org/html/2610.18879#bib.bib1),[3](https://arxiv.org/html/2610.18879#bib.bib2)\]\. Class\-incremental learning \(CIL\) provides a natural framework for this setting: a model learns new classes sequentially while preserving discriminative ability on those seen before\[[2](https://arxiv.org/html/2610.18879#bib.bib1),[4](https://arxiv.org/html/2610.18879#bib.bib9),[5](https://arxiv.org/html/2610.18879#bib.bib8)\]\. At the same time, protein annotation increasingly relies on pretrained encoders that provide reusable representations without task\-specific backbone training\[[6](https://arxiv.org/html/2610.18879#bib.bib5),[7](https://arxiv.org/html/2610.18879#bib.bib22)\]\. This raises a question that has not been systematically examined: do CIL methods developed largely for computer vision transfer to frozen protein representations?

Two lines of work meet at this question\. On one side, protein representation learning has shifted from task\-specific supervised models toward general\-purpose pretrained encoders\. Sequence\-based protein language models such as TAPE\[[8](https://arxiv.org/html/2610.18879#bib.bib38)\], ESM\[[9](https://arxiv.org/html/2610.18879#bib.bib39)\], ESM2\[[6](https://arxiv.org/html/2610.18879#bib.bib5)\], and ProtTrans\[[10](https://arxiv.org/html/2610.18879#bib.bib7)\]capture structural and functional signal from sequence, while structure\-aware models such as GearNet\[[7](https://arxiv.org/html/2610.18879#bib.bib22)\], SaProt\[[11](https://arxiv.org/html/2610.18879#bib.bib6)\], and PST\[[12](https://arxiv.org/html/2610.18879#bib.bib28)\]incorporate geometric information\. These encoders support enzyme function and EC\-number prediction\[[13](https://arxiv.org/html/2610.18879#bib.bib21),[14](https://arxiv.org/html/2610.18879#bib.bib20),[15](https://arxiv.org/html/2610.18879#bib.bib30),[16](https://arxiv.org/html/2610.18879#bib.bib31),[17](https://arxiv.org/html/2610.18879#bib.bib29)\], and their embedding spaces organize functionally related proteins\[[18](https://arxiv.org/html/2610.18879#bib.bib18),[19](https://arxiv.org/html/2610.18879#bib.bib19)\]\. Yet this work largely assumes a fixed label space\[[8](https://arxiv.org/html/2610.18879#bib.bib38),[13](https://arxiv.org/html/2610.18879#bib.bib21)\]\. On the other side, class\-incremental learning studies sequential class arrival\[[2](https://arxiv.org/html/2610.18879#bib.bib1)\]\. Classical methods address forgetting through regularization, replay, exemplar memory, or distillation, including EWC\[[5](https://arxiv.org/html/2610.18879#bib.bib8)\], iCaRL\[[4](https://arxiv.org/html/2610.18879#bib.bib9)\], A\-GEM\[[20](https://arxiv.org/html/2610.18879#bib.bib13)\], DER\+\+\[[21](https://arxiv.org/html/2610.18879#bib.bib10)\], and RER\[[22](https://arxiv.org/html/2610.18879#bib.bib14)\]\. Recent approaches increasingly build on pretrained backbones through prompt tuning\[[23](https://arxiv.org/html/2610.18879#bib.bib34),[24](https://arxiv.org/html/2610.18879#bib.bib35)\], classifier alignment\[[25](https://arxiv.org/html/2610.18879#bib.bib33)\], random projection\[[26](https://arxiv.org/html/2610.18879#bib.bib32)\], subspace ensembles\[[27](https://arxiv.org/html/2610.18879#bib.bib36)\], and model surgery\[[28](https://arxiv.org/html/2610.18879#bib.bib37)\]\. Prototype\-based methods provide a complementary route from feature statistics\[[29](https://arxiv.org/html/2610.18879#bib.bib17)\], with FeCAM\[[30](https://arxiv.org/html/2610.18879#bib.bib11)\]using covariance\-aware Mahalanobis classification with shrinkage estimation\[[31](https://arxiv.org/html/2610.18879#bib.bib26),[32](https://arxiv.org/html/2610.18879#bib.bib27)\]\. These methods have been developed and validated largely on vision benchmarks, and whether their assumptions transfer to frozen protein representations remains untested\.

To answer this, we conduct a controlled benchmark of nine representative CIL methods, with Naive fine\-tuning and Joint training as lower and offline reference points, across five frozen protein encoders and two enzyme datasets at different scales\. The encoders span sequence language models, a structure\-based graph network, and a sequence–structure fusion model, and differ substantially in embedding dimension and geometry, as illustrated in Fig\.[1](https://arxiv.org/html/2610.18879#S2.F1)\. We use four phase\-construction protocols, including EC\-aligned and alternative class groupings, to test whether conclusions depend on the incremental stream\. Each method uses one fixed hyperparameter configuration across encoders and datasets, directly testing the plug\-and\-play transfer setting rather than encoder\-specific tuning\. The result is consistent across protocols: no CIL method is reliably strong across representations, and the dominant variation is encoder\-dependent rather than phase\-dependent\. Diagnostic analyses trace this instability to three recurring causes: weak gradient\-based stability signals when only the classifier head is trainable, large differences in encoder\-induced geometry, and covariance capacity requirements that change jointly with dataset scale and representation geometry\.

Our contributions are threefold\. First, we establish the first systematic benchmark of CIL on frozen protein representations, spanning nine CIL methods, five encoders, two enzyme datasets, and four phase construction protocols\. Across this benchmark, no existing method transfers reliably across encoders and dataset scales\. Second, we show that this instability is primarily representation dependent rather than phase dependent\. Method rankings vary substantially across encoders, while the performance patterns remain stable under alternative phase constructions\. We further trace this instability to weak gradient based stability signals, variation in embedding geometry, and covariance capacity requirements that change with both dataset scale and encoder geometry\. Third, we show that these failures reflect a mismatch between the fixed assumptions of existing CIL methods and the representations on which they operate\. This points toward encoder aware protein CIL, where statistical capacity adapts to representation geometry rather than being fixed in advance\.

## 2Method

![Refer to caption](https://arxiv.org/html/2610.18879v1/figure/diagram-protocil-NIPS_main.jpg)Figure 1:Motivation for frozen protein CIL\. Frozen protein encoders induce distinct class geometries, creating a covariance\-capacity trade\-off as new classes arrive\.We formalize class\-incremental learning over frozen protein encoders, then describe the evaluation protocol and the taxonomy of methods our benchmark spans\. Concrete datasets, encoder extraction details, and hyperparameters are deferred to the experimental setup\.

#### Problem formulation\.

We consider class\-incremental learning over a sequence ofTTphases𝒫=\{P0,P1,…,PT−1\}\\mathcal\{P\}=\\\{P\_\{0\},P\_\{1\},\\ldots,P\_\{T\-1\}\\\}, where each phasePtP\_\{t\}introduces a disjoint set of new classes\. A frozen pretrained protein encoderfθf\_\{\\theta\}maps each protein to add\-dimensional embedding

x=fθ​\(protein\)∈ℝd,x=f\_\{\\theta\}\(\\text\{protein\}\)\\in\\mathbb\{R\}^\{d\},and its parametersθ\\thetaare never updated\. After observing phasePtP\_\{t\}, the learner must classify any query protein over all classes encountered so far,

C≤t=⋃i=0tPi,C\_\{\\leq t\}=\\bigcup\_\{i=0\}^\{t\}P\_\{i\},using the frozen protein embeddings together with the method\-specific components learned or maintained during the incremental sequence\. This setting separates continual\-learning behavior from backbone adaptation\[[33](https://arxiv.org/html/2610.18879#bib.bib15),[34](https://arxiv.org/html/2610.18879#bib.bib16)\], allowing different incremental\-learning strategies to be compared under a fixed representation space\.

#### Benchmark workflow\.

We evaluate all methods within a single controlled pipeline\. Each protein is first encoded by a frozen backbone, and the resulting embeddings are cached for all subsequent experiments\. We include five frozen protein encoders with different geometric properties: sequence\-based models ESM2\-650M \(d=1280d\{=\}1280\), ESM2\-150M \(d=640d\{=\}640\)\[[6](https://arxiv.org/html/2610.18879#bib.bib5)\], and ProtBert \(d=1024d\{=\}1024\)\[[10](https://arxiv.org/html/2610.18879#bib.bib7)\]; structure\-based GearNet\-Edge \(d=3072d\{=\}3072\)\[[7](https://arxiv.org/html/2610.18879#bib.bib22)\]; and the sequence\-structure fusion model SaProt \(d=1280d\{=\}1280\)\[[11](https://arxiv.org/html/2610.18879#bib.bib6)\]\. The encoders remain frozen throughout\. Method\-specific hyperparameters are fixed across all encoder\-dataset configurations, with no setting\-specific tuning\.

Classes are introduced under four phase construction protocols that vary in their alignment with the Enzyme Commission \(EC\) hierarchy\[[35](https://arxiv.org/html/2610.18879#bib.bib25)\]\.EC\-firstassigns one phase to each top\-level EC category;Randomshuffles classes across phases while preserving the EC\-first phase sizes;Finer\-ECgroups classes using a finer EC prefix and greedily packs these groups into phases; andMixedcombines classes from different top\-level EC groups within each phase\. Phase sizes are kept comparable across protocols so that performance differences are not driven simply by the number of classes introduced at each step\. For each random seed, phase order is shuffled once, and all methods evaluated under that seed observe exactly the same incremental stream\.

#### Methods and evaluation\.

Nine CIL methods are evaluated under this common protocol, with Naive fine\-tuning and Joint training serving as incremental lower and offline upper reference points, respectively\. Regularization\-based methods are represented by EWC\[[5](https://arxiv.org/html/2610.18879#bib.bib8)\]\. Replay\-based methods include Replay\[[20](https://arxiv.org/html/2610.18879#bib.bib13)\], iCaRL\[[4](https://arxiv.org/html/2610.18879#bib.bib9)\], DER\+\+\[[21](https://arxiv.org/html/2610.18879#bib.bib10)\], and RER\[[22](https://arxiv.org/html/2610.18879#bib.bib14)\], which retain examples from previous phases and differ in how stored information is reused or distilled\. Prototype\- and covariance\-based methods include FeCAM\[[30](https://arxiv.org/html/2610.18879#bib.bib11)\]and FeTrIL\[[36](https://arxiv.org/html/2610.18879#bib.bib12)\], which update class representations using feature statistics\. Projection\- and subspace\-based methods include RanPAC\[[26](https://arxiv.org/html/2610.18879#bib.bib32)\]and EASE\[[27](https://arxiv.org/html/2610.18879#bib.bib36)\], which construct classifiers in transformed embedding spaces\.

After each phase, every method is evaluated on all classes observed so far\. This produces a lower triangular accuracy matrix

R∈ℝT×T,R\\in\\mathbb\{R\}^\{T\\times T\},whereR​\[i\]​\[j\]R\[i\]\[j\]denotes the classification accuracy on phasePjP\_\{j\}after training through phasePiP\_\{i\}, withj≤ij\\leq i\. FromRR, we report three standard continual learning metrics\[[2](https://arxiv.org/html/2610.18879#bib.bib1)\]\. Average accuracy measures final retained performance\[[37](https://arxiv.org/html/2610.18879#bib.bib3)\], forgetting measures degradation from the best historical performance\[[38](https://arxiv.org/html/2610.18879#bib.bib4)\], and plasticity measures new\-class acquisition at each phase:

AvgAcc\\displaystyle\\mathrm\{AvgAcc\}=1T​∑j=0T−1R⁡\[T−1\]​\[j\],\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{j=0\}^\{T\-1\}R\[T\{\-\}1\]\[j\],\(1a\)Fgt\\displaystyle\\mathrm\{Fgt\}=1T−1​∑j=0T−2\(maxi∈\[j,T−2\]⁡R⁡\[i\]​\[j\]−R⁡\[T−1\]​\[j\]\),\\displaystyle=\\frac\{1\}\{T\-1\}\\sum\_\{j=0\}^\{T\-2\}\\left\(\\max\_\{i\\in\[j,T\-2\]\}R\[i\]\[j\]\-R\[T\{\-\}1\]\[j\]\\right\),\(1b\)Plast\\displaystyle\\mathrm\{Plast\}=1T​∑i=0T−1R⁡\[i\]​\[i\]\.\\displaystyle=\\frac\{1\}\{T\}\\sum\_\{i=0\}^\{T\-1\}R\[i\]\[i\]\.\(1c\)All reported results are averaged over random seeds\.

## 3Results

### 3\.1Overall results\.

Table[1](https://arxiv.org/html/2610.18879#S3.T1)reveals that method rankings are unstable across encoders and datasets: the strongest baseline on one encoder is often mediocre on another, and no method dominates all five encoders on either dataset\. Regularization \(EWC\) provides inconsistent gains over the naive lower bound, consistent with the limited gradient signal available when the encoder is frozen\. Replay\-based methods retain high new\-class plasticity but still incur substantial forgetting, particularly on EC Number\. The two prototype methods fail in opposite directions: FeTrIL retains high plasticity yet forgets severely as classes accumulate, while FeCAM preserves old classes effectively but degrades sharply on EC Number, where its plasticity falls near the bottom of the table\. RanPAC is strong on most sequence and sequence–structure encoders but deteriorates sharply on GearNet\-Edge\. Overall, transferring vision\-based CIL recipes to protein encoders yields no uniformly reliable winner, motivating a diagnosis of the representation\-dependent failure modes\.

Table 1:Benchmark results across two datasets and five encoders\. Encoder\-specific results are reported as mean±\\pmstd over 3 seeds\. Best per column inbold, secondunderlined\(Naive and Joint excluded from ranking\)\. Category abbreviations: Reg\. = Regularization, Rep\. = Replay, Sub\. = Subspace, Proj\. = Projection, Proto\. = Prototype\.Enzyme Reaction \(348 classes\)EC Number \(4788 classes\)ESM2ESM2GearNetESM2ESM2GearNetCat\.Method650M150MEdgeSaProtProtBert650M150MEdgeSaProtProtBertAverage Incremental Accuracy \(%\)↑\\uparrowLBNaive13\.7±\\pm0\.512\.6±\\pm1\.57\.4±\\pm1\.413\.4±\\pm0\.710\.7±\\pm0\.525\.4±\\pm1\.623\.5±\\pm1\.48\.6±\\pm0\.320\.7±\\pm0\.821\.4±\\pm0\.2JointJoint MLP73\.9±\\pm0\.869\.1±\\pm0\.852\.1±\\pm1\.171\.2±\\pm0\.465\.5±\\pm0\.399\.5±\\pm0\.299\.5±\\pm0\.168\.4±\\pm0\.799\.4±\\pm0\.099\.5±\\pm0\.0Reg\.EWC21\.8±\\pm4\.017\.7±\\pm4\.16\.9±\\pm1\.120\.8±\\pm4\.015\.0±\\pm1\.845\.2±\\pm5\.440\.2±\\pm2\.98\.7±\\pm1\.232\.6±\\pm1\.840\.2±\\pm1\.3Rep\.Replay57\.2±\\pm2\.052\.5±\\pm1\.534\.5±\\pm1\.953\.9±\\pm0\.644\.9±\\pm1\.958\.2±\\pm2\.652\.8±\\pm2\.225\.7±\\pm0\.451\.0±\\pm1\.551\.5±\\pm1\.6iCaRL62\.5±\\pm0\.656\.4±\\pm0\.836\.9±\\pm0\.860\.1±\\pm0\.851\.2±\\pm0\.459\.5±\\pm1\.752\.9±\\pm2\.124\.0±\\pm1\.950\.6±\\pm1\.052\.2±\\pm0\.9DER\+\+66\.8±\\pm3\.161\.2±\\pm5\.233\.1±\\pm2\.762\.0±\\pm1\.854\.7±\\pm5\.064\.5±\\pm5\.159\.4±\\pm5\.725\.9±\\pm2\.957\.9±\\pm6\.756\.3±\\pm6\.5RER56\.4±\\pm1\.452\.3±\\pm2\.134\.6±\\pm1\.654\.6±\\pm1\.045\.8±\\pm1\.858\.2±\\pm2\.552\.7±\\pm2\.325\.8±\\pm0\.750\.9±\\pm1\.551\.4±\\pm1\.6Sub\.EASE55\.4±\\pm0\.849\.9±\\pm0\.829\.4±\\pm1\.053\.0±\\pm0\.943\.7±\\pm0\.860\.0±\\pm0\.658\.6±\\pm0\.344\.7±\\pm0\.259\.1±\\pm0\.353\.7±\\pm0\.9Proj\.RanPAC74\.3±\\pm0\.070\.5±\\pm0\.021\.8±\\pm0\.073\.5±\\pm0\.065\.8±\\pm0\.082\.2±\\pm0\.079\.9±\\pm0\.026\.1±\\pm0\.084\.7±\\pm0\.079\.0±\\pm0\.0Proto\.FeTrIL64\.1±\\pm6\.646\.9±\\pm6\.431\.0±\\pm4\.547\.6±\\pm5\.643\.0±\\pm4\.947\.6±\\pm6\.946\.5±\\pm6\.323\.8±\\pm4\.246\.0±\\pm6\.246\.3±\\pm6\.5FeCAM75\.0±\\pm0\.067\.1±\\pm0\.043\.4±\\pm0\.072\.5±\\pm0\.069\.6±\\pm0\.055\.8±\\pm0\.051\.7±\\pm0\.037\.7±\\pm0\.047\.7±\\pm0\.030\.2±\\pm0\.0Forgetting \(%\)↓\\downarrowLBNaive76\.3±\\pm1\.273\.3±\\pm1\.554\.9±\\pm3\.073\.3±\\pm0\.968\.9±\\pm1\.085\.1±\\pm2\.386\.3±\\pm2\.052\.6±\\pm5\.787\.6±\\pm1\.585\.6±\\pm0\.7Reg\.EWC66\.5±\\pm4\.666\.4±\\pm3\.953\.9±\\pm2\.465\.2±\\pm4\.762\.5±\\pm1\.862\.0±\\pm6\.566\.7±\\pm3\.252\.2±\\pm4\.573\.7±\\pm1\.463\.7±\\pm2\.2Rep\.Replay23\.0±\\pm2\.524\.7±\\pm2\.124\.2±\\pm1\.325\.3±\\pm0\.228\.0±\\pm1\.247\.5±\\pm3\.452\.8±\\pm3\.736\.1±\\pm5\.053\.6±\\pm2\.851\.7±\\pm2\.8iCaRL16\.3±\\pm1\.420\.1±\\pm2\.321\.0±\\pm1\.917\.4±\\pm1\.519\.8±\\pm1\.346\.5±\\pm2\.353\.6±\\pm2\.436\.5±\\pm6\.654\.9±\\pm1\.852\.3±\\pm1\.3DER\+\+12\.7±\\pm1\.015\.3±\\pm3\.123\.2±\\pm1\.915\.3±\\pm1\.116\.0±\\pm1\.139\.7±\\pm5\.544\.4±\\pm5\.826\.4±\\pm4\.544\.7±\\pm6\.745\.3±\\pm5\.2RER24\.5±\\pm1\.925\.7±\\pm2\.922\.6±\\pm0\.624\.4±\\pm1\.027\.2±\\pm0\.847\.5±\\pm3\.352\.9±\\pm3\.735\.5±\\pm5\.153\.7±\\pm2\.851\.7±\\pm2\.8Sub\.EASE11\.0±\\pm2\.09\.9±\\pm3\.412\.7±\\pm2\.011\.6±\\pm1\.79\.8±\\pm1\.78\.0±\\pm0\.76\.7±\\pm0\.35\.8±\\pm1\.06\.6±\\pm1\.29\.1±\\pm1\.0Proj\.RanPAC6\.8±\\pm2\.16\.1±\\pm1\.87\.3±\\pm3\.84\.0±\\pm1\.75\.7±\\pm2\.57\.3±\\pm1\.37\.5±\\pm1\.23\.3±\\pm1\.45\.1±\\pm0\.96\.9±\\pm1\.3Proto\.FeTrIL21\.4±\\pm7\.340\.5±\\pm8\.127\.8±\\pm5\.339\.8±\\pm9\.439\.3±\\pm8\.167\.9±\\pm10\.767\.9±\\pm10\.532\.9±\\pm4\.267\.2±\\pm10\.366\.4±\\pm10\.9FeCAM5\.8±\\pm1\.68\.0±\\pm0\.99\.8±\\pm2\.65\.6±\\pm0\.76\.1±\\pm1\.23\.4±\\pm1\.13\.4±\\pm1\.33\.9±\\pm1\.43\.8±\\pm1\.24\.6±\\pm1\.5Plasticity \(%\)↑\\uparrowLBNaive79\.1±\\pm0\.775\.4±\\pm0\.854\.4±\\pm1\.676\.3±\\pm0\.469\.7±\\pm0\.398\.4±\\pm0\.797\.5±\\pm0\.753\.6±\\pm5\.295\.8±\\pm1\.294\.8±\\pm0\.8Reg\.EWC78\.9±\\pm0\.274\.7±\\pm0\.953\.1±\\pm1\.176\.6±\\pm0\.168\.6±\\pm0\.898\.3±\\pm0\.697\.4±\\pm0\.753\.4±\\pm5\.095\.8±\\pm1\.294\.8±\\pm0\.7Rep\.Replay76\.9±\\pm0\.373\.7±\\pm1\.152\.0±\\pm2\.675\.1±\\pm0\.468\.7±\\pm0\.698\.2±\\pm0\.697\.3±\\pm0\.753\.6±\\pm4\.095\.7±\\pm1\.194\.6±\\pm0\.8iCaRL76\.3±\\pm0\.673\.6±\\pm2\.253\.4±\\pm2\.574\.5±\\pm0\.768\.1±\\pm0\.898\.8±\\pm0\.898\.2±\\pm0\.954\.8±\\pm6\.896\.7±\\pm1\.496\.0±\\pm1\.4DER\+\+76\.5±\\pm1\.773\.6±\\pm2\.648\.8±\\pm1\.273\.8±\\pm1\.567\.0±\\pm3\.197\.8±\\pm0\.396\.7±\\pm0\.345\.2±\\pm2\.295\.0±\\pm0\.494\.0±\\pm1\.4RER77\.3±\\pm0\.274\.3±\\pm0\.750\.4±\\pm2\.675\.0±\\pm0\.268\.8±\\pm0\.998\.2±\\pm0\.697\.3±\\pm0\.753\.1±\\pm4\.095\.7±\\pm1\.194\.6±\\pm0\.8Sub\.EASE64\.2±\\pm1\.458\.4±\\pm2\.140\.3±\\pm2\.462\.9±\\pm0\.651\.8±\\pm1\.466\.9±\\pm0\.864\.2±\\pm0\.449\.6±\\pm0\.764\.6±\\pm1\.261\.4±\\pm0\.5Proj\.RanPAC80\.1±\\pm1\.875\.7±\\pm1\.527\.6±\\pm3\.176\.8±\\pm1\.470\.5±\\pm1\.988\.5±\\pm1\.186\.3±\\pm1\.128\.9±\\pm1\.289\.1±\\pm0\.884\.8±\\pm1\.1Proto\.FeTrIL79\.7±\\pm0\.877\.2±\\pm0\.452\.7±\\pm0\.377\.0±\\pm0\.673\.0±\\pm1\.398\.8±\\pm0\.298\.1±\\pm0\.444\.3±\\pm1\.497\.1±\\pm0\.497\.0±\\pm0\.5FeCAM80\.0±\\pm1\.473\.9±\\pm0\.851\.8±\\pm2\.277\.2±\\pm0\.574\.8±\\pm1\.058\.7±\\pm0\.854\.6±\\pm1\.141\.0±\\pm1\.250\.9±\\pm1\.034\.1±\\pm1\.3

Figure[2](https://arxiv.org/html/2610.18879#S3.F2)provides a representative view of the phase\-wise behavior on EC Number with ESM2\-650M\. The trajectories show how the aggregate differences in Table[1](https://arxiv.org/html/2610.18879#S3.T1)emerge as classes are introduced, while the right panel shows that performance also varies across top\-level EC families\. Per\-encoder trajectories for the remaining configurations are provided in Appendix Figs\.[4](https://arxiv.org/html/2610.18879#A2.F4)and[5](https://arxiv.org/html/2610.18879#A2.F5)\.

![Refer to caption](https://arxiv.org/html/2610.18879v1/figure/diagram-protocil-NIPS_acc-fgt.jpg)Figure 2:Continual\-learning performance on EC Number with ESM2\-650M\.Left:Average incremental accuracy across phases\.Middle:Forgetting across phases\.Right:Final accuracy across the seven top\-level EC families\.
### 3\.2Robustness Across Phase Construction Protocols

#### Robustness to phase construction\.

Table[2](https://arxiv.org/html/2610.18879#S3.T2)examines whether method rankings depend on the EC\-first phase construction used in the main experiments\. The overall performance pattern remains broadly stable across all three alternative protocols\. RanPAC remains the strongest method on EC Number, while RanPAC and FeCAM are consistently strong on Enzyme Reaction; DER\+\+ also remains competitive across both datasets\. EWC and FeTrIL remain among the weakest methods\. On Enzyme Reaction, RanPAC and FeCAM achieve 60–63% accuracy across all protocols\. On EC Number, RanPAC remains near 69%, whereas FeCAM reaches 44–50%\. Finer\-EC gives modestly higher accuracy than Random or Mixed for several methods\. For example, FeCAM increases from 44\.7% to 49\.5% on EC Number\. This may reflect reduced inter\-phase interference when functionally related classes are grouped together, but it does not alter the broader performance pattern\. Forgetting and plasticity also preserve the same qualitative structure across protocols\. FeCAM, EASE, and RanPAC maintain low forgetting, whereas most replay\- and regularization\-based methods show substantially higher forgetting\. Large cross\-encoder standard deviations also persist across protocols, with RanPAC varying by approximately±\\pm20–28%\. This suggests that the encoder\-dependent variability observed in the main experiments is not specific to a particular phase schedule\. Overall, the main diagnostic patterns remain robust across different phase constructions\. Per\-encoder results for the three alternative protocols are reported in Appendix Tables[9](https://arxiv.org/html/2610.18879#A2.T9)and[10](https://arxiv.org/html/2610.18879#A2.T10)\.

Table 2:Protocol sanity check with baselines under alternative phase constructions\. We report encoder\-average Avg Acc, Forgetting, and Plasticity \(%, mean±\\pmstd over the five encoders; per\-encoder values are averaged over three seeds\) under the random, finer\-EC, and mixed phase\-construction protocols\.Data\-setMethodRandom classesFiner\-EC bucketsMixed hierarchyAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowEnzymeReactionEWC14\.5±\\pm2\.876\.3±\\pm4\.679\.9±\\pm6\.514\.8±\\pm3\.171\.5±\\pm4\.076\.1±\\pm6\.515\.4±\\pm3\.376\.5±\\pm3\.581\.0±\\pm6\.2FeTrIL21\.4±\\pm2\.268\.3±\\pm6\.479\.9±\\pm7\.623\.4±\\pm3\.062\.6±\\pm4\.877\.0±\\pm7\.222\.1±\\pm2\.768\.1±\\pm5\.580\.5±\\pm7\.4EASE39\.2±\\pm9\.414\.8±\\pm0\.551\.8±\\pm9\.442\.8±\\pm9\.912\.6±\\pm0\.753\.5±\\pm9\.939\.3±\\pm9\.314\.5±\\pm1\.051\.6±\\pm9\.8RER46\.6±\\pm6\.935\.2±\\pm2\.176\.6±\\pm7\.149\.3±\\pm8\.129\.2±\\pm1\.574\.1±\\pm7\.249\.4±\\pm7\.332\.8±\\pm1\.577\.5±\\pm7\.2Replay46\.5±\\pm6\.935\.5±\\pm2\.076\.8±\\pm6\.748\.9±\\pm8\.229\.8±\\pm2\.074\.3±\\pm6\.749\.1±\\pm6\.933\.1±\\pm0\.977\.6±\\pm6\.7iCaRL49\.5±\\pm7\.730\.7±\\pm1\.875\.7±\\pm6\.253\.7±\\pm8\.723\.9±\\pm3\.373\.9±\\pm6\.253\.5±\\pm8\.126\.9±\\pm2\.976\.6±\\pm5\.9DER\+\+55\.3±\\pm10\.519\.9±\\pm5\.772\.0±\\pm6\.158\.3±\\pm11\.116\.6±\\pm5\.171\.7±\\pm7\.157\.0±\\pm9\.919\.6±\\pm5\.173\.5±\\pm5\.8RanPAC60\.1±\\pm20\.112\.6±\\pm0\.570\.8±\\pm19\.962\.6±\\pm21\.96\.4±\\pm2\.068\.0±\\pm20\.661\.5±\\pm20\.89\.8±\\pm0\.869\.9±\\pm20\.4FeCAM60\.6±\\pm11\.912\.1±\\pm1\.770\.9±\\pm12\.262\.0±\\pm12\.38\.7±\\pm1\.369\.4±\\pm11\.761\.2±\\pm12\.19\.8±\\pm0\.570\.4±\\pm12\.6ECNumberFeTrIL22\.7±\\pm1\.288\.6±\\pm2\.098\.5±\\pm3\.018\.8±\\pm1\.093\.0±\\pm2\.398\.5±\\pm2\.925\.2±\\pm1\.485\.3±\\pm2\.398\.1±\\pm4\.0EWC31\.5±\\pm8\.978\.2±\\pm6\.998\.6±\\pm3\.037\.4±\\pm12\.671\.3±\\pm11\.498\.6±\\pm2\.928\.4±\\pm6\.881\.0±\\pm2\.897\.8±\\pm4\.6FeCAM44\.7±\\pm10\.26\.5±\\pm0\.850\.3±\\pm10\.849\.5±\\pm10\.33\.7±\\pm0\.452\.7±\\pm10\.446\.2±\\pm10\.55\.5±\\pm0\.850\.8±\\pm10\.9RER49\.3±\\pm8\.658\.1±\\pm8\.098\.7±\\pm2\.653\.5±\\pm10\.753\.1±\\pm10\.598\.7±\\pm2\.546\.5±\\pm6\.661\.0±\\pm5\.198\.1±\\pm3\.9Replay49\.2±\\pm8\.458\.4±\\pm7\.998\.7±\\pm2\.652\.6±\\pm10\.354\.1±\\pm10\.098\.7±\\pm2\.546\.5±\\pm6\.861\.1±\\pm5\.398\.1±\\pm3\.8iCaRL55\.3±\\pm10\.351\.4±\\pm10\.699\.0±\\pm1\.956\.0±\\pm10\.950\.6±\\pm11\.399\.1±\\pm1\.953\.5±\\pm10\.753\.4±\\pm10\.998\.6±\\pm2\.9EASE56\.8±\\pm7\.38\.5±\\pm1\.464\.0±\\pm8\.356\.0±\\pm5\.15\.9±\\pm1\.260\.7±\\pm5\.558\.0±\\pm7\.06\.6±\\pm1\.463\.3±\\pm8\.0DER\+\+63\.6±\\pm12\.441\.3±\\pm11\.998\.5±\\pm3\.165\.3±\\pm13\.439\.0±\\pm12\.698\.5±\\pm3\.262\.2±\\pm12\.442\.8±\\pm11\.898\.0±\\pm4\.2RanPAC68\.7±\\pm26\.09\.6±\\pm2\.476\.9±\\pm27\.569\.3±\\pm27\.87\.1±\\pm1\.875\.3±\\pm28\.968\.9±\\pm26\.111\.1±\\pm2\.578\.5±\\pm27\.6

### 3\.3Diagnostic Analysis

#### Why regularization and geometry assumptions break\.

Table[3](https://arxiv.org/html/2610.18879#S3.T3)isolates two root causes\. For the gradient signal \(Hypothesis A\), inter\-phase conflict cosines are near zero and the Fisher information is highly concentrated in a small fraction of head parameters\. With a frozen encoder, there is therefore little conflicting signal for regularization to preserve, consistent with EWC collapsing toward the naive baseline and RER toward plain Replay\. For embedding geometry \(Hypothesis B\), the within\-class covariance condition number spans roughly five orders of magnitude across encoders, while intrinsic dimensionality and class separability also vary substantially with both encoder and dataset\. These differences show that the representations induce markedly different class geometries, making a single fixed geometric assumption unlikely to transfer uniformly across encoders\.

Table 3:Diagnostic statistics for gradient signal \(Hypothesis A\) and embedding geometry \(Hypothesis B\) across datasets and encoders, averaged over three seeds\. Gradient diagnostics report inter\-phase conflict cosine and top\-1% Fisher concentration on the MLP head\. Geometry diagnostics report embedding dimension, intrinsic rankr95r\_\{95\}, effective rank, within\-class covariance condition number, and between/within class separability\.Gradient signal \(Hypo\.A\)Embedding geometry \(Hypo\.B\)DatasetEncoderConflictcosineFisherconcentrationEmbeddingdimensionIntrinsicrankr95r\_\{95\}EffectiverankConditionnumberClassseparabilityEnzymeReaction\(348 classes\)ESM2\-650M−0\.047\-0\.0470\.90128017917\.2×1051\.5\\\!\\times\\\!10^\{5\}0\.205ESM2\-150M−0\.044\-0\.0440\.88640607\.5×1051\.6\\\!\\times\\\!10^\{5\}0\.203GearNet\-Edge−0\.048\-0\.0480\.9630728431\.7×1097\.1\\\!\\times\\\!10^\{9\}0\.241SaProt−0\.045\-0\.0450\.78128017729\.4×1066\.9\\\!\\times\\\!10^\{6\}0\.209ProtBert−0\.046\-0\.0460\.921024759\.6×1048\.6\\\!\\times\\\!10^\{4\}0\.165EC Number\(4788 classes\)ESM2\-650M−0\.010\-0\.0100\.94128021748\.1×1043\.8\\\!\\times\\\!10^\{4\}1\.167ESM2\-150M−0\.012\-0\.0120\.9664010537\.1×1041\.8\\\!\\times\\\!10^\{4\}1\.179GearNet\-Edge−0\.015\-0\.0150\.9830729736\.6×1093\.8\\\!\\times\\\!10^\{9\}0\.966SaProt−0\.013\-0\.0130\.85128018949\.3×1063\.4\\\!\\times\\\!10^\{6\}0\.856ProtBert−0\.016\-0\.0160\.9610241149\.8×1046\.8\\\!\\times\\\!10^\{4\}0\.450

The representation dependence is also visible in Fig\.[3](https://arxiv.org/html/2610.18879#S3.F3)\. The phase\-wise matrices show distinct retention patterns among representative methods under the same frozen representation, while the hierarchy\-level error profiles differ across encoders and datasets\. These patterns are consistent with the geometric differences measured in Table[3](https://arxiv.org/html/2610.18879#S3.T3)\. Additional phase\-wise accuracy matrices across encoder–dataset configurations are provided in Appendix Fig\.[6](https://arxiv.org/html/2610.18879#A2.F6)\.

![Refer to caption](https://arxiv.org/html/2610.18879v1/figure/diagram-protocil-NIPS_Rmatrix-error.jpg)Figure 3:Representative diagnostic patterns across methods and protein encoders\.\(a\)Phase\-wise accuracy matricesRRfor RanPAC, FeCAM, EWC, and FeTrIL on EC Number with ESM2\-650M\.\(b\)\(c\)EC hierarchy error decomposition across encoders on Enzyme Reaction and EC Number\.
#### How the shared\-covariance assumption fails\.

Table[4](https://arxiv.org/html/2610.18879#S3.T4)shows that failures under a single shared covariance are structured rather than uniformly distributed across classes\. Errors concentrate particularly on transferases and hydrolases, two large and functionally distinct EC families that are repeatedly mapped into one another under the shared metric\. Importantly, with the covariance estimator held fixed, the severity of this confusion varies sharply across encoders\. Transferase–hydrolase confusions are especially prominent for ESM2\-150M, whereas the same failure mode is much weaker for SaProt\. This encoder dependence suggests that the limitation is geometric: a single global covariance can be too coarse when different regions of the representation space exhibit substantially different local structure\. Moving to finer covariance estimates can alleviate this mismatch for several encoders, but finer granularity is not universally preferable\. The full cross\-family confusion breakdowns for all five encoders on both datasets are provided in Appendix Table[11](https://arxiv.org/html/2610.18879#A2.T11)\.

Table 4:Representative cross\-family confusions under a single shared covariance on EC Number\. ESM2\-150M and SaProt provide contrasting examples of cross\-family error structure across encoders\. Each row reports a true EC family, its predicted family, the number of misclassified instances, and one representative EC\-level confusion\.ESM2\-150MSaProtTruePred\.\# mis\.ExampleTruePred\.\# mis\.Example3 \(hydro\.\)2 \(transf\.\)4613\.1\.4\.35→\\to2\.4\.1\.2292 \(transf\.\)3 \(hydro\.\)842\.3\.2\.27→\\to3\.2\.2\.62 \(transf\.\)3 \(hydro\.\)2702\.7\.2\.3→\\to3\.5\.1\.763 \(hydro\.\)2 \(transf\.\)773\.5\.4\.26→\\to2\.1\.1\.852 \(transf\.\)1 \(oxidored\.\)1702\.7\.1\.148→\\to1\.1\.1\.1933 \(hydro\.\)1 \(oxidored\.\)393\.5\.4\.19→\\to1\.1\.1\.233 \(hydro\.\)1 \(oxidored\.\)1593\.1\.21\.4→\\to1\.13\.11\.587 \(transl\.\)1 \(oxidored\.\)337\.1\.1\.9→\\to1\.7\.2\.52 \(transf\.\)4 \(lyases\)1172\.4\.2\.9→\\to4\.2\.1\.912 \(transf\.\)1 \(oxidored\.\)282\.7\.2\.3→\\to1\.2\.1\.282 \(transf\.\)6 \(ligases\)1022\.7\.7\.101→\\to6\.5\.1\.22 \(transf\.\)4 \(lyases\)222\.5\.1\.18→\\to4\.4\.1\.20

Table[5](https://arxiv.org/html/2610.18879#S3.T5)probes this trade\-off by comparing three covariance\-free controls with four Mahalanobis variants of increasing covariance granularity\. Euclidean and Cosine NCM perform substantially below the Mahalanobis variants on both datasets, while Ridge also degrades sharply on EC Number\. Among the Mahalanobis variants, the preferred granularity differs across datasets and encoders\. On Enzyme Reaction, full shared covariance is best for all five encoders, outperforming per\-class covariance by 6\.3–22\.4%\. On EC Number, per\-class covariance is best for four of five encoders, improving over full shared covariance by 3\.4–6\.5%\. GearNet is the consistent exception: full shared covariance improves accuracy from 37\.1% to 47\.2% on Enzyme Reaction and from 58\.5% to 72\.4% on EC Number\. Together, these results show that the appropriate covariance capacity depends jointly on dataset scale and encoder\-induced geometry\.

Table 5:Effect of covariance granularity on prototype classification across datasets and encoders\. Classification accuracy is reported on 256\-dimensional PCA projections for three covariance\-free controls and four Mahalanobis variants, averaged over three seeds\. Best result for each dataset–encoder pair is shown inbold\.No covariance assumptionMahalanobis \(covariance granularity\)DatasetEncoderEuclideanNCMCosineNCMRidgeclassifierShared\(diagonal\)FullsharedGroupedK=8K\\\!=\\\!8Per\-classEnzymeReaction\(348 classes\)ESM2\-650M0\.1300\.1670\.5060\.5940\.6100\.4900\.541ESM2\-150M0\.1080\.1470\.4840\.5860\.5910\.4780\.523GearNet\-Edge0\.1830\.1980\.2950\.3710\.4720\.2670\.248SaProt0\.2200\.2360\.4740\.6030\.6150\.5660\.552ProtBert0\.0580\.0740\.4510\.5760\.6040\.4510\.473EC Number\(4788 classes\)ESM2\-650M0\.4360\.4260\.3400\.7600\.7850\.7340\.849ESM2\-150M0\.4680\.4510\.3340\.7410\.7680\.7020\.833GearNet\-Edge0\.3720\.3810\.2070\.5850\.7240\.5530\.563SaProt0\.4290\.4120\.3270\.7520\.7960\.7300\.830ProtBert0\.2060\.2250\.2970\.7120\.7620\.6990\.806

## 4Discussion and Conclusion

Our results show that class\-incremental learning on frozen protein representations is not primarily a matter of choosing a stronger existing CIL baseline\. Across five encoders and two dataset scales, no method is consistently reliable, and the same strategy can move from near\-best to poor performance when only the encoder changes\. The diagnostic analysis identifies three recurring causes\. First, gradient\-based regularization receives little useful stability signal once the encoder is frozen: inter\-phase gradient conflict remains near zero and Fisher information is highly concentrated in a small fraction of classifier parameters\. Second, protein encoders induce markedly different embedding geometries, with large variation in intrinsic dimensionality, covariance conditioning, and class separability\. Third, the statistical capacity required by prototype classifiers changes with both dataset scale and encoder geometry\. These findings suggest that the central difficulty is not catastrophic forgetting alone, but a mismatch between the assumptions built into existing CIL methods and the representation on which they operate\.

This distinguishes frozen protein CIL from the vision settings in which most existing CIL methods, including recent approaches built on pretrained representations, were developed and evaluated\[[2](https://arxiv.org/html/2610.18879#bib.bib1),[33](https://arxiv.org/html/2610.18879#bib.bib15),[34](https://arxiv.org/html/2610.18879#bib.bib16)\]\. Our benchmark spans sequence language models, a structure\-based graph network, and a sequence–structure fusion model\[[6](https://arxiv.org/html/2610.18879#bib.bib5),[7](https://arxiv.org/html/2610.18879#bib.bib22),[11](https://arxiv.org/html/2610.18879#bib.bib6)\], which encode different sources of protein information and produce substantially different representation geometries\. A geometric assumption that is appropriate for one encoder therefore need not transfer to another\. RanPAC, for example, is highly competitive on several encoders but deteriorates sharply on GearNet, while FeCAM preserves old classes extremely well yet loses plasticity as its shared\-covariance assumption becomes too restrictive\. The phase\-construction experiments reinforce this conclusion: changing how classes are distributed across phases alters absolute performance only modestly, while the broader encoder\-dependent patterns remain\. The failures are therefore representation\-dependent rather than driven by phase construction\.

The covariance analysis provides a concrete example of this broader problem\. A single shared covariance offers stable estimation but can be too coarse to represent heterogeneous class geometry, whereas per\-class covariance is more flexible but relies on substantially fewer samples per estimate\. This bias–variance tension is particularly relevant in high\-dimensional covariance estimation, where shrinkage is commonly used to stabilize estimates under limited samples\[[31](https://arxiv.org/html/2610.18879#bib.bib26),[32](https://arxiv.org/html/2610.18879#bib.bib27)\]\. Neither extreme is uniformly preferable in our experiments\. Full shared covariance is best across all five encoders on Enzyme Reaction, while per\-class covariance is best for four of five encoders on EC Number; GearNet remains the exception, preferring shared covariance even at the larger scale\. This suggests a natural direction for protein CIL: covariance capacity should be adapted between these two extremes rather than fixed in advance\. Classes with compatible geometry could share statistical strength, while geometrically distinct regions could retain separate covariance structure\. Such adaptation could also incorporate the EC hierarchy as a biological prior, while allowing representation geometry to determine when that hierarchy should or should not define the statistical grouping\.

Our findings point toward encoder\-aware continual learning as a more suitable design principle for protein representations\. Covariance granularity is one instance of this idea, but the same principle may extend to projection dimension, distance metrics, regularization strength, and memory allocation\. Rather than searching for a vision\-derived CIL recipe that transfers unchanged across protein encoders, future methods should adapt their statistical capacity to the geometry of the representation they receive\. In this sense, the benchmark provides not only a negative transfer result but also a design criterion: reliable protein CIL should respond to encoder geometry instead of assuming it away\.

Our study deliberately isolates continual\-learning behavior under frozen protein representations, and the conclusions should therefore be interpreted within this setting\. Partially or fully adapted encoders may exhibit different stability and geometric properties and remain an important direction for future work\. Our evaluation is also limited to enzyme function classification under the EC hierarchy, and whether the same failure modes extend to other protein tasks remains open\. In addition, our fixed\-hyperparameter protocol is designed to evaluate plug\-and\-play transfer across encoders rather than the best performance achievable through encoder\-specific tuning\. These choices define the scope of the study rather than weaknesses of the benchmark, and they help separate representation\-dependent failure modes from gains that could arise through additional adaptation or tuning\.

## References

- \[1\]\(2023\)UniProt: the universal protein knowledgebase in 2023\.Nucleic Acids Research51\(D1\),pp\. D523–D531\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p1.1)\.
- \[2\]M\. Masana, X\. Liu, B\. Twardowski, M\. Menta, A\. D\. Bagdanov, and J\. van de Weijer\(2023\)Class\-incremental learning: survey and performance evaluation on image classification\.IEEE Transactions on Pattern Analysis and Machine Intelligence45\(5\),pp\. 5513–5533\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p2.2),[§4](https://arxiv.org/html/2610.18879#S4.p2.1)\.
- \[3\]D\. Patil, P\. Malviya, M\. Reymond, Q\. Fournier, and S\. Chandar\(2026\)CoPeP: benchmarking continual pretraining for protein language models\.External Links:2603\.00253Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p1.1)\.
- \[4\]S\. Rebuffi, A\. Kolesnikov, G\. Sperl, and C\. H\. Lampert\(2017\)ICaRL: incremental classifier and representation learning\.In2017 IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 5533–5542\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.6.1),[§1](https://arxiv.org/html/2610.18879#S1.p1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[5\]J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness, G\. Desjardins, A\. A\. Rusu, K\. Milan, J\. Quan, T\. Ramalho, A\. Grabska\-Barwinska, D\. Hassabis, C\. Clopath, D\. Kumaran, and R\. Hadsell\(2017\)Overcoming catastrophic forgetting in neural networks\.Proceedings of the National Academy of Sciences114\(13\),pp\. 3521–3526\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.4.1),[§1](https://arxiv.org/html/2610.18879#S1.p1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[6\]Z\. Lin, H\. Akin, R\. Rao, B\. Hie, Z\. Zhu, W\. Lu, N\. Smetanin, R\. Verkuil, O\. Kabeli, Y\. Shmueli, A\. dos Santos Costa, M\. Fazel\-Zarandi, T\. Sercu, S\. Candido, and A\. Rives\(2023\)Evolutionary\-scale prediction of atomic\-level protein structure with a language model\.Science379\(6637\),pp\. 1123–1130\.Cited by:[Table 7](https://arxiv.org/html/2610.18879#A1.T7.2.2.1.1),[Table 7](https://arxiv.org/html/2610.18879#A1.T7.2.3.1.1),[§1](https://arxiv.org/html/2610.18879#S1.p1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.18879#S4.p2.1)\.
- \[7\]Z\. Zhang, M\. Xu, A\. Jamasb, V\. Chenthamarakshan, A\. Lozano, P\. Das, and J\. Tang\(2023\)Protein representation learning by geometric structure pretraining\.InInternational Conference on Learning Representations,Cited by:[§A\.2](https://arxiv.org/html/2610.18879#A1.SS2.p1.1),[Table 7](https://arxiv.org/html/2610.18879#A1.T7.2.6.1.1),[§1](https://arxiv.org/html/2610.18879#S1.p1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.18879#S4.p2.1)\.
- \[8\]R\. Rao, N\. Bhattacharya, N\. Thomas, Y\. Duan, X\. Chen, J\. Canny, P\. Abbeel, and Y\. S\. Song\(2019\)Evaluating protein transfer learning with tape\.InAdvances in Neural Information Processing Systems 32,Red Hook, NY, USA\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[9\]A\. Rives, J\. Meier, T\. Sercu, S\. Goyal, Z\. Lin, J\. Liu, D\. Guo, M\. Ott, C\. L\. Zitnick, J\. Ma, and R\. Fergus\(2021\)Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences\.Proceedings of the National Academy of Sciences118\(15\),pp\. e2016239118\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[10\]A\. Elnaggar, M\. Heinzinger, C\. Dallago, G\. Rehawi, Y\. Wang, L\. Jones, T\. Gibbs, T\. Feher, C\. Angerer, M\. Steinegger, D\. Bhowmik, and B\. Rost\(2022\)ProtTrans: toward understanding the language of life through self\-supervised learning\.IEEE Transactions on Pattern Analysis and Machine Intelligence44\(10\),pp\. 7112–7127\.Cited by:[Table 7](https://arxiv.org/html/2610.18879#A1.T7.2.4.1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px2.p1.1)\.
- \[11\]J\. Su, C\. Han, Y\. Zhou, J\. Shan, X\. Zhou, and F\. Yuan\(2024\)SaProt: protein language modeling with structure\-aware vocabulary\.InInternational Conference on Learning Representations,pp\. 6987–7009\.Cited by:[§A\.2](https://arxiv.org/html/2610.18879#A1.SS2.p1.1),[Table 7](https://arxiv.org/html/2610.18879#A1.T7.2.5.1.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2610.18879#S4.p2.1)\.
- \[12\]P\. Hartout, D\. Chen, P\. Pellizzoni, C\. Oliver, and K\. Borgwardt\(2025\)Endowing protein language models with structural knowledge\.Bioinformatics41\(11\),pp\. btaf582\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[13\]T\. Yu, H\. Cui, J\. C\. Li, Y\. Luo, G\. Jiang, and H\. Zhao\(2023\)Enzyme function prediction using contrastive learning\.Science379\(6639\),pp\. 1358–1363\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[14\]P\. Hermosilla, M\. Schäfer, M\. Lang, G\. Fackelmann, P\. Vázquez, B\. Kozlikova, M\. Krone, T\. Ritschel, and T\. Ropinski\(2021\)Intrinsic\-extrinsic convolution and pooling for learning on 3d protein structures\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[15\]Y\. Yang, A\. Jerger, S\. Feng, Z\. Wang, C\. Brasfield, M\. S\. Cheung, J\. Zucker, and Q\. Guan\(2024\)Improved enzyme functional annotation prediction using contrastive learning with structural inference\.Communications Biology7\(1\),pp\. 1690\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[16\]W\. Wang, Y\. Shuai, M\. Zeng,et al\.\(2025\)DPFunc: accurately predicting protein function via deep learning with domain\-guided structure information\.Nature Communications16\(1\),pp\. 70\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[17\]R\. Schmirler, M\. Heinzinger, and B\. Rost\(2024\)Fine\-tuning protein language models boosts predictions across diverse tasks\.Nature Communications15,pp\. 7407\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[18\]D\. Wang, M\. Pourmirzaei, U\. L\. Abbas, S\. Zeng, N\. Manshour, F\. Esmaili, B\. Poudel, Y\. Jiang, Q\. Shao, J\. Chen, and D\. Xu\(2025\)S\-plm: structure\-aware protein language model via contrastive learning between sequence and structure\.Advanced Science12\(5\),pp\. 2404212\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[19\]W\. Yeung, Z\. Zhou, L\. Mathew, N\. Gravel, R\. Taujale, B\. O’Boyle, M\. Salcedo, A\. Venkat, W\. Lanzilotta, S\. Li, and N\. Kannan\(2023\)Tree visualizations of protein sequence embedding space enable improved functional clustering of diverse protein superfamilies\.Briefings in Bioinformatics24\(1\),pp\. bbac619\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[20\]A\. Chaudhry, M\. Rohrbach, M\. Elhoseiny, T\. Ajanthan, P\. K\. Dokania, P\. H\. S\. Torr, and M\. Ranzato\(2019\)On tiny episodic memories in continual learning\.External Links:1902\.10486Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.5.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[21\]P\. Buzzega, M\. Boschini, A\. Porrello, D\. Abati, and S\. Calderara\(2020\)Dark experience for general continual learning: a strong, simple baseline\.InAdvances in Neural Information Processing Systems 33,Red Hook, NY, USA\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.7.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[22\]Q\. Wang, R\. Wang, Y\. Li, D\. Wei, H\. Wang, K\. Ma, Y\. Zheng, and D\. Meng\(2024\)Relational experience replay: continual learning by adaptively tuning task\-wise relationship\.IEEE Transactions on Multimedia26,pp\. 9683–9698\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.8.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[23\]Z\. Wang, Z\. Zhang, S\. Ebrahimi, R\. Sun, H\. Zhang, C\. Lee, X\. Ren, G\. Su, V\. Perot, J\. Dy, and T\. Pfister\(2022\)DualPrompt: complementary prompting for rehearsal\-free continual learning\.InProceedings of the European Conference on Computer Vision,pp\. 631–648\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[24\]J\. S\. Smith, L\. Karlinsky, V\. Gutta, P\. Cascante\-Bonilla, D\. Kim, A\. Arbelle, R\. Panda, R\. Feris, and Z\. Kira\(2023\)CODA\-prompt: continual decomposed attention\-based prompting for rehearsal\-free continual learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11909–11919\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[25\]G\. Zhang, L\. Wang, G\. Kang, L\. Chen, and Y\. Wei\(2023\)SLCA: slow learner with classifier alignment for continual learning on a pre\-trained model\.In2023 IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 19091–19101\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[26\]M\. D\. McDonnell, D\. Gong, A\. Parveneh, E\. Abbasnejad, and A\. van den Hengel\(2023\)RanPAC: random projections and pre\-trained models for continual learning\.InAdvances in Neural Information Processing Systems 36,Red Hook, NY, USA\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.10.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[27\]D\. Zhou, H\. Sun, H\. Ye, and D\. Zhan\(2024\)Expandable subspace ensemble for pre\-trained model\-based class\-incremental learning\.In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 23554–23564\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.11.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[28\]H\. Sun, D\. Zhou, H\. Zhao, L\. Gan, D\. Zhan, and H\. Ye\(2025\)MOS: model surgery for pre\-trained model\-based class\-incremental learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 20699–20707\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[29\]T\. Mensink, J\. Verbeek, F\. Perronnin, and G\. Csurka\(2013\)Distance\-based image classification: generalizing to new classes at near\-zero cost\.IEEE Transactions on Pattern Analysis and Machine Intelligence35\(11\),pp\. 2624–2637\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1)\.
- \[30\]D\. Goswami, Y\. Liu, B\. Twardowski, and J\. van de Weijer\(2023\)FeCAM: exploiting the heterogeneity of class distributions in exemplar\-free continual learning\.InAdvances in Neural Information Processing Systems 36,Red Hook, NY, USA\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.12.1),[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[31\]O\. Ledoit and M\. Wolf\(2004\)A well\-conditioned estimator for large\-dimensional covariance matrices\.Journal of Multivariate Analysis88\(2\),pp\. 365–411\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§4](https://arxiv.org/html/2610.18879#S4.p3.1)\.
- \[32\]Y\. Chen, A\. Wiesel, Y\. C\. Eldar, and A\. O\. Hero\(2010\)Shrinkage algorithms for mmse covariance estimation\.IEEE Transactions on Signal Processing58\(10\),pp\. 5016–5029\.Cited by:[§1](https://arxiv.org/html/2610.18879#S1.p2.1),[§4](https://arxiv.org/html/2610.18879#S4.p3.1)\.
- \[33\]D\. Zhou, H\. Sun, J\. Ning, H\. Ye, and D\. Zhan\(2024\)Continual learning with pre\-trained models: a survey\.InProceedings of the International Joint Conference on Artificial Intelligence \(IJCAI\),pp\. 8363–8371\.Cited by:[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px1.p1.3),[§4](https://arxiv.org/html/2610.18879#S4.p2.1)\.
- \[34\]D\. Zhou, Z\. Cai, H\. Ye, D\. Zhan, and Z\. Liu\(2024\)Revisiting class\-incremental learning with pre\-trained models: generalizability and adaptivity are all you need\.International Journal of Computer Vision133\(3\),pp\. 1012–1032\.Cited by:[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px1.p1.3),[§4](https://arxiv.org/html/2610.18879#S4.p2.1)\.
- \[35\]E\. C\. Webb\(1992\)Enzyme nomenclature 1992: recommendations of the nomenclature committee of the international union of biochemistry and molecular biology on the nomenclature and classification of enzymes\.Academic Press\.Cited by:[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px2.p2.1)\.
- \[36\]G\. Petit, A\. Popescu, H\. Schindler, D\. Picard, and B\. Delezoide\(2023\)FeTrIL: feature translation for exemplar\-free class\-incremental learning\.In2023 IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\),pp\. 3900–3909\.Cited by:[Table 8](https://arxiv.org/html/2610.18879#A1.T8.2.1.9.1),[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p1.1)\.
- \[37\]D\. Lopez\-Paz and M\. Ranzato\(2017\)Gradient episodic memory for continual learning\.InAdvances in Neural Information Processing Systems 30,Red Hook, NY, USA,pp\. 6470–6479\.Cited by:[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p2.2)\.
- \[38\]A\. Chaudhry, P\. K\. Dokania, T\. Ajanthan, and P\. H\. S\. Torr\(2018\)Riemannian walk for incremental learning: understanding forgetting and intransigence\.InProceedings of the European Conference on Computer Vision \(ECCV\),pp\. 556–572\.Cited by:[§2](https://arxiv.org/html/2610.18879#S2.SS0.SSS0.Px3.p2.2)\.
- \[39\]M\. van Kempen, S\. S\. Kim, C\. Tumescheit, M\. Mirdita, J\. Lee, C\. L\. M\. Gilchrist, J\. Söding, and M\. Steinegger\(2024\)Fast and accurate protein structure search with foldseek\.Nature Biotechnology42\(2\),pp\. 243–246\.Cited by:[§A\.1](https://arxiv.org/html/2610.18879#A1.SS1.p1.1)\.

## Declaration of LLM Usage

Large language models were used solely as writing and coding assistants during the preparation of this work\. Specifically, they were used to polish prose, improve clarity, and assist with routine coding tasks such as debugging, result formatting, and figure generation\. All experimental design, analyses, results, and conclusions were determined and verified by the authors\.

## Ethics Statement

This work evaluates class\-incremental learning methods on publicly available protein sequence and function datasets\. It involves no human subjects, personally identifiable information, or newly collected biological samples\. The study is methodological and diagnostic in nature, with the aim of understanding how continual\-learning methods interact with different frozen protein representations\. We are not aware of foreseeable harmful applications arising directly from this work\.

## Reproducibility Statement

We provide anonymized code at[https://anonymous\.4open\.science/r/ProteinCIL/](https://anonymous.4open.science/r/ProteinCIL/)\. Dataset construction, preprocessing, phase protocols, and encoder details are documented in Appendix[A](https://arxiv.org/html/2610.18879#A1), while experimental settings and baseline configurations are provided in Appendix[B](https://arxiv.org/html/2610.18879#A2)\. All protein encoders are kept frozen throughout continual learning, and experiments are evaluated under consistent protocols across representations\. Results are averaged over three random seeds unless otherwise specified\. All datasets and pretrained encoders used in this work are publicly available and cited to their original sources\. All experiments were run on a single NVIDIA RTX 4090 GPU\.

## Appendix ABenchmark Details

### A\.1Datasets and Preprocessing

Table[6](https://arxiv.org/html/2610.18879#A1.T6)summarizes the two datasets and the preprocessing steps used in our benchmark\. Both datasets are converted to a unified format\. Each sample contains an amino acid sequence, a functional label, and an associated protein structure when available\. For Enzyme Reaction, multi\-label EC annotations are converted to a single label using argmax, and the original training and validation splits are merged\. For EC Number, only the first annotation is retained for proteins with multiple labels\. AlphaFold v4 predicted structures are downloaded for EC Number\. Enzyme Reaction structures are obtained from the TorchDrug distribution\. Samples without matched structures are excluded from GearNet\-Edge and SaProt but retained for sequence\-only encoders\. All PDB files are processed with Foldseek\[[39](https://arxiv.org/html/2610.18879#bib.bib23)\]to generate the per\-residue 3Di structural tokens required by SaProt\.

Table 6:Dataset statistics\. Counts report the final cached sequence\-encoder splits; structure\-dependent encoders may exclude samples without matched structures\.DatasetClassesTrainTestTrain/classPhasesEnzyme Reaction34829,2155,651∼\\sim847EC Number4,78873,2599,112∼\\sim157
### A\.2Encoders and Embedding Extraction

Table[7](https://arxiv.org/html/2610.18879#A1.T7)lists the five frozen encoders and their extraction settings\. Each encoder is applied once to every protein, and the resulting embeddings are cached and reused across all CIL methods\. ESM2 models use mean pooling over non\-special token representations\. ProtBert and SaProt use attention\-weighted mean pooling\. SaProt\[[11](https://arxiv.org/html/2610.18879#bib.bib6)\]combines each amino acid with its corresponding Foldseek 3Di structural token\. Proteins without matched 3Di sequences are excluded from SaProt\. GearNet\-Edge\[[7](https://arxiv.org/html/2610.18879#bib.bib22)\]constructs multi\-relational residue graphs from 3D coordinates\. Its embeddings are L2\-normalized to reduce the effect of magnitude differences on distance\-based computations\. Sequences that exceed encoder\-specific length limits are truncated\.

Table 7:Frozen encoders and embedding extraction details\.EncoderTypeInputDim\.PoolingESM2\-650M\[[6](https://arxiv.org/html/2610.18879#bib.bib5)\]Sequence PLMAA sequence1280Mean pool, final layerESM2\-150M\[[6](https://arxiv.org/html/2610.18879#bib.bib5)\]Sequence PLMAA sequence640Mean pool, final layerProtBert\[[10](https://arxiv.org/html/2610.18879#bib.bib7)\]Sequence PLMSpace\-separated AA1024Attention\-weighted meanSaProt\[[11](https://arxiv.org/html/2610.18879#bib.bib6)\]Seq\.\+Struct\.AA \+ 3Di tokens1280Attention\-weighted meanGearNet\-Edge\[[7](https://arxiv.org/html/2610.18879#bib.bib22)\]Structure GNNResidue graph from 3D coords3072Node mean; L2 norm\.
### A\.3Baselines and Hyperparameters

Table[8](https://arxiv.org/html/2610.18879#A1.T8)lists the baseline methods and their configurations\. All MLP\-based methods use a two\-layer classifier with hidden dimension 1024\. They are trained with Adam at a learning rate of10−310^\{\-3\}for 100 epochs per phase\. Replay, iCaRL, DER\+\+, and RER use a reservoir buffer of 2,000 samples\. DER\+\+ usesα=0\.5\\alpha\{=\}0\.5andβ=0\.5\\beta\{=\}0\.5\. EWC uses a Fisher penalty weight ofλ=2000\\lambda\{=\}2000\. FeCAM uses a global shared covariance with ridge regularization \(λ=10−4\\lambda\{=\}10^\{\-4\}\)\. FeTrIL generates pseudo\-features for old classes by translating new\-class embeddings using the difference between old and new class means\. It then retrains an MLP classifier on the combined real and pseudo features\. RanPAC uses a fixed Gaussian random projection withD=12800D\{=\}12800and ridge parameterλ=1\.0\\lambda\{=\}1\.0\. EASE trains a rank\-256 low\-rank adapter for each phase using the same learning rate and epoch count as the MLP\-based methods\. All hyperparameters are fixed across encoder\-dataset configurations, with no setting\-specific tuning\.

Table 8:Baseline CIL methods and configurations\.MethodStrategyCore MechanismKey HyperparametersNaiveLower boundTrain on new phase onlyHidden=1024, lr=10−310^\{\-3\}, 100 epochsJoint MLPJoint referenceRetrain on all dataSame as NaiveEWC\[[5](https://arxiv.org/html/2610.18879#bib.bib8)\]RegularizationFisher\-weighted L2 penaltyλ=2000\\lambda\{=\}2000Replay\[[20](https://arxiv.org/html/2610.18879#bib.bib13)\]ReplayFixed\-size reservoir bufferBuffer=2,000iCaRL\[[4](https://arxiv.org/html/2610.18879#bib.bib9)\]Replay \+ DistillationExemplar replay \+ distillationBuffer=2,000DER\+\+\[[21](https://arxiv.org/html/2610.18879#bib.bib10)\]Replay \+ DistillationLogit replay \+ label replayBuffer=2,000,α=0\.5\\alpha\{=\}0\.5,β=0\.5\\beta\{=\}0\.5RER\[[22](https://arxiv.org/html/2610.18879#bib.bib14)\]Adaptive replayGradient\-cosine adaptiveα\\alphaBuffer=2,000FeTrIL\[[36](https://arxiv.org/html/2610.18879#bib.bib12)\]Pseudo\-featureClass\-mean translation \+ MLP classifierHidden=1024, lr=10−310^\{\-3\}, 100 epochsRanPAC\[[26](https://arxiv.org/html/2610.18879#bib.bib32)\]ProjectionRandom projection \+ ridge regressionD=12800D\{=\}12800,λ=1\.0\\lambda\{=\}1\.0EASE\[[27](https://arxiv.org/html/2610.18879#bib.bib36)\]SubspaceLearned per\-task adapter \+ prototypesRank=256, lr=10−310^\{\-3\}, 100 epochsFeCAM\[[30](https://arxiv.org/html/2610.18879#bib.bib11)\]PrototypeShared Mahalanobis distanceRidgeλ=10−4\\lambda\{=\}10^\{\-4\}

## Appendix BExperimental Details

### B\.1Per\-Encoder Breakdown

Tables[9](https://arxiv.org/html/2610.18879#A2.T9)and[10](https://arxiv.org/html/2610.18879#A2.T10)report the per\-encoder results underlying the encoder\-averaged values in Table[2](https://arxiv.org/html/2610.18879#S3.T2)\. Each block covers one encoder on both datasets, with values averaged over three seeds\. Across encoders, changing the phase construction produces only modest within\-encoder variation, while substantial differences between encoders persist\. On EC Number, replay\- and regularization\-based methods generally retain high plasticity but incur substantial forgetting, whereas FeCAM and EASE maintain much lower forgetting with reduced plasticity\. These results support the main finding that phase construction has a smaller effect than the choice of frozen protein representation\.

Table 9:Per\-encoder breakdown of the protocol sanity check \(part 1/2: ESM2\-150M, ESM2\-650M\), in the same format as Table[2](https://arxiv.org/html/2610.18879#S3.T2)but reported separately for each encoder instead of averaged over them\. Each encoder block contains both datasets\. Values are mean±\\pmstd over three seeds \(%\)\. Acc = Avg Acc \(↑\\uparrow\), Fgt = Forgetting \(↓\\downarrow\), Plast = Plasticity \(↑\\uparrow\)\.DatasetMethodRandom classesFiner\-EC bucketsMixed hierarchyAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowESM2\-150MEnzymeReactionEWC14\.8±\\pm0\.878\.9±\\pm1\.882\.4±\\pm2\.315\.7±\\pm2\.573\.2±\\pm3\.078\.4±\\pm0\.115\.8±\\pm1\.878\.0±\\pm2\.082\.6±\\pm0\.5FeTrIL22\.4±\\pm5\.070\.7±\\pm6\.883\.0±\\pm1\.724\.4±\\pm5\.262\.9±\\pm6\.078\.3±\\pm0\.423\.3±\\pm4\.969\.9±\\pm5\.083\.2±\\pm0\.6EASE41\.8±\\pm1\.914\.2±\\pm1\.653\.8±\\pm1\.745\.1±\\pm1\.211\.4±\\pm1\.854\.7±\\pm0\.741\.0±\\pm4\.114\.6±\\pm3\.953\.5±\\pm1\.5RER47\.0±\\pm3\.436\.8±\\pm2\.678\.5±\\pm2\.252\.8±\\pm1\.628\.0±\\pm2\.176\.7±\\pm0\.451\.6±\\pm2\.633\.1±\\pm2\.080\.0±\\pm1\.0Replay46\.5±\\pm3\.437\.1±\\pm2\.878\.2±\\pm2\.652\.5±\\pm1\.828\.5±\\pm2\.176\.9±\\pm0\.250\.7±\\pm2\.033\.3±\\pm1\.479\.3±\\pm1\.2iCaRL52\.0±\\pm3\.230\.1±\\pm3\.177\.2±\\pm3\.556\.7±\\pm1\.022\.7±\\pm1\.775\.9±\\pm0\.654\.5±\\pm4\.527\.2±\\pm3\.877\.9±\\pm1\.3DER\+\+57\.6±\\pm3\.119\.4±\\pm0\.574\.2±\\pm3\.161\.6±\\pm5\.315\.3±\\pm2\.873\.9±\\pm2\.358\.4±\\pm2\.819\.9±\\pm1\.975\.2±\\pm1\.8RanPAC68\.3±\\pm3\.111\.9±\\pm2\.678\.4±\\pm2\.070\.8±\\pm0\.36\.5±\\pm2\.076\.4±\\pm1\.470\.9±\\pm0\.99\.0±\\pm0\.478\.6±\\pm0\.8FeCAM62\.3±\\pm4\.514\.6±\\pm7\.774\.8±\\pm2\.563\.4±\\pm1\.19\.6±\\pm2\.271\.7±\\pm1\.064\.2±\\pm3\.310\.8±\\pm3\.773\.4±\\pm0\.1ECNumberEWC35\.7±\\pm3\.175\.0±\\pm3\.699\.9±\\pm0\.043\.5±\\pm6\.465\.8±\\pm7\.499\.9±\\pm0\.030\.6±\\pm5\.280\.9±\\pm6\.199\.9±\\pm0\.0FeTrIL23\.1±\\pm7\.689\.5±\\pm8\.999\.8±\\pm0\.119\.2±\\pm4\.794\.1±\\pm5\.499\.9±\\pm0\.026\.0±\\pm10\.086\.1±\\pm12\.099\.8±\\pm0\.2EASE60\.9±\\pm0\.68\.4±\\pm1\.268\.0±\\pm1\.660\.1±\\pm0\.25\.6±\\pm3\.864\.6±\\pm2\.962\.0±\\pm0\.86\.2±\\pm1\.967\.3±\\pm1\.4RER52\.5±\\pm2\.955\.2±\\pm3\.499\.9±\\pm0\.158\.2±\\pm4\.948\.6±\\pm5\.799\.8±\\pm0\.149\.6±\\pm2\.958\.7±\\pm3\.499\.9±\\pm0\.0Replay52\.1±\\pm4\.955\.8±\\pm5\.799\.9±\\pm0\.057\.4±\\pm5\.149\.5±\\pm6\.099\.8±\\pm0\.048\.9±\\pm2\.859\.4±\\pm3\.299\.8±\\pm0\.1iCaRL60\.1±\\pm4\.246\.5±\\pm5\.099\.9±\\pm0\.061\.3±\\pm2\.145\.0±\\pm2\.499\.9±\\pm0\.058\.4±\\pm4\.948\.4±\\pm5\.799\.9±\\pm0\.0DER\+\+68\.4±\\pm4\.436\.8±\\pm5\.299\.9±\\pm0\.071\.0±\\pm3\.833\.8±\\pm4\.499\.9±\\pm0\.066\.8±\\pm3\.338\.6±\\pm3\.999\.9±\\pm0\.0RanPAC78\.6±\\pm0\.512\.0±\\pm2\.188\.9±\\pm2\.080\.5±\\pm0\.18\.7±\\pm1\.987\.9±\\pm1\.679\.6±\\pm1\.312\.8±\\pm2\.990\.6±\\pm2\.5FeCAM51\.7±\\pm1\.37\.2±\\pm1\.957\.9±\\pm2\.656\.2±\\pm0\.13\.9±\\pm1\.059\.5±\\pm1\.053\.8±\\pm1\.25\.9±\\pm1\.558\.8±\\pm0\.7ESM2\-650MEnzymeReactionEWC16\.2±\\pm1\.179\.6±\\pm0\.784\.4±\\pm1\.717\.3±\\pm1\.275\.2±\\pm2\.581\.8±\\pm0\.918\.4±\\pm2\.879\.7±\\pm3\.086\.7±\\pm0\.3FeTrIL22\.9±\\pm4\.973\.6±\\pm5\.985\.6±\\pm0\.725\.7±\\pm5\.766\.5±\\pm6\.982\.7±\\pm0\.323\.8±\\pm5\.373\.1±\\pm5\.886\.5±\\pm0\.5EASE47\.2±\\pm2\.515\.1±\\pm3\.460\.1±\\pm0\.549\.9±\\pm1\.512\.8±\\pm2\.260\.8±\\pm0\.447\.2±\\pm4\.215\.9±\\pm3\.660\.8±\\pm1\.2RER53\.8±\\pm3\.833\.1±\\pm2\.282\.1±\\pm1\.956\.0±\\pm1\.227\.8±\\pm0\.779\.8±\\pm0\.656\.4±\\pm2\.031\.1±\\pm1\.383\.1±\\pm1\.3Replay54\.0±\\pm3\.932\.8±\\pm2\.782\.1±\\pm1\.755\.4±\\pm0\.928\.2±\\pm1\.179\.6±\\pm0\.455\.8±\\pm1\.831\.7±\\pm0\.783\.0±\\pm1\.3iCaRL56\.1±\\pm3\.729\.4±\\pm3\.981\.3±\\pm0\.460\.7±\\pm1\.121\.4±\\pm1\.679\.0±\\pm0\.459\.7±\\pm2\.325\.2±\\pm0\.881\.3±\\pm1\.8DER\+\+65\.3±\\pm3\.414\.8±\\pm1\.177\.9±\\pm2\.768\.6±\\pm3\.612\.0±\\pm1\.178\.6±\\pm2\.765\.9±\\pm4\.416\.2±\\pm3\.179\.4±\\pm1\.7RanPAC72\.9±\\pm3\.012\.7±\\pm4\.383\.8±\\pm1\.075\.6±\\pm0\.16\.3±\\pm1\.780\.8±\\pm1\.574\.7±\\pm1\.19\.5±\\pm0\.682\.9±\\pm0\.7FeCAM69\.1±\\pm4\.211\.9±\\pm5\.679\.0±\\pm0\.970\.0±\\pm0\.59\.2±\\pm1\.577\.9±\\pm0\.869\.4±\\pm2\.611\.3±\\pm3\.879\.1±\\pm0\.7ECNumberEWC34\.9±\\pm2\.075\.8±\\pm2\.399\.9±\\pm0\.046\.7±\\pm4\.162\.1±\\pm4\.899\.9±\\pm0\.030\.7±\\pm5\.180\.7±\\pm6\.099\.8±\\pm0\.1FeTrIL23\.4±\\pm7\.889\.3±\\pm9\.199\.9±\\pm0\.119\.2±\\pm4\.794\.0±\\pm5\.499\.8±\\pm0\.126\.1±\\pm10\.286\.1±\\pm11\.999\.9±\\pm0\.0EASE63\.1±\\pm0\.79\.6±\\pm0\.771\.3±\\pm0\.659\.5±\\pm0\.27\.1±\\pm3\.965\.0±\\pm3\.163\.9±\\pm0\.67\.4±\\pm2\.870\.1±\\pm3\.2RER56\.6±\\pm1\.350\.6±\\pm1\.599\.9±\\pm0\.062\.0±\\pm6\.044\.2±\\pm7\.099\.9±\\pm0\.050\.8±\\pm3\.657\.2±\\pm4\.299\.8±\\pm0\.1Replay56\.1±\\pm1\.351\.2±\\pm1\.599\.9±\\pm0\.059\.8±\\pm5\.246\.7±\\pm6\.199\.9±\\pm0\.050\.6±\\pm3\.457\.4±\\pm4\.099\.8±\\pm0\.1iCaRL63\.9±\\pm3\.842\.0±\\pm4\.499\.9±\\pm0\.062\.6±\\pm4\.043\.5±\\pm4\.799\.9±\\pm0\.061\.6±\\pm4\.044\.6±\\pm4\.699\.8±\\pm0\.1DER\+\+72\.4±\\pm4\.432\.2±\\pm5\.199\.9±\\pm0\.075\.1±\\pm4\.129\.0±\\pm4\.899\.9±\\pm0\.070\.7±\\pm2\.934\.0±\\pm3\.399\.8±\\pm0\.1RanPAC81\.9±\\pm0\.310\.0±\\pm1\.990\.4±\\pm1\.683\.1±\\pm0\.18\.0±\\pm1\.589\.9±\\pm1\.382\.1±\\pm1\.311\.6±\\pm2\.592\.1±\\pm2\.1FeCAM55\.3±\\pm1\.36\.8±\\pm1\.861\.1±\\pm2\.560\.1±\\pm0\.13\.5±\\pm0\.763\.1±\\pm0\.756\.6±\\pm1\.05\.8±\\pm1\.761\.5±\\pm0\.7

Table 10:Per\-encoder breakdown of the protocol sanity check \(part 2/2: GearNet\-Edge, ProtBert, SaProt\)\. Same format as Table[9](https://arxiv.org/html/2610.18879#A2.T9)\. Values are mean±\\pmstd over three seeds \(%\)\. Acc = Avg Acc \(↑\\uparrow\), Fgt = Forgetting \(↓\\downarrow\), Plast = Plasticity \(↑\\uparrow\)\.DatasetMethodRandom classesFiner\-EC bucketsMixed hierarchyAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowAcc↑\\uparrowFgt↓\\downarrowPlast↑\\uparrowGearNet\-EdgeEnzymeReactionEWC10\.5±\\pm0\.768\.3±\\pm2\.069\.1±\\pm2\.49\.9±\\pm1\.265\.0±\\pm1\.865\.5±\\pm1\.310\.0±\\pm0\.370\.7±\\pm2\.070\.6±\\pm1\.9FeTrIL17\.5±\\pm3\.957\.3±\\pm4\.166\.6±\\pm2\.818\.5±\\pm2\.954\.3±\\pm4\.764\.6±\\pm1\.517\.7±\\pm4\.858\.6±\\pm4\.667\.9±\\pm0\.9EASE24\.0±\\pm3\.215\.0±\\pm4\.136\.9±\\pm1\.127\.1±\\pm0\.912\.8±\\pm2\.837\.8±\\pm1\.925\.0±\\pm2\.613\.3±\\pm3\.336\.4±\\pm0\.4RER36\.6±\\pm2\.833\.6±\\pm2\.264\.7±\\pm2\.836\.3±\\pm1\.431\.2±\\pm2\.162\.3±\\pm2\.137\.5±\\pm2\.132\.2±\\pm2\.765\.1±\\pm0\.3Replay36\.5±\\pm1\.934\.9±\\pm2\.565\.7±\\pm2\.935\.9±\\pm1\.232\.7±\\pm0\.563\.3±\\pm1\.937\.9±\\pm2\.133\.0±\\pm3\.166\.2±\\pm0\.5iCaRL36\.8±\\pm2\.533\.4±\\pm1\.765\.2±\\pm2\.040\.2±\\pm2\.728\.8±\\pm2\.864\.0±\\pm2\.640\.9±\\pm3\.829\.9±\\pm3\.966\.6±\\pm0\.4DER\+\+37\.8±\\pm3\.129\.5±\\pm3\.562\.0±\\pm3\.439\.8±\\pm1\.225\.3±\\pm2\.560\.2±\\pm0\.640\.3±\\pm2\.028\.0±\\pm2\.364\.0±\\pm2\.2RanPAC24\.6±\\pm2\.213\.4±\\pm5\.435\.6±\\pm6\.824\.0±\\pm0\.29\.3±\\pm4\.631\.5±\\pm4\.124\.8±\\pm0\.610\.5±\\pm2\.033\.7±\\pm1\.2FeCAM39\.8±\\pm3\.911\.1±\\pm3\.349\.4±\\pm3\.440\.4±\\pm1\.49\.9±\\pm3\.148\.8±\\pm1\.339\.7±\\pm3\.09\.9±\\pm3\.648\.1±\\pm0\.8ECNumberEWC15\.7±\\pm1\.090\.4±\\pm3\.893\.2±\\pm3\.415\.5±\\pm0\.990\.8±\\pm2\.193\.4±\\pm2\.616\.4±\\pm1\.985\.3±\\pm5\.689\.5±\\pm6\.6FeTrIL20\.5±\\pm6\.585\.0±\\pm8\.393\.1±\\pm1\.217\.0±\\pm5\.189\.0±\\pm6\.693\.3±\\pm0\.622\.8±\\pm9\.281\.3±\\pm12\.491\.0±\\pm2\.2EASE44\.4±\\pm0\.96\.5±\\pm1\.149\.9±\\pm0\.748\.6±\\pm0\.14\.2±\\pm1\.952\.0±\\pm1\.746\.3±\\pm1\.34\.4±\\pm2\.249\.7±\\pm2\.5RER34\.5±\\pm0\.371\.8±\\pm3\.494\.0±\\pm2\.834\.8±\\pm1\.271\.2±\\pm1\.294\.3±\\pm2\.535\.1±\\pm1\.069\.5±\\pm3\.391\.2±\\pm5\.4Replay34\.7±\\pm0\.471\.7±\\pm3\.394\.1±\\pm2\.734\.4±\\pm0\.971\.6±\\pm1\.494\.3±\\pm2\.334\.5±\\pm1\.070\.1±\\pm4\.191\.3±\\pm5\.2iCaRL37\.4±\\pm3\.269\.6±\\pm2\.895\.6±\\pm2\.636\.8±\\pm3\.070\.5±\\pm2\.195\.7±\\pm1\.934\.7±\\pm3\.372\.3±\\pm2\.193\.4±\\pm4\.1DER\+\+41\.7±\\pm3\.462\.2±\\pm6\.693\.0±\\pm2\.641\.7±\\pm3\.661\.0±\\pm6\.592\.7±\\pm2\.840\.2±\\pm4\.063\.5±\\pm9\.190\.4±\\pm5\.5RanPAC22\.4±\\pm1\.16\.3±\\pm2\.927\.8±\\pm2\.619\.7±\\pm0\.24\.8±\\pm2\.523\.8±\\pm1\.922\.5±\\pm3\.17\.8±\\pm2\.929\.1±\\pm3\.4FeCAM39\.1±\\pm0\.66\.5±\\pm1\.744\.6±\\pm1\.845\.0±\\pm0\.24\.2±\\pm1\.448\.6±\\pm1\.339\.8±\\pm1\.46\.1±\\pm1\.645\.0±\\pm1\.1ProtBertEnzymeReactionEWC13\.2±\\pm0\.876\.9±\\pm2\.679\.1±\\pm2\.913\.9±\\pm1\.870\.6±\\pm1\.474\.4±\\pm0\.915\.3±\\pm2\.076\.0±\\pm1\.980\.5±\\pm1\.1FeTrIL21\.3±\\pm5\.169\.0±\\pm6\.380\.5±\\pm2\.022\.6±\\pm4\.864\.2±\\pm5\.577\.7±\\pm0\.221\.5±\\pm5\.368\.9±\\pm6\.180\.6±\\pm0\.7EASE37\.1±\\pm1\.814\.3±\\pm3\.349\.2±\\pm1\.240\.3±\\pm0\.213\.0±\\pm2\.551\.3±\\pm2\.436\.1±\\pm4\.114\.8±\\pm2\.448\.3±\\pm1\.8RER43\.5±\\pm2\.438\.0±\\pm2\.476\.0±\\pm2\.846\.6±\\pm1\.430\.2±\\pm1\.972\.3±\\pm1\.647\.7±\\pm3\.435\.1±\\pm3\.777\.8±\\pm0\.9Replay43\.8±\\pm3\.037\.9±\\pm2\.176\.3±\\pm2\.745\.9±\\pm1\.431\.2±\\pm1\.272\.5±\\pm0\.847\.9±\\pm3\.534\.3±\\pm3\.177\.3±\\pm1\.2iCaRL48\.5±\\pm2\.631\.7±\\pm1\.675\.6±\\pm1\.350\.2±\\pm1\.525\.7±\\pm1\.271\.9±\\pm0\.751\.3±\\pm3\.229\.5±\\pm2\.776\.6±\\pm0\.9DER\+\+55\.1±\\pm4\.019\.4±\\pm3\.171\.5±\\pm4\.057\.4±\\pm5\.616\.1±\\pm3\.769\.9±\\pm3\.257\.5±\\pm3\.218\.8±\\pm1\.973\.2±\\pm2\.6RanPAC65\.2±\\pm3\.212\.4±\\pm2\.175\.8±\\pm3\.367\.4±\\pm0\.26\.3±\\pm1\.572\.6±\\pm1\.365\.6±\\pm1\.610\.9±\\pm1\.574\.9±\\pm0\.4FeCAM65\.7±\\pm4\.710\.0±\\pm3\.974\.3±\\pm1\.767\.7±\\pm0\.26\.9±\\pm0\.973\.6±\\pm0\.465\.9±\\pm3\.111\.0±\\pm3\.575\.3±\\pm0\.6ECNumberEWC36\.9±\\pm4\.173\.5±\\pm4\.899\.9±\\pm0\.043\.1±\\pm6\.466\.2±\\pm7\.599\.8±\\pm0\.031\.0±\\pm5\.880\.4±\\pm6\.899\.9±\\pm0\.0FeTrIL23\.2±\\pm7\.589\.5±\\pm8\.899\.9±\\pm0\.019\.1±\\pm4\.694\.2±\\pm5\.499\.8±\\pm0\.025\.4±\\pm9\.886\.8±\\pm11\.499\.8±\\pm0\.1EASE56\.5±\\pm0\.410\.0±\\pm1\.564\.9±\\pm1\.652\.7±\\pm0\.77\.2±\\pm4\.858\.7±\\pm4\.556\.9±\\pm1\.58\.1±\\pm3\.663\.3±\\pm3\.5RER52\.0±\\pm3\.655\.8±\\pm4\.399\.9±\\pm0\.156\.0±\\pm3\.951\.2±\\pm4\.599\.9±\\pm0\.050\.6±\\pm2\.457\.4±\\pm2\.999\.8±\\pm0\.1Replay52\.8±\\pm2\.855\.0±\\pm3\.399\.9±\\pm0\.056\.8±\\pm3\.750\.2±\\pm4\.399\.9±\\pm0\.050\.6±\\pm2\.557\.5±\\pm2\.899\.8±\\pm0\.1iCaRL58\.3±\\pm5\.548\.5±\\pm6\.599\.9±\\pm0\.057\.5±\\pm3\.449\.4±\\pm4\.099\.9±\\pm0\.055\.2±\\pm2\.352\.0±\\pm2\.699\.9±\\pm0\.0DER\+\+68\.1±\\pm5\.937\.2±\\pm6\.999\.9±\\pm0\.069\.3±\\pm4\.435\.7±\\pm5\.199\.9±\\pm0\.067\.3±\\pm3\.538\.0±\\pm4\.199\.9±\\pm0\.0RanPAC76\.6±\\pm0\.811\.5±\\pm1\.986\.4±\\pm2\.478\.0±\\pm0\.18\.4±\\pm2\.085\.2±\\pm1\.676\.6±\\pm1\.413\.9±\\pm3\.588\.6±\\pm2\.9FeCAM30\.1±\\pm0\.45\.1±\\pm1\.834\.4±\\pm1\.734\.0±\\pm0\.13\.1±\\pm0\.436\.7±\\pm0\.531\.2±\\pm1\.64\.1±\\pm1\.434\.8±\\pm1\.0SaProtEnzymeReactionEWC17\.9±\\pm2\.277\.9±\\pm2\.884\.7±\\pm0\.317\.4±\\pm1\.273\.5±\\pm1\.280\.4±\\pm0\.217\.5±\\pm1\.878\.0±\\pm2\.584\.4±\\pm1\.0FeTrIL22\.7±\\pm4\.671\.0±\\pm5\.783\.6±\\pm0\.525\.6±\\pm5\.965\.2±\\pm6\.581\.5±\\pm0\.624\.2±\\pm5\.570\.0±\\pm5\.784\.3±\\pm0\.7EASE46\.1±\\pm2\.215\.4±\\pm4\.859\.2±\\pm2\.851\.8±\\pm1\.813\.1±\\pm2\.662\.8±\\pm2\.247\.4±\\pm3\.114\.1±\\pm4\.059\.2±\\pm0\.2RER52\.2±\\pm3\.734\.3±\\pm5\.281\.6±\\pm0\.754\.8±\\pm1\.928\.6±\\pm2\.279\.3±\\pm0\.153\.6±\\pm1\.832\.5±\\pm2\.281\.5±\\pm1\.1Replay51\.9±\\pm3\.234\.7±\\pm4\.581\.7±\\pm0\.654\.6±\\pm2\.228\.6±\\pm2\.479\.1±\\pm0\.153\.4±\\pm1\.433\.4±\\pm1\.782\.0±\\pm0\.9iCaRL54\.2±\\pm4\.329\.0±\\pm4\.879\.0±\\pm1\.460\.6±\\pm0\.620\.9±\\pm0\.478\.5±\\pm0\.861\.0±\\pm1\.822\.9±\\pm3\.480\.6±\\pm1\.4DER\+\+60\.7±\\pm2\.916\.5±\\pm1\.274\.6±\\pm3\.164\.3±\\pm5\.014\.1±\\pm2\.975\.7±\\pm2\.962\.7±\\pm3\.415\.2±\\pm0\.675\.7±\\pm3\.5RanPAC69\.7±\\pm3\.712\.6±\\pm3\.580\.2±\\pm2\.475\.4±\\pm0\.33\.6±\\pm0\.678\.5±\\pm0\.471\.5±\\pm1\.19\.3±\\pm0\.979\.5±\\pm0\.5FeCAM65\.9±\\pm4\.012\.8±\\pm6\.076\.9±\\pm1\.968\.3±\\pm0\.87\.7±\\pm1\.374\.8±\\pm0\.666\.6±\\pm2\.311\.0±\\pm3\.376\.1±\\pm0\.6ECNumberEWC34\.3±\\pm4\.776\.5±\\pm5\.499\.9±\\pm0\.038\.3±\\pm7\.771\.7±\\pm8\.899\.8±\\pm0\.133\.5±\\pm5\.277\.5±\\pm6\.099\.9±\\pm0\.0FeTrIL23\.1±\\pm7\.689\.6±\\pm8\.999\.9±\\pm0\.019\.3±\\pm4\.893\.9±\\pm5\.599\.8±\\pm0\.125\.8±\\pm9\.986\.4±\\pm11\.699\.9±\\pm0\.0EASE59\.0±\\pm1\.18\.0±\\pm0\.565\.9±\\pm1\.559\.0±\\pm0\.35\.6±\\pm3\.663\.4±\\pm3\.260\.7±\\pm0\.96\.7±\\pm2\.366\.2±\\pm2\.0RER51\.1±\\pm2\.356\.9±\\pm2\.699\.9±\\pm0\.056\.6±\\pm3\.050\.3±\\pm3\.499\.8±\\pm0\.146\.6±\\pm3\.962\.0±\\pm4\.599\.8±\\pm0\.1Replay50\.1±\\pm1\.558\.2±\\pm1\.799\.9±\\pm0\.054\.7±\\pm4\.452\.6±\\pm5\.199\.8±\\pm0\.147\.7±\\pm2\.460\.9±\\pm2\.899\.9±\\pm0\.0iCaRL56\.7±\\pm3\.550\.5±\\pm4\.099\.9±\\pm0\.061\.6±\\pm1\.944\.6±\\pm2\.199\.9±\\pm0\.057\.4±\\pm5\.049\.6±\\pm5\.899\.9±\\pm0\.0DER\+\+67\.2±\\pm5\.638\.1±\\pm6\.699\.9±\\pm0\.069\.4±\\pm4\.235\.6±\\pm4\.999\.9±\\pm0\.065\.8±\\pm3\.039\.8±\\pm3\.599\.9±\\pm0\.0RanPAC83\.8±\\pm0\.78\.2±\\pm1\.190\.8±\\pm1\.685\.1±\\pm0\.15\.7±\\pm1\.289\.9±\\pm1\.083\.8±\\pm0\.79\.6±\\pm1\.992\.0±\\pm1\.8FeCAM47\.5±\\pm1\.26\.9±\\pm2\.653\.4±\\pm2\.852\.0±\\pm0\.14\.0±\\pm1\.555\.4±\\pm1\.449\.5±\\pm1\.55\.4±\\pm1\.654\.1±\\pm1\.6

### B\.2Cross\-family confusion patterns\.

Across both datasets, the dominant cross\-family errors consistently involve transferases and hydrolases, indicating that these families remain difficult to separate under a single shared covariance\. However, the magnitude of these confusions varies substantially across encoders and datasets\. This variation further supports the conclusion that the effectiveness of a shared covariance assumption depends strongly on the geometry induced by the underlying protein representation\.

Table 11:Cross\-family confusion breakdowns under a single shared covariance for all five protein encoders on both datasets\. For each encoder and dataset, we report the six most frequent cross\-family misclassifications together with one representative EC\-level confusion\.Enzyme ReactionEC NumberTruePred\.\# mis\.ExampleTruePred\.\# mis\.ExampleESM2\-150MESM2\-150M2 \(transf\.\)3 \(hydro\.\)962\.1\.1\.\-→\\to3\.1\.21\.43 \(hydro\.\)2 \(transf\.\)4613\.1\.4\.35→\\to2\.4\.1\.2293 \(hydro\.\)2 \(transf\.\)483\.4\.19\.12→\\to2\.7\.10\.\-2 \(transf\.\)3 \(hydro\.\)2702\.7\.2\.3→\\to3\.5\.1\.762 \(transf\.\)4 \(lyases\)342\.7\.7\.\-→\\to4\.2\.1\.\-2 \(transf\.\)1 \(oxidored\.\)1702\.7\.1\.148→\\to1\.1\.1\.1933 \(hydro\.\)5 \(isomer\.\)163\.6\.4\.\-→\\to5\.6\.1\.\-3 \(hydro\.\)1 \(oxidored\.\)1593\.1\.21\.4→\\to1\.13\.11\.582 \(transf\.\)1 \(oxidored\.\)152\.7\.11\.1→\\to1\.13\.12\.\-2 \(transf\.\)4 \(lyases\)1172\.4\.2\.9→\\to4\.2\.1\.914 \(lyases\)2 \(transf\.\)144\.1\.1\.\-→\\to2\.7\.4\.62 \(transf\.\)6 \(ligases\)1022\.7\.7\.101→\\to6\.5\.1\.2ESM2\-650MESM2\-650M2 \(transf\.\)3 \(hydro\.\)592\.7\.7\.\-→\\to3\.1\.13\.\-3 \(hydro\.\)2 \(transf\.\)4253\.1\.3\.16→\\to2\.3\.1\.843 \(hydro\.\)2 \(transf\.\)533\.4\.22\.28→\\to2\.5\.1\.\-2 \(transf\.\)3 \(hydro\.\)2702\.3\.1\.48→\\to3\.1\.21\.42 \(transf\.\)4 \(lyases\)312\.7\.7\.48→\\to4\.2\.1\.\-2 \(transf\.\)1 \(oxidored\.\)1592\.7\.1\.148→\\to1\.1\.1\.1933 \(hydro\.\)4 \(lyases\)213\.6\.4\.\-→\\to4\.2\.1\.\-3 \(hydro\.\)1 \(oxidored\.\)1243\.1\.4\.17→\\to1\.14\.11\.302 \(transf\.\)6 \(ligases\)112\.4\.2\.\-→\\to6\.3\.5\.22 \(transf\.\)6 \(ligases\)1242\.7\.2\.3→\\to6\.4\.1\.24 \(lyases\)5 \(isomer\.\)94\.2\.1\.\-→\\to5\.3\.3\.82 \(transf\.\)4 \(lyases\)922\.3\.2\.27→\\to4\.99\.1\.10GearNet\-EdgeGearNet\-Edge2 \(transf\.\)3 \(hydro\.\)1242\.4\.2\.\-→\\to3\.1\.13\.\-3 \(hydro\.\)2 \(transf\.\)3693\.2\.1\.15→\\to2\.7\.1\.1612 \(transf\.\)1 \(oxidored\.\)782\.5\.1\.\-→\\to1\.2\.1\.112 \(transf\.\)3 \(hydro\.\)2772\.3\.2\.27→\\to3\.6\.1\.623 \(hydro\.\)2 \(transf\.\)713\.2\.1\.4→\\to2\.4\.1\.\-2 \(transf\.\)1 \(oxidored\.\)1122\.8\.4\.1→\\to1\.1\.99\.373 \(hydro\.\)1 \(oxidored\.\)553\.2\.1\.4→\\to1\.8\.1\.\-3 \(hydro\.\)1 \(oxidored\.\)1053\.5\.1\.28→\\to1\.6\.99\.12 \(transf\.\)6 \(ligases\)392\.7\.1\.\-→\\to6\.1\.1\.152 \(transf\.\)4 \(lyases\)732\.3\.1\.15→\\to4\.1\.1\.312 \(transf\.\)4 \(lyases\)372\.7\.11\.1→\\to4\.2\.99\.\-1 \(oxidored\.\)2 \(transf\.\)711\.15\.1\.1→\\to2\.4\.2\.31ProtBertProtBert2 \(transf\.\)3 \(hydro\.\)1062\.7\.11\.1→\\to3\.4\.19\.123 \(hydro\.\)2 \(transf\.\)1243\.1\.4\.35→\\to2\.7\.11\.13 \(hydro\.\)2 \(transf\.\)593\.6\.4\.\-→\\to2\.7\.7\.\-2 \(transf\.\)3 \(hydro\.\)1232\.7\.11\.1→\\to3\.4\.19\.123 \(hydro\.\)4 \(lyases\)213\.5\.1\.2→\\to4\.2\.1\.\-1 \(oxidored\.\)3 \(hydro\.\)371\.15\.1\.1→\\to3\.1\.1\.32 \(transf\.\)4 \(lyases\)122\.7\.1\.\-→\\to4\.2\.1\.\-3 \(hydro\.\)1 \(oxidored\.\)343\.1\.1\.4→\\to1\.4\.3\.24 \(lyases\)2 \(transf\.\)124\.1\.1\.\-→\\to2\.7\.7\.487 \(transl\.\)1 \(oxidored\.\)317\.1\.1\.9→\\to1\.12\.98\.32 \(transf\.\)1 \(oxidored\.\)112\.7\.1\.\-→\\to1\.16\.3\.14 \(lyases\)3 \(hydro\.\)244\.2\.2\.10→\\to3\.2\.1\.133SaProtSaProt2 \(transf\.\)3 \(hydro\.\)892\.7\.7\.\-→\\to3\.1\.13\.\-2 \(transf\.\)3 \(hydro\.\)842\.3\.2\.27→\\to3\.2\.2\.63 \(hydro\.\)2 \(transf\.\)643\.1\.3\.\-→\\to2\.8\.1\.13 \(hydro\.\)2 \(transf\.\)773\.5\.4\.26→\\to2\.1\.1\.852 \(transf\.\)6 \(ligases\)182\.4\.2\.\-→\\to6\.3\.5\.23 \(hydro\.\)1 \(oxidored\.\)393\.5\.4\.19→\\to1\.1\.1\.232 \(transf\.\)4 \(lyases\)152\.7\.11\.1→\\to4\.6\.1\.27 \(transl\.\)1 \(oxidored\.\)337\.1\.1\.9→\\to1\.7\.2\.52 \(transf\.\)1 \(oxidored\.\)142\.7\.13\.\-→\\to1\.6\.5\.\-2 \(transf\.\)1 \(oxidored\.\)282\.7\.2\.3→\\to1\.2\.1\.284 \(lyases\)2 \(transf\.\)124\.1\.1\.\-→\\to2\.7\.7\.62 \(transf\.\)4 \(lyases\)222\.5\.1\.18→\\to4\.4\.1\.20

Figure 4:Per\-encoder continual\-learning results on Enzyme Reaction \(348 classes, 7 reaction type groups\)\.Left:Average incremental accuracy across phases\.Middle:Forgetting across phases\.Right:Final accuracy broken down by reaction type\.![Refer to caption](https://arxiv.org/html/2610.18879v1/figure/diagram-protocil-NIPS_curve-appendix-ecnumber-acc-fgt.jpg)Figure 5:Per\-encoder continual\-learning results on EC Number \(4,788 classes, 7 EC top\-level groups\)\.Left:Average incremental accuracy across phases\.Middle:Forgetting across phases\.Right:Final accuracy broken down by EC class\.![Refer to caption](https://arxiv.org/html/2610.18879v1/figure/diagram-protocil-NIPS_heatmap-appendix-rmatrix.jpg)Figure 6:Phase\-wise accuracy matrices across encoder–dataset configurations\. Within each configuration, matrices from left to right correspond to FeTrIL, RanPAC, and FeCAM\. Rows indicate the training phase and columns the evaluated phase\.