REDAR：用于类别增量学习的强化驱动自适应正则化

# REDAR: Reinforcement-Driven Adaptive Regularisation for Class Incremental Learning
Source: [https://arxiv.org/html/2610.31991](https://arxiv.org/html/2610.31991)
###### Abstract

Regularisation\-based class\-incremental learning \(CIL\) methods control the stability\-plasticity trade\-off through fixed coefficients tuned offline, independently of the continual learning objective and blind to phase\-specific variations in data volume, task difficulty, and class similarity\. We argue this is a fundamental mismatch and proposeREDAR:Reinforcement\-DrivenAdaptiveRegularisation for Class Incremental Learning, which treats regularisation strength selection as a sequential decision problem\. At each incremental phase, a lightweight reinforcement learning \(RL\) agent observes a compact state of current performance metrics and incoming data statistics, and outputs the distillation coefficients for that phase\. To respect CIL’s constraint on historical data access, the policy is trained entirely on pseudo\-CIL tasks constructed from the initial phase, then deployed on the incremental sequence without further adaptation\. Experiments on CIFAR\-100 and Tiny\-ImageNet demonstrate thatREDARreduces forgetting and improves forward transfer over fixed and heuristic baselines, and that policies trained on short pseudo\-task horizons transfer effectively to longer incremental runs\.

## 1Introduction

Class\-incremental learning \(CIL\) requires a model to absorb new categories sequentially while retaining performance on all previously seen classes, without access to the full historical dataset\. This capability is essential for deployed perception systems and embodied agents that operate under non\-stationary data streams\. The central challenge is catastrophic forgetting: naive fine\-tuning overwrites earlier representations as the network adapts to new data, exposing the fundamental stability–plasticity dilemma\.

Prior work addresses this from three angles: rehearsal, which interleaves stored exemplars with current training data\[[23](https://arxiv.org/html/2610.31991#bib.bib15),[17](https://arxiv.org/html/2610.31991#bib.bib29)\]; architecture\-based methods, which grow or partition capacity to reduce interference\[[32](https://arxiv.org/html/2610.31991#bib.bib37),[19](https://arxiv.org/html/2610.31991#bib.bib33)\]; regularisation\-based methods, which constrain updates via parameter penalties\[[12](https://arxiv.org/html/2610.31991#bib.bib26),[34](https://arxiv.org/html/2610.31991#bib.bib6)\]or knowledge distillation\[[15](https://arxiv.org/html/2610.31991#bib.bib21),[9](https://arxiv.org/html/2610.31991#bib.bib19),[5](https://arxiv.org/html/2610.31991#bib.bib27)\]\. Despite this progress, one design choice has received little attention: how strongly should a model be regularised at each incremental step? Existing methods fix the regularisation coefficient across all phases, selected once via offline hyperparameter search\[[23](https://arxiv.org/html/2610.31991#bib.bib15),[9](https://arxiv.org/html/2610.31991#bib.bib19),[5](https://arxiv.org/html/2610.31991#bib.bib27),[6](https://arxiv.org/html/2610.31991#bib.bib34),[25](https://arxiv.org/html/2610.31991#bib.bib3)\]\. This is a poor fit for realistic CIL scenarios, where successive phases differ in data volume, class imbalance, task difficulty, and similarity to prior classes, so the stability–plasticity trade\-off shifts in ways a single global coefficient cannot track\.

We treat regularisation strength as a control variable and formulate its selection as a sequential decision problem\. To this end, we proposeREDAR\(Reinforcement\-DrivenAdaptiveRegularisation for Class Incremental Learning\), which casts per\-phase coefficient selection as a policy learned via Reinforcement Learning \(RL\)\. A lightweight RL agent observes a compact state consisting of current performance and basic data statistics, and outputs a per\-phase distillation coefficient\. To respect CIL’s constraint on historical data access, we train the policy offline on pseudo\-CIL tasks constructed from the initial phase data alone, then deploy it on the true incremental sequence without retraining, using distributed experience collection\.

We make three empirical claims\. First, a state\-dependent learned policy reduces forgetting and improves forward transfer relative to fixed and heuristic baselines on CIFAR\-100 and Tiny\-ImageNet \(Sec\.[6](https://arxiv.org/html/2610.31991#S6)\)\. Second, policies trained on short pseudo\-task horizons transfer to longer incremental runs \(Sec\.[6\.1](https://arxiv.org/html/2610.31991#S6.SS1)\)\. Third, the gains are attributable to learned state\-conditioned selection: the policy outperforms both the best fixed coefficient and random per\-phase draws from the same action space \(Sec\.[6\.2](https://arxiv.org/html/2610.31991#S6.SS2)\)\.

## 2Related Work

### 2\.1Class\-Incremental Learning

Existing CIL methods fall into three broad categories\.Replay\-based methodsmaintain a small exemplar buffer or synthesise past data, interleaving stored samples with current training\[[23](https://arxiv.org/html/2610.31991#bib.bib15),[4](https://arxiv.org/html/2610.31991#bib.bib24),[18](https://arxiv.org/html/2610.31991#bib.bib13)\]\. When real data cannot be reused, synthetic images or features are replayed instead\[[26](https://arxiv.org/html/2610.31991#bib.bib7),[30](https://arxiv.org/html/2610.31991#bib.bib16)\], while exemplar\-free variants replay class prototypes and compensate for their drift\[[22](https://arxiv.org/html/2610.31991#bib.bib2)\]\.Architecture\-based methodsgrow or route capacity to reduce inter\-task interference\[[19](https://arxiv.org/html/2610.31991#bib.bib33),[6](https://arxiv.org/html/2610.31991#bib.bib34)\], but become impractical as the number of tasks grows\.Regularisation\-based methodsadd penalty terms to prevent drift\. Weight\-based approaches penalise changes to important parameters\[[12](https://arxiv.org/html/2610.31991#bib.bib26),[5](https://arxiv.org/html/2610.31991#bib.bib27),[35](https://arxiv.org/html/2610.31991#bib.bib36)\], while activation\-based approaches distill knowledge from the previous model at the logit\[[15](https://arxiv.org/html/2610.31991#bib.bib21)\], feature\[[9](https://arxiv.org/html/2610.31991#bib.bib19)\], or embedding\-geometry level\[[24](https://arxiv.org/html/2610.31991#bib.bib1)\]\. Flashback learning\[[20](https://arxiv.org/html/2610.31991#bib.bib38)\]makes regularisation bidirectional, pulling toward both old and new knowledge\. Most of these methods use fixed regularisation strengths\[[23](https://arxiv.org/html/2610.31991#bib.bib15),[1](https://arxiv.org/html/2610.31991#bib.bib9)\]or hand\-crafted schedules\[[9](https://arxiv.org/html/2610.31991#bib.bib19),[5](https://arxiv.org/html/2610.31991#bib.bib27)\]\. Recent work adapts regularisation via Bayesian optimisation\[[33](https://arxiv.org/html/2610.31991#bib.bib30)\]or prompt\-based regularisers on pre\-trained backbones\[[10](https://arxiv.org/html/2610.31991#bib.bib31)\], but treats each phase in isolation without a long\-horizon objective\.REDARaddresses this gap by casting coefficient selection as a sequential decision problem, accounting for the downstream effects of each phase’s regularisation strength\.

### 2\.2Reinforcement Learning for CIL

RL has achieved strong results in robot control\[[11](https://arxiv.org/html/2610.31991#bib.bib14)\], game playing\[[27](https://arxiv.org/html/2610.31991#bib.bib22)\], and neural architecture search\[[38](https://arxiv.org/html/2610.31991#bib.bib23)\], and has been applied to vision problems such as active learning\[[2](https://arxiv.org/html/2610.31991#bib.bib4)\]and object detection\[[14](https://arxiv.org/html/2610.31991#bib.bib17)\]\. In continual learning, it has been used to optimise specific design choices: RCL\[[31](https://arxiv.org/html/2610.31991#bib.bib28)\]learns to expand network capacity for new tasks, and RMM\[[17](https://arxiv.org/html/2610.31991#bib.bib29)\]learns a memory allocation policy across phases, extended to select which samples\[[37](https://arxiv.org/html/2610.31991#bib.bib8)\]or tasks\[[13](https://arxiv.org/html/2610.31991#bib.bib18)\]to replay\. Most closely related,\[[16](https://arxiv.org/html/2610.31991#bib.bib25)\]adapts regularisation strength as an online bandit problem, but must be re\-optimised per phase and does not condition on a rich phase description;\[[33](https://arxiv.org/html/2610.31991#bib.bib30)\]similarly tunes regularisation independently at each step via Bayesian optimisation\.

In contrast,REDARformulates regularisation selection as a sequential decision problem with an explicit state of performance metrics and data properties\. Trained offline on pseudo\-CIL tasks and deployed without retraining, it optimises a long\-horizon CIL objective and generalises across incremental horizons\.

## 3Preliminaries

### 3\.1Class Incremental Learning

Class incremental learning \(CIL\) spans\(N\+1\)\(N\+1\)phases: an initial phase followed byNNincremental phases, with the class count growing at each stage\.

Initial Phase\.The modelθ0\\theta\_\{0\}is randomly initialised and trained onD0D\_\{0\}, comprising all samples from the first 50% of classes \(train\-from\-half\)\. A fixed\-size memory bufferℳ\\mathcal\{M\}\(budgetKm​e​mK\_\{mem\}\) is populated via herding, selecting exemplars per class fromD0D\_\{0\}\.

Incremental Phases\.In phaseii,θi\\theta\_\{i\}is initialised fromθi−1\\theta\_\{i\-1\}and trained on new classesDiD\_\{i\}combined with memoryℳ\\mathcal\{M\}, then fine\-tuned on a class\-balanced subset of all observed classes to correct recency bias from the new\-memory data imbalance\. After training,ℳ\\mathcal\{M\}is updated via herding to retainKm​e​m/\|C0:i\|K\_\{mem\}/\|C\_\{0:i\}\|exemplars per class, whereC0:iC\_\{0:i\}is the set of all classes seen up to phaseii\.

## 4Methodology

![Refer to caption](https://arxiv.org/html/2610.31991v1/imgs/HPO-Architecture.png)Figure 1:Overview ofREDAR\.\(a\)Reinforcement\-driven regularisation is formulated as a finite\-horizon Markov Decision Process \(MDP\),ℳ=\(𝒮,𝒜,𝒯,ℛ,γ\)\\mathcal\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{T\},\\mathcal\{R\},\\gamma\), where the statesi=\(Ai−1,Fi−1,C​Ri,I​Si\)s\_\{i\}=\(A\_\{i\-1\},F\_\{i\-1\},CR\_\{i\},IS\_\{i\}\)summarises the current learning dynamics, the actionai=\(αi,βi\)a\_\{i\}=\(\\alpha\_\{i\},\\beta\_\{i\}\)specifies the phase\-specific regularisation coefficients, and the rewardrir\_\{i\}is the validation accuracy after CIL training\.\(b\)The CIL model jointly optimises cross\-entropy, feature distillation, logit distillation, and margin ranking losses while learning from replay memory and incoming samples\.\(c\)The policy is trained offline on pseudo\-CIL tasks using a distributed actor–learner architecture with prioritised experience replay before being deployed on the target incremental sequence\.We proposeREDAR\(Figure[1](https://arxiv.org/html/2610.31991#S4.F1)\), an RL framework for adaptive regularisation in CIL with two components: a CIL model parameterised by phase\-specific distillation coefficients \(αi,βi\\alpha\_\{i\},\\beta\_\{i\}\), and an RL agent that learns to set them\. We describe the CIL model in Section[4\.1](https://arxiv.org/html/2610.31991#S4.SS1), formulate coefficient selection as an MDP in Section[4\.2](https://arxiv.org/html/2610.31991#S4.SS2), and detail agent training on pseudo\-tasks in Section[4\.3](https://arxiv.org/html/2610.31991#S4.SS3)\.

### 4\.1CIL Model

The CIL modelfθ,w:𝒳→ℝKf\_\{\\theta,w\}:\\mathcal\{X\}\\rightarrow\\mathbb\{R\}^\{K\}is a deep convolutional network composed of a feature extractorhθ:𝒳→ℝdh\_\{\\theta\}:\\mathcal\{X\}\\rightarrow\\mathbb\{R\}^\{d\}and a cosine\-normalised classifiergw:ℝd→ℝKg\_\{w\}:\\mathbb\{R\}^\{d\}\\rightarrow\\mathbb\{R\}^\{K\}\[[9](https://arxiv.org/html/2610.31991#bib.bib19)\], given byfθ,w​\(x\)=cos⁡\(gw,hθ​\(x\)\)f\_\{\\theta,w\}\(x\)=\\cos\(g\_\{w\},h\_\{\\theta\}\(x\)\)\. Training combines four losses\. First, cross\-entropy over all observed classes provides the classification objective:

Lc​e\(x\)=−∑k=1KyklogPwk\(x\),\\displaystyle L\_\{ce\}\(x\)=\-\\sum\_\{k=1\}^\{K\}y\_\{k\}\\log P\_\{w\_\{k\}\}\(x\),\(1\)
wherePw​\(x\)P\_\{w\}\(x\)denotes the softmax probabilities from the cosine\-normalised classifier\. Then, to preserve prior knowledge, we apply distillation in both feature and logit space\. The feature distillation loss penalises drift in representation orientation:

Lf​d​i​s​t​\(x\)=1−cos⁡\(hθi​\(x\),hθi−1​\(x\)\),\\displaystyle L\_\{fdist\}\(x\)=1\-\\cos\\\!\\left\(h\_\{\\theta\}^\{i\}\(x\),\\,h\_\{\\theta\}^\{i\-1\}\(x\)\\right\),\(2\)and the logit distillation loss penalises divergence in output distributions over previously seen classes \(with cosine normalisation omitted to preserve logit scale\):

ℒl​d​i​s​t\(x\)=DK​L\(p^i−1\(x\)∥p^i\(x\)\),\\displaystyle\\mathcal\{L\}\_\{ldist\}\(x\)=D\_\{KL\}\\big\(\\hat\{p\}^\{\\,i\-1\}\(x\)\\,\\\|\\,\\hat\{p\}^\{\\,i\}\(x\)\\big\),\(3\)wherep^wi\\hat\{p\}\_\{w\}^\{\\,i\}andp^wi−1\\hat\{p\}\_\{w\}^\{\\,i\-1\}denote the softmax distributions of the current and previous models restricted to theKi−1K\_\{i\-1\}previously seen classes\.

Finally, a margin ranking loss separates old\-class exemplar embeddings from new\-class embeddings by a marginmm:

ℒm​r​\(x\)=∑h=1Hmax⁡\(m−zy​\(x\)\+z\(h\)​\(x\),0\),\\displaystyle\\mathcal\{L\}\_\{mr\}\(x\)=\\sum\_\{h=1\}^\{H\}\\max\\big\(m\-z^\{y\}\(x\)\+z^\{\(h\)\}\(x\),\\,0\\big\),\(4\)wherezy​\(x\)z^\{y\}\(x\)is the ground\-truth class score,z\(h\)​\(x\)z^\{\(h\)\}\(x\)is thehh\-th hardest new\-class score, andHHis the number of hard negatives\.\. The overall training objective at phaseiiis:

Loverall=1\|ℬ\|​∑x∈ℬ\(Lc​e​\(x\)\+αi​Lf​d​i​s​t​\(x\)\+βi​Ll​d​i​s​t​\(x\)\)\\displaystyle L\_\{\\text\{overall\}\}=\\frac\{1\}\{\|\\mathcal\{B\}\|\}\\sum\_\{x\\in\\mathcal\{B\}\}\\Bigl\(L\_\{ce\}\(x\)\+\\alpha\_\{i\}\\,L\_\{fdist\}\(x\)\+\\beta\_\{i\}\\,L\_\{ldist\}\(x\)\\Bigr\)\(5\)\+1\|ℬo​l​d\|∑x∈ℬo​l​dLm​r\(x\),\\displaystyle\+\\frac\{1\}\{\|\\mathcal\{B\}\_\{old\}\|\}\\sum\_\{x\\in\\mathcal\{B\}\_\{old\}\}L\_\{mr\}\(x\),whereℬ\\mathcal\{B\}denotes a training mini\-batch drawn from the current phase dataset, andℬo​l​d⊆ℬ\\mathcal\{B\}\_\{old\}\\subseteq\\mathcal\{B\}denotes the subset of replay samples belonging to previously learned classes, andαi\\alpha\_\{i\}andβi\\beta\_\{i\}are phase\-specific coefficients weighting representation and prediction stability, respectively\. SinceLc​eL\_\{ce\}andLm​rL\_\{mr\}are always active, the model continually acquires new classes; adjusting\(αi,βi\)\(\\alpha\_\{i\},\\beta\_\{i\}\)controls how strongly it must simultaneously preserve past knowledge\. These two coefficients therefore govern the stability\-plasticity trade\-off at each phase and constitute the action space of our RL agent, described next in Section[4\.2](https://arxiv.org/html/2610.31991#S4.SS2)\.

### 4\.2Reinforcement\-Driven Regularisation

We formulate regularisation selection as a finite\-horizon Markov Decision Process \(MDP\),ℳ=\(𝒮,𝒜,𝒯,ℛ,γ\)\\mathcal\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{T\},\\mathcal\{R\},\\gamma\)\. The environment is the CIL training process: each episode is a full run ofNNincremental phases, each phase is one environment step, and we setγ=1\\gamma=1\. At stepi∈\{0,…,N−1\}i\\in\\\{0,\\dots,N\-1\\\}, the agent observes a statesi∈𝒮s\_\{i\}\\in\\mathcal\{S\}summarising current model performance and incoming\-phase data statistics, selects an actionai∈𝒜a\_\{i\}\\in\\mathcal\{A\}corresponding to the distillation coefficients\(αi,βi\)\(\\alpha\_\{i\},\\beta\_\{i\}\)for that phase, and receives a scalar rewardri=ℛ⁡\(si,ai\)r\_\{i\}=\\mathcal\{R\}\(s\_\{i\},a\_\{i\}\)after the CIL model has been trained underaia\_\{i\}\. The policyπθ​\(a∣s\)\\pi\_\{\\theta\}\(a\\mid s\)is trained to maximise the undiscounted cumulative return∑i=0Nri\\sum\_\{i=0\}^\{N\}r\_\{i\}\. We instantiate𝒮\\mathcal\{S\},𝒜\\mathcal\{A\}, andℛ\\mathcal\{R\}in the following subsections\.

#### 4\.2\.1State Representation

The statesis\_\{i\}is constructed by concatenating two groups of features that jointly characterise the factors governing the appropriate regularisation strength at each phase: the model’s current learning behaviour and the statistical properties of the incoming data\.

Performance metrics\.We include two scalar indicators\.*Average accuracy*after phaseiimeasures overall competence across all tasks seen so far:

Ai=1i​∑j=1iAi,j,\\displaystyle A\_\{i\}=\\frac\{1\}\{i\}\\sum\_\{j=1\}^\{i\}A\_\{i,j\},\(6\)whereAi,jA\_\{i,j\}is the accuracy on taskjjafter phaseii\. SinceAiA\_\{i\}conflates retention of old knowledge with acquisition of new, we also track*average forgetting*\. The forgetting on taskjjafter phasei\>ji\>jis defined as the drop from its peak accuracy,Fij=maxl∈\{j,…,i−1\}⁡Al,j−Ai,j,∀j<iF^\{j\}\_\{i\}=\\max\_\{l\\in\\\{j,\\dots,i\-1\\\}\}A\_\{l,j\}\-A\_\{i,j\},\\ \\forall j<i\. The average forgetting across all prior tasks is:

Fi=1i−1​∑j=1i−1ℱij\.\\displaystyle F\_\{i\}=\\frac\{1\}\{i\-1\}\\sum\_\{j=1\}^\{i\-1\}\\mathcal\{F\}\_\{i\}^\{j\}\.\(7\)
Data properties\.Performance metrics alone do not capture the demands of the incoming task\. We include two data\-level cues: class ratio and inter\-task similarity\.

The*class ratio*quantifies the relative volume of new data at phaseii:

C​Ri=∑j=0i−1\|Cj\|\|Ci\|,\\displaystyle CR\_\{i\}=\\frac\{\\sum\_\{j=0\}^\{i\-1\}\|C\_\{j\}\|\}\{\|C\_\{i\}\|\},\(8\)where\|Cj\|\|C\_\{j\}\|is the number of classes introduced at phasejj\. A largeC​RiCR\_\{i\}means the incoming phase adds far fewer classes than those already consolidated, placing greater pressure on stability\.

The*inter\-task similarity*measures how closely the new classes relate to previously learned ones, informing whether prior representations can be reused or stronger plasticity is needed\. We estimate it from class\-mean embeddings under a frozen ImageNet\-pretrained ResNet\-34:

μc=1\|Pc\|​∑p∈Pcϕ⁡\(xp\),\\displaystyle\\mu\_\{c\}=\\frac\{1\}\{\|P\_\{c\}\|\}\\sum\_\{p\\in P\_\{c\}\}\\phi\(x\_\{p\}\),\(9\)
wherePcP\_\{c\}is the sample set of classccandϕ⁡\(xp\)∈ℝd\\phi\(x\_\{p\}\)\\in\\mathbb\{R\}^\{d\}is the embedding of samplexpx\_\{p\}\. The mean distance between incoming and consolidated classes at phaseiiis:

Di=1\|Ci\|​∑c∈Ci1∑j<i\|Cj\|​∑j<i∑c′∈Cj∥μc−μc′∥2,\\displaystyle D\_\{i\}=\\frac\{1\}\{\|C\_\{i\}\|\}\\sum\_\{c\\in C\_\{i\}\}\\frac\{1\}\{\\sum\_\{j<i\}\|C\_\{j\}\|\}\\sum\_\{j<i\}\\sum\_\{c^\{\\prime\}\\in C\_\{j\}\}\\lVert\\mu\_\{c\}\-\\mu\_\{c^\{\\prime\}\}\\rVert\_\{2\},\(10\)which we convert to a similarity score viaI​Si=11\+DiIS\_\{i\}=\\frac\{1\}\{1\+D\_\{i\}\}so that higherI​SiIS\_\{i\}indicates greater similarity to prior classes\. The pretrained extractor is used solely for this estimate and plays no role in CIL model training\. Combining all four cues, the state at phaseiiissi=\(Ai−1,Fi−1,C​Ri,I​Si\)s\_\{i\}=\(A\_\{i\-1\},F\_\{i\-1\},CR\_\{i\},IS\_\{i\}\)\.

#### 4\.2\.2Action Space

The action at phaseiiis the pairai=\(αi,βi\)a\_\{i\}=\(\\alpha\_\{i\},\\beta\_\{i\}\)controlling the two distillation coefficients in Eq\.[5](https://arxiv.org/html/2610.31991#S4.E5)\. We discretise the action space into a finite grid,𝒜=Λα×Λβ\\mathcal\{A\}=\\Lambda\_\{\\alpha\}\\times\\Lambda\_\{\\beta\}, whereΛα\\Lambda\_\{\\alpha\}andΛβ\\Lambda\_\{\\beta\}are predefined value sets for each coefficient\. Discretisation yields a compact action space that reduces policy\-gradient variance and avoids constrained continuous\-action methods, at the cost of slightly coarser regularisation control\.

#### 4\.2\.3Reward

We use per\-phase validation accuracy as the reward:rir\_\{i\}is the accuracy of the CIL model on the held\-out reward split at the end of phaseii\. The episodic return overNNincremental phases isR=∑i=1NriR=\\sum\_\{i=1\}^\{N\}r\_\{i\}\. MaximisingRRincentivises the agent to sustain strong performance throughout the full learning trajectory rather than trading early\-phase retention for higher terminal accuracy\.

### 4\.3Agent Training

#### 4\.3\.1Pseudo\-Task Construction

Applying RL to CIL requires reward signals at each phase, which are unavailable during standard incremental training since past and future data are inaccessible\. Following\[[17](https://arxiv.org/html/2610.31991#bib.bib29)\], we construct pseudo\-CIL tasks entirely fromD0D\_\{0\}, which contains half the full dataset and thus provides sufficient data for meaningful smaller\-scale incremental episodes\. Each pseudo\-task uses a 90/10 train–validation split per phase, enabling reliable reward computation throughout\. Each episode proceeds forNNphases: at each step, the agent selects\(αi,βi\)\(\\alpha\_\{i\},\\beta\_\{i\}\), the CIL model is trained under Eq\.[5](https://arxiv.org/html/2610.31991#S4.E5), and the episodic return is the sum of per\-phase validation accuracies\.

#### 4\.3\.2Learning Algorithm

Table 1:Performance comparison on average accuracyA​C​CACC, average forgettingF​R​GFRGand forward transferF​W​TFWTon CIFAR\-100\.Table 2:Performance comparison on average accuracyA​C​CACC, average forgettingF​R​GFRGand forward transferF​W​TFWTon Tiny\-ImageNet\.We train the agent with Ape\-X DQN\[[8](https://arxiv.org/html/2610.31991#bib.bib11)\]\. Since each environment step requires a full CIL training phase, data collection is slow; Ape\-X addresses this by distributing collection across parallel actors, each running an independent CIL instance at a different exploration rate\. The resulting replay buffer captures a diverse range of experiences, which a central learner consumes in a prioritised fashion\.

We use Double DQN\[[7](https://arxiv.org/html/2610.31991#bib.bib12),[29](https://arxiv.org/html/2610.31991#bib.bib10)\]withnn\-step bootstrap targets\[[21](https://arxiv.org/html/2610.31991#bib.bib5)\]\. Thenn\-step return and training loss are:

Gi\\displaystyle G\_\{i\}=Ri\+1\+γ​Ri\+2\+⋯\+γn−1​Ri\+n\\displaystyle=R\_\{i\+1\}\+\\gamma R\_\{i\+2\}\+\\cdots\+\\gamma^\{n\-1\}R\_\{i\+n\}\(11\)\+γn​q​\(Si\+n,arg⁡max𝑎​q​\(Si\+n,a,θ\),θ−\),\\displaystyle\+\\gamma^\{n\}q\\\!\\left\(S\_\{i\+n\},\\underset\{a\}\{\\arg\\max\}\\;q\(S\_\{i\+n\},a;\\theta\),\\theta^\{\-\}\\right\),li​\(θ\)=12​\(Gi−q⁡\(Si,𝒜i,θ\)\)2,l\_\{i\}\(\\theta\)=\\frac\{1\}\{2\}\\bigl\(G\_\{i\}\-q\(S\_\{i\},\\mathcal\{A\}\_\{i\};\\theta\)\\bigr\)^\{2\},\(12\)whereθ−\\theta^\{\-\}are the parameters of the slowly updated target network\. We setnnequal to the number of incremental phases per pseudo\-task andγ=1\\gamma=1, consistent with the finite\-horizon MDP in Section[4\.2](https://arxiv.org/html/2610.31991#S4.SS2)\.

## 5Experimental Setup

### 5\.1Datasets

We evaluate on CIFAR\-100 \(100 classes,32×3232\\times 32images\) and Tiny\-ImageNet \(200 classes,64×6464\\times 64images\)\. Following the standard train\-from\-half protocol, models are first trained on half the classes \(50 and 100, respectively\), with the remainder introduced overN∈\{2,5,10\}N\\in\\\{2,5,10\\\}incremental phases\.

### 5\.2Implementation Details

We use ResNet\-32 for CIFAR\-100 and ResNet\-18 for Tiny\-ImageNet, trained with SGD \(momentum0\.90\.9, weight decay5×10−45\\times 10^\{\-4\}\) and batch size of128128for160160epochs under a step\-decay learning rate schedule, with a fixed memory budget of20002000exemplars selected by herding\[[23](https://arxiv.org/html/2610.31991#bib.bib15)\]\. The RL agent is a three\-layer MLPq⁡\(s,a,θ\)q\(s,a;\\theta\)mapping the four\-dimensional state\(Ai−1,Fi−1,C​Ri,I​Si\)\(A\_\{i\-1\},F\_\{i\-1\},CR\_\{i\},IS\_\{i\}\)to Q\-values over a discrete action set withΛα=Λβ=\{0\.1,0\.5,1,1\.5,2,2\.5,5\}\\Lambda\_\{\\alpha\}=\\Lambda\_\{\\beta\}=\\\{0\.1,0\.5,1,1\.5,2,2\.5,5\\\}, optimised with Adam \(learning rate0\.010\.01\)\. We train the policy offline using Ape\-X DQN\[[8](https://arxiv.org/html/2610.31991#bib.bib11)\]with 7 parallel actors \(ϵ\\epsilon\-greedy exploration,100100episodes each\) and a centralised learner with prioritised replay\. To keep offline cost tractable, CIL models during policy training use2020epochs per phase on pseudo tasks rather than the160160used at deployment\. At inference, a single MLP forward pass per phase adds negligible overhead\.

### 5\.3Evaluation Metrics

We report three metrics: mean accuracy \(A​C​CACC\) over all classes seen so far, mean forgetting \(F​R​GFRG\), and forward transfer \(F​W​TFWT\), defined asF​W​T=Ai−1,θCL−Ai,θrandomFWT=A\_\{i\-1,\\,\\theta\_\{\\text\{CL\}\}\}\-A\_\{i,\\,\\theta\_\{\\text\{random\}\}\}, whereAi,θrandomA\_\{i,\\theta\_\{\\text\{random\}\}\}is the performance of a randomly initialised model on phaseiiandAi−1,θCLA\_\{i\-1,\\theta\_\{\\text\{CL\}\}\}is that of the CIL model immediately before learning phaseii\. A positiveF​W​TFWTindicates that prior learning facilitates adaptation to new classes\.

## 6Results

Tables[1](https://arxiv.org/html/2610.31991#S4.T1)and[2](https://arxiv.org/html/2610.31991#S4.T2)report accuracy, forgetting, and forward transfer across CIFAR\-100 \(three incremental settings,N∈\{2,5,10\}N\\in\\\{2,5,10\\\}\) and Tiny\-ImageNet \(two incremental settings,N∈\{5,10\}N\\in\\\{5,10\\\}\)\. Figure[2](https://arxiv.org/html/2610.31991#S6.F2)shows phase\-wise accuracy forN=5N\{=\}5\.REDARachieves the best or near\-best accuracy on both datasets across most settings, and consistently leads forward transfer by the widest margin of any method, indicating that the learned regularisation policy is particularly effective at shaping representations that generalise to future tasks\.

CIFAR\-100\.REDARachieves the best or near\-best accuracy across all three incremental settings \(67\.60%, 64\.43%, and 61\.53% atN=2,5,10,N\{=\}2,5,10,respectively\), and leads forward transfer by a substantial margin in every case \(43\.78%, 51\.77%, and 49\.36%, each of which is the highest among all methods\), indicating that the learned regularisation policy consistently produces representations that generalise well to future tasks rather than merely resisting forgetting on past ones\. WA occasionally reports lower forgetting, most notably atN=10N\{=\}10\(28\.88% vs\.REDAR30\.17%\), but this comes paired with a 3\.76\-point accuracy deficit relative toREDAR\(57\.77% vs\. 61\.53%\), consistent with a stability\-biased regime that suppresses forgetting by under\-acquiring new classes rather than genuinely retaining prior knowledge more effectively\. Taken together, these results suggest that our adaptive policy resolves the stability\-plasticity trade\-off more favourably than static or heuristic baselines, rather than simply trading one failure mode for another\.

Tiny\-ImageNet\.AtN=5N\{=\}5,REDARleads all three metrics: highest accuracy \(57\.68%\), lowest forgetting \(14\.2%\), and best forward transfer \(61\.4%\)\. AtN=10N\{=\}10, OHOCIL and LUCIR attain higher accuracy \(54\.93% and 53\.41% vs\. our 47\.37%\) and lower forgetting \(30\.65% and 27\.60% vs\. our 39\.61%\), but our forward transfer remains the strongest overall \(71\.75% forREDARvs\. 68\.12% for LUCIR\), indicating that even where raw retention lags at longer horizons, the learned policy still yields representations that transfer better to future tasks\.

Figure 2:Top: Phase\-wise accuracy on CIFAR\-100 and Tiny\-ImageNet under theN=5N=5incremental setting\. Accuracy is reported as the mean over all classes seen after each phase\. Bottom: coefficients \(α\\alpha,β\\beta\) used to train each phase\.Figure[2](https://arxiv.org/html/2610.31991#S6.F2)\(bottom\) shows the coefficients selected on each dataset: the policy starts feature\-heavy \(α=2\\alpha\{=\}2,β=0\.1\\beta\{=\}0\.1\) and shifts weight towards logit distillation in the middle phases\. Across both datasets, the consistent advantage in forward transfer indicatesREDARshapes representations that generalise better to future tasks than static or heuristic schedules\.

### 6\.1Cross\-Setting Transfer

A key advantage ofREDARover fixed hyperparameter methods is that the learned policy is not tied to a specific number of incremental phases\. Figure[3](https://arxiv.org/html/2610.31991#S6.F3)evaluates this by training the policy onNtrainN\_\{\\text\{train\}\}phases and deploying it onNtestN\_\{\\text\{test\}\}phases without any retraining\. We compare this with a fixed\-regularisation hyperparameter setting, representing the best that a static approach can achieve without retuning\.

REDARgeneralises well across settings without retraining\. A policy trained onN=2N\{=\}2achieves 64\.74% when deployed onN=10N\{=\}10, outperforming fixed hyperparameters tuned specifically forN=10N\{=\}10\(61\.24%\)\. Similarly, a policy trained onN=5N\{=\}5achieves 63\.95% onN=10N\{=\}10, again surpassing the fixed baseline\. In contrast, fixed hyperparameters collapse under cross\-setting evaluation: hyperparameters tuned forN=10N\{=\}10drop to 49\.36% when applied toN=2N\{=\}2, a gap of 17\.08% compared to our policy’s 67\.60%\. This highlights a fundamental limitation of static regularisation: a single globally tuned coefficient cannot adapt to varying incremental horizons, whereasREDARadjusts dynamically based on the observed state at each phase\.

Why shorter training horizons transfer to longer ones?With fewer phases, each decision carries disproportionate weight on the final return, forcing the policy to learn a more conservative, generalisable strategy; with more phases, it can afford early suboptimal decisions and recover later, yielding a more horizon\-specific policy that transfers less well\. We attribute this to our state design: conditioning onAi−1A\_\{i\-1\},Fi−1F\_\{i\-1\},C​RiCR\_\{i\}, andI​SiIS\_\{i\}rather than the phase index lets the policy react to current training dynamics rather than a fixed schedule, remaining applicable whenever the observed state matches the training distribution, a phase\-agnostic inductive bias that static hyperparameter selection cannot achieve\.

![Refer to caption](https://arxiv.org/html/2610.31991v1/transfer_heatmap.png)Figure 3:Cross\-setting transfer evaluation on CIFAR\-100\. Each cell reports ACC \(%\) for a policy trained onNtrainN\_\{\\text\{train\}\}phases and deployed onNtestN\_\{\\text\{test\}\}phases\. Diagonal entries \(pink outline\) correspond to standard in\-distribution evaluation\.
### 6\.2Benefit of Adaptive Regularisation

Table 3:Benefit of varying hyperparameters\.Table[3](https://arxiv.org/html/2610.31991#S6.T3)comparesREDARagainst two baselines: a fixed regularisation strength of 0\.1 and random selection from the same discrete action space𝒜\\mathcal\{A\}\. Random selection serves as a lower bound that isolates the contribution of the learned policy from the benefit of simply varying the regularisation strength across phases\. Random selection performs substantially worse than both fixed and learned regularisation \(59\.84% ACC, 43\.16% FWT\), confirming that varying the regularisation strength alone is not beneficial\. The*choice*of strength at each phase matters\.REDARoutperforms the fixed baseline across all three metrics \(64\.43% vs 63\.93% ACC, 28\.02% vs 31\.92% FRG, 51\.77% vs 49\.42% FWT\), demonstrating that the policy learns a meaningful regularisation strategy rather than simply introducing variation\. Together, these results confirm that the performance gains of our method stem from the quality of the learned policy, not from the flexibility of the action space alone\.

### 6\.3Ablation Studies

#### 6\.3\.1Ablation of State Representation

We validate the contribution of each component in the state representation through a systematic ablation study, with results presented in Table[4](https://arxiv.org/html/2610.31991#S6.T4)\. The full statesi=\(Ai−1,Fi−1,C​Ri,I​Si\)s\_\{i\}=\(A\_\{i\-1\},F\_\{i\-1\},CR\_\{i\},IS\_\{i\}\)combines two performance metrics: accuracyA​C​CACCand forgettingF​R​GFRG, with two data properties, class ratioC​RCRand inter\-task similarityI​SIS\. RemovingI​SISwhile retaining all other components causes a notable drop across all metrics \(64\.43%→\\to59\.97% ACC, 28\.02%→\\to45\.46% FRG\), indicating that inter\-task similarity is the most influential component\. This is intuitive: without knowledge of how closely the incoming classes relate to previously seen ones, the agent cannot anticipate whether stronger plasticity or stronger stability is required for the current phase\.

RemovingF​R​GFRGwhile retainingI​SIShas a smaller but still measurable effect, suggesting that forgetting provides complementary signal to accuracy alone\. A model with similar overall accuracy may be forgetting old classes at very different rates, and the agent benefits from observing this directly\. The full combination of all four components yields the best performance across all metrics, confirming that each component contributes a distinct and non\-redundant signal to the policy’s decision making\.

Table 4:Ablation results of the state representationState RepresentationMetricsCRACCFRGISACCFRGFWT✓✓✗✗62\.4837\.3945\.42✓✓✓✗59\.9745\.4639\.96✓✓✗✓62\.4937\.4448\.82✓✓✓✓64\.4328\.0251\.77
#### 6\.3\.2Ablation of Reward Design

Table 5:Ablation results of the reward design\.We validate the choice of reward function by comparing per\-phase validation accuracy with learning effectiveness\[[28](https://arxiv.org/html/2610.31991#bib.bib20)\], a composite metric defined as a harmonic\-like combination of accuracyA​C​CACCand forgettingF​R​GFRG:

L​E=2F​R​G\+1A​C​C\.\\displaystyle LE=\\frac\{2\}\{FRG\+\\frac\{1\}\{ACC\}\}\.\(13\)
Results are presented in Table[5](https://arxiv.org/html/2610.31991#S6.T5)\. While learning effectiveness combines both accuracy and forgetting into a single scalar, using it as the reward signal leads to worse performance across all three metrics compared to using validation accuracy directly \(64\.27% vs 64\.43% ACC, 33\.38% vs 28\.02% FRG, 48\.66% vs 51\.77% FWT\)\. We attribute this to the composite nature ofL​ELE: by conflating accuracy and forgetting into a single value, the agent receives an ambiguous training signal that does not clearly distinguish between a model that learns new classes and one that simply retains old knowledge\. Per\-phase validation accuracy, by contrast, provides a direct and unambiguous measure of the model’s current competence across all observed classes, giving the agent cleaner credit assignment at each incremental step\.

## 7Conclusions

We presentREDAR, which shows that viewing regularisation strength in CIL as a controllable signal, and learning a phase\-wise policy for it via reinforcement learning, yields a more favourable stability–plasticity trade\-off than fixed or heuristic schedules\. This adaptivity is most evident as the incremental horizon lengthens: our policy’s advantage over the strongest baseline widens with more phases, and its gains concentrate in forward transfer, indicating that it does more than resist forgetting on past classes; it actively shapes representations that generalise to future ones\. These findings suggest that the per\-phase stability–plasticity balance is itself a learnable quantity, rather than something that needs to be fixed in advance or hand\-tuned per dataset\.REDARcurrently operates over a discretised action space for the regularisation coefficients and is trained offline on a single rehearsal\-based backbone; extending it to continuous control, other CIL backbones \(e\.g\., prompt\- or expansion\-based methods\), and online adaptation during deployment are natural directions for future work\.

## Acknowledgment

The authors would like to acknowledge support from the CSIRO’s Data61 Embodied AI Cluster and partial support from the Australian Research Council \(ARC\) Discovery Project Grant DP250103634\.

## References

- \[1\]P\. Buzzega, M\. Boschini, A\. Porrello, D\. Abati, and S\. Calderara\(2020\)Dark experience for general continual learning: a strong, simple baseline\.Advances in neural information processing systems33,pp\. 15920–15930\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[2\]J\. C\. Caicedo and S\. Lazebnik\(2015\)Active object localization with deep reinforcement learning\.InProceedings of the IEEE international conference on computer vision,pp\. 2488–2496\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[3\]F\. M\. Castro, M\. J\. Marín\-Jiménez, N\. Guil, C\. Schmid, and K\. Alahari\(2018\)End\-to\-end incremental learning\.InProceedings of the European conference on computer vision \(ECCV\),pp\. 233–248\.Cited by:[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.5.1)\.
- \[4\]A\. Chaudhry, M\. Rohrbach, M\. Elhoseiny, T\. Ajanthan,et al\.\(2019\)On tiny episodic memories in continual learning\.arXiv preprint arXiv:1902\.10486\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[5\]A\. Douillard, M\. Cord, C\. Ollion, T\. Robert, and E\. Valle\(2020\)Podnet: pooled outputs distillation for small\-tasks incremental learning\.InEuropean conference on computer vision,pp\. 86–102\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.31991#S4.T1.2.7.1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.8.1)\.
- \[6\]Z\. Feng, M\. Zhou, Z\. Gao, A\. Stefanidis, and Z\. Sui\(2024\)SCREAM: knowledge sharing and compact representation for class incremental learning\.Information Processing & Management61\(3\),pp\. 103629\.External Links:ISSN 0306\-4573,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.ipm.2023.103629),[Link](https://www.sciencedirect.com/science/article/pii/S0306457323003667)Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.11.1)\.
- \[7\]H\. Hasselt\(2010\)Double q\-learning\.Advances in neural information processing systems23\.Cited by:[§4\.3\.2](https://arxiv.org/html/2610.31991#S4.SS3.SSS2.p2.2)\.
- \[8\]D\. Horgan, J\. Quan, D\. Budden, G\. Barth\-Maron, M\. Hessel, H\. van Hasselt, and D\. Silver\(2018\)Distributed prioritized experience replay\.External Links:1803\.00933,[Link](https://arxiv.org/abs/1803.00933)Cited by:[§4\.3\.2](https://arxiv.org/html/2610.31991#S4.SS3.SSS2.p1.1),[§5\.2](https://arxiv.org/html/2610.31991#S5.SS2.p1.1)\.
- \[9\]S\. Hou, X\. Pan, C\. C\. Loy, Z\. Wang, and D\. Lin\(2019\)Learning a unified classifier incrementally via rebalancing\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 831–839\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[§4\.1](https://arxiv.org/html/2610.31991#S4.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.31991#S4.T1.2.5.1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.6.1)\.
- \[10\]W\. Huang, C\. R\. Chen, and H\. Hsu\(2024\)OVOR: oneprompt with virtual outlier regularization for rehearsal\-free class\-incremental learning\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2402.04129)Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[11\]J\. Ibarz, J\. Tan, C\. Finn, M\. Kalakrishnan, P\. Pastor, and S\. Levine\(2021\)How to train your robot with deep reinforcement learning: lessons we have learned\.The International Journal of Robotics Research40\(4\-5\),pp\. 698–721\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[12\]J\. Kirkpatrick, R\. Pascanu, N\. Rabinowitz, J\. Veness,et al\.\(2017\)Overcoming catastrophic forgetting in neural networks\.Proceedings of the national academy of sciences114\(13\),pp\. 3521–3526\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[13\]M\. Klasson, H\. Kjellström, and C\. Zhang\(2022\)Learn the time to learn: replay scheduling in continual learning\.arXiv preprint arXiv:2209\.08660\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[14\]K\. Kotar and R\. Mottaghi\(2022\)Interactron: embodied adaptive object detection\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 14860–14869\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[15\]Z\. Li and D\. Hoiem\(2017\)Learning without forgetting\.IEEE transactions on pattern analysis and machine intelligence40\(12\),pp\. 2935–2947\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[16\]Y\. Liu, Y\. Li, B\. Schiele, and Q\. Sun\(2023\)Online hyperparameter optimization for class\-incremental learning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 8906–8913\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2610.31991#S4.T1.2.9.1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.10.1)\.
- \[17\]Y\. Liu, B\. Schiele, and Q\. Sun\(2021\)Rmm: reinforced memory management for class\-incremental learning\.Advances in neural information processing systems34,pp\. 3478–3490\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1),[§4\.3\.1](https://arxiv.org/html/2610.31991#S4.SS3.SSS1.p1.1)\.
- \[18\]D\. Lopez\-Paz and M\. Ranzato\(2017\)Gradient episodic memory for continual learning\.Advances in neural information processing systems30\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[19\]Z\. Luo, Y\. Liu, B\. Schiele, and Q\. Sun\(2023\)Class\-incremental exemplar compression for class\-incremental learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 11371–11380\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.9.1)\.
- \[20\]L\. Mahmoodi, P\. Moghadam, M\. Hayat, C\. Simon, and M\. Harandi\(2025\)Flashbacks to harmonize stability and plasticity in continual learning\.Neural Networks190,pp\. 107616\.External Links:ISSN 0893\-6080,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.neunet.2025.107616),[Link](https://www.sciencedirect.com/science/article/pii/S0893608025004964)Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[21\]V\. Mnih, A\. P\. Badia, M\. Mirza, A\. Graves, T\. Lillicrap,et al\.\(2016\)Asynchronous methods for deep reinforcement learning\.InInternational conference on machine learning,pp\. 1928–1937\.Cited by:[§4\.3\.2](https://arxiv.org/html/2610.31991#S4.SS3.SSS2.p2.2)\.
- \[22\]C\. Peng, P\. Koniusz, K\. Guo, B\. C\. Lovell, and P\. Moghadam\(2024\)Multivariate prototype representation for domain\-generalized incremental learning\.Computer Vision and Image Understanding249,pp\. 104215\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[23\]S\. Rebuffi, A\. Kolesnikov, G\. Sperl, and C\. H\. Lampert\(2017\)Icarl: incremental classifier and representation learning\.InProceedings of the IEEE conference on Computer Vision and Pattern Recognition,pp\. 2001–2010\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.31991#S4.T1.2.4.1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.4.1),[§5\.2](https://arxiv.org/html/2610.31991#S5.SS2.p1.1)\.
- \[24\]K\. Roy, P\. Moghadam, and M\. Harandi\(2023\)L3DMC: lifelong learning using distillation via mixed\-curvature space\.InInternational conference on medical image computing and computer\-assisted intervention,pp\. 123–133\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[25\]K\. Roy, C\. Simon, P\. Moghadam, and M\. Harandi\(2023\)Subspace distillation for continual learning\.Neural Networks167,pp\. 65–79\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1)\.
- \[26\]H\. Shin, J\. K\. Lee, J\. Kim, and J\. Kim\(2017\)Continual learning with deep generative replay\.Advances in neural information processing systems30\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[27\]D\. Silver, A\. Huang, C\. J\. Maddison, A\. Guez, L\. Sifre,et al\.\(2016\)Mastering the game of go with deep neural networks and tree search\.Nature529\(7587\),pp\. 484–489\.External Links:[Document](https://dx.doi.org/10.1038/nature16961)Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[28\]P\. Singh Y\. Liet al\.\(2023\)Learning to learn: how to continuously teach humans and machines\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 11708–11719\.Cited by:[§6\.3\.2](https://arxiv.org/html/2610.31991#S6.SS3.SSS2.p1.1)\.
- \[29\]H\. Van Hasselt, A\. Guez, and D\. Silver\(2016\)Deep reinforcement learning with double q\-learning\.InProceedings of the AAAI conference on artificial intelligence,Vol\.30\.Cited by:[§4\.3\.2](https://arxiv.org/html/2610.31991#S4.SS3.SSS2.p2.2)\.
- \[30\]Y\. Xiang, Y\. Fu, P\. Ji, and H\. Huang\(2019\)Incremental learning using conditional adversarial networks\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 6619–6628\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1)\.
- \[31\]J\. Xu and Z\. Zhu\(2018\)Reinforced continual learning\.Advances in neural information processing systems31\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[32\]S\. Yan, J\. Xie, and X\. He\(2021\)DER: dynamically expandable representation for class incremental learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 3014–3023\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1)\.
- \[33\]E\. C\. G\. Yildirim, M\. O\. Yildirim, M\. Kilickaya, and J\. Vanschoren\(2023\)Adaptive regularization for class\-incremental learning\.CoRRabs/2303\.13113\.External Links:[Link](https://arxiv.org/abs/2303.13113),2303\.13113Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[34\]F\. Zenke, B\. Poole, and S\. Ganguli\(2017\)Continual learning through synaptic intelligence\.InInternational conference on machine learning,pp\. 3987–3995\.Cited by:[§1](https://arxiv.org/html/2610.31991#S1.p2.1)\.
- \[35\]B\. Zhao, X\. Xiao, G\. Gan, B\. Zhang, and S\. Xia\(2020\)Maintaining discrimination and fairness in class incremental learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 13208–13217\.Cited by:[§2\.1](https://arxiv.org/html/2610.31991#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.31991#S4.T1.2.6.1.1),[Table 2](https://arxiv.org/html/2610.31991#S4.T2.2.1.7.1)\.
- \[36\]D\. Zhou, H\. Ye, and D\. Zhan\(2021\)Co\-transport for class\-incremental learning\.InProceedings of the 29th ACM International Conference on Multimedia,pp\. 1645–1654\.Cited by:[Table 1](https://arxiv.org/html/2610.31991#S4.T1.2.8.1.1)\.
- \[37\]L\. Zhu, T\. Chen, J\. Yin, S\. See, and J\. Liu\(2023\)Continual semantic segmentation with automatic memory sample selection\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 3082–3092\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.
- \[38\]B\. Zoph and Q\. V\. Le\(2016\)Neural architecture search with reinforcement learning\.arXiv preprint arXiv:1611\.01578\.Cited by:[§2\.2](https://arxiv.org/html/2610.31991#S2.SS2.p1.1)\.