Beyond Class Marginals: Bounding Rehearsal Gaps without Freezing Class Co-occurrence

# Beyond Class Marginals:Bounding Rehearsal Gaps without Freezing Class Co-occurrence
Source: [https://arxiv.org/html/2609.25735](https://arxiv.org/html/2609.25735)
Nat RoongjiraratAffiliation:King’s College LondonFei Ye††thanks:Corresponding author\.Affiliation:University of Electronic Science and Technology of China

###### Abstract

Class\-balanced replay controls class frequency but does not determine the interval between successive replay appearances of a class\. We study this interval, the*rehearsal gap*, separately from the class marginal and class co\-occurrence, and introduce*randomised\-pass replay*\(RPR\), which visits each resident class once per shuffled pass\. For a fixed set ofCCresident classes and replay batch sizeb≤Cb\\leq C, RPR preserves the balanced time\-averaged class marginal and bounds every gap by2​⌈C/b⌉−12\\lceil C/b\\rceil\-1; a churn\-conditional bound applies while the resident set changes\. The scheduler uses no future class information and adds no replay examples or forward passes\. In a linear\-head ER\-ACE diagnostic, joint absence from the incoming and replay batches produces a one\-sided classifier\-bias gradient\. Longer absence episodes are associated with larger negative bias displacement, and removing the incoming\-loss mask attenuates the scheduling effect\. In the primary ER\-ACE experiments, RPR improves final average accuracy by0\.720\.72–1\.671\.67percentage points relative to independent class\-balanced retrieval under reservoir storage, with positive effects also observed under balanced storage\. Pretrained ViTs show positive effects on the tested LT10 streams with small replay batches, while matched larger\-batch controls show no material effect\. Fixed\-cycle and reused\-pass controls change more than one temporal statistic, so the experiments do not isolate rehearsal\-gap length from all other forms of temporal dependence\. The accuracy effects depend on the learner and operating regime\.

## 1Introduction

Online continual learning requires a model to acquire new knowledge while retaining classes that become infrequent or disappear from the incoming stream\. Experience replay addresses this by revisiting a small memory of past examples\([Chaudhry et al\., 2019](https://arxiv.org/html/2609.25735#bib.bib3)\)\. Under an imbalanced stream, storage alone does not determine when retained examples are replayed\. Storage\-side balancing\([Chrysakis and Moens, 2020](https://arxiv.org/html/2609.25735#bib.bib14);[Buzzega et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib18)\)controls which classes occupy memory, and class\-balanced retrieval controls their expected share of a replay batch\. Neither specifies*when*a stored class returns\.

Replay order determines the sequence of gradient updates even when aggregate class exposure is unchanged\. Equal average exposure can coexist with long periods of absence, during which other classes continue to update the model\. We call the interval between successive replay appearances of a class its*rehearsal gap*\. Table[4](https://arxiv.org/html/2609.25735#A3.T4)gives a fixed\-memory example\. On a fixed balanced memory with200200classes and eight replay examples per step, independent balanced retrieval and a shuffled\-pass schedule have mean gaps of24\.8224\.82and25\.0025\.00steps, and maximum gaps over3,0003\{,\}000steps of263263and4949, respectively\.

We distinguish three properties of a replay sequence: the*class marginal*is how often a class appears; the*rehearsal gap*is how long it waits between appearances; and*class co\-occurrence*is which classes share a batch \(Figure[1](https://arxiv.org/html/2609.25735#S1.F1)a\)\. On a fixed resident set larger than the replay batch, independent class\-balanced retrieval gives a geometric rehearsal\-gap distribution with unbounded support\. Fixed class cycling\([Hickok and Kudithipudi, 2024](https://arxiv.org/html/2609.25735#bib.bib1)\)gives bounded gaps and repeated pairings determined by the cycle\. At a fixed replay budget, we study schedules that bound resident\-class rehearsal gaps while preserving the fixed\-set time\-averaged marginal and resampling class co\-occurrence across passes, and evaluate their effect on learning\.

![Refer to caption](https://arxiv.org/html/2609.25735v1/rpr_figure.png)Figure 1:Replay properties and randomised\-pass replay \(RPR\)\.\(a\) On a fixed resident set with replay batchb≤Cb\\leq C, independent balanced retrieval, fixed cycling, and RPR have the same time\-averaged class marginalb/Cb/Cbut may differ in gap tails and class co\-occurrence\. \(b\) RPR keeps a persistent shuffled deck of resident classes, samples one stored example per selected class, and reshuffles when the deck empties\. On a fixed resident set, it preservesμ¯c=b/C\\bar\{\\mu\}\_\{c\}=b/Cand boundsGc≤2​⌈C/b⌉−1G\_\{c\}\\leq 2\\lceil C/b\\rceil\-1\. It adds no replay examples or forward passes\.Randomised\-pass replay \(RPR\) maintains a persistent shuffled deck of resident classes\. Each class is visited once per pass, after which the deck is reshuffled \(Figure[1](https://arxiv.org/html/2609.25735#S1.F1)b\)\. The deck is updated as classes enter or leave memory\. The scheduler requires no future class identities, total class count, or task boundaries\. On a fixed set ofCCresident classes with replay batchb≤Cb\\leq C, its time\-averaged marginal is balanced and every gap is bounded by2​⌈C/b⌉−12\\lceil C/b\\rceil\-1\. A churn\-conditional bound covers changing memory; exact marginal equality applies only in the fixed\-set limit\.

The scheduling guarantees do not imply an accuracy improvement\. We analyse a linear\-head ER\-ACE diagnostic\([Caccia et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib12)\)to relate replay gaps to parameter updates\. The incoming\-loss mask removes a seen but absent class from incoming competition, while replay examples of other classes continue to contribute a positive gradient to its classifier bias\. During joint absence, the class\-specific bias receives gradients of the same sign at every step while the class remains resident\. We evaluate this mechanism using measured bias displacement and mask interventions\. The primary accuracy study uses a bias\-free cosine head\. The bias\-drift analysis therefore applies directly to the linear\-head diagnostic and does not by itself explain the primary cosine\-head results\. We compare fixed cycling, reused permutations, storage policies, and replay budgets to separate scheduling effects from storage balance\. Pretrained\-backbone controls evaluate the same retrieval intervention outside the primary ER\-ACE configuration\.

The shuffled\-pass primitive is standard; our contributions concern its use in replay\.1\) Replay\-schedule decomposition\.We distinguish class marginal frequency, rehearsal gaps, and class co\-occurrence as separate properties of a replay sequence\.2\) Scheduling guarantees\.For RPR, we derive the fixed\-set time\-averaged marginal, a worst\-case resident\-class gap bound, and a churn\-conditional bound\. We also give a lower bound on the maximum gap of any recurrent schedule and, under the stated conditions, the fixed\-partition structure induced by attaining it exactly\.3\) Mechanistic and empirical evaluation\.We measure gap\-conditioned bias displacement in ER\-ACE and evaluate retrieval order across storage policies, replay losses, replay budgets, and pretrained backbones\. The temporal controls change more than one schedule statistic, so the experiments do not isolate rehearsal\-gap length from all other forms of temporal dependence\.

## 2Related Work

#### Storage and retrieval\.

Reservoir sampling\([Vitter, 1985](https://arxiv.org/html/2609.25735#bib.bib2)\)retains a stream\-uniform sample and is standard in online continual learning\([Chaudhry et al\., 2019](https://arxiv.org/html/2609.25735#bib.bib3)\)\. CBRS\([Chrysakis and Moens, 2020](https://arxiv.org/html/2609.25735#bib.bib14)\)and Balanced Reservoir Sampling \(BRS\)\([Buzzega et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib18)\)instead favour balanced occupancy; GSS diversifies stored gradients\([Aljundi et al\., 2019b](https://arxiv.org/html/2609.25735#bib.bib6)\), and InfoRS gates reservoir eligibility by information\([Sun et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib15)\)\. These methods modify memory contents rather than the temporal order of class retrieval\. Among retrieval methods, MIR selects examples whose loss increases after a virtual incoming update\([Aljundi et al\., 2019a](https://arxiv.org/html/2609.25735#bib.bib17)\), and ASER uses a Shapley\-value proxy\([Shim et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib7)\)\. Both optimise per\-example criteria rather than the class\-level rehearsal\-gap statistics studied here\. CeCR samples classes without replacement within a joint retrieval, storage, and loss design\([Sun et al\., 2024](https://arxiv.org/html/2609.25735#bib.bib23)\)\. We compare schedules at fixed storage, loss, and replay budget; exact time\-averaged marginal matching applies to the fixed\-resident\-set analysis\.

#### Cyclic scheduling and reshuffling\.

The Balanced primitive of[Hickok and Kudithipudi \(2024\)](https://arxiv.org/html/2609.25735#bib.bib1)continues through classes in fixed class\-id order; their separate deduplication schedules limit repeated examples\. We include this class\-cycle primitive as a baseline\. It gives bounded class gaps and, on a fixed resident set, a repeated class partition whenb\|Cb\\mid C\. Task\-level scheduling instead learns which previous tasks to replay\([Klasson et al\., 2023](https://arxiv.org/html/2609.25735#bib.bib24)\), whereas RPR orders resident classes without task boundaries\. Random reshuffling is well established for finite\-sum optimisation\([Gürbüzbalaban et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib20);[HaoChen and Sra, 2019](https://arxiv.org/html/2609.25735#bib.bib21);[Mishchenko et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib22)\)\. RPR applies it to a changing set of resident*class identifiers*instead of a static set of objective components\. Our analysis concerns replay gaps, class co\-occurrence, and their interaction with learning; it does not use finite\-sum convergence results\.

#### Replay losses\.

ER\-ACE masks seen\-but\-absent classes from the incoming loss\([Caccia et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib12)\)\. DER regresses stored logits, and DER\+\+ adds supervised replay cross\-entropy\([Buzzega et al\., 2020](https://arxiv.org/html/2609.25735#bib.bib4)\)\. Contrastive and prototype\-based methods couple replay examples through batch\-level objectives\([Mai et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib13);[Guo et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib19);[Wei et al\., 2023](https://arxiv.org/html/2609.25735#bib.bib5)\)\. Definition[1](https://arxiv.org/html/2609.25735#Thmdefinition1)characterises replay terms in which each example’s gradient contribution is independent of the other examples in the batch; Section[5\.2](https://arxiv.org/html/2609.25735#S5.SS2.SSS0.Px4)relates this property to full training trajectories\.

## 3Setting and Notation

A task\-free online learner processes a streamS=\(\(xi,yi\)\)i=1NS=\(\(x\_\{i\},y\_\{i\}\)\)\_\{i=1\}^\{N\}once, without task\-boundary signals\. At steptt, it receives an incoming minibatchAtA\_\{t\}, retrievesBtB\_\{t\}from a memoryMtM\_\{t\}of at mostmmpreviously observed examples, and takes one gradient step on

ℒ⁡\(θt\)=ℓon​\(θt,At\)\+ℓre​\(θt,Bt\)\.\\mathcal\{L\}\(\\theta\_\{t\}\)=\\ell^\{\\mathrm\{on\}\}\(\\theta\_\{t\};A\_\{t\}\)\+\\ell^\{\\mathrm\{re\}\}\(\\theta\_\{t\};B\_\{t\}\)\.\(1\)The requested replay budget isnt=min⁡\(b,\|Mt\|\)n\_\{t\}=\\min\(b,\|M\_\{t\}\|\)and the realised size obeys\|Bt\|≤nt\|B\_\{t\}\|\\leq n\_\{t\}\(Appendix[D](https://arxiv.org/html/2609.25735#A4)\); an empty memory contributes no replay loss\. Write𝒞⁡\(B\)=\{y:\(x,y\)∈B\}\\mathcal\{C\}\(B\)=\\\{y:\(x,y\)\\in B\\\}and𝒮t=⋃s≤t𝒞⁡\(As\)\\mathcal\{S\}\_\{t\}=\\bigcup\_\{s\\leq t\}\\mathcal\{C\}\(A\_\{s\}\)for the classes in a batch and those seen so far\. The scheduler uses only the current resident set

𝒦t=𝒞⁡\(Mt\),Ct=\|𝒦t\|\.\\mathcal\{K\}\_\{t\}=\\mathcal\{C\}\(M\_\{t\}\),\\qquad C\_\{t\}=\|\\mathcal\{K\}\_\{t\}\|\.\(2\)It requires neither the identities nor the number of future classes: a class enters𝒦t\\mathcal\{K\}\_\{t\}only after observation and storage\. We useCCfor a fixed resident count in the analysis, not an advance input to RPR\. Storage determines which examples remain inMtM\_\{t\}; retrieval determines which return inBtB\_\{t\}\. Section[5\.2](https://arxiv.org/html/2609.25735#S5.SS2.SSS0.Px1)varies these rules independently\.

## 4Randomised Pass Replay

### 4\.1Replay Marginals, Gaps, and Co\-occurrence

For a retrieval policyπ\\pi, distinguish the history\-conditional inclusion probabilityμc\(t\)=Pr⁡\[c∈𝒞⁡\(Bt\)∣ℱt\]\\mu\_\{c\}^\{\(t\)\}=\\Pr\[c\\in\\mathcal\{C\}\(B\_\{t\}\)\\mid\\mathcal\{F\}\_\{t\}\], conditional on the history available before retrieval at steptt, from the time\-averaged marginal

μ¯c\(π\)=limN→∞1N∑t=1NPr\[c∈𝒞\(Bt\)\],\\bar\{\\mu\}\_\{c\}\(\\pi\)=\\lim\_\{N\\to\\infty\}\\frac\{1\}\{N\}\\sum\_\{t=1\}^\{N\}\\Pr\[c\\in\\mathcal\{C\}\(B\_\{t\}\)\],\(3\)when this limit exists on a fixed resident set\. Finite online runs use empirical visit frequencies\. I\.i\.d\. class\-balanced retrieval independently selectsbbdistinct classes per step, giving both marginalsb/Cb/Cforb≤Cb\\leq C\. RPR does not preserve the history\-conditional inclusion probability, since classes already visited in the current pass are temporarily unavailable\. We use*matched marginal*only for the fixed\-set time average in Equation \([3](https://arxiv.org/html/2609.25735#S4.E3)\)\. Under class arrivals, evictions, and incomplete passes, finite\-horizon empirical frequencies may differ across retrieval policies\. The online comparisons match storage, loss, and replay size, but do not establish exact per\-class frequency equality under churn\.

The empirical frequency overTTsteps isμ^c=T−1∑t=1T𝟏\[c∈𝒞\(Bt\)\]\\hat\{\\mu\}\_\{c\}=T^\{\-1\}\\sum\_\{t=1\}^\{T\}\\mathbf\{1\}\[c\\in\\mathcal\{C\}\(B\_\{t\}\)\]\. Equal replay size fixes the total example budget but does not fix eachμ^c\\hat\{\\mu\}\_\{c\}, and a similar aggregate mean gap does not establish per\-class equality\.

Letτc​\(k\)\\tau\_\{c\}\(k\)denote thekk\-th replay appearance of classcc\. Its*rehearsal gap*and the empirical frequency of a distinct class pair are

Gc\(k\)=τc\(k\+1\)−τc\(k\),Qc​d\(N\)=1N∑t=1N𝟏\[\{c,d\}⊆𝒞\(Bt\)\]\.G\_\{c\}\(k\)=\\tau\_\{c\}\(k\+1\)\-\\tau\_\{c\}\(k\),\\qquad Q\_\{cd\}^\{\(N\)\}=\\frac\{1\}\{N\}\\sum\_\{t=1\}^\{N\}\\mathbf\{1\}\[\\\{c,d\\\}\\subseteq\\mathcal\{C\}\(B\_\{t\}\)\]\.\(4\)Neither statistic is determined byμ¯c\\bar\{\\mu\}\_\{c\}\. For i\.i\.d\. balanced retrieval on a fixed set withb<Cb<C,

Gc∼Geometric\(p\),𝔼\[Gc\]=C/b,Pr\[Gc\>g\]=\(1−p\)g,p=b/C\.G\_\{c\}\\sim\\mathrm\{Geometric\}\(p\),\\qquad\\mathbb\{E\}\[G\_\{c\}\]=C/b,\\qquad\\Pr\[G\_\{c\}\>g\]=\(1\-p\)^\{g\},\\qquad p=b/C\.\(5\)For i\.i\.d\. balanced retrieval,𝔼⁡\[Gc\]=C/b\\mathbb\{E\}\[G\_\{c\}\]=C/b, with unbounded gap support whenb<Cb<C\. Fixed class cycling gives bounded rehearsal gaps and, whenb\|Cb\\mid C, repeated class pairings within a fixed partition \(Figure[1](https://arxiv.org/html/2609.25735#S1.F1)a\)\.

### 4\.2Randomised Pass Scheduler

RPR maintains a persistent deck of resident classes not yet visited in the current pass\. Whenb≤Ctb\\leq C\_\{t\}, the scheduler removesbbclass identifiers from the deck and samples one stored example from each selected class\. When the deck empties, it is refilled with a uniform random permutation of𝒦t\\mathcal\{K\}\_\{t\}\. If a pass boundary falls within a minibatch, classes already visited in that step move to the end of the new permutation; this prevents duplicates whenb≤Ctb\\leq C\_\{t\}\. A newly resident class enters a uniformly random position among unvisited entries, and an evicted class leaves the deck\. Insertions and evictions update the current deck without restarting the pass\. The scheduler requires no information about future classes\. Ifb≥Ctb\\geq C\_\{t\}, retrieval covers every resident class\. Appendix[D](https://arxiv.org/html/2609.25735#A4)specifies the complete rule, including warm\-up and within\-class sampling\.

###### Proposition 1\(Preserved time\-averaged marginal\)\.

On a fixed resident set of sizeCCwithb≤Cb\\leq C, RPR visits each class once per pass\. Henceμ¯c=b/C\\bar\{\\mu\}\_\{c\}=b/Cand the event\-average gap is𝔼⁡\[Gc\]=C/b\\mathbb\{E\}\[G\_\{c\}\]=C/b, as under i\.i\.d\. balanced retrieval\. The history\-conditional per\-step marginals need not coincide\.

###### Proposition 2\(Bounded gap on a fixed resident set\)\.

Under the same conditions, every class and visit satisfy

Gc​\(k\)≤2​⌈Cb⌉−1\.G\_\{c\}\(k\)\\leq 2\\Big\\lceil\\frac\{C\}\{b\}\\Big\\rceil\-1\.\(6\)The bound is tight whenbbdividesCC: a class can appear in the first batch of one pass and the last batch of the next\.

###### Proof\.

Letq=⌈C/b⌉q=\\lceil C/b\\rceil, and let a pass start afteraaconsumed class slots\. Its last slot isa\+Ca\+C\. Writea\+C=h​b\+ra\+C=hb\+r, with0≤r<b0\\leq r<b\. Ifr\>0r\>0, its finalrrclasses share a batch with the next pass and move to the end of that next pass\. A class outside this deferred set therefore appears among the firstC−rC\-rslots of the next pass, no later than batchh\+qh\+q\. Its preceding appearance is no earlier than batch⌊a/b⌋\+1\\lfloor a/b\\rfloor\+1, so its gap is at mosth\+q−⌊a/b⌋−1≤2​q−1h\+q\-\\lfloor a/b\\rfloor\-1\\leq 2q\-1\. A deferred class instead appears in batchh\+1h\+1and returns no later thanh\+⌈\(C\+r\)/b⌉h\+\\lceil\(C\+r\)/b\\rceil, giving a gap at mostq≤2​q−1q\\leq 2q\-1\. Whenr=0r=0, the first case covers every class\. Every pass remains a permutation and no step repeats a class forb≤Cb\\leq C; consequently each complete pass contributes one class visit andCCslots, which also proves Proposition[1](https://arxiv.org/html/2609.25735#Thmproposition1)\. ∎

The bound does not assert tightness at other ratios\. Online storage requires a separate statement\.

###### Proposition 3\(Churn\-conditional bound\)\.

Supposeccremains resident between consecutive replay appearances\. LetCmaxC\_\{\\max\}be the largest resident count during this interval andIIthe number of class insertions into the deck\. Then

Gc​\(k\)≤⌈2​Cmax−1\+Ib⌉\.G\_\{c\}\(k\)\\leq\\Big\\lceil\\frac\{2C\_\{\\max\}\-1\+I\}\{b\}\\Big\\rceil\.\(7\)

This conservative slot\-count bound need not equal Equation \([6](https://arxiv.org/html/2609.25735#S4.E6)\) even whenI=0I=0\. Both require residency: no retrieval rule bounds the latency of a class that storage removes entirely\. We distinguish*resident gaps*, whose endpoints and intervening steps retain the class in memory, from*wall gaps*, which also include periods of eviction\. Section[5\.2](https://arxiv.org/html/2609.25735#S5.SS2.SSS0.Px2)tests the churn bound inside the online runs\.

### 4\.3Gap Bounds and Co\-occurrence

Any recurrent schedule has maximum rehearsal gap at least⌈C/b⌉\\lceil C/b\\rceil, since fewer than⌈C/b⌉\\lceil C/b\\rceilsteps cannot cover allCCclasses\. Proposition[2](https://arxiv.org/html/2609.25735#Thmproposition2)givesGc​\(k\)≤2​⌈C/b⌉−1G\_\{c\}\(k\)\\leq 2\\lceil C/b\\rceil\-1\. Whenb\|Cb\\mid C, attainingGc​\(k\)=C/bG\_\{c\}\(k\)=C/bfor every class and visit forces a fixed partition into batches: each class always shares its batch with the sameb−1b\-1classes\. This statement requires exact equality and does not extend to schedules whose gaps only approach the lower bound\.

Each RPR pass induces a new uniform partition whenb\|Cb\\mid C, with expected pair frequency

𝔼⁡\[Qc​d\]=bC​b−1C−1\(c≠d\)\.\\mathbb\{E\}\[Q\_\{cd\}\]=\\frac\{b\}\{C\}\\frac\{b\-1\}\{C\-1\}\\quad\(c\\neq d\)\.\(8\)A fixed partition restricts each class to theb−1b\-1classes in its block\. Under this divisibility condition, every distinct class pair has the co\-occurrence probability in Equation \([8](https://arxiv.org/html/2609.25735#S4.E8)\)\. This co\-occurrence result alone does not imply an improvement in predictive accuracy\.

Overnnrehearsal gaps per class, a union bound gives an i\.i\.d\. maximum\-gap scale ofO⁡\(log⁡\(n​C\)/log⁡\(1/\(1−p\)\)\)O\(\\log\(nC\)/\\log\(1/\(1\-p\)\)\)forp=b/C<1p=b/C<1\. A single\-class lower bound has the same logarithmic dependence\. The RPR bound is independent ofnn\.

Whenb\|Cb\\mid CandC/bC/bincreases, the ratio between the i\.i\.d\. and RPR gap variances approaches66, corresponding to a standard\-deviation ratio of6≈2\.449\\sqrt\{6\}\\approx 2\.449\. The four static configurations in Table[4](https://arxiv.org/html/2609.25735#A3.T4)give standard\-deviation ratios2\.412\.41,2\.422\.42,2\.382\.38, and2\.432\.43\. Appendix[E](https://arxiv.org/html/2609.25735#A5)gives the derivations and the convex gap\-cost result\. These results characterise the replay schedule and do not imply an accuracy improvement; the learning consequences depend on the loss and the evolving parameters\.

### 4\.4Interaction with ER\-ACE

We first consider replay losses that decompose over examples\.

###### Definition 1\(Composition\-blind replay\)\.

A replay term is*composition\-blind*if it decomposes over examples,

ℓre​\(θ,B\)=∑\(x,y\)∈Bϕ⁡\(θ,x,y\),\\ell^\{\\mathrm\{re\}\}\(\\theta;B\)\\;=\\;\\sum\_\{\(x,y\)\\in B\}\\phi\(\\theta;x,y\),\(9\)with per\-exampleϕ\\phiindependent of the other examples, including their labels and any joint forward computation\.

If Equation \([9](https://arxiv.org/html/2609.25735#S4.E9)\) holds then∇θℓre​\(θ,B\)=∑\(x,y\)∈B∇θϕ​\(θ,x,y\)\\nabla\_\{\\theta\}\\ell^\{\\mathrm\{re\}\}\(\\theta;B\)=\\sum\_\{\(x,y\)\\in B\}\\nabla\_\{\\theta\}\\phi\(\\theta;x,y\), so one example’s contribution is unaffected by its batchmates’ labels, and

𝔼B∼π\[∇θℓre\(θ;B\)\]=∑\(x,y\)∈MtPrπ\[\(x,y\)∈B\]∇θϕ\(θ;x,y\),\\mathbb\{E\}\_\{B\\sim\\pi\}\\big\[\\nabla\_\{\\theta\}\\ell^\{\\mathrm\{re\}\}\(\\theta;B\)\\big\]\\;=\\;\\sum\_\{\(x,y\)\\in M\_\{t\}\}\\\!\\\!\\Pr\_\{\\pi\}\\big\[\(x,y\)\\in B\\big\]\\,\\nabla\_\{\\theta\}\\phi\(\\theta;x,y\),\(10\)which depends only on the per\-*example*inclusion probabilities\. At fixed\(θ,Mt\)\(\\theta,M\_\{t\}\), two policies that match those probabilities have the same one\-step expected replay gradient\. Their full training trajectories need not coincide: class\-balanced and uniform retrieval need not match per\-example probabilities in a finite, uneven store, and temporal dependence changes the future parameters at which gradients are evaluated\.

Dark Experience Replay\([Buzzega et al\., 2020](https://arxiv.org/html/2609.25735#bib.bib4)\)satisfies Definition[1](https://arxiv.org/html/2609.25735#Thmdefinition1)for a batch\-independent forward map\. Its replay term isα​‖fθ​\(x\)−z‖22\\alpha\\,\\\|f\_\{\\theta\}\(x\)\-z\\\|\_\{2\}^\{2\}summed over the batch, a per\-example regression on stored logits that does not depend on the class label\.

ER\-ACE applies standard cross\-entropy to replay examples and masks previously observed classes that are absent fromAtA\_\{t\}in the incoming\-example softmax\([Caccia et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib12)\)\. For a seen classccabsent from both batches, the incoming term supplies no class\-cclogit gradient, while replay supplies only positive softmax contributions\. To obtain an explicit parameter\-level statement, consider a nonempty replay batch and the linear\-head classifierzc​\(x\)=wc⊤​hθ​\(x\)\+βcz\_\{c\}\(x\)=w\_\{c\}^\{\\top\}h\_\{\\theta\}\(x\)\+\\beta\_\{c\},

∂ℒt∂βc=∑x∈Btpc​\(x\)\>0\(c∈𝒮t∖\[𝒞⁡\(At\)∪𝒞⁡\(Bt\)\]\)\.\\frac\{\\partial\\mathcal\{L\}\_\{t\}\}\{\\partial\\beta\_\{c\}\}=\\sum\_\{x\\in B\_\{t\}\}p\_\{c\}\(x\)\>0\\qquad\\bigl\(c\\in\\mathcal\{S\}\_\{t\}\\setminus\[\\mathcal\{C\}\(A\_\{t\}\)\\cup\\mathcal\{C\}\(B\_\{t\}\)\]\\bigr\)\.\(11\)Herepc​\(x\)p\_\{c\}\(x\)is the replay softmax probability for classccat inputxx\. A replay gap ofggsteps hasg−1g\-1interior steps without rehearsal\. If the class also remains absent from the stream, each contributes one\-sided bias pressure; plain gradient descent decreasesβc\\beta\_\{c\}at each such step\. Gap length alone does not fix the magnitude, since the probabilities change during training\.

Equation \([11](https://arxiv.org/html/2609.25735#S4.E11)\) characterises the bias gradient and does not imply a decrease in every class\-cclogit\. Shared representations couple outputs through the empirical neural tangent kernel\([Jacot et al\., 2018](https://arxiv.org/html/2609.25735#bib.bib25)\); momentum and weight decay introduce additional terms described in Appendix[E\.3](https://arxiv.org/html/2609.25735#A5.SS3)\. The experiments measure realised bias displacement and mask–schedule interactions directly\. The proposed mechanism predicts sensitivity to long replay gaps at fixed mean rehearsal frequency\. The primary cosine classifier has no bias parameter, so Equation \([11](https://arxiv.org/html/2609.25735#S4.E11)\) applies only to the linear\-head diagnostic and does not directly explain the gains of the primary classifier\.

## 5Experiments

The experiments evaluate replay order under class\-balanced retrieval, rehearsal gaps in online runs, the linear\-head ER\-ACE mechanism, and sensitivity to replay budget, storage policy, loss, and backbone\.

### 5\.1Experimental Setup

#### Benchmarks and streams\.

The evaluation uses class\-incremental streams from CIFAR\-10 and CIFAR\-100\([Krizhevsky and Hinton, 2009](https://arxiv.org/html/2609.25735#bib.bib8)\), Tiny\-ImageNet\([Le and Yang, 2015](https://arxiv.org/html/2609.25735#bib.bib26)\), and ImageNet\-R\([Hendrycks et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib9)\)\. The CIFAR\-10 control uses a replay batch that accommodates every class once all classes are resident\. The main experiments use CIFAR\-100 and Tiny\-ImageNet with balanced streams or exponential long\-tailed streams at imbalance factorsρ∈\{10,100\}\\rho\\in\\\{10,100\\\}\. Within each task, a seeded random permutation assigns class rankrr, and the retained example count isnr=max\{1,round\(nmaxρ−r/\(Ctask−1\)\)\}n\_\{r\}=\\max\\\{1,\\operatorname\{round\}\(n\_\{\\max\}\\rho^\{\-r/\(C\_\{\\mathrm\{task\}\}\-1\)\}\)\\\}\. This is a within\-task adaptation of the standard artificial long\-tailed CIFAR construction\([Cui et al\., 2019](https://arxiv.org/html/2609.25735#bib.bib27)\)\. ImageNet\-R is used for the pretrained\-backbone evaluation\.

#### Methods and baselines\.

ER\-ACE with a randomly initialised ResNet\-18\([He et al\., 2016](https://arxiv.org/html/2609.25735#bib.bib10)\)serves as the primary learner\. We additionally examine DER and DER\+\+\([Buzzega et al\., 2020](https://arxiv.org/html/2609.25735#bib.bib4)\), and Online Continual Learning through Mutual Information Maximization \(OCM\)\([Guo et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib19)\)\. These additional losses test whether retrieval effects depend on the replay objective\. The pretrained\-backbone study uses a 224\-pixel ViT\-B/16\([Dosovitskiy et al\., 2021](https://arxiv.org/html/2609.25735#bib.bib11)\), pretrained on ImageNet\-21k and AugReg fine\-tuned on ImageNet\-1k\([Steiner et al\., 2022](https://arxiv.org/html/2609.25735#bib.bib28)\)\. We load the exactvit\_base\_patch16\_224\.augreg2\_in21k\_ft\_in1kcheckpoint from timm\([Wightman, 2019](https://arxiv.org/html/2609.25735#bib.bib29)\)\. Retrieval policies include uniform sampling, i\.i\.d\. class\-balanced sampling, fixed class cycling, MIR, and RPR\. Reservoir sampling and Balanced Reservoir Sampling provide the two storage policies in the storage–retrieval study\.

#### Evaluation\.

We report final task\-mean test accuracy and mean paired differences in percentage points\. Comparisons use ten paired random seeds and two\-sided95%95\\%Student\-ttconfidence intervals\. Full paired\-test results include counts of positive paired differences\. We define an effect as*material*when its relative magnitude exceeds2%2\\%and its confidence interval excludes zero\. Separate validation runs select the tested method and regimes without entering test comparisons\.

#### Implementation details\.

All experiments process each stream once\. The primary ResNet\-18 ER\-ACE setting uses SGD with learning rate0\.030\.03, no momentum or weight decay, incoming batch size3232, and replay batch size88\. CIFAR\-100 and Tiny\-ImageNet use ten class\-incremental tasks of ten and twenty classes, respectively, with random crops, horizontal flips, and dataset\-specific normalisation\. Task boundaries organise the stream and evaluation, but do not enter the replay rule\. Buffer capacities range from200200to5,1205\{,\}120where indicated\. The pretrained ViT setting uses a replay batch of3232examples unless the replay\-budget ablation states otherwise\. Accuracy is evaluated with an exponential moving average of the weights over a1,0241\{,\}024\-sample horizon\([Soutif–Cormerais et al\., 2023](https://arxiv.org/html/2609.25735#bib.bib16)\), held fixed within every comparison\. Section[F](https://arxiv.org/html/2609.25735#A6)reports the corresponding comparison without the EMA readout\. Appendix[D](https://arxiv.org/html/2609.25735#A4)gives the RPR pseudocode, and the remaining appendices provide complete paired statistics and additional controls\.

#### Classifier and comparison scope\.

The primary ER\-ACE accuracy study uses a normalised cosine classifier with scale1010and no bias\. The bias\-drift, mask–schedule, renewal, and buffer diagnostics use an ordinary linear classifier with bias, with the same configuration across each block\. This permits bias\-displacement measurements and limits direct mechanistic attribution to that head\. The from\-scratch accuracies characterise a one\-pass, small\-replay\-budget regime; they are not a reproduction of published ER\-ACE scores under other training protocols\. Comparisons estimate retrieval effects within each protocol, including the pretrained setting, and do not establish generality across training regimes\.

### 5\.2Results

#### Retrieval order and storage balance\.

Table 1:Storage and retrieval contrasts in final accuracy\. Entries give differences in percentage points \(relative differences in parentheses\), with95%95\\%paired confidence intervals where shown\. Storage compares BRS with reservoir at uniform retrieval; draw compares i\.i\.d\. class\-balanced with uniform retrieval at reservoir storage; both compares BRS with RPR against reservoir with uniform retrieval\. RPR comparisons specify the storage rule \(ℛ\\mathcal\{R\}: reservoir\)\. For RPR–fixed contrasts, stars denote Holm\-adjustedp<\.05/\.01/\.001p<\.05/\.01/\.001\. Full accuracies and paired tests appear in Appendices[G](https://arxiv.org/html/2609.25735#A7)and[H](https://arxiv.org/html/2609.25735#A8)\.The primary comparison replaces independent class\-balanced retrieval with RPR at fixed storage, learner, and replay budget\. Table[1](https://arxiv.org/html/2609.25735#S5.T1)reports gains of0\.720\.72–1\.671\.67percentage points under reservoir storage\. These online comparisons test the scheduling policy as a whole; Proposition[1](https://arxiv.org/html/2609.25735#Thmproposition1)does not guarantee identical finite\-run replay frequencies in their evolving memories\. Crossing storage and retrieval rules evaluates scheduling at different levels of storage imbalance\. On CIFAR\-100 LT10, BRS reduces the occupancy spread from15\.4×15\.4\\timesto1\.02×1\.02\\times\. Within BRS, RPR increases accuracy by1\.381\.38points over independent balanced retrieval, with all ten paired differences positive\. The mean RPR effect is positive under both storage rules in every tested stream\.

Fixed cycling provides a bounded\-gap comparator without permutation renewal\. RPR has higher mean accuracy in all eight storage–stream combinations, but only six intervals exclude zero; both CIFAR\-100 LT100 contrasts remain inconclusive\. All eight RPR–i\.i\.d\. contrasts and six RPR–fixed contrasts remain significant after the within\-family Holm adjustment \(Appendix[H](https://arxiv.org/html/2609.25735#A8)\)\.

#### Rehearsal gaps under changing memory\.

Telemetry evaluates the churn\-conditional bound as memory changes\. Across CIFAR\-100 LT10, Tiny\-ImageNet LT10, and Tiny\-ImageNet LT100, RPR’s mean per\-run resident maxima are23\.923\.9,48\.148\.1, and46\.346\.3steps, with no violation of the corresponding churn\-conditional bound in any run \(Appendix[I](https://arxiv.org/html/2609.25735#A9)\)\. Independent balanced draws have similar aggregate mean gaps, but mean per\-run maxima of8181–176176steps\. Similar means do not establish matched per\-class frequencies\.

![Refer to caption](https://arxiv.org/html/2609.25735v1/mechanism_summary.png)Figure 2:Temporal coverage and pretrained ViT evaluation\.a, Linear\-head ER\-ACE resident\-gap 99th percentiles and maxima on LT10 streams \(ten\-seed means\); arrows show relative reductions\.b, RPR minus indicated comparator \(percentage points;95%95\\%paired confidence intervals, ten seeds\)\. Row labels give replay batchbband buffer capacityMM; colours are centred at zero\. Table[3](https://arxiv.org/html/2609.25735#S5.T3)gives baseline gaps\.Tiny\-ImageNet LT100 has7\.47\.4class deactivations per run\. Its maximum wall gap exceeds the resident maximum because it includes eviction periods, during which the class is unavailable to any retrieval schedule\. Relative to independent balanced retrieval, mean accuracy differences in the from\-scratch settings are positive in all stream\-frequency thirds, while macro per\-class forgetting falls by1\.741\.74–2\.752\.75points \(Appendix[J](https://arxiv.org/html/2609.25735#A10)\)\.

#### Linear\-head bias displacement and mask interventions\.

Under masked linear\-head ER\-ACE, every completed joint\-absence episode in the telemetry has negative classifier\-bias displacement\. Mean displacement magnitude increases across the duration bins\. For CIFAR\-100 under independent balanced retrieval, the mean changes from−0\.00191\-0\.00191at11–44steps to−0\.00926\-0\.00926at≥33\\geq 33steps\. Tiny\-ImageNet shows the same ordering \(Appendix[K](https://arxiv.org/html/2609.25735#A11)\)\. These measured parameter changes are consistent with the accumulated bias pressure in Equation \([11](https://arxiv.org/html/2609.25735#S4.E11)\)\. The gradient\-sign result alone does not establish an accuracy effect\.

Table 2:Mask–schedule interaction in linear\-head ER\-ACE on LT10 streams\. RPR minus i\.i\.d\. accuracy \(percentage points;95%95\\%paired CI\); interaction: masked minus unmasked effect\.In the linear\-head diagnostics, RPR reduces the resident\-gap9999th percentile by about47%47\\%on both datasets \(Figure[2](https://arxiv.org/html/2609.25735#S5.F2)a\)\. With the incoming mask, RPR gains1\.231\.23and1\.161\.16points; without it, both intervals include zero\. The paired mask–schedule interactions are0\.930\.93and1\.061\.06points with intervals above zero \(Table[2](https://arxiv.org/html/2609.25735#S5.T2)\), supporting attenuation after mask removal\.

Reusing a shuffled pass for44or1616traversals reduces pair coverage and entropy, and fixed cycling reduces both further\. All these bounded schedules have higher mean accuracy than independent balanced retrieval, but accuracy is non\-monotone in pair diversity, and fixed cycling has the shortest gaps without the largest gain\. These controls do not isolate the gap tail from other temporal dependencies\.

#### Dependence on replay loss, storage, budget, and backbone\.

Loss interventions evaluate the scope of the ER\-ACE mechanism\. Removing its mask, or adding it to DER\+\+, leaves the balanced\-draw versus uniform contrast inconclusive \(Appendix[L](https://arxiv.org/html/2609.25735#A12)\)\. RPR increases accuracy by1\.021\.02and0\.870\.87points over independent balanced retrieval in masked DER\+\+ on CIFAR\-100 and Tiny\-ImageNet, respectively\. DER, DER\+\+/Refresh, and OCM yield inconclusive or dataset\-dependent results; these comparisons do not support a common trajectory\-level effect based on the loss’s one\-step form\.

Table 3:Pretrained ViT\-B/16\. Gap: i\.i\.d\. resident\-gap9999th percentile\.Δ\\Delta: RPR minus i\.i\.d\. final task\-mean accuracy \(percentage points;95%95\\%paired CI\)\.Setting\(b,M\)\(b,M\)GapΔ\\DeltaaccuracyCIFAR\-100Bal\.\(32,500\)\(32,500\)9\.00−1\.09​\[−2\.03,−0\.15\]\-1\.09\\;\[\-2\.03,\-0\.15\]Bal\.\(16,500\)\(16,500\)18\.30\+0\.30​\[−1\.14,\+1\.73\]\+0\.30\\;\[\-1\.14,\+1\.73\]Bal\.\(8,500\)\(8,500\)37\.60\+0\.66​\[−0\.27,\+1\.60\]\+0\.66\\;\[\-0\.27,\+1\.60\]LT10\(32,500\)\(32,500\)8\.00−0\.14​\[−0\.92,\+0\.64\]\-0\.14\\;\[\-0\.92,\+0\.64\]LT10\(8,500\)\(8,500\)34\.20\+2\.08​\[\+1\.14,\+3\.02\]\+2\.08\\;\[\+1\.14,\+3\.02\]LT10\(8,5120\)\(8,5120\)36\.10\+3\.51​\[\+2\.31,\+4\.71\]\+3\.51\\;\[\+2\.31,\+4\.71\]ImageNet\-RBal\.\(32,500\)\(32,500\)15\.80\+0\.22​\[−0\.76,\+1\.20\]\+0\.22\\;\[\-0\.76,\+1\.20\]Bal\.\(16,500\)\(16,500\)31\.70\+0\.48​\[−0\.27,\+1\.23\]\+0\.48\\;\[\-0\.27,\+1\.23\]Bal\.\(8,500\)\(8,500\)63\.90\+0\.85​\[−0\.33,\+2\.02\]\+0\.85\\;\[\-0\.33,\+2\.02\]LT10\(32,500\)\(32,500\)15\.60\+0\.17​\[−0\.46,\+0\.79\]\+0\.17\\;\[\-0\.46,\+0\.79\]LT10\(8,500\)\(8,500\)61\.40\+1\.53​\[\+0\.84,\+2\.23\]\+1\.53\\;\[\+0\.84,\+2\.23\]LT10\(8,5120\)\(8,5120\)67\.60\+2\.36​\[\+1\.04,\+3\.67\]\+2\.36\\;\[\+1\.04,\+3\.67\]Buffer\-capacity comparisons evaluate sensitivity to class residency\. In the CIFAR\-100 LT10 buffer sweep, class deactivations fall from16\.116\.1to zero from capacity200200to5,1205\{,\}120, while RPR shortens gaps throughout\. Its mean accuracy difference varies non\-monotonically, with endpoint estimates of0\.300\.30and1\.231\.23points \(Appendix[K](https://arxiv.org/html/2609.25735#A11)\)\. Storage controls class availability, and retrieval controls visits during residency\.

The pretrained ViT controls evaluate stream and replay\-budget dependence\. On balanced streams with buffer500500andb=32b=32, RPR changes accuracy by−1\.09\-1\.09points on CIFAR\-100 and\+0\.22\+0\.22on ImageNet\-R\. Smaller replay batches have longer gap tails, but show no material positive effect on these balanced streams \(Table[3](https://arxiv.org/html/2609.25735#S5.T3)\)\. The gap statistics summarise the evolving resident set, not a stationary full\-vocabulary process\.

On LT10 streams with the same buffer andb=8b=8, RPR increases accuracy by2\.082\.08and1\.531\.53points on the respective datasets, with both intervals above zero\. Raisingbbto3232reduces the independent\-draw gap9999th percentile from34\.234\.2to8\.08\.0steps on CIFAR\-100 and from61\.461\.4to15\.615\.6on ImageNet\-R\. The corresponding accuracy effects are−0\.14\-0\.14and\+0\.17\+0\.17points, and both intervals include zero\. Within the same ten seeds, the RPR–i\.i\.d\. effect decreases by2\.222\.22points \(95%95\\%CI\[−3\.81,−0\.63\]\[\-3\.81,\-0\.63\]\) and1\.371\.37points \(\[−2\.01,−0\.72\]\[\-2\.01,\-0\.72\]\)\. Mean accuracy across the ten task\-boundary evaluations shows the same small\- versus large\-batch pattern \(Appendix[M](https://arxiv.org/html/2609.25735#A13)\)\.

Pretrained effects depend on stream and replay budget\. Three of four LT10 ViT contrasts with fixed cycling include zero; the fourth is strictly positive before rounding \(Figure[2](https://arxiv.org/html/2609.25735#S5.F2)b\)\. These controls establish neither small batches nor imbalance as necessary\. Readout and MIR controls appear in Appendices[F](https://arxiv.org/html/2609.25735#A6)and[N](https://arxiv.org/html/2609.25735#A14)\.

## 6Conclusion and Limitations

Class\-balanced retrieval does not determine replay spacing\. On a fixed resident set, RPR bounds rehearsal gaps and renews class co\-occurrence without additional replay examples or forward passes\. It improves final accuracy across the primary ER\-ACE storage settings, while pretrained gains concentrate in the tested LT10 small\-batch settings\. Linear\-head diagnostics are consistent with accumulated bias pressure during joint absence, but the controls do not isolate the gap tail\. Marginal preservation under churn remains unestablished, and effects depend on the learner and regime\.

These conclusions have a defined scope\. The fixed\-set gap bound is vacuous whenb≥Cb\\geq Cand is conditional on storage under churn\. The CIFAR\-10 control atb=32b=32, balanced pretrained streams, and matched LT10 controls atb=32b=32show no material positive gain\. Evidence covers single\-pass CIFAR\-100, Tiny\-ImageNet, and ImageNet\-R, not multi\-epoch, offline, or ImageNet\-scale continual learning\. Reused\-pass and fixed\-cycle controls change gaps and co\-occurrence jointly, and the linear\-head bias diagnostic does not fully explain cosine\-head or cross\-host results\.

## 7AI Use Statement

We use AI as auxiliary tools for linguistic refinement, restructuring and formatting text and tables, and assistance with experiment orchestration, analysis, and verification code\. The authors determine the research questions, methodology, experimental design, statistical criteria, and interpretation; they review generated code and recompute every reported quantity from per\-run records\. AI output does not constitute evidence and requires author verification\. The authors retain all decisions about content, wording, analysis, and presentation\.

## References

- Aljundiet al\.\(2019a\)R\. Aljundi, E\. Belilovsky, T\. Tuytelaars, L\. Charlin, M\. Caccia, M\. Lin, and L\. Page\-CacciaOnline continual learning with maximal interfered retrieval\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\. 11849–11860\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Aljundiet al\.\(2019b\)R\. Aljundi, M\. Lin, B\. Goujaud, and Y\. BengioGradient based sample selection for online continual learning\.Advances in neural information processing systems32\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Buzzegaet al\.\(2020\)P\. Buzzega, M\. Boschini, A\. Porrello, D\. Abati, and S\. CalderaraDark experience for general continual learning: a strong, simple baseline\.InAdvances in Neural Information Processing Systems,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. F\. Balcan, and H\. Lin \(Eds\.\),Vol\.33,pp\. 15920–15930\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px3.p1.1),[§4\.4](https://arxiv.org/html/2609.25735#S4.SS4.p3.1),[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px2.p1.1)\.
- Buzzegaet al\.\(2021\)P\. Buzzega, M\. Boschini, A\. Porrello, and S\. CalderaraRethinking experience replay: a bag of tricks for continual learning\.In25th International Conference on Pattern Recognition \(ICPR\),pp\. 2180–2187\.Cited by:[§1](https://arxiv.org/html/2609.25735#S1.p1.1),[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Cacciaet al\.\(2022\)L\. Caccia, R\. Aljundi, N\. Asadi, T\. Tuytelaars, J\. Pineau, and E\. BelilovskyNew insights on reducing abrupt representation change in online continual learning\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.25735#S1.p5.1),[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px3.p1.1),[§4\.4](https://arxiv.org/html/2609.25735#S4.SS4.p4.1)\.
- Chaudhryet al\.\(2019\)A\. Chaudhry, M\. Rohrbach, M\. Elhoseiny, T\. Ajanthan, P\. K\. Dokania, P\. H\. S\. Torr, and M\. RanzatoOn tiny episodic memories in continual learning\.External Links:1902\.10486,[Link](https://arxiv.org/abs/1902.10486)Cited by:[§1](https://arxiv.org/html/2609.25735#S1.p1.1),[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Chrysakis and Moens \(2020\)A\. Chrysakis and M\. MoensOnline continual learning from imbalanced data\.InProceedings of the 37th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.119,pp\. 1952–1961\.Cited by:[§1](https://arxiv.org/html/2609.25735#S1.p1.1),[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Cuiet al\.\(2019\)Y\. Cui, M\. Jia, T\. Lin, Y\. Song, and S\. BelongieClass\-balanced loss based on effective number of samples\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 9268–9277\.Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px1.p1.1)\.
- Dosovitskiyet al\.\(2021\)A\. Dosovitskiy, L\. Beyer, A\. Kolesnikov, D\. Weissenborn, X\. Zhai, T\. Unterthiner, M\. Dehghani, M\. Minderer, G\. Heigold, S\. Gelly, J\. Uszkoreit, and N\. HoulsbyAn image is worth 16x16 words: transformers for image recognition at scale\.External Links:2010\.11929,[Link](https://arxiv.org/abs/2010.11929)Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px2.p1.1)\.
- Guoet al\.\(2022\)Y\. Guo, B\. Liu, and D\. ZhaoOnline continual learning through mutual information maximization\.InProceedings of the 39th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.162,pp\. 8109–8126\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px3.p1.1),[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px2.p1.1)\.
- Gürbüzbalabanet al\.\(2021\)M\. Gürbüzbalaban, A\. Ozdaglar, and P\. A\. ParriloWhy random reshuffling beats stochastic gradient descent\.SIAM Journal on Optimization31\(3\),pp\. 1880–1915\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px2.p1.1)\.
- HaoChen and Sra \(2019\)J\. Z\. HaoChen and S\. SraRandom shuffling beats SGD after finite epochs\.InProceedings of the 36th International Conference on Machine Learning,Vol\.97,pp\. 2624–2633\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px2.p1.1)\.
- Heet al\.\(2016\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.InProc\. of IEEE Conf\. on Computer Vision and Pattern Recognition \(CVPR\),pp\. 770–778\.Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px2.p1.1)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, S\. Basart, N\. Mu, S\. Kadavath, F\. Wang, E\. Dorundo, R\. Desai, T\. Zhu, S\. Parajuli, M\. Guo, D\. Song, J\. Steinhardt, and J\. GilmerThe many faces of robustness: a critical analysis of out\-of\-distribution generalization\.ICCV\.Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px1.p1.1)\.
- Hickok and Kudithipudi \(2024\)T\. Hickok and D\. KudithipudiWatch your step: optimal retrieval for continual learning at scale\.Note:Presented at the CLVision Workshop at CVPR 2024External Links:2404\.10758Cited by:[§1](https://arxiv.org/html/2609.25735#S1.p3.1),[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px2.p1.1)\.
- Jacotet al\.\(2018\)A\. Jacot, F\. Gabriel, and C\. HonglerNeural tangent kernel: convergence and generalization in neural networks\.InAdvances in Neural Information Processing Systems,Vol\.31\.Cited by:[§E\.3](https://arxiv.org/html/2609.25735#A5.SS3.p1.2),[§4\.4](https://arxiv.org/html/2609.25735#S4.SS4.p5.1)\.
- Klassonet al\.\(2023\)M\. Klasson, H\. Kjellström, and C\. ZhangLearn the time to learn: replay scheduling in continual learning\.Transactions on Machine Learning Research\.External Links:ISSN 2835\-8856,[Link](https://openreview.net/forum?id=Q4aAITDgdP)Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px2.p1.1)\.
- Krizhevsky and Hinton \(2009\)A\. Krizhevsky and G\. HintonLearning multiple layers of features from tiny images\.Technical reportUniversity of Toronto\.Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px1.p1.1)\.
- Le and Yang \(2015\)Y\. Le and X\. YangTiny imagenet visual recognition challenge\.Note:Stanford CS231N project reportExternal Links:[Link](https://cs231n.stanford.edu/reports/2015/pdfs/yle_project.pdf)Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px1.p1.1)\.
- Maiet al\.\(2021\)Z\. Mai, R\. Li, H\. Kim, and S\. SannerSupervised contrastive replay: revisiting the nearest class mean classifier in online class\-incremental continual learning\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,pp\. 3589–3599\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px3.p1.1)\.
- Mishchenkoet al\.\(2022\)K\. Mishchenko, A\. Khaled, and P\. RichtárikProximal and federated random reshuffling\.InProceedings of the 39th International Conference on Machine Learning,Vol\.162,pp\. 15718–15749\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px2.p1.1)\.
- Shimet al\.\(2021\)D\. Shim, Z\. Mai, J\. Jeong, S\. Sanner, H\. Kim, and J\. JangOnline class\-incremental continual learning with adversarial shapley value\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.35,pp\. 9630–9638\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Soutif–Cormeraiset al\.\(2023\)A\. Soutif–Cormerais, A\. Carta, and J\. van de WeijerImproving online continual learning performance and stability with temporal ensembles\.InProceedings of the 2nd Conference on Lifelong Learning Agents,Proceedings of Machine Learning Research, Vol\.232,pp\. 828–845\.Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px4.p1.1)\.
- Steineret al\.\(2022\)A\. Steiner, A\. Kolesnikov, X\. Zhai, R\. Wightman, J\. Uszkoreit, and L\. BeyerHow to train your ViT? data, augmentation, and regularization in vision transformers\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=4nPswr1KcP)Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px2.p1.1)\.
- Sunet al\.\(2024\)G\. Sun, B\. Ji, L\. Liang, and M\. ChenCeCR: cross\-entropy contrastive replay for online class\-incremental continual learning\.Neural Networks173,pp\. 106163\.External Links:[Document](https://dx.doi.org/10.1016/j.neunet.2024.106163)Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Sunet al\.\(2022\)S\. Sun, D\. Calandriello, H\. Hu, A\. Li, and M\. TitsiasInformation\-theoretic online memory selection for continual learning\.InInternational Conference on Learning Representations,Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Vitter \(1985\)J\. S\. VitterRandom sampling with a reservoir\.ACM Trans\. Math\. Softw\.11\(1\),pp\. 37–57\.External Links:ISSN 0098\-3500,[Link](https://doi.org/10.1145/3147.3165),[Document](https://dx.doi.org/10.1145/3147.3165)Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px1.p1.1)\.
- Weiet al\.\(2023\)Y\. Wei, J\. Ye, Z\. Huang, J\. Zhang, and H\. ShanOnline prototype learning for online continual learning\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 18764–18774\.Cited by:[§2](https://arxiv.org/html/2609.25735#S2.SS0.SSS0.Px3.p1.1)\.
- Wightman \(2019\)R\. WightmanPyTorch image models\.GitHub\.Note:[https://github\.com/huggingface/pytorch\-image\-models](https://github.com/huggingface/pytorch-image-models)External Links:[Document](https://dx.doi.org/10.5281/zenodo.4414861)Cited by:[§5\.1](https://arxiv.org/html/2609.25735#S5.SS1.SSS0.Px2.p1.1)\.

## Appendix

## Appendix AReproducibility Statement

Every experiment records its configuration, command line, random seed, evaluation split, and campaign identifier\. The256256validation runs comprise9696hybrid\-schedule runs,9696replay\-scope runs, and6464stream\-regime runs; none contributes to a test contrast\. EMA and live\-readout arms are separate training runs rather than repeated evaluations of one checkpoint\. For the long\-tailed construction,r∈\{0,…,Ctask−1\}r\\in\\\{0,\\ldots,C\_\{\\mathrm\{task\}\}\-1\\\},nmaxn\_\{\\max\}is the largest original class count in that task, androunduses ties\-to\-even rounding\. The permutation seed is the run’s imbalance seed plus10091009times the zero\-based task index\. The two hybrid candidates assign1616or2424of3232replay slots to RPR and fill the remainder by an independent balanced draw\. The release includes the scheduler, integrations for each continual\-learning method, experiment configurations, and analysis scripts\. Test comparisons use paired seeds, while validation runs remain separate from test evaluation\. The replay\-budget extension reuses the ten LT10 seeds to estimate a pairedb=32b=32minusb=8b=8interaction and is not treated as an independent confirmation block\. Unit tests cover the scheduler and its limiting cases\.

## Appendix BEthics Statement

This work trains image classifiers on CIFAR\-10, CIFAR\-100, Tiny\-ImageNet, and ImageNet\-R, all established public benchmarks\. We collect no new data, recruit no human subjects, and use no annotations about personal identity\. As with other image benchmarks, these datasets may reflect biases in their source data\. The methods in this study change which classes a learner rehearses, and Section[5\.2](https://arxiv.org/html/2609.25735#S5.SS2.SSS0.Px4)shows that their effects depend on the stream and replay budget\. Rare classes can benefit in the from\-scratch long\-tailed settings but can also lose accuracy in a pretrained control\. Retrieval policies of this kind should therefore be evaluated per class rather than only by aggregate accuracy when class\-level disparities matter\.

## Appendix CMeasured rehearsal gaps

Table[4](https://arxiv.org/html/2609.25735#A3.T4)isolates the scheduling statistics from learning by holding the resident class set and balanced memory fixed for3,0003\{,\}000steps\. The comparison between class\-balanced retrieval \(cb\) and RPR holds the long\-run class\-visit marginal fixed; uniform example retrieval \(uni\) provides an additional reference\. The balanced draw takes a new uniform random class permutation each step, whereas RPR follows Algorithm[1](https://arxiv.org/html/2609.25735#alg1)\. Uniform retrieval samples example slots and can select multiple examples of the same class within a batch, so its class\-inclusion probability need not match that of the two class\-based policies\. The mean describes how often a class returns, whereas the upper quantiles and maximum describe the long absences that the mean alone leaves unconstrained\.

Table 4:Rehearsal\-gap distributions on a fixed balanced memory\. All gaps are in steps\.Across all four configurations, RPR andcbhave nearly identical mean gaps, but RPR approximately halves the9999th percentile\. For example, atC=200C=200andb=8b=8, the mean changes from24\.8224\.82to25\.0025\.00steps while the maximum falls from263263to4949\. Each RPR maximum equals the static bound in Proposition[2](https://arxiv.org/html/2609.25735#Thmproposition2)\. These measurements illustrate the intended change in the gap distribution; they do not by themselves establish an accuracy gain\. Appendix[I](https://arxiv.org/html/2609.25735#A9)tests the corresponding guarantee when the resident set changes during learning\.

## Appendix DThe schedule in pseudocode

Algorithm 1Randomised\-pass replay as used in the experiments\. The class deck persists across calls; example lists are freshly shuffled at each replay step\. The resident set is read from the current memory; no future class identities or total class count are given to the algorithm\.1:state:deck

D←\[\]D\\leftarrow\[\\,\], known classes

K←∅K\\leftarrow\\emptyset
2:input:memory

MM, requested replay size

bb
3:

n←min⁡\(b,\|M\|\)n\\leftarrow\\min\(b,\|M\|\);

B←\[\]B\\leftarrow\[\\,\]
4:if

n=0n=0then

5:return

BB
6:endif

7:

𝒦←𝒞⁡\(M\)\\mathcal\{K\}\\leftarrow\\mathcal\{C\}\(M\)
8:

D←\[c∈D:c∈𝒦\]D\\leftarrow\[\\,c\\in D:c\\in\\mathcal\{K\}\\,\]
9:for all

c∈𝒦∖Kc\\in\\mathcal\{K\}\\setminus Kdo

10:insert

ccinto

DDat a uniformly random position

11:endfor

12:

K←𝒦K\\leftarrow\\mathcal\{K\}
13:for all

c∈𝒦c\\in\\mathcal\{K\}do

14:

Hc←H\_\{c\}\\leftarrowuniformly shuffled list of stored example indices of class

cc
15:endfor

16:

g←0g\\leftarrow 0
17:while

\|B\|<n\|B\|<nand

g≤\|M\|g\\leq\|M\|do

18:

V←ClassSlots⁡\(𝒦,n−\|B\|,D\)V\\leftarrow\\operatorname\{ClassSlots\}\(\\mathcal\{K\},n\-\|B\|,D\);

g←g\+1g\\leftarrow g\+1
19:for all

c∈Vc\\in Vdo

20:if

Hc≠\[\]H\_\{c\}\\neq\[\\,\]and

\|B\|<n\|B\|<nthen

21:pop one index from

HcH\_\{c\}and append it to

BB
22:endif

23:endfor

24:endwhile

25:returnthe examples indexed by

BB

ClassSlots\(𝒦,k,D\)\(\\mathcal\{K\},k,D\)\.LetC=\|𝒦\|C=\|\\mathcal\{K\}\|andq=⌊k/C⌋q=\\lfloor k/C\\rfloor\. First outputqqcomplete, independently shuffled class permutations, without advancingDD\. For the remainingk−q​Ck\-qCslots, remove class identifiers fromDD\. WheneverDDempties, refill it with a uniformly shuffled permutation of𝒦\\mathcal\{K\}, moving classes already output in this partial round to the end while preserving their relative order\. Persist the remaining deck for the next call\. Ifk<Ck<C, only this persistent\-deck part operates\. Ifk≥Ck\\geq C, the complete rounds ensure that every resident class is visited before the scheduled remainder\. Removing example indices fromHcH\_\{c\}samples without replacement within a replay batch; the same stored example can return at a later step\. The bounded loop matches the implementation’s guard against non\-progress and may return fewer thannnexamples when a large\-batch request repeatedly visits classes whose within\-step lists are exhausted\. This edge case is outside the fixed\-set analysis below, which assumesb≤Cb\\leq Cand enough stored examples for every requested class slot\.

## Appendix EAdditional Analysis and Proofs

We denote RPR byπrp\\pi\_\{\\mathrm\{rp\}\}in the following derivations\.

### E\.1Static Schedule Diagnostics

Table[4](https://arxiv.org/html/2609.25735#A3.T4)reports the static schedule diagnostics\. Across the four configurations, the absolute deviation of the mean gap fromC/bC/bis at most0\.0050\.005for RPR and0\.180\.18for the i\.i\.d\. balanced draw\. The corresponding i\.i\.d\.\-to\-RPR ratios for the9999th\-percentile gap are2\.02\.0–2\.52\.5, and the ratios for the observed maximum are4\.74\.7–5\.45\.4\. In all four configurations, the observed RPR maximum equals2​⌈C/b⌉−12\\lceil C/b\\rceil\-1\.

### E\.2Composition Dependence and the ER\-ACE Mask

The per\-step marginal corresponding to Equation \([3](https://arxiv.org/html/2609.25735#S4.E3)\) is

μc\(t\)​\(π\)=Pr⁡\[c∈𝒞⁡\(Bt\)∣ℱt\],c∈𝒦t\.\\mu\_\{c\}^\{\(t\)\}\(\\pi\)=\\Pr\[c\\in\\mathcal\{C\}\(B\_\{t\}\)\\mid\\mathcal\{F\}\_\{t\}\],\\qquad c\\in\\mathcal\{K\}\_\{t\}\.\(12\)Equality of time\-averaged marginals does not imply equality of these history\-conditional laws\.

Definition[1](https://arxiv.org/html/2609.25735#Thmdefinition1)and Equation \([10](https://arxiv.org/html/2609.25735#S4.E10)\) characterise the one\-step replay gradient\. ER\-ACE introduces an additional asymmetry through the incoming loss\. The from\-scratch ResNet\-18 uses training\-mode BatchNorm, so its forward map is batch\-dependent and Equation \([10](https://arxiv.org/html/2609.25735#S4.E10)\) is an idealised scope statement for that configuration\. The ViT instead uses per\-example LayerNorm\.

ER\-ACE’s incoming loss retains current\-batch and unseen output classes:

ℓon\(θ;At\)=∑\(x,y\)∈At−logexp⁡zy​\(x\)∑c∈𝒞⁡\(At\)∪𝒰texp⁡zc​\(x\),𝒰t=𝒴∖𝒮t\.\\ell^\{\\mathrm\{on\}\}\(\\theta;A\_\{t\}\)=\\sum\_\{\(x,y\)\\in A\_\{t\}\}\-\\log\\frac\{\\exp z\_\{y\}\(x\)\}\{\\sum\_\{c\\in\\mathcal\{C\}\(A\_\{t\}\)\\cup\\mathcal\{U\}\_\{t\}\}\\exp z\_\{c\}\(x\)\},\\qquad\\mathcal\{U\}\_\{t\}=\\mathcal\{Y\}\\setminus\\mathcal\{S\}\_\{t\}\.\(13\)Here𝒴\\mathcal\{Y\}denotes the host classifier’s output vocabulary; RPR does not inspect that vocabulary or use unseen labels to construct its deck\. Forc∈𝒮t∖𝒞⁡\(At\)c\\in\\mathcal\{S\}\_\{t\}\\setminus\\mathcal\{C\}\(A\_\{t\}\),

∂ℓon∂zc=0\.\\frac\{\\partial\\ell^\{\\mathrm\{on\}\}\}\{\\partial z\_\{c\}\}=0\.\(14\)For a resident replay gap ofg\>1g\>1starting att=τc​\(k\)t=\\tau\_\{c\}\(k\), ifccis also absent from every interior incoming batch, Equation \([11](https://arxiv.org/html/2609.25735#S4.E11)\) gives

∑s=1g−1∂ℒt\+s∂βc=∑s=1g−1∑x∈Bt\+spc​\(x\)\>0\.\\sum\_\{s=1\}^\{g\-1\}\\frac\{\\partial\\mathcal\{L\}\_\{t\+s\}\}\{\\partial\\beta\_\{c\}\}=\\sum\_\{s=1\}^\{g\-1\}\\sum\_\{x\\in B\_\{t\+s\}\}p\_\{c\}\(x\)\>0\.\(15\)There areg−1g\-1interior steps, since the final endpoint is a replay visit\. The sum characterises accumulated bias\-gradient pressure\. Its magnitude is not determined by gap length alone, and Equation \([15](https://arxiv.org/html/2609.25735#A5.E15)\) does not imply a universal relationship between gap length and final accuracy\.

### E\.3From Logit Gradients to Parameter Updates

The logit\-space gradient underlying Equation \([11](https://arxiv.org/html/2609.25735#S4.E11)\) is∂ℓre/∂zc​\(x\)=pc​\(x\)\>0\\partial\\ell^\{\\mathrm\{re\}\}/\\partial z\_\{c\}\(x\)=p\_\{c\}\(x\)\>0at each replay input during joint absence\. The sign of∂ℓre/∂zc​\(x\)\\partial\\ell^\{\\mathrm\{re\}\}/\\partial z\_\{c\}\(x\)does not determine the sign of the parameter\-induced change inzc​\(x′\)z\_\{c\}\(x^\{\\prime\}\)\. Under a gradient step onθ\\thetathe induced logit movement at an arbitrary inputx′x^\{\\prime\}is, to first order,

Δ​zc​\(x′\)≈−η​Jc​\(x′\)⊤​∇θℒ,Jc​\(x′\)=∇θzc​\(x′\),\\Delta z\_\{c\}\(x^\{\\prime\}\)\\;\\approx\\;\-\\,\\eta\\,J\_\{c\}\(x^\{\\prime\}\)^\{\\\!\\top\}\\nabla\_\{\\theta\}\\mathcal\{L\},\\qquad J\_\{c\}\(x^\{\\prime\}\)=\\nabla\_\{\\theta\}z\_\{c\}\(x^\{\\prime\}\),\(16\)so a positive∂ℓ/∂zc\\partial\\ell/\\partial z\_\{c\}at the replayed points does not by itself fix the sign ofΔ​zc​\(x′\)\\Delta z\_\{c\}\(x^\{\\prime\}\)at other points: the shared representation couples classes and inputs throughJc​\(x′\)⊤​Jc′​\(x\)J\_\{c\}\(x^\{\\prime\}\)^\{\\\!\\top\}J\_\{c^\{\\prime\}\}\(x\), the empirical neural tangent kernel\([Jacot et al\., 2018](https://arxiv.org/html/2609.25735#bib.bib25)\)\. For the class\-specific parameters of the final linear layer, the bias gradient has a fixed sign during joint absence\. Writingzc​\(x\)=wc⊤​hθ​\(x\)\+βcz\_\{c\}\(x\)=w\_\{c\}^\{\\\!\\top\}h\_\{\\theta\}\(x\)\+\\beta\_\{c\}, the bias gradient is∂ℒ/∂βc=∑x∈Btpc​\(x\)\>0\\partial\\mathcal\{L\}/\\partial\\beta\_\{c\}=\\sum\_\{x\\in B\_\{t\}\}p\_\{c\}\(x\)\>0at every step of joint absence\. Under plain gradient descent, the bias update is negative at each such step\. Momentum, adaptive preconditioning, and weight decay add terms that depend on past gradients, the preconditioner, and the current parameter value; the gradient sign alone does not establish an optimizer\-independent update sign\. At a single plain gradient step, the update ofwcw\_\{c\}likewise has non\-positive projection along its current gradient, but that direction can change across steps\. The shared\-trunk contribution remains sign\-indefinite\. Section[5\.2](https://arxiv.org/html/2609.25735#S5.SS2.SSS0.Px3)measures the realised bias displacement\.

### E\.4Proof of the Churn\-Conditional Bound

We prove Proposition[3](https://arxiv.org/html/2609.25735#Thmproposition3), the counterpart of Equation \([6](https://arxiv.org/html/2609.25735#S4.E6)\) for a resident class set that changes over time\.

###### Proof of Proposition[3](https://arxiv.org/html/2609.25735#Thmproposition3)\.

Count from the end of stepτc​\(k\)\\tau\_\{c\}\(k\), rather than from the position ofccinside its minibatch\. If the current pass still containscc, at mostCmaxC\_\{\\max\}existing slots remain up to its visit\. Otherwise, at mostCmax−1C\_\{\\max\}\-1existing slots remain in that pass, followed by at mostCmaxC\_\{\\max\}slots up toccin the next pass\. An insertion adds at most one further slot before this visit; deletions remove slots\. Hence at most2​Cmax−1\+I2C\_\{\\max\}\-1\+Islots remain\. A refill within a later step cannot deferccunless that step already visits it, in which case the waiting interval ends\. Each intervening step either suppliesbbdistinct class slots or, when fewer thanbbclasses remain resident, visits every resident class and hencecc\. Since the count starts at a step boundary, at most⌈\(2​Cmax−1\+I\)/b⌉\\lceil\(2C\_\{\\max\}\-1\+I\)/b\\rceilfurther steps are required\. ∎

The bound requiresccto remain resident\. Ifc∉𝒦tc\\notin\\mathcal\{K\}\_\{t\}during an interval, no retrieval policy can include it inBtB\_\{t\}, so no retrieval rule can bound the corresponding wall gap\.

### E\.5Additional Scheduling Results

The following results give a worst\-case lower bound, characterise schedules that attain the lower bound exactly whenb\|Cb\\mid C, and describe the stream\-length dependence of the maximum rehearsal gap\. A final result gives a convex cost bound at fixed mean rehearsal gap\.

###### Proposition 4\(Worst\-case gap lower bound\)\.

For any recurrent retrieval policy on a fixed resident set,

max⁡supkc⁡Gc​\(k\)≥⌈Cb⌉\.\\max\_\{c\}\\ \\sup\_\{k\}G\_\{c\}\(k\)\\ \\geq\\ \\Big\\lceil\\tfrac\{C\}\{b\}\\Big\\rceil\.\(17\)

###### Proof\.

A window ofwwconsecutive steps contains at mostb​wbwreplay class slots and therefore at mostb​wbwdistinct classes\. Takew=⌈C/b⌉−1w=\\lceil C/b\\rceil\-1; thenb​w<Cbw<Cwhether or notbbdividesCC, so some classccis absent from the whole window\. The adjacent replay visits tocclie on opposite sides of the window, givingGc≥w\+1G\_\{c\}\\geq w\+1\. ∎

Combining Proposition[4](https://arxiv.org/html/2609.25735#Thmproposition4)with Proposition[2](https://arxiv.org/html/2609.25735#Thmproposition2)gives an upper bound of2−1/⌈C/b⌉2\-1/\\lceil C/b\\rceilon RPR’s worst\-case ratio to the lower bound\. The geometric gap distribution in Equation \([5](https://arxiv.org/html/2609.25735#S4.E5)\) has unbounded support\. The next proposition characterises schedules that attain the lower bound exactly whenb\|Cb\\mid C\.

###### Proposition 5\(Exact lower\-bound attainment implies a fixed partition\)\.

On a fixed resident set ofCCclasses, suppose a bi\-infinite schedule visits exactlybbdistinct classes per step,b\|Cb\\mid C, and attainsGc​\(k\)=C/bG\_\{c\}\(k\)=C/bfor every successive pair of visits\. Then there is a fixed partition of the resident class set intoC/bC/bblocks of sizebbsuch that the replay batch’s class set is always one block\. Two classes in the same block co\-occur at every visit; two classes in different blocks never co\-occur\.

###### Proof\.

Writem=C/bm=C/b\. A uniform gap ofmmmeans classccis visited exactly at the steps congruent to some fixedϕ⁡\(c\)\\phi\(c\)modulomm\. Hence the class set at stepttis\{c:ϕ⁡\(c\)≡t\}\\\{c:\\phi\(c\)\\equiv t\\\}, which depends onttonly throughtmodmt\\bmod mand partitions the resident class set into themmlevel sets ofϕ\\phi, each necessarily of sizebbsince every step visitsbbclasses\. ∎

For a batch\-coupled loss, the fixed partition restricts classccto co\-occurrence with theb−1b\-1other classes in its block\. Proposition[5](https://arxiv.org/html/2609.25735#Thmproposition5)applies only whenGc​\(k\)=C/bG\_\{c\}\(k\)=C/bfor every class and every visit; it does not characterise schedules whose gaps are only close to the lower bound\. RPR samples a new class permutation each pass, whereas deterministic class cycling induces the fixed partition whenb\|Cb\\mid C\.

For Equation \([8](https://arxiv.org/html/2609.25735#S4.E8)\), condition on the position ofccin a uniformly permuted pass\. Exactlyb−1b\-1of the otherC−1C\-1positions share its replay batch, soddshares the replay batch withccwith probability\(b−1\)/\(C−1\)\(b\-1\)/\(C\-1\)\. That batch occupies one of theC/bC/bsteps in the pass, giving the additional factorb/Cb/Cin the time\-averaged pair frequency\.

###### Proposition 6\(Stream\-length scaling of the maximum gap\)\.

On a fixed resident set withb<Cb<C, over the firstnnrehearsal gaps of each class, the i\.i\.d\. balanced draw satisfies

𝔼⁡\[maxc,k≤n⁡Gc​\(k\)\]=O⁡\(log⁡\(n​C\)log⁡\(1/\(1−p\)\)\),p=bC,\\mathbb\{E\}\\Big\[\\max\_\{c,k\\leq n\}G\_\{c\}\(k\)\\Big\]\\;=\\;O\\\!\\left\(\\frac\{\\log\(nC\)\}\{\\log\\\!\\big\(1/\(1\-p\)\\big\)\}\\right\),\\qquad p=\\tfrac\{b\}\{C\},\(18\)and the maximum over thennindependent gaps of any fixed class isΩ⁡\(log⁡n/log⁡\(1/\(1−p\)\)\)\\Omega\(\\log n/\\log\(1/\(1\-p\)\)\)\. The i\.i\.d\. maximum therefore has logarithmic dependence onnn, whereasπrp\\pi\_\{\\mathrm\{rp\}\}satisfiesmaxc,k⁡Gc​\(k\)≤2​⌈C/b⌉−1\\max\_\{c,k\}G\_\{c\}\(k\)\\leq 2\\lceil C/b\\rceil\-1independently ofnn\. Whenb\|Cb\\mid C, asm=C/b→∞m=C/b\\to\\infty,

Var⁡\[Gciid\]Var⁡\[Gcrp\]=\(1−p\)/p2\(m2−1\)/6⟶6,\\frac\{\\operatorname\{Var\}\\big\[G\_\{c\}^\{\\mathrm\{iid\}\}\\big\]\}\{\\operatorname\{Var\}\\big\[G\_\{c\}^\{\\mathrm\{rp\}\}\\big\]\}\\;=\\;\\frac\{\(1\-p\)/p^\{2\}\}\{\(m^\{2\}\-1\)/6\}\\;\\longrightarrow\\;6,\(19\)so the limiting ratio of standard deviations is6≈2\.449\\sqrt\{6\}\\approx 2\.449\.

The upper bound in Equation \([18](https://arxiv.org/html/2609.25735#A5.E18)\) follows by a union bound overn​CnCgeometric tails and does not assume independence across classes; the lower bound uses the renewal gaps of one class\. Equation \([19](https://arxiv.org/html/2609.25735#A5.E19)\) follows fromGcrp=m−i\+jG\_\{c\}^\{\\mathrm\{rp\}\}=m\-i\+jwithi,ji,jthe batch positions ofccin two consecutive passes, which are independent and uniform on\{1,…,m\}\\\{1,\\dots,m\\\}whenb\|Cb\\mid C\.

For the four configurations, the geometric extreme\-value scales are121121,3030,247247, and6666, respectively; the measured maxima are117117,3737,263263, and6464\. The measured standard\-deviation ratios are2\.412\.41,2\.422\.42,2\.382\.38, and2\.432\.43\. The finite\-mmvariance expression is exact whenbbdividesCC; elsewhere a partial pass changes the finite\-sample distribution, so6\\sqrt\{6\}is an asymptotic reference rather than an exact prediction\.

Proposition[6](https://arxiv.org/html/2609.25735#Thmproposition6)predicts logarithmic growth of the i\.i\.d\. maximum gap with the number of draws, while the RPR bound is independent of stream length\. The proposition concerns the scheduling statistic and does not imply a monotone relationship with accuracy\.

The following proposition considers a convex cost functional over rehearsal gaps at fixed mean gap\. A balanced long\-run class marginal gives𝔼⁡\[Gc\]=C/b\\mathbb\{E\}\[G\_\{c\}\]=C/bonce finite boundary effects vanish\.

###### Proposition 7\(Jensen bound for convex gap cost\)\.

Leth:ℝ≥0→ℝ≥0h:\\mathbb\{R\}\_\{\\geq 0\}\\to\\mathbb\{R\}\_\{\\geq 0\}be non\-decreasing and convex, modelling the cost of a rehearsal gap ofggsteps\. Consider recurrent retrieval policies for which the event\-average gap exists and satisfies𝔼⁡\[Gc\]=C/b\\mathbb\{E\}\[G\_\{c\}\]=C/b\. Their expected cost obeys

𝔼⁡\[h⁡\(Gc\)\]≥h⁡\(C/b\),\\mathbb\{E\}\\big\[h\(G\_\{c\}\)\\big\]\\ \\geq\\ h\\big\(C/b\\big\),\(20\)Ifhhis strictly convex on the convex hull of the support ofGcG\_\{c\}, equality holds if and only ifGcG\_\{c\}is almost surely constant\. Under mere convexity, nonconstant gaps can also attain equality whenhhis affine on their support\.

Equation \([20](https://arxiv.org/html/2609.25735#A5.E20)\) is Jensen’s inequality and is conditional on the choice ofhh\. It does not assume that accuracy is convex in the rehearsal gap\. Policies with the same marginal may also differ in temporal dependence and class co\-occurrence; the proposition isolates gap dispersion within the specified cost model\.

## Appendix FRobustness to the evaluation\-time weight average

We compare separately trained arms with and without the averaged readout of Section[5\.1](https://arxiv.org/html/2609.25735#S5.SS1)to assess its interaction with retrieval\. At buffer500500, the320320\-run block crosses four dataset–stream cells with live and EMA readouts under reservoir storage and replay batch3232\. Removing the exponential moving average \(EMA\) increases the relative effect of the balanced draw from\+6\.5\+6\.5–\+11\.5%\+11\.5\\%to\+12\.8\+12\.8–\+17\.9%\+17\.9\\%\. These comparisons show that the relative balanced\-draw effect depends on the readout in these settings\.

At the primary operating point, a separate120120\-run block evaluates reservoir storage, buffer5,1205\{,\}120, andb=8b=8without the EMA, paired to the seeds and cells of Table[5](https://arxiv.org/html/2609.25735#A7.T5)\. Removing the readout reduces accuracy by1\.61\.6to5\.25\.2points in eleven of the twelve arms\. The balanced\-draw effects over uniform retrieval are−0\.22\-0\.22,\+1\.08\+1\.08,\+0\.36\+0\.36, and\+0\.65\+0\.65points for CIFAR\-100 balanced, CIFAR\-100 LT10, CIFAR\-100 LT100, and Tiny\-ImageNet LT10, respectively; only the LT10 CIFAR\-100 effect is material\. In contrast, RPR remains material in all four cells, with gains of\+2\.41\+2\.41,\+2\.22\+2\.22,\+1\.39\+1\.39, and\+1\.92\+1\.92points \(\+16\.7%\+16\.7\\%to\+35\.7%\+35\.7\\%relative\) over uniform retrieval\. Its margins over the balanced draw are\+2\.63\+2\.63,\+1\.14\+1\.14,\+1\.03\+1\.03, and\+1\.28\+1\.28points, with at least eight of ten paired differences positive in each cell\. These runs have positive mean RPR effects without the averaged readout; the balanced\-draw effects are material in only one cell\.

## Appendix GAccuracies for crossed storage and retrieval policies

Table[5](https://arxiv.org/html/2609.25735#A7.T5)provides the absolute accuracies underlying the storage–retrieval contrasts in Table[1](https://arxiv.org/html/2609.25735#S5.T1)\. Every arm uses ER\-ACE with a ResNet\-18 initialised from scratch, buffer5,1205\{,\}120, replay batchb=8b=8, and one stream pass\. Entries report mean test accuracy and sample standard deviation over ten paired seeds; bold identifies the highest mean within each stream\. LT10 and LT100 denote exponential long\-tailed streams\. Reservoir sampling \(ℛ\\mathcal\{R\}\) and Balanced Reservoir Sampling \(BRS\), which evicts from the largest class, form the two storage policies\. Comparing columns within a row holds storage fixed and changes retrieval; comparing the two rows within a stream holds retrieval fixed and changes storage\. This factorial comparison separates which examples remain available from when their classes return to the learner\. The reported standard deviations describe individual\-arm variability; paired inference appears in Appendix[H](https://arxiv.org/html/2609.25735#A8)\.

Table 5:Final test accuracy \(%\) for crossed storage and retrieval policies\.RPR has the highest mean accuracy within every storage–stream row, and BRS with RPR gives the best mean in all three long\-tailed streams\. On balanced CIFAR\-100, reservoir storage with RPR is slightly higher than BRS with RPR \(20\.6120\.61versus20\.5520\.55\), so storage balancing is not uniformly beneficial\. RPR also has higher mean accuracy than independent balanced retrieval within every BRS row\.

## Appendix HPaired statistical tests

Table[6](https://arxiv.org/html/2609.25735#A8.T6)quantifies the uncertainty in the RPR contrasts of Table[1](https://arxiv.org/html/2609.25735#S5.T1)\. Pairing compares retrieval policies under the same seed, sosΔs\_\{\\Delta\}describes variation in the within\-seed difference, not the accuracy variation of either arm separately\. The test statistic has nine degrees of freedom,ppis its two\-sided Student\-ttprobability, andkkcounts positive paired differences out of ten\. Each Holm family contains the four streams for one storage policy and one comparator; the adjustment does not pool all sixteen tests into a single family\.

Table 6:Paired tests of RPR against independent balanced retrieval and fixed cycling\.Every RPR–i\.i\.d\. comparison remains significant at the5%5\\%level after this adjustment, under both reservoir and BRS storage\. RPR also exceeds fixed cycling in six of the eight adjusted tests\. Both exceptions concern CIFAR\-100 LT100, where the positive mean differences do not establish an additional benefit over fixed cycling\. Across these comparisons, evidence for RPR over independent draws is more consistent than evidence for RPR over fixed cycling\.

## Appendix IMeasured gaps inside the online runs

Table[7](https://arxiv.org/html/2609.25735#A9.T7)extends the static diagnostic to an evolving replay buffer\. Its class countCCdenotes the benchmark vocabulary, not an input available to RPR in advance; the scheduler uses only the current resident set\. All runs use ER\-ACE, buffer5,1205\{,\}120, replay batchb=8b=8, and ten seeds per cell; entries average the per\-run statistics across seeds\. Because this set grows and can lose classes, the aggregate mean gap need not equal the stationary valueC/bC/b\. The resident maximum tests the churn\-conditional guarantee, while the wall maximum also counts time when storage makes a class unavailable to every retrieval rule\. The violation count records steps whose resident gap exceeds Equation \([7](https://arxiv.org/html/2609.25735#S4.E7)\) using the run’s ownCmaxC\_\{\\max\}and insertion count\. Class deactivations record departures from memory\.

Table 7:Online rehearsal\-gap statistics \(steps\), bound exceedances, and class deactivations\.RPR yields no observed violation of the churn\-conditional bound in any of the three settings, while the independent draw exceeds the same threshold hundreds of times per run\. Fixed cycling has shorter tails than RPR in these settings\. On Tiny\-ImageNet LT100, the RPR wall maximum of47\.847\.8exceeds its resident maximum of46\.346\.3steps, alongside7\.47\.4class deactivations per run\. This distinction makes the guarantee explicitly conditional on storage retaining the class\.

## Appendix JHead, mid and tail accuracy and per\-class forgetting

Table[8](https://arxiv.org/html/2609.25735#A10.T8)examines whether the aggregate scheduling gain reflects a redistribution of accuracy towards frequent classes\. Within each run, classes are sorted by observed online frequency, with class identifier breaking ties\. The first and second⌊C/3⌋\\lfloor C/3\\rfloorclasses form the tail and mid groups, and the remainder forms the head group; the resulting sizes are33/33/3433/33/34for CIFAR\-100 and66/66/6866/66/68for ImageNet\-R and Tiny\-ImageNet\. Macro accuracy weights all classes equally\. Each entry compares RPR with i\.i\.d\. class\-balanced retrieval on the test split, with95%95\\%Student\-ttintervals over ten paired seeds\. The from\-scratch ResNet\-18 settings use buffer5,1205\{,\}120andb=8b=8; pretrained ViT\-B/16 uses buffer500500andb=32b=32\. For classcc, forgetting is its highest accuracy over task\-boundary evaluations minus its final accuracy; the table averages this quantity over classes\. Positive accuracy differences favour RPR; negative forgetting differences indicate improved retention\.

Table 8:RPR minus independent balanced retrieval in frequency\-group accuracy and forgetting\. Differences are in percentage points; macro accuracy weights classes equally and therefore differs from task\-mean accuracy when class sizes are unequal\.In the from\-scratch settings, all three frequency groups have positive mean differences, and tail gains are at least as large as head gains\. The intervals exclude zero for every group except the head of Tiny\-ImageNet LT100; macro forgetting decreases in all three settings\. The pretrainedb=32b=32controls show a different pattern: CIFAR\-100 loses2\.252\.25points on tail classes and has greater forgetting, while every ImageNet\-R interval includes zero\. These results support a setting\-dependent benefit rather than a universal improvement in rare\-class retention, consistent with the gap\-opportunity analysis in Table[3](https://arxiv.org/html/2609.25735#S5.T3)\.

## Appendix KGap drift, co\-occurrence, and buffer diagnostics

### K\.1Bias displacement during joint absence

Table[9](https://arxiv.org/html/2609.25735#A11.T9)measures the realised classifier\-bias change over episodes in which a resident class is absent from both incoming and replay batches under masked ER\-ACE with a linear classifier\. All three diagnostics in this appendix use the same linear classifier with bias, whereas the primary storage–retrieval comparison uses the bias\-free cosine head\. This distinction is necessary for interpreting the measured bias displacement\. An episode starts at a replay visit and closes at the next replay visit\. An incoming\-only return neither closes nor censors it, but that update is not included in the accumulated displacement because the class is not jointly absent\. Eviction discards the open episode\. Columns group completed episodes by replay\-to\-replay gap; the displacement sums only jointly absent updates\. Within each seed we average completed episodes in a bin, then average those per\-seed means equally across the ten seeds\. This diagnostic measures bias displacement along the training trajectory to evaluate the bias\-pressure mechanism\.

Table 9:Classifier\-bias displacement by replay\-to\-replay gap \(steps\), accumulated only over jointly absent updates\.All populated bins have negative\-displacement fraction1\.001\.00, and the magnitude of the mean displacement increases with episode length under both retrieval policies\. RPR has no completed CIFAR\-100 episode in the≥33\\geq 33\-step bin; the dash denotes absence of observations, not zero drift\. These measurements support accumulated bias pressure during long gaps\. However, the bins condition on different episodes and classes, so a within\-bin difference between policies is not an isolated causal estimate of scheduling, nor does the bias displacement determine the movement of every logit through the shared representation\.

### K\.2Renewal of class co\-occurrence

Table[10](https://arxiv.org/html/2609.25735#A11.T10)varies how often the class order renews at buffer size5,1205\{,\}120\. The parameterKKcounts traversals that reuse one shuffled pass: RPR setsK=1K=1, the intermediate schedules setK=4K=4or1616, and fixed cycling never renews the order\. Pair coverage and pair entropy describe the diversity of within\-batch class co\-occurrence, while the gap columns record the remaining temporal variation\. Accuracy differences use independent balanced retrieval as the comparator and report95%95\\%paired intervals\. Both pair statistics cover the full training stream\. Pair coverage is the number of distinct observed replay\-class pairs divided by\(Cmax2\)\\binom\{C\_\{\\max\}\}\{2\}, whereCmaxC\_\{\\max\}is the largest resident class count in the run\. Normalised pair entropy is−∑pqplogqp/log\(Cmax2\)\-\\sum\_\{p\}q\_\{p\}\\log q\_\{p\}/\\log\\binom\{C\_\{\\max\}\}\{2\}, whereqpq\_\{p\}is a pair’s share of all within\-batch pair observations and the logarithm is natural\.

Table 10:Co\-occurrence diversity, rehearsal gaps, and accuracy under different renewal frequencies\. Accuracy differences are in percentage points\.Less frequent renewal reduces pair coverage and entropy on both datasets, but accuracy does not follow a universal monotone ordering\. On CIFAR\-100, the mean gain decreases from1\.231\.23points for RPR to0\.470\.47for fixed cycling; on Tiny\-ImageNet, the intermediate schedules have slightly larger point estimates than RPR\. All bounded schedules have positive mean gains over independent draws\. Since renewal also changes the gap distribution, these interventions do not independently identify the contribution of pair diversity\. The comparisons associate bounded gaps with positive mean accuracy differences but do not isolate which temporal property accounts for them\.

### K\.3Buffer capacity and class residency

Table[11](https://arxiv.org/html/2609.25735#A11.T11)tests how memory capacity mediates the scheduling effect on CIFAR\-100 LT10\. The accuracy columns report final test accuracy, andΔ\\Deltagives RPR minus independent balanced retrieval with a95%95\\%paired interval\. Deactivations count departure events from the resident set, so a class can contribute more than once if it leaves, returns, and leaves again\. Reporting this count alongside the gap tail distinguishes failure to retain a class from failure to revisit one that remains available\.

Table 11:Accuracy \(%\), class deactivations, and rehearsal gaps \(steps\) across buffer capacities on CIFAR\-100 LT10\.RPR shortens the gap tail at every capacity, but the accuracy effect is small at buffers200200and500500, and the interval at500500includes zero\. The gains increase to0\.680\.68,1\.161\.16, and1\.231\.23points at the three larger capacities, where class deactivations fall from1\.91\.9to zero\. The effect is not strictly monotone across the entire sweep\. Residency and gap statistics vary together in these comparisons, so their contributions are not isolated\. Retrieval cannot revisit a class while it is absent from memory\.

## Appendix LLoss interventions and cross\-host comparisons

Table[12](https://arxiv.org/html/2609.25735#A12.T12)examines how the learner’s loss changes its response to class\-balanced retrieval\. These contrasts compare the independent balanced draw with uniform retrieval, not RPR with the balanced draw\. Mask removal tests dependence on ER\-ACE’s incoming\-loss asymmetry; mask addition tests whether the same intervention transfers to DER\+\+\. The remaining rows probe the limits of a prediction based only on the per\-example form of the replay loss\. This confirmation block contains470470test runs\. Entries report mean paired effects with95%95\\%Student\-ttintervals; an effect is material when the interval excludes zero and its relative magnitude exceeds the pre\-specified2%2\\%threshold\. CE denotes per\-example cross\-entropy, whose normalizer ranges over output classes, not other replay examples\. Daggers identify null predictions specified before execution\.

Table 12:Class\-balanced minus uniform retrieval under loss interventions and across hosts\. Absolute effects are in percentage points\.Without the ER\-ACE mask, the balanced\-draw point estimates are near zero and both intervals include zero\. The mask itself improves ER\-ACE by5\.395\.39points on CIFAR\-100 and4\.164\.16on Tiny\-ImageNet, a separate contrast from the retrieval effect in the table\. Adding the mask to DER\+\+ moves both point estimates in the positive direction, but neither interval excludes zero\. DER is inconclusive, whereas DER\+\+/Refresh on CIFAR\-100 contradicts the predicted null, with a\+0\.90\+0\.90\-point effect\. Together, these controls support mask dependence in ER\-ACE but not a necessary or sufficient rule across hosts\. They complement the RPR mask interaction in Table[2](https://arxiv.org/html/2609.25735#S5.T2), which compares RPR with the independent balanced draw at fixed storage and replay budget\.

The scheduling and cross\-host comparisons further qualify this account\. With an ACE mask, DER\+\+ gains1\.021\.02points on CIFAR\-100 and0\.870\.87on Tiny\-ImageNet from RPR over independent balanced retrieval; the corresponding unmasked effects are0\.100\.10and0\.240\.24points\. OCM has an inconclusive0\.300\.30\-point effect \(95%95\\%CI\[−0\.44,\+1\.05\]\[\-0\.44,\+1\.05\]\), rather than the negative effect suggested by a simple batch\-coupling argument\. Here DER\+\+/Refresh denotes the unlearn\-then\-relearn perturbation with Fisher damping\. Its task\-free adaptation updates Fisher estimates through an online moving average triggered by observed\-example count rather than task boundaries\. These results separate a local loss mechanism from the cumulative response of an entire training trajectory\.

## Appendix MAccuracy across the learning trajectory

To summarise accuracy during training, Table[13](https://arxiv.org/html/2609.25735#A13.T13)compares the arithmetic mean of the ten task\-boundary average\-accuracy measurements within each pretrained ViT\-B/16 LT10 run\. Entries give RPR minus the indicated comparator, with95%95\\%Student\-ttintervals over ten paired seeds\. This metric summarises the evaluated trajectory; it is not sample\-wise online or prequential accuracy and does not resolve changes between evaluation points\.

Table 13:RPR effects on mean task\-boundary accuracy for pretrained ViT\-B/16 on LT10 streams\. Differences are in percentage points\.Atb=8b=8, RPR improves this trajectory summary over independent balanced retrieval by1\.301\.30points on CIFAR\-100 and1\.131\.13points on ImageNet\-R, with both intervals above zero\. Atb=32b=32, both intervals include zero, consistent with the shorter resident\-gap tail and weaker final\-accuracy effect in the matched replay\-budget controls\. Every fixed\-cycle contrast also includes zero\. These measurements support higher mean task\-boundary accuracy for RPR than for independent draws in the tested small\-batch setting, but do not establish an additional effect over fixed cycling\.

## Appendix NAdditional retrieval controls

Table[14](https://arxiv.org/html/2609.25735#A14.T14)compares four retrieval objectives in ER\-ACE with a5,1205\{,\}120\-example buffer and a replay batch of3232\. Entries report test\-accuracy differences from uniform retrieval in percentage points, with relative changes in parentheses\. MIR prioritises examples by interference; the independent balanced draw samples classes uniformly; RPR uses shuffled class passes\. This comparison evaluates these retrieval policies within the same ER\-ACE configuration\.

Table 14:Test\-accuracy effects of retrieval policies relative to uniform retrieval\. Absolute effects are in percentage points\.In these runs, MIR retrieves7\.47\.4distinct classes per batch on average, compared with uniform retrieval’s21\.721\.7, and its accuracy is lower on both datasets\. Class\-balanced retrieval and RPR instead improve over uniform retrieval, with RPR giving the largest point estimate\. On CIFAR\-100, for example, the relative changes are−50\.1%\-50\.1\\%,0%0\\%,\+5\.8%\+5\.8\\%, and\+7\.4%\+7\.4\\%for MIR, uniform retrieval, independent balanced retrieval, and RPR, respectively\. The association between class coverage and accuracy is consistent with the ER\-ACE mechanism, but this comparison changes the retrieval objective as well as batch composition\. It does not isolate a causal effect of class count or establish a general ranking of MIR and RPR across hosts\. The separate readout control in Appendix[F](https://arxiv.org/html/2609.25735#A6)tests whether the scheduling benefit depends on evaluation\-time weight averaging\.