Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions

arXiv cs.LG Papers
continual-learning class-incremental long-tailed spectral-regularization analytic-learning deep-learning

Summary

Proposes Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem, achieving new state-of-the-art results for analytic class-incremental learning.

arXiv:2607.22931v1 Announce Type: new Abstract: Analytic Continual Learning (ACL) offers a computationally efficient alternative to gradient-based approaches. Recent ACL methods are based on Recursive Least Squares (RLS) and have achieved the state-of-the-art results compared to other alternatives. However, they falter significantly in Class-Incremental Learning scenarios characterized by Long-Tailed distributions. While the ill-conditioning of the autocorrelation (Gram) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise. Standard Ridge Regression ($L_2$) fails to address this effectively as it applies isotropic regularization - a uniform penalty that is insufficient to stabilize the tail without over-shrinking the head. To address this, we propose Geometry-Spectral Rectification (GSR), a theoretically grounded framework that treats long-tailed learning as a spectral regularization problem. Unlike standard isotropic regularization (Ridge) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes. We construct a structured, data-dependent spectral perturbation matrix $\Delta$ that selectively inflates collapsed tail eigen-directions of the Gram matrix. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability. Extensive experiments show that GSR establishes a new state-of-the-art for analytic CIL, offering a superior trade-off between computational efficiency and robust generalization in long-tailed settings.
Original Article
View Cached Full Text

Cached at: 07/28/26, 06:23 AM 

# Spectral-Aware Analytic Class-Incremental Learning for Long-Tailed Distributions
Source: [https://arxiv.org/abs/2607.22931](https://arxiv.org/abs/2607.22931)
[View PDF](https://arxiv.org/pdf/2607.22931)

> Abstract:Analytic Continual Learning \(ACL\) offers a computationally efficient alternative to gradient\-based approaches\. Recent ACL methods are based on Recursive Least Squares \(RLS\) and have achieved the state\-of\-the\-art results compared to other alternatives\. However, they falter significantly in Class\-Incremental Learning scenarios characterized by Long\-Tailed distributions\. While the ill\-conditioning of the autocorrelation \(Gram\) matrix is a known limitation of RLS, we demonstrate that class imbalance exacerbates this issue into a distinct spectral pathology: "tail" classes suffer from severe spectral collapse, rendering their subspaces numerically indistinguishable from noise\. Standard Ridge Regression \($L\_2$\) fails to address this effectively as it applies isotropic regularization \- a uniform penalty that is insufficient to stabilize the tail without over\-shrinking the head\. To address this, we propose Geometry\-Spectral Rectification \(GSR\), a theoretically grounded framework that treats long\-tailed learning as a spectral regularization problem\. Unlike standard isotropic regularization \(Ridge\) which uniformly penalizes all eigenvalues, GSR acts as an anisotropic spectral filter, selectively inflating the collapsed eigenvalues of tail classes\. We construct a structured, data\-dependent spectral perturbation matrix $\\Delta$ that selectively inflates collapsed tail eigen\-directions of the Gram matrix\. Theoretical analysis proves that GSR guarantees an improved stable rank for the Gram matrix, ensuring numerical stability\. Extensive experiments show that GSR establishes a new state\-of\-the\-art for analytic CIL, offering a superior trade\-off between computational efficiency and robust generalization in long\-tailed settings\.

## Submission history

From: Quan Dao \[[view email](https://arxiv.org/show-email/c38520a4/2607.22931)\] **\[v1\]**Fri, 24 Jul 2026 22:12:15 UTC \(120 KB\)

Similar Articles

Spectral Gradient Surgery for Domain-Generalizable Dataset Distillation

arXiv cs.LG

This paper introduces Domain Generalizable Dataset Distillation (DGDD), a new problem setting that targets out-of-distribution generalization of distilled datasets, and proposes Spectral Gradient Surgery (SGS) to disentangle class-discriminative and domain-specific information by leveraging cross-domain gradient agreement in the spectral domain.

Spectral Signatures of Large Language Models

arXiv cs.CL

This paper introduces a spectral shape-based metric using Heavy-Tailed Self-Regularization theory to characterize, compare, and manage large language models. The approach is data-free, computationally efficient, and scale-invariant, enabling model lineage tracing, unsupervised clustering, and performance quantification across diverse model collections.