An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning
Summary
This paper introduces MMOT, an online mixture model learning framework based on optimal transport theory that addresses incremental learning with distributional shifts through dynamic centroid updates and improved class similarity estimation. The approach includes a Dynamic Preservation strategy to mitigate catastrophic forgetting and maintain class separability in latent space.
View Cached Full Text
Cached at: 04/20/26, 08:27 AM
Paper page - An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning
Source: https://huggingface.co/papers/2211.16780
Abstract
An online mixture model learning framework based on optimal transport theory addresses challenges in incremental learning with distributional shifts by enabling dynamic centroid updates and improving class similarity estimation.
In online incremental learning (https://huggingface.co/papers?q=online%20incremental%20learning), data continuously arrives with substantial distributional shifts (https://huggingface.co/papers?q=distributional%20shifts), creating a significant challenge because previous samples have limited replay value when learning a new task. Prior research has typically relied on either a single adaptive centroid (https://huggingface.co/papers?q=adaptive%20centroid) or multiple fixed centroids (https://huggingface.co/papers?q=fixed%20centroids) to represent each class in the latent space (https://huggingface.co/papers?q=latent%20space). However, such methods struggle when class data streams are inherently multimodal and require continual centroid updates. To overcome this, we introduce an online Mixture Model (https://huggingface.co/papers?q=Mixture%20Model) learning framework grounded in Optimal Transport theory (https://huggingface.co/papers?q=Optimal%20Transport%20theory) (MMOT), where centroids evolve incrementally with new data. This approach offers two main advantages: (i) it provides a more precise characterization of complex data streams, and (ii) it enables improved class similarity estimation (https://huggingface.co/papers?q=class%20similarity%20estimation) for unseen samples during inference through MMOT-derived centroids. Furthermore, to strengthen representation learning (https://huggingface.co/papers?q=representation%20learning) and mitigate catastrophic forgetting (https://huggingface.co/papers?q=catastrophic%20forgetting), we design a Dynamic Preservation strategy (https://huggingface.co/papers?q=Dynamic%20Preservation%20strategy) that regulates the latent space (https://huggingface.co/papers?q=latent%20space) and maintains class separability (https://huggingface.co/papers?q=class%20separability) over time. Experimental evaluations on benchmark datasets confirm the superior effectiveness of our proposed method.
View arXiv page (https://arxiv.org/abs/2211.16780)View PDF (https://arxiv.org/pdf/2211.16780)Add to collection (https://huggingface.co/login?next=%2Fpapers%2F2211.16780)
Get this paper in your agent:
hf papers read 2211.16780
Don’t have the latest CLI?curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2211.16780 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2211.16780 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2211.16780 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to a collection (https://huggingface.co/new-collection) to link it from this page.
Similar Articles
Accelerating LMO-Based Optimization via Implicit Gradient Transport
This paper proposes LMO-IGT, a new class of stochastic optimization methods that accelerates convergence using implicit gradient transport while maintaining a single-gradient-per-iteration structure. It introduces a unified theoretical framework and demonstrates improved performance over existing LMO-based optimizers like Muon.
One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning
This paper proposes a unified optimal transport framework for cold-start active learning, introducing a Sinkhorn-based algorithm (ε-AS) that adapts regularization strength to data and achieves state-of-the-art results on six datasets, including improving ImageNet-1k accuracy by 1.29% over prior methods while reducing selection time by 56.2%.
DOT-MoE: Differentiable Optimal Transport for MoEfication
DOT-MoE formulates dense layer decomposition as a differentiable optimal transport problem, enabling efficient training of sparse MoE models that retain 90% of original performance while reducing active parameters by 50%.
SAOT: Self-Supervised Continual Graph Learning with Structure-Aware Optimal Transport
Proposes SAOT, a structure-aware optimal transport framework for self-supervised continual graph learning that preserves relational structure across tasks. Achieves significant performance gains over state-of-the-art methods on multiple benchmarks, including up to 15% improvement on Products-CL.
IGT-OMD: Implicit Gradient Transport for Decision-Focused Learning under Delayed Feedback
This paper identifies 'staleness amplification' in bilevel optimization under delayed feedback and proposes IGT-OMD, which uses Implicit Gradient Transport to achieve sublinear regret and improve decision loss on benchmarks like Warcraft shortest-path and LQR.