Tag
提出了一种基于Kolmogorov–Arnold Networks（KAN）的非线性投影方法来增强低秩分解，用于NLP嵌入层压缩，实验表明该方法在固定维度下能生成更忠实的压缩嵌入。
SpecMRL 提出在参数化谱空间中重新组织特征以改进嵌入的前缀读出，在 MTEB 上以更少的嵌入维度超越标准 MRL 及其全维表现。
Proposes DIVE, a compression adapter for embedding dimensionality reduction that uses self-limiting gradient updates and head-wise NT-Xent contrastive loss to prevent overfitting on small datasets, outperforming existing methods on BEIR benchmarks.
This article introduces a polynomial autoencoder that improves upon PCA for compressing transformer embeddings by using a quadratic decoder to capture nonlinear variance. Benchmarks on BEIR show it significantly outperforms standard PCA and Matryoshka embeddings in retrieval quality while maintaining high compression ratios.
Spectral Tempering (SpecTemp) proposes a learning-free method for embedding compression in dense passage retrieval that adaptively determines optimal spectral scaling based on signal-to-noise ratio analysis, outperforming fixed hyperparameter approaches like PCA and whitening.