No More K-means:Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval @Veritas2026 et al. replace vector clustering with efficient sparse autoencoders & natural inverted indexing to accelerate multi-vector retrieval. https://arxiv.org/abs/2605.30120 https://github.com/Y-Research-SBU/SSR…

Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval

Source: https://arxiv.org/html/2605.30120

Abstract

Multi-vector retrieval (MVR) models, exemplified by ColBERT, have established new benchmarks in retrieval accuracy by preserving fine-grained token-level interactions. However, this granularity imposes prohibitive storage and retrieval efficiency bottlenecks: to manage the immense memory footprint and computational overhead of billion-scale token vectors, state-of-the-art systems are forced to rely on aggressive dimension reduction and complex clustering (e.g., K-means). This compromise introduces two critical limitations: excessive indexing latency of clustering large-scale corpora and semantic information loss inherent to compression. In this paper, we proposeSingle-stage Sparse Retrieval(SSR), a paradigm shift that replaces expensive clustering with efficient sparse coding. Instead of compressing features into low-dimensional dense vectors, we utilize Sparse Autoencoder (SAE) to project token embeddings into a high-dimensional but highly sparse representation. This transformation enables us to bypass vector clustering entirely and leverage inverted indexing for precise, high-throughput retrieval. Extensive experiments on the BEIR benchmark demonstrate that SSR achieves a ”trifecta” of improvements: it reduces indexing time by 15x compared to ColBERTv2, halves retrieval latency, and simultaneously improves retrieval performance over leading baselines.

Figure 1:Single-stage Sparse Retrieval (SSR).(a)Paradigm Comparison.Unlike dense MVR, which compresses embeddings into low-dimensional vectors and requires exhaustive token-pair computations, our sparse MVR projects features into a high-dimensional sparse space via Sparse Autoencoders (SAE). This enables efficient interaction calculation solely on overlapping activated neurons. (b)Trade-off Analysis.Single Vector Retrieval offers high efficiency but suffers from semantic compression loss. Dense MVR preserves token-level details but incurs heavy computational overheads due to clustering and multi-stage pruning (inefficient indexing). SSR bridges this gap, accelerating retrieval via natural inverted indexing while retaining fine-grained semantic information. (c)Performance vs. Efficiency.SSR achieves state-of-the-art retrieval performance with halved retrieval latency compared to competitive baselines. Notably, it delivers a 15x reduction in indexing time (indicated by bubble size) by eliminating the clustering bottleneck.

1Introduction

Neural information retrieval has long faced a fundamental trade-off between precision and efficiency(Khattab and Zaharia,2020; Caoet al.,2023; Zhouet al.,2023; Santhanamet al.,2022a). While Single-Vector Retrieval (SVR) enables high-throughput search via simple dot products, it often struggles to capture the intricate semantic nuances of complex queries. In contrast, Multi-Vector Retrieval (MVR) paradigms, pioneered by ColBERT(Khattab and Zaharia,2020), have established a new standard for effective retrieval. By preserving documents as sequences of token-level embeddings and employing late interaction (e.g., MaxSim) mechanisms, MVR achieves fine-grained semantic alignment that SVR typically misses.

However, such precision imposes substantial storage and computational overheads. Representing documents as bags of vectors causes the index size to explode by orders of magnitude compared to single-vector baselines. To make deployment feasible, state-of-the-art systems like PLAID(Santhanamet al.,2022a)rely on aggressive approximation strategies, such as Vector Quantization (VQ) and large-scale clustering (e.g., K-means). While these engineering optimizations mitigate retrieval latency, they leave two critical issues unresolved. First, compressing rich token embeddings into short codes or centroids inevitably incurs information loss, sacrificing the very semantic details MVR aims to preserve. Second, the indexing process remains prohibitively complex: performing clustering on billion-scale token datasets is computationally expensive, creating a severe bottleneck for index construction and real-time updates. This leads us to a pivotal question:

Can we retain the token-level granularity of MVR while achieving similar retrieval speed of lexical search, without the heavy burden of clustering?

In this paper, we propose a paradigm shift from dense approximation toSingle-stageSparseRetrieval (SSR). As shown in Figure1(Left), instead of compressing token embeddings into lower-dimensional dense embeddings, we project them into a significantly higher-dimensional but highly sparse feature space using Sparse Autoencoder (SAE)(Gaoet al.,2024; Leeet al.,2006). This transformation disentangles complex token semantics into sparse vectors with few active neurons (e.g., 32). Crucially, this sparsity unlocks a structural advantage that dense methods lack: it enables us to discard K-means entirely and utilize the inverted index, allowing each active dimension to function as a ”pseudo token”, which is the same efficient data structure powering traditional keyword search (e.g., BM25(Robertsonet al.,1995)). We present two implementations: SSR-tok, which only focuses on token-level interactions, and SSR-CLS, which incorporates global semantics by integrating [CLS] embedding similarities.

We empirically validate SSR on the MS MARCO(Nguyenet al.,2016)(in-domain) and BEIR(Thakuret al.,2021)(out-of-domain) benchmarks. Our results demonstrate that SSR simultaneously optimizes accuracy, speed, and scalability: it matches or exceeds the performance of state-of-the-art retrievers (achieving an average 2.2% improvement over the strongest baseline) while nearly halving retrieval latency and reducing index construction time by over 15×\times(by eliminating the clustering overhead). Furthermore, we demonstrate the scalability of our approach by applying it to modern Large Language Model backbones (Llama-Embed-8B(Babakhinet al.,2025)), achieving performance competitive with the latest single-vector embedding models. We discuss in detail the evolution of single-vector retrieval and multi-vector retrieval techniques in AppendixB, highlighting the correlations and limitations of existing methods that motivate SSR. Our code is released athttps://github.com/Y-Research-SBU/SSR. Our contributions are as follows:

• •We proposeSingle-stage Sparse Retrieval, a framework for efficient and effective multi-vector sparse coding, enabling direct use of inverted indices for single-stage semantic retrieval.
• •We introduce a hybrid training objective that combines reconstruction loss with a multi-vector contrastive loss, ensuring that the learned sparse features are discriminative for ranking tasks.
• •Extensive in-domain and out-of-domain evaluations confirm that SSR effectively improves the efficiency-effectiveness trade-off frontier, delivering sub-20ms retrieval latency and state-of-the-art indexing speed without compromising retrieval quality.

Figure 2:Conceptual comparison between standard retrieval paradigms (e.g., PLAID(Santhanamet al.,2022b)) and our proposed SSR.

2Background

2.1Problem Formulation

The objective of a retrieval task is to get a ranked list of documents from a large-scale corpus𝒟={D1,…,D|𝒟|}\mathcal{D}=\{D_{1},\dots,D_{|\mathcal{D}|}\}that are semantically relevant to a user queryQQ. Both the queryQ=(q1,…,q|Q|)Q=(q_{1},\dots,q_{|Q|})and the documentD=(d1,…,d|D|)D=(d_{1},\dots,d_{|D|})are typically represented as sequences of tokens. The core challenge is to learn a parameterized scoring functionsθ​(Q,D)∈ℝs_{\theta}(Q,D)\in\mathbb{R}that quantifies the relevance between the query-document pair. Ideally, for a relevant documentD+D^{+}and a non-relevant documentD−D^{-}, the model should satisfysθ​(Q,D+)>sθ​(Q,D−)s_{\theta}(Q,D^{+})>s_{\theta}(Q,D^{-}).

2.2Existing Multi-Vector Retrieval Paradigm

Despite architectural variations in recent literature, the inference process of modern Multi-Vector Retrieval (MVR) models generally converges to a three-stage filter-and-refine paradigm, as illustrated in Figure2. Formally, given a queryQ={q1,…,qN}Q=\{q_{1},\dots,q_{N}\}and a document corpus𝒟\mathcal{D}, the goal is to efficiently identify top-kkdocuments that maximize the late interaction scoreS​(Q,D)S(Q,D).

Stage I: Indexing & Candidate Generation.Handling billion-scale token vectors necessitates an efficient pre-filtering mechanism. Specifically, document tokens are mapped to a codebook of centroids𝒞={𝐜1,…,𝐜K}\mathcal{C}=\{\mathbf{c}_{1},\dots,\mathbf{c}_{K}\}via clustering engines such as FAISS(Douzeet al.,2024)in the indexing stage. During retrieval, for each query tokenqiq_{i}, the system identifies a set of nearest centroids𝒞i⊂𝒞\mathcal{C}_{i}\subset\mathcal{C}and retrieves the associated document list. The initial candidate set𝒟cand\mathcal{D}_{\text{cand}}is the union of documents hit by these active centroids:

𝒟cand=⋃qi∈Q⋃𝐜∈𝒞iϕ​(𝐜)\mathcal{D}_{\text{cand}}=\bigcup_{q_{i}\in Q}\bigcup_{\mathbf{c}\in\mathcal{C}_{i}}\phi(\mathbf{c})(1) whereϕ​(𝐜)\phi(\mathbf{c})represents the set of documents that include tokens in the centroid𝐜\mathbf{c}. This stage typically prunes the search space by filtering out>98%>98\%of the original corpus.

Stage II: Approximate Scoring & Pruning.Performing fine-grained interaction on𝒟cand\mathcal{D}_{\text{cand}}is still computationally prohibitive due to the high cost of memory access and floating-point operations. Therefore, this stage employs approximate scoring using compressed representations. For instance, PLAID(Santhanamet al.,2022a)and EMVB(Nardiniet al.,2024)estimate the similarity using centroid-level interactions or bit-vectors rather than reconstructing the full embeddings. The approximate scoreS^​(Q,D)\hat{S}(Q,D)is often calculated as:

S^​(Q,D)=∑i=1|Q|maxj=1|D|⁡(𝐪i⋅𝐜dj)\hat{S}(Q,D)=\sum_{i=1}^{|Q|}\max_{j=1}^{|D|}(\mathbf{q}_{i}\cdot\mathbf{c}_{d_{j}})(2)where𝐜dj\mathbf{c}_{d_{j}}represents the centroid of thejj-th document tokendjd_{j}. Based onS^\hat{S}, the candidate set is aggressively pruned to a much smaller subset𝒟top⊂𝒟cand\mathcal{D}_{\text{top}}\subset\mathcal{D}_{\text{cand}}(typically thousands of documents).

Stage III: Final Reranking with Decompression.In the final stage, full-precision vectors are reconstructed to resolve minor semantic differences. Methods like ColBERTv2(Santhanamet al.,2022b)utilize residual compression, where the reconstructed vector𝐝~j≈𝐜dj+𝐫dj\tilde{\mathbf{d}}_{j}\approx\mathbf{c}_{d_{j}}+\mathbf{r}_{d_{j}}. The final ranking is determined by the precise MaxSim operation:

S​(Q,D)=∑i=1Nmaxj=1M⁡(𝐪𝐢⋅𝐝~j)S(Q,D)=\sum_{i=1}^{N}\max_{j=1}^{M}(\mathbf{q_{i}}\cdot\tilde{\mathbf{d}}_{j})(3)This ensures that the final top-ranked documents are selected based on the highest fidelity representation.

Analysis: Efficiency Gain and Limitations.The paradigm described above significantly reduces retrieval costs by shifting the heavy computation from the entire corpus to a progressively smaller candidate set. By utilizing quantization and approximate scoring, modern engines achieve sub-second latency for million-scale corpora. However, substantial challenges remain.First, the indexing process is computationally expensive and slow, primarily due to the overhead of clustering billion-scale tokens and computing residuals.Second, the multi-stage pruning pipeline introduces complex control logic and memory access patterns, where the overhead of decompression and repeated scoring can become a bottleneck.Finally, the MaxSim operation in the final stage, even with SIMD optimizations, remains theoretically quadratic with respect to sequence length (i.e.,O​(N×M)O(N\times M)), limiting the throughput for long-context retrieval.

3New Paradigm: from Density to Sparsity

To address the efficiency limitations and indexing overhead inherent in the past multi-vector systems, we proposeSingle-stage Sparse Retrieval (SSR). Instead of relying on expensive clustering and aggressive dimensionality reduction, SSR projects token embeddings into a high-dimensional, highly sparse feature space via Sparse Autoencoder (SAE), enabling direct and efficient semantic retrieval.

3.1Sparse Late Interaction Scoring

The core of our SSR paradigm is to maintain the token-level granularity of late interaction while operating in a sparse latent space. Given a queryQ={q1,…,q|Q|}Q=\{q_{1},...,q_{|Q|}\}with|Q||Q|tokens and a documentD={d1,…,d|D|}D=\{d_{1},...,d_{|D|}\}with|D||D|tokens, each token is first encoded into dense representation by backbone encoder such as BERT, and then mapped into a sparse vector𝐳∈ℝh\mathbf{z}\in\mathbb{R}^{h}utilizing the learned SAE, after which the query is converted toQ′={𝐳q1,𝐳q2,…,𝐳q|Q|}∈ℝh×|Q|Q^{\prime}=\{\mathbf{z}_{q_{1}},\mathbf{z}_{q_{2}},...,\mathbf{z}_{q_{|Q|}}\}\in\mathbb{R}^{h\times|Q|}while document is converted toD′={𝐳d1,𝐳d2,…,𝐳d|D|}∈ℝh×|D|D^{\prime}=\{\mathbf{z}_{d_{1}},\mathbf{z}_{d_{2}},...,\mathbf{z}_{d_{|D|}}\}\in\mathbb{R}^{h\times|D|}. We adopt the MaxSim operator to calculate the fine-grained similarity. However, unlike dense retrieval, the interaction only occurs between activated neurons:

S​(Q,D)=∑i=1Nmaxj=1M⁡(∑u∈𝒜K​(𝐳qi)∩𝒜K​(𝐳dj)𝐳qi(u)⋅𝐳dj(u))S(Q,D)=\sum_{i=1}^{N}\max_{j=1}^{M}\left(\sum_{u\in\mathcal{A}_{K}(\mathbf{z}_{q_{i}})\cap\mathcal{A}_{K}(\mathbf{z}_{d_{j}})}\mathbf{z}_{q_{i}}^{(u)}\cdot\mathbf{z}_{d_{j}}^{(u)}\right)(4)where𝒜K​(⋅)\mathcal{A}_{K}(\cdot)denotes the set of indices for the top-KKlargest neurons.

AppendixAprovides a bounded-distortion analysis of the sparse scoring rule in Equation (4). Under small SAE reconstruction error and restricted decoder near-orthogonality on active sparse supports, the sparse inner product𝐳qi⊤​𝐳dj\mathbf{z}_{q_{i}}^{\top}\mathbf{z}_{d_{j}}approximates the dense token similarity𝐪i⊤​𝐝j\mathbf{q}_{i}^{\top}\mathbf{d}_{j}with bounded error. This token-level guarantee further extends through the MaxSim operator, showing that the full SSR late-interaction score remains close to the dense late-interaction score.

3.2Hybrid Training of Sparse Projectors

To ensure that the sparse features are both reconstructive and discriminative for retrieval, we train two separate SAEs, with𝐄tok\mathbf{E}_{\text{tok}}for regular tokens and𝐄[CLS]\mathbf{E}_{\text{[CLS]}}for global semantic tokens.

Unsupervised TopK Sparse Autoencoding.For feature𝐱∈ℝd\mathbf{x}\in\mathbb{R}^{d}, we apply the autoencoding process with encoder𝐖enc∈ℝh×d\mathbf{W}_{\text{enc}}\in\mathbb{R}^{h\times d}to reach the targeted sparsity levelKKwith TopK operator and reconstruct original feature𝐱^\mathbf{\hat{x}}with decoder𝐖dec∈ℝd×h\mathbf{W}_{\text{dec}}\in\mathbb{R}^{d\times h}:

𝐳=TopK​(𝐖enc​(𝐱−𝐛pre)+𝐛enc)\mathbf{z}=\text{TopK}(\mathbf{W}_{\text{enc}}(\mathbf{x}-\mathbf{b}_{\text{pre}})+\mathbf{b}_{\text{enc}})(5)𝐱^=𝐖dec​𝐳+𝐛pre\mathbf{\hat{x}}=\mathbf{W}_{\text{dec}}\mathbf{z}+\mathbf{b}_{\text{pre}}(6)wherehhis the hidden dimension,𝐛pre\mathbf{b}_{\text{pre}}and𝐛enc\mathbf{b}_{\text{enc}}are bias terms and the TopK operation sets all values to zero except theKKlargest. The training goal is to minimize the difference between original feature and the reconstructed feature under sparsityℒrecon​(k)=‖𝐱−𝐱^‖22\mathcal{L}_{\text{recon}}(k)=\|\mathbf{x}-\hat{\mathbf{x}}\|_{2}^{2}under sparsitykk.

Following(Gaoet al.,2024; Wenet al.,2025; Guoet al.,2026; Wanget al.,2024), we expand standard reconstruction loss with Multi-TopK loss, auxiliary lossℒaux\mathcal{L}_{\text{aux}}and sparse contrastive loss and the overall loss is:

ℒunsup=ℒrecon​(k)+18​ℒrecon​(4​k)+α​ℒaux​(kaux)+β​ℒcl\mathcal{L}_{\text{unsup}}=\mathcal{L}_{\text{recon}}(k)+\frac{1}{8}\mathcal{L}_{\text{recon}}(4k)+\alpha\mathcal{L}_{\text{aux}}(k_{\text{aux}})+\beta\mathcal{L}_{\text{cl}}(7)whereLaux​(kaux)L_{\text{aux}}(k_{\text{aux}})calculates the reconstruction loss using the top-kauxk_{\text{aux}}neurons that have not been activated for a long time and sparse contrastive lossLclL_{\text{cl}}encourages SAE to distinguish between positive and negative pairs with the following formula:

ℒcl=−1|ℬ|​∑i=1|ℬ|log⁡e𝐳iT​𝐳ie𝐳iT​𝐳i+∑j≠i|ℬ|e𝐳iT​𝐳j\mathcal{L}_{\text{cl}}=-\frac{1}{|\mathcal{B}|}\sum_{i=1}^{\mathcal{|B|}}\log\frac{e^{\mathbf{z}_{i}^{T}\mathbf{z}_{i}}}{e^{\mathbf{z}_{i}^{T}\mathbf{z}_{i}}+\sum_{j\neq i}^{|\mathcal{B}|}e^{\mathbf{z}_{i}^{T}\mathbf{z}_{j}}}(8)whereℬ\mathcal{B}is the set of all tokens in the training sentence batch.

Supervised Contrastive Learning.Furthermore, we incorporate an additional supervised contrastive loss term to help Sparse Autoencoder capture the semantic difference between one query’s positive and negative documents, which is followed by most mature dense MVR paradigms(Khattab and Zaharia,2020; Gaoet al.,2021; Leeet al.,2023):

ℒCE=−log⁡esim​(Q,D+)∑D∈𝒟esim​(Q,D)\mathcal{L}_{\text{CE}}=-\log\frac{e^{\text{sim}(Q,D^{+})}}{\sum_{D\in\mathcal{D}}e^{\text{sim}(Q,D)}}(9)whereQQis a query,𝒟\mathcal{D}is a set of mini-batch documents with one positive documentD+D^{+}and|𝒟|−1|\mathcal{D}|-1negative documentsD−D^{-}forQQandsim​()\text{sim}()is the similarity score that measures semantic similarity between two contexts. When trained on regular tokens, the similarity score is sum of MaxSim of each query token, while when trained on special token [CLS], the similarity score is cosine similarity.

Overall Training Objective.Finally, we optimize the sparse projector through a combination of unsupervised and supervised sparse learning, whose final training objective is formulated as:

ℒS​S​R=ℒunsup+γ​ℒCE\mathcal{L}_{SSR}=\mathcal{L}_{\text{unsup}}+\gamma\mathcal{L}_{\text{CE}}(10)whereγ\gammais the hyperparameter to balance the two loss components and is set to 0.05 for default.

Table 1:Comparative analysis of retrieval performance and efficiency against lightweight multi-vector and sparse lexical baselines. All methods are evaluated under a controlled experimental setup. The best results are highlighted inbold, and the second-best areunderlined. Retrieval performance is reported in nDCG@10, while efficiency is measured by per-query retrieval latency on the MSMARCO passage ranking dataset. Unless otherwise specified, the reported SSR-tok and SSR-CLS latency numbers use the SSR++ acceleration strategy described in Section3.3.MethodTimeMSARCFCVDBFEFQHQNFNQQUSDSFTOAvg.Late-interaction dense retrievalColBERT57.336.023.618.167.939.276.931.759.230.751.885.714.666.820.541.5COIL12.635.329.521.766.839.983.831.371.333.252.183.815.570.728.147.4ColBERTv256.939.846.317.373.544.978.736.166.633.556.385.315.269.126.149.2PLAID37.139.746.117.473.744.778.836.366.733.856.485.015.469.226.349.3AligneR51.838.828.918.268.841.672.433.561.534.052.482.314.370.334.846.6CITADEL15.739.951.118.371.542.276.633.066.333.754.085.315.969.034.249.4XTR33.445.040.720.873.640.873.734.864.734.052.885.914.671.131.348.8Learned sparse retrievalSplade-v216.343.347.623.571.543.478.733.868.433.552.183.815.969.436.350.1Splade-v316.643.950.823.374.845.279.837.569.235.858.681.415.871.029.651.2Late-interaction sparse retrievalSSR-tok17.545.246.122.876.448.281.939.469.538.860.788.317.572.933.152.9SSR-CLS19.545.546.623.576.848.482.840.069.939.161.288.717.973.333.753.4

3.3Sparsity-Enabled Efficient Indexing and Retrieval

Leveraging the high sparsity (K≪hK\ll h) of sparse features, we design a retrieval structure that circumvents the expensive cluster-based scanning of dense MVR. We first introduce the base retrieval paradigm (SSR) which utilizes neuron-level inverted indexing, followed by an accelerated variant (SSR++) employing a coarse-to-fine pruning strategy.

Neuron-Level Inverted Indexing.Unlike dense methods that require clustering, we directly build inverted indices on active neurons. Let𝐳t∈ℝh\mathbf{z}_{t}\in\mathbb{R}^{h}denote the sparse representation of a document tokendd. For each neuron dimensionu∈{1,…,h}u\in\{1,...,h\}, we construct a posting listℐu\mathcal{I}_{u}. To support the MaxSim operator efficiently, we store the maximum impact of neuronuuwithin a documentDD:

μD,u=maxt∈D⁡𝐳t(u)\mu_{D,u}=\max_{t\in D}\mathbf{z}_{t}^{(u)}(11)The posting list is thus defined asℐu={(D,μD,u)|μD,u>0}\mathcal{I}_{u}=\{(D,\mu_{D,u})|\ \mu_{D,u}>0\}. Furthermore, to facilitate retrieval, each listℐu\mathcal{I}_{u}is partitioned into fixed-size blocks, where each blockBBstores an upper-bound scoreUB=maxD∈B⁡μd,uU_{B}=\max_{D\in B}\mu_{d,u}. This block-level organization enables early pruning during posting-list traversal, allowing SSR to avoid scoring documents whose maximum possible contribution is already insufficient for the current topKKset.

Single-stage Sparse Retrieval (SSR).Given a queryQ={q1,…,qN}Q=\{q_{1},...,q_{N}\}, the goal of SSR is to compute the exact late-interaction score using the full set of activated neurons. For each query token embedding𝐪i\mathbf{q}_{i}, we identify its top-KKactivated neurons𝒜K​(𝐪i)\mathcal{A}_{K}(\mathbf{q}_{i}). The system traverses the union of posting lists corresponding to these neurons. Since the number of activated neuronsKKis small (e.g.,K=32K=32), this process is significantly faster than scanning dense vector clusters. However, traversingN×KN\times Klists can still be optimized for ultra-low latency scenarios.

Accelerated Retrieval with Pruning (SSR++).To further reduce latency, we propose SSR++, a coarse-to-fine pipeline that progressively prunes the search space.

Step 1: Coarse Scoring.For each query tokenqiq_{i}, we extract only the principal active neurons𝒜Kcoarse​(𝐪i)\mathcal{A}_{K_{\text{coarse}}}(\mathbf{q}_{i}), whereKcoarse<KK_{\text{coarse}}<K(we selectKcoarseK_{\text{coarse}}as 4). We compute an approximate upper-bound scoreS^coarse\hat{S}_{\text{coarse}}by traversing only these principal posting lists:

S^coarse​(Q,D)=∑i=1N∑u∈𝒜Kcoarse​(𝐪i)(𝐪i(u)⋅μD,u)\hat{S}_{\text{coarse}}(Q,D)=\sum_{i=1}^{N}\sum_{u\in\mathcal{A}_{K_{\text{coarse}}}(\mathbf{q}_{i})}(\mathbf{q}_{i}^{(u)}\cdot\mu_{D,u})(12)During this traversal, we utilize block upper-boundsUBU_{B}to skip blocks that cannot contribute enough to exceed the current top-k threshold, rapidly producing a small candidate set𝒞1\mathcal{C}_{1}.

Step 2: Exact Refinement.For the refined subset𝒞1\mathcal{C}_{1}, we revert to the full set of activated neurons𝒜K​(𝐪i)\mathcal{A}_{K}(\mathbf{q}_{i})to compute the precise late-interaction score as defined in Equation4. This ensures that the final ranking captures the precise semantic alignment using allKKfeatures while performing expensive computations on only a fraction of documents.

Table 2:Retrieval performance comparison when utilizing llama-embed-nemotron-8b(Babakhinet al.,2025)as backbone.The maximum values are indicated inbold, while the second-highest values areunderlined.ModelMSARCFCVDBFEFQHQNFNQQUSDSFTOAvg.llama-8B61.775.744.989.240.193.561.476.745.166.288.128.282.068.865.8Qwen3-8B62.476.946.091.338.892.764.677.041.364.687.829.778.473.066.0SFR-Embed59.067.228.387.627.386.660.473.840.334.888.919.976.255.257.5Linq-Embed60.569.630.387.127.887.661.275.441.141.989.221.976.454.558.9e5-mistral-7b59.161.728.587.034.687.056.873.233.466.488.516.375.155.458.8gte-Qwen2-7B50.354.632.280.447.390.662.075.040.465.687.923.579.559.260.6bge-large-v1.548.764.527.374.741.987.345.075.037.151.188.922.673.647.156.1SSR-tok61.275.445.589.641.593.962.177.246.067.787.528.181.869.466.4SSR-CLS62.076.246.490.143.394.662.877.946.868.488.629.582.970.567.1Complexity Analysis.The standard SSR scales withO​(N⋅K⋅L¯)O(N\cdot K\cdot\bar{L}), whereL¯\bar{L}is the average posting length. In contrast, SSR++ reduces the Stage I complexity toO​(N⋅Kcoarse⋅Lskip)O(N\cdot K_{\text{coarse}}\cdot L_{\text{skip}}), whereKcoarseK_{\text{coarse}}is significantly smaller thanKKandLskip≪L¯L_{\text{skip}}\ll\bar{L}due to block skipping. This efficiency gain is further supported by the bounded-distortion analysis in AppendixA, which shows that sparse active-neuron interactions can remain faithful to dense late interaction under suitable reconstruction and local decoder-geometry conditions.

4Experiments

4.1Benchmark performance

Evaluation under Controlled Setup.Under a controlled zero-shot setting, we compare SSR against representative state-of-the-art multi-vector dense retrievers (e.g., PLAID(Santhanamet al.,2022a)) and learned sparse retrievers (Splade-v2(Formalet al.,2021a)and Splade-v3(Lassanceet al.,2024)). All models are trained in-domain on MSMARCO(Nguyenet al.,2016)and evaluated on both MSMARCO and 13 BEIR(Thakuret al.,2021)datasets using nDCG@10 as the primary metric. As shown in Table1, SSR consistently delivers the best overall trade-off between effectiveness and efficiency. In particular, SSR-CLS achieves the highest average nDCG@10 of 53.4, outperforming the strongest sparse baseline Splade-v3 (51.2) and dense baseline PLAID (49.3), while SSR-tok reduces retrieval latency to 17.5ms, nearly 2× faster than ColBERTv2 and PLAID, yet still surpasses all baselines in average effectiveness. Moreover, SSR shows strong out-of-domain robustness, achieving the best results on 9 of 13 BEIR datasets. These results indicate that SSR effectively mitigates the conventional effectiveness-efficiency trade-off, and that its gains stem from the sparse coding mechanism itself rather than auxiliary token design alone. More details are in AppendixD.1.

Evaluation on Scalability to Modern Backbone.To examine whether SSR scales to modern large embedding backbones, we instantiate it on top of Llama-embed-nemotron-8b by freezing the backbone and training the Sparse Autoencoder on last-layer token embeddings, with the same MSMARCO training setup and sparsity constraint (K=32K=32). Table2compares SSR against leading open-source embedding models of similar scale on the MTEB(Muennighoffet al.,2022)leaderboard. The results show that SSR can effectively unlock the rich semantic capacity of modern LLM-based encoders: SSR-CLS achieves the best average score of 67.1, outperforming strong baselines such as Qwen3-Embedding-8B(Zhanget al.,2025b)and other competitive dense retrievers including e5-mistral-7b(Wanget al.,2023). Moreover, compared with the original Llama-embed-8B backbone, our sparse adaptation yields consistent gains across tasks, with particularly clear improvements on benchmarks requiring precise entity matching, such as DBpedia and FEVER. To address the concern that SSR may benefit simply from adding an extra trained module on top of a frozen backbone, we additionally train a ColBERT-style linear projector under the same frozen-backbone setting on two representative LLM embedding backbones. This control keeps the backbone fixed and only learns a lightweight projection layer. Results in Table3show that ColBERT-style linear projectors improve the frozen backbones only marginally, whereas SSR achieves larger gains, indicating that the improvement comes from the sparse coding objective and retrieval mechanism rather than merely from adding an extra layer. These findings also suggest that SSR is not limited to controlled BERT-scale settings, but generalizes effectively to high-dimensional modern foundation models. More details are in AppendixD.2.

Table 3:Frozen-backbone control for LLM-backbone experiments.All methods keep the LLM backbone frozen. The linear projector controls for the effect of adding an extra trainable projection layer.ModelAvg.Llama-8B65.8e5-mistral-7b58.8Llama-8B + linear66.1e5-mistral-7b + linear60.2Llama-8B + SSR-tok66.4Llama-8B + SSR-CLS67.1

(a)End-to-end analysis.

(b)Effect of data scale.

Figure 3:(Left): Efficiency analysis on the training (left), indexing (middle) and retrieval (right) phase in the end-to-end retrieval.(Right): Retrieval performance (left) and efficiency (right) comparison with ColBERTv2(Santhanamet al.,2022b)under different data scale.Evaluation on Robustness to Long-Tail Distributions.We further evaluate SSR’s capability to handle rare, domain-specific terminology with LoTTE(Santhanamet al.,2022b)benchmark. We find that across five corpus domains and two query sources, SSR outperforms the strongest representative baseline (i.e., ColBERTv2) with +2.5% and +2.2% on Search and Forum queries respectively. Notably, the performance gap widens in the more challenging ”Pooled” setting where models must retrieve relevant documents from a massive, multi-domain merged corpus, with SSR exceeding ColBERTv2 by a substantial +3.9% on Forum queries. More detailed setup and results are available in AppendixD.3.

Evaluation on Scalability to Long Document Sequences.To further evaluate SSR’s scalability and robustness to long sequences, we extend our evaluation to MSMARCO(Nguyenet al.,2016)document ranking subset. Compared to passage ranking set which is evaluated in Section4.1, it presents a significantly greater challenge due to much larger document length (average sentence length from less than 100 to 1131 tokens, with many exceeding 4096). Specifically, on MS MARCO document ranking, SSR-CLS achieves the best nDCG@10 of 48.8%, while SSR-tok delivers a competitive 48.3% nDCG@10 with only 27.5ms latency. SSR achieves both performance and latency superiority compared to various dense MVR frameworks. Detailed setup and results are available in AppendixD.4.

Stress Testing on Representational Bounds.To stress-test the representational bound of SSR, we evaluate it on the LIMIT diagnostic benchmark(Welleret al.,2025). Unlike standard evaluation sets that measure average-case retrieval accuracy, LIMIT is designed to systematically stress-test a model’s capacity to encode all possible top-kkdocument combinations within a given query space. We find that LIMIT exposes a severe representational bottleneck in standard single-vector embedding models, with Recall@5 below 5% even for state-of-the-art models such as the Qwen-Embedding series(Zhanget al.,2025b). By contrast, Multi-Vector Retrieval methods mitigate this bottleneck, and SSR establishes clear superiority, achieving 78.6% Recall@5 and 98.1% Recall@100, outperforming ColBERTv2 by 6.8 points on Recall@5. Detailed setup and results are available in AppendixD.5.

4.2Efficiency and Resource Analysis

End-to-End Efficiency Analysis.We conduct an end-to-end efficiency analysis of the multi-vector retrieval pipeline on the MSMARCO passage ranking dataset(Nguyenet al.,2016), comparing SSR with representative state-of-the-art systems, including COIL(Gaoet al.,2021), ColBERTv2 with PLAID acceleration(Santhanamet al.,2022b,a), XTR optimized by WARP(Leeet al.,2023; Scheereret al.,2025). All models are trained and evaluated using default hyper-parameters, with the sparsity level of SSR set toK=32K=32. For a fair end-to-end comparison, the reported indexing time includes the full offline index construction pipeline after training. For SSR, this covers corpus encoding, sparse projection, inverted-list construction, list partitioning, and final block upper-bound computation. For dense MVR baselines such as ColBERTv2/PLAID and XTR/WARP, it covers corpus encoding and the final centroid-based index construction, including clustering, residual/code computation, and auxiliary retrieval structures. As shown in Figure3(Left), SSR achieves substantial efficiency gains across training, indexing, and retrieval. In training, SSR-tok reduces the cost by approximately 53% compared with ColBERTv2, which requires 24.2 hours. The most significant advantage appears in indexing: while dense MVR systems such as ColBERTv2 and XTR suffer from the clustering bottleneck and require over 100 hours to process MSMARCO due to expensive K-means over billions of embeddings, SSR completes indexing in about 7.5 hours, yielding over 15×\timesspeedup. For online retrieval, SSR achieves sub-20ms latency, outperforming ColBERTv2 (37.1ms) and XTR (33.4ms). Although COIL is slightly faster (12.6ms), SSR offers a better balance by preserving high-precision late-interaction semantic matching while substantially improving throughput.

(a)Hidden dimension.

(b)Sparsity level.

Figure 4:(Left): Effect of hidden dimensionℝh\mathbb{R}^{h}on performance.(Right): Effect of sparsity constraintKKon retrieval performance.CPU-Only Retrieval Efficiency.We compare SSR’s index and retrieval time compared to state-of-the-art baselines, including classic methods such as PLAID(Santhanamet al.,2022a)and recent algorithms based on ColBERT, such as IGP(Bianet al.,2025). Evaluation on MSMARCO passage ranking shows that SSR achieves superior efficiency compared to these modern MVR engines, with about 30%/85% retrieval/index time gain compared to the most efficient engine DESSERT(Engelset al.,2023). Moreover, SSR does not gain speed by sacrificing retrieval quality, resulting in the best MRR@10 among all compared methods. Experiment setup and more results are available in AppendixE.2.

System-Level Resource Footprint.Apart from effectiveness and efficiency, system-level resource consumption is another significant factor to evaluate MVR methods. We measure the systematic consumption along three explicit dimensions: (1) Build-time cost; (2) Serving-time footprint; (3) Maintenance/update cost. Table4summarizes the system-level footprint. Compared with clustering-based dense MVR engines, SSR substantially lowers peak build memory, reducing it from 274.2 GB for ColBERTv2/PLAID and 186.7 GB for XTR/WARP to 34.6 GB for SSR-tok and 55.1 GB for SSR-CLS. SSR also keeps a smaller persistent main index of 18.5 GB and supports append-only updates, since new documents only require sparse projection and posting-list insertion rather than rebuilding clustering structures. This update property is particularly important for dynamic retrieval corpora, where documents are frequently added or refreshed. Detailed experiment setups can be found in AppendixE.3.

Table 4:System-level resource breakdown.SSR avoids clustering-based rebuilds and supports append-only index updates.MethodPeak mem.IndexUpdate(GB)(GB)modeColBERTv2/PLAID274.222.1rebuildXTR/WARP186.755.9rebuildSSR-tok34.618.5append-onlySSR-CLS55.118.5append-only

4.3Empirical Analysis

Ablations.We conduct ablation experiments on weights of different loss terms with sparsity levelK=32K=32during training:α\alphafor auxiliary loss,β\betafor sparse contrastive loss andγ\gammafor supervised contrastive loss. We find that each loss term leads to performance improvement, with supervised contrastive loss resulting in the most significant performance improvement due to its promotion for understanding semantic difference between positive and negative documents. Additionally, we conduct experiments on MSMARCO passage ranking subset to isolate SSR++’s acceleration effect. Table5shows that SSR++ reduces the number of hit candidates from 54,278 to 3,196 and cuts retrieval latency from 38.6 ms to 17.5 ms, while maintaining essentially the same retrieval quality, with nDCG@10 changing only from 45.3 to 45.2. This shows that the coarse-to-fine sparse pruning strategy improves efficiency without introducing a meaningful effectiveness loss. More detailed results are presented in AppendixE.1.

Table 5:Ablation on the SSR++’s acceleration strategy.Evaluation is conducted on the MS MARCO passage ranking subset.MethodCandidatesLatency (ms)[email protected]++319617.545.2Performance Under Different Data Scale.To assess the effect of corpus scale on retrieval robustness and latency, we conduct a controlled MSMARCO evaluation by varying the collection size with five stratified subsets,N∈{0.5​M,1​M,2​M,5​M,10​M}N\in\{0.5\text{M},1\text{M},2\text{M},5\text{M},10\text{M}\}, randomly sampled from the full corpus. For validity, we retain only queries whose positive passages are included in each subset, and fix the sparsity level atK=32K=32for all runs. As shown in Figure3, larger corpora degrade performance for all models, but SSR is more robust than ColBERTv2: from 0.5M to 10M documents, SSR-tok drops by only 5.3%, compared with ColBERTv2’s 8.9% decline. In terms of efficiency, SSR’s inverted index-based pruning yields increasing latency advantages asNNgrows, demonstrating its scalability for large-scale retrieval.

Effect of Hidden Dimensionℝh\mathbb{R}^{h}.We study how SAE’s hidden dimensionhhaffects retrieval quality and latency by sweepinghhfrom2122^{12}to2162^{16}, with sparsity fixed atK=32K=32and all other hyper-parameters kept unchanged. The model is trained on MSMARCO(Nguyenet al.,2016)with a BERT-base-uncased backbone and evaluated on BEIR following Table1. As Figure4(a)shows, increasinghhcreates a trade-off between effectiveness and efficiency. Retrieval performance follows an inverted-U trend, peaking ath=214h=2^{14}. This suggests that moderate overcompleteness provides enough capacity for the sparse codes to preserve fine-grained token interactions under the fixed Top-KKconstraint, whereas an overly large hidden space makes the active supports more diffuse and less consistently shared across semantically related query-document pairs. This behavior also highlights a key difference between sparse late interaction and dense over-parameterization. In SSR, retrieval quality depends not only on the expressiveness of individual sparse codes, but also on whether semantically related query and document tokens activate overlapping supports. Whenhhbecomes too large under a fixedKK, the activation space is fragmented: related tokens may be assigned to different neurons, reducing useful overlap in the inverted index. Therefore, the optimal hidden dimension balances feature disentanglement and support sharing.

Sensitivity to Sparsity ConstraintKK.We study SSR’s sensitivity to the sparsity constraintKKby sweepingKKfrom 8 to 128, using BERT-base-uncased as the backbone with a fixed hidden dimension ofh=16384h=16384and keeping all other hyperparameters unchanged. As shown in Figure4(b), performance exhibits a turning point atK=32K=32: largerKKvalues bring only marginal gains, whereas reducingKKbelow 16 causes substantial degradation, likely due to the loss of fine-grained semantic information. Interestingly, the model relies more heavily on the global [CLS] token whenKKbecomes smaller, indicating that global context helps compensate for aggressively pruned token-level features. From the efficiency perspective, smallerKKvalues significantly reduce latency, mainly because they lower the cost of similarity scoring and induce a much sparser inverted index. This trade-off further motivates adaptive sparsity selection, where the model can allocate more active neurons only when queries or domains require finer-grained matching.

4.4Further Discussions

Adaptive Query-based Sparsity Control.Since queries with different lengths may require different levels of semantic granularity, we further explore whether query sparsity can be adaptively controlled by query length. The adaptive strategy achieves comparable performance to fixedK=64K=64(53.0% vs. 53.1%) while reducing latency from 19.9 ms to 16.3 ms, slightly improving the performance-efficiency Pareto frontier. More detailed settings and results are provided in AppendixF.1.

Sweet Spot on Sparsity across Domains.We also investigate whether the optimal sparsity level varies across domains. Overall,K=32K=32is a stable choice: for example, it reaches 67.7% on fact-oriented tasks and 65.1% on multi-hop tasks. IncreasingKKto 64 brings limited gains in most domains, but improves multi-hop retrieval from 65.1% to 66.5%. These observations suggest that sparsity in SSR should be viewed not merely as a fixed compression knob, but as a controllable interface for adapting retrieval granularity to query and domain complexity. We provide the full domain grouping and detailed comparison in AppendixF.2.

5Conclusion & Discussion

We introducedSingle-stage Sparse Retrieval, a multi-vector retrieval framework that replaces the computational bottleneck of dense clustering with efficient sparse autoencoding. By disentangling complex token semantics into high-dimensional sparse activations, SSR enables direct neuron-level inverted indexing, thereby optimizing the trade-off between indexing speed and retrieval precision. This approach is effective across both standard discriminative encoders and modern language model backbones, establishing sparse coding as a scalable alternative for next-generation retrieval systems. We believe SSR opens up a practical path toward systems that combine the semantic fidelity of multi-vector interaction with the operational simplicity of sparse inverted indexing, enabling more efficient deployment in large-scale corpora. Future work may further explore hardware-aware index layouts to improve the robustness of SSR across heterogeneous retrieval workloads.

Impact Statement

This work aims to advance efficient large-scale retrieval in machine learning by removing the clustering bottleneck in indexing and improving empirical indexing and retrieval efficiency. These improvements may reduce the computational cost of deploying retrieval systems and make high-throughput retrieval more accessible in practical applications. However, SSR also involves important system-level trade-offs. In particular, it relies on high-dimensional sparse representations and neuron-level inverted lists, which may incur nontrivial memory and storage overhead in very large-scale deployments. The actual efficiency gains may further depend on factors such as hidden dimensionality, sparsity level, posting-list length, auxiliary index structures, hardware configuration, memory bandwidth, cache behavior, inverted-index layout, and implementation-level optimizations. Therefore, the reported latency and throughput improvements should be interpreted as empirical results under our implementation and evaluation setup, rather than hardware-independent guarantees.

However, more efficient retrieval systems may also introduce broader societal risks if deployed without appropriate safeguards. Like other high-throughput retrieval models, SSR could be used in applications that amplify existing ranking biases, accelerate the retrieval or dissemination of harmful content, or support surveillance-oriented use cases. These risks are not unique to SSR, but increased retrieval efficiency may make harmful or biased retrieval pipelines easier to scale. Responsible deployment therefore requires careful evaluation beyond standard efficiency metrics, including bias auditing, monitoring of downstream ranking behavior, content-governance mechanisms, and safeguards appropriate to the application domain.

References

• Y. Babakhin, R. Osmulski, R. Ak, G. Moreira, M. Xu, B. Schifferer, B. Liu, and E. Oldridge (2025)Llama-embed-nemotron-8b: a universal text embedding model for multilingual and cross-lingual tasks.External Links:2511.07025,LinkCited by:Appendix B,§1,Table 2,Table 2.
• Z. Bian, M. L. Yiu, and B. Tang (2025)IGP: efficient multi-vector retrieval via proximity graph index.InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp. 2524–2533.Cited by:Appendix B,§E.2,§4.2.
• A. Bondarenko, M. Fröbe, M. Beloucif, L. Gienapp, Y. Ajjour, A. Panchenko, C. Biemann, B. Stein, H. Wachsmuth, M. Potthast,et al.(2020)Overview of touché 2020: argument retrieval.InInternational Conference of the Cross-Language Evaluation Forum for European Languages,pp. 384–395.Cited by:14th item.
• V. Boteva, D. Gholipour, A. Sokolov, and S. Riezler (2016)A full-text learning to rank dataset for medical information retrieval.InAdvances in Information Retrieval,N. Ferro, F. Crestani, M. Moens, J. Mothe, F. Silvestri, G. M. Di Nunzio, C. Hauff, and G. Silvello (Eds.),Cham,pp. 716–722.Cited by:8th item.
• B. Bussmann, P. Leask, and N. Nanda (2024)Batchtopk sparse autoencoders.arXiv preprint arXiv:2412.06410.Cited by:Appendix B.
• Y. Cao, L. Yang, C. Wang, Z. Liu, H. Peng, C. You, and P. S. Yu (2023)Multi-task item-attribute graph pre-training for strict cold-start item recommendation.InProceedings of the 17th ACM conference on recommender systems,Cited by:Appendix B,§1.
• C. Choi, J. Kim, S. Lee, J. Kwon, S. Gu, Y. Kim, M. Cho, and J. Sohn (2024)Linq-embed-mistral technical report.arXiv preprint arXiv:2412.03223.Cited by:§D.2.
• A. Cohan, S. Feldman, I. Beltagy, D. Downey, and D. S. Weld (2020)Specter: document-level representation learning using citation-informed transformers.arXiv preprint arXiv:2004.07180.Cited by:11st item.
• H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey (2023)Sparse autoencoders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600.Cited by:Appendix B.
• J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)Bert: pre-training of deep bidirectional transformers for language understanding.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers),pp. 4171–4186.Cited by:§D.1.
• L. Dhulipala, M. Hadian, R. Jayaram, J. Lee, and V. Mirrokni (2024)Muvera: multi-vector retrieval via fixed dimensional encodings.arXiv preprint arXiv:2405.19504.Cited by:Appendix B,§E.2.
• T. Diggelmann, J. Boyd-Graber, J. Bulian, M. Ciaramita, and M. Leippold (2020)Climate-fever: a dataset for verification of real-world climate claims.arXiv preprint arXiv:2012.00614.Cited by:2nd item.
• M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P. Mazaré, M. Lomeli, L. Hosseini, and H. Jégou (2024)The faiss library.arXiv preprint arXiv:2401.08281.External Links:2401.08281Cited by:§2.2.
• J. Engels, B. Coleman, V. Lakshman, and A. Shrivastava (2023)DESSERT: an efficient algorithm for vector set search with vector set queries.Advances in Neural Information Processing Systems36,pp. 67972–67992.Cited by:Appendix B,§E.2,§4.2.
• T. Formal, C. Lassance, B. Piwowarski, and S. Clinchant (2021a)SPLADE v2: sparse lexical and expansion model for information retrieval.arXiv preprint arXiv:2109.10086.Cited by:Appendix B,2nd item,§4.1.
• T. Formal, B. Piwowarski, and S. Clinchant (2021b)SPLADE: sparse lexical and expansion model for first stage ranking.InProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval,pp. 2288–2292.Cited by:Appendix B.
• L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, A. Radford, I. Sutskever, J. Leike, and J. Wu (2024)Scaling and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093.Cited by:Appendix B,§1,§3.2.
• L. Gao, Z. Dai, and J. Callan (2021)COIL: revisit exact lexical match in information retrieval with contextualized inverted list.arXiv preprint arXiv:2104.07186.Cited by:Appendix B,1st item,§D.4,§3.2,§4.2.
• L. Guo, Y. Wang, T. Wen, Y. Wang, A. Feng, B. Chen, S. Jegelka, and C. You (2026)CSRv2: unlocking ultra-sparse embeddings.InInternational Conference on Learning Representations (ICLR),Cited by:Appendix B,Appendix B,§3.2.
• F. Hasibi, F. Nikolaev, C. Xiong, K. Balog, S. E. Bratsberg, A. Kotov, and J. Callan (2017)DBpedia-entity v2: a test collection for entity search.InProceedings of the 40th international ACM SIGIR conference on research and development in information retrieval,pp. 1265–1268.Cited by:3rd item.
• Z. Ji, H. Jain, A. Veit, S. J. Reddi, S. Jayasumana, A. S. Rawat, A. K. Menon, F. Yu, and S. Kumar (2024)Efficient document ranking with learnable late interactions.arXiv preprint arXiv:2406.17968.Cited by:Appendix B.
• O. Khattab and M. Zaharia (2020)ColBERT: efficient and effective passage search via contextualized late interaction over bert.InProceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval,pp. 39–48.Cited by:Appendix B,1st item,§1,§3.2.
• A. Kusupati, G. Bhatt, A. Rege, M. Wallingford, A. Sinha, V. Ramanujan, W. Howard-Snyder, K. Chen, S. Kakade, P. Jain,et al.(2022)Matryoshka representation learning.InAdvances in Neural Information Processing Systems,Cited by:Appendix B.
• T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee,et al.(2019)Natural questions: a benchmark for question answering research.Transactions of the Association for Computational Linguistics7,pp. 453–466.Cited by:9th item.
• C. Lassance, H. Déjean, T. Formal, and S. Clinchant (2024)SPLADE-v3: new baselines for splade.External Links:2403.06789Cited by:Appendix B,2nd item,§4.1.
• C. Lee, R. Roy, M. Xu, J. Raiman, M. Shoeybi, B. Catanzaro, and W. Ping (2024)Nv-embed: improved techniques for training llms as generalist embedding models.arXiv preprint arXiv:2405.17428.Cited by:Appendix B.
• H. Lee, A. Battle, R. Raina, and A. Ng (2006)Efficient sparse coding algorithms.Advances in neural information processing systems19.Cited by:Appendix B,§1.
• J. Lee, Z. Dai, S. M. K. Duddu, T. Lei, I. Naim, M. Chang, and V. Zhao (2023)Rethinking the role of token retrieval in multi-vector retrieval.Advances in Neural Information Processing Systems36,pp. 15384–15405.Cited by:Appendix B,1st item,§D.4,§3.2,§4.2.
• M. Li, S. Lin, B. Oguz, A. Ghoshal, J. Lin, Y. Mehdad, W. Yih, and X. Chen (2023a)CITADEL: conditional token interaction via dynamic lexical routing for efficient and effective multi-vector retrieval.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),pp. 11891–11907.Cited by:Appendix B,1st item.
• Z. Li, X. Zhang, Y. Zhang, D. Long, P. Xie, and M. Zhang (2023b)Towards general text embeddings with multi-stage contrastive learning.arXiv preprint arXiv:2308.03281.Cited by:§D.2.
• F. Liu, X. Wu, C. You, S. Ge, Y. Zou, and X. Sun (2021)Aligning source visual and target language domains for unpaired video captioning.IEEE Transactions on Pattern Analysis and Machine Intelligence.Cited by:Appendix B.
• F. Liu, B. Yang, C. You, X. Wu, S. Ge, Z. Liu, X. Sun, Y. Yang, and D. Clifton (2022)Retrieve, reason, and refine: generating accurate and faithful patient instructions.Advances in Neural Information Processing Systems.Cited by:Appendix B.
• M. Maia, S. Handschuh, A. Freitas, B. Davis, R. McDermott, M. Zarrouk, and A. Balahur (2018)Www’18 open challenge: financial opinion mining and question answering.InCompanion proceedings of the the web conference 2018,pp. 1941–1942.Cited by:5th item.
• D. Min, Z. Xu, G. Qi, L. Huang, and C. You (2025)UniHGKR: unified instruction-aware heterogeneous knowledge retrievers.InProceedings of the Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies,pp. 4577–4594.Cited by:Appendix B.
• N. Muennighoff, N. Tazi, L. Magne, and N. Reimers (2022)MTEB: massive text embedding benchmark.arXiv preprint arXiv:2210.07316.External Links:Link,DocumentCited by:Appendix B,§D.2,§4.1.
• F. M. Nardini, C. Rulli, and R. Venturini (2024)Efficient multi-vector dense retrieval using bit vectors.arXiv preprint arXiv:2404.02805.Cited by:Appendix B,§E.2,§2.2.
• T. Nguyen, M. Rosenberg, X. Song, J. Gao, S. Tiwary, R. Majumder, and L. Deng (2016)Ms marco: a human-generated machine reading comprehension dataset.arXiv:1611.09268.Cited by:7th item,§D.1,§1,§4.1,§4.1,§4.2,§4.3.
• L. Peng, Y. Zhang, Z. Wang, J. Srinivasa, G. Liu, Z. Wang, and J. Shang (2024)Answer is all you need: instruction-following text embedding via answering the question.arXiv preprint arXiv:2402.09642.Cited by:Appendix B.
• Y. Qian, J. Lee, S. M. K. Duddu, Z. Dai, S. Brahma, I. Naim, T. Lei, and V. Y. Zhao (2022)Multi-vector retrieval as sparse alignment.arXiv preprint arXiv:2211.01267.Cited by:Appendix B,1st item.
• S. Rajamanoharan, A. Conmy, L. Smith, T. Lieberum, V. Varma, J. Kramár, R. Shah, and N. Nanda (2024a)Improving dictionary learning with gated sparse autoencoders.arXiv preprint arXiv:2404.16014.Cited by:Appendix B.
• S. Rajamanoharan, T. Lieberum, N. Sonnerat, A. Conmy, V. Varma, J. Kramár, and N. Nanda (2024b)Jumping ahead: improving reconstruction fidelity with jumprelu sparse autoencoders.arXiv preprint arXiv:2407.14435.Cited by:Appendix B.
• S. E. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, M. Gatford,et al.(1995)Okapi at trec-3.British Library Research and Development Department.Cited by:§D.1,§1.
• S. R. J. Rui Meng, C. Xiong, and S. Y. Yingbo Zhou (2024)SFR-embedding-mistral:enhance text retrieval with transfer learning.Note:Salesforce AI Research BlogExternal Links:LinkCited by:§D.2.
• K. Santhanam, O. Khattab, C. Potts, and M. Zaharia (2022a)PLAID: an efficient engine for late interaction retrieval.InProceedings of the 31st ACM International Conference on Information & Knowledge Management,pp. 1747–1756.Cited by:Appendix B,1st item,§E.2,§E.3,§1,§1,§2.2,§4.1,§4.2,§4.2.
• K. Santhanam, O. Khattab, J. Saad-Falcon, C. Potts, and M. Zaharia (2022b)ColBERTv2: effective and efficient retrieval via lightweight late interaction.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp. 3715–3734.Cited by:Appendix B,15th item,Appendix C,1st item,§D.3,§D.3,§D.4,Table 8,Table 8,Figure 2,Figure 2,§2.2,Figure 3,Figure 3,§4.1,§4.2.
• J. L. Scheerer, M. Zaharia, C. Potts, G. Alonso, and O. Khattab (2025)WARP: an efficient engine for multi-vector retrieval.InProceedings of the 48th international ACM SIGIR conference on research and development in information retrieval,pp. 2504–2512.Cited by:Appendix B,§E.2,§E.3,§4.2.
• S. Sturua, I. Mohr, M. K. Akram, M. Günther, B. Wang, M. Krimmel, F. Wang, G. Mastrapas, A. Koukounas, N. Wang,et al.(2024)Jina-embeddings-v3: multilingual embeddings with task lora.arXiv preprint arXiv:2409.10173.Cited by:Appendix B.
• A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones, H. Cunningham, N. L. Turner, C. McDougall, M. MacDiarmid, A. Tamkin, E. Durmus, T. Hume, F. Mosconi, C. D. Freeman, T. R. Sumers, E. Rees, J. Batson, A. Jermyn, S. Carter, C. Olah, and T. Henighan (2024)Scaling monosemanticity: extracting interpretable features from claude 3 sonnet.Transformer Circuits Thread.External Links:LinkCited by:Appendix B.
• N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych (2021)Beir: a heterogenous benchmark for zero-shot evaluation of information retrieval models.arXiv preprint arXiv:2104.08663.Cited by:10th item,Appendix C,§D.1,§1,§4.1.
• J. Thorne, A. Vlachos, C. Christodoulopoulos, and A. Mittal (2018)FEVER: a large-scale dataset for fact extraction and verification.arXiv preprint arXiv:1803.05355.Cited by:4th item.
• J. Veneroso, R. Jayaram, J. Rao, G. H. Ábrego, M. Hadian, and D. Cer (2025)CRISP: clustering multi-vector representations for denoising and pruning.arXiv preprint arXiv:2505.11471.Cited by:Appendix B.
• E. Voorhees, T. Alam, S. Bedrick, D. Demner-Fushman, W. R. Hersh, K. Lo, K. Roberts, I. Soboroff, and L. L. Wang (2021)TREC-covid: constructing a pandemic information retrieval test collection.InACM SIGIR Forum,Vol.54,pp. 1–12.Cited by:13rd item.
• H. Wachsmuth, S. Syed, and B. Stein (2018)Retrieval of the best counterargument without prior topic knowledge.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),pp. 241–251.Cited by:1st item.
• D. Wadden, S. Lin, K. Lo, L. L. Wang, M. van Zuylen, A. Cohan, and H. Hajishirzi (2020)Fact or fiction: verifying scientific claims.arXiv preprint arXiv:2004.14974.Cited by:12nd item.
• L. Wang, N. Yang, X. Huang, L. Yang, R. Majumder, and F. Wei (2023)Improving text embeddings with large language models.arXiv preprint arXiv:2401.00368.Cited by:Appendix B,§D.2,§4.1.
• Y. Wang, Q. Zhang, Y. Guo, and Y. Wang (2024)Non-negative contrastive learning.arXiv preprint arXiv:2403.12459.Cited by:§3.2.
• J. Wei, X. Guo, P. Yu, X. Zhang, W. Ouyang, S. Sun, Q. Wang, and C. You (2026)When to think, when to speak: learning disclosure policies for llm reasoning.arXiv preprint arXiv:2605.03314.Cited by:Appendix B.
• O. Weller, M. Boratko, I. Naim, and J. Lee (2025)On the theoretical limitations of embedding-based retrieval.arXiv preprint arXiv:2508.21038.Cited by:16th item,Appendix C,§4.1.
• T. Wen, Y. Wang, Z. Zeng, Z. Peng, Y. Su, X. Liu, B. Chen, H. Liu, S. Jegelka, and C. You (2025)Beyond matryoshka: revisiting sparse coding for adaptive representation.InInternational Conference on Machine Learning,Cited by:Appendix B,Appendix B,§3.2.
• S. Xiao, Z. Liu, P. Zhang, and N. Muennighoff (2023)C-pack: packaged resources to advance general chinese embedding.External Links:2309.07597Cited by:§D.2.
• Y. Xie, T. Wen, T. Huang, B. Chen, C. You, S. Jegelka, and Y. Wang (2026)Scaling attention via feature sparsity.arXiv preprint arXiv:2603.22300.Cited by:Appendix B.
• Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning (2018)HotpotQA: a dataset for diverse, explainable multi-hop question answering.InProceedings of the 2018 conference on empirical methods in natural language processing,pp. 2369–2380.Cited by:6th item.
• C. You, H. Dai, Y. Min, J. S. Sekhon, S. Joshi, and J. S. Duncan (2025)Uncovering memorization effect in the presence of spurious correlations.Nature Communications16(1),pp. 5424.Cited by:Appendix B.
• C. You, Y. Mint, W. Dai, J. S. Sekhon, L. Staib, and J. S. Duncan (2024)Calibrating multi-modal representations: a pursuit of group robustness without annotations.InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),pp. 26140–26150.Cited by:Appendix B.
• S. Zhang, Y. Lin, and H. Li (2025a)Memory retrieval and consolidation in large language models through function tokens.arXiv preprint arXiv:2510.08203.Cited by:Appendix B.
• X. Zhang, J. Cao, and C. You (2024)Counting ability of large language models and impact of tokenization.arXiv preprint arXiv:2410.19730.Cited by:Appendix B.
• Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, F. Huang, and J. Zhou (2025b)Qwen3 embedding: advancing text embedding and reranking through foundation models.arXiv preprint arXiv:2506.05176.Cited by:Appendix B,§D.2,§4.1,§4.1.
• X. Zhao, X. Hu, Z. Shan, S. Huang, Y. Zhou, X. Zhang, Z. Sun, Z. Liu, D. Li, X. Wei,et al.(2025)Kalm-embedding-v2: superior training techniques and data inspire a versatile embedding model.arXiv preprint arXiv:2506.20923.Cited by:Appendix B.
• P. Zhou, Q. Ye, Y. Xie, J. Gao, S. Wang, J. B. Kim, C. You, and S. Kim (2023)Attention calibration for transformer-based sequential recommendation.InProceedings of the ACM international conference on information and knowledge management,Cited by:Appendix B,§1.

Appendix AOn the Bounded-Distortion Property of SSR

Setup.We provide a theoretical characterization of when Single-stage Sparse Retrieval (SSR) preserves the semantic discriminability of dense late interaction. We focus first on a pair of token embeddings, which serves as the minimal unit of late-interaction scoring, and then extend the result to the MaxSim score used in multi-vector retrieval.

Let𝐱,𝐲∈ℝd\mathbf{x},\mathbf{y}\in\mathbb{R}^{d}denote two dense token embeddings. Let𝐳x,𝐳y∈ℝh\mathbf{z}_{x},\mathbf{z}_{y}\in\mathbb{R}^{h}be their Top-KKsparse SAE codes. The SAE decoder reconstructs

𝐱^=𝐖dec​𝐳x+𝐛pre,𝐲^=𝐖dec​𝐳y+𝐛pre.\hat{\mathbf{x}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{x}+\mathbf{b}_{\mathrm{pre}},\quad\hat{\mathbf{y}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{y}+\mathbf{b}_{\mathrm{pre}}.(13)Without loss of generality, we analyze the centered representation and absorb𝐛pre\mathbf{b}_{\mathrm{pre}}into the embeddings. Thus,

𝐱^=𝐖dec​𝐳x,𝐲^=𝐖dec​𝐳y.\hat{\mathbf{x}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{x},\quad\hat{\mathbf{y}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{y}.(14) For queryQ={𝐪1,…,𝐪N}Q=\{\mathbf{q}_{1},\ldots,\mathbf{q}_{N}\}and documentD={𝐝1,…,𝐝M}D=\{\mathbf{d}_{1},\ldots,\mathbf{d}_{M}\}, the dense late-interaction score is

Sdense​(Q,D)=∑i=1Nmax1≤j≤M⁡𝐪i⊤​𝐝j.S_{\mathrm{dense}}(Q,D)=\sum_{i=1}^{N}\max_{1\leq j\leq M}\mathbf{q}_{i}^{\top}\mathbf{d}_{j}.(15)The corresponding sparse SSR score is

SSSR​(Q,D)=∑i=1Nmax1≤j≤M⁡𝐳qi⊤​𝐳dj.S_{\mathrm{SSR}}(Q,D)=\sum_{i=1}^{N}\max_{1\leq j\leq M}\mathbf{z}_{q_{i}}^{\top}\mathbf{z}_{d_{j}}.(16)Since the sparse codes are Top-KKactivated codes, the sparse inner product is equivalently computed only over overlapping activated neurons:

𝐳qi⊤​𝐳dj=∑u∈𝒜K​(𝐳qi)∩𝒜K​(𝐳dj)𝐳qi(u)​𝐳dj(u).\mathbf{z}_{q_{i}}^{\top}\mathbf{z}_{d_{j}}=\sum_{u\in\mathcal{A}_{K}(\mathbf{z}_{q_{i}})\cap\mathcal{A}_{K}(\mathbf{z}_{d_{j}})}\mathbf{z}_{q_{i}}^{(u)}\mathbf{z}_{d_{j}}^{(u)}.(17) A sufficient regime.The following assumptions describe a sufficient local regime under which SSR is a bounded-distortion approximation to dense late interaction.

Assumption 1. Reconstruction fidelity.For every dense token embedding𝐱\mathbf{x}and its sparse reconstruction𝐱^=𝐖dec​𝐳x\hat{\mathbf{x}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{x},

‖𝐱−𝐱^‖2≤ϵ.\|\mathbf{x}-\hat{\mathbf{x}}\|_{2}\leq\epsilon.(18)Equivalently, for two token embeddings𝐱\mathbf{x}and𝐲\mathbf{y},

‖𝐱−𝐱^‖2≤ϵ,‖𝐲−𝐲^‖2≤ϵ.\|\mathbf{x}-\hat{\mathbf{x}}\|_{2}\leq\epsilon,\quad\|\mathbf{y}-\hat{\mathbf{y}}\|_{2}\leq\epsilon.(19) Assumption 2. Bounded dense-token norm.Dense token embeddings are uniformly bounded:

‖𝐱‖2≤B,‖𝐲‖2≤B.\|\mathbf{x}\|_{2}\leq B,\quad\|\mathbf{y}\|_{2}\leq B.(20) Assumption 3. Restricted decoder near-orthogonality.Let

S=supp⁡(𝐳x)∪supp⁡(𝐳y)S=\operatorname{supp}(\mathbf{z}_{x})\cup\operatorname{supp}(\mathbf{z}_{y})(21)be the union of active sparse coordinates. The decoder isδ\delta-near-orthogonal on this active support:

‖𝐖dec⊤​𝐖dec−𝐈‖S→S≤δ.\left\|\mathbf{W}_{\mathrm{dec}}^{\top}\mathbf{W}_{\mathrm{dec}}-\mathbf{I}\right\|_{S\rightarrow S}\leq\delta.(22)That is, for all sparse vectors𝐚\mathbf{a}and𝐛\mathbf{b}supported onSS,

|𝐚⊤​(𝐖dec⊤​𝐖dec−𝐈)​𝐛|≤δ​‖𝐚‖2​‖𝐛‖2.\left|\mathbf{a}^{\top}\left(\mathbf{W}_{\mathrm{dec}}^{\top}\mathbf{W}_{\mathrm{dec}}-\mathbf{I}\right)\mathbf{b}\right|\leq\delta\|\mathbf{a}\|_{2}\|\mathbf{b}\|_{2}.(23) Assumption 4. Bounded sparse-code norm.When extending the result to full query-document scoring, we assume the sparse codes are uniformly bounded:

‖𝐳x‖2≤C,‖𝐳y‖2≤C.\|\mathbf{z}_{x}\|_{2}\leq C,\quad\|\mathbf{z}_{y}\|_{2}\leq C.(24)This assumption is used only to obtain a uniform late-interaction bound.

Theorem A(Token-level bounded distortion). UnderAssumptions 1–3, the dense token inner product and the SSR sparse inner product satisfy

|𝐱⊤​𝐲−𝐳x⊤​𝐳y|≤2​B​ϵ+ϵ2+δ​‖𝐳x‖2​‖𝐳y‖2.\left|\mathbf{x}^{\top}\mathbf{y}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|\leq 2B\epsilon+\epsilon^{2}+\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}.(25)IfAssumption 4also holds, then

|𝐱⊤​𝐲−𝐳x⊤​𝐳y|≤2​B​ϵ+ϵ2+δ​C2.\left|\mathbf{x}^{\top}\mathbf{y}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|\leq 2B\epsilon+\epsilon^{2}+\delta C^{2}.(26) Proof.We decompose the approximation error into a reconstruction term and a decoder-geometry term:

|𝐱⊤​𝐲−𝐳x⊤​𝐳y|\displaystyle\left|\mathbf{x}^{\top}\mathbf{y}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|≤|𝐱⊤​𝐲−𝐱^⊤​𝐲^|+|𝐱^⊤​𝐲^−𝐳x⊤​𝐳y|.\displaystyle\leq\left|\mathbf{x}^{\top}\mathbf{y}-\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}\right|+\left|\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|.(27) We first bound the reconstruction term. Let

𝐞x=𝐱−𝐱^,𝐞y=𝐲−𝐲^.\mathbf{e}_{x}=\mathbf{x}-\hat{\mathbf{x}},\quad\mathbf{e}_{y}=\mathbf{y}-\hat{\mathbf{y}}.(28)Then𝐱^=𝐱−𝐞x\hat{\mathbf{x}}=\mathbf{x}-\mathbf{e}_{x}and𝐲^=𝐲−𝐞y\hat{\mathbf{y}}=\mathbf{y}-\mathbf{e}_{y}. Therefore,

𝐱⊤​𝐲−𝐱^⊤​𝐲^\displaystyle\mathbf{x}^{\top}\mathbf{y}-\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}=𝐱⊤​𝐲−(𝐱−𝐞x)⊤​(𝐲−𝐞y)\displaystyle=\mathbf{x}^{\top}\mathbf{y}-(\mathbf{x}-\mathbf{e}_{x})^{\top}(\mathbf{y}-\mathbf{e}_{y})(29)=𝐱⊤​𝐞y+𝐞x⊤​𝐲−𝐞x⊤​𝐞y.\displaystyle=\mathbf{x}^{\top}\mathbf{e}_{y}+\mathbf{e}_{x}^{\top}\mathbf{y}-\mathbf{e}_{x}^{\top}\mathbf{e}_{y}.By Cauchy-Schwarz,

|𝐱⊤​𝐲−𝐱^⊤​𝐲^|\displaystyle\left|\mathbf{x}^{\top}\mathbf{y}-\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}\right|≤‖𝐱‖2​‖𝐞y‖2+‖𝐞x‖2​‖𝐲‖2+‖𝐞x‖2​‖𝐞y‖2.\displaystyle\leq\|\mathbf{x}\|_{2}\|\mathbf{e}_{y}\|_{2}+\|\mathbf{e}_{x}\|_{2}\|\mathbf{y}\|_{2}+\|\mathbf{e}_{x}\|_{2}\|\mathbf{e}_{y}\|_{2}.(30)UsingAssumptions 1–2, we obtain

|𝐱⊤​𝐲−𝐱^⊤​𝐲^|≤2​B​ϵ+ϵ2.\left|\mathbf{x}^{\top}\mathbf{y}-\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}\right|\leq 2B\epsilon+\epsilon^{2}.(31) We now bound the decoder-geometry term. Since

𝐱^=𝐖dec​𝐳x,𝐲^=𝐖dec​𝐳y,\hat{\mathbf{x}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{x},\quad\hat{\mathbf{y}}=\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{y},(32)we have

𝐱^⊤​𝐲^=𝐳x⊤​𝐖dec⊤​𝐖dec​𝐳y.\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}=\mathbf{z}_{x}^{\top}\mathbf{W}_{\mathrm{dec}}^{\top}\mathbf{W}_{\mathrm{dec}}\mathbf{z}_{y}.(33)Thus,

𝐱^⊤​𝐲^−𝐳x⊤​𝐳y=𝐳x⊤​(𝐖dec⊤​𝐖dec−𝐈)​𝐳y.\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}=\mathbf{z}_{x}^{\top}\left(\mathbf{W}_{\mathrm{dec}}^{\top}\mathbf{W}_{\mathrm{dec}}-\mathbf{I}\right)\mathbf{z}_{y}.(34)ByAssumption 3,

|𝐱^⊤​𝐲^−𝐳x⊤​𝐳y|≤δ​‖𝐳x‖2​‖𝐳y‖2.\left|\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|\leq\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}.(35)Combining the two bounds gives

|𝐱⊤​𝐲−𝐳x⊤​𝐳y|≤2​B​ϵ+ϵ2+δ​‖𝐳x‖2​‖𝐳y‖2.\left|\mathbf{x}^{\top}\mathbf{y}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|\leq 2B\epsilon+\epsilon^{2}+\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}.(36)If‖𝐳x‖2,‖𝐳y‖2≤C\|\mathbf{z}_{x}\|_{2},\|\mathbf{z}_{y}\|_{2}\leq C, then

|𝐱⊤​𝐲−𝐳x⊤​𝐳y|≤2​B​ϵ+ϵ2+δ​C2.\left|\mathbf{x}^{\top}\mathbf{y}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|\leq 2B\epsilon+\epsilon^{2}+\delta C^{2}.(37)This completes the proof.□\square

Remark.Theorem A shows that, at the token level, SSR is a bounded-distortion approximation to dense interaction. The distortion consists of two parts. The first part,

2​B​ϵ+ϵ2,2B\epsilon+\epsilon^{2},(38)comes from SAE reconstruction error. The second part,

δ​‖𝐳x‖2​‖𝐳y‖2,\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2},(39)comes from the deviation of the decoder geometry from an orthonormal embedding of the active sparse coordinates. Therefore, if dense token embeddings are well reconstructed and the decoder is approximately orthogonal on the active support, then

𝐱⊤​𝐲=𝐳x⊤​𝐳y+O​(B​ϵ+ϵ2)+O​(δ​‖𝐳x‖2​‖𝐳y‖2).\mathbf{x}^{\top}\mathbf{y}=\mathbf{z}_{x}^{\top}\mathbf{z}_{y}+O(B\epsilon+\epsilon^{2})+O\!\left(\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}\right).(40)In particular, under bounded sparse-code norms,

𝐱⊤​𝐲=𝐳x⊤​𝐳y+O​(B​ϵ+ϵ2+δ​C2).\mathbf{x}^{\top}\mathbf{y}=\mathbf{z}_{x}^{\top}\mathbf{z}_{y}+O(B\epsilon+\epsilon^{2}+\delta C^{2}).(41) Theorem B(Late-interaction bounded distortion). SupposeAssumptions 1–4hold uniformly for every query-document token pair(𝐪i,𝐝j)(\mathbf{q}_{i},\mathbf{d}_{j}). Define

η:=2​B​ϵ+ϵ2+δ​C2.\eta:=2B\epsilon+\epsilon^{2}+\delta C^{2}.(42)Then the dense late-interaction score and the SSR sparse late-interaction score satisfy

|Sdense​(Q,D)−SSSR​(Q,D)|≤N​η.\left|S_{\mathrm{dense}}(Q,D)-S_{\mathrm{SSR}}(Q,D)\right|\leq N\eta.(43) Proof.Let

si​j=𝐪i⊤​𝐝j,s~i​j=𝐳qi⊤​𝐳dj.s_{ij}=\mathbf{q}_{i}^{\top}\mathbf{d}_{j},\quad\tilde{s}_{ij}=\mathbf{z}_{q_{i}}^{\top}\mathbf{z}_{d_{j}}.(44)By Theorem A and the uniform boundedness assumptions,

|si​j−s~i​j|≤η,∀i,j.|s_{ij}-\tilde{s}_{ij}|\leq\eta,\quad\forall i,j.(45)For each query token embedding𝐪i\mathbf{q}_{i}, we use the elementary inequality

|maxj⁡si​j−maxj⁡s~i​j|≤maxj⁡|si​j−s~i​j|≤η.\left|\max_{j}s_{ij}-\max_{j}\tilde{s}_{ij}\right|\leq\max_{j}|s_{ij}-\tilde{s}_{ij}|\leq\eta.(46)Therefore,

|Sdense​(Q,D)−SSSR​(Q,D)|\displaystyle\left|S_{\mathrm{dense}}(Q,D)-S_{\mathrm{SSR}}(Q,D)\right|=|∑i=1Nmaxj⁡si​j−∑i=1Nmaxj⁡s~i​j|\displaystyle=\left|\sum_{i=1}^{N}\max_{j}s_{ij}-\sum_{i=1}^{N}\max_{j}\tilde{s}_{ij}\right|(47)≤∑i=1N|maxj⁡si​j−maxj⁡s~i​j|\displaystyle\leq\sum_{i=1}^{N}\left|\max_{j}s_{ij}-\max_{j}\tilde{s}_{ij}\right|≤N​η.\displaystyle\leq N\eta.This completes the proof.□\square

Consequence.Combining Theorem A and Theorem B, SSR preserves dense late-interaction scores up to a bounded distortion. At the token level, if dense token embeddings𝐱,𝐲\mathbf{x},\mathbf{y}are well reconstructed, i.e.,

‖𝐱−𝐱^‖2,‖𝐲−𝐲^‖2<ϵ,\|\mathbf{x}-\hat{\mathbf{x}}\|_{2},\ \|\mathbf{y}-\hat{\mathbf{y}}\|_{2}<\epsilon,(48)and their norms are bounded as

‖𝐱‖2,‖𝐲‖2<B,\|\mathbf{x}\|_{2},\ \|\mathbf{y}\|_{2}<B,(49)then the dense similarity differs from the reconstructed dense similarity by at most

O​(B​ϵ+ϵ2).O(B\epsilon+\epsilon^{2}).(50)Moreover, if the decoder is approximately orthogonal on the active support, namely

‖𝐖dec⊤​𝐖dec−𝐈‖S→S≤δ,\left\|\mathbf{W}_{\mathrm{dec}}^{\top}\mathbf{W}_{\mathrm{dec}}-\mathbf{I}\right\|_{S\rightarrow S}\leq\delta,(51)then the reconstructed dense similarity is further close to the sparse inner product:

𝐱^⊤​𝐲^=𝐳x⊤​𝐳y+O​(δ​‖𝐳x‖2​‖𝐳y‖2).\hat{\mathbf{x}}^{\top}\hat{\mathbf{y}}=\mathbf{z}_{x}^{\top}\mathbf{z}_{y}+O\!\left(\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}\right).(52)Therefore,

𝐱⊤​𝐲=𝐳x⊤​𝐳y+O​(B​ϵ+ϵ2)+O​(δ​‖𝐳x‖2​‖𝐳y‖2).\mathbf{x}^{\top}\mathbf{y}=\mathbf{z}_{x}^{\top}\mathbf{z}_{y}+O(B\epsilon+\epsilon^{2})+O\!\left(\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}\right).(53)Equivalently, SSR is a bounded-distortion approximation to dense late interaction:

|𝐱⊤​𝐲−𝐳x⊤​𝐳y|=O​(B​ϵ+ϵ2)+O​(δ​‖𝐳x‖2​‖𝐳y‖2).\left|\mathbf{x}^{\top}\mathbf{y}-\mathbf{z}_{x}^{\top}\mathbf{z}_{y}\right|=O(B\epsilon+\epsilon^{2})+O\!\left(\delta\|\mathbf{z}_{x}\|_{2}\|\mathbf{z}_{y}\|_{2}\right).(54)For the full MaxSim late-interaction score, if the above conditions hold uniformly over all query-document token pairs and‖𝐳qi‖2,‖𝐳dj‖2≤C\|\mathbf{z}_{q_{i}}\|_{2},\|\mathbf{z}_{d_{j}}\|_{2}\leq C, then

|Sdense​(Q,D)−SSSR​(Q,D)|≤N​(2​B​ϵ+ϵ2+δ​C2).\left|S_{\mathrm{dense}}(Q,D)-S_{\mathrm{SSR}}(Q,D)\right|\leq N\left(2B\epsilon+\epsilon^{2}+\delta C^{2}\right).(55)Thus, when the SAE reconstruction error is small and the decoder is near-orthogonal on active sparse supports, SSR preserves the semantic discriminability of dense late interaction while replacing dense token similarity with sparse overlap-based scoring.

Appendix BAdditional Related Work

Single-Vector Retrieval in the LLM Era.Leveraging the generative capabilities of Large Language Models (LLMs), recent research has significantly advanced single vector representation for documents(Liuet al.,2021,2022; Caoet al.,2023; Zhouet al.,2023; Youet al.,2024; Zhanget al.,2024; Minet al.,2025; Youet al.,2025; Xieet al.,2026; Weiet al.,2026). Through techniques such as synthetic data generation(Sturuaet al.,2024; Leeet al.,2024), hard negative mining(Zhanget al.,2025b; Wanget al.,2023), and instruction tuning(Penget al.,2024), representative models like Llama-Embed-8B(Babakhinet al.,2025)and Kalm-embedding-v2(Zhaoet al.,2025)have achieved state-of-the-art performance on the MTEB benchmark(Muennighoffet al.,2022). To mitigate the computational and storage cost, various compression techniques have been proposed. MRL(Kusupatiet al.,2022)employs multi-length representation learning, allowing for adaptive truncation with minimal performance loss. CSR(Wenet al.,2025; Guoet al.,2026)and the Splade series(Formalet al.,2021b,a; Lassanceet al.,2024)design sparse structure for high-efficiency retrieval, respectively exploring sparse projections and lexical expansions.

Multi-Vector Retrieval.Following the fine-grained interaction paradigm established by ColBERT(Khattab and Zaharia,2020), recent research has focused on addressing its scalability bottlenecks. On the storage front, ColBERTv2(Santhanamet al.,2022b), CRISP(Venerosoet al.,2025), and EMVB(Nardiniet al.,2024)significantly reduce memory footprints through techniques ranging from residual compression to product quantization. To enhance retrieval efficiency, COIL(Gaoet al.,2021)and CITADEL(Liet al.,2023a)adopt inverted list structures for faster indexing, while PLAID(Santhanamet al.,2022a)and EMVB(Nardiniet al.,2024)implement multi-stage pruning mechanisms. More recent works optimize the entire pipeline semantics: XTR(Leeet al.,2023)improves candidate quality via a token-retrieval objective, and WARP(Scheereret al.,2025)further optimizes this with dynamic similarity imputation and implicit decompression. Finally, ALIGNER(Qianet al.,2022)and LITE(Jiet al.,2024)explore alternative interaction optimizations via salience pruning and learnable MLP scoring, respectively. There have also been a few recent works that specially optimize the index and retrieval of ColBert-style models, such as IGP(Bianet al.,2025), MUVERA(Dhulipalaet al.,2024)and DESSERT(Engelset al.,2023).

Sparse Autoencoder(SAE).Sparse coding algorithms(Leeet al.,2006)serve as powerful techniques for compressing high-dimensional features into semantically meaningful, sparse representations, among which Sparse Autoencoder (SAE) pioneers for adaptive representation(Wenet al.,2025; Guoet al.,2026)and foundation model interpretability(Zhanget al.,2025a; Cunninghamet al.,2023; Templetonet al.,2024). SAEs project these high-dimensional vectors into sparse feature directions, effectively translating abstract model activations into human-readable concept dictionaries. To enhance the precision of this semantic extraction, recent studies have introduced advanced sparsity control mechanisms, ranging from TopK and Batch TopK constraints(Gaoet al.,2024; Bussmannet al.,2024)to dynamic activation functions like JumpReLU and Gated architectures(Rajamanoharanet al.,2024b,a).

Appendix CDatasets

Three distinct categories of datasets are involved in this paper: the BEIR benchmark(Thakuret al.,2021)for diverse retrieval tasks, LoTTE(Santhanamet al.,2022b)for long-tail retrieval, and LIMIT(Welleret al.,2025)for rigorous stress testing.

• •Arguana(AR)(Wachsmuthet al.,2018): Arguana comprises67536753argument-counterargument pairs extracted from10691069debates on idebate.org across1515diverse themes, uniquely featuring explicit counterarguments that attack the premises or conclusions of specific points.
• •ClimateFever(CF)(Diggelmannet al.,2020): ClimateFever is a challenging real-world dataset for climate-related claim verification, consisting of 1,535 claims collected from the internet and paired with 7,675 Wikipedia-sourced evidence sentences annotated as supporting, refuting, or providing insufficient information.
• •DBpedia(DB)(Hasibiet al.,2017): DBpedia is a standard and updated test collection for entity search that provides graded relevance judgments for 467 queries across various categories, mapped to 4.6 million entities from DBpedia.
• •Fever(FE)(Thorneet al.,2018): FEVER is a large-scale dataset consisting of 185,445 claims derived from Wikipedia, where systems are tasked with retrieving sentence-level evidence to classify each claim as Supported, Refuted, or NotEnoughInfo.
• •FiQA-2018(FQ)(Maiaet al.,2018): Constructed from the StackExchange Investment community, the FiQA-2018 QA dataset focuses on opinion-based question answering over financial data, containing a knowledge base of 57,640 posts and over 17,000 labeled question-answer pairs for training.
• •HotpotQA(HQ)(Yanget al.,2018): HotpotQA is a large-scale dataset containing about 113k Wikipedia-based question-answer pairs designed to test multi-hop reasoning, with sentence-level supporting facts provided for explainability.
• •MSMARCO(MS)(Nguyenet al.,2016): MS MARCO is a large-scale machine reading comprehension dataset comprising real-world user queries sampled from Bing search logs, where answers are human-generated summaries derived from retrieved web documents rather than simple extracted spans.
• •NFCorpus(NF)(Botevaet al.,2016): NFCorpus is a full-text learning-to-rank dataset in the medical domain, consisting of over 3,000 layman’s queries collected from NutritionFacts.org mapped to technical medical abstracts from PubMed with three-level relevance judgments.
• •NQ(Kwiatkowskiet al.,2019): Natural Questions (NQ) is a large-scale dataset consisting of 307,373 real-world Google search queries paired with Wikipedia pages, where systems are tasked with identifying a long answer (e.g., a paragraph) and a short answer (e.g., an entity), or determining if no answer exists.
• •Quora(QU)(Thakuret al.,2021): Quora is a duplicate question retrieval dataset where systems are tasked with retrieving semantically equivalent questions from a large corpus of community-generated questions given an input query.
• •SCIDOCS(SD)(Cohanet al.,2020): SCIDOCS is a scientific document evaluation benchmark where the retrieval task involves predicting citation links between research papers based on their titles and abstracts.
• •SciFact(SF)(Waddenet al.,2020): SciFact is a scientific fact-checking dataset where the retrieval task involves identifying research abstracts containing evidence that either supports or refutes a given expert-written claim.
• •TREC-COVID(CV)(Voorheeset al.,2021): TREC-COVID is an ad-hoc retrieval dataset constructed from the CORD-19 corpus, consisting of scientific articles related to COVID-19 and coronaviruses paired with pandemic-related queries annotated by biomedical experts.
• •Touché-2020(TO)(Bondarenkoet al.,2020): Focusing on argument retrieval, Touché-2020 consists of search scenarios on controversial issues where the goal is to retrieve grounded arguments that comprise conclusions and premises from online debate portals.
• •LoTTE(Santhanamet al.,2022b): LoTTE evaluates out-of-domain retrieval performance on information-seeking queries across long-tail topics. It encompasses five domains, including writing, recreation, science, technology, and lifestyle, along with an aggregated pooled setting. To capture varying user intents, each domain provides two distinct query subsets: search queries sourced from GooAQ and forum queries derived from StackExchange post titles.
• •LIMIT(Welleret al.,2025): Designed as a diagnostic retrieval benchmark, LIMIT empirically probes the theoretical representational bounds of embedding models. Unlike standard evaluation sets, it systematically stress-tests a model’s capacity to encode all possible top-kkdocument combinations within a given query space.

Appendix DDetails on Benchmark Performance Experiments

D.1Evaluation under Controlled Setup

Baselines.We benchmark against two categories of state-of-the-art retrieval models:

• •Multi-vector dense retrievers: We include late-interaction models such as ColBERT(Khattab and Zaharia,2020)and its efficiency-optimized variants ColBERTv2(Santhanamet al.,2022b)and PLAID(Santhanamet al.,2022a)). Additionally, we compare against recent state-of-the-art advancements, like AligneR(Qianet al.,2022), CITADEL(Liet al.,2023a), and XTR(Leeet al.,2023). Moreover, COIL(Gaoet al.,2021), which utilizes inverted index structure, is analyzed.
• •Sparse retrievers: We select the Splade family (Splade-v2(Formalet al.,2021a)and Splade-v3(Lassanceet al.,2024)) as the primary sparse baseline, which currently represents the state-of-the-art in learned sparse retrieval.

Experiment Setup.We conduct evaluations under a rigorous zero-shot setting to assess generalization capabilities. All models are trained in-domain on the MSMARCO(Nguyenet al.,2016)passage ranking dataset with negative documents sampled by BM25(Robertsonet al.,1995)and evaluated on both in-domain MSMARCO and 13 out-of-domain datasets from the BEIR(Thakuret al.,2021)benchmark. where primary evaluation metric is nDCG@10. Intuitively, nDCG@10 computes a weighted sum of relevance scores for the top 10 result and is normalized against the maximum possible score. To ensure a fair comparison, SSR is trained on MSMARCO using BERT-base-uncased(Devlinet al.,2019)backbone, with sparsity levelKKfixed at 32 and max sequence length set 32. Both SSR and SSR+CLS have been accelerated by SSR++ pruning. Latency is measured on A100 40G GPU, after 1000 warmup queries.

Implementation Details.During training, we include two separate SAEs: one for regular token embedding (EtokE_{\text{tok}}and one for the global [CLS] token (EtokE_{\text{tok}}). Setup for hyper-parameters in Table6. During evaluation, SSR++ partitions lists into blocks of size 64, uses the top-4 activated neurons for coarse pruning, keeps top-2000 candidates and then applies exact refinement with fullK=32K=32.

Table 6:Implementation details on evaluation under controlled setup.dhTopKKKcoarseK_{\text{coarse}}Block Sizekauxk_{\text{aux}}OptimizerlrBatch SizeEpochWarmupγ\gammaα\alphaβ\beta76816384324642048AdamW0.001642200000.050.031250.1

D.2Evaluation on Scalability to Modern Backbone

Baselines.We choose several embedding models that are competitive on MTEB benchmark(Muennighoffet al.,2022). These models are Qwen3-Embedding-8B(Zhanget al.,2025b), SFR-Embedding-Mistral(Rui Menget al.,2024), Linq-Embed-Mistral(Choiet al.,2024), e5-mistral-7b-instruct(Wanget al.,2023), gte-Qwen2-7B-instruct(Liet al.,2023b)and bge-large-en-v1.5(Xiaoet al.,2023).

Experiment Setup.We use Llama-embed-nemotron-8b as backbone. During training, we freeze the backbone parameters and train Sparse Autoencoder directly on the last-layer token embeddings. Training is conducted on the MSMARCO passage ranking dataset with the sparsity constraintKKmaintained at 32. Detailed setup is presented in Table7, with other parameters set as the same in Table6.

Table 7:Implementation details on evaluation on scalability to modern backbone.dhPrecisionlrGlobal Batch Size409665536BF160.000232

D.3Evaluation on Robustness to Long-Tail Distribution.

Datasets and Baselines.We benchmark SSR on the LoTTE(Santhanamet al.,2022b)dataset with five specialized domains (writing, recreation, science, technology, and lifestyle) and an aggregated pooled setting. Each domain includes two query sources, with search queries sourced from GooAQ and forum queries derived from StackExchange post titles. We compare against two representative MVR frameworks, XTR and ColBERTv2.

Experimental SetupWe utilize the same training and evaluation configurations as described in AppendixD.2to ensure a fair comparison, except that the max sequence length is set 1024 rather than 512.

Evaluation ResultsTable8demonstrates SSR’s performance on LOTTE(Santhanamet al.,2022b). Results show that SSR achieves superior performance compared with representative baselines.

Table 8:Evaluation on Long-Tail Benchmark LoTTE(Santhanamet al.,2022b).We report Success@5 as evaluation metrics, where the maximum values are indicated inbold.Query TypeModelCorpusWritingRecreationScienceTechnologyLifestylePooledAvg.SearchColBERTv279.270.855.565.984.371.871.3XTR78.669.556.164.683.569.370.3SSR81.873.959.367.187.673.373.8ForumColBERTv278.771.445.853.977.363.565.1XTR76.470.944.253.475.861.763.6SSR81.874.347.355.979.767.467.3

D.4Evaluation on Scalability to Long Document Sequences

Datasets and Baselines.We compare against a few representative baselines, including ColBERTv2(Santhanamet al.,2022b), XTR(Leeet al.,2023)and COIL(Gaoet al.,2021). Both MSMARCO passage and document ranking subsets are evaluated to demonstrate performance and efficiency difference on various document lengths in similar domain.

Experiment Setup.We utilize the same training and evaluation configurations as described in AppendixD.2, except that the max sequence length is set 4096 rather than 512. Similarity, SSR has been accelerated by SSR++.

Table 9:Performance and efficiency comparison on Long-Context Retrieval.We report nDCG@10 (%) for retrieval quality and retrieval time per query (ms) for efficiency. The maximum values are indicated inbold.MethodMS PassageMS DocumentnDCG@10 (%)latency (ms)nDCG@10 (%)latency (ms)ColBERTv239.837.141.579.3XTR47.645.048.152.6COIL35.312.638.818.8SSR-tok45.217.548.327.5SSR-CLS45.519.548.829.3Evaluation Results.Results in Table9demonstrate that while dense interaction mechanism of ColBERTv2 causes latency to spike to 79.3ms on longer documents, SSR-CLS maintains a sub-30ms latency (29.3ms), achieving a 2.7×\timesspeedup by leveraging sparse inverted indexing to restrict interactions to activated neurons. Crucially, this efficiency gain incurs no performance penalty: SSR-CLS attains the highest retrieval accuracy (48.8), outperforming the long-context optimized XTR (48.1) and significantly surpassing the lexical-only COIL baseline, thereby confirming its capability to preserve fine-grained details over long sequences with high throughput.

D.5Stress Testing on Representational Bounds.

Baselines and Experiment Setup.We select three outstanding SVR models (Qwen3-Embedding-4B, GritLM-7B and e5-mistral-7B-instruct) and two MVR frameworks (ColBERTv2 and XTR) to fully compare the representational bounds across different methods and paradigms. Same training configuration as described in AppendixD.1is utilized, with sparsityK=32K=32.

Evaluation Results.As shown in Table10, the LIMIT benchmark exposes a severe representational bottleneck in standard embedding models, with Single-Vector Retrieval (SVR) systems experiencing catastrophic failure (e.g., scoring below 5% on Recall@5). This empirically validates that forcing diverse document semantics into a single fixed-length vector causes irreversible information loss under stress. While dense Multi-Vector Retrieval (MVR) methods like ColBERTv2 mitigate this bottleneck by preserving token-level granularity, SSR establishes clear superiority by achieving a Recall@5 of 78.6% and a Recall@100 of 98.1%. SSR outperforms the strongest dense baseline (ColBERTv2) by substantial margins (+6.8% on Recall@5).

Table 10:Stress Testing on various SVR and MVR methods.We report Recall@5, Recall@10 and Recall@100 as evaluation metrics, where the maximum values are indicated inbold.MethodRecall@5Recall@10Recall@100Single-Vector RetrievalQwen3-4B1.24.66.3GritLM 7B4.76.119.4e5-mistral-7B2.85.210.9Multi-Vector RetrievalColBERTv271.882.694.4XTR68.479.993.2SSR78.687.398.1

Appendix EDetails on Efficiency and Empirical Analysis

E.1Ablations

Table11,12and13respectively demonstrates different settings on loss weightsα\alpha,β\betaandγ\gamma, with other parameters set as default as AppendixD.1. Evaluation is done on BEIR benchmark.

Table 11:Different settings on auxiliary loss weightα\alpha.α\alpha1/641/321/160.10.2SSR-tok52.152.952.451.650.8SSR-CLS52.353.452.751.851.3Table 12:Different settings on sparse contrastive loss weightβ\beta.β\beta0.050.10.150.20.25SSR-tok52.552.952.852.351.5SSR-CLS53.153.453.152.751.6Table 13:Different settings on supervised contrastive loss weightγ\gamma.γ\gamma0.010.020.050.10.2SSR-tok49.751.352.950.447.6SSR-CLS50.351.753.451.248.5We further conduct experiments to ablate each loss term respectively. Results in Table14show that each loss term leads to performance improvement while supervised contrastive loss results in the most significant improvement. Sparsity levelKKis set 32.

Table 14:Ablation on each loss term.Loss termSSR-tokSSR-CLSα\alphaβ\betaγ\gamma00046.446.71/320047.548.61/320.1048.249.51/320.10.0552.953.4

E.2CPU-based Efficiency Analysis

Baselines and Experiment Setup.We compare against six MVR engines, including PLAID(Santhanamet al.,2022a), WARP(Scheereret al.,2025), EMVB(Nardiniet al.,2024), DESSERT(Engelset al.,2023), MUVERA(Dhulipalaet al.,2024)and IGP(Bianet al.,2025). Index and Retrieval are done on Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz with 96 cores. Retrieval depth is set 100 in retrieval efficiency calculation, while other settings are set as the same in the original papers.

Results.Table15demonstrates different methods’ performance (MRR@10), index time (hour) and retrieval time (ms). Results show that SSR achieve better performance-efficiency trade-off compared to various modern engines.

Table 15:CPU-based Efficiency Analysis (the best results are highlighted inbold).MethodMRR@10(%)Index(h)Retrieval(ms)PLAID39.6122.9156.3WARP38.7103.7129.7EMVB39.861.593.4DESSERT37.249.875.6MUVERA39.782.694.7IGP39.1119.7113.2SSR-tok39.57.349.1SSR-CLS40.27.857.3

E.3System Resource Analysis

Baselines and Experiment Setup.We compare SSR against two representative engines: ColBERTv2 (accelerated by PLAID(Santhanamet al.,2022a)) and XTR (accelerated by WARP(Scheereret al.,2025)). We analyze through three dimensions:

• •Build-time cost:the peak build-time memory during training and indexing
• •Serving-time footprint:the persistent main index size from transient build resources, including auxiliary retrieval structures such as centroids/codebooks/residual metadata for PLAID-style systems, and posting lists/block metadata for SSR.
• •Maintenance/update cost:the update mode when new candidate documents appear after initial indexing.

Appendix FFurther Discussions

F.1Adaptive Query-based Sparsity Control

The optimal sparsity level for different queries may be different as longer queries may contain more complex semantic information, therefore requiring more fine-grained features for effective retrieval. To fully explore the performance-efficiency trade-off across different sparsity settings, we evaluate on BEIR in four query sparsity settings with models trained in benchmark experiments under controlled setup (i.e., Section4.1): fixed sparsityK=16K=16,K=32K=32,K=64K=64and adaptive sparsity based on query length. To be specific, for query with less than 3 tokens,KKis set 16, while 32 for those with 4-7 tokens and 64 for those with more than 8 tokens. Table16shows that adaptive sparsity slightly improves the Pareto frontier of performance-efficiency in MVR compared to fixed strategies.

Table 16:Comparison of sparsity settings based on query length.Sparsity SettingPerformance(%)Latency(ms)Fixed-1650.516.4Fixed-3252.917.5Fixed-6453.119.9Adaptive53.016.3

F.2Sweet Spot on Sparsity across Domains

Due to difference in semantic complexity, the most stable sparsity setting may vary for different domains. We select 11 representative tasks in BEIR, divide them into 4 domains based on content and analyze each domain’s average performance under differentKK:

• •fact: DBpedia, Fever, SciFact
• •multi-hop: NQ, HotpotQA
• •scientific: TREC-COVID, NFCorpus, SCIDOCS
• •opinion: Arguana, Touché-2020, FiQA-2018

Results in Table17show thatK=32K=32is the most stable setting for most domains. However, for domains whose semantics tend to be clear and easily separable (fact),K=16K=16leads to relatively minor performance degradation while for domains where more fine-grained search is needed (multi-hop),K=64K=64results in obvious performance improvement but introduces additional latency cost.

Table 17:Sparsity Setting Difference across domains.K8163264128fact50.465.967.768.268.4multi-hop50.258.665.166.566.8scientific29.339.644.244.845.3opinion20.933.339.540.340.6