Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers

# Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers
Source: [https://arxiv.org/html/2608.08809](https://arxiv.org/html/2608.08809)
\\authorOne

\[1\]Yu Wang\\authorOne\[2\]Shengyao Zhuang\\authorOne\[3\]Xueguang Ma\\authorOne\[4\]Zongyu Wu\\authorOne\[3\]Jimmy Lin\\authorOne\[1\]Vivek Srikumar\\authorOne\[1\]Zhichao Xu 1\]University of Utah 2\]The University of Queensland 3\]University of Waterloo 4\]Pennsylvania State University\\contribution\[\]yuki\.wang@utah\.edusvivek@cs\.utah\.eduzhichao\.xu@utah\.edu

###### Abstract

A single model scale challenges the flexibility of a production retrieval system: some settings need it faster, others need a smaller index, and the right trade\-off changes with the workload\. In the context of information retrieval \(IR\), a transformer\-based model can be made smaller in three ways—using fewer layers, passing fewer tokens through the upper layers, or producing a shorter embedding—and each way saves a different compute resource\. These options have been studied one at a time, each as its own method with its own code and training setup, which makes them hard to combine or adapt to a new model\. We presentTevatron\-Elasticto bring all three under one simple abstraction: a single object names any size the model can run at, and a short schedule lists the sizes to train\. Training then produces one checkpoint that serves all of those sizes, and at deployment the user picks any of them\. The same abstraction covers both retrievers and rerankers and both encoder and decoder models, as it works through interfaces that Hugging Face transformers already expose; a new backbone is a configuration change, not new modeling code\. Prior methods—Matryoshka embeddings\(Kusupati et al\.,[2022](https://arxiv.org/html/2608.08809#bib.bib13)\), early exit\(Liu et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib17); Xin et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib25)\), 2D Matryoshka\(e\.g\., Starbucks, Zhuang et al\.,[2026a](https://arxiv.org/html/2608.08809#bib.bib38)\), and layerwise token compression\(LTC, Zhuang et al\.,[2026b](https://arxiv.org/html/2608.08809#bib.bib39)\)—become special cases of our unified abstraction\. The same interface also enables Matryoshka LTC \(MLTC\), which jointly trains several token\-compression ratios in one retriever checkpoint\. To validate our framework, we train 20 checkpoints across three backbones and two tasks: the quality curves are smooth, one checkpoint costs little over a model trained for a single size, and a controlled study confirms the wallclock speedups\. We release the framework and all checkpoints as a resource for building elastic retrieval systems\.

\\setmaintable

## 1Introduction

A transformer\-based information retrieval \(IR\) model\(Lin et al\.,[2022](https://arxiv.org/html/2608.08809#bib.bib16); Xu et al\.,[2026b](https://arxiv.org/html/2608.08809#bib.bib31)\)has three largely separable axes that control its cost at inference time:

- •depth— how many transformer layers are executed before a readout\. Fewer layers means less encoder/scorer compute and lower latency\(Devlin et al\.,[2019](https://arxiv.org/html/2608.08809#bib.bib2); Liu et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib17); Xin et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib25); Warner et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib23)\)\.
- •token— how many sequence positions flow through the upper layers\. Pooling the sequence partway up the stack reduces the floating\-point operations \(FLOPs\) of those upper layers\(Goyal et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib8); Zhang et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib37)\)\.
- •width— the dimensionality of the output embedding\. A smaller embedding reduces the size of the stored index and the cost of nearest\-neighbor search\(Johnson et al\.,[2017](https://arxiv.org/html/2608.08809#bib.bib12); Kusupati et al\.,[2022](https://arxiv.org/html/2608.08809#bib.bib13)\)\.

A useful mental model to group these axes is by the deployment cost each one reduces: depth and token reduce*compute*, while width reduces*storage and search*\. Depth and token act at compute time and leave the index unchanged; width acts at storage time and leaves the per\-input compute unchanged\.

Prior work on elastic modeling tends to commit to a single axis\. Matryoshka representation learning \(MRL\) varies width\(Kusupati et al\.,[2022](https://arxiv.org/html/2608.08809#bib.bib13)\); layer\-dropping and early\-exit methods vary depth\(Liu et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib17); Xin et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib25); Elhoushi et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib4)\);Zhang et al\. \([2025](https://arxiv.org/html/2608.08809#bib.bib37)\)andZhuang et al\. \([2026b](https://arxiv.org/html/2608.08809#bib.bib39)\)compress the retriever and reranker on the token axis; and 2D Matryoshka retrievers such as 2DMSE and Starbucks vary depth and width together\(Li et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib15); Zhuang et al\.,[2026a](https://arxiv.org/html/2608.08809#bib.bib38)\)\. Each of the aforementioned methods is effective in its own setting, but each comes as a separate implementation with its own model class definition and training objective, so combining them, or adapting one to a new task or backbone, requires re\-implementation\.

We posit that these compression axes are complementary rather than competing: because they reduce different costs, they can be applied independently and composed\. Existing work, however, provides no single abstraction to unify the definitions of different compressed model classes, as well as the corresponding training and inference\.

Seeing this gap, we presentTevatron\-Elasticto provide a unified abstraction of these axes based onTevatron\(Ma et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib18); Xu et al\.,[2026a](https://arxiv.org/html/2608.08809#bib.bib30)\)\. We first abstract the action of compressing a model as an*operating point*, e\.g\., early exit at a certain layer means defining the early exit operating point for that layer\. Based on this definition,Tevatron\-Elasticintroduces two core components\. A frozen dataclass,Granularity, names any operating point with one optional field per axis; and aGranularitySchedulelists the operating points to train the underlying elastic model\. Specifically, the training loop sums a per\-granularity loss over the schedule from a single backbone forward pass, producing one checkpoint that serves every listed operating point\. Instead of editing the model internals: the abstraction routes through the hidden\-states tuple, the layer module list, and the pooling step that every Hugging Face transformer exposes, so a new backbone is a launch flag rather than new code\.

Our core contribution is the unified abstraction that accommodates different types of compression strategies:

1. 1\.one unified abstraction spans two tasks \(retrieval, reranking\), three axes, and five backbone families \(BERT, ModernBERT, Qwen3, Llama3, Mistral\) with no new model code per backbone \([Section˜3](https://arxiv.org/html/2608.08809#S3)\);
2. 2\.it reproduces prior elastic methods: MRL, early exit, 2D Matryoshka, Starbucks, and LTC as configurations \([Section˜4](https://arxiv.org/html/2608.08809#S4)\);
3. 3\.the operating points trained jointly behave as expected — the quality curves from our empirical studies are smooth and the cost of joint training relative to a dedicated single\-point model is small \([Section˜4](https://arxiv.org/html/2608.08809#S4)\)\. Further, controlled study shows the measured inference speedup follows the analytic cost model \([Section˜4\.3](https://arxiv.org/html/2608.08809#S4.SS3)\)\.
4. 4\.it introduces*Matryoshka Layerwise Token Compression*\(MLTC\), which jointly trains several token\-compression ratios in one checkpoint as an extension of LTC\(Zhuang et al\.,[2026b](https://arxiv.org/html/2608.08809#bib.bib39)\)andZhang et al\. \([2025](https://arxiv.org/html/2608.08809#bib.bib37)\)\. MLTC uses the framework’s existing schedule and token readout path; supporting several mid\-stack pooling points would otherwise require modifying the model class\.

We note thatTevatron\-Elasticdoes not improve on prior methods in absolute quality, nor that combining all three axes is preferable to using one\. In our empirical experiments, we hold the training recipe fixed \(supervised fine\-tuning from base checkpoints, without the pre\-training some prior systems use\(Gao and Callan,[2022](https://arxiv.org/html/2608.08809#bib.bib7); Zhuang et al\.,[2026a](https://arxiv.org/html/2608.08809#bib.bib38); Xu et al\.,[2026f](https://arxiv.org/html/2608.08809#bib.bib35)\)\), so that we can study the trained elastic checkpoints in isolation\. We plan to release our framework and checkpoints to help researchers reproduce existing elastic model recipes and develop new ones\.

## 2Related Works and Background

We first review work on IR model compression along the three compression axes \([Section˜2\.1](https://arxiv.org/html/2608.08809#S2.SS1)\), then define the notation used in the rest of the paper \([Section˜2\.2](https://arxiv.org/html/2608.08809#S2.SS2)\)\.

### 2\.1Prior methods

We focus on work that makes a retrieval or ranking model*elastic*— adjustable in cost after training — and group it by the axis it compresses\.

#### Width\.

Matryoshka representation learning trains an embedding whose leading coordinates remain usable on their own, so one model serves several vector sizes and, in retrieval, several index sizes\(Kusupati et al\.,[2022](https://arxiv.org/html/2608.08809#bib.bib13)\)\. The truncation is at the output, so width changes storage and nearest\-neighbor search cost\(Johnson et al\.,[2017](https://arxiv.org/html/2608.08809#bib.bib12)\)but not the per\-input compute\.

#### Depth\.

Early\-exit and layer\-dropping methods read out the representations at an intermediate layer to trade quality for fewer layers of compute, either with a fixed exit or an input\-adaptive one\(Liu et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib17); Xin et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib25); Elhoushi et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib4)\)\. In IR this yields a shallower encoder \(retriever\) or scorer \(reranker\)\.

#### Token\.

Sequence\-compression methods shorten the token sequence inside the stack so the upper layers process fewer positions\(Goyal et al\.,[2020](https://arxiv.org/html/2608.08809#bib.bib8)\)\. Prior works Jasper\-Token\-Compression\-600M\(Zhang et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib37)\)and Layer\-wise token compression\(LTC, Zhuang et al\.,[2026b](https://arxiv.org/html/2608.08809#bib.bib39)\)pool the sequence of tokens at a chosen layer and run the remaining layers on the shortened sequence for retrieval and reranking, respectively\.

#### Composition\.

A few methods combine axes\. 2D Matryoshka embeddings vary depth and width together\(Li et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib15)\), and Starbucks trains a retriever elastic on both, so one checkpoint serves a grid of \(layer, dim\) points\(Zhuang et al\.,[2026a](https://arxiv.org/html/2608.08809#bib.bib38)\)\. These are the closest prior systems to ours; each, however, fixes one particular pair of axes and one task\.

#### Our position\.

In existing works, each axis \(or fixed pair\) is realized as a separate method with its own model class definition and implementation as well as training strategy, and almost always on a single task\. We do not introduce a new pooling operation; instead, we provide one abstraction in which depth, token, and width are named uniformly, the set of operating points to train is written as data, and the methods above — MRL, early exit, LTC, and 2D Matryoshka — appear as configurations for both retrieval and reranking\. The per\-operating\-point objective is itself pluggable and can be a standard contrastive loss, a learning\-to\-rank loss\(Cao et al\.,[2007](https://arxiv.org/html/2608.08809#bib.bib1); Xia et al\.,[2008](https://arxiv.org/html/2608.08809#bib.bib24); Oord et al\.,[2018](https://arxiv.org/html/2608.08809#bib.bib19)\), or distillation\(Hinton et al\.,[2015](https://arxiv.org/html/2608.08809#bib.bib10); Xu et al\.,[2025b](https://arxiv.org/html/2608.08809#bib.bib28)\)\([Section˜3](https://arxiv.org/html/2608.08809#S3)\)\. MLTC is one new configuration enabled by this interface: it applies the same token\-pooling path to several ratios in a single retriever checkpoint\.

### 2\.2Notations

We set up notation shared by the three axes, which[Section˜3](https://arxiv.org/html/2608.08809#S3)then unifies\. A transformer withLLlayers maps an input ofTTtokens to a sequence of hidden states; writeH\(ℓ\)​ℝT​DH^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{T\\times D\}for the output of layerℓ\\ell, withDDthe hidden size\. A dense encoder pools the top layer to a vectorz=pool⁡\(H\(L\)\)​ℝDz=\\operatorname\{pool\}\(H^\{\(L\)\}\)\\in\\mathbb\{R\}^\{D\}\(CLS, mean, or last token\); a cross\-encoder mapsH\(L\)H^\{\(L\)\}to a scalar relevance score\. An operating point chooses*how*this readout is taken, and each compression axis is one such choice\.

#### Width \(MRL\)

Width keeps the firstd​Dd\\le Dcoordinates of the pooled vector,z1:dz\_\{1:d\}, and scores with the truncated embedding\. Training over a set of widths𝒟=\{d1,…,dk\}\\mathcal\{D\}=\\\{d\_\{1\},\\dots,d\_\{k\}\\\}sums the loss over the nested prefixes\(Kusupati et al\.,[2022](https://arxiv.org/html/2608.08809#bib.bib13)\):

ℒMRL=\\slimits@d​𝒟​ℒ​\(z1:d\)\.\\mathcal\{L\}\_\{\\text\{MRL\}\}=\\tsum\\slimits@\_\{d\\in\\mathcal\{D\}\}\\mathcal\{L\}\\big\(z\_\{1:d\}\\big\)\.\(1\)The forward pass is unchanged; only the output dimension varies, so width affects index size and search cost, not per\-input compute\.

#### Depth \(early exit\)

Depth reads out at an intermediate layerℓ​L\\ell\\le Linstead of the top, usingpool⁡\(H\(ℓ\)\)\\operatorname\{pool\}\(H^\{\(\\ell\)\}\)\(or the scalar head applied at layerℓ\\ell\)\. Training over a set of exit layersℰ=\{ℓ1,…,ℓk\}\\mathcal\{E\}=\\\{\\ell\_\{1\},\\dots,\\ell\_\{k\}\\\}gives

ℒdepth=\\slimits@ℓ​ℰ​ℒ​\(pool⁡\(H\(ℓ\)\)\)\.\\mathcal\{L\}\_\{\\text\{depth\}\}=\\tsum\\slimits@\_\{\\ell\\in\\mathcal\{E\}\}\\mathcal\{L\}\\big\(\\operatorname\{pool\}\(H^\{\(\\ell\)\}\)\\big\)\.\(2\)At inference, exiting atℓ\\ellruns only the firstℓ\\elllayers, so the compute scales roughly asℓ/L\\ell/L\.

#### Token \(LTC\)

Token compression pools the sequence at a chosen layerpp, reducing its length fromTTtom=\\lceil​r​T​\\rceilm=\\lceil rT\\rceilfor a keep\-ratior​\(0,1\]r\\in\(0,1\], then runs the remaining layersp\+1,…,Lp\{\+\}1,\\dots,Lon the shortened sequence\(Zhuang et al\.,[2026b](https://arxiv.org/html/2608.08809#bib.bib39)\)\. Writingpoolseq\\operatorname\{pool\}\_\{\\text\{seq\}\}for the sequence pooling andfp:Lf\_\{p:L\}for the upper\-layer stack,

H~\(p\)=poolseq⁡\(H\(p\),r\),z=pool⁡\(fp:L​\(H~\(p\)\)\),\\tilde\{H\}^\{\(p\)\}=\\operatorname\{pool\}\_\{\\text\{seq\}\}\\\!\\big\(H^\{\(p\)\},r\\big\),\\qquad z=\\operatorname\{pool\}\\big\(f\_\{p:L\}\(\\tilde\{H\}^\{\(p\)\}\)\\big\),\(3\)The layers belowppstill run on the full sequence, so the compute saving applies only to the upper layers and is largest for long inputs\.

Although the three choices intervene at different components of transformer computations, they all read from or transform the same hidden\-state sequence\. This shared structure suggests a common interface: can we name and train these choices in a single framework, while retaining the deployment trade\-offs each one offers?

## 3Framework Design

To turn this shared structure into a framework, we first clarify the concept of residual stream and the readout\. We then describe the requirements for a unified elastic framework\.

#### Preliminary: residual stream and readout\.

A backbone processes an input and returns a sequence of hidden representations as output\. This sequence is referred to as*residual stream*in literature\(Elhage et al\.,[2021](https://arxiv.org/html/2608.08809#bib.bib3)\)\. When we setoutput\_hidden\_states=True, the standard Hugging Face interface returns it as a tuple\(H\(0\),H\(1\),…,H\(L\)\)\(H^\{\(0\)\},H^\{\(1\)\},\\dots,H^\{\(L\)\}\), whereH\(0\)H^\{\(0\)\}is the embedding output andH\(ℓ\)​ℝT​DH^\{\(\\ell\)\}\\in\\mathbb\{R\}^\{T\\times D\}is the output after layerℓ\\ell\. A*readout*is the method we use to turn this residual stream into the model output at the operating point: choose a layer, optionally shorten its token sequence, then pool it to a retrieval vector or map it to a relevance score\. For a retriever, the readout can also keep only part of the vector\. Depth chooses*which*layer to read; token compression changes the sequence lengthTT; and width truncates the feature dimensionDD\.

out=backbone\(\*\*inputs,output\_hidden\_states=True\)

H=out\.hidden\_states

h=H\[layer\]

z=pool\(h,attention\_mask\)

z=z\[:dim\]

Token compression is the one case that changes the stream*between*layers: it poolsH\(p\)H^\{\(p\)\}alongTTand runs the upper layers on the shorter sequence\. The remaining operations are simple choices in the readout above\. This view is the bridge between the three axes\.

#### Desiderata\.

Starting from this common readout, a unified framework should meet four practical requirements:

1. 1\.Name every operating point in one convention,whether it changes depth, token count, or embedding width\.
2. 2\.Let users combine operating points without new modeling code,and select the points to train in a small configuration rather than in a new training loop\.
3. 3\.Work with the transformer interfaces models already expose,so changing a backbone does not mean rewriting the elastic model\.
4. 4\.Train and serve many operating points efficiently,so one checkpoint is useful in practice rather than merely expressive on paper\.

The rest of this section answers these four goals in turn\.

#### One description for every operating point\.

Every operating point can be represented as a single frozen dataclass with one optional field per axis:

@dataclass\(frozen=True\)

classGranularity:

layer:int

dim:int\|None=None

keep\_ratio:float\|None=None

pool\_layer:int\|None=None

AGranularityprints as a compact, collision\-free key \(L12,L6xD128,L20xR0\.8,L28xR0\.6xP20\) that doubles as the operating point’s name in logs and on disk\. Fields are independent: setting onlylayergives depth, addingdimgives width, and addingkeep\_ratiowithpool\_layergives token compression\.

#### A schedule instead of custom training code\.

The set of operating points to train is a list, written as a short spec string or JSON:

schedule=GranularitySchedule\.from\_spec\("2:32,6:128,12:768"\)

[Table˜1](https://arxiv.org/html/2608.08809#S3.T1)maps spec strings to the prior methods they reproduce\. The same grammar expresses single axes and their compositions; the user can choose the operating points by editing the string without changing code\.

Table 1:The schedule grammar\. One line of configuration selects an axis or a composition, and reproduces a prior method as a special case\.what to trainspec / presetreproducesdepth\-only \(early exit\)"2,4,6,8,10,12"/pure\_depthearly exitwidth\-only \(MRL\)pure\_mrl\(12,\(32,\.\.\.,768\)\)MRLdepthwidthstarbucks\_bert/diagonal\(\.\.\.\)Starbuckstoken\-only \(reranker\)"20@0\.6"LTCtoken\-only \(retriever\)"28@1\.0/20,\.\.\.,28@0\.4/20"MLTC \(this work\)width \+ token"28:256@0\.6/20"— \(composition\)full three\-axis"24:256@0\.6/20"left open \([Section˜4](https://arxiv.org/html/2608.08809#S4)\)
#### How readout operates on the standard Hugging Face interface\.

Because the readout only uses the hidden\-states tuple, the layer module list, and a pooling step — all standard across Hugging Face transformers — we can use one function to realize all three axes for any backbone, parameterized by aGranularityg:

def\_readout\(self,hidden\_states,attention\_mask,g\):

ifg\.pool\_layerisnotNone:

h\_p=hidden\_states\[g\.pool\_layer\]

pooled\_h,mask=pool\_sequence\(h\_p,attention\_mask,g\.keep\_ratio\)

h=self\.\_range\_runner\(self\.encoder,pooled\_h,mask,g\.pool\_layer,end=g\.layer\)

else:

h,mask=hidden\_states\[g\.layer\],attention\_mask

reps=self\.\_pooling\(h,mask\)

ifg\.dimisnotNone:

reps=reps\[:,:g\.dim\]

returnF\.normalize\(reps\)ifself\.normalizeelsereps

The only backbone\-specific code is the token path’s upper\-layer runner, which rebuilds the attention mask \(and, for decoders, the rotary position embeddings\) for the shortened sequence; it is registered per model family:

@register\_upper\_runner\("qwen3","qwen2","mistral","llama"\)

def\_decoder\_upper\(backbone,hidden,mask,l\_star,end=None\):\.\.\.

#### Matryoshka Layerwise Token Compression \(MLTC\)\.

With the proposed framework, we further extends LTC to the Matryoshka setting, where we train multiple compression ratios in one training run\. We name this approach as MLTC and showcases its effectiveness in the retriever setting\. Specifically, instead of using a single keep ratiorras LTC in[Section˜2\.2](https://arxiv.org/html/2608.08809#S2.SS2), we use multiple keep ratios packed in theGranularitySchedule, and the final loss is a sum over individual losses, similar to the MRL retriever setting\.

MLTC requires no additional model path as all keep ratios use same token readout\. Without the framework’s token runner and schedule interface, supporting these mid\-stack pooling points would require modifying the Hugging Face model class\. We verified that no new model code is required to train across BERT\(Devlin et al\.,[2019](https://arxiv.org/html/2608.08809#bib.bib2)\), ModernBERT\(Warner et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib23)\), Qwen3\(Yang et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib36)\), Llama3\(Grattafiori et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib9)\), and Mistral\-7B\(Jiang et al\.,[2023](https://arxiv.org/html/2608.08809#bib.bib11)\)\([Table˜2](https://arxiv.org/html/2608.08809#S3.T2)\)\.

Table 2:Adding a backbone is a launch flag, not new model code\. The pooling column is the only per\-backbone choice for the depth and width axes\.backbonefamilypoolingnew model codeBERT\-base\(Devlin et al\.,[2019](https://arxiv.org/html/2608.08809#bib.bib2)\)encodercls—ModernBERT\-base\(Warner et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib23)\)encodercls—Qwen3\-0\.6B\(Yang et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib36)\)decoder \(GQA\)eos—Llama3\-\{1B,3B,8B\}\(Grattafiori et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib9)\)decoder \(GQA\)eos— \(full path verified\)Mistral\-7B\-v0\.3\(Jiang et al\.,[2023](https://arxiv.org/html/2608.08809#bib.bib11)\)decoder \(GQA\)eos— \(full path verified\)
#### Joint training\.

We run the backbone model for*a single forward pass*per batch; every operating point in the schedule is then a cheap readout off that shared forward pass, and the training loss is the sum of the per\-granularity losses:

out\_q=model\(\*\*query,output\_hidden\_states=True\)

out\_p=model\(\*\*passage,output\_hidden\_states=True\)

loss=sum\(gran\_loss\(out\_q,out\_p,g\)forginschedule\.points\)

The per\-granularity lossgran\_lossis a pluggable function of the readouts, not part of the training framework: the schedule decides*which*operating points to train, whilegran\_lossdecides*how*each is supervised\. In our experiments it is the task’s standard objective\. For the retriever it is in\-batch InfoNCE\(Oord et al\.,[2018](https://arxiv.org/html/2608.08809#bib.bib19)\)over the query and passage readouts, which is softmax cross\-entropy over each query’s candidate list together with in\-batch negatives\(Xiong et al\.,[2021](https://arxiv.org/html/2608.08809#bib.bib26); Qu et al\.,[2021](https://arxiv.org/html/2608.08809#bib.bib20)\)— equivalently, the ListNet objective\(Cao et al\.,[2007](https://arxiv.org/html/2608.08809#bib.bib1)\)with a single relevant item:

defgran\_loss\(out\_q,out\_p,g\):

q=self\.\_readout\(out\_q\.hidden\_states,q\_mask,g\)

p=self\.\_readout\(out\_p\.hidden\_states,p\_mask,g\)

scores=\(q@p\.T\)/self\.temperature

target=torch\.arange\(B,device=q\.device\)\*n

returnF\.cross\_entropy\(scores,target\)

Becausegran\_lossonly sees the readouts for an operating point, substituting a different objective is a local change to this one function\. Staying within learning\-to\-rank literature, we use ListMLE\(Xia et al\.,[2008](https://arxiv.org/html/2608.08809#bib.bib24)\)as an example, which replaces the ListNet term with the Plackett–Luce likelihood of the target ordering:

defgran\_loss\(out\_q,out\_p,g\):

q=self\.\_readout\(out\_q\.hidden\_states,q\_mask,g\)

p=self\.\_readout\(out\_p\.hidden\_states,p\_mask,g\)

scores=\(q@p\.T\)/self\.temperature

returnlistmle\(scores,target\_order\)

Distillation instead supervises each operating point with a teacher model\(Hinton et al\.,[2015](https://arxiv.org/html/2608.08809#bib.bib10); Xu et al\.,[2025b](https://arxiv.org/html/2608.08809#bib.bib28)\), wherescore\_with\_teacherreturns the teacher’s query–passage scores for the batch\(Xu et al\.,[2025b](https://arxiv.org/html/2608.08809#bib.bib28)\):

defgran\_loss\(out\_q,out\_p,g\):

q=self\.\_readout\(out\_q\.hidden\_states,q\_mask,g\)

p=self\.\_readout\(out\_p\.hidden\_states,p\_mask,g\)

student=\(q@p\.T\)/self\.temperature

teacher=score\_with\_teacher\(query,passage\)

returnF\.kl\_div\(student\.log\_softmax\(\-1\),teacher\.softmax\(\-1\),reduction="batchmean"\)

The reranker follows the same pattern with grouped cross\-entropy over each query’s candidate list \(the positive at index 0\)\. In every case the schedule and the readout are unchanged; substituting ListNet with ListMLE or a distillation loss is a matter of plugging in a differentgran\_loss\.

#### Checkpoint serving\.

We load the full checkpoint and prune in place to a chosen operating point\. Depth pruning physically removes the upper layers, so the saved compute is incurred once per input; width and token are applied at readout time:

model=ElasticDenseModel\.load\(ckpt\)

model\.prune\_to\(Granularity\(16\)\)

We store one checkpoint rather than a separate checkpoint per operating point\.

#### A cost model for the three axes\.

We can use a simple function to map aGranularityto its relative cost: encode/score FLOPs=L/Lmax=L/L\_\{\\max\}for a depth point at layerLL, and\(p\+\(Lmax−p\)​r\)/Lmax\(p\+\(L\_\{\\max\}\-p\)\\,r\)/L\_\{\\max\}for a token point that pools at layerppwith keep\-ratiorr; relative index size=d/dmax=d/d\_\{\\max\}for widthdd\.[Section˜4\.3](https://arxiv.org/html/2608.08809#S4.SS3)checks the measured wall clock speedup and index size reduction\.

## 4Experiments

This section presents the experimental setup and results, examining whether the proposed framework meets the desiderata above\.

### 4\.1Experimental setup

![Refer to caption](https://arxiv.org/html/2608.08809v1/x3.png)Figure 1:Retriever quality \(BEIR\-15 nDCG@10\) along each axis, three backbones\. Dashed lines mark each backbone’s plain full point\. \(a\) depth at full width; \(b\) width \(MRL\) at full depth, log dimension axis; \(c\) depthwidth \(Starbucks\), where layer and dimension grow jointly\. Every curve is smooth and monotonic, and the elastic full points sit close to the plain reference\.![Refer to caption](https://arxiv.org/html/2608.08809v1/x4.png)\(a\)Token axis on both tasks \(Qwen3\-0\.6B\): the retriever \(MLTC\) and reranker \(LTC\) curves have similar gentle shapes under the same pooling primitive\.
![Refer to caption](https://arxiv.org/html/2608.08809v1/x5.png)\(b\)Reranker depth, three backbones\. A floor at the shallow end then a long plateau, distinct from the gradual retriever depth curve in[Figure˜1](https://arxiv.org/html/2608.08809#S4.F1)a\.

Figure 2:Comparison of token\-axis behavior and reranker depth\.#### Backbones\.

We experiment with three backbone models BERT\-base\(Devlin et al\.,[2019](https://arxiv.org/html/2608.08809#bib.bib2)\)\(110M, encoder, CLS pooling\), ModernBERT\-base\(Warner et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib23)\)\(150M, encoder, CLS pooling, alternating global/local attention\), and Qwen3\-0\.6B\(Yang et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib36)\)\(decoder, GQA, last\-token pooling\) on the full set of axes\. Llama3 family models\(Grattafiori et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib9)\)and Mistral\-7B\(Jiang et al\.,[2023](https://arxiv.org/html/2608.08809#bib.bib11)\)are used only to verify the backbone\-agnostic path end to end\.

#### Data and evaluation\.

All models are trained on RLHN\-680K\(Thakur et al\.,[2025](https://arxiv.org/html/2608.08809#bib.bib22)\)withquery:andpassage:prefixes, at input length 512, albeit the different tokenizers\. We evaluate on BEIR\-15\(Thakur et al\.,[2021](https://arxiv.org/html/2608.08809#bib.bib21)\)as a single macro average \(cqadupstack collapsed to the mean of its 12 subforums and counted once; MS MARCO dev included as one member\), reporting nDCG@10 and Recall@100\. Retrieval reports absolute nDCG@10; reranking reports nDCG@10 after reranking BM25 top\-100, so the BM25 first stage \(0\.415\) is the reference\. The per\-backbone*plain*\(non\-elastic, single full point\) model is the baseline against which we read each compression curve\. We refer the complete per\-granularity numbers for Qwen3\-0\.6B checkpoints to[Appendix˜A](https://arxiv.org/html/2608.08809#A1), and skip BERT and ModernBERT for the sake of space\.

#### What the experiments assess\.

The experiments assess the desiderata discussed in[Section˜3](https://arxiv.org/html/2608.08809#S3)\. The backbone coverage exercises the common interface across encoder and decoder families\. The quality curves test whether a common operating\-point description and schedule can express the expected depth, width, token, and composed trade\-offs\. The experiments also measure the cost of joint training at the full point \(the*elasticity tax*: elastic full\-point minus the dedicated plain full\-point\), and separately measure the deployment costs that each axis is intended to change\. We focus less on absolute performance numbers and instead on diagnosing the framework\.

### 4\.2Results and analysis

We first evaluate the single\-axis operating points and then their compositions\. These configurations correspond to prior elastic methods where applicable\.[Figures˜1](https://arxiv.org/html/2608.08809#S4.F1),[2\(a\)](https://arxiv.org/html/2608.08809#S4.F2.sf1)and[2\(b\)](https://arxiv.org/html/2608.08809#S4.F2.sf2)show the quality curves \(BEIR\-15 nDCG@10\)\.

#### Depth\.

For the retriever, performance improves smoothly with the number of layers on all three backbones \([Figure˜1](https://arxiv.org/html/2608.08809#S4.F1)a; Qwen3 goes from0\.4310\.431at layer 6 to0\.5180\.518at layer 28, full width\)\. The reranker has a different curve \([Figure˜2\(b\)](https://arxiv.org/html/2608.08809#S4.F2.sf2)\): quality is very low when only a few layers run, then rises and levels off at about half the layers \(Qwen3 reaches0\.5550\.555by layer 16, matching layer 28; BERT levels off at layer 10, ModernBERT at layer 16\)\. The retriever, which reads out a pooled vector, degrades gradually; the reranker, which reads out a single score, has a low\-quality region at shallow depth and a plateau at greater depth\. ModernBERT shows a step around layer 13 on both tasks, at the transition in its alternating global/local attention layout\.

#### Width \(MRL\)

The pure\-width curve is smooth and monotonic on all three backbones \([Figure˜1](https://arxiv.org/html/2608.08809#S4.F1)b; e\.g\. Qwen30\.3650\.5050\.365\\to 0\.505from dim 64 to 1024 at full depth\)\. This reproduces MRL as a configuration; the corresponding cost is index size rather than compute \([Section˜4\.3](https://arxiv.org/html/2608.08809#S4.SS3)\)\.

#### Token \(LTC Reranker and MLTC Retriever\)

On the reranker, we followZhuang et al\. \([2026b](https://arxiv.org/html/2608.08809#bib.bib39)\)to pool the sequence at layer 20 and sweep the keep\-ratio for multiple runs; quality falls gently and never collapses \([Figure˜2\(a\)](https://arxiv.org/html/2608.08809#S4.F2.sf1); Qwen30\.5520\.552at ratio 1\.0 to0\.5380\.538at ratio 0\.4\)\. We then apply the same pooling operation to the*retriever*— the identical code, now on a bi\-encoder — and obtain a curve with a similarly gentle shape \(0\.4970\.497to0\.5110\.511\)\. Different from the LTC reranker which trains a single \(layer, ratio\) point, we use MLTC, which jointly trains several ratios in one checkpoint\. Compared to the plain Qwen3\-0\.6B retriever, we note the MLTC retriever achieves competitive performance \(0\.511 vs 0\.513\), suggesting the sanity of our implementation\. We do not compare it directly with separately trained LTC retrievers due to limited bandwidth\.

#### Composition: depth width\.

The 2D Matryoshka schedule reproduces Starbucks on all three backbones \([Figure˜1](https://arxiv.org/html/2608.08809#S4.F1)c\); each frontier is smooth and monotonic \(BERT0\.2630\.4480\.263\\to 0\.448; Qwen30\.2500\.5210\.250\\to 0\.521; ModernBERT0\.1870\.4500\.187\\to 0\.450across six operating points from one checkpoint\)\. The shape matches Starbucks\. The comparison does not include the masked\-autoencoding pretraining used in that work, which is not part of our training recipe\.

#### Cost of serving many sizes\.

Training elastic models may degrade model performance compared to the standard training; we refer to this cost as*elasticity tax*\.[Table˜3](https://arxiv.org/html/2608.08809#S4.T3)compares the full model trained in the elastic way to a model trained for that size alone \(the*elasticity tax*\), using the depthwidth schedule so the comparison is the same across backbones\. We note the measured difference is small: on the retriever it is−0\.013\-0\.013and−0\.026\-0\.026for the two encoders and\+0\.008\+0\.008for the decoder, and on the reranker it is within0\.006\\pm 0\.006for all three\. The reranker tax is close to zero, while the retriever’s full point changes modestly on the two encoder backbones\. Among the retriever backbones, the Qwen3 decoder has the smallest tax\.

Table 3:Elasticity tax \(BEIR\-15 nDCG@10\)\. The depthwidth schedule is used for all backbones for comparability\.plainelastic \(2D\)taxBERT @ 12:7680\.4610\.448−0\.013\-0\.013ModernBERT @ 22:7680\.4760\.450−0\.026\-0\.026Qwen3 @ 28:10240\.5130\.521\+0\.008\+0\.008
#### Which axes each task admits\.

A retriever produces a vector and admits all three axes; a reranker produces a scalar score and admits depth and token but not width, since there is no embedding to truncate\. This is the one structural exclusion in the task\-by\-axis matrix, and it is why the reranker has fewer configurations than the retriever above\.

#### The three\-axis composition is left open\.

A depth width token retriever is expressible today \(e\.g\."24:256@0\.6/20"\), and the machinery runs it \(training and inference both route through thepool\_layerbranch of[Section˜3](https://arxiv.org/html/2608.08809#S3)\)\. We do not report an operating point for it; we provide the capability and leave the configuration to future use\.

### 4\.3Efficiency and deployment trade\-offs

We measure the deployment costs associated with each axis on a single controlled testbed: Qwen3\-0\.6B on MS MARCO dev, with MRR@10 as the quality axis \(a single in\-domain corpus avoids the domain\- and length\-confounds of averaging across BEIR\)\. We time on one GPU with a fixed workload, separately from the parallel evaluation runs\. The retriever has two encode paths with opposite profiles, and we report both document encoding \(offline, batch 128, length 512, reported as documents/second\) and query encoding \(on the request path, batch 1, length 32, reported as latency per query\)\. For reranking, we host the model through a Hugging Face backend, send HTTP requests in batches of 128 query–document pairs, and compute throughput from the end\-to\-end request latency\. We report efficiency results in[Figures˜3](https://arxiv.org/html/2608.08809#S4.F3)and[4](https://arxiv.org/html/2608.08809#S4.F4)\.

![Refer to caption](https://arxiv.org/html/2608.08809v1/figures/flops_improvement.png)Figure 3:Quality versus relative analytic cost\.Left:retrieval document encoding, depth and MLTC, against relative FLOPs; MRL is absent because it does not change FLOPs\.Center:retrieval against relative index size; only MRL moves along this axis\.Right:reranking against relative score FLOPs\. Depth and token reduce compute and leave the index unchanged; width reduces the index and leaves compute unchanged\.![Refer to caption](https://arxiv.org/html/2608.08809v1/figures/wallclock_improvement.png)Figure 4:Quality retained versus measured speedup\.Left:retrieval document throughput\.Center:online query latency at batch 1\.Right:reranking throughput\. Depth speeds up every path; the token axis helps document encoding modestly and online query encoding not at all, because pooling an eight\-token query saves little and the layers below the pooling point still run\.#### The measured speedup follows the cost model\.

For reranking at the batch size 128, the measured speedup matches the prediction \(1/relative FLOPs1/\\text\{relative FLOPs\}\) to within about two percent at every depth \(layer 4 measures6\.866\.86\\timesagainst a predicted7\.007\.00\\times; layer 16,1\.751\.75\\timesagainst1\.751\.75\\times\)\. Document encoding tracks just as closely \(layer 6,4\.614\.61\\timesmeasured against4\.674\.67\\timespredicted\)\. When the batch is large the work is dominated by arithmetic, so counting FLOPs is enough to predict the wall\-clock speedup\.

#### Depth and token help different paths\.

At layer 16, the reranker retains the full\-depth quality \(0\.3420\.342MRR@10\) while running1\.751\.75\\timesfaster\. The token axis gives a smaller speedup because the layers below the pooling point still run on the full sequence \(pooling at layer 20 of 28 leaves about0\.710\.71of the work in place\), though its quality falls more gently\. The gap is starker for queries: cutting depth still speeds up query encoding by about44\\timesat layer 6, while token pooling barely helps \(around0\.990\.99\\times\), since an eight\-token query has almost nothing to pool and the lower layers run regardless\. Token compression is therefore a lever for long documents, while depth speeds up both the offline corpus pass and the online query\. At batch 1 the query pass spends much of its time on overhead rather than arithmetic, so its speedup falls short of the FLOP prediction — which is why we measure query latency directly instead of reading it off the FLOP count\.

#### Width reduces storage and search\.

With an exact inner\-product index, both the stored index size and the search FLOPs are proportional to the embedding dimension, so the index size is a hardware\-independent measure of the width axis’s cost\. Reducing the Qwen3 embedding from dim 1024 to dim 64 reduces the MS MARCO index from about 36 GB to about 2\.3 GB while MRR@10 moves from 0\.329 to 0\.261; depth and token leave the index size unchanged\.

#### Which axis for which constraint\.

Taken together, the three axes address different deployment constraints: depth reduces compute on every path and is the most broadly applicable axis; the token axis reduces compute on long inputs specifically \(corpus encoding, not short queries\); and width reduces index storage and search cost\. The cost model and the figures above indicate which axis to reach for given a latency or storage budget\.

## 5Conclusion and Future Work

Tevatron\-Elasticexpresses depth, token, and width compression for both retrievers and rerankers through one small dataclass and a declarative schedule\. It reproduces four prior methods—MRL, early exit, 2D Matryoshka, and LTC—as configurations, and runs across five encoder and decoder backbone families with no new model code \(the token axis adds one small per\-family shim\)\. Across 20 checkpoints the elastic curves are smooth and monotonic, the cost of joint training relative to a dedicated model is small, and the measured inference speedup follows the analytic cost model\. The same interface enables MLTC, which serves several token\-compression ratios from one retriever checkpoint\. We release the code and all checkpoints so that the three axes can be combined and extended directly\.

We identify two promising future directions based on experiences buildingTevatron\-Elastic\. First, the token axis currently targets global\-attention backbones; extending it to sliding\-window\-attention models \(such as ModernBERT\) needs a pooling rule compatible with a fixed local window, after which the token axis applies to that family as well\. Second, the abstraction already expresses the full depthwidthtoken composition on the retriever \([Section˜4](https://arxiv.org/html/2608.08809#S4)\); a practitioner with a specific latency and storage budget can train and select that operating point directly\. Because a new axis, backbone, or per\-granularity loss enters through a single interface, we expect further compression methods to slot in as additional configurations rather than as new systems\. Lastly, learned sparse retrieval\(Formal et al\.,[2021b](https://arxiv.org/html/2608.08809#bib.bib6),[a](https://arxiv.org/html/2608.08809#bib.bib5); Lassance et al\.,[2024](https://arxiv.org/html/2608.08809#bib.bib14); Xu et al\.,[2025a](https://arxiv.org/html/2608.08809#bib.bib27),[2026f](https://arxiv.org/html/2608.08809#bib.bib35)\)remains an interesting yet underexplored direction for elastic modeling\.

## References

- Cao et al\. \(2007\)Zhe Cao, Tao Qin, Tie\-Yan Liu, Ming\-Feng Tsai, and Hang Li\.Learning to rank: from pairwise approach to listwise approach\.In*Proceedings of the 24th international conference on Machine learning*, pages 129–136, 2007\.
- Devlin et al\. \(2019\)Jacob Devlin, Ming\-Wei Chang, Kenton Lee, and Kristina Toutanova\.BERT: Pre\-training of deep bidirectional transformers for language understanding\.In Jill Burstein, Christy Doran, and Thamar Solorio, editors,*Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\)*, pages 4171–4186, Minneapolis, Minnesota, June 2019\. Association for Computational Linguistics\.[10\.18653/v1/N19\-1423](https://arxiv.org/doi.org/10.18653/v1/N19-1423)\.URL[https://aclanthology\.org/N19\-1423/](https://aclanthology.org/N19-1423/)\.
- Elhage et al\. \(2021\)Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield\-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah\.A mathematical framework for transformer circuits\.*Transformer Circuits Thread*, 2021\.https://transformer\-circuits\.pub/2021/framework/index\.html\.
- Elhoushi et al\. \(2024\)Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole\-Jean Wu\.Layerskip: Enabling early exit inference and self\-speculative decoding, August 2024\.URL[https://aclanthology\.org/2024\.acl\-long\.681](https://aclanthology.org/2024.acl-long.681)\.
- Formal et al\. \(2021a\)Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant\.Splade v2: Sparse lexical and expansion model for information retrieval, 2021a\.URL[https://arxiv\.org/abs/2109\.10086](https://arxiv.org/abs/2109.10086)\.
- Formal et al\. \(2021b\)Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant\.*SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking*, page 2288–2292\.Association for Computing Machinery, New York, NY, USA, 2021b\.ISBN 9781450380379\.URL[https://doi\.org/10\.1145/3404835\.3463098](https://doi.org/10.1145/3404835.3463098)\.
- Gao and Callan \(2022\)Luyu Gao and Jamie Callan\.Unsupervised corpus aware language model pre\-training for dense passage retrieval\.In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors,*Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 2843–2853, Dublin, Ireland, May 2022\. Association for Computational Linguistics\.[10\.18653/v1/2022\.acl\-long\.203](https://arxiv.org/doi.org/10.18653/v1/2022.acl-long.203)\.URL[https://aclanthology\.org/2022\.acl\-long\.203/](https://aclanthology.org/2022.acl-long.203/)\.
- Goyal et al\. \(2020\)Saurabh Goyal, Anamitra Roy Choudhury, Saurabh Raje, Venkatesan Chakaravarthy, Yogish Sabharwal, and Ashish Verma\.PoWER\-BERT: Accelerating BERT inference via progressive word\-vector elimination\.In Hal Daumé III and Aarti Singh, editors,*Proceedings of the 37th International Conference on Machine Learning*, volume 119 of*Proceedings of Machine Learning Research*, pages 3690–3699\. PMLR, 13–18 Jul 2020\.URL[https://proceedings\.mlr\.press/v119/goyal20a\.html](https://proceedings.mlr.press/v119/goyal20a.html)\.
- Grattafiori et al\. \(2024\)Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al\.The llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*, 2024\.
- Hinton et al\. \(2015\)Geoffrey Hinton, Oriol Vinyals, and Jeff Dean\.Distilling the knowledge in a neural network, 2015\.URL[https://arxiv\.org/abs/1503\.02531](https://arxiv.org/abs/1503.02531)\.
- Jiang et al\. \(2023\)Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie\-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed\.Mistral 7b\.*ArXiv*, abs/2310\.06825, 2023\.URL[https://api\.semanticscholar\.org/CorpusID:263830494](https://api.semanticscholar.org/CorpusID:263830494)\.
- Johnson et al\. \(2017\)Jeff Johnson, Matthijs Douze, and Hervé Jégou\.Billion\-scale similarity search with gpus, 2017\.URL[https://arxiv\.org/abs/1702\.08734](https://arxiv.org/abs/1702.08734)\.
- Kusupati et al\. \(2022\)Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard\-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, et al\.Matryoshka representation learning\.*Advances in Neural Information Processing Systems*, 35:30233–30249, 2022\.
- Lassance et al\. \(2024\)Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant\.Splade\-v3: New baselines for splade\.*arXiv preprint arXiv:2403\.06789*, 2024\.
- Li et al\. \(2024\)Xianming Li, Zongxi Li, Jing Li, Haoran Xie, and Qing Li\.2d matryoshka sentence embeddings, 2024\.URL[https://arxiv\.org/abs/2402\.14776](https://arxiv.org/abs/2402.14776)\.
- Lin et al\. \(2022\)Jimmy Lin, Rodrigo Nogueira, and Andrew Yates\.*Pretrained transformers for text ranking: Bert and beyond*\.Springer Nature, 2022\.
- Liu et al\. \(2020\)Weijie Liu, Peng Zhou, Zhiruo Wang, Zhe Zhao, Haotang Deng, and Qi Ju\.FastBERT: a self\-distilling BERT with adaptive inference time\.In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors,*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 6035–6044, Online, July 2020\. Association for Computational Linguistics\.[10\.18653/v1/2020\.acl\-main\.537](https://arxiv.org/doi.org/10.18653/v1/2020.acl-main.537)\.URL[https://aclanthology\.org/2020\.acl\-main\.537/](https://aclanthology.org/2020.acl-main.537/)\.
- Ma et al\. \(2025\)Xueguang Ma, Luyu Gao, Shengyao Zhuang, Jiaqi Samantha Zhan, Jamie Callan, and Jimmy Lin\.Tevatron 2\.0: Unified document retrieval toolkit across scale, language, and modality\.In*Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR ’25, page 4061–4065, New York, NY, USA, 2025\. Association for Computing Machinery\.ISBN 9798400715921\.[10\.1145/3726302\.3730135](https://arxiv.org/doi.org/10.1145/3726302.3730135)\.URL[https://doi\.org/10\.1145/3726302\.3730135](https://doi.org/10.1145/3726302.3730135)\.
- Oord et al\. \(2018\)Aaron van den Oord, Yazhe Li, and Oriol Vinyals\.Representation learning with contrastive predictive coding\.*arXiv preprint arXiv:1807\.03748*, 2018\.
- Qu et al\. \(2021\)Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang\.RocketQA: An optimized training approach to dense passage retrieval for open\-domain question answering\.In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani\-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors,*Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 5835–5847, Online, June 2021\. Association for Computational Linguistics\.[10\.18653/v1/2021\.naacl\-main\.466](https://arxiv.org/doi.org/10.18653/v1/2021.naacl-main.466)\.URL[https://aclanthology\.org/2021\.naacl\-main\.466/](https://aclanthology.org/2021.naacl-main.466/)\.
- Thakur et al\. \(2021\)Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych\.BEIR: A heterogeneous benchmark for zero\-shot evaluation of information retrieval models\.In*Thirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\)*, 2021\.URL[https://openreview\.net/forum?id=wCu6T5xFjeJ](https://openreview.net/forum?id=wCu6T5xFjeJ)\.
- Thakur et al\. \(2025\)Nandan Thakur, Crystina Zhang, Xueguang Ma, and Jimmy Lin\.Hard negatives, hard lessons: Revisiting training data quality for robust information retrieval with LLMs\.In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,*Findings of the Association for Computational Linguistics: EMNLP 2025*, pages 9064–9083, Suzhou, China, November 2025\. Association for Computational Linguistics\.ISBN 979\-8\-89176\-335\-7\.[10\.18653/v1/2025\.findings\-emnlp\.481](https://arxiv.org/doi.org/10.18653/v1/2025.findings-emnlp.481)\.URL[https://aclanthology\.org/2025\.findings\-emnlp\.481/](https://aclanthology.org/2025.findings-emnlp.481/)\.
- Warner et al\. \(2024\)Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, et al\.Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference\.*arXiv preprint arXiv:2412\.13663*, 2024\.
- Xia et al\. \(2008\)Fen Xia, Tie\-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li\.Listwise approach to learning to rank: theory and algorithm\.In*Proceedings of the 25th International Conference on Machine Learning*, ICML ’08, page 1192–1199, New York, NY, USA, 2008\. Association for Computing Machinery\.ISBN 9781605582054\.[10\.1145/1390156\.1390306](https://arxiv.org/doi.org/10.1145/1390156.1390306)\.URL[https://doi\.org/10\.1145/1390156\.1390306](https://doi.org/10.1145/1390156.1390306)\.
- Xin et al\. \(2020\)Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin\.DeeBERT: Dynamic early exiting for accelerating BERT inference\.In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors,*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 2246–2251, Online, July 2020\. Association for Computational Linguistics\.[10\.18653/v1/2020\.acl\-main\.204](https://arxiv.org/doi.org/10.18653/v1/2020.acl-main.204)\.URL[https://aclanthology\.org/2020\.acl\-main\.204/](https://aclanthology.org/2020.acl-main.204/)\.
- Xiong et al\. \(2021\)Lee Xiong, Chenyan Xiong, Ye Li, Kwok\-Fung Tang, Jialin Liu, Paul N\. Bennett, Junaid Ahmed, and Arnold Overwijk\.Approximate nearest neighbor negative contrastive learning for dense text retrieval\.In*International Conference on Learning Representations*, 2021\.URL[https://openreview\.net/forum?id=zeFrfgyZln](https://openreview.net/forum?id=zeFrfgyZln)\.
- Xu et al\. \(2025a\)Zhichao Xu, Aosong Feng, Yijun Tian, Haibo Ding, and Lin Lee Cheong\.CSPLADE: Learned sparse retrieval with causal language models\.In Kentaro Inui, Sakriani Sakti, Haofen Wang, Derek F\. Wong, Pushpak Bhattacharyya, Biplab Banerjee, Asif Ekbal, Tanmoy Chakraborty, and Dhirendra Pratap Singh, editors,*Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics*, pages 99–114, Mumbai, India, December 2025a\. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics\.ISBN 979\-8\-89176\-298\-5\.[10\.18653/v1/2025\.ijcnlp\-long\.7](https://arxiv.org/doi.org/10.18653/v1/2025.ijcnlp-long.7)\.URL[https://aclanthology\.org/2025\.ijcnlp\-long\.7/](https://aclanthology.org/2025.ijcnlp-long.7/)\.
- Xu et al\. \(2025b\)Zhichao Xu, Zhiqi Huang, Shengyao Zhuang, and Vivek Srikumar\.Distillation versus contrastive learning: How to train your rerankers\.In Kentaro Inui, Sakriani Sakti, Haofen Wang, Derek F\. Wong, Pushpak Bhattacharyya, Biplab Banerjee, Asif Ekbal, Tanmoy Chakraborty, and Dhirendra Pratap Singh, editors,*Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics*, pages 564–578, Mumbai, India, December 2025b\. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics\.ISBN 979\-8\-89176\-303\-6\.[10\.18653/v1/2025\.findings\-ijcnlp\.33](https://arxiv.org/doi.org/10.18653/v1/2025.findings-ijcnlp.33)\.URL[https://aclanthology\.org/2025\.findings\-ijcnlp\.33/](https://aclanthology.org/2025.findings-ijcnlp.33/)\.
- Xu et al\. \(2025c\)Zhichao Xu, Jinghua Yan, Ashim Gupta, and Vivek Srikumar\.State space models are strong text rerankers\.In Vaibhav Adlakha, Alexandra Chronopoulou, Xiang Lorraine Li, Bodhisattwa Prasad Majumder, Freda Shi, and Giorgos Vernikos, editors,*Proceedings of the 10th Workshop on Representation Learning for NLP \(RepL4NLP\-2025\)*, pages 152–169, Albuquerque, NM, May 2025c\. Association for Computational Linguistics\.ISBN 979\-8\-89176\-245\-9\.[10\.18653/v1/2025\.repl4nlp\-1\.12](https://arxiv.org/doi.org/10.18653/v1/2025.repl4nlp-1.12)\.URL[https://aclanthology\.org/2025\.repl4nlp\-1\.12/](https://aclanthology.org/2025.repl4nlp-1.12/)\.
- Xu et al\. \(2026a\)Zhichao Xu, Xueguang Ma, Shengyao Zhuang, Luyu Gao, Wenqian Ye, Yu Wang, Jamie Callan, and Jimmy Lin\.Tevatron meets megatron: Expert\-parallel llm reranker training on an academic budget, 2026a\.URL[https://arxiv\.org/abs/2608\.00916](https://arxiv.org/abs/2608.00916)\.
- Xu et al\. \(2026b\)Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang Phillips, Jimmy Lin, and Vivek Srikumar\.A survey of model architectures in information retrieval\.*Transactions on Machine Learning Research*, 2026b\.ISSN 2835\-8856\.URL[https://openreview\.net/forum?id=xAIbTbHRrX](https://openreview.net/forum?id=xAIbTbHRrX)\.Survey Certification\.
- Xu et al\. \(2026c\)Zhichao Xu, Minheng Wang, Yawei Wang, Wenqian Ye, Yuntao Du, Yunpu Ma, and Yijun Tian\.Recon: Reasoning with condensation for efficient retrieval\-augmented generation, 2026c\.URL[https://arxiv\.org/abs/2510\.10448](https://arxiv.org/abs/2510.10448)\.
- Xu et al\. \(2026d\)Zhichao Xu, Zongyu Wu, Yun Zhou, Aosong Feng, Kang Zhou, Sangmin Woo, Kiran Ramnath, Yijun Tian, Xuan Qi, Weikang Qiu, Lin Lee Cheong, and Haibo Ding\.Beyond correctness: Rewarding faithful reasoning in retrieval\-augmented generation\.*Transactions on Machine Learning Research*, 2026d\.ISSN 2835\-8856\.URL[https://openreview\.net/forum?id=mZ0gGlXelF](https://openreview.net/forum?id=mZ0gGlXelF)\.
- Xu et al\. \(2026e\)Zhichao Xu, Shengyao Zhuang, Xueguang Ma, Bingsen Chen, Yijun Tian, Fengran Mo, Tao Li, Jie Cao, and Vivek Srikumar\.Rethinking on\-policy optimization for query augmentation\.*Transactions on Machine Learning Research*, 2026e\.ISSN 2835\-8856\.URL[https://openreview\.net/forum?id=mmqbjhz5Br](https://openreview.net/forum?id=mmqbjhz5Br)\.
- Xu et al\. \(2026f\)Zhichao Xu, Shengyao Zhuang, Crystina Zhang, Xueguang Ma, Yijun Tian, Maitrey Mehta, Jimmy Lin, and Vivek Srikumar\.Laconic: Dense\-level effectiveness for scalable sparse retrieval via a two\-phase training curriculum\.In*Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR ’26, page 4298–4304, New York, NY, USA, 2026f\. Association for Computing Machinery\.ISBN 9798400725999\.[10\.1145/3805712\.3809869](https://arxiv.org/doi.org/10.1145/3805712.3809869)\.URL[https://doi\.org/10\.1145/3805712\.3809869](https://doi.org/10.1145/3805712.3809869)\.
- Yang et al\. \(2025\)An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu\.Qwen3 technical report, 2025\.URL[https://arxiv\.org/abs/2505\.09388](https://arxiv.org/abs/2505.09388)\.
- Zhang et al\. \(2025\)Dun Zhang, Ziyang Zeng, Yudong Zhou, and Shuyang Lu\.Jasper\-token\-compression\-600m technical report, 2025\.URL[https://arxiv\.org/abs/2511\.14405](https://arxiv.org/abs/2511.14405)\.
- Zhuang et al\. \(2026a\)Shengyao Zhuang, Shuai Wang, Fabio Zheng, Bevan Koopman, and Guido Zuccon\.Starbucks: Improved training for 2d matryoshka embeddings\.In*Advances in Information Retrieval: 48th European Conference on Information Retrieval, ECIR 2026, Delft, The Netherlands, March 29 – April 2, 2026, Proceedings, Part I*, page 67–82, Berlin, Heidelberg, 2026a\. Springer\-Verlag\.ISBN 978\-3\-032\-21288\-7\.[10\.1007/978\-3\-032\-21289\-4\_5](https://arxiv.org/doi.org/10.1007/978-3-032-21289-4_5)\.URL[https://doi\.org/10\.1007/978\-3\-032\-21289\-4\_5](https://doi.org/10.1007/978-3-032-21289-4_5)\.
- Zhuang et al\. \(2026b\)Shengyao Zhuang, Zhichao Xu, and Ivano Lauriola\.Layer\-wise token compression for efficient document reranking\.In*Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval*, SIGIR ’26, page 4426–4432, New York, NY, USA, 2026b\. Association for Computing Machinery\.ISBN 9798400725999\.[10\.1145/3805712\.3809871](https://arxiv.org/doi.org/10.1145/3805712.3809871)\.URL[https://doi\.org/10\.1145/3805712\.3809871](https://doi.org/10.1145/3805712.3809871)\.

## Appendix AComplete per\-dataset results

We report nDCG@10 on each of the 15 BEIR datasets for every operating point of all 20 trained checkpoints \(13 retriever, 7 reranker\)\. Each*elastic*checkpoint is one model serving all listed operating points viaprune\_to/encode\_at;*plain*rows are dedicated single\-point baselines with the identical recipe\. TheAVGcolumn is the BEIR\-15 single macro average \(the 14 standalone datasets plus CQA, the mean of the 12 cqadupstack subforums; this matches the aggregate figures in the main text\)\. Dataset abbreviations: ARG arguana, CFE climate\-fever, DBP dbpedia\-entity, FEV fever, FIQ fiqa, HOP hotpotqa, NFC nfcorpus, NQ nq, QUO quora, SCD scidocs, SCF scifact, TRC trec\-covid, TOU webis\-touche2020, MSM ms\-marco dev, CQA cqadupstack \(mean of 12\)\. Granularity labels:L:DL\{:\}Dexit layerLLand dimDD\(retriever\);LLexit layer \(reranker\);L​R​r​P​pL\{\\times\}\\mathrm\{R\}r\{\\times\}\\mathrm\{P\}pexitLL, keep\-ratiorr, pool layerpp\(token / M\-LTC\)\. These are the checkpoints we release as artifacts\.

Table 4:BERT retriever, plain\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG12:7680\.5760\.2940\.3680\.8620\.2840\.6450\.2860\.5020\.8460\.1700\.5860\.5800\.2350\.3600\.3120\.461

Table 5:BERT retriever, depth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG2:7680\.3680\.1830\.2780\.7390\.1620\.4590\.2130\.3070\.7880\.1050\.4090\.5170\.1880\.2460\.2030\.3444:7680\.4490\.2220\.3070\.8090\.1980\.5340\.2290\.3780\.8110\.1260\.4790\.4930\.2140\.2930\.2360\.3856:7680\.4890\.2450\.3250\.8340\.2220\.5780\.2380\.4270\.8240\.1430\.5230\.4810\.2200\.3220\.2640\.4098:7680\.5220\.2750\.3580\.8550\.2440\.6220\.2600\.4730\.8360\.1580\.5460\.5080\.2440\.3420\.2860\.43510:7680\.5450\.2870\.3640\.8640\.2610\.6380\.2640\.5020\.8420\.1600\.5710\.5410\.2550\.3510\.2940\.44912:7680\.5520\.2810\.3710\.8590\.2670\.6360\.2720\.5050\.8440\.1570\.5700\.5840\.2540\.3520\.2970\.453

Table 6:BERT retriever, width \(MRL\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG12:320\.4610\.1610\.1960\.5990\.1390\.2750\.1620\.3020\.7860\.0970\.3570\.3750\.1690\.2560\.1660\.30012:640\.5260\.2240\.2920\.7840\.1990\.4750\.2130\.4060\.8220\.1250\.4600\.5150\.2150\.3220\.2320\.38712:1280\.5490\.2570\.3310\.8310\.2340\.5550\.2360\.4450\.8340\.1400\.5050\.5400\.2170\.3410\.2600\.41812:2560\.5600\.2700\.3500\.8530\.2550\.5890\.2560\.4670\.8390\.1480\.5200\.5520\.2350\.3500\.2740\.43512:5120\.5660\.2820\.3550\.8640\.2560\.6060\.2610\.4770\.8410\.1500\.5230\.5440\.2370\.3550\.2830\.44012:7680\.5650\.2850\.3640\.8660\.2600\.6130\.2650\.4830\.8420\.1550\.5270\.5640\.2370\.3560\.2880\.445

Table 7:BERT retriever, depthwidth \(Starbucks\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG2:320\.3050\.1080\.2250\.5770\.1000\.2220\.1520\.2210\.7390\.0760\.2810\.4510\.1510\.1870\.1420\.2634:640\.4400\.1700\.2680\.7400\.1650\.4020\.1970\.3150\.7940\.1100\.3700\.4340\.1910\.2560\.1980\.3376:1280\.4820\.2090\.2880\.7960\.2020\.5100\.2170\.3890\.8190\.1330\.4590\.4210\.2050\.3050\.2400\.3788:2560\.5110\.2550\.3340\.8410\.2370\.5920\.2410\.4580\.8370\.1530\.5150\.5030\.2200\.3330\.2790\.42110:5120\.5300\.2820\.3570\.8560\.2570\.6240\.2650\.5000\.8460\.1550\.5580\.5390\.2360\.3480\.2950\.44312:7680\.5450\.2750\.3600\.8540\.2710\.6280\.2730\.5060\.8470\.1560\.5670\.5440\.2350\.3540\.3020\.448

Table 8:ModernBERT retriever, plain\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG22:7680\.5290\.3170\.3450\.8400\.3410\.6300\.2960\.4970\.8650\.1770\.6280\.7400\.2310\.3610\.3400\.476

Table 9:ModernBERT retriever, depth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG4:7680\.2810\.1180\.1450\.5410\.1030\.2810\.1320\.1170\.7730\.0760\.2500\.3690\.1910\.1190\.1450\.2437:7680\.3530\.1380\.1660\.6080\.1390\.3680\.1560\.1730\.7930\.0900\.3180\.4330\.2100\.1640\.1880\.28710:7680\.3800\.2140\.2050\.6980\.1770\.4330\.1830\.2410\.8160\.1060\.4020\.5170\.2270\.2090\.2380\.33613:7680\.3460\.2180\.2020\.6760\.1780\.4060\.1670\.2370\.8200\.1030\.3650\.5130\.2170\.2120\.2350\.32616:7680\.3690\.2680\.2470\.7530\.2520\.4960\.2280\.3600\.8440\.1300\.4820\.6720\.2310\.2840\.2860\.39319:7680\.4360\.3100\.3170\.7870\.3060\.5740\.2760\.4390\.8570\.1590\.5900\.7700\.2580\.3250\.3260\.44922:7680\.4880\.2840\.3170\.7950\.3130\.5910\.2840\.4460\.8600\.1600\.5890\.7810\.2640\.3310\.3360\.456

Table 10:ModernBERT retriever, width \(MRL\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG22:320\.2670\.1640\.1220\.4440\.1500\.1860\.1520\.1890\.8070\.0910\.2970\.4180\.1680\.1990\.1460\.25322:640\.3520\.2430\.2300\.6780\.2350\.3990\.2100\.3240\.8440\.1290\.4810\.5700\.1980\.2860\.2360\.36122:1280\.4050\.2860\.2840\.7480\.2770\.5250\.2480\.4100\.8560\.1480\.5300\.6490\.2510\.3180\.2880\.41522:2560\.4370\.3000\.3160\.8010\.2950\.5830\.2760\.4360\.8610\.1630\.5750\.6840\.2580\.3360\.3070\.44222:5120\.4510\.3190\.3370\.8240\.3170\.6100\.2850\.4510\.8630\.1700\.5980\.7050\.2460\.3450\.3250\.45622:7680\.4510\.3240\.3450\.8300\.3240\.6190\.2920\.4580\.8640\.1710\.6080\.7180\.2590\.3460\.3310\.463

Table 11:ModernBERT retriever, depthwidth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG4:320\.3000\.0850\.0880\.3930\.0740\.1360\.0820\.0670\.7220\.0500\.1840\.2760\.1630\.0790\.1020\.1877:640\.3610\.1150\.1460\.6150\.1240\.2820\.1310\.1480\.7810\.0860\.3170\.4040\.2470\.1400\.1770\.27210:1280\.2950\.2040\.2030\.7190\.1670\.3990\.1780\.2310\.8170\.1130\.4010\.5160\.2320\.2140\.2220\.32713:2560\.3000\.2190\.2320\.7530\.1900\.4570\.1860\.2930\.8350\.1220\.4130\.5940\.2290\.2450\.2570\.35516:5120\.3710\.2780\.2880\.7630\.2720\.5370\.2630\.4060\.8560\.1470\.5650\.6860\.2430\.3110\.2990\.41922:7680\.4200\.2840\.3160\.7920\.3020\.5990\.3010\.4370\.8630\.1650\.6310\.7410\.2260\.3390\.3350\.450

Table 12:Qwen3\-0\.6B retriever, plain\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG28:10240\.6500\.3440\.3860\.8550\.3930\.7380\.3440\.5390\.8580\.2250\.6860\.7040\.2340\.3920\.3490\.513

Table 13:Qwen3\-0\.6B retriever, depth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG6:10240\.4880\.2600\.3140\.8040\.2480\.6030\.2790\.4050\.8350\.1680\.5800\.6460\.2190\.3300\.2840\.43110:10240\.5540\.2920\.3300\.8110\.2890\.6090\.3140\.4470\.8440\.1860\.6050\.7390\.2570\.3410\.2830\.46014:10240\.5920\.3210\.3260\.8170\.3200\.6210\.3170\.4930\.8450\.1960\.6200\.7670\.2370\.3500\.2890\.47418:10240\.6340\.3120\.3380\.8200\.3360\.6190\.3200\.4920\.8380\.1990\.6160\.7530\.2100\.3550\.3100\.47724:10240\.6410\.2930\.3680\.8370\.3530\.6730\.3120\.5090\.8450\.2030\.6390\.6990\.2300\.3680\.3140\.48628:10240\.6610\.3390\.3890\.8560\.3870\.7340\.3460\.5500\.8540\.2140\.6830\.7350\.2680\.3930\.3590\.518

Table 14:Qwen3\-0\.6B retriever, width \(MRL\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG28:640\.4700\.2310\.2130\.6660\.2650\.3920\.2260\.3340\.8200\.1720\.4810\.4820\.2280\.3110\.1830\.36528:1280\.5420\.2780\.2950\.7860\.3180\.5670\.2790\.4180\.8380\.1970\.5490\.5770\.2440\.3510\.2510\.43328:2560\.5790\.3010\.3380\.8160\.3510\.6440\.3060\.4600\.8460\.2070\.5970\.6250\.2570\.3680\.2870\.46528:5120\.6120\.3200\.3620\.8330\.3750\.6890\.3230\.4870\.8530\.2150\.6200\.6630\.2750\.3810\.3200\.48928:7680\.6230\.3330\.3700\.8460\.3840\.7130\.3330\.4960\.8550\.2200\.6400\.6710\.2710\.3870\.3380\.49928:10240\.6280\.3430\.3820\.8550\.3890\.7260\.3390\.5030\.8570\.2220\.6460\.6760\.2660\.3910\.3450\.505

Table 15:Qwen3\-0\.6B retriever, depthwidth \(2D\-Matryoshka\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVG6:640\.3460\.1170\.1010\.5010\.0980\.2280\.1390\.1440\.7750\.1060\.3180\.4570\.1950\.1410\.0850\.25010:1280\.4600\.2160\.1950\.6830\.1760\.3930\.2100\.2890\.8240\.1500\.4530\.5630\.2480\.2350\.1600\.35014:2560\.5650\.3010\.2870\.7770\.2720\.5270\.2910\.4400\.8420\.1910\.5640\.6690\.2460\.3170\.2370\.43518:5120\.6200\.3290\.3260\.8070\.3330\.5810\.3130\.4950\.8390\.1970\.6010\.7150\.2420\.3460\.2930\.46924:7680\.6570\.3200\.3510\.8200\.3510\.6410\.3020\.5110\.8420\.2020\.6380\.6350\.2260\.3620\.3120\.47828:10240\.6770\.3620\.3890\.8500\.3960\.7310\.3410\.5600\.8570\.2200\.6930\.7300\.2560\.3900\.3600\.521

Table 16:Qwen3\-0\.6B retriever, token \(M\-LTC, pool@20\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL28xR0\.4xP200\.6460\.3490\.3570\.8270\.3420\.6850\.3340\.5320\.8410\.2110\.6770\.7120\.2320\.3700\.3330\.497L28xR0\.6xP200\.6570\.3630\.3690\.8320\.3660\.7040\.3290\.5400\.8500\.2150\.6830\.7340\.2450\.3740\.3420\.507L28xR0\.8xP200\.6600\.3690\.3760\.8450\.3760\.7130\.3380\.5460\.8520\.2170\.6810\.7650\.2580\.3780\.3480\.515L28xR1\.0xP200\.6550\.3770\.3810\.8420\.3680\.7220\.3390\.5460\.8550\.2190\.6690\.7120\.2450\.3890\.3490\.511

Table 17:BERT reranker, plain\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL120\.2900\.2260\.4070\.8460\.3000\.7570\.3520\.4770\.7800\.1540\.6860\.6660\.3310\.3420\.3210\.462

Table 18:BERT reranker, depth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL20\.0030\.0250\.0620\.1140\.0110\.0290\.1410\.0160\.0050\.0110\.0110\.3000\.0500\.0110\.0040\.053L40\.2560\.0940\.2040\.4880\.1350\.3430\.2740\.2050\.5630\.0890\.4760\.5370\.2670\.1440\.1790\.284L60\.2610\.2050\.3860\.8220\.2860\.7010\.3480\.4280\.7600\.1450\.6880\.6540\.3360\.3290\.3100\.444L80\.2510\.2310\.3920\.8300\.2970\.7240\.3470\.4430\.7660\.1490\.6960\.6570\.3440\.3310\.3140\.451L100\.2600\.2260\.3960\.8290\.3030\.7300\.3460\.4470\.7580\.1490\.7030\.6560\.3370\.3310\.3190\.453L120\.2650\.2160\.3880\.8130\.3000\.7310\.3460\.4480\.7590\.1500\.7070\.6570\.3410\.3330\.3100\.451

Table 19:ModernBERT reranker, plain\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL220\.2170\.1610\.3820\.8320\.3460\.7220\.3340\.4670\.7650\.1350\.6830\.7700\.3730\.3380\.3250\.457

Table 20:ModernBERT reranker, depth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL40\.1030\.0600\.1190\.4420\.0880\.3100\.2100\.1200\.4150\.0710\.4240\.4370\.2300\.0870\.1300\.216L70\.1460\.0930\.2150\.6400\.1730\.4220\.2720\.2320\.5910\.0990\.5190\.5050\.2920\.1790\.2310\.307L100\.1370\.0840\.2170\.6520\.1840\.4210\.2850\.2430\.6240\.1040\.5220\.5210\.3000\.1920\.2360\.315L130\.2340\.1710\.3550\.8050\.2980\.6570\.3460\.4010\.8130\.1460\.6740\.6720\.3110\.3110\.3250\.435L160\.2240\.2020\.3950\.8400\.3380\.7230\.3480\.4510\.8030\.1550\.7080\.7680\.3630\.3350\.3410\.466L190\.2220\.1980\.3940\.8270\.3460\.7160\.3490\.4510\.8050\.1560\.7000\.7510\.3440\.3330\.3400\.462L220\.2360\.1750\.3900\.8330\.3490\.7210\.3490\.4580\.8090\.1520\.7150\.7540\.3280\.3360\.3380\.463

Table 21:Qwen3\-0\.6B reranker, plain\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL280\.6040\.3420\.4490\.8940\.4320\.7990\.3920\.5960\.8530\.2070\.7620\.8670\.3570\.3990\.3900\.556

Table 22:Qwen3\-0\.6B reranker, depth\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL40\.0110\.0470\.0620\.1110\.0090\.0270\.1300\.0160\.0060\.0110\.0110\.3070\.1020\.0150\.0050\.058L80\.5250\.2510\.3540\.8480\.2880\.7240\.3230\.4030\.8480\.1730\.6760\.7210\.2670\.2870\.3340\.468L120\.5350\.2730\.3650\.8540\.2950\.7370\.3290\.4380\.8560\.1660\.6740\.7440\.2900\.3120\.3140\.479L160\.5930\.3400\.4560\.8900\.4240\.7960\.3820\.5910\.8600\.2010\.7580\.8700\.3610\.3990\.3960\.554L200\.6000\.3440\.4570\.8920\.4200\.7960\.3900\.5920\.8290\.2030\.7590\.8640\.3680\.4000\.3910\.554L240\.5970\.3300\.4470\.8890\.4200\.7930\.3900\.5890\.7780\.1990\.7540\.8580\.3500\.4000\.3860\.545L280\.6030\.3370\.4530\.8910\.4220\.7970\.3890\.5920\.8480\.2010\.7550\.8620\.3680\.4000\.3890\.554

Table 23:Qwen3\-0\.6B reranker, token \(LTC, pool@20\)\.ModelARGCFEDBPFEVFIQHOPNFCNQQUOSCDSCFTRCTOUMSMCQAAVGL20xR0\.40\.5720\.3370\.4500\.8840\.4060\.7960\.3800\.5930\.7550\.2030\.7550\.8630\.3050\.4010\.3680\.538L20xR0\.60\.5930\.3380\.4430\.8850\.4060\.7960\.3830\.5900\.7580\.2040\.7420\.8530\.3020\.4000\.3940\.539L20xR0\.80\.5780\.3380\.4480\.8860\.4090\.7960\.3840\.5910\.8140\.2030\.7530\.8590\.3190\.4010\.3820\.544L20xR1\.00\.6100\.3310\.4490\.8910\.4230\.7980\.3880\.5910\.8420\.2040\.7540\.8690\.3410\.4010\.3940\.552