Introducing the Ettin Reranker Family

Source: https://huggingface.co/blog/ettin-reranker Back to Articles

- TL;DR - Table of contents - What is a reranker, and why pair one with an embedder? - Usage- End-to-end retrieve-then-rerank pipeline - Architecture Details - Results- MTEB(eng, v2) Retrieval - Speed - Training- Distillation recipe - Dataset - Training Arguments - Evaluation - Overall Training Script - Conclusion - Acknowledgements - Citation https://huggingface.co/blog/ettin-reranker#tldrTL;DR

Today I’m releasing six newSentence TransformersCrossEncoder rerankers, state-of-the-art at their respective sizes, built on top of theEttinModernBERT encoders, together with the data and full training recipe that produced them:

• cross\-encoder/ettin\-reranker\-17m\-v1
• cross\-encoder/ettin\-reranker\-32m\-v1
• cross\-encoder/ettin\-reranker\-68m\-v1
• cross\-encoder/ettin\-reranker\-150m\-v1
• cross\-encoder/ettin\-reranker\-400m\-v1
• cross\-encoder/ettin\-reranker\-1b\-v1

The models were trained with adistillation recipe: pointwise MSE onmixedbread\-ai/mxbai\-rerank\-large\-v2scores overcross\-encoder/ettin\-reranker\-v1\-data, which is a subset oflightonai/embeddings\-pre\-trainingmixed with a reranked subset oflightonai/embeddings\-fine\-tuning.

Our six rerankers paired withgoogle/embeddinggemma\-300mon MTEB(eng, v2) Retrieval. SeeResultsfor five more embedder pairings.

If you’re new to rerankers and want the “why” first, jump toWhat is a reranker, and why pair one with an embedder?. If you just want to plug a model in, jump toUsage. If you want to train your own, jump toTraining.

I bootstrapped the training recipe below with the newtrain\-sentence\-transformersAgent Skillshipped inSentence Transformers v5.5.0. Install it withhf skills add train\-sentence\-transformers \[\-\-global\] \[\-\-claude\]and ask your AI coding agent (Claude Code, Codex, Cursor, Gemini CLI, ...) to fine-tune aSentenceTransformer,CrossEncoder, orSparseEncodermodel on your data.

https://huggingface.co/blog/ettin-reranker#table-of-contentsTable of contents

• What is a reranker, and why pair one with an embedder?
• Usage- End-to-end retrieve-then-rerank pipeline
• Architecture Details
• Results- MTEB(eng, v2) Retrieval - Speed
• Training- Distillation recipe - Dataset - Training Arguments - Evaluation - Overall Training Script
• Conclusion
• Acknowledgements

https://huggingface.co/blog/ettin-reranker#what-is-a-reranker-and-why-pair-one-with-an-embedderWhat is a reranker, and why pair one with an embedder?

A reranker (a.k.a. pointwise cross-encoder) is a neural model that takes a\(query, document\)pair and outputs a single relevance score. Unlike an embedding model, which encodes the query and document separately and computes their similarity from the two embedding vectors, a reranker lets the two texts attend to each other through every transformer layer. That joint encoding is more accurate but also more expensive: the model has to be run once per\(query, document\)pair rather than once per text.

Because cross-encoders are too expensive to run over a full corpus, the common production pattern isretrieve-then-rerank: a fast embedding model retrieves the top-K candidates (cheap), then a cross-encoder re-orders just those K with high accuracy. The total cost stays bounded while the final ranking is much closer to what an exhaustive cross-encoder pass would produce.

Throughout this blogpost I’ll use “reranker” and “cross-encoder” interchangeably.

https://huggingface.co/blog/ettin-reranker#usageUsage

The released models are normal Sentence TransformersCrossEncodermodels, so you can use them with just 3 lines of code:

from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ettin-reranker-32m-v1")
scores = model.predict([
    ("Where was Apple founded?", "Apple Inc. was founded in Cupertino, California in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne."),
    ("Where was Apple founded?", "The Fuji apple is an apple cultivar developed in the late 1930s and brought to market in 1962."),
])
print(scores)
# [11.393298  2.968891]   <- larger means more relevant

For a query and a list of candidates, you can also userankto get back sorted indices and scores:

ranked = model.rank(
    query="Which planet is known as the Red Planet?",
    documents=[
        "Venus is often called Earth's twin because of its similar size and proximity.",
        "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
        "Jupiter, the largest planet in our solar system, has a prominent red spot.",
        "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
    ],
    top_k=4,
    return_documents=True,
)
for r in ranked:
    print(f"({r['score']:.2f}): {r['text']}")
# (10.82): Mars, known for its reddish appearance, is often referred to as the Red Planet.
# (9.86): Saturn, famous for its rings, is sometimes mistaken for the Red Planet.
# (8.55): Jupiter, the largest planet in our solar system, has a prominent red spot.
# (6.21): Venus is often called Earth's twin because of its similar size and proximity.

You can swapcross\-encoder/ettin\-reranker\-32m\-v1for any other size to trade quality for speed. All six accept up to 8K tokens of context (useful for long-document reranking) thanks to ModernBERT’s long-context pre-training.

It is recommended to installkernelsand setmodel\_kwargs=\{"dtype": "bfloat16", "attn\_implementation": "flash\_attention\_2"\}for the highest throughput. See theSpeedsection below for more details, but in general you can expect a 1.7x-8.3x speedup over default loading depending on model size and sequence length.

from sentence_transformers import CrossEncoder

model = CrossEncoder(
    "cross-encoder/ettin-reranker-32m-v1",
    model_kwargs={"dtype": "bfloat16", "attn_implementation": "flash_attention_2"},
)

https://huggingface.co/blog/ettin-reranker#end-to-end-retrieve-then-rerank-pipelineEnd-to-end retrieve-then-rerank pipeline

A complete example with a fast embedder for retrieval and the reranker for the final ordering:

from sentence_transformers import SentenceTransformer, CrossEncoder

# Fast retrieval with a static embedder (sub-millisecond on CPU per query)
embedder = SentenceTransformer("sentence-transformers/static-retrieval-mrl-en-v1")
reranker = CrossEncoder("cross-encoder/ettin-reranker-68m-v1")

corpus = [
    "Apple Inc. was founded in Cupertino, California in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne.",
    "The Fuji apple is an apple cultivar developed in the late 1930s.",
    "Steve Jobs introduced the iPhone in 2007 at Macworld.",
    "Macintosh computers were sold by Apple from 1984 onward.",
    # ... thousands or millions more in production
]
query = "Where was Apple founded?"

# Step 1: encode + retrieve top-100
query_emb = embedder.encode_query(query, convert_to_tensor=True)
corpus_emb = embedder.encode_document(corpus, convert_to_tensor=True)
scores = embedder.similarity(query_emb, corpus_emb)[0]
top_k_idx = scores.topk(min(100, len(corpus))).indices.tolist()

# Step 2: rerank
top_k_docs = [corpus[i] for i in top_k_idx]
ranked = reranker.rank(query, top_k_docs, top_k=5, return_documents=True)
for r in ranked:
    print(f"({r['score']:.2f}): {r['text']}")
# (11.63): Apple Inc. was founded in Cupertino, California in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne.
# (4.71): Steve Jobs introduced the iPhone in 2007 at Macworld.
# (1.96): The Fuji apple is an apple cultivar developed in the late 1930s.
# (1.49): Macintosh computers were sold by Apple from 1984 onward.

This is the same shape used by most modern search systems. The retriever decides what enters the funnel, the reranker decides what wins.

https://huggingface.co/blog/ettin-reranker#architecture-detailsArchitecture Details

All six rerankers share the same architecture and differ only in their backbone size. The backbone is one of the sixEttin encodersfrom Johns Hopkins University’s Ettin suite. These are ModernBERT-style models with unpadded attention, RoPE positional encodings, GeGLU, and 2T tokens of open-license pre-training, supporting up to 8192 tokens of context.

On top of each encoder, the reranker uses a 4-module classification head that mirrorsModernBertForSequenceClassificationbut is built from Sentence Transformers’ modular components. The underlyingTransformeris a plainAutoModelrather thanAutoModelForSequenceClassification, which lets us use sequence unpadding for variable-length inputs for Flash Attention 2. At medium-document sequence lengths this is a 1.7x-8.3x speedup over fp32+SDPA depending on model size (seeSpeedfor the full benchmark):

1. Transformer(FA2)
2. Pooling(cls)
3. Dense(H, H, bias=False, GELU)
4. LayerNorm(H)
5. Dense(H, 1, scores)

In my ablations, CLS pooling outperformed mean pooling. That was a little surprising. ModernBERT uses global attention only every third layer and the other two-thirds use local-window attention that cannot reach CLS from distant positions. Empirically, those few global layers carry enough signal to make CLS the better pooling choice.

All six models are released under theApache 2.0license, matching the Ettin encoders.

https://huggingface.co/blog/ettin-reranker#resultsResults

https://huggingface.co/blog/ettin-reranker#mtebeng-v2-retrievalMTEB(eng, v2) Retrieval

I ran each released model through the fullMTEB\(eng, v2\)Retrieval benchmark(10 tasks, top-100 reranked) using MTEB’stwo-stage reranking flow, pairing each reranker with six embedding models that span the speed/quality spectrum:

Thedashed retriever-only linein each chart below is the headline number to beat. Anything below it means the reranker actively hurts the pipeline on average:

Full table of results (click to expand)Mean NDCG@10 over the 6 embedder pairings, sorted descending. Our six models are inbold, and the teachermixedbread\-ai/mxbai\-rerank\-large\-v2is underlined.

†Capped tomax\_seq\_length=8192(the 4B Qwen3-based rerankers don’t fit on a single H100 80GB at native context). Native-context evaluation is likely higher.

Full table of NanoBEIR results (click to expand)NanoBEIRis a fast 13-dataset subset ofBEIRthat uses 50 queries per dataset against up to 5000 documents each. NanoBEIR is whatmetric\_for\_best\_modelwas set to during training (seeEvaluation), and what I used to guide the experimentation.

The smallest model I’m releasing, our 17M, beats the 33Mms\-marco\-MiniLM\-L12\-v2by +0.051 NDCG@10 (0.5576 vs 0.5066) on MTEB and +0.038 (0.6746 vs 0.6369) on NanoBEIR at roughly half the parameter count. The 32M beats the 568MBAAI/bge\-reranker\-v2\-m3by +0.025 (0.5779 vs 0.5526) on MTEB, a 17x parameter gap. If you’ve been using one of the legacy MiniLM rerankers as the default in your retrieve-then-rerank stack, swapping in our 17M (or 32M) is a low-risk drop-in replacement, with a noticeable quality bump on both benchmarks.

Moving up the table, our 150M is the strongest reranker I tested in the under-600M range on MTEB, edging out the recentQwen/Qwen3\-Reranker\-0\.6B(596M) by +0.005 (0.5994 vs 0.5940) and beating every BAAI bge-reranker variant by 0.03 to 0.05. The 68M is also worth a mention: at 0.5915 it lands almost exactly onQwen3\-Reranker\-0\.6B(0.5940) while using a ninth of the parameters.

At the top of the released range, our 1B model closely tracks its teacher. It comes within 0.0001 of the 1.54Bmxbai\-rerank\-large\-v2on MTEB (0.6114 vs 0.6115) and within 0.008 on NanoBEIR, despite distilling from a model 54% larger than itself. The distillation effectively closes the gap to the teacher, which is what I was hoping to see going into this release.

The overall strongest reranker in the comparison isQwen/Qwen3\-Reranker\-4Bat 0.6367 MTEB, +0.025 above our 1B model. Closing that gap from the current recipe would likely require distilling from a stronger teacher (our teacher itself sits belowQwen3\-Reranker\-4B). For most retrieve-then-rerank workloads, our 1B at a quarter of the parameters (seeSpeed) is a much more practical pick.

https://huggingface.co/blog/ettin-reranker#speedSpeed

Quality numbers are only half of what matters for a reranker. The other half is whether its latency fits inside the budget you have between retrieval and showing results to the user. Let me walk through what I measured.

I benchmarked all six released models against thirteen public rerankers (strong baselines up to about 1B parameters) on a single NVIDIA H100 80GB. The queries and documents come fromsentence\-transformers/natural\-questionsat its natural document-length distribution: most NQ answers are short, some are long. Documents are truncated atmax\_length=512to avoid giving the older models an unfair advantage. Each model uses its best supported attention implementation: Flash Attention 2 wherever the architecture supports it (BERT, XLM-RoBERTa, ModernBERT, Qwen2), SDPA where it doesn’t, and eager for DeBERTa-v2 (which currently has neither FA2 nor SDPA support intransformers).

For every model an auto-batch search starts at batch size 8 and doubles until the GPU runs out of memory. At each batch size I run three timed passes and keep the median throughput, so a single unlucky run doesn’t drag the number around. The reported throughput is at whichever batch size won.

Table 1.Throughput in pairs per second, all inbfloat16. Our six rerankers are inbold.

Our 17M is the fastest reranker in the whole comparison, at 7517 pairs per second. That’s almost twice the throughput ofms\-marco\-MiniLM\-L6\-v2(3817) and faster even than the smallerms\-marco\-MiniLM\-L4\-v2(4029). And as you saw in the MTEB table earlier, our 17M is also more accurate than every MiniLM variant. If you’re currently running a MiniLM cross-encoder, swapping to our 17M is a one-line change that improves both your latency and search quality.

Our 150M is an even more interesting comparison, because there are two direct architectural peers at exactly 150M parameters:Alibaba\-NLP/gte\-reranker\-modernbert\-baseandibm\-granite/granite\-embedding\-reranker\-english\-r2. Both are built on the same ModernBERT-base backbone. Our 150M runs at 3237 pairs per second, while the two peers come in at 1418 and 1404 respectively, for a 2.3x speed gap.

All three 150M models use Flash Attention 2, but the two peers load throughAutoModelForSequenceClassification, which keeps the inputs padded. So attention itself runs the FA2 kernel, but the rest of the model is still doing dense compute on padding tokens that don’t contribute anything. Our modularTransformermodule (seeArchitecture Detailsabove) propagates unpadded inputs all the way through the model, so every layer only spends compute on real tokens. That’s the difference between getting some of FA2’s benefit and getting all of it.

At the bottom of the table, our 1B model hits 928 pairs per second, which is 2.4x faster than the 1.54B teachermxbai\-rerank\-large\-v2(387 pairs per second) while matching its MTEB score within 0.0001. The teacher is Qwen2-based with a prompt-template overhead per pair, so the distilled student inherits the teacher’s calibration and judgement but skips all the runtime baggage. This is honestly the most satisfying single number in the whole release for me.

One unfortunate note: the DeBERTa-v2-basedmxbai\-rerank\-\{xsmall,base,large\}\-v1series ends up much slower than the rest of the table because DeBERTa-v2 currently supports neither Flash Attention 2 nor SDPA intransformers. The 70Mmxbai\-rerank\-xsmall\-v1runs at 2636 pairs per second, about half the throughput of our 68M at almost the same parameter count. The models themselves are perfectly fine, they just don’t get to use modern attention kernels.

Same benchmark on a consumer GPU (RTX 3090, 24 GB)If you’re self-hosting on a consumer card rather than a datacenter GPU, here’s the same throughput sweep on an RTX 3090. Same benchmark setup as Table 1:bfloat16, best-supported attention per model, three-trial median throughput at the largest batch that fits.

Our 17M is still the fastest model in the table at 9008 pairs per second, actually higher than its H100 number, which suggests that at tiny sizes raw compute isn’t the bottleneck and the H100’s extra muscle doesn’t translate. The middle of the table reshuffles a bit, with the MiniLM rerankers overtaking our 32M and 68M, and the 1B slipping behindmxbai\-rerank\-base\-v2(189 vs 221 pairs per second). Our 150M model still holds a solid lead over the two 150M ModernBERT-based peers, and the teacher-replacement story still holds, with our 1B at 2.7x the throughput of the 1.5Bmxbai\-rerank\-large\-v2(189 vs 69 pairs per second).

Same benchmark on CPU (Intel Core i7-13700K)On CPU, we can’t take advantage of bf16, Flash Attention 2, or unpadding, so the latency story is a bit simpler: the higher the parameter count, the slower the model. The 17M model is considerably faster thanms\-marco\-MiniLM\-L6\-v2(267.4 vs 143.9 pairs per second) and even faster than the smallerms\-marco\-MiniLM\-L4\-v2(206.2). As expected, our 150M model lands alongside the two 150M peers (14.0 vs 14.5 and 14.7 pairs per second) now that unpadding no longer applies. If you’re CPU-bound, our 17M and 32M are the practical picks.

To explain where the speed comes from, the next table sweepsfp32\+SDPA,bf16\+SDPA, andbf16\+FA2for our six models using the same bench config. The FA2 column is split in two: one with the inputs still padded (what a wrapped model would see) and one with unpadded inputs (what our modularTransformeractually does). The rightmost column is what our models use by default when FA2 is enabled.

Table 2.Precision and attention ablation for the six released sizes atmax\_length=512on natural NQ documents. Each cell shows pairs / second with the multiplier relative tofp32\+SDPAin parentheses, and peak GPU memory on the second line. The rightmost column (inbold) is the configuration our models use by default when FA2 is enabled.

The total speedup frombf16\+FA2 w\.o\. paddingover thefp32\+SDPAbaseline grows sharply with model size, from 1.71x on the 17M to 8.26x on the 1B. Most of that growth comes frombf16alone: thefp32\+SDPAtobf16\+SDPAstep gives the 17M only a 1.03x speedup but gives the 1B a full 5.60x speedup, also due to the lowered memory cost allowing for bigger batch sizes. In short,bfloat16is the biggest single contributor to the overall speedup.

Unexpectedly, turning on FA2 while the inputs are still padded is actually slower thanbf16\+SDPAat every size in the release. The FA2 kernel prefers an unpadded format, and when you feed it padded inputs you pay the bookkeeping overhead of converting between formats while still spending compute on the padding tokens themselves. So thebf16\+FA2 w\. paddingcolumn is roughly what you’d measure if you swappedsdpaforflash\_attention\_2inmodel\_kwargswithout changing anything else about the model loader. This is the situation thatgte\-reranker\-modernbert\-baseandgranite\-embedding\-reranker\-english\-r2from Table 1 are in.

Lastly, going frombf16\+FA2 w\. paddingtobf16\+FA2 w\.o\. paddingis worth between 1.78x (1B) and 2.45x (68M) of additional throughput, and it also cuts peak memory considerably, allowing for higher batch sizes.

So my recommendation is simple: enablebf16and FA2 together. The six Ettin rerankers will use unpadded inputs by default, since that’s what the modularTransformermodule from theArchitecture Detailssection is set up for. The full snippet is the same as in theUsagesection above:

from sentence_transformers import CrossEncoder

model = CrossEncoder(
    "cross-encoder/ettin-reranker-150m-v1",
    model_kwargs={
        "dtype": "bfloat16",
        "attn_implementation": "flash_attention_2",  # See tip below
    },
)

Usepip install kernelsto install FA2. It ships pre-built kernels for a wide range of GPU architectures, CUDA versions, and operating systems.

One caveat for other CrossEncoders: the full speedup is only available for models built with a modularTransformerlike the Ettin rerankers. Applying the same two flags to a CrossEncoder that loads throughAutoModelForSequenceClassificationlands you in the slowerbf16\+FA2 w\. paddingcolumn of Table 2 instead.

https://huggingface.co/blog/ettin-reranker#trainingTraining

The training script below started as the output of the newtrain\-sentence\-transformersAgent Skill, shipped inSentence Transformers v5.5.0. If you use an AI coding agent (Claude Code, Codex, Cursor, Gemini CLI, ...), you can install the skill and ask it to fine-tune aSentenceTransformer,CrossEncoder, orSparseEncodermodel on your data. The skill carries version-aware guidance for base model selection, loss and evaluator choice, hard-negative mining, distillation, LoRA, Matryoshka, multilingual training, and static embeddings, plus template scripts for each model type.

hf skills add train-sentence-transformers --claude   # symlinks into .claude/skills/
hf skills add train-sentence-transformers --global   # under ~/.agents/skills/

A prompt like*“Fine-tune a cross-encoder reranker on\(query, document\)pairs from my dataset, mine hard negatives, and push to my Hub repo”*will produce a runnable script you can then iterate on. That’s how I started working on the recipe below.

All six rerankers were trained with the same single-stage recipe. Only the learning rate and the per-device batch size vary per model size. The full training script is ~150 lines and uses one published dataset.

The recipe converged after a single sweep across model sizes. Each size’s learning rate was tuned by a small grid search on a ~15% subset of the final training data, and the resulting LRs transferred cleanly to the full-data runs without re-tuning. No per-size tuning beyond LR was needed.

https://huggingface.co/blog/ettin-reranker#distillation-recipeDistillation recipe

Most published reranker recipes train on human-labeled relevance triples (a query, one positive document, and optionally hard negatives) with a contrastive, pointwise, pairwise, or listwise loss likeMultipleNegativesRankingLoss,BinaryCrossEntropyLoss,RankNetLoss, orLambdaLoss, respectively. See my earlierTraining and Finetuning Reranker Models with Sentence Transformersblogpost, for example.

But this approach has a few practical and theoretical drawbacks. First, positives need to be human-labeled, which is expensive and slow to scale across many domains. Second, the model only ever sees a label for the small subset of\(query, document\)pairs that someone went through. Especially after hard negative mining, you end up with a lot of false negatives, e.g. as shown inHard Negatives, Hard Lessons. Third, the binary nature of this labeling doesn’t match reality, where some documents are simply more relevant than others.

I took a different route here: pointwise MSE distillation from an existing strong teacher reranker. The setup is simple enough to describe in three lines:

• Teacher:mixedbread\-ai/mxbai\-rerank\-large\-v2(1.54B parameters).
• Loss:MSELosson the raw teacher logits (range ~[−12, 22]), i.e. without rescaling.
• Training data: ~143M\(query, document, teacher\_score\)triples.

https://huggingface.co/blog/ettin-reranker#datasetDataset

I’ve released the training data as a single Hugging Face dataset,cross\-encoder/ettin\-reranker\-v1\-data, assembled from two sources. Each source is kept as its own split so the provenance is transparent:

1. LightOn pre-training data (lightonai/embeddings\-pre\-training, non-curated): 32 splits covering broad-domain text similarity signal (MTP, FW-EDU, Reddit, PAQ, S2ORC, Amazon, Wikipedia, MS MARCO, etc.). I limit the number of samples for some of the splits, resulting in ~110M\(query, document, similarity\)triples in total.
2. Rescored retrieval data fromlightonai/embeddings\-fine\-tuning: 7 splits (msmarco,hotpotqa,trivia,nq,squadv2,fiqa,fever). The source dataset has up to 2048 candidate documents per query (initially scored withAlibaba\-NLP/gte\-modernbert\-base), which I rescored withmixedbread\-ai/mxbai\-rerank\-large\-v2and uploaded ascross\-encoder/lightonai\-embeddings\-fine\-tuning\-reranked\-v1. That dataset subsamples each query’s 2048 candidates down to 256 using theJang et al.quantile-anchor recipe (all positives + top-16 hard + ~239 quantile-anchor stratified). For training, I pick 64 of those 256 per query: 32 from the score-sorted head (the positive plus the hardest negatives) and 32 medium-difficulty negatives sampled from a band further down the teacher’s ranking. See thedataset cardfor the exact rank positions.

Total: ~143M\(query, document, score\)triples, plus a held-out 5K-row eval split (the tail ofquora) that drives the in-training eval loss.

https://huggingface.co/blog/ettin-reranker#training-argumentsTraining Arguments

Most hyperparameters are constant across model sizes:

CrossEncoderTrainingArguments(
    num_train_epochs=1,                    # I chose more data over more epochs
    per_device_train_batch_size=...,       # global_batch_size // world_size (see table below)
    gradient_accumulation_steps=1,
    learning_rate=...,                     # per-size, see table
    warmup_ratio=0.03,                     # ~3% linear warmup, then linear decay (default)
    bf16=True,                             # FA2 + bf16 throughout
    eval_strategy="steps",
    eval_steps=0.05,                       # NanoBEIR every 5% of training
    save_strategy="steps",
    save_steps=0.05,
    save_total_limit=5,
    load_best_model_at_end=True,
    metric_for_best_model="eval_NanoBEIR_R100_mean_ndcg@10",
    seed=12,
)

Only the learning rate and global batch size very per model size.

SizeLearning rateGlobal batch size17m2.4e-4102432m1.2e-451268m3e-5256150m1.5e-5192400m7e-62561b3e-6512 global\_batch\_sizeisper\_device\_batch\_size x world\_size x gradient\_accumulation\_steps. On a single 8-GPU node, the 1024 global batch for 17m meansper\_device=128. On 8 nodes, it meansper\_device=8. The training script computesper\_device\_batch\_sizefromglobal\_batch\_size // world\_sizeso the same script works at any node count. The global batch size could be made more consistent, but I found that the above values worked well and didn’t want to retune them just for the sake of consistency.

https://huggingface.co/blog/ettin-reranker#evaluationEvaluation

I monitored NanoBEIR mean NDCG@10 during training (eval every 5% of steps) and used it as themetric\_for\_best\_modelforload\_best\_model\_at\_end. NanoBEIR is fast, so I could afford it 20 times per training run. After training, I evaluated both the best checkpoint (according to NanoBEIR) and the last checkpoint on the full MTEB(eng, v2) Retrieval benchmark. The final release checkpoint was the one that did best on MTEB. The NanoBEIR-preferred checkpoint won for all sizes except 68m, where the last checkpoint was slightly stronger.

https://huggingface.co/blog/ettin-reranker#overall-training-scriptOverall Training Script

The complete script (what every released model was trained with) is a single file. OnlyENCODER\_SIZEchanges per run, and everything else is automatic:

from __future__ import annotations

import logging
import os
from pathlib import Path

import torch
import torch.nn as nn
from datasets import concatenate_datasets, get_dataset_config_names, load_dataset

from sentence_transformers import CrossEncoder
from sentence_transformers.base.modules import Dense
from sentence_transformers.cross_encoder import (
    CrossEncoderModelCardData,
    CrossEncoderTrainer,
    CrossEncoderTrainingArguments,
)
from sentence_transformers.cross_encoder.evaluation import CrossEncoderNanoBEIREvaluator
from sentence_transformers.cross_encoder.losses import MSELoss
from sentence_transformers.sentence_transformer.modules import LayerNorm, Pooling, Transformer

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s", datefmt="%H:%M:%S")
logging.getLogger("httpx").setLevel(logging.WARNING)

# Per-size config. I swept the learning rates with these global (effective) batch sizes,
# also by incorporating accum_steps
CONFIGS: dict[str, dict] = {
    "17m":  {"base_model_name": "jhu-clsp/ettin-encoder-17m",  "learning_rate": 2.4e-4, "global_batch_size": 1024},
    "32m":  {"base_model_name": "jhu-clsp/ettin-encoder-32m",  "learning_rate": 1.2e-4, "global_batch_size": 512},
    "68m":  {"base_model_name": "jhu-clsp/ettin-encoder-68m",  "learning_rate": 3e-5,   "global_batch_size": 256},
    "150m": {"base_model_name": "jhu-clsp/ettin-encoder-150m", "learning_rate": 1.5e-5, "global_batch_size": 192},
    "400m": {"base_model_name": "jhu-clsp/ettin-encoder-400m", "learning_rate": 7e-6,   "global_batch_size": 256},
    "1b":   {"base_model_name": "jhu-clsp/ettin-encoder-1b",   "learning_rate": 3e-6,   "global_batch_size": 512},
}
ENCODER_SIZE = "17m"

def main() -> None:
    config = CONFIGS[ENCODER_SIZE]
    encoder_id = config["base_model_name"]
    learning_rate = config["learning_rate"]
    global_batch_size = config["global_batch_size"]

    world_size = int(os.environ.get("WORLD_SIZE", 1))
    per_device_batch_size = global_batch_size // world_size
    dataloader_workers = 0 if world_size > 8 else 4
    run_name = f"ettin-reranker-{ENCODER_SIZE}-lr{learning_rate:.0e}"

    # 1. Load a model to finetune with model card data
    # The model mirrors ModernBertForSequenceClassification, but with a 'headless' Transformer that just loads
    # AutoModel. This allows for unpadding with FA2, which isn't possible with AutoModelForSequenceClassification.
    # This speeds up training considerably, while heavily reducing memory usage.
    torch.manual_seed(12)
    transformer = Transformer(encoder_id, model_kwargs={"attn_implementation": "flash_attention_2"})
    transformer.model.config.num_labels = 1
    embedding_dimension = transformer.get_embedding_dimension()
    pooling = Pooling(embedding_dimension=embedding_dimension, pooling_mode="cls")
    dense_inner = Dense(
        in_features=embedding_dimension, out_features=embedding_dimension, bias=False,
        activation_function=nn.GELU(),
        module_input_name="sentence_embedding", module_output_name="sentence_embedding",
    )
    norm = LayerNorm(dimension=embedding_dimension)
    dense_score = Dense(
        in_features=embedding_dimension, out_features=1, bias=True,
        activation_function=nn.Identity(),
        module_input_name="sentence_embedding", module_output_name="scores",
    )
    model = CrossEncoder(
        modules=[transformer, pooling, dense_inner, norm, dense_score],
        num_labels=1,
        activation_fn=nn.Identity(),
        model_card_data=CrossEncoderModelCardData(
            model_name=f"Ettin Reranker {ENCODER_SIZE} distilled from mxbai-rerank-large-v2",
            language="en",
            license="apache-2.0",
        ),
    )
    actual_attn = getattr(model[0].model.config, "_attn_implementation", None)
    if not (actual_attn and "flash" in actual_attn.lower()):
        logging.warning(f"FA2 may not be active (attn_impl={actual_attn!r}); training will be slower.")

    # 2. Load the dataset. Each config is one source subset (32 lighton + 7 rerank retrieval
    # domains). The held-out eval rows live as the 'validation' split of the 'quora' config.
    dataset_repo = "cross-encoder/ettin-reranker-v1-data"
    train_pieces = []
    eval_dataset = None
    for config_name in get_dataset_config_names(dataset_repo):
        dataset = load_dataset(dataset_repo, config_name)
        train_pieces.append(dataset["train"])
        if "validation" in dataset:
            eval_dataset = dataset["validation"]
    train_dataset = concatenate_datasets(train_pieces)
    print(train_dataset)

    # 3. Define a loss function
    loss = MSELoss(model)

    # 4. Specify training arguments
    args = CrossEncoderTrainingArguments(
        output_dir=f"models/{run_name}",
        num_train_epochs=1,
        per_device_train_batch_size=per_device_batch_size,
        per_device_eval_batch_size=per_device_batch_size,
        gradient_accumulation_steps=1,
        learning_rate=learning_rate,
        warmup_ratio=0.03,
        bf16=True,
        eval_strategy="steps",
        eval_steps=0.05,
        save_strategy="steps",
        save_steps=0.05,
        save_total_limit=5,
        logging_steps=0.025,
        logging_first_step=True,
        load_best_model_at_end=True,
        metric_for_best_model="eval_NanoBEIR_R100_mean_ndcg@10",
        dataloader_num_workers=dataloader_workers,
        run_name=run_name,
        seed=12,
    )

    # 5. Create an evaluator
    evaluator = CrossEncoderNanoBEIREvaluator(
        dataset_names=["msmarco", "nfcorpus", "nq", "fiqa2018", "touche2020", "scifact",
                       "hotpotqa", "arguana", "fever", "dbpedia", "climatefever", "scidocs",
                       "quoraretrieval"],
        batch_size=per_device_batch_size,
        always_rerank_positives=False,
        show_progress_bar=False,
    )

    # 6. Create a trainer
    trainer = CrossEncoderTrainer(
        model=model,
        args=args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        loss=loss,
        evaluator=evaluator,
    )

    # 7. Evaluate before training
    if trainer.is_world_process_zero():
        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
            evaluator(model)

    # 8. Train
    trainer.train()

    # 9. Evaluate the final model
    if trainer.is_world_process_zero():
        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
            evaluator(model)

    # 10. Save the final model
    final_dir = f"models/{run_name}/final"
    model.save_pretrained(final_dir)

if __name__ == "__main__":
    main()

For multi-node training (anything past 17m/32m), launch the same script withtorchrun:

# Single-node (17m, 32m): defaults work
python train.py

# Multi-node 4n setup for 150m, preserves global_batch_size=192:
torchrun --nproc_per_node=8 --nnodes=4 ... train.py

https://huggingface.co/blog/ettin-reranker#conclusionConclusion

The ettin-reranker-v1 family, trained with a single simple recipe, is state-of-the-art at every released size up to 1B parameters. Pointwise MSE distillation from a strong teacher onto a broad-domain and retrieval-specific mix scales cleanly from 17M to 1B parameters, with only the learning rate and per-device batch size changing between sizes.

Every ettin-reranker-v1 model beats thems\-marco\-MiniLM\-L\*\-v2family by a comfortable margin on MTEB and NanoBEIR.cross\-encoder/ettin\-reranker\-150m\-v1is the strongest mid-tier reranker I tested in the under-600M range,cross\-encoder/ettin\-reranker\-400m\-v1lands within 0.0024 of the 1.54B teacher’s MTEB score, andcross\-encoder/ettin\-reranker\-1b\-v1matches that teacher within 0.0001.

Everything in one place:

• Models:- cross\-encoder/ettin\-reranker\-17m\-v1 - cross\-encoder/ettin\-reranker\-32m\-v1 - cross\-encoder/ettin\-reranker\-68m\-v1 - cross\-encoder/ettin\-reranker\-150m\-v1 - cross\-encoder/ettin\-reranker\-400m\-v1 - cross\-encoder/ettin\-reranker\-1b\-v1
• Dataset:cross\-encoder/ettin\-reranker\-v1\-datawith ~143M\(query, document, label\)triples, kept as 39 named splits so the provenance of every row is visible.
• Training script: the ~150 lines inOverall Training Scriptabove, which is the same script used for all six models.

If you build something on top of these, please let me know! I’d genuinely love to see what people do with them, and if you manage to train better rerankers using the released data, even better. The recipe is intentionally simple, partly so that there’s plenty of headroom for someone else to improve it. Train a stronger teacher and the same script can keep producing better students.

https://huggingface.co/blog/ettin-reranker#acknowledgementsAcknowledgements

I’d like to thank the Ettin team (Orion Weller, Kathryn Ricci, Marc Marone, Antoine Chaffin, Dawn Lawrie, and Benjamin Van Durme) forbuilding the base encodersthat these rerankers are built on, the LightOn team (Antoine Chaffin, Raphael Sourty, Paulo Moura, and Amélie Chatelain) fortheir work on the training data collection, and the Mixedbread AI team (Xianming Li, Aamir Shakir, Rui Huang, Tsz-fung Andrew Lee, Julius Lipp, Benjamin Clavié, and Jing Li) fortheir work on the teacher model.

https://huggingface.co/blog/ettin-reranker#citationCitation

If you use the ettin-reranker-v1 family or any of the released artifacts, please cite this blogpost:

@misc{aarsen2026ettin-reranker,
    title = "Introducing the Ettin Reranker Family",
    author = "Aarsen, Tom",
    year = "2026",
    publisher = "Hugging Face",
    url = "https://huggingface.co/blog/ettin-reranker",
}