KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking

Hugging Face Daily Papers Papers
reranking retrieval encoder-decoder matryoshka-embedding cross-attention efficient-nlp

Summary

KaLM-Reranker-V1 is a fast reranker that decouples query and passage computation using an encoder-decoder architecture with Matryoshka embedding pooling and cross-attention, achieving state-of-the-art reranking performance on BEIR and competitive results on multilingual benchmarks.

As retrieval systems scale, high-quality reranking becomes increasingly important. However, most existing rerankers, whether encoder-based or decoder-based, jointly encode the query and passage, tightly coupling their computation and limiting deployment efficiency as well as flexibility. We present KaLM-Reranker-V1, a fast but not late-interaction (FBNL) reranker that decouples query and passage computation while retaining expressive relevance modeling. Built on an encoder-decoder architecture, KaLM-Reranker-V1 uses the encoder to pre-encode passages with Matryoshka embedding pooling, while the decoder models the system instruction, user instruction, and query intent; cross-attention then captures relevance between the query context and passage representations. This design makes KaLM-Reranker-V1 efficient through decoupled passage encoding, yet not late interaction, by preserving rich relevance modeling through cross-attention. We instantiate KaLM-Reranker-V1 in three sizes, Nano, Small, and Large, with 0.27B, 1B, and 4B activated parameters, respectively. Extensive experiments on BEIR, MIRACL, and LMEB demonstrate that KaLM-Reranker-V1 achieves strong reranking performance with superior efficiency. On BEIR, KaLM-Reranker-V1 achieves state-of-the-art performance, on par with strong industrial models such as the Qwen3-Reranker series; on MIRACL, despite not being extensively trained on multilingual data, KaLM-Reranker-V1 still shows excellent reranking performance. Moreover, on LMEB, reranking models demonstrate a clear advantage, with even the 0.27B Nano model remaining competitive with 7-12B embedding models.
Original Article
View Cached Full Text

Cached at: 06/23/26, 05:41 AM

Paper page - KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking

Source: https://huggingface.co/papers/2606.22807

Abstract

KaLM-Reranker-V1 is a fast reranker that decouples query and passage computation using encoder-decoder architecture with Matryoshka embedding pooling and cross-attention for efficient relevance modeling.

As retrieval systems scale, high-quality reranking becomes increasingly important. However, most existingrerankers, whether encoder-based or decoder-based, jointly encode the query and passage, tightly coupling their computation and limiting deployment efficiency as well as flexibility. We present KaLM-Reranker-V1, a fast but notlate-interaction(FBNL)rerankerthat decouples query and passage computation while retaining expressive relevance modeling. Built on anencoder-decoder architecture, KaLM-Reranker-V1 uses the encoder to pre-encode passages withMatryoshka embedding pooling, while the decoder models the system instruction, user instruction, and query intent;cross-attentionthen captures relevance between the query context and passage representations. This design makes KaLM-Reranker-V1 efficient through decoupled passage encoding, yet not late interaction, by preserving rich relevance modeling throughcross-attention. We instantiate KaLM-Reranker-V1 in three sizes, Nano, Small, and Large, with 0.27B, 1B, and 4B activated parameters, respectively. Extensive experiments onBEIR,MIRACL, andLMEBdemonstrate that KaLM-Reranker-V1 achieves strong reranking performance with superior efficiency. OnBEIR, KaLM-Reranker-V1 achieves state-of-the-art performance, on par with strong industrial models such as the Qwen3-Rerankerseries; onMIRACL, despite not being extensively trained on multilingual data, KaLM-Reranker-V1 still shows excellent reranking performance. Moreover, onLMEB, reranking models demonstrate a clear advantage, with even the 0.27B Nano model remaining competitive with 7-12B embedding models.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2606\.22807

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper3

#### KaLM-Embedding/KaLM-Reranker-V1-Nano Text Ranking• 0.8B• Updatedabout 3 hours ago • 1 #### KaLM-Embedding/KaLM-Reranker-V1-Small Text Ranking• 2B• Updatedabout 3 hours ago • 2 • 1 #### KaLM-Embedding/KaLM-Reranker-V1-Large Text Ranking• 8B• Updatedabout 3 hours ago • 1

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.22807 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.22807 in a Space README.md to link it from this page.

Collections including this paper2

Similar Articles

LAMAR: An Open Language-Aware Multilingual Alignment Reranker

Hugging Face Daily Papers

LAMAR is a language-aware multilingual cross-encoder reranker that uses English-anchored relevance distillation and preference alignment to prioritize documents in the same language as the query while preserving semantic relevance, achieving strong performance on multilingual benchmarks.

hLLM: Single Pass Decoding for Generative Reranking

arXiv cs.LG

This paper introduces hLLM, a decoding strategy for generative reranking that uses the Hungarian algorithm to achieve single-pass decoding, resulting in a 64x speed-up while maintaining ranking quality.