late-interaction

Turbopuffer announces beta support for late interaction, enabling models like ColBERT to represent text as token-level vectors, combining a fast single-vector ANN first pass with exact late interaction reranking to improve recall.

This paper presents fully open DenseOn and LateOn retrieval models, trained on curated English data and extended to multilingual settings via translate-train, achieving state-of-the-art BEIR results for their parameter size.

ColGraphRAG replaces single-vector bi-encoder similarity with late-interaction MaxSim scoring for ranking graph-linked image candidates in multimodal GraphRAG, improving retrieval and QA accuracy on MultimodalQA.

This paper quantifies and expands the theoretical capacity of late-interaction retrieval models, showing that MaxSim can replicate inner products between non-negative vectors and proposing Signed MaxSim for arbitrary real-valued vectors, revealing a representation gap between inner product and late-interaction models.

Introduces SaMer, an object-aware token merging framework that compresses image-side tokens for vision-language retrieval while preserving object-level evidence, achieving significant storage reduction and improved retrieval performance.

Mixedbread Search introduces asymmetric quantization for late interaction retrieval, achieving near-lossless quality with 97% storage reduction by storing document vectors as binary signs while keeping query vectors at higher precision.

Mixedbread AI introduces asymmetric quantization for late interaction retrieval, achieving 32x storage reduction with minimal quality loss by storing document vectors as binary signs while keeping query vectors high-precision, making late interaction practical for billion-scale production systems.

Mixedbread announces that users can now bring their own cloud bucket, enabling zero-retention indexing and search with late-interaction models.

TopK introduces semantic_index, a single schema annotation that abstracts multi-vector retrieval complexity for production systems, achieving state-of-the-art performance with sub-second latency and high throughput.

The release of PyLate introduces MaxSim kernels for GPU-accelerated training with lower memory requirements and TACHIOM for fast multi-vector indexing and search on CPU.

Argus-Retriever is a new late-interaction visual document retriever that adapts document representation to the query, achieving SOTA performance on ViDoRe benchmarks with a smaller index.

A curated list of top models, engines, libraries, and datasets for late-interaction multivector retrieval, organized in an 'Awesome Multivector Retrieval' resource.

LateOn, a new generation ColBERT model, achieves a nearly 10-point improvement over v2 on BEIR and generalizes well outside BEIR, with the same usage in PyLate.

The author shares their work on reducing the cost of multi-vector retrieval by using k-means as top-1 sparse coding. Omar Khattab adds that late-interaction sparse retrieval with neuron-level inverted indexing on unsupervised sparse autoencoders works well.

This paper presents a single-stage sparse coding method using unsupervised sparse autoencoders and natural inverted indexing to accelerate multi-vector retrieval, outperforming traditional k-means based approaches.

ProtoCol applies late-interaction retrieval to protein homology search, representing proteins as sets of residue embeddings and using MaxSim for scoring, outperforming pooled and alignment-based methods on remote homology benchmarks.

SMART is a framework that unlocks latent multi-vector capabilities in single-vector models for multimodal retrieval, improving state-of-the-art performance with reduced computational costs via contrastive training and late-interaction inference.

Releases a kernel on Hugging Face that accelerates MaxSim late-interaction retrieval by using tiled scoring with SIMD group matrix operations (Metal and WMMA), achieving 3–5× speedup over the naive implementation.

LightOn achieves GPT-5-level deep research retrieval performance using a 150M-parameter late-interaction model, a remarkable feat.

The article highlights how ColBERT models, despite being smaller and older, outperform larger models like Qwen3-embed-8B when coupled with late interaction techniques and minimal fine-tuning.