Xetrieval: Mechanistically Explaining Dense Retrieval

Hugging Face Daily Papers Papers
dense-retrieval interpretability mechanistic-explanation embedding chain-of-thought sparse-features

Summary

Xetrieval is a mechanistic framework that explains dense retrieval by enhancing sentence embeddings with reasoning information and decomposing them into interpretable sparse features, providing feature-level explanations for retrieval decisions without expensive autoregressive generation.

Explaining why dense retrievers assign high relevance scores remains challenging because retrieval decisions are made through opaque high-dimensional embeddings. Existing explanations often focus on surface signals, such as lexical matches, token alignments, or post-hoc textual rationales, and thus provide limited insight into the latent factors that shape dense retrieval behavior at the embedding level. We propose Xetrieval, an embedding-level mechanistic framework for explaining dense retrieval. Xetrieval first introduces a lightweight reasoning internalizer that approximates Chain-of-Thought reasoning directly in the embedding space with a single forward pass, enriching sentence embeddings with reasoning-oriented information while avoiding expensive autoregressive generation. It then decomposes these reasoning-enhanced embeddings into sparse, human-interpretable features, each associated with a coherent natural language description. By aggregating sparse feature overlaps across multiple document-side views, Xetrieval provides feature-level explanations of individual retrieval decisions. Experiments on diverse retrievers and benchmarks show that Xetrieval uncovers coherent interpretable features, yields stronger pair-level intervention effects, and supports task-level feature steering. The project page and source code are available at https://hihiczx.github.io/Xetrieval .
Original Article
View Cached Full Text

Cached at: 05/29/26, 03:02 PM

Paper page - Xetrieval: Mechanistically Explaining Dense Retrieval

Source: https://huggingface.co/papers/2605.29507

Abstract

Xetrieval is a mechanistic framework that explains dense retrieval by enhancing sentence embeddings with reasoning information and decomposing them into interpretable sparse features for retrieval decision explanations.

Explaining whydense retrieversassign high relevance scores remains challenging becauseretrieval decisionsare made through opaquehigh-dimensional embeddings. Existing explanations often focus on surface signals, such as lexical matches, token alignments, or post-hoc textual rationales, and thus provide limited insight into the latent factors that shape dense retrieval behavior at the embedding level. We propose Xetrieval, an embedding-level mechanistic framework for explaining dense retrieval. Xetrieval first introduces a lightweightreasoning internalizerthat approximatesChain-of-Thought reasoningdirectly in theembedding spacewith a single forward pass, enriching sentence embeddings with reasoning-oriented information while avoiding expensive autoregressive generation. It then decomposes these reasoning-enhanced embeddings into sparse,human-interpretable features, each associated with a coherent natural language description. By aggregating sparse feature overlaps across multiple document-side views, Xetrieval providesfeature-level explanationsof individualretrieval decisions. Experiments on diverse retrievers and benchmarks show that Xetrieval uncovers coherent interpretable features, yields strongerpair-level intervention effects, and supportstask-level feature steering. The project page and source code are available at https://hihiczx.github.io/Xetrieval .

View arXiv pageView PDFProject pageGitHub12Add to collection

Get this paper in your agent:

hf papers read 2605\.29507

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.29507 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.29507 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.29507 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

DREAM: Dense Retrieval Embeddings via Autoregressive Modeling

Hugging Face Daily Papers

DREAM trains dense retrieval embeddings by using autoregressive language model attention to supervise query-document similarity, eliminating the need for labeled data. It consistently outperforms baselines on BEIR and RTEB benchmarks across model scales.

Retrieval Grounding Latent Reasoning for Dense Retrieval

arXiv cs.AI

Proposes Retrieval Grounding Latent Reasoning (RGLR), a latent reasoning framework for dense retrieval that explicitly connects intermediate latent transitions with retrieval improvements, outperforming baselines on reasoning-intensive tasks.