When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections

Hugging Face Daily Papers Papers
dense-retrieval asymmetric-geometry bias-variance-theory shared-projections dual-projections retrieval-augmented-generation cars-algorithm

Summary

The paper introduces a bias-variance theory for dense retrieval, comparing shared and dual projections, and proposes the CARS method to select optimal geometry based on directional signal estimation from training data.

Dense retrieval powers retrieval-augmented generation, semantic search, and question answering, yet the theoretical basis for choosing between shared and dual query-document projections remains unclear. We introduce a bias-variance theory for low-rank bilinear scoring. Shared projections induce positive-semidefinite operators, whereas dual projections realize arbitrary low-rank operators. We derive their exact approximation gap and prove a local Gaussian boundary: dual has lower risk exactly when squared directional signal exceeds the estimation cost of its additional degrees of freedom. This boundary motivates the Cross-fitted Asymmetry Risk Selector (CARS), which estimates reproducible directional signal from training pairs; its Gaussian counterpart admits exact selection-power and regret formulas. Guided by the theory, we run retrieval experiments across multiple datasets and embedding models. The mean Dual-minus-Shared NDCG@10 advantage more than doubles as query rotation increases from 0 degrees to 90 degrees. In the rank-sample-size grids, Shared wins 13 of 16 cells at n=32, whereas Dual wins all 32 cells at n=1024 and n=2048. Consistent with this shift, all 168 comparable operator-risk curves move toward Dual as training data grow. Compared to the two fixed-geometry baselines, CARS reduces held-out regret by 49-96% and achieves 90.1% mean geometry-selection accuracy.
Original Article
View Cached Full Text

Cached at: 09/30/26, 04:20 AM

Paper page - When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections

Source: https://huggingface.co/papers/2609.32488

Abstract

Denseretrievalpowersretrieval-augmentedgeneration,semanticsearch,andquestionanswering,yetthetheoreticalbasisforchoosingbetweensharedanddualquery-documentprojectionsremainsunclear.Weintroduceabias-variancetheoryforlow-rankbilinearscoring.Sharedprojectionsinducepositive-semidefiniteoperators,whereasdualprojectionsrealizearbitrarylow-rankoperators.WederivetheirexactapproximationgapandprovealocalGaussianboundary:dualhaslowerriskexactlywhensquareddirectionalsignalexceedstheestimationcostofitsadditionaldegreesoffreedom.ThisboundarymotivatestheCross-fittedAsymmetryRiskSelector(CARS),whichestimatesreproducibledirectionalsignalfromtrainingpairs;itsGaussiancounterpartadmitsexactselection-powerandregretformulas.Guidedbythetheory,werunretrievalexperimentsacrossmultipledatasetsandembeddingmodels.ThemeanDual-minus-SharedNDCG@10advantagemorethandoublesasqueryrotationincreasesfrom0degreesto90degrees.Intherank-sample-sizegrids,Sharedwins13of16cellsatn=32,whereasDualwinsall32cellsatn=1024andn=2048.Consistentwiththisshift,all168comparableoperator-riskcurvesmovetowardDualastrainingdatagrow.Comparedtothetwofixed-geometrybaselines,CARSreducesheld-outregretby49-96%andachieves90.1%meangeometry-selectionaccuracy.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2609\.32488

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.32488 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.32488 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.32488 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Relation Geometry in Semantic Space of Language Models

arXiv cs.CL

This paper explores how semantic relations are encoded in the geometry of language model semantic spaces, finding that asymmetric relations occupy distinct regions and that lexical information matters more for causal models while contextual information matters more for masked and diffusion models.

RADAR: Relative Angular Divergence Across Representations

arXiv cs.LG

RADAR is a geometrically grounded metric that estimates cross-domain transferability in foundation models by analyzing layer-wise angular and distance changes in representations, using KL divergence between within-domain and cross-domain trajectory distributions.