Query-aware routing for Cross-lingual performance gains in Encoders

# Query-aware routing for Cross-lingual performance gains in Encoders
Source: [https://arxiv.org/html/2610.02875](https://arxiv.org/html/2610.02875)
###### Abstract

Multilingual encoders can exhibit reduced retrieval effectiveness when queries and relevant documents differ in language, despite strong same\-language performance\. We investigate whether Finnish and Swedish cross\-lingual retrieval can improve while preserving an encoder’s existing same\-language performance and document index\. We combine a query\-only low\-rank adapter, trained against frozen document embeddings, with deterministic routing based on query and index languages\. Cross\-language queries use the adapter, while same\-language queries use the original encoder\. SampoTron111[https://huggingface\.co/ConfidentialMind/SampoTron](https://huggingface.co/ConfidentialMind/SampoTron), our fine\-tuned low\-rank \(LoRA\) adapter alongwith the Nemotron\-3\-Embed\-1B model, improves average retrieval quality across six English, Finnish, and Swedish directions from 0\.241 to 0\.291 in normalized discounted cumulative gain \(nDCG\) at rank ten, a 20\.9% relative gain on a sampled financial benchmark\. All six cross\-lingual directions improve, and routing preserves the original same\-language performance, including two full\-corpus Finnish evaluations\. The approach enables selective cross\-language specialization with reusable document embedding vectors\.

## 1Introduction

Multilingual organizations and public information services require access to relevant documents independently of the language used to formulate a query\. For example, Finnish or Swedish queries may target English technical documentation, while English queries may target local\-language public information\.*Cross\-lingual retrieval*addresses this query–document language mismatch by ranking documents according to relevance across language boundaries\. In contrast,*multi\-monolingual retrieval*uses a shared model across languages while restricting each query to documents in the same language\.

Dense retrievers encode queries and passages as vector representations, or embeddings, and rank passages by their similarity to the query embedding\[[6](https://arxiv.org/html/2610.02875#bib.bib6)\]\. In retrieval\-augmented generation, the retrieved passages provide evidence for response generation\. Reranking is restricted to the retrieved candidate set, making initial retrieval quality a constraint on the evidence available downstream\. Improved cross\-lingual retrieval can support access to organizational and public\-service documents without requiring users to reformulate queries in a second language\.

Multilingual pretraining and retrieval fine\-tuning have substantially expanded language coverage\[[15](https://arxiv.org/html/2610.02875#bib.bib15),[19](https://arxiv.org/html/2610.02875#bib.bib19)\]\. However, this does not ensure equally effective retrieval across language boundaries\. Recent work documents cross\-lingual misalignment and develops training objectives that improve it while limiting monolingual regressions\[[3](https://arxiv.org/html/2610.02875#bib.bib3),[7](https://arxiv.org/html/2610.02875#bib.bib7)\]\. An addtional constraint that arises in deployed systems is that changing the document encoder requires computing the stored corpus vectors again\. The practical problem is thus to improve cross\-language retrieval while retaining both a useful same\-language retriever and its existing document index\.

We address this problem through query\-only adaptation and language\-routed selection at inference\. A low\-rank adapter introduces trainable parameter updates into the frozen encoder and learns query representations against document embeddings produced by the original model\. The proposed*query\-aware routing*policy uses explicit query and index language metadata to activate the adapter when the languages differ\. Consequently, a single cross\-language adapter is able to serve all evaluated language pairs\.

Our principal implementation to achieve this, SampoTron, adapts Nemotron\-3\-Embed\-1B\[[11](https://arxiv.org/html/2610.02875#bib.bib11)\]\. We evaluate its performance over English, Finnish, and Swedish financial question\-answer retrieval and full\-corpus Finnish benchmarks\. Our contributions can be outlined as follows:

1. 1\.A language\-routed retrieval architecture that combines a query\-only adapter with the original encoder and reuses the original document index\. Its same\-language score preservation follows directly from retaining the original embedding path\.
2. 2\.A mathematical analysis of how contrastive learning directs adapted queries toward relevant, fixed document embeddings relative to distractors, together with the conditions under which routing preserves original rankings\.
3. 3\.A performance evaluation of SampoTron demonstrating 20\.9% mean cross\-language gain on the sampled FIQA corpus, with all six language pairs improving and same\-language scores preserved\. Additionally, SampoHarrier applies the same architecture tomicrosoft/harrier\-oss\-v1\-0\.6bwith a separately trained query adapter and also improves cross\-language retrieval\. These results support the architecture’s generalizability across encoder models through model\-specific query\-adapter training\.

#### Notation\.

We writeqqfor a query,ddfor a document, andℓq,ℓD\\ell\_\{q\},\\ell\_\{D\}for query and index languages\. Bold letters denote vectors\. All retrieval vectors have unit Euclidean norm, so their inner product equals cosine similarity\. The upright subscriptsbase\\mathrm\{base\}andadapt\\mathrm\{adapt\}label quantities associated with the original and adapted encoders, respectively\. The subscriptR\\mathrm\{R\}denotes the routed system\. Parameter sets are denoted separately byθ\\thetafor frozen base parameters andϕ\\phifor trainable adapter parameters\.

Section[2](https://arxiv.org/html/2610.02875#S2)positions the proposed approach against the state\-of\-the\-art\. Section[3](https://arxiv.org/html/2610.02875#S3)presents the architecture and geometry\. Sections[4](https://arxiv.org/html/2610.02875#S4)and[5](https://arxiv.org/html/2610.02875#S5)describe the experiments and results\. Section[6](https://arxiv.org/html/2610.02875#S6)concludes the paper\.

## 2State of the art

#### Multilingual representations and cross\-lingual alignment\.

Multilingual E5 uses large\-scale contrastive pretraining followed by supervised fine\-tuning\[[15](https://arxiv.org/html/2610.02875#bib.bib15)\]\. Qwen3 Embedding adapts pretrained Qwen3 language models for embedding and reranking tasks\[[19](https://arxiv.org/html/2610.02875#bib.bib19)\]\. Its training pipeline also incorporates synthetic multilingual data\. Both model families provide representations for multiple tasks and languages\.

Beyond multilingual coverage, targeted methods address the balance between monolingual and cross\-lingual retrieval\. Hybrid batch training combines both types of query–document pairs to improve performance in each setting\[[1](https://arxiv.org/html/2610.02875#bib.bib1)\]\. Furthermore, Hong et al\.\[[3](https://arxiv.org/html/2610.02875#bib.bib3)\]combine distributional alignment with contrastive retrieval training\. Their evaluation also identifies language bias in mixed\-language document pools\. CLEAR uses English passages as a bridge between languages\[[7](https://arxiv.org/html/2610.02875#bib.bib7)\]\. It combines reverse\-direction contrastive training with a distribution\-matching objective to improve cross\-lingual retrieval while limiting English degradation\. These methods seek to preserve monolingual performance through training\. In contrast, our method preserves the original encoder and document vectors for same\-language requests at inference\.

#### Adapters and fixed document indices\.

Low\-rank adaptation \(LoRA\) freezes a pretrained model and learns small matrix updates\[[4](https://arxiv.org/html/2610.02875#bib.bib4)\]\. Its use in multilingual retrieval is well established in literature\. Yang et al\.\[[16](https://arxiv.org/html/2610.02875#bib.bib16)\]study language and retrieval adapters for zero\-shot cross\-language transfer\. They show that changing language adapters at inference can create alignment problems\. Additionally, Jina\-embeddings\-v3 provides task\-specific LoRA adapters, including separate adapters for query and passage encoding\[[12](https://arxiv.org/html/2610.02875#bib.bib12)\]\.

Query\-only adaptation is also an explored area in the state\-of\-the\-art\. Specifically, Jain et al\.\[[5](https://arxiv.org/html/2610.02875#bib.bib5)\]adapt query encoders while keeping document indices fixed\. They evaluate this approach on enterprise retrieval and public benchmarks\. Moreover, their multi\-tenant system selects tenant\-specific adapters over a shared base model\.

#### Contribution relative to this work\.

Our contribution combines these components through an explicit language\-routing rule\. We train one cross\-language query adapter against document embeddings from a frozen encoder\. The router selects this adapter when the query and index languages differ\. Otherwise, it retains the vanilla encoder\. Same\-language preservation therefore follows from the inference policy, rather than from the adapted model’s ability to retain monolingual performance\.

Our evaluation covers Finnish and Swedish with English, including both Finnish–Swedish directions\. Consequently, the results demonstrate gains over the selected base models on translated financial retrieval dataset, while maintaining base model performance on monolingual retrieval for Wikipedia dataset in Finnish\.

## 3Architecture and embedding alignment

### 3\.1A fixed document encoder and an adaptable query path

LetFbaseF\_\{\\mathrm\{base\}\}denote the original text encoder with frozen parametersθ\\theta\. It includes a pooling operation that aggregates token representations into a single embedding\. For SampoTron,PmP\_\{m\}selects the firstm=1024m=1024coordinates of its 2048\-dimensional output\. The normalized representation is

Ebase​\(x\)=Pm​Fbase​\(x\)‖Pm​Fbase​\(x\)‖2,𝐝base​\(d\)=Ebase​\(TD​\(d\)\),𝐪base​\(q\)=Ebase​\(TQ​\(q\)\),E\_\{\\mathrm\{base\}\}\(x\)=\\frac\{P\_\{m\}F\_\{\\mathrm\{base\}\}\(x\)\}\{\\left\\lVert P\_\{m\}F\_\{\\mathrm\{base\}\}\(x\)\\right\\rVert\_\{2\}\},\\qquad\\mathbf\{d\}\_\{\\mathrm\{base\}\}\(d\)=E\_\{\\mathrm\{base\}\}\(T\_\{D\}\(d\)\),\\qquad\\mathbf\{q\}\_\{\\mathrm\{base\}\}\(q\)=E\_\{\\mathrm\{base\}\}\(T\_\{Q\}\(q\)\),\(1\)whereTQT\_\{Q\}andTDT\_\{D\}denote query and document preprocessing, respectively\. Since, document encoding uses frozen weights, our query adaptation approach does not require rebuilding the inference index\.

Concretely, for a frozen weight matrixW∈ℝa×bW\\in\\mathbb\{R\}^\{a\\times b\}, LoRA uses

Wadapt=W\+αr​B​A,A∈ℝr×b,B∈ℝa×r\.W\_\{\\mathrm\{adapt\}\}=W\+\\frac\{\\alpha\}\{r\}BA,\\qquad A\\in\\mathbb\{R\}^\{r\\times b\},\\quad B\\in\\mathbb\{R\}^\{a\\times r\}\.\(2\)Only the parameter matricesAAandBBreceive updates\. The parameter setϕ\\phicollects these matrices across all adapted projections\. The rankrrcontrols each update’s information depth\. We apply the updates to the attention and feed\-forward projections\. The resulting normalized query vector,𝐪adapt​\(q\)\\mathbf\{q\}\_\{\\mathrm\{adapt\}\}\(q\), depends on the frozen parametersθ\\thetaand trainable parametersϕ\\phi\. In contrast, documents always use the original weightsθ\\theta\. Figure[1](https://arxiv.org/html/2610.02875#S3.F1)shows how training and inference share the frozen document encoder\.

TrainingQuery encoder \+ LoRATrainable adapterDocument encoderFrozen base weightsContrastive lossRetrieval scoresPositive vectorsNegative vectorsFrozen cacheInferenceQuery language \+ index languageDeterministic language ruleSame languageBase encoderDifferent languagesBase \+ LoRASimilarity searchOriginal document indexThe frozen encoder anchors document geometry\.Only query representations are adapted\.Figure 1:Training and inference for language\-routed query adaptation\. Training updates only the query adapter using positive and negative document vectors from the frozen encoder\. The dashed arrow denotes adapter updates\. At inference, query and index language labels select the original or adapted query path\. Both paths share one base model and search the same original document index\.
### 3\.2Contrastive objective and query–document alignment

The training objective promotes higher query similarity to relevant documents than to competing documents\. For queryii, let𝐝i\+\\mathbf\{d\}\_\{i\}^\{\+\}denote its positive document vector\. The candidate set𝒞i\\mathcal\{C\}\_\{i\}contains the batch’s positive and hard\-negative vectors\. Here, hard negatives receive high lexical\-ranking scores but are not labeled relevant\. For a batch ofNNqueries, inverse temperatureβ=20\\beta=20controls the concentration of the softmax distribution\. The objective function, as defined in\[[14](https://arxiv.org/html/2610.02875#bib.bib14)\],ℒ\\mathcal\{L\}is expressed as:

ℒ=1N​∑i=1Nℒi,ℒi=−log⁡exp⁡\(β​𝐪i⊤​𝐝i\+\)∑𝐝j∈𝒞iexp⁡\(β​𝐪i⊤​𝐝j\),𝐪i=𝐪adapt​\(qi\)\.\\mathcal\{L\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathcal\{L\}\_\{i\},\\qquad\\mathcal\{L\}\_\{i\}=\-\\log\\frac\{\\exp\(\\beta\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{d\}\_\{i\}^\{\+\}\)\}\{\\sum\_\{\\mathbf\{d\}\_\{j\}\\in\\mathcal\{C\}\_\{i\}\}\\exp\(\\beta\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{d\}\_\{j\}\)\},\\quad\\mathbf\{q\}\_\{i\}=\\mathbf\{q\}\_\{\\mathrm\{adapt\}\}\(q\_\{i\}\)\.\(3\)A full batch contains 256 positive and 256 explicit negative vectors\. Additionally, other rows’ positives serve as negatives for each query\. Gradient caching computes this large\-batch loss and replays smaller query batches for backpropagation\[[2](https://arxiv.org/html/2610.02875#bib.bib2)\]\.

For unit vectors,‖𝐪i−𝐝j‖22=2−2​𝐪i⊤​𝐝j\\left\\lVert\\mathbf\{q\}\_\{i\}\-\\mathbf\{d\}\_\{j\}\\right\\rVert\_\{2\}^\{2\}=2\-2\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{d\}\_\{j\}\. Consequently, the same loss can be written as

ℒi=log∑𝐝j∈𝒞iexp\[β2\(‖𝐪i−𝐝i\+‖22−‖𝐪i−𝐝j‖22\)\]\.\\mathcal\{L\}\_\{i\}=\\log\\\!\\sum\_\{\\mathbf\{d\}\_\{j\}\\in\\mathcal\{C\}\_\{i\}\}\\exp\\\!\\left\[\\frac\{\\beta\}\{2\}\\left\(\\left\\lVert\\mathbf\{q\}\_\{i\}\-\\mathbf\{d\}\_\{i\}^\{\+\}\\right\\rVert\_\{2\}^\{2\}\-\\left\\lVert\\mathbf\{q\}\_\{i\}\-\\mathbf\{d\}\_\{j\}\\right\\rVert\_\{2\}^\{2\}\\right\)\\right\]\.\(4\)Reducing this objective favors a smaller distance to the relevant document relative to competing documents\. Thus, relevance supervision can align a Finnish query with an English answer without requiring literal translations\.

The gradient makes the direction of the training signal explicit\. Letpi​jp\_\{ij\}be the softmax weight assigned to candidatejj:

pi​j=exp⁡\(β​𝐪i⊤​𝐝j\)∑kexp⁡\(β​𝐪i⊤​𝐝k\),−∇𝐪iℒi=β⁡\(𝐝i\+−∑jpi​j​𝐝j\)\.p\_\{ij\}=\\frac\{\\exp\(\\beta\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{d\}\_\{j\}\)\}\{\\sum\_\{k\}\\exp\(\\beta\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{d\}\_\{k\}\)\},\\qquad\-\\nabla\_\{\\mathbf\{q\}\_\{i\}\}\\mathcal\{L\}\_\{i\}=\\beta\\left\(\\mathbf\{d\}\_\{i\}^\{\+\}\-\\sum\_\{j\}p\_\{ij\}\\mathbf\{d\}\_\{j\}\\right\)\.\(5\)The positive vector contributes attraction, partly offset by its own softmax term\. Other candidates contribute repulsion, with higher\-scoring distractors receiving larger weights\. To isolate the attractive component, consider the normalized step𝐪i​\(ϵ\)=\(𝐪i\+ϵ​𝐝i\+\)/‖𝐪i\+ϵ​𝐝i\+‖2\\mathbf\{q\}\_\{i\}\(\\epsilon\)=\(\\mathbf\{q\}\_\{i\}\+\\epsilon\\mathbf\{d\}\_\{i\}^\{\+\}\)/\\left\\lVert\\mathbf\{q\}\_\{i\}\+\\epsilon\\mathbf\{d\}\_\{i\}^\{\+\}\\right\\rVert\_\{2\}\. Its local effect is

dd​ϵ​‖𝐪i​\(ϵ\)−𝐝i\+‖22\|ϵ=0=−2​\[1−\(𝐪i⊤​𝐝i\+\)2\]≤0\.\\left\.\\frac\{\\mathrm\{d\}\}\{\\mathrm\{d\}\\epsilon\}\\left\\lVert\\mathbf\{q\}\_\{i\}\(\\epsilon\)\-\\mathbf\{d\}\_\{i\}^\{\+\}\\right\\rVert\_\{2\}^\{2\}\\right\|\_\{\\epsilon=0\}=\-2\\left\[1\-\(\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{d\}\_\{i\}^\{\+\}\)^\{2\}\\right\]\\leq 0\.\(6\)This local result describes the attractive component in embedding space\. However, the complete training update also depends on candidate repulsion and the network’s parameterization\. Note that, the frozen document vectors provide stationary targets throughout training\.

#### Geometrical understanding through the document subspace\.

For a given candidate set, define𝒮i=span⁡\{𝐝j:𝐝j∈𝒞i\}\\mathcal\{S\}\_\{i\}=\\operatorname\{span\}\\\{\\mathbf\{d\}\_\{j\}:\\mathbf\{d\}\_\{j\}\\in\\mathcal\{C\}\_\{i\}\\\}\. This span contains all linear combinations of the candidate document vectors\. The gradient in Equation[5](https://arxiv.org/html/2610.02875#S3.E5)lies in this span\. Normalization projects the correction onto the unit sphere’s tangent space at𝐪i\\mathbf\{q\}\_\{i\}and scales it by the inverse pre\-normalization norm\. The projected direction changes orientation while preserving unit norm to first order\. Let𝐡i\\mathbf\{h\}\_\{i\}be the unnormalized query vector, with𝐪i=𝐡i/‖𝐡i‖2\\mathbf\{q\}\_\{i\}=\\mathbf\{h\}\_\{i\}/\\left\\lVert\\mathbf\{h\}\_\{i\}\\right\\rVert\_\{2\}\. Then

∇𝐡iℒi=β‖𝐡i‖2​\(I−𝐪i​𝐪i⊤\)​\(∑jpi​j​𝐝j−𝐝i\+\)\.\\nabla\_\{\\mathbf\{h\}\_\{i\}\}\\mathcal\{L\}\_\{i\}=\\frac\{\\beta\}\{\\left\\lVert\\mathbf\{h\}\_\{i\}\\right\\rVert\_\{2\}\}\(I\-\\mathbf\{q\}\_\{i\}\\mathbf\{q\}\_\{i\}^\{\\top\}\)\\left\(\\sum\_\{j\}p\_\{ij\}\\mathbf\{d\}\_\{j\}\-\\mathbf\{d\}\_\{i\}^\{\+\}\\right\)\.\(7\)The loss therefore induces an angular correction determined by the relevant document and competing candidates\. Backpropagation maps this correction to the trainable low\-rank parameters\. This gradient explains how the objective promotes alignment with document geometry\.

However, a language’s corpus need not occupy a distinct linear subspace\. Its full span can be all ofℝm\\mathbb\{R\}^\{m\}\. Moreover, the objective does not guarantee that every update reduces the distance to every positive document\. Candidate repulsion, shared parameters, and limited adapter capacity affect the actual update\. Subspace alignment, from a training perspective, is indicative of geometric agreement and not an absolute measure of convergence\.

### 3\.3Language routing and same\-language preservation

The routing policy determines when to use the adapted query representation\. It receives explicit query and index language labels and selects

g\(ℓq,ℓD\)=𝟏\[ℓq≠ℓD\],𝐪R=\(1−g\)𝐪base\+g𝐪adapt,sR\(q,d\)=𝐪R⊤𝐝base\(d\)\.g\(\\ell\_\{q\},\\ell\_\{D\}\)=\\mathbf\{1\}\[\\ell\_\{q\}\\neq\\ell\_\{D\}\],\\qquad\\mathbf\{q\}\_\{\\mathrm\{R\}\}=\(1\-g\)\\mathbf\{q\}\_\{\\mathrm\{base\}\}\+g\\mathbf\{q\}\_\{\\mathrm\{adapt\}\},\\qquad s\_\{\\mathrm\{R\}\}\(q,d\)=\\mathbf\{q\}\_\{\\mathrm\{R\}\}^\{\\top\}\\mathbf\{d\}\_\{\\mathrm\{base\}\}\(d\)\.\(8\)The binary gateggselects either𝐪base\\mathbf\{q\}\_\{\\mathrm\{base\}\}or𝐪adapt\\mathbf\{q\}\_\{\\mathrm\{adapt\}\}without interpolation\. The base scoring function issbase​\(q,d\)=𝐪base​\(q\)⊤​𝐝base​\(d\)s\_\{\\mathrm\{base\}\}\(q,d\)=\\mathbf\{q\}\_\{\\mathrm\{base\}\}\(q\)^\{\\top\}\\mathbf\{d\}\_\{\\mathrm\{base\}\}\(d\)\. For same\-language requests,

ℓq=ℓD⟹sR​\(q,d\)=sbase​\(q,d\)for every indexed​d\.\\ell\_\{q\}=\\ell\_\{D\}\\ \\Longrightarrow\\ s\_\{\\mathrm\{R\}\}\(q,d\)=s\_\{\\mathrm\{base\}\}\(q,d\)\\quad\\text\{for every indexed \}d\.\(9\)This identity preserves the original ranking when preprocessing, document vectors, search settings, and tie\-breaking remain unchanged\. Consequently, every metric computed from that ranking is also preserved\. The result is independent of the learned adapter parameters\. However, it requires correct language metadata and faithful execution of the selected encoder\. It does not imply that the original model is equally accurate across languages\.

The proposed approach requires one copy of the base model and an unmerged adapter\. During serving, the adapter is enabled or disabled for each route\. If requests share the adapter state, selection and encoding must form one uninterrupted operation\. Furthermore, the evaluated policy assumes one known language per index\. Code\-switching and mixed\-language document pools require a separate routing policy and evaluation, and are beyond the scope of this paper\.

## 4Experimental setup

### 4\.1SampoTron: Query \- Routed LoRA Adapted Nemotron\-3\-1B\-BF16 Embedder

We disclose in this paper a rank\-64 LoRA adapter called SampoTron\. It uses NVIDIA’s Nemotron\-3\-Embed\-1B\-BF16 embedder as the base model222Model:nvidia/Nemotron\-3\-Embed\-1B\-BF16\. Full revision:a5e0f804b9e90a1ca6784ecbf6e41595774fc834\.\. The model uses average pooling and supports a normalized 1024\-dimensional prefix of its native output\[[11](https://arxiv.org/html/2610.02875#bib.bib11)\]\. We compare the original model, the adapter applied to every query \(*ungated*\), and the routed system\. All three configurations use the same original document vectors\. This comparison also assists in separating the adapter’s effect from the routing policy’s own contribution\.

#### Training data\.

The training mixture contains 113,043 query–positive–negative rows\. These comprise 76,377 FIQA rows, 25,674 MIRACL\-derived rows, and 10,992 Mr\. TyDi\-derived rows\[[8](https://arxiv.org/html/2610.02875#bib.bib8),[13](https://arxiv.org/html/2610.02875#bib.bib13),[18](https://arxiv.org/html/2610.02875#bib.bib18),[17](https://arxiv.org/html/2610.02875#bib.bib17)\]\. The FIQA examples come from its training split\. Each row contains one relevant passage and one explicit hard negative\. The FIQA portion has 12,729 rows in each of five cross\-language directions and 12,732 in Swedish\-to\-English\. However, the full mixture is not balanced across directions\. The Finnish datasets add proportionally more English\-to\-Finnish examples\. The final counts are 28,041 \(en→\\tofi\), 16,977 \(en→\\tosv\), 16,914 \(fi→\\toen\), 16,956 \(fi→\\tosv\), 17,076 \(sv→\\toen\), and 17,079 \(sv→\\tofi\)\.

Cross\-language training rows include machine\-translated Finnish and Swedish text\. For retrieval against English passages, we reused FIQA training examples containing Finnish and Swedish queries translated using NLLB\-200\[[10](https://arxiv.org/html/2610.02875#bib.bib10)\]\. For additional translations a draft\-and\-refinement pipeline was utilized, wherein the drafting and refinement models wereglm\-5\-3\-flashandinferact\-glm53\-nvfp4, respectively\. These models were deployed on an on\-premises cluster consisting of NVIDIA B300 GPUs\.

Further, we also mine new FIQA hard negatives with BM25 lexical ranking in the document language\. A translated query retrieves the top\-50 candidates, and deterministic cycling selects eligible negatives\. Positive–negative text duplicates are filtered\. Additionally, rows sharing a query can contain different relevant passages\. During training\-data preparation, we exclude any FIQA training example whose query identifier or query text matches a query in the FIQA test set\. MIRACL\-derived rows originate from the training split\. Although evaluation queries are excluded from training, some passages in the evaluation corpus may also appear in the training data as passages that are positives or negatives for a given query in training might be utilized as positive or negative for another query in evaluation phase\.

#### Training setup\.

Table[1](https://arxiv.org/html/2610.02875#S4.T1)presents the training settings utilized for developing the LoRA adapter\. The training workload used one NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB physical memory\. The run imposed an 18 GiB allocator cap\. Note that, while the training settings only mention LoRA rank6464, in the actual training runs other LoRA ranks were also used\. However, for the sake of brevity, training parameters and results for the most optimal SampoTron settings are reported in this paper\. The target projections are the internal linear layers receiving LoRA updates: q, k, v, and o denote the attention query, key, value, and output projections\. Additionally, gate, up, and down denote the feed\-forward gating, expansion, and contraction projections\. Early stopping monitors the trailing 20\-step mean training loss from epoch 2\.2\. Training stops at a checkpoint boundary after 60 optimizer steps without at least a 1% relative decrease from the monitor’s best recorded mean\.

Table 1:SampoTron training configuration\. Only LoRA parameters are trainable\. The document cache contains 90,490 unique texts encoded by the frozen base model\.

### 4\.2Evaluation methodology and metrics

#### Cross\-lingual Financial retrieval\.

We evaluate 194 questions selected from FIQA’s 648 test questions by a random shuffle with a fixed seed, corresponding to approximately 30% of the test set\. Each question is available in English, Finnish, and Swedish\. Queries in each language search passages in the same language and in each of the other two languages\. This gives three same\-language and six cross\-language tests, each using the same 194 questions\.

Each test searches a collection of 14,330 passages drawn from FIQA’s 57,638\-passage corpus\. The collection contains all 1,706 passages labeled relevant across the full set of 648 test questions, plus 12,624 additional distractor passages\. These distractors were selected using BM25 lexical ranking, pooling the top\-30 non\-relevant passages per test question and removing duplicates\. They provide challenging alternatives with lexical similarity to the questions\. The same passage identifiers are used for the English, Finnish, and Swedish versions of this collection\.

#### Finnish\-only retrieval\.

We evaluate Finnish\-only retrieval using questions originally written in Finnish from the MIRACL\-fi and Mr\. TyDi\-fi datasets\. MIRACL\-fi uses 1,271 development questions to search 1,883,509 Finnish passages\. Mr\. TyDi\-fi uses 1,738 development questions to search 1,908,757 Finnish passages\. Both evaluations search the complete passage collections rather than sampled subsets\.

#### Scoring\.

We rank documents by the dot product of normalized embeddings\. The main metric is normalized discounted cumulative gain at rank ten \(nDCG@10\)\. It rewards relevant passages near the top of the list and normalizes the score against an ideal ranking\. We treat each passage as either relevant or not relevant to a query\. For each query\-language and document\-language pair, we report the average score across all evaluated queries\. We then report equal\-weight means across the six cross\-language or three same\-language directions\.

## 5Results and discussion

### 5\.1Cross\-language gains in the final system

Figure[2](https://arxiv.org/html/2610.02875#S5.F2)reports the complete language matrix\. SampoTron increases the six\-direction cross\-lingual mean from 0\.2406 to 0\.2908 nDCG@10\. This corresponds to an absolute gain of 0\.0502 and a relative gain of 20\.9%\. Moreover, every cross\-language direction improves\. Finnish\-to\-English rises from 0\.285 to 0\.382, while Finnish\-to\-Swedish rises from 0\.146 to 0\.209\. Swedish\-to\-English remains the highest\-scoring cross\-language direction, increasing from 0\.370 to 0\.436\. English\-to\-Swedish has the smallest absolute gain, from 0\.303 to 0\.315\.

![Refer to caption](https://arxiv.org/html/2610.02875v1/retrieval_matrix.png)Figure 2:Final SampoTron evaluation on 194 query identities per direction and a 14,330\-passage FIQA corpus sample\. Cells show nDCG@10\. Rows indicate query language and columns indicate index language\. The routed matrix follows the language rule: off\-diagonal cells use measured adapter scores, while diagonal cells retain original scores\. The right panel gives routed\-minus\-base differences\. All six cross\-language directions improve\. en: English, fi: Finnish, sv: Swedish\.These results show improved cross\-language ranking of financial passages\. Additionally, we evaluated the same query\-routed architecture onmicrosoft/harrier\-oss\-v1\-0\.6b\[[9](https://arxiv.org/html/2610.02875#bib.bib9)\]\. This implementation uses a separately trained rank\-32 adapter\. Its six\-direction cross\-lingual financial retrieval \(FIQA dataset\) mean rises from 0\.278 to 0\.334, approximately 20%\. Language routing retains the original same\-language FIQA mean of 0\.384\. It also retains the Finnish monolingual performance scores of 0\.723 on MIRACL and 0\.709 on Mr\. TyDi\. Together, the SampoTron and SampoHarrier results support the generalizability of the architecture across encoder models, i\.e\., separately trained query adapters improve cross\-language retrieval for both evaluated models\.

### 5\.2Preserving multi\-monolingual performance through routing

Table[2](https://arxiv.org/html/2610.02875#S5.T2)separates the effects of query adaptation from those of language routing\. Applying the adapter to every query slightly increases mean same\-language FIQA performance\. However, this mean combines an English\-to\-English decrease with Finnish\-to\-Finnish and Swedish\-to\-Swedish improvements\. Larger decreases occur on the native Finnish benchmarks\. Ungated MIRACL\-fi falls from 0\.756 to 0\.675, and Mr\. TyDi\-fi falls from 0\.728 to 0\.665\. Thus, query\-only training does not by itself preserve native Finnish performance\.

Table 2:nDCG@10 for the original model, the final adapter applied to all queries, and SampoTron with language routing\. FIQA values average six cross\-language or three same\-language directions on the sampled corpus\. The Finnish development evaluations search the full corpora\. Routed same\-language values follow the original scoring path and are inherited from the base measurements\.In contrast, routing preserves every original diagonal score and both native\-Finnish scores\. Same\-language requests select the original encoder, as established in Equation[9](https://arxiv.org/html/2610.02875#S3.E9)\. Cross\-language requests retain the adapted\-query gains\. The routed system therefore combines improved cross\-language retrieval with unchanged same\-language rankings on these benchmarks\.

### 5\.3Deployment implications and limitations

The SampoTron adapter occupies approximately 84 MB\. It reuses the original document index, avoiding corpus re\-encoding and duplicate storage for the adapted route\. The architecture therefore adds a specialized query path without duplicating document representations\. The evaluation covers Finnish and Swedish with English, not other Nordic languages\. Additionally, native Swedish has no dedicated benchmark in this study\. Translated FIQA queries and passages may share systematic translation artifacts\. As a result, the gains may not fully represent naturally authored queries\. Moreover, in this paper we do not measure the sensitivity to incorrect language labels, downstream answer quality, organizational productivity, or public\-service outcomes\. The demonstrated benefit is cross\-lingual retrieval improvement with preserved same\-language behavior and reusable document vectors\.

## 6Conclusions

Query\-aware routing supports cross\-language specialization while preserving the original same\-language retrieval path\. A query\-only low\-rank adapter learns relevance against fixed document representations\. A deterministic language rule then selects when to use the adapter\. On the sampled FIQA benchmark, SampoTron increases mean cross\-language nDCG@10 from 0\.241 to 0\.291\. All six English–Finnish–Swedish directions improve, while the original same\-language and native\-Finnish scores remain unchanged\. Additionally, SampoHarrier shows that the architecture can be applied to another base model with a separately trained query adapter to obtain cross\-language gains\.

The approach also reuses the base model and its document index\. The reported cross\-language gains are demonstrated on the sampled FIQA financial\-retrieval benchmark\. Future work should evaluate naturally authored Swedish queries, additional Nordic languages, broader domains, and mixed\-language corpora\. Future work could combine the proposed routing architecture with other cross\-lingual training objectives to assess whether they provide additional retrieval gains while preserving same\-language performance\.

## Acknowledgments and Disclosure of Funding

We acknowledge the contributions of Lucas Björklund, Dan Kozytskyi, Henri Dahl, Nico Roos and Sakari Tanskanen to this work\.

## References

- \[1\]Adel Elmahdy, Sheng\-Chieh Lin, and Amin Ahmad\.Synergistic approach for simultaneous optimization of monolingual, cross\-lingual, and multilingual information retrieval\.arXiv:2408\.10536, 2024\.
- \[2\]Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan\.Scaling deep contrastive learning batch size under memory limited setup\.In*Proceedings of the Sixth Workshop on Representation Learning for NLP*, 2021\.URL[https://arxiv\.org/abs/2101\.06983](https://arxiv.org/abs/2101.06983)\.
- \[3\]Seongtae Hong, Youngjoon Jang, Jungseob Lee, Hyeonseok Moon, and Heuiseok Lim\.Improving semantic proximity in information retrieval through cross\-lingual alignment\.In*International Conference on Learning Representations*, 2026\.URL[https://proceedings\.iclr\.cc/paper\_files/paper/2026/hash/4e5432baa6f217e24f9a4b5921cad30b\-Abstract\-Conference\.html](https://proceedings.iclr.cc/paper_files/paper/2026/hash/4e5432baa6f217e24f9a4b5921cad30b-Abstract-Conference.html)\.
- \[4\]Edward J\. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen\.LoRA: Low\-rank adaptation of large language models\.In*International Conference on Learning Representations*, 2022\.URL[https://arxiv\.org/abs/2106\.09685](https://arxiv.org/abs/2106.09685)\.
- \[5\]Prateek Jain, Shabari S Nair, Ritesh Goru, Prakhar Agarwal, Ajay Yadav, Yoga Sri Varshan Varadharajan, and Constantine Caramanis\.Succeeding at scale: Enterprise retrieval benchmark construction and index\-preserving query adaptation for multi\-tenant search\.In*ICML Workshop on Connecting Low\-rank Representations in AI*, 2026\.doi:10\.48550/arXiv\.2601\.04646\.arXiv:2601\.04646v4\.
- \[6\]Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen\-tau Yih\.Dense passage retrieval for open\-domain question answering\.In*Proceedings of EMNLP*, pages 6769–6781, 2020\.doi:10\.18653/v1/2020\.emnlp\-main\.550\.
- \[7\]Seungyoon Lee, Minhyuk Kim, Seongtae Hong, Youngjoon Jang, Dongsuk Oh, and Heuiseok Lim\.CLEAR: Cross\-lingual enhancement in retrieval via reverse\-training\.In*Proceedings of ACL \(Volume 1: Long Papers\)*, pages 347–362, 2026\.doi:10\.18653/v1/2026\.acl\-long\.13\.
- \[8\]Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur\.WWW’18 open challenge: Financial opinion mining and question answering\.In*Companion Proceedings of The Web Conference*, pages 1941–1942, 2018\.doi:10\.1145/3184558\.3192301\.
- \[9\]Microsoft\.harrier\-oss\-v1\-0\.6b model card\.Hugging Face, 2026\.URL[https://huggingface\.co/microsoft/harrier\-oss\-v1\-0\.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b)\.Accessed September 15, 2026\.
- \[10\]NLLB Team\.No language left behind: Scaling human\-centered machine translation\.arXiv:2207\.04672, 2022\.
- \[11\]NVIDIA\.Nemotron\-3\-Embed\-1B\-BF16 model card\.Hugging Face, 2026\.URL[https://huggingface\.co/nvidia/Nemotron\-3\-Embed\-1B\-BF16](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16)\.Accessed September 15, 2026\. Experiments use the pinned revision specified in Section 4\.
- \[12\]Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, and Han Xiao\.jina\-embeddings\-v3: Multilingual embeddings with task LoRA\.arXiv:2409\.10173, 2024\.
- \[13\]Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych\.BEIR: A heterogeneous benchmark for zero\-shot evaluation of information retrieval models\.In*NeurIPS Datasets and Benchmarks*, 2021\.URL[https://arxiv\.org/abs/2104\.08663](https://arxiv.org/abs/2104.08663)\.
- \[14\]Aaron van den Oord, Yazhe Li, and Oriol Vinyals\.Representation learning with contrastive predictive coding\.arXiv:1807\.03748, 2018\.
- \[15\]Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei\.Multilingual E5 text embeddings: A technical report\.arXiv:2402\.05672, 2024\.
- \[16\]Eugene Yang, Suraj Nair, Dawn Lawrie, James Mayfield, and Douglas W\. Oard\.Parameter\-efficient zero\-shot transfer for cross\-language dense retrieval with adapters\.arXiv:2212\.10448, 2022\.
- \[17\]Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin\.Mr\. TyDi: A multi\-lingual benchmark for dense retrieval\.In*Proceedings of the First Workshop on Multilingual Representation Learning*, 2021\.URL[https://arxiv\.org/abs/2108\.08787](https://arxiv.org/abs/2108.08787)\.
- \[18\]Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso\-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin\.MIRACL: A multilingual retrieval dataset covering 18 diverse languages\.*Transactions of the Association for Computational Linguistics*, 11:1114–1131, 2023\.doi:10\.1162/tacl\_a\_00595\.
- \[19\]Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou\.Qwen3 Embedding: Advancing text embedding and reranking through foundation models\.arXiv:2506\.05176, 2025\.