MLAIRE: Multilingual Language-Aware Information Retrieval Evaluation Protocal
Summary
MLAIRE is a multilingual language-aware information retrieval evaluation protocol that separates semantic retrieval accuracy from query-language preference to better assess retrieval utility across mixed-language corpora.
View Cached Full Text
Cached at: 05/18/26, 06:27 PM
Paper page - MLAIRE: Multilingual Language-Aware Information Retrieval Evaluation Protocal
Source: https://huggingface.co/papers/2605.07249
Abstract
Multilingual information retrieval evaluation protocol MLAIRE separates semantic retrieval accuracy from query-language preference to better assess retrieval utility across mixed-language corpora.
Multilingual Information Retrievalis increasingly important in real-world search settings, where users issue queries over mixed-language corpora. Existing evaluations mainly reward language-agnostic semantic relevance, treating relevant passages equally regardless of language. Yet retrieval utility also depends on the language of the retrieved passages: users may prefer results they can read and verify in the query language, and query--passage language mismatch can complicate downstream grounding and answer verification inRetrieval-Augmented Generationsystems. To evaluate this language-aware dimension, we introduce MLAIRE, a Multilingual Language-Aware Information Retrieval Evaluation protocol that disentanglescross-lingual semantic retrievalfromquery-language preference. MLAIRE constructs controlled pools with parallel passages across languages, enabling measurement of semantic retrieval accuracy andquery-language preferencewhen equivalent translations are available. We proposelanguage-aware metrics, includingLanguage Preference Rate(LPR) andLang-nDCG, together with a 4-way decomposition separating semantic andquery-language preferencefailures. Evaluating 31 dense, sparse, andlate-interaction retrievers, we show that standard metrics obscure distinct behaviors: semantically strong retrievers may return correct content in a non-query language, while retrievers with strongerquery-language preferencemay retrieve less semantically relevant passages.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.07249
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.07249 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.07249 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.07249 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG
Researchers identify systematic English and query-language bias in multilingual RAG rerankers and introduce LAURA, a utility-driven alignment method that boosts performance by retrieving answer-critical documents across languages.
MMed-Bench-IR: A Heterogeneous Benchmark for Multilingual Medical Information Retrieval
MMed-Bench-IR is a heterogeneous benchmark for multilingual medical information retrieval across six languages, evaluating cross-lingual alignment, concept discrimination, and evidence retrieval. It reveals severe performance drops for non-English queries, highlighting gaps in existing English-only evaluations.
LAMAR: An Open Language-Aware Multilingual Alignment Reranker
LAMAR is a language-aware multilingual cross-encoder reranker that uses English-anchored relevance distillation and preference alignment to prioritize documents in the same language as the query while preserving semantic relevance, achieving strong performance on multilingual benchmarks.
Libra: Training the Environment for Agentic Information Retrieval
The paper presents Libra, a self-evolving framework that introduces mutable catalogs into repositories to improve code localization for LLM agents, achieving logarithmic improvements and zero-shot transfer across different models and problem sets.
MEUSLI: a Multilingual Projector for LLM-based ASR and Beyond
MEUSLI is an open-source multilingual projector family that connects Whisper encoder with multilingual LLMs, enabling end-to-end ASR in 28 European languages and extending to speech translation and topic identification.