MULTI3IR: A Benchmark for Multi-perspective Multi-domain Multi-modal Information Retrieval
Summary
The paper introduces Multi3IR, a benchmark for multi-perspective, multi-domain, multi-modal information retrieval, and proposes SPIN, a method to improve perspective coverage in retrieval systems.
View Cached Full Text
Cached at: 09/03/26, 03:51 AM
Paper page - MULTI3IR: A Benchmark for Multi-perspective Multi-domain Multi-modal Information Retrieval
Source: https://huggingface.co/papers/2608.30949
Abstract
A new benchmark and steering method improve retrieval coverage of diverse perspectives across domains and modalities for open-ended queries.
Information retrieval (IR) increasingly targetsopen-ended queriesthat admit diverse perspectives. Existing IR benchmarks, however, focus primarily on closed-ended queries, while even open-ended benchmarks largely consist of queries whose supporting documents span a single subject domain and modality. We introduce Multi^3IR, a benchmark that evaluates how well retrievers cover the multifaceted perspectives ofopen-ended queriesacross diverse domains and modalities. It comprises 104.9K Stack Exchange queries, each annotated with perspective descriptions that capture the query’s implicit viewpoints. We further propose SPIN, a parameter- and label-efficient method that learnsnoise vectorsto steer embeddings toward diverse yet meaningful semantic directions. Experiments show that existingmultimodal retrieverssuffer fromsingle-perspective bias, while SPIN substantially improvesperspective coverageon Multi^3IR and generalizes well to unseen open-ended IR benchmarks. The dataset and experimental code are available at https://github.com/seokwon99/Multi3IR.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2608\.30949
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.30949 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.30949 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.30949 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MMed-Bench-IR: A Heterogeneous Benchmark for Multilingual Medical Information Retrieval
MMed-Bench-IR is a heterogeneous benchmark for multilingual medical information retrieval across six languages, evaluating cross-lingual alignment, concept discrimination, and evidence retrieval. It reveals severe performance drops for non-English queries, highlighting gaps in existing English-only evaluations.
MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs
MultiView-Bench is a diagnostic benchmark for evaluating vision-language models on their ability to integrate multiple viewpoints into a coherent 3D mental model, revealing systematic failures in 3D spatial reasoning, and introducing ViewNavigator to mitigate these issues.
Scaling Retrieval-Augmented Reasoning with Parallel Search and Explicit Merging
Introduces MultiSearch, an RL-based framework that generates multiple queries at each reasoning step and explicitly merges retrieved information to improve signal-to-noise ratio and reasoning accuracy in question-answering tasks.
UMER: Unifying Embedding and Ranking via Pair-Aware Discriminative Reasoning for Universal Multimodal Retrieval
UMER introduces a unified framework for multimodal retrieval that combines embedding and ranking via pair-aware discriminative reasoning, achieving state-of-the-art performance on the MMEB-V2 benchmark.
M$^3$R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding
This paper introduces M3R-Bench, a unified evidence-grounded benchmark for multimodal metaphor understanding with 1,000 image-text instances, and proposes M3R-Reasoner, an 8B-parameter model combining curriculum-based reasoning supervision and reinforcement learning that outperforms larger proprietary models.