Tag
The paper proposes Direct Embedding Optimization (DEO), a training-free method that decomposes queries into positive and negative components and optimizes query embeddings with a contrastive objective for negation-aware text and multimodal retrieval, outperforming baselines on NegConstraint and improving Recall@5 over OpenAI CLIP.
This paper compares multimodal foundation model embeddings with late fusion of separately trained modality embeddings via Pareto-based multi-objective optimization for cross-modal galaxy retrieval, finding that the multi-objective approach preserves modality-specific structure and retrieves neighbors 8% closer in raw-data space.
The paper introduces SABER, a controlled benchmark exposing widespread attribute-object binding failures in multimodal retrievers, and shows that fine-tuning an MLLM-based embedder solely on synthetic attribute-binding text achieves SOTA on SABER while improving cross-modal and image-image retrieval without image supervision.
该论文系统研究了混合模态检索器的模态偏好问题,发现无关文本比无关图像对检索性能的干扰更严重(称为 "Chaos in the Text"),并提出 Trident 方法,通过 Multi-Positive View InfoNCE 联合优化文本、图像和融合视图来缓解该偏差。
SnapBench introduces a paired corruption benchmark for robust snap-and-ask multimodal retrieval on mobile, revealing image noise as a key degradation factor and proposing an adaptive fusion method for modality reliability.
This paper proposes STeReO, a reranker for orchestrating heterogeneous speech and text retrievers to enhance Retrieval-Augmented Generation systems by improving evidence selection in multi-modality scenarios.
UMER introduces a unified framework for multimodal retrieval that combines embedding and ranking via pair-aware discriminative reasoning, achieving state-of-the-art performance on the MMEB-V2 benchmark.
This paper introduces UniME-R1, an embedder-adviser framework for unified multimodal retrieval that generates Retrieval-Centric Chain-of-Thought (RC-CoT) conditioned on retrieval feedback, improving retrieval performance by learning from hard negatives.
This paper introduces Bayesian Data Reweighting, a probabilistic framework that adaptively weights query-document pairs to mitigate false negatives in contrastive training, improving multimodal retrieval across multiple knowledge-based VQA benchmarks.
Introduces a language interface for visual evidence attribution in document understanding, using verbatim quotes instead of coordinate-based bounding boxes, achieving significantly higher evidence recall and lower attribution hallucination on CiteVQA.
Proposes SERAF, a multimodal retrieval-augmented framework for time series forecasting that uses both numerical similarity and self-generated textual descriptions to retrieve historical patterns, improving forecasting under non-stationarity. Experiments on seven real-world datasets show effectiveness over state-of-the-art baselines.
This paper introduces Fine-grained Fragment Retrieval (FFR), a new task for locating semantically coherent multi-modal fragments (text and images) within long-form dialogues. The authors propose F2RVLM, a generation-based retrieval model trained with reinforcement learning, and FFRS, a two-stage retrieval system, along with a new dataset MLDR for evaluation.
SMART is a framework that unlocks latent multi-vector capabilities in single-vector models for multimodal retrieval, improving state-of-the-art performance with reduced computational costs via contrastive training and late-interaction inference.