Unveil: Unified Visual-Textual Integration and Distillation for Multi-modal Document Retrieval

arXiv cs.CL Papers
multi-modal document-retrieval knowledge-distillation visual-textual embedding parsing-free

Summary

Unveil introduces a unified visual-textual embedding framework for multi-modal document retrieval, using knowledge distillation to transfer semantic understanding from a visual-textual model to a purely visual model, achieving robust and efficient retrieval.

arXiv:2605.24530v1 Announce Type: new Abstract: Document retrieval in real-world scenarios faces significant challenges due to diverse document formats and modalities. Traditional text-based approaches rely on tailored parsing techniques that disregard layout information and are prone to errors, while recent parsing-free visual methods often struggle to capture fine-grained textual semantics in text-rich scenarios. To address these limitations, we propose \textbf{Unveil}, a novel visual-textual embedding framework that effectively integrates textual and visual features for robust document representation. Through knowledge distillation, we transfer the semantic understanding capabilities from the visual-textual embedding model to a purely visual model, enabling efficient parsing-free retrieval while preserving semantic fidelity. Experimental results demonstrate that our visual-textual embedding method surpasses existing approaches, while knowledge distillation successfully bridges the performance gap between visual-textual and visual-only methods, improving both retrieval accuracy and efficiency.
Original Article
View Cached Full Text

Cached at: 05/26/26, 09:03 AM 

# Unveil: Unified Visual-Textual Integration and Distillation for Multi-modal Document Retrieval
Source: [https://arxiv.org/abs/2605.24530](https://arxiv.org/abs/2605.24530)
[View PDF](https://arxiv.org/pdf/2605.24530)

> Abstract:Document retrieval in real\-world scenarios faces significant challenges due to diverse document formats and modalities\. Traditional text\-based approaches rely on tailored parsing techniques that disregard layout information and are prone to errors, while recent parsing\-free visual methods often struggle to capture fine\-grained textual semantics in text\-rich scenarios\. To address these limitations, we propose \\textbf\{Unveil\}, a novel visual\-textual embedding framework that effectively integrates textual and visual features for robust document representation\. Through knowledge distillation, we transfer the semantic understanding capabilities from the visual\-textual embedding model to a purely visual model, enabling efficient parsing\-free retrieval while preserving semantic fidelity\. Experimental results demonstrate that our visual\-textual embedding method surpasses existing approaches, while knowledge distillation successfully bridges the performance gap between visual\-textual and visual\-only methods, improving both retrieval accuracy and efficiency\.

## Submission history

From: Hao Sun \[[view email](https://arxiv.org/show-email/6d19fec6/2605.24530)\] **\[v1\]**Sat, 23 May 2026 11:48:28 UTC \(8,190 KB\)

Similar Articles

ReToken: One Token to Improve Vision-Language Models for Visual Retrieval

Hugging Face Daily Papers

ReToken introduces a single learnable retrieval token that selects sparse query-relevant visual tokens from a pre-filled visual KV cache, improving long-context vision-language model performance on visual retrieval tasks. It yields consistent gains across image and video benchmarks while enabling efficient long-video inference on a single H100.

OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video

Hugging Face Daily Papers

该论文提出 OneSearch-VL，一个以视觉接地证据图（VGEG）为核心的统一多模态深度研究智能体，用于单图、多图和视频研究任务，并构建了 SFT/RL 数据集及细粒度评测基准，在多个图文与视频基准上显著超越 Qwen3-VL-8B。