Paper page - ViMU: Benchmarking Video Metaphorical Understanding

Source: https://huggingface.co/papers/2605.14607

Abstract

Video understanding models lack capability to interpret implicit meanings and social contexts beyond literal visual comprehension, necessitating new benchmarking approaches.

Any new medium, once it emerges, is used for more than the transmission of overt content alone. The information it carries typically operates on two levels: one is the content directly presented, while the other is the subtext beneath it-the implicit ideas and intentions the creator seeks to convey through the medium. Likewise, since video technologies became widely adopted, video has served not only as a powerful tool for recording and communicating visual information, but also as a vehicle for emotions, attitudes, andsocial meaningsthat are often difficult to articulate explicitly. Thus, the true meaning of many videos does not reside solely in what is shown on screen; it is often embedded in context, style of expression, and the viewer’s social experience. Some forms of such video subtext are humorous, while others carry irony, mockery, or criticism. Theseimplicit meanings can also be interpreted very differently across cultural backgrounds and social groups. However, most existingvideo understanding modelsstill focus primarily on literal visual comprehension, such as recognizing objects, actions, or temporal relations, and lack a systematic ability to understand the metaphorical, ironic, andsocial meaningsembedded in videos. To bridge this gap, we introduce ViMU, the first benchmark designed to systematically evaluate thesubtext understandingcapabilities of frontier models in videos. ViMU assesses whethervideo understanding modelscan go beyond literal perception to inferimplicit meaningwhile grounding their interpretations inmultimodal evidenceand answering both open-ended and multiple-choice questions. Importantly, all questions are designed to be hint-free, ensuring that no key evidence is disclosed to models before answering.

View arXiv pageView PDFProject pageGitHub10Add to collection

Get this paper in your agent:

hf papers read 2605\.14607

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.14607 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.14607 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.14607 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.