Paper page - VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding

Source: https://huggingface.co/papers/2608.14718

Abstract

VideoGAIA introduces a multi-turn, tool-augmented benchmark that evaluates agentic video understanding for advanced multimodal models through complex real-world tasks.

Video understanding is a fundamental task for evaluating the capabilities ofmultimodal large language models(MLLMs). However, existing leading models have already achieved approximately 90% accuracy on the Video-MME leaderboard, suggesting that conventional single-turn video understanding tasks are becoming increasingly saturated and insufficient for assessing the intelligence of advanced MLLMs. Towards this end, we introduceVideoGAIA, anagentic video understandingbenchmark for general artificial intelligence (AI) assistants. Moving beyond one-shot video question answering,VideoGAIAformulates video understanding as a multi-turn,tool-augmented interactionprocess, where models must iteratively perceive videos, invoke external tools, gather complementary information, and integrate multimodal evidence across turns.VideoGAIAcontains 271 model-human co-designed tasks covering diverse and complex real-world scenarios. Each video-question-answer instance is independently verified by three human experts to ensure both correctness and appropriate difficulty. All evaluated MLLMs, including frontier models such as GPT-5.5 and Kimi-K3, achieve less than 60% accuracy onVideoGAIA, highlighting its value as a high-quality and timely benchmark for evaluating next-generation MLLMs. We hope thatVideoGAIAwill facilitate the transition from conventional video understanding towardagentic video understanding.

View arXiv pageView PDFGitHubAdd to collection

Get this paper in your agent:

hf papers read 2608\.14718

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.14718 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.14718 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.14718 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.