VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance
Summary
Introduces VIABench, a comprehensive video benchmark for evaluating multimodal large language models in real-world visual assistance for blind and visually impaired individuals, covering 761 videos and 14,526 annotations across three tasks.
View Cached Full Text
Cached at: 07/20/26, 09:42 AM
Paper page - VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance
Source: https://huggingface.co/papers/2607.14660 We introduceVIABench, a comprehensive, time-aligned video benchmark for evaluating multimodal large language models (MLLMs) in real-world visual assistance scenarios for blind and visually impaired people.
VIABench contains761 videos,14,526 manually curated annotations, and46.9 hours of footage. It covers three complementary tasks:Proactive Reminder,Visual Question Answering, andVision-Guided Interaction. We also proposeToken-Level Prompt Activation Decoding (TPAD), a two-stage framework for evaluating proactive assistance in both online and offline settings.
Our evaluation shows that current MLLMs still struggle with reliable real-world assistance, especially in anticipating navigation-critical events and responding in real time. We hope VIABench encourages progress toward safer and more useful visual assistants.
Code and data:https://github.com/MCG-NJU/VIABench
Similar Articles
@samsja19: great benchmark
Introducing VGI-Bench, a multimodal benchmark that probes 12 distinct visual and audio-visual skills with 550 human-curated questions to expose failures in today's video benchmarks.
WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark
Introduces WorldBench, a visually diverse multimodal reasoning benchmark that reveals significant limitations in current multimodal large language models' visual understanding.
VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?
Introduces VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks for vision-language models, covering feedback-guided repair and reference-guided restyling. Evaluates 20 VLMs and proposes VisEditAgent, a render-grounded editing framework that improves pass rates from 55.75% to 67.99%.
VGI-BENCH: Probing Visual Intelligence in Video Generation Models
VGI-Bench evaluates visual reasoning in video generation models through 27 tasks, revealing limited reliability and self-correction in current systems.
Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
Video-IFBench introduces a comprehensive benchmark for evaluating how well multimodal large language models follow diverse instructions in video understanding tasks, covering various constraints and task types.