CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
Summary
Introduces CLIP-CC-Bench, a benchmark for evaluating video-language models on paragraph-level descriptions of movie clips, with 200 clips and human reference paragraphs. Tests 17 models and finds that long-form video description remains unsolved.
View Cached Full Text
Cached at: 08/10/26, 10:17 PM
Paper page - CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
Source: https://huggingface.co/papers/2608.04302 Can a video-language model write a faithful paragraph about a minute of film?
Video benchmarks still lean on short clips, one-sentence captions, and multiple-choice QA β none of which test long-form description. CLIP-CC-Bench does.
π¬200 movie clips(~90 s each, 5 h total, 140+ films from 1959β2024), each paired with a human-narrated reference paragraph averaging ~400 words. Narrators describe only what is on screen β no proper nounsβ so recognising the movie earns a model nothing.
βοΈ17 video-language modelsscored by anensemble of 5 embedding judges, combining paragraph-level (coarse) and sentence-level (fine) matching into one harmonic mean, aggregated with Borda count.
πModels get the story, not the details.Coarse beats fine for essentially every model, and the best system reaches a mean HM-CF of only0.67β long-form video description is far from solved.
π We also stress-test the protocol itself: the five judges agree on rankings atmean Spearman 0.98, and across1,000 bootstrap resamplesof the clip set the top model stays #1every timeβ so the leaderboard is a property of the task, not of which 200 clips we happened to pick.
Dataset, per-clip scores for all 17 models, and the full evaluation pipeline are released.
Similar Articles
Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
Video-IFBench introduces a comprehensive benchmark for evaluating how well multimodal large language models follow diverse instructions in video understanding tasks, covering various constraints and task types.
VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance
Introduces VIABench, a comprehensive video benchmark for evaluating multimodal large language models in real-world visual assistance for blind and visually impaired individuals, covering 761 videos and 14,526 annotations across three tasks.
Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning
Introduces Video-MME-Logical, a controlled benchmark for evaluating video temporal-logical reasoning in multimodal large language models, revealing a substantial human-model gap.
AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models
Introduces AnyGroundBench, a domain-adaptation benchmark for spatio-temporal video grounding, evaluating 15 VLMs across five specialized domains and finding current models fail in zero-shot and in-context learning adaptation.
CoVEBench: Can Video Editing Models Handle Complex Instructions?
Introduces CoVEBench, a new benchmark for evaluating compositional video editing capabilities, addressing limitations in handling complex multi-step instructions. The benchmark includes 416 videos, 626 instructions, and 9,990 checklist items, revealing that current models struggle with compositional editing tasks.