TempCloze: Can Video-LLMs Identify the Missing Middle?
Summary
TempCloze is a video cloze benchmark for evaluating visual temporal reasoning in Video-LLMs, requiring models to identify missing video segments from distractors to reduce linguistic shortcuts in semantics, alignment, and progression.
View Cached Full Text
Cached at: 09/11/26, 10:17 AM
Paper page - TempCloze: Can Video-LLMs Identify the Missing Middle?
Source: https://huggingface.co/papers/2609.01515
Abstract
TempCloze evaluates visual temporal reasoning in Video-LLMs by requiring identification of missing video segments from distractors targeting semantics, alignment, and progression.
Temporal reasoningbenchmarks forVideo-LLMsare often mediated by language, leaving room for linguistic shortcuts from option wording, answer correlations, or language priors. To reduce such shortcuts, we introduce TempCloze, avideo clozebenchmark for evaluating visualtemporal reasoninginVideo-LLMs. Given the beginning and ending clips of a video, models must identify the true missing middle from four candidates. TempCloze contains 1,521 carefully filtered videos from seven sources, mainly long-take and egocentric videos. We constructsame-source distractorsalong three dimensions: Semantic asks what event should happen, Alignment probes when it should occur, andProgressiontests how it should unfold, while shared scenes and objects reduce appearance cues. Our evaluation of 10 proprietary and 21 open-sourceVideo-LLMsreveals Alignment as the primary bottleneck: models often recognize plausible semantic content and local eventprogressionbut struggle with temporal alignment. We further conduct error pattern and behavioral sensitivity analyses on TempCloze-Mixed and TempCloze-Hard with four representative models to examine where errors arise and how candidate order, context direction, visible span, frame density, andtest-time scalinginfluence model choices.
View arXiv pageView PDFGitHub5Add to collection
Get this paper in your agent:
hf papers read 2609\.01515
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.01515 in a model README.md to link it from this page.
Datasets citing this paper1
#### CedPei/TempCloze Viewer• Updatedabout 16 hours ago • 1.52k
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.01515 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning
Introduces Video-MME-Logical, a controlled benchmark for evaluating video temporal-logical reasoning in multimodal large language models, revealing a substantial human-model gap.
TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
TimeLens2 introduces a generalist video temporal grounding method using multimodal LLMs, treating temporal evidence as an interval set and achieving state-of-the-art performance across multiple benchmarks.
When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding
This paper studies the ability of multimodal large language models (MLLMs) to detect when the correct answer is absent in video understanding tasks, finding that models systematically fail by selecting plausible distractors instead of recognizing no valid option exists. The failure worsens in temporal reasoning and dense frame sampling, and chain-of-thought prompting only partially mitigates the issue.
When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning
The paper constructs a benchmark to evaluate LLMs on temporal legal reasoning, revealing biases towards applying the most recently enacted laws and an inverse relationship between general reasoning ability and temporal performance.
The Machine's Internal Clock: Do LLMs Share Human Temporal Illusions?
This paper explores if large language models exhibit human-like temporal illusions by testing them on a narrative benchmark, finding that LLMs often choose scenarios consistent with psychology literature rather than mimicking human perception.