TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

Hugging Face Daily Papers Papers
video-temporal-grounding multimodal-llm interval-set temporal-wasserstein time-lens2 video-understanding

Summary

TimeLens2 introduces a generalist video temporal grounding method using multimodal LLMs, treating temporal evidence as an interval set and achieving state-of-the-art performance across multiple benchmarks.

Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.
Original Article
View Cached Full Text

Cached at: 07/21/26, 06:35 AM

Paper page - TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

Source: https://huggingface.co/papers/2607.17423 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Videomultimodallargelanguagemodels(MLLMs)candescribewhathappensinavideo,butrarelyidentifywhenthesupportingevidenceoccurs.Westudygeneralistvideotemporalgrounding,inwhichonemodelpredictsavariable-cardinalitysetofevidenceintervalsacrossvideolengths,domains,queryforms,andviewpoints.Existingtrainingstrategiesaremisalignedwiththisset-valuedtask:long-videolabelsoftenrelyonbrittleone-passannotation,whilereinforcement-learningrewardseitherfailtodistinguishnon-overlappingpredictionsorrequirefragilesegmentmatching.TimeLens2treatstemporalevidenceasanintervalsetthroughoutsupervisionandoptimization.TimeLens2-93Kconstructsreliablemulti-spansupervisionthroughcaption-derivedproposals,independentlocalization,cross-agentconsensus,semanticverification,andboundaryrefinement.OurtemporalWassersteinrewardcomputesexactone-dimensional\(W_1\)betweenuniformdistributionsovermergedintervalsupports,providingdense,matching-freefeedbackunderunequalcardinalitiesandequivalentfragmentation;temporalIoUcomplementsitwithprecise-overlapfeedback.Acrosssevenbenchmarks,TimeLens2-2Boutperformsallsize-matchedbaselinesoneverybenchmark,whilethe4Band8Bvariantsachievestate-of-the-artperformance,surpassingopen-sourcemodelswithupto397Bparameters.The2B,4B,and8BvariantsimproveovertheirQwen3-VLbackbonesby14.2,13.0,and18.1mIoUpoints,respectively.

View arXiv pageView PDFProject pageGitHub9Add to collection

Get this paper in your agent:

hf papers read 2607\.17423

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper3

#### MCG-NJU/TimeLens2-8B Video-Text-to-Text• 9B• Updated16 minutes ago • 708 • 8 #### MCG-NJU/TimeLens2-4B Video-Text-to-Text• 4B• Updated16 minutes ago • 191 • 7 #### MCG-NJU/TimeLens2-2B Video-Text-to-Text• 2B• Updated16 minutes ago • 5

Datasets citing this paper1

#### MCG-NJU/TimeLens2-93K Viewer• Updated16 minutes ago • 71.4k • 791 • 8

Spaces citing this paper2

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles