VideoJev: System One Model for Video Understanding and Interaction

Hugging Face Daily Papers Papers
video-understanding multimodal-llm temporal-grounding visual-agents inference-efficiency

Summary

VideoJev提出System One范式的视频理解与交互模型家族，通过共享打分头替代自回归解码，统一时序定位与视频问答，并利用缓存和联合推理大幅提升效率。VideoJev-4B和VideoJev-35B-A3B在时序定位基准上显著超越骨干模型，视频问答推理加速超过100倍。

Visual agents must turn video observations into timely decisions to track progress and choose what to do next. Building on the emerging System One paradigm, we introduce VideoJev, a family of models that cast video understanding and interaction as structured decision tasks. We adapt pretrained multimodal large language models (MLLMs) by replacing autoregressive answer decoding with a shared head that scores temporal intervals and textual options against video evidence. These scores yield event spans or answer choices, unifying temporal grounding and multiple-choice video question answering. This combines the semantic flexibility of MLLMs with efficient decisions that agents can use directly. As an interaction unfolds, new goals and intermediate results prompt further queries about the same visual history. VideoJev therefore reuses visual computation through Cached inference for queries as they arrive and Joint inference for multiple queries in a single forward pass. Across three temporal grounding benchmarks, VideoJev-4B and VideoJev-35B-A3B improve mean mIoU over their backbones by 6.7 and 10.8 points, respectively, with measured inference speedups of 2.8-3.1times. On four video QA benchmarks, both achieve performance comparable to their backbones in Thinking mode, with measured inference speedups exceeding 100times. These results offer a unified, reusable decision interface for composing structured visual judgments into complex, open-ended workflows.
Original Article
View Cached Full Text

Cached at: 10/14/26, 06:51 AM

Paper page - VideoJev: System One Model for Video Understanding and Interaction

Source: https://huggingface.co/papers/2610.18615 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Visualagentsmustturnvideoobservationsintotimelydecisionstotrackprogressandchoosewhattodonext.BuildingontheemergingSystemOneparadigm,weintroduceVideoJev,afamilyofmodelsthatcastvideounderstandingandinteractionasstructureddecisiontasks.Weadaptpretrainedmultimodallargelanguagemodels(MLLMs)byreplacingautoregressiveanswerdecodingwithasharedheadthatscorestemporalintervalsandtextualoptionsagainstvideoevidence.Thesescoresyieldeventspansoranswerchoices,unifyingtemporalgroundingandmultiple-choicevideoquestionanswering.ThiscombinesthesemanticflexibilityofMLLMswithefficientdecisionsthatagentscanusedirectly.Asaninteractionunfolds,newgoalsandintermediateresultspromptfurtherqueriesaboutthesamevisualhistory.VideoJevthereforereusesvisualcomputationthroughCachedinferenceforqueriesastheyarriveandJointinferenceformultiplequeriesinasingleforwardpass.Acrossthreetemporalgroundingbenchmarks,VideoJev-4BandVideoJev-35B-A3BimprovemeanmIoUovertheirbackbonesby6.7and10.8points,respectively,withmeasuredinferencespeedupsof2.8-3.1times.OnfourvideoQAbenchmarks,bothachieveperformancecomparabletotheirbackbonesinThinkingmode,withmeasuredinferencespeedupsexceeding100times.Theseresultsofferaunified,reusabledecisioninterfaceforcomposingstructuredvisualjudgmentsintocomplex,open-endedworkflows.

View arXiv pageView PDFProject pageGitHub4Add to collection

Get this paper in your agent:

hf papers read 2610\.18615

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper2

#### ZhuYuhan/VideoJev-4B Feature Extraction• 4B• Updatedabout 2 hours ago • 25 • 5 #### ZhuYuhan/VideoJev-35B-A3B Feature Extraction• 34B• Updatedabout 2 hours ago • 27 • 3

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.18615 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper1

Similar Articles

SearchJev: A Fast and Calibrated System-1 Model for Search Agents

Hugging Face Daily Papers

SearchJev introduces a fast, calibrated System-1 model for search agents that scores legal decisions without autoregressive generation, using Soft-Label Learning for Calibrated Decisions (SLCD) to reduce latency and improve confidence reliability. A dual-system agent combining SearchJev with a System-2 model achieves up to 4.7x speedups on BrowseComp-Plus while raising answer accuracy from 45% to 54%.