Supervising the Visible State: Separating Representation from View Selection in Streaming Video
Summary
This paper introduces Visible-State Supervision (ViSS) and Test-Time Causal View Search (TVS) to separate representation learning from evidence selection in streaming video understanding, achieving state-of-the-art accuracy on StreamingBench and OVO-Bench with far fewer processed frames.
View Cached Full Text
Cached at: 11/02/26, 11:28 AM
Paper page - Supervising the Visible State: Separating Representation from View Selection in Streaming Video
Source: https://huggingface.co/papers/2610.34926 Authors:
,
,
,
,
,
,
,
,
,
Abstract
Visible-State Supervision and Test-Time Causal View Search improve streaming action recognition by separating representation learning from evidence selection, boosting accuracy with fewer frames.
An action seen once and the same action seen three times receive the same action-recognition label, so that supervision does not require a model to count. Answers shared across histories that differ in event order, accumulation, or relations leave those distinctions unconstrained; we call this state under-specification.Visible-State Supervision(ViSS) addresses it by training on source-grounded probes of event identity, order, accumulation, relation change, and answer sufficiency; under the same recent-frame interface it improvesOVO-Benchover a strict-matched generic temporal supervision baseline while remaining comparable onStreamingBench, with the largest gain inhallucination detection.Test-Time Causal View Search(TVS) then adapts evidence access while keeping the learned model fixed, expanding from recent frames to sparse event and compact history views on demand. The two interventions separate representation from view selection: supervision particularly benefitshallucination detectionandcausal reasoning, access particularly benefitsevent recallandforward reasoning. Together they reach the best reported accuracy among the published systems compared here onStreamingBenchandOVO-Benchat a fraction of the processed frames of exhaustive selection, and strict replay shows that most of the remaining gap to a three-view oracle is a correct answer that was later replaced. Streaming accuracy depends on what supervision asks the model to keep, what evidence inference exposes, and whether the model keeps a correct answer once it has one.
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2610\.34926
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.34926 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.34926 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.34926 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
LatentStream introduces a progressive latent working memory framework that internalizes streaming visual evidence for continuous reasoning, achieving state-of-the-art results on video benchmarks.
@PinzhiHuang: State tracking is a core pillar of video understanding: it requires identifying entities and events, and mapping how th…
Introduces VSTAT, a new benchmark to measure how well multimodal LLMs track states in videos, revealing that frontier models struggle with tasks humans find easy.
Benchmarking Visual State Tracking in Multimodal Video Understanding
Introduces VSTAT, a benchmark for evaluating visual state tracking in multimodal large language models (MLLMs) using 834 clips and 1,500 questions. Current MLLMs perform poorly compared to humans, failing at visual perception rather than reasoning.
OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
OneStreamer introduces a unified framework for streaming video LLMs that jointly learns query-independent evidence recording and task response via Proactive Hierarchical Caption Memory and Proactive State Transition Learning, delivering a 4B model that achieves state-of-the-art results across eight streaming video understanding benchmarks along with the OneStreamer-1M dataset.
SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning
The paper proposes SVoT, a reinforcement learning framework that generates interleaved, verifiable intermediate states and visualizations for multi-hop spatial reasoning in MLLMs, achieving significant accuracy gains on new benchmarks involving multi-object interactions and numerical reasoning.