StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

Hugging Face Daily Papers Papers
streaming-video benchmark multimodal long-horizon video-understanding agentic

Summary

StreamArena is a benchmark for hour-scale interactive streaming video understanding, paired with a two-tier architecture called StreamMind that outperforms existing streaming baselines across real-time perception, historical recall, proactive interaction, and tool use.

Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design allows minimal baselines that process only the last four frames to match or surpass complex streaming models, while answer options also expose language shortcuts. We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding. StreamArena contains 243 full-length videos averaging 88.8 minutes and 3,646 rigorously annotated, open-ended question-answer pairs that evaluate real-time perception, historical retrospection, proactive interaction, and multimodal tool utilization. Evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension. Methods that retain only recent frames cannot recover distant events, methods that convert past observations into text lose visual evidence, and methods that repeatedly compress visual memory struggle to preserve fine-grained details over time. We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled frontend workers, while backend workers asynchronously construct persistent multimodal memory and perform historical recall and external search. StreamMind outperforms existing streaming baselines across all four capabilities and reduces query-to-answer latency by reusing persistent state.
Original Article
View Cached Full Text

Cached at: 08/10/26, 06:14 AM

Paper page - StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

Source: https://huggingface.co/papers/2608.05703

Abstract

Deployingautonomousmultimodalagentsincontinuous,real-worldenvironmentsrequiresthemtoingestunboundedaudio-visualstreamsandmaintainhour-scalememory.However,currentevaluationspredominantlyrelyonbriefclipsandmultiple-choiceformats.Thisdesignallowsminimalbaselinesthatprocessonlythelastfourframestomatchorsurpasscomplexstreamingmodels,whileansweroptionsalsoexposelanguageshortcuts.WeintroduceStreamArena,abenchmarkforhour-scale,interactivestreamingvideounderstanding.StreamArenacontains243full-lengthvideosaveraging88.8minutesand3,646rigorouslyannotated,open-endedquestion-answerpairsthatevaluatereal-timeperception,historicalretrospection,proactiveinteraction,andmultimodaltoolutilization.Evaluationacrossdiversesystemsexposesatensionbetweencontinuousinteractionandlong-horizonmultimodalcomprehension.Methodsthatretainonlyrecentframescannotrecoverdistantevents,methodsthatconvertpastobservationsintotextlosevisualevidence,andmethodsthatrepeatedlycompressvisualmemorystruggletopreservefine-graineddetailsovertime.WeaddressthistensionwithStreamMind,atwo-tierarchitecturethatassignslatency-criticalinteractionandproactivemonitoringtoindependentlyscheduledfrontendworkers,whilebackendworkersasynchronouslyconstructpersistentmultimodalmemoryandperformhistoricalrecallandexternalsearch.StreamMindoutperformsexistingstreamingbaselinesacrossallfourcapabilitiesandreducesquery-to-answerlatencybyreusingpersistentstate.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2608\.05703

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.05703 in a model README.md to link it from this page.

Datasets citing this paper1

#### hkuzxc/StreamArena Viewer• Updated3 days ago • 7.78k • 126

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.05703 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction

Hugging Face Daily Papers

HorizonStream introduces a long-horizon attention mechanism for streaming 3D reconstruction that explicitly models geometric propagation via an evidence influence kernel, achieving stable, scalable reconstruction with constant memory and linear time complexity, and generalizing to sequences over 10,000 frames.

Video = World + Event Stream

Hugging Face Daily Papers

Wan-Streamer v0.3 reframes video as a world plus event stream, introducing a general-purpose pretraining task for real-time audio-visual interaction. It achieves low-latency full-duplex interaction with 200ms model-side response latency.

Streaming Communication in Multi-Agent Reasoning

Hugging Face Daily Papers

StreamMA introduces a streaming communication paradigm for multi-agent reasoning that pipelines intermediate results to reduce latency and improve effectiveness by leveraging more reliable early steps, outperforming baselines across benchmarks and revealing a step-level scaling law.