StreamVibe

Proactive Video Agents for Evolving User Intent

paper: https://t.co/mvEgnsy8JB https://t.co/1c2Qxo3VkB

Paper page - StreamVibe: Proactive Video Agents for Evolving User Intent

Source: https://huggingface.co/papers/2610.34662 Published on Oct 26

#3 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

StreamVibe introduces a benchmark, streaming vision-language model, and hierarchical memory harness to enable proactive, intent-aware interaction with implicit and evolving user goals.

Vision-language modelsare shifting from offline passive understanding toward onlineproactive interaction. However, prevailing methods largely rely on explicit requests while miss theimplicit user intent(e.g., “where should I click next?” while a user navigate a website), meanwhile fail to update the state simultaneously as user requests might update continuously. These limitations make them less suited to intent-aware interaction in practical. To address these challenges, we introduceStreamVibe-Bench, a new benchmark serving as testbed for user intent interaction, with 4.1K evaluation samples and 85K training samples spanning explicit, implicit, anddynamic intents. Building on this benchmark, we developStreamVibe-VL, astreaming proactive modelthat infersimplicit user intents from continuous observations and dynamically tailors its interactions as these intents evolve. We further introduceStreamVibe-Harness, which maintains user intents as persistent states with a fleixble lifecycles via ahierarchical memoryto track the latest user intent and environmental state. Our experiments reveal three key findings: (i)StreamVibe-VLpreserves strong general video understanding while improving response timing and answer quality by 10.7% over other baselines; (ii) existing proactive models remain limited in understanding user intent and deciding when to respond, especially under implicit anddynamic intents; and (iii)StreamVibe-Harnessfurther improves response consistency and adaptation over sustained interaction, resulting in a 12.3% gain inproactive interaction.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.34662 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.34662 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper1