Paper page - StreamEval: Evaluating Interactive Streaming Video Generation Models

Source: https://huggingface.co/papers/2610.28402

Abstract

StreamEval benchmarks interactive streaming video generation across five tasks, revealing that reliable request execution and sustained state consistency remain major challenges despite high visual quality.

Asstreaming video generationadvances, enabling real-time interaction with generated video is becoming increasingly important. We introduce StreamEval, a unified benchmark forinteractive streaming video generationspanning five tasks:avatar dialogue,avatar performance,world generation,instruction editing, andreference editing. StreamEval comprises 300 scripted test cases lasting 15-300s, each specifying initial assets, incrementally arriving multimodal requests, and explicit requirements for state changes and content preservation. Performance is assessed along four evaluation dimensions: task execution,state consistency, visual fidelity, andaudiovisual coordination, instantiated through ten task-specific metrics on a 0-100 scale. Evaluating 37 distinct models across 300 test cases, we find that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustainedstate consistency. In particular, request execution remains challenging: the highest task-specific scores are 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditionedworld generation, and 15.00 for Edit Success among three reported reference-editing models. These findings identify reliable request execution under timing and preservation constraints as a central challenge for advancingomni-modal real-time interactive systems. The leaderboard is available at https://streameval.ai

View arXiv pageView PDFProject pageGitHub90Add to collection

Get this paper in your agent:

hf papers read 2610\.28402

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.28402 in a model README.md to link it from this page.

Datasets citing this paper1

#### StreamEval/StreamEval Viewer• Updatedabout 6 hours ago • 1.32k • 57 • 6

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.