SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence
Summary
Introduces SVI-Bench, a large-scale benchmark for strategic video intelligence using team sports, designed to evaluate models on dynamic scene understanding, causal reasoning, strategic simulation, and agentic synthesis. The benchmark reveals a capability cliff where models perform well on perceptual tasks but sharply degrade on higher-level strategic reasoning.
View Cached Full Text
Cached at: 06/02/26, 07:33 PM
Paper page - SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence
Source: https://huggingface.co/papers/2605.31529
Abstract
Strategic Video Intelligence requires understanding, causal reasoning, and planning capabilities that current benchmarks fail to evaluate adequately, leading to significant performance gaps in complex cognitive tasks.
True video intelligence demands more than recognizing what is visible: it requires reasoning about why events unfold, predicting what would change under different conditions, and deciding what to do next. We refer to this progression, from perception throughcausal reasoningand simulation tostrategic planning, asStrategic Video Intelligence(SVI). No existing benchmark evaluates this capability stack: in-the-wild videos lack verifiable ground truth for causal and strategic questions, while synthetic environments sacrifice the complexity of real multi-agent systems. To bridge this gap, we introduce SVI-Bench, a large-scale benchmark that leverages team sports as a dynamic microworld, combining the complexity of real-world multi-agent interaction (10-22 agents making coordinated decisions under adversarial pressure) with the verifiability of explicit rules and definitive outcomes. SVI-Bench comprises approximately 35K hours of broadcast video, 15M annotated actions, 15K hours of expert commentary, 23K game reports, and 103K structured statistical records across basketball, soccer, and hockey, all constructed via a data engine that transforms raw game data into a dense, cross-referenced corpus. We organize evaluation into 9 tasks spanning a progressive four-pillar hierarchy:Dynamic Scene Understanding,Causal Reasoning,Strategic Simulation, andAgentic Synthesis. Evaluating strongmultimodalandagentic baselines, we find a capability cliff: models perform competently on perceptual tasks, achieving approximately 73% on fine-grainedaction QA, but degrade sharply at each successive cognitive level. Agentic tasks prove hardest: the strongest model achieves only 5% accuracy when required to autonomously gather and integrate evidence across a corpus of 1.8M clips.
View arXiv pageView PDFProject pageGitHub0Add to collection
Get this paper in your agent:
hf papers read 2605\.31529
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper1
Datasets citing this paper1
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.31529 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Sci-VBench: Evaluating Knowledge- and Reasoning-Intensive Video Generation in Science Domains
Introduces Sci-VBench, a benchmark for evaluating knowledge- and reasoning-intensive video generation across scientific domains, finding that visual realism has not translated into reliable scientific and causal correctness.
@samsja19: great benchmark
Introducing VGI-Bench, a multimodal benchmark that probes 12 distinct visual and audio-visual skills with 550 human-curated questions to expose failures in today's video benchmarks.
RTSGameBench: An RTS Benchmark for Strategic Reasoning by Vision-Language Models
RTSGameBench is a benchmark for evaluating strategic reasoning in vision-language models using the real-time strategy game Beyond All Reason. It provides diverse matchups, diagnostic mini-games, and a self-evolving framework to generate new scenarios.
VGenST-Bench: A Benchmark for Spatio-Temporal Reasoning via Active Video Synthesis
VGenST-Bench is a benchmark that uses generative models to actively synthesize controlled spatio-temporal reasoning scenarios, with a multi-agent pipeline and human quality control, to evaluate multimodal large language models.
OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs
OVO-S-Bench introduces a comprehensive human-annotated benchmark of 1,680 questions across 348 videos to evaluate streaming spatial intelligence in multimodal LLMs, revealing that even the best model (Gemini-3.1-Pro) trails human experts by 27 points. The benchmark exposes key limitations including allocentric mapping as a major bottleneck and chain-of-thought reasoning amplifying spatial errors.