Tag
StreamArena is a benchmark for hour-scale interactive streaming video understanding, paired with a two-tier architecture called StreamMind that outperforms existing streaming baselines across real-time perception, historical recall, proactive interaction, and tool use.
GST-Bench is a new VQA benchmark for evaluating global spatial awareness in video understanding, testing whether VLMs can build coherent global scene representations from long-horizon egocentric video. Evaluation of 22 state-of-the-art VLMs shows a large gap versus humans, with the best model scoring 42.68 vs 79.08.
Engineering notes on optimizing frame selection for feeding video to LLMs, covering scene detection, deduplication strategies, and token budget management.
ViSAGE is a multimodal agentic memory framework for long-form video understanding that builds self-correcting, entity-centric memories via cross-modal binding, bidirectional memory refinement, and multi-agent cross-verification, achieving 5.9% higher accuracy than baselines.
VideoAgent is an all-in-one open-source framework for comprehensive video intelligence, combining understanding, editing, and creative generation through a unified agentic workflow.
Mage-VL is introduced as a multimodal AI model that handles images, text, and video understanding in a single model, enabling richer interactive applications.
This paper introduces Reflective Retrieval Memory (RRM), a memory framework that distills procedural retrieval experience from historical task trajectories to improve evidence retrieval for long-horizon multimodal reasoning. RRM matches or exceeds prior state-of-the-art on M3-Bench-Robot, M3-Bench-Web, and Video-MME-Long benchmarks.
Introduces Pegasus, a low-resource framework that translates human demonstration videos into robot-executable data using graph-based task representation, hierarchical affordance latent space, and closed-loop physics verification, aiming to turn hardware data collection into scalable knowledge transfer.
Google DeepMind launches Gemini Robotics ER 2, an embodied reasoning model that enables robots to understand video, orchestrate tasks, and collaborate with multiple robots, now publicly available via the Gemini API.
Microsoft introduces Mage-VL, a codec-native streaming multimodal foundation model for image and video understanding that achieves up to 3.5x inference speedup by using a sparsity pattern inspired by video codecs, cutting visual tokens by over 75%.
Induction Labs introduces imagination models, a new foundation model architecture that learns from internet-scale video. Their first model, Photon-1, learns to use a computer by watching 18 years of screen recordings without action labels, achieving better results at 30× lower pretraining cost than Gemini 3.1 Flash.
TimeLens2 introduces a generalist video temporal grounding method using multimodal LLMs, treating temporal evidence as an interval set and achieving state-of-the-art performance across multiple benchmarks.
Audio-Visual Flamingo (AV-Flamingo) is a fully open audio-visual large language model designed for understanding and reasoning over long and complex videos, outperforming similarly sized open models and competitive with larger models.
VideoChat3 is a fully open, efficient, and generalist video-centric multimodal large language model that introduces Inflated 3D Vision Transformer (I3D-ViT) and Adaptive Frame Resolution for streaming video perception, along with scalable video data synthesis pipelines, achieving superior performance with only 4B parameters.
360 AI Research presents MoSA, a method that learns object recognition from 10,000 hours of unlabeled video, generating 21 million+ self-labels without human intervention, solving AI's expensive labeling bottleneck.
MOSS-VL-Realtime is a realtime streaming vision-language model that processes continuous video frames, supports interruptible interaction, proactive silence, and dynamic correction, with timestamp-aware encoding and a 256K context window.
Light-Omni is a multimodal agent framework for efficient video understanding that uses dual contextual states (global state and parametric latent state) to avoid iterative reasoning, achieving faster and more accurate processing with significant speedup and memory savings.
The article discusses optimal methods for AI agents to process and understand video content, exploring various techniques for video analysis.
This paper introduces PadCaptioner, a 3B parameter model for omni-modal dense video captioning that uses parallelized autoregressive decoding to achieve high efficiency and quality, outperforming 7B counterparts. A latent planning mechanism enables lossless parallel generation by exploiting weak local dependencies among events.
Video-Oasis reveals that 55% of existing video benchmarks can be solved without visual input, exposing significant capability gaps in current video understanding models. State-of-the-art models perform only marginally above random guessing on the remaining video-native challenges.