Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Summary
Video-RSI 提出一种通过 harness 进化实现视频理解智能体递归自我改进的框架,无需更新模型权重,通过重新审视训练视频、收集证据并诊断失败来迭代改进提示、工具和记忆,从而在更少帧处理量下提升准确率。
View Cached Full Text
Cached at: 10/12/26, 06:55 AM
Paper page - Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
Source: https://huggingface.co/papers/2609.37950 We introduceVideo-RSI, a framework for recursive self-improvement of video understanding agents through harness evolution. Instead of updating model weights, Video-RSI enables the agent to revisit the original training videos used for evolution, gather additional visual, transcript, and OCR evidence, and test competing explanations for its failures. Evidence-supported diagnoses are then translated into reusable changes to the agent’s prompts, tools, memory, and execution logic.
Video-RSI uses a cost-aware selection process to retain revisions based on both answer accuracy and visual processing cost. Across the evaluated video understanding benchmarks, the evolved harness improves accuracy while processing fewer frames, yielding competitive accuracy–efficiency trade-offs.
We release the inference and evaluation code, the initial and final harnesses, and the implementation of the harness-evolution workflow. We welcome feedback and discussion from the community.
Similar Articles
Self-Evolving Agent Harness
提出一个自我进化代理框架(Harness RSI),在保持模型权重固定的情况下,通过反馈驱动的修订与进化优化来持续改进代理的软件脚手架(提示、记忆、工作流和可执行组件),并借助持续 PAC 学习给出泛化与保留的理论保证。
RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
This paper introduces Regularized Recursive Self-Improvement (RRSI) for AI agent harnesses, which applies regularization to prevent overfitting during recursive evolution, demonstrating performance gains on multiple benchmarks.
Reels-RSI: Program-Level Self-Improvement for Code-Rendered Explainer Videos
本文提出 Reels-RSI,一个开源的可编程短视频(explainer reel)导演系统,将讲解视频视为程序而非像素,通过代码层面的日志、可执行规则与回顾反思实现 agent 的程序级自我改进,并在 111 个 reel 的实验中验证了规则审计效果及偏好信号的失败模式。
ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
ModularRSI introduces a modular and generalizable framework for recursive self-improvement in AI agent harnesses, using contrastive learning across tasks to evolve modules independently and enhance performance on unseen tasks.
EasyVideoR1: Easier RL for Video Understanding
EasyVideoR1 is an efficient reinforcement learning framework for training large vision-language models on video understanding tasks, featuring offline preprocessing with tensor caching for 1.47x throughput improvement, a task-aware reward system covering 11 problem types, and evaluation across 22 video benchmarks. It also supports joint image-video training and a mixed offline-online data training paradigm.