EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment

Hugging Face Daily Papers Papers
audiovisual-captioning multimodal-learning video-understanding grpo fine-tuning omni-language-models

Summary

EVA-Cap 提出一种事件中心的音视频视频描述框架，将描述分解为原子事件并通过 SFT 与 GRPO 优化对齐，在 ChronusAV 上平均提升 9.0%，在 WorldSense 上提升 3.3%。

Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning. Nevertheless, existing approaches predominantly treat captioning as an unstructured text generation process, neglecting the inherent semantic structure of videos and leading to suboptimal fine-grained audiovisual alignment. To bridge this gap, we propose EVA-Cap, a novel framework that decomposes captions into atomic audiovisual events and transforms unstructured text supervision into event-centric alignment. We train EVA-Cap via Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) on a self-curated dataset, EVA-Data, which is built with event-centric quality control. During the GRPO phase, the model is explicitly optimized against three event-centric alignment targets: global event completeness, intra-event attribution, and inter-event synchronization. Extensive experiments demonstrate that EVA-Cap significantly outperforms existing methods in event-centric evaluation (e.g., +9.0% average gain on ChronusAV), while achieving competitive performance on general audiovisual captioning benchmarks (e.g., +3.3% on WorldSense).
Original Article
View Cached Full Text

Cached at: 10/16/26, 07:04 AM

Paper page - EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment

Source: https://huggingface.co/papers/2610.22186 Published on Oct 15

·

Submitted byhttps://huggingface.co/JPShi

JPShion Oct 16

Abstract

EVA-Cap improves audiovisual video captioning by decomposing captions into atomic events and optimizing event-centric alignment through supervised fine-tuning and group relative policy optimization.

Recent advances inOmni Language Models(OLMs) provide a unified solution for audiovisual video captioning. Nevertheless, existing approaches predominantly treat captioning as an unstructured text generation process, neglecting the inherent semantic structure of videos and leading to suboptimal fine-grained audiovisual alignment. To bridge this gap, we proposeEVA-Cap, a novel framework that decomposes captions intoatomic audiovisual eventsand transforms unstructured text supervision intoevent-centric alignment. We trainEVA-CapviaSupervised Fine-Tuning(SFT) andGroup Relative Policy Optimization(GRPO) on a self-curated dataset,EVA-Data, which is built with event-centric quality control. During the GRPO phase, the model is explicitly optimized against threeevent-centric alignmenttargets:global event completeness,intra-event attribution, andinter-event synchronization. Extensive experiments demonstrate thatEVA-Capsignificantly outperforms existing methods in event-centric evaluation (e.g., +9.0% average gain on ChronusAV), while achieving competitive performance on general audiovisual captioning benchmarks (e.g., +3.3% on WorldSense).

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.22186

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.22186 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.22186 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.22186 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

Hugging Face Daily Papers

AVC-Reward 提出一个事件中心的音视频一致性评估框架，通过可训练奖励模型在全局和谐、时间同步和属性对齐三个层面评估复杂多事件场景，并在与人类判断对齐上达到最先进水平，还可作为在线强化学习信号提升联合生成模型的一致性。

EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning

Hugging Face Daily Papers

EvaOmni 提出了一个证据接地的音视频推理基准，包含 392 个视频和 1,027 个人工标注问答对，要求模型同时返回答案与证据区间，并提出 Coverage Grounded Score 与 Bottleneck Grounded Score 评估推理链；Gemini3.7-Flash 表现最佳，但复杂推理层面的接地定位仍明显不足。