EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment
Abstract
EVA-Cap improves audiovisual video captioning by decomposing captions into atomic events and optimizing event-centric alignment through supervised fine-tuning and group relative policy optimization.
Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning. Nevertheless, existing approaches predominantly treat captioning as an unstructured text generation process, neglecting the inherent semantic structure of videos and leading to suboptimal fine-grained audiovisual alignment. To bridge this gap, we propose EVA-Cap, a novel framework that decomposes captions into atomic audiovisual events and transforms unstructured text supervision into event-centric alignment. We train EVA-Cap via Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) on a self-curated dataset, EVA-Data, which is built with event-centric quality control. During the GRPO phase, the model is explicitly optimized against three event-centric alignment targets: global event completeness, intra-event attribution, and inter-event synchronization. Extensive experiments demonstrate that EVA-Cap significantly outperforms existing methods in event-centric evaluation (e.g., +9.0% average gain on ChronusAV), while achieving competitive performance on general audiovisual captioning benchmarks (e.g., +3.3% on WorldSense).
Community
Event-Centric Optimization for Audiovisual Video Captioning: Novel Perspective, Dataset, and Post-Training Framework.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning (2026)
- OmniFysics-Captioner Technical Report: Grounding Omni-Modal Understanding in the Physical World for Better Captioning (2026)
- SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation (2026)
- TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming (2026)
- TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models (2026)
- Advancing Video-Text Pretraining with Multi-View Captions (2026)
- EventLens: Event-Structure Reinforcement Learning for Video Understanding (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.22186 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper