Papers
arxiv:2610.16172

EventLens: Event-Structure Reinforcement Learning for Video Understanding

Published on Oct 12
ยท Submitted by
Guowen Zhang
on Oct 14
Authors:
,
,
,
,
,
,
,

Abstract

Recent multimodal large language models (MLLMs) achieve strong results on video question answering, yet often fail to recover the temporal structure that makes a video coherent. They may merge neighboring events, miss single-event progression, or infer implausible relations across events. We study structured temporal understanding as the ability to recover event structure from visual evidence, including event boundaries, single-event progression, and multi-event relations. We argue that this gap arises from a mismatch between current post-training objectives and temporal reasoning, where caption-style supervision allows models to rely on language shortcuts rather than visual temporal grounding. To address this, we propose EventLens, a vision-centric reinforcement learning framework that learns video understanding through event-structure recovery. Instead of reconstructing captions, EventLens trains models to recover temporal structure under controlled perturbations, such as altered temporal granularity, reversed progression, and shuffled event order. We instantiate this framework with a three-level temporal ontology and derive three verifiable task families: event segmentation, progression discrimination, and multi-event relation reasoning. These tasks admit deterministic rewards, enabling task-specialized GRPO training without human preference labels or learned reward models. We further introduce multi-teacher on-policy distillation (MT-OPD) to consolidate specialized policies into a unified model. Experiments on Qwen3-VL backbones show consistent improvements on temporal grounding and event-sensitive reasoning benchmarks, while preserving general video-QA performance. Learning curves further demonstrate that EventLens achieves comparable downstream transfer with significantly reduced training cost compared to direct mixed RL optimization.

Community

Paper author Paper submitter

Excited to share EventLens (NeurIPS 2026)! ๐Ÿš€

Can video MLLMs truly understand how events unfold over time?

We introduce Event-Structure Reinforcement Learning, a vision-centric framework that learns temporal structure through verifiable rewards rather than caption-style supervision. By explicitly modeling event boundaries, progression, and inter-event relations, EventLens improves temporal grounding and event-sensitive reasoning while preserving general video understanding capabilities.

๐ŸŒ Project: https://lostgreen.github.io/EventLens/

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.16172
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.16172 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.16172 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.16172 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.