Papers
arxiv:2607.05511

Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory

Published on Jul 6
Β· Submitted by
Nie
on Jul 8
Authors:
,
,
,
,

Abstract

Light-Omni is a multimodal agent framework that enables efficient video understanding through dual contextual states, achieving faster and more accurate video processing by eliminating iterative reasoning while maintaining semantic alignment.

Agentic video understanding equips models with long-term memory to autonomously process and respond to continuous, long-horizon multimodal streams. However, advanced video agents often rely on ``detective-style'' iterative reasoning for action control (e.g., search) and evidence aggregation, incurring prohibitive costs and latency. We argue that such heavy reasoning primarily compensates for the lack of global context and semantic misalignment in retrieval. This paper introduces Light-Omni, a multimodal agent framework for reflexive and lightweight video understanding. It achieves this through dual contextual states that instantly build the required context in a single forward pass. First, we maintain a global state, a finite-sized multimodal script continuously consolidated from episodic memory, serving as the global context for Light-Omni. Through hierarchical merging, it preserves recent details while summarizing past events. Second, conditioned on this global context, we generate a parametric latent state that directly drives autonomous actions and produces retrieval embeddings, with minimal latency. Benefiting from this coupled design, Light-Omni achieves semantically aligned retrieval and reflexive responses while avoiding iterative reasoning. Extensive experiments validate the effectiveness of Light-Omni across multiple video benchmarks. Notably, it outperforms M3-Agent with an average 2.4% accuracy gain, a 12.1times speedup, and a 2.6times improvement in GPU memory efficiency. Furthermore, it serves as a memory system to enhance both the performance and efficiency of existing MLLMs. Project page: https://clare-nie.github.io/Light-Omni.

Community

Paper submitter

We introduce Light-Omni, a highly efficient multimodal agent framework for long-term video understanding. Advanced video agents usually suffer from prohibitive latency due to heavy "detective-style" iterative reasoning. To solve this, Light-Omni enables instant "reflexive" responses via a novel dual-state mechanism (Global & Latent states) in a single forward pass.

Key Highlights:

  • πŸ”₯ Incredible Efficiency: Achieves a 12.1Γ— speedup and 2.6Γ— GPU memory reduction compared to M3-Agent, with near-constant latency (~2.3s) regardless of video length.
  • πŸ“ˆ SOTA Performance: Delivers an average accuracy of 64.8% across VideoMME-long, LVBench, and HippoVlog.
  • πŸ› οΈ Plug-and-Play: Acts as a foundational memory system that seamlessly boosts existing MLLMs (e.g., Qwen2.5-VL, Qwen3-VL, Gemini-2.0-Flash).

Code, models, and dataset are available in our repo!

This comment has been hidden

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2607.05511
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 1

Spaces citing this paper 1

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.