Video-R1: Towards Super Reasoning Ability in Video Understanding
This work aims to integrate deep thinking capabilities into video understanding tasks through the R1 paradigm.
For the first time, we achieved a simultaneous increase in both accuracy and thinking length in video understanding domain.
This is a preliminary repo, and we will continue to develop our Video-R1 model in the future.
Updates
- [2025/02/23] We release training code and data of Video-R1
Findings
Shared Growth of Accuracy and Thinking Length is Possible in Video
In many previous multimodal R1 repositories, the thinking length either showed little to no increase (e.g., Open R1 Video ) or even decreased (e.g., R1-V ).
In this work, we demonstrate that this issue can be addressed by using an appropriate base model and a strong reasoning dataset. We train Qwen2-VL-7B-Instruct using GRPO with accuracy and format rewards on the DVD-counting dataset. Training the 7B model for 900 steps can be completed in approximately 10 hours using 4 x A100 (80G) GPUs. The training curve is as follows:
Weak Base Model Hinders the Emergence of Deep Thinking in Video
We train Qwen2-VL-2B-Instruct using the same setting on the DVD-counting dataset. In contrast, this model shows a decrease in thinking length.
In some cases, the model even skips the thinking process and outputs sentences like this: <think>\n</think>\n<answer>2</answer>.
Weak Reasoning Data Maybe Not Beneficial for Reinforcing Deep Thinking
We train Qwen2-VL-7B-Instruct on a subset of NExT-QA dataset with little reasoning. We can notice that there is almost no increase in thinking length. This indicates that reinforcing deep thinking may require strong reasoning data.
Datasets
The video files are in the zip file and the train/test splits are in the jsonl file.
🤗 Video-R1 Dataset: DVD-counting
This dataset is extracted from "DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue"
Performance
We can observe that RL training results in an accuracy boost of around 10% on DVD-counting-test
| Dataset | Qwen2-VL-7B-Instruct | Video-R1-7B |
|---|---|---|
| DVD-counting-test | 25.0 | 34.5 |
Reasoning Samples:
Set up
git clone https://github.com/tulerfeng/Video-R1
cd Video-R1
# build environment
conda create -n video-r1 python=3.11
conda activate video-r1
bash setup.sh
# qwen video extraction setting
cd src/qwen-vl-utils
pip install -e .
cd ..
# download dataset
git lfs install
git clone https://huggingface.co/datasets/Video-R1/DVD-counting
Please put the downloaded dataset to src/r1-v/data/
Training
Train Qwen2-VL-7B-Instruct with GRPO
bash src/scripts/run_grpo_video.sh
Evaluation
Evaluation on video counting task
python ./src/eval/test_qwen2vl_video_counting.py
Acknowledgements
We sincerely appreciate the contributions of the open-source community. The related projects are as follows:
- R1-V (our initial codebase)
- Open R1 Video (concurrent work)