Dataset Preparation
Training data
annotation: We place the annotation file in dataset/trainval/train_2k5.json.
original videos: You can download download from our organized version in Boshenxx/TimeR1-Dataset;
or you can download and organize the original data from VTG-IT, TimeIT, TimePro, HTStep and LongVid.
Testing data
folder structure:
dataset
├─ timer1
│ ├─ annotations
│ │ ├─ train_2k5.json
│ │ └─ tvgbench.json
│ ├─ videos
│ │ ├─ timerft_data
│ │ | ├─ xxx.mp4
│ │ │ └─ ...
│ │ ├─ tvgbench_data
│ │ | ├─ xxx.mp4
│ │ │ └─ ...
├─ activitynet
│ ├─ annotations
│ │ ├─ sentence_temporal_grounding
│ │ │ └─ test.json
│ ├─ videos
│ | ├─ v_zzz_3yWpTXo.mp4
│ │ └─ ...
├─ charades
│ ├─ Charades_anno
│ │ └─ Charades_v1_test.csv
│ ├─ Charades_v1
│ | ├─ 0I0FX.mp4
│ │ └─ ...
├─ mvbench
│ ├─ json
│ │ ├─ action_antonym.json
│ │ └─ ...
│ ├─ videos
│ │ ├─ clevrer
│ │ └─ ...
├─ tempcompass
│ ├─ questions
│ │ ├─ multi-choice.json
│ │ └─ ...
│ ├─ videos
│ │ ├─ 315784.mp4
│ │ └─ ...
├─ egoschema
│ ├─ MC
│ │ └─ test-00000-of-00001.parquet
│ ├─ Subset
│ │ └─ test-00000-of-00001.parquet
│ ├─ videos
│ │ ├─ 001934bb-81bd-4cd8-a574-0472ef3f6678.mp4
│ │ └─ ...
├─ videomme
│ ├─ videomme
│ │ └─ test-00000-of-00001.parquet
│ ├─ data
│ │ ├─ _8lBR0E_Tx8.mp4
└─ └─ └─ ...
ActivityNet
Download link: ActivityNet
For fine-tuning setting, before training, you need to preprocess the video data.
bash preprocess_video.sh
Specify the path to the Charades-STA dataset (video files, annotations, etc.).
Charades
Download link: Charades-v1
For fine-tuning setting, before training, you need to preprocess the video data.
bash preprocess_video.sh
Specify the path to the Charades-STA dataset (video files, annotations, etc.).
TVGBench
Download link: hf: Boshenxx/TimeR1-Dataset
MVBench
We place the annotation file of tvgbench in dataset/trainval/tvgbench.json.
Download link: hf: OpenGVLab/MVBench
VideoMME
Download link: hf: lmms-lab/MVBench
Egoschema
Download link: hf: lmms-lab/egoschema
TempCompass
Download link: hf: lmms-lab/TempCompass