third_party/InternVideo/InternVideo2/single_modality/README.md

Single-modality

Installation

Please follow the installation instructions in INSTALL.

Datasets

You can find the dataset instructions in DATASET.

Model ZOO

You can find all the models and the scripts in MODEL_ZOO.

Pre-Training

We use InternVL and VideoMAEv2 pretrained models as teachers by default

For training, you can simply run the pretraining scripts in scripts/pretraining as follows:

bash ./scripts/pretraining/1B_pt.sh

:warning: Notes:

1. Chage DATA_PATH to your data path before running the scripts.
2. --sampling_rate is set to 1 for sprase sampling.
3. The latest checkpoint will be automatically saved while training, thus we use a large --save_ckpt_freq.
4. For InternVideo2-1B and 6B, we use InternVL-C-13B and VideoMAEv2-g.

Finetuning

For finetuning, you can simply run the pretraining scripts in scripts/finetuning as follows:

bash ./scripts/finetuning/full_tuning/k400/1B_ft_k710_ft_k400_f8.sh

:warning: Notes:

1. Chage DATA_PATH And PREFIX to your data path before running the scripts.
2. Chage MODEL_PATH to your model path.
3. Set --use_checkpoint and --checkpoint_num to save GPU memory.
4. The best checkpoint will be automatically evaluated with --test_best.
5. Set --test_num_segment and --test_num_crop for different evaluation strategies.
6. To only run evaluation, just set --eval.