File size: 1,180 Bytes
032e687 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
## Single Transformer SFT code
#### Train
For CMD (single node):
```commandline
./tools/dist.sh train projects/single_transformer/configs/solo_sft_our_mmistral_llava_resize448.py 8
```
the logs and ckpts will be saved in "work_dirs/solo_sft_our_mmistral_llava_resize448""
For multi-node training:
see this trial [link](https://ml.bytedance.net/development/instance/jobs/3662970356615cd3?trialId=32717073), fork this trial and modified number gpus and nodes.
#### Evaluation
For CMD (single node):
```commandline
./tools/dist.sh test projects/single_transformer/configs/solo_sft_our_mmistral_llava_resize448_test.py 8 --checkpoint work_dirs/solo_sft_our_mmistral_llava_resize448/xxxx.pth/
```
Note that please set the test datasets in the configs.
#### Datasets
put the datasets in the ./data folder.
% training data
./data
---./Cambrian-10M
---./llava_data
---------./llava_images
---------./LLaVA-Instruct-150K
---------./LLaVA-Pretrain
% evaluation data
./data
---./eval
Download pre-processed data from huggingface. https://huggingface.co/datasets/OMG-Research/VLM_EVAL
See the folder: /mnt/bn/xiangtai-training-data/project/VLM/data/eval
#### Configs
|