## Single Transformer SFT code


#### Train 

For CMD (single node):

```commandline
./tools/dist.sh train projects/single_transformer/configs/solo_sft_our_mmistral_llava_resize448.py 8 
```

the logs and ckpts will be saved in "work_dirs/solo_sft_our_mmistral_llava_resize448""

For multi-node training: 

see this trial [link](https://ml.bytedance.net/development/instance/jobs/3662970356615cd3?trialId=32717073), fork this trial and modified number gpus and nodes.

#### Evaluation

For CMD (single node):

```commandline
./tools/dist.sh test projects/single_transformer/configs/solo_sft_our_mmistral_llava_resize448_test.py 8 --checkpoint work_dirs/solo_sft_our_mmistral_llava_resize448/xxxx.pth/
```

Note that please set the test datasets in the configs.


#### Datasets

put the datasets in the ./data folder.

% training data
./data
---./Cambrian-10M
---./llava_data
---------./llava_images
---------./LLaVA-Instruct-150K
---------./LLaVA-Pretrain

% evaluation data 
./data
---./eval

Download pre-processed data from huggingface. https://huggingface.co/datasets/OMG-Research/VLM_EVAL

See the folder: /mnt/bn/xiangtai-training-data/project/VLM/data/eval


#### Configs