Training:
- Download tsv files from here: https://github.com/google-research-datasets/wit/blob/main/DATA.md
- Use
prepare_wit.pyto download images from Wikipedia as annotated on each TSV file. - Use
scale_converter.pyto remove corrupt images and resize suitable images to 224x224. - Use
join_datasets_custom_split.pyto group all JSONs from different subsets of the dataset together. - Use
discard_incorrect_files.pyto filter out images that we were not able to convert. - Finally, use
run-clip.shto train.