we have used MOCO-V3, Dino, MAE to pretrain the ViT-base-224 model. The official codes are implemented here.
