we have used MOCO-V3, Dino, MAE to pretrain the ViT-base-224 model.
The official codes are implemented here.