File size: 1,452 Bytes
3c76023 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | y label
1 \textbf{Controls}
2 random weights, ViT-S/16
3 DINOv2 ViT-S/14 (natural images)
4 ResNet-50, ImageNet (Aug.)
5.6 \textbf{Ours: ssl-vits16}
6.6 ssl-vits16 step 5k
7.6 ssl-vits16 step 20k
8.6 ssl-vits16 step 35k
9.6 ssl-vits16 step 40k
10.6 ssl-vits16 step 45k
11.6 ssl-vits16 step 50k
12.6 ssl-vits16 step 55k
13.6 ssl-vits16 step 60k
14.6 ssl-vits16 step 65k
15.6 ssl-vits16 final (93k)
17.2 \textbf{Ours: ssl-vitb16}
18.2 ssl-vitb16 step 10k
19.2 ssl-vitb16 step 35k
20.2 ssl-vitb16 step 40k
21.8 \textbf{Ours: distilled students}
22.8 D13: Midnight-12k teacher, 20k steps
23.8 D14: H-optimus-0 teacher, 20k steps
24.8 D15: Midnight, 5k, from random
25.8 D15: Midnight, 5k, from step 50k
26.8 D15: Midnight, 5k, from final
27.8 D16: H-optimus-0, 5k, from random
28.8 D16: H-optimus-0, 5k, from step 50k
29.8 \textbf{\textcolor{hrBest}{D16: H-optimus-0, 5k, from final}}
31.4 \textbf{Ours: ViT-B/16 students}
32.4 D18: H-optimus-0, ViT-B/16, 5k, from step 40k
33.4 D18: H-optimus-0, ViT-B/16, 5k, from step 45k
34.4 D18: H-optimus-0, ViT-B/16, 5k, from step 50k
36 \textbf{Released encoders}
37 Hibou-B (Aug.)
38 Phikon v1, ViT-B/16 (Aug.)
39 Virchow2 (Aug.)
40 Midnight-12k, CLS
41 Midnight-12k, CLS+mean
42 Midnight-12k, CLS, 1.0\,\textmu m/px
43 H-optimus-0, ImageNet norm. (Aug.)
44 H-optimus-0, ImageNet norm. (Sep.)
45 H-optimus-0, card norm., CLS
46 H-optimus-0, card norm., CLS+mean
47 H-optimus-0, card norm., CLS, 1.0\,\textmu m/px
|