updated results with flickr ablation inline

dinov2_bert_large_results.txt

@@ -4,10 +4,13 @@ STAGE 2: FULL VALIDATION BATTERY
 
 Loading cached datasets...
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12921.78it/s]
+ 49/49 [00:00<00:00, 14609.11it/s]
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 14461.08it/s]
+ 49/49 [00:00<00:00, 13073.85it/s]
   COCO train: 40504, COCO test: 40775
+Loading dataset from disk: 100%
+ 37/37 [00:00<00:00, 9806.59it/s]
+  Flickr30k:  31014
 
 ======================================================================
 EXP 1: MAIN (5 seeds, FULL train)
@@ -15,38 +18,43 @@ EXP 1: MAIN (5 seeds, FULL train)
 
   Seed 42:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 16096.56it/s]
-  seed_42 pretrain val (N=6076): R@1=0.0000 R@5=0.0002 cos=-0.010 CVj=0.167
-    Ep  1: loss=0.3334 c=0.2841 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.537 (37s)
-  Seed 42 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.977 CVj=0.142
+ 49/49 [00:00<00:00, 15620.65it/s]
+  seed_42 pretrain val (N=6076): R@1=0.0002 R@5=0.0016 cos=-0.015 CVj=0.149
+    Ep  1: loss=0.3604 c=0.3112 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.543 (38s)
+  Seed 42 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.198
+  Seed 42 Flickr (N=31014): R@1=1.0000 R@5=1.0000 cos=0.981 CVj=0.248
 
   Seed 123:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12929.91it/s]
-  seed_123 pretrain val (N=6076): R@1=0.0001 R@5=0.0005 cos=-0.004 CVj=0.168
-    Ep  1: loss=0.3222 c=0.2729 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.531 (36s)
-  Seed 123 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.977 CVj=0.155
+ 49/49 [00:00<00:00, 13374.17it/s]
+  seed_123 pretrain val (N=6076): R@1=0.0002 R@5=0.0009 cos=-0.005 CVj=0.161
+    Ep  1: loss=0.3313 c=0.2821 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.537 (38s)
+  Seed 123 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.978 CVj=0.193
+  Seed 123 Flickr (N=31014): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.207
 
   Seed 456:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13957.28it/s]
-  seed_456 pretrain val (N=6076): R@1=0.0000 R@5=0.0005 cos=0.006 CVj=0.165
-    Ep  1: loss=0.3120 c=0.2628 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.524 (36s)
-  Seed 456 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.975 CVj=0.151
+ 49/49 [00:00<00:00, 13673.14it/s]
+  seed_456 pretrain val (N=6076): R@1=0.0000 R@5=0.0005 cos=0.011 CVj=0.161
+    Ep  1: loss=0.3464 c=0.2972 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.533 (38s)
+  Seed 456 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.192
+  Seed 456 Flickr (N=31014): R@1=1.0000 R@5=1.0000 cos=0.981 CVj=0.237
 
   Seed 789:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13756.42it/s]
-  seed_789 pretrain val (N=6076): R@1=0.0000 R@5=0.0010 cos=0.016 CVj=0.184
-    Ep  1: loss=0.3166 c=0.2673 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.531 (37s)
-  Seed 789 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.977 CVj=0.145
+ 49/49 [00:00<00:00, 13702.31it/s]
+  seed_789 pretrain val (N=6076): R@1=0.0001 R@5=0.0005 cos=-0.013 CVj=0.153
+    Ep  1: loss=0.3281 c=0.2789 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.534 (38s)
+  Seed 789 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.979 CVj=0.186
+  Seed 789 Flickr (N=31014): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.225
 
   Seed 2024:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13427.47it/s]
-  seed_2024 pretrain val (N=6076): R@1=0.0005 R@5=0.0020 cos=0.004 CVj=0.159
-    Ep  1: loss=0.3293 c=0.2800 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.534 (37s)
-  Seed 2024 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.978 CVj=0.145
+ 49/49 [00:00<00:00, 13214.23it/s]
+  seed_2024 pretrain val (N=6076): R@1=0.0003 R@5=0.0010 cos=0.024 CVj=0.157
+    Ep  1: loss=0.3103 c=0.2610 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.532 (38s)
+  Seed 2024 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.207
+  Seed 2024 Flickr (N=31014): R@1=1.0000 R@5=1.0000 cos=0.981 CVj=0.219
 
   R@1: 1.0000 ± 0.0000
 
@@ -54,10 +62,10 @@ Loading dataset from disk: 100%
 EXP 2: CONTRASTIVE ONLY
 ======================================================================
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12913.66it/s]
-  contrastive_only pretrain val (N=6076): R@1=0.0000 R@5=0.0004 cos=-0.019 CVj=0.147
-    Ep  1: loss=0.3113 c=0.3113 p=0.0000 a=0.0000 val_R@1=1.0000 val_R@5=1.0000 temp=14.543 (20s)
-  Contrastive only (N=40775): R@1=1.0000 R@5=1.0000 cos=0.971 CVj=0.133
+ 49/49 [00:00<00:00, 13378.52it/s]
+  contrastive_only pretrain val (N=6076): R@1=0.0001 R@5=0.0007 cos=-0.026 CVj=0.161
+    Ep  1: loss=0.3136 c=0.3136 p=0.0000 a=0.0000 val_R@1=1.0000 val_R@5=1.0000 temp=14.548 (22s)
+  Contrastive only (N=40775): R@1=1.0000 R@5=1.0000 cos=0.976 CVj=0.191
 
 ======================================================================
 EXP 3: DEPTH (1,2,4,6 layers)
@@ -65,79 +73,89 @@ EXP 3: DEPTH (1,2,4,6 layers)
 
   1 layers:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 14139.72it/s]
-  depth_1 pretrain val (N=6076): R@1=0.0001 R@5=0.0003 cos=0.003 CVj=0.158
-    Ep  1: loss=0.2801 c=0.2308 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.497 (34s)
-  1L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.988 CVj=0.153
+ 49/49 [00:00<00:00, 12061.79it/s]
+  depth_1 pretrain val (N=6076): R@1=0.0002 R@5=0.0012 cos=-0.002 CVj=0.158
+    Ep  1: loss=0.2866 c=0.2374 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.499 (37s)
+  1L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.979 CVj=0.162
 
   2 layers:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12575.47it/s]
-  depth_2 pretrain val (N=6076): R@1=0.0001 R@5=0.0009 cos=0.007 CVj=0.152
-    Ep  1: loss=0.2803 c=0.2310 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.504 (35s)
-  2L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.988 CVj=0.178
+ 49/49 [00:00<00:00, 13452.96it/s]
+  depth_2 pretrain val (N=6076): R@1=0.0002 R@5=0.0010 cos=-0.014 CVj=0.150
+    Ep  1: loss=0.2941 c=0.2449 p=0.0002 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.509 (37s)
+  2L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.976 CVj=0.181
 
   4 layers:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13950.64it/s]
-  depth_4 pretrain val (N=6076): R@1=0.0002 R@5=0.0007 cos=-0.004 CVj=0.159
-    Ep  1: loss=0.3358 c=0.2866 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.531 (37s)
-  4L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.978 CVj=0.148
+ 49/49 [00:00<00:00, 14176.79it/s]
+  depth_4 pretrain val (N=6076): R@1=0.0000 R@5=0.0002 cos=0.005 CVj=0.146
+    Ep  1: loss=0.3524 c=0.3031 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.542 (38s)
+  4L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.193
 
   6 layers:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13785.95it/s]
-  depth_6 pretrain val (N=6076): R@1=0.0005 R@5=0.0009 cos=-0.001 CVj=0.147
-    Ep  1: loss=0.3918 c=0.3426 p=0.0003 a=0.9844 val_R@1=1.0000 val_R@5=1.0000 temp=14.554 (38s)
-  6L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.974 CVj=0.128
+ 49/49 [00:00<00:00, 13204.04it/s]
+  depth_6 pretrain val (N=6076): R@1=0.0002 R@5=0.0014 cos=0.011 CVj=0.143
+    Ep  1: loss=0.3972 c=0.3479 p=0.0003 a=0.9844 val_R@1=1.0000 val_R@5=1.0000 temp=14.558 (39s)
+  6L (N=40775): R@1=1.0000 R@5=1.0000 cos=0.968 CVj=0.169
 
 ======================================================================
 EXP 4: SCALE (1K → FULL)
 ======================================================================
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12613.29it/s]
+ 49/49 [00:00<00:00, 12187.68it/s]
 
   1000 pairs:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12365.88it/s]
-  scale_1000 pretrain val (N=151): R@1=0.0066 R@5=0.0298 cos=0.002 CVj=0.213
-    Ep  1: loss=4.6368 c=4.5876 p=0.0001 a=0.9849 val_R@1=0.7583 val_R@5=0.9503 temp=14.284 (3s)
-  1000 (N=40775): R@1=0.8682 R@5=0.9725 cos=0.788 CVj=0.381
+ 49/49 [00:00<00:00, 14658.08it/s]
+  scale_1000 pretrain val (N=151): R@1=0.0066 R@5=0.0364 cos=0.015 CVj=0.231
+    Ep  1: loss=4.7240 c=4.6747 p=0.0000 a=0.9849 val_R@1=0.7053 val_R@5=0.9139 temp=14.284 (3s)
+  1000 (N=40775): R@1=0.1257 R@5=0.2825 cos=0.551 CVj=0.417
 
   2000 pairs:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 15297.42it/s]
-  scale_2000 pretrain val (N=301): R@1=0.0000 R@5=0.0199 cos=-0.008 CVj=0.190
-    Ep  1: loss=3.5262 c=3.4770 p=0.0002 a=0.9847 val_R@1=0.9884 val_R@5=1.0000 temp=14.287 (4s)
-  2000 (N=40775): R@1=0.9918 R@5=1.0000 cos=0.900 CVj=0.394
+ 49/49 [00:00<00:00, 14147.51it/s]
+  scale_2000 pretrain val (N=301): R@1=0.0000 R@5=0.0116 cos=-0.014 CVj=0.193
+    Ep  1: loss=3.6022 c=3.5529 p=0.0001 a=0.9847 val_R@1=0.9651 val_R@5=1.0000 temp=14.287 (5s)
+  2000 (N=40775): R@1=0.4119 R@5=0.7025 cos=0.804 CVj=0.618
 
   5000 pairs:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13456.49it/s]
-  scale_5000 pretrain val (N=751): R@1=0.0027 R@5=0.0100 cos=0.002 CVj=0.222
-    Ep  1: loss=1.9448 c=1.8955 p=0.0004 a=0.9846 val_R@1=1.0000 val_R@5=1.0000 temp=14.306 (7s)
-  5000 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.966 CVj=0.300
+ 49/49 [00:00<00:00, 13074.68it/s]
+  scale_5000 pretrain val (N=751): R@1=0.0020 R@5=0.0113 cos=0.014 CVj=0.204
+    Ep  1: loss=2.0504 c=2.0012 p=0.0003 a=0.9846 val_R@1=1.0000 val_R@5=1.0000 temp=14.306 (8s)
+  5000 (N=40775): R@1=0.9942 R@5=1.0000 cos=0.907 CVj=0.364
 
   10000 pairs:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13460.01it/s]
-  scale_10000 pretrain val (N=1501): R@1=0.0003 R@5=0.0017 cos=-0.008 CVj=0.191
-    Ep  1: loss=1.0957 c=1.0464 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.342 (11s)
-  10000 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.978 CVj=0.276
+ 49/49 [00:00<00:00, 14572.85it/s]
+  scale_10000 pretrain val (N=1501): R@1=0.0003 R@5=0.0050 cos=0.004 CVj=0.179
+    Ep  1: loss=1.1547 c=1.1055 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.341 (12s)
+  10000 (N=40775): R@1=0.9996 R@5=1.0000 cos=0.959 CVj=0.351
 
   20000 pairs:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 12668.49it/s]
-  scale_20000 pretrain val (N=3001): R@1=0.0000 R@5=0.0008 cos=0.003 CVj=0.176
-    Ep  1: loss=0.5638 c=0.5145 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.415 (20s)
-  20000 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.206
+ 49/49 [00:00<00:00, 13250.01it/s]
+  scale_20000 pretrain val (N=3001): R@1=0.0000 R@5=0.0005 cos=-0.011 CVj=0.177
+    Ep  1: loss=0.5679 c=0.5186 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.415 (21s)
+  20000 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.977 CVj=0.271
 
   40504 pairs:
 Loading dataset from disk: 100%
- 49/49 [00:00<00:00, 13687.71it/s]
-  scale_40504 pretrain val (N=6076): R@1=0.0003 R@5=0.0017 cos=0.006 CVj=0.149
-    Ep  1: loss=0.3689 c=0.3196 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.535 (36s)
-  40504 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.975 CVj=0.133
+ 49/49 [00:00<00:00, 13208.28it/s]
+  scale_40504 pretrain val (N=6076): R@1=0.0002 R@5=0.0005 cos=0.001 CVj=0.153
+    Ep  1: loss=0.3731 c=0.3238 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.552 (38s)
+  40504 (N=40775): R@1=1.0000 R@5=1.0000 cos=0.979 CVj=0.198
+
+======================================================================
+EXP 5: CROSS-DATASET
+======================================================================
+Loading dataset from disk: 100%
+ 49/49 [00:00<00:00, 12989.56it/s]
+  cross_dataset pretrain val (N=6076): R@1=0.0000 R@5=0.0002 cos=0.005 CVj=0.146
+    Ep  1: loss=0.3524 c=0.3031 p=0.0003 a=0.9845 val_R@1=1.0000 val_R@5=1.0000 temp=14.542 (38s)
+  COCO test (full) (N=40775): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.193
+  Flickr30k (zero-shot, full) (N=31014): R@1=1.0000 R@5=1.0000 cos=0.980 CVj=0.236
 
   Saved to /home/claude/geo_results.json
 
@@ -147,6 +165,7 @@ SUMMARY
 
   EXP 1 — Multi-seed:
     R@1: 1.0000 ± 0.0000
+    Flickr R@1: 1.0000 ± 0.0000
 
   EXP 2 — Ablation:
     Contrastive only: R@1=1.0000
@@ -159,12 +178,16 @@ SUMMARY
     6L (60,971,008): R@1=1.0000
 
   EXP 4 — Scale:
-      1000: R@1=0.8682
-      2000: R@1=0.9918
-      5000: R@1=1.0000
-     10000: R@1=1.0000
+      1000: R@1=0.1257
+      2000: R@1=0.4119
+      5000: R@1=0.9942
+     10000: R@1=0.9996
      20000: R@1=1.0000
      40504: R@1=1.0000
 
+  EXP 5 — Transfer:
+    COCO:   R@1=1.0000
+    Flickr: R@1=1.0000
+
   Ref: chance=0.000025, CLIP~0.60 (400M pairs)
 Done.