Hugging Face's logo

AIStudyHi
/
asyira

Model card Files
xet
 Community
AsyiraFitri commited on
Commit
39aa18b
·
1 Parent(s): 02a99db

Add fusion experiments results

Browse files
Files changed (4) hide show
  1. README.md +20 -6
  2. tabacco3482_benchmarks/figures/fusion_results.png +3 -0
  3. tabacco3482_benchmarks/fusion_summary.txt +80 -0
  4. tabacco3482_benchmarks/results/fusion_summary.json +280 -0
README.md CHANGED
@@ -88,6 +88,8 @@ Document Image Transformer — a ViT pretrained specifically on 42 million scann
88
  | DiT Base | `microsoft/dit-base` | 64 |
89
  | DiT Large | `microsoft/dit-large` | 32 |
90
 
 
 
91
  ---
92
 
93
  ## Evaluation Protocol
@@ -113,29 +115,41 @@ PCA post-processing (mean-centering → PCA whitening → L2 re-normalisation) w
113
 
114
  ## Results
115
 
116
- ![Results](tabacco3482_benchmarks/figures/results.png)
 
117
 
118
- Best classification accuracy: SigLIP SO400M (patch14-384) + postprocess
119
- Best kNN: SigLIP SO400M (patch14-384)
120
- Best Overall: SigLIP SO400M (patch14-384)
121
 
122
  ---
123
 
124
  ## Embedding Fusion (In Progress)
125
 
126
- Experiments combining embeddings from two complementary models via weighted concatenation. Three fusion pairs are being tested:
127
 
128
  | Pair | Rationale |
129
  |---|---|
130
  | SigLIP2 SO400M + DiT Large | VLM features + doc-specific pretraining |
131
  | SigLIP2 SO400M + DINOv2 Base | VLM features + self-supervised vision |
 
 
 
 
132
  | DiT Large + DINOv2 Base | Both vision-focused, different pretraining signals |
133
 
134
- Three strategies per pair:
 
135
  - **Raw concat** (α=0.5) — simple weighted concatenation + L2 renorm
136
  - **PCA-whitened concat** — reduces redundancy across the two embedding spaces
137
  - **Alpha sweep** (α ∈ {0.6, 0.7, 0.8}) — up-weights the stronger model
138
 
 
 
 
 
 
 
139
  ---
140
 
141
  ## Repo Structure
 
88
  | DiT Base | `microsoft/dit-base` | 64 |
89
  | DiT Large | `microsoft/dit-large` | 32 |
90
 
91
+ Saved to [model_registry.txt](tabacco3482_benchmarks/model_registry.txt) and [model_registry.json](tabacco3482_benchmarks/results/model_registry.json)
92
+
93
  ---
94
 
95
  ## Evaluation Protocol
 
115
 
116
  ## Results
117
 
118
+ ![Results](tabacco3482_benchmarks/figures/results.png) <br>
119
+ Results are checkpointed during execution at [`results_checkpoint.txt`](tabacco3482_benchmarks/results_checkpoint.txt) and [`results_checkpoint.json`](tabacco3482_benchmarks/results/results_checkpoint.json), while the final aggregated results are saved at [`benchmark_summary.txt`](tabacco3482_benchmarks/benchmark_summary.txt) and [`benchmark_summary.json`](tabacco3482_benchmarks/results/benchmark_summary.json).
120
 
121
+ - Best classification accuracy: SigLIP SO400M (patch14-384) + postprocess
122
+ - Best kNN: SigLIP SO400M (patch14-384)
123
+ - Best Overall: SigLIP SO400M (patch14-384)
124
 
125
  ---
126
 
127
  ## Embedding Fusion (In Progress)
128
 
129
+ Experiments combining embeddings from two complementary models via weighted concatenation.
130
 
131
  | Pair | Rationale |
132
  |---|---|
133
  | SigLIP2 SO400M + DiT Large | VLM features + doc-specific pretraining |
134
  | SigLIP2 SO400M + DINOv2 Base | VLM features + self-supervised vision |
135
+ | SigLIP SO400M + DiT Large | VLM features + doc-specific pretraining |
136
+ | SigLIP SO400M + DINOv2 Base | VLM features + self-supervised vision |
137
+ | ModernVBERT Bi + DiT Large | Doc-retrieval features + doc-specific pretraining |
138
+ | ModernVBERT Bi + DINOv2 Base | Doc-retrieval features + self-supervised vision |
139
  | DiT Large + DINOv2 Base | Both vision-focused, different pretraining signals |
140
 
141
+ Three strategies tested per pair:
142
+
143
  - **Raw concat** (α=0.5) — simple weighted concatenation + L2 renorm
144
  - **PCA-whitened concat** — reduces redundancy across the two embedding spaces
145
  - **Alpha sweep** (α ∈ {0.6, 0.7, 0.8}) — up-weights the stronger model
146
 
147
+ ### Results
148
+
149
+ ![Fusion Results](tabacco3482_benchmarks/figures/fusion_results.png)
150
+
151
+ Results saved to [`fusion_summary.txt`](results/fusion_summary.txt) and [`fusion_summary.json`](results/fusion_summary.json) on completion.
152
+
153
  ---
154
 
155
  ## Repo Structure
tabacco3482_benchmarks/figures/fusion_results.png ADDED

Git LFS Details

  • SHA256: 94b62b7a81e00d060766d4b1a790cd1667f60a99ffc9f16f0327a3d42d7f67cb
  • Pointer size: 131 Bytes
  • Size of remote file: 106 kB
tabacco3482_benchmarks/fusion_summary.txt ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ================================================================================
3
+ Embedding Fusion Results
4
+ ================================================================================
5
+ Variant Clf kNN@1 kNN@5
6
+ --------------------------------------------------------------------------------
7
+ SigLIP2-SO400M + DiT Large | concat a=0.5 Clf 87.95% kNN@1 87.66% kNN@5 97.27%
8
+ SigLIP2-SO400M + DiT Large | PCA n=512 Clf 88.67% kNN@1 75.61% kNN@5 95.41%
9
+ SigLIP2-SO400M + DiT Large | concat a=0.6 Clf 88.67% kNN@1 87.09% kNN@5 96.41%
10
+ SigLIP2-SO400M + DiT Large | concat a=0.7 Clf 88.95% kNN@1 86.66% kNN@5 96.27%
11
+ SigLIP2-SO400M + DiT Large | concat a=0.8 Clf 88.67% kNN@1 84.79% kNN@5 95.98%
12
+ SigLIP2-SO400M + DINOv2 Base | concat a=0.5 Clf 87.95% kNN@1 86.37% kNN@5 95.98%
13
+ SigLIP2-SO400M + DINOv2 Base | PCA n=512 Clf 88.09% kNN@1 77.62% kNN@5 95.98%
14
+ SigLIP2-SO400M + DINOv2 Base | concat a=0.6 Clf 88.67% kNN@1 87.37% kNN@5 96.84%
15
+ SigLIP2-SO400M + DINOv2 Base | concat a=0.7 Clf 88.38% kNN@1 87.09% kNN@5 96.56%
16
+ SigLIP2-SO400M + DINOv2 Base | concat a=0.8 Clf 88.95% kNN@1 86.08% kNN@5 96.27%
17
+ SigLIP-SO400M + DiT Large | concat a=0.5 Clf 90.10% kNN@1 85.94% kNN@5 97.27%
18
+ SigLIP-SO400M + DiT Large | PCA n=512 Clf 90.24% kNN@1 75.04% kNN@5 94.40%
19
+ SigLIP-SO400M + DiT Large | concat a=0.6 Clf 89.96% kNN@1 87.95% kNN@5 97.56%
20
+ SigLIP-SO400M + DiT Large | concat a=0.7 Clf 90.10% kNN@1 87.66% kNN@5 97.42%
21
+ SigLIP-SO400M + DiT Large | concat a=0.8 Clf 89.96% kNN@1 86.94% kNN@5 96.84%
22
+ SigLIP-SO400M + DINOv2 Base | concat a=0.5 Clf 89.24% kNN@1 86.37% kNN@5 95.98%
23
+ SigLIP-SO400M + DINOv2 Base | PCA n=512 Clf 89.38% kNN@1 78.62% kNN@5 96.13%
24
+ SigLIP-SO400M + DINOv2 Base | concat a=0.6 Clf 90.10% kNN@1 88.38% kNN@5 97.13%
25
+ SigLIP-SO400M + DINOv2 Base | concat a=0.7 Clf 90.39% kNN@1 87.66% kNN@5 96.84%
26
+ SigLIP-SO400M + DINOv2 Base | concat a=0.8 Clf 90.10% kNN@1 86.80% kNN@5 96.56%
27
+ ModernVBERT Bi + DiT Large | concat a=0.5 Clf 86.37% kNN@1 82.07% kNN@5 94.40%
28
+ ModernVBERT Bi + DiT Large | PCA n=512 Clf 86.37% kNN@1 77.19% kNN@5 95.84%
29
+ ModernVBERT Bi + DiT Large | concat a=0.6 Clf 85.65% kNN@1 82.50% kNN@5 93.83%
30
+ ModernVBERT Bi + DiT Large | concat a=0.7 Clf 85.51% kNN@1 81.06% kNN@5 93.54%
31
+ ModernVBERT Bi + DiT Large | concat a=0.8 Clf 84.94% kNN@1 81.35% kNN@5 93.11%
32
+ ModernVBERT Bi + DINOv2 Base | concat a=0.5 Clf 88.09% kNN@1 83.79% kNN@5 95.41%
33
+ ModernVBERT Bi + DINOv2 Base | PCA n=512 Clf 88.38% kNN@1 79.34% kNN@5 95.84%
34
+ ModernVBERT Bi + DINOv2 Base | concat a=0.6 Clf 88.09% kNN@1 83.07% kNN@5 94.84%
35
+ ModernVBERT Bi + DINOv2 Base | concat a=0.7 Clf 86.51% kNN@1 84.07% kNN@5 94.40%
36
+ ModernVBERT Bi + DINOv2 Base | concat a=0.8 Clf 86.23% kNN@1 83.07% kNN@5 94.12%
37
+ DiT Large + DINOv2 Base | concat a=0.5 Clf 77.33% kNN@1 71.45% kNN@5 90.96%
38
+ DiT Large + DINOv2 Base | PCA n=512 Clf 82.07% kNN@1 72.88% kNN@5 94.26%
39
+ DiT Large + DINOv2 Base | concat a=0.6 Clf 76.47% kNN@1 70.01% kNN@5 90.82%
40
+ DiT Large + DINOv2 Base | concat a=0.7 Clf 75.32% kNN@1 69.01% kNN@5 92.11%
41
+ DiT Large + DINOv2 Base | concat a=0.8 Clf 71.02% kNN@1 65.71% kNN@5 90.39%
42
+
43
+ ================================================================================
44
+ Per-Family Best
45
+ ================================================================================
46
+
47
+ SigLIP2
48
+ Best Clf :
49
+ SigLIP2-SO400M + DiT Large | concat a=0.7
50
+ Clf 88.95% kNN@5 96.27%
51
+ Best kNN :
52
+ SigLIP2-SO400M + DiT Large | concat a=0.5
53
+ Clf 87.95% kNN@5 97.27%
54
+
55
+ SigLIP
56
+ Best Clf :
57
+ SigLIP-SO400M + DINOv2 Base | concat a=0.7
58
+ Clf 90.39% kNN@5 96.84%
59
+ Best kNN :
60
+ SigLIP-SO400M + DiT Large | concat a=0.6
61
+ Clf 89.96% kNN@5 97.56%
62
+
63
+ ModernVBERT
64
+ Best Clf :
65
+ ModernVBERT Bi + DINOv2 Base | PCA n=512
66
+ Clf 88.38% kNN@5 95.84%
67
+ Best kNN :
68
+ ModernVBERT Bi + DiT Large | PCA n=512
69
+ Clf 86.37% kNN@5 95.84%
70
+
71
+ ================================================================================
72
+ Overall Best
73
+ ================================================================================
74
+
75
+ Best Clf :
76
+ SigLIP-SO400M + DINOv2 Base | concat a=0.7
77
+ Clf 90.39% kNN@5 96.84%
78
+ Best kNN :
79
+ SigLIP-SO400M + DiT Large | concat a=0.6
80
+ Clf 89.96% kNN@5 97.56%
tabacco3482_benchmarks/results/fusion_summary.json ADDED
@@ -0,0 +1,280 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "results": {
3
+ "SigLIP2-SO400M + DiT Large | concat a=0.5": {
4
+ "clf": 0.8794835007173601,
5
+ "f1": 0.8776871268181167,
6
+ "knn1": 0.8766140602582496,
7
+ "knn5": 0.9727403156384505
8
+ },
9
+ "SigLIP2-SO400M + DiT Large | PCA n=512": {
10
+ "clf": 0.8866571018651362,
11
+ "f1": 0.8836676621246795,
12
+ "knn1": 0.7560975609756098,
13
+ "knn5": 0.9540889526542324
14
+ },
15
+ "SigLIP2-SO400M + DiT Large | concat a=0.6": {
16
+ "clf": 0.8866571018651362,
17
+ "f1": 0.8854715884433103,
18
+ "knn1": 0.8708751793400287,
19
+ "knn5": 0.9641319942611191
20
+ },
21
+ "SigLIP2-SO400M + DiT Large | concat a=0.7": {
22
+ "clf": 0.8895265423242468,
23
+ "f1": 0.8884065034233251,
24
+ "knn1": 0.866571018651363,
25
+ "knn5": 0.9626972740315638
26
+ },
27
+ "SigLIP2-SO400M + DiT Large | concat a=0.8": {
28
+ "clf": 0.8866571018651362,
29
+ "f1": 0.8854512250510171,
30
+ "knn1": 0.8479196556671449,
31
+ "knn5": 0.9598278335724534
32
+ },
33
+ "SigLIP2-SO400M + DINOv2 Base | concat a=0.5": {
34
+ "clf": 0.8794835007173601,
35
+ "f1": 0.8775456088248054,
36
+ "knn1": 0.8637015781922525,
37
+ "knn5": 0.9598278335724534
38
+ },
39
+ "SigLIP2-SO400M + DINOv2 Base | PCA n=512": {
40
+ "clf": 0.8809182209469153,
41
+ "f1": 0.8781900259671098,
42
+ "knn1": 0.776183644189383,
43
+ "knn5": 0.9598278335724534
44
+ },
45
+ "SigLIP2-SO400M + DINOv2 Base | concat a=0.6": {
46
+ "clf": 0.8866571018651362,
47
+ "f1": 0.885323301090747,
48
+ "knn1": 0.8737446197991392,
49
+ "knn5": 0.9684361549497847
50
+ },
51
+ "SigLIP2-SO400M + DINOv2 Base | concat a=0.7": {
52
+ "clf": 0.8837876614060258,
53
+ "f1": 0.8822008131287516,
54
+ "knn1": 0.8708751793400287,
55
+ "knn5": 0.9655667144906743
56
+ },
57
+ "SigLIP2-SO400M + DINOv2 Base | concat a=0.8": {
58
+ "clf": 0.8895265423242468,
59
+ "f1": 0.8880499787458564,
60
+ "knn1": 0.860832137733142,
61
+ "knn5": 0.9626972740315638
62
+ },
63
+ "SigLIP-SO400M + DiT Large | concat a=0.5": {
64
+ "clf": 0.9010043041606887,
65
+ "f1": 0.8986619791211244,
66
+ "knn1": 0.8593974175035868,
67
+ "knn5": 0.9727403156384505
68
+ },
69
+ "SigLIP-SO400M + DiT Large | PCA n=512": {
70
+ "clf": 0.9024390243902439,
71
+ "f1": 0.9006061486749664,
72
+ "knn1": 0.7503586800573888,
73
+ "knn5": 0.9440459110473458
74
+ },
75
+ "SigLIP-SO400M + DiT Large | concat a=0.6": {
76
+ "clf": 0.8995695839311334,
77
+ "f1": 0.8982238782458929,
78
+ "knn1": 0.8794835007173601,
79
+ "knn5": 0.975609756097561
80
+ },
81
+ "SigLIP-SO400M + DiT Large | concat a=0.7": {
82
+ "clf": 0.9010043041606887,
83
+ "f1": 0.8995867606939418,
84
+ "knn1": 0.8766140602582496,
85
+ "knn5": 0.9741750358680057
86
+ },
87
+ "SigLIP-SO400M + DiT Large | concat a=0.8": {
88
+ "clf": 0.8995695839311334,
89
+ "f1": 0.8982964367381786,
90
+ "knn1": 0.8694404591104734,
91
+ "knn5": 0.9684361549497847
92
+ },
93
+ "SigLIP-SO400M + DINOv2 Base | concat a=0.5": {
94
+ "clf": 0.8923959827833573,
95
+ "f1": 0.8905458282341152,
96
+ "knn1": 0.8637015781922525,
97
+ "knn5": 0.9598278335724534
98
+ },
99
+ "SigLIP-SO400M + DINOv2 Base | PCA n=512": {
100
+ "clf": 0.8938307030129125,
101
+ "f1": 0.8922013234383107,
102
+ "knn1": 0.7862266857962698,
103
+ "knn5": 0.9612625538020086
104
+ },
105
+ "SigLIP-SO400M + DINOv2 Base | concat a=0.6": {
106
+ "clf": 0.9010043041606887,
107
+ "f1": 0.8987865269335131,
108
+ "knn1": 0.8837876614060258,
109
+ "knn5": 0.9713055954088953
110
+ },
111
+ "SigLIP-SO400M + DINOv2 Base | concat a=0.7": {
112
+ "clf": 0.9038737446197992,
113
+ "f1": 0.9014950999730301,
114
+ "knn1": 0.8766140602582496,
115
+ "knn5": 0.9684361549497847
116
+ },
117
+ "SigLIP-SO400M + DINOv2 Base | concat a=0.8": {
118
+ "clf": 0.9010043041606887,
119
+ "f1": 0.8995705654328235,
120
+ "knn1": 0.8680057388809183,
121
+ "knn5": 0.9655667144906743
122
+ },
123
+ "ModernVBERT Bi + DiT Large | concat a=0.5": {
124
+ "clf": 0.8637015781922525,
125
+ "f1": 0.8630863906707585,
126
+ "knn1": 0.8206599713055954,
127
+ "knn5": 0.9440459110473458
128
+ },
129
+ "ModernVBERT Bi + DiT Large | PCA n=512": {
130
+ "clf": 0.8637015781922525,
131
+ "f1": 0.8605465592114369,
132
+ "knn1": 0.7718794835007173,
133
+ "knn5": 0.9583931133428981
134
+ },
135
+ "ModernVBERT Bi + DiT Large | concat a=0.6": {
136
+ "clf": 0.8565279770444764,
137
+ "f1": 0.8561551157775931,
138
+ "knn1": 0.8249641319942611,
139
+ "knn5": 0.9383070301291249
140
+ },
141
+ "ModernVBERT Bi + DiT Large | concat a=0.7": {
142
+ "clf": 0.8550932568149211,
143
+ "f1": 0.8544534705350197,
144
+ "knn1": 0.8106169296987088,
145
+ "knn5": 0.9354375896700143
146
+ },
147
+ "ModernVBERT Bi + DiT Large | concat a=0.8": {
148
+ "clf": 0.8493543758967002,
149
+ "f1": 0.8490501017968091,
150
+ "knn1": 0.8134863701578192,
151
+ "knn5": 0.9311334289813487
152
+ },
153
+ "ModernVBERT Bi + DINOv2 Base | concat a=0.5": {
154
+ "clf": 0.8809182209469153,
155
+ "f1": 0.8807322770447767,
156
+ "knn1": 0.8378766140602583,
157
+ "knn5": 0.9540889526542324
158
+ },
159
+ "ModernVBERT Bi + DINOv2 Base | PCA n=512": {
160
+ "clf": 0.8837876614060258,
161
+ "f1": 0.8822042043992898,
162
+ "knn1": 0.793400286944046,
163
+ "knn5": 0.9583931133428981
164
+ },
165
+ "ModernVBERT Bi + DINOv2 Base | concat a=0.6": {
166
+ "clf": 0.8809182209469153,
167
+ "f1": 0.8803378238967797,
168
+ "knn1": 0.830703012912482,
169
+ "knn5": 0.9483500717360115
170
+ },
171
+ "ModernVBERT Bi + DINOv2 Base | concat a=0.7": {
172
+ "clf": 0.8651362984218077,
173
+ "f1": 0.8643561442009781,
174
+ "knn1": 0.8407460545193687,
175
+ "knn5": 0.9440459110473458
176
+ },
177
+ "ModernVBERT Bi + DINOv2 Base | concat a=0.8": {
178
+ "clf": 0.8622668579626973,
179
+ "f1": 0.8616144252149917,
180
+ "knn1": 0.830703012912482,
181
+ "knn5": 0.9411764705882353
182
+ },
183
+ "DiT Large + DINOv2 Base | concat a=0.5": {
184
+ "clf": 0.7733142037302726,
185
+ "f1": 0.763217169706105,
186
+ "knn1": 0.7144906743185079,
187
+ "knn5": 0.9096126255380201
188
+ },
189
+ "DiT Large + DINOv2 Base | PCA n=512": {
190
+ "clf": 0.8206599713055954,
191
+ "f1": 0.8117425399405073,
192
+ "knn1": 0.7288378766140603,
193
+ "knn5": 0.9426111908177905
194
+ },
195
+ "DiT Large + DINOv2 Base | concat a=0.6": {
196
+ "clf": 0.7647058823529411,
197
+ "f1": 0.754804984878742,
198
+ "knn1": 0.7001434720229556,
199
+ "knn5": 0.9081779053084649
200
+ },
201
+ "DiT Large + DINOv2 Base | concat a=0.7": {
202
+ "clf": 0.7532281205164992,
203
+ "f1": 0.738924955000803,
204
+ "knn1": 0.6901004304160688,
205
+ "knn5": 0.921090387374462
206
+ },
207
+ "DiT Large + DINOv2 Base | concat a=0.8": {
208
+ "clf": 0.7101865136298422,
209
+ "f1": 0.6853458025846235,
210
+ "knn1": 0.6571018651362984,
211
+ "knn5": 0.9038737446197992
212
+ }
213
+ },
214
+ "per_family_best": {
215
+ "SigLIP2": {
216
+ "best_clf": {
217
+ "variant": "SigLIP2-SO400M + DiT Large | concat a=0.7",
218
+ "clf": 0.8895265423242468,
219
+ "f1": 0.8884065034233251,
220
+ "knn1": 0.866571018651363,
221
+ "knn5": 0.9626972740315638
222
+ },
223
+ "best_knn": {
224
+ "variant": "SigLIP2-SO400M + DiT Large | concat a=0.5",
225
+ "clf": 0.8794835007173601,
226
+ "f1": 0.8776871268181167,
227
+ "knn1": 0.8766140602582496,
228
+ "knn5": 0.9727403156384505
229
+ }
230
+ },
231
+ "SigLIP": {
232
+ "best_clf": {
233
+ "variant": "SigLIP-SO400M + DINOv2 Base | concat a=0.7",
234
+ "clf": 0.9038737446197992,
235
+ "f1": 0.9014950999730301,
236
+ "knn1": 0.8766140602582496,
237
+ "knn5": 0.9684361549497847
238
+ },
239
+ "best_knn": {
240
+ "variant": "SigLIP-SO400M + DiT Large | concat a=0.6",
241
+ "clf": 0.8995695839311334,
242
+ "f1": 0.8982238782458929,
243
+ "knn1": 0.8794835007173601,
244
+ "knn5": 0.975609756097561
245
+ }
246
+ },
247
+ "ModernVBERT": {
248
+ "best_clf": {
249
+ "variant": "ModernVBERT Bi + DINOv2 Base | PCA n=512",
250
+ "clf": 0.8837876614060258,
251
+ "f1": 0.8822042043992898,
252
+ "knn1": 0.793400286944046,
253
+ "knn5": 0.9583931133428981
254
+ },
255
+ "best_knn": {
256
+ "variant": "ModernVBERT Bi + DiT Large | PCA n=512",
257
+ "clf": 0.8637015781922525,
258
+ "f1": 0.8605465592114369,
259
+ "knn1": 0.7718794835007173,
260
+ "knn5": 0.9583931133428981
261
+ }
262
+ }
263
+ },
264
+ "overall_best": {
265
+ "best_clf": {
266
+ "variant": "SigLIP-SO400M + DINOv2 Base | concat a=0.7",
267
+ "clf": 0.9038737446197992,
268
+ "f1": 0.9014950999730301,
269
+ "knn1": 0.8766140602582496,
270
+ "knn5": 0.9684361549497847
271
+ },
272
+ "best_knn": {
273
+ "variant": "SigLIP-SO400M + DiT Large | concat a=0.6",
274
+ "clf": 0.8995695839311334,
275
+ "f1": 0.8982238782458929,
276
+ "knn1": 0.8794835007173601,
277
+ "knn5": 0.975609756097561
278
+ }
279
+ }
280
+ }