Hugging Face's logo

AbstractPhil
/
geolip-axis-prototype

Model card Files
xet
 Community
geolip-axis-prototype
File size: 9,225 Bytes
dff07c7
 
 
 
da85368
 
 
 
1650c76
 
 
 
da85368
 
 
 
 
 
 
 
 
 
 
 
5f8027e
 
340870a
5f8027e
340870a
5f8027e
340870a
5f8027e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
f9f1c81
 
 
 
 
3b820cd
 
f9f1c81
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3b820cd
 
04ba8f5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3b820cd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
---
license: apache-2.0
---

# Partial Success: Prototype 5.5

Full geodesic alignment possible at a cost of additional dimensions per geodesic complexity grade applied.

I'm thinking the losses have too many changes, and the system is drifting from the directed goals specified in the earlier versions.
There many be too many differences between the losses here, the process here, and the system here - in conjunction with the geolip structural integrity.
This process may be useful for head training, but it may not be useful for distillation training.


```
  Mode                    Dim   Train     Val     Gap
  --------------------------------------------------
  raw_768                1536   0.498   0.357   0.141
  raw+diff               3072   0.567   0.475   0.092
  bank_enriched          1792   0.766   0.532   0.235
  bank+diff              3584   0.722   0.670   0.052
  geo_explicit              6   0.326   0.363  -0.037

```

# prototype 1; 
full anchor bank, minimal anchors, positive outcome from mini distilled expert student

bert-base aligned.

Positive enough to continue refinement.

```
=================================================================
PHASE 3: GEOMETRIC VERIFICATION
=================================================================
  Passthrough integrity: 1.000000 (should be ~1.000)
  Geo context CV: 0.1651
  Geo context eff_dim: 21.5
  Geo context shape: torch.Size([2000, 64])

=================================================================
PHASE 4: CLASSIFIER STABILITY TEST
=================================================================
  with_bank      : train_acc=0.481  val_acc=0.390  gap=0.091
  without_bank   : train_acc=0.443  val_acc=0.330  gap=0.113

=================================================================
DONE
=================================================================

  Student: mini_student.pt
  Bank: alignment_bank.pt
  Consensus CV: 0.1316
  Student v_cos: 0.612
```

# prototype 2; 
full anchor bank self aligned, 512 anchors, reduced training overfit, increased validation on distilled expert student

bert-base aligned.

```
=================================================================
PHASE 3: GEOMETRIC VERIFICATION
=================================================================
  Passthrough:     1.000000 (target: 1.000)
  Emb CV:          0.1592 (consensus: 0.1316)
  Geo context CV:  0.1790
  Geo eff_dim:     33.7 / 128
  Expert cos:      0.794 ± 0.052
  Anchor max cos:  0.656
  Cross-expert:    0.032

=================================================================
PHASE 4: CLASSIFIER STABILITY TEST
=================================================================
  with_bank      : train=0.782  val=0.505  gap=0.277
  without_bank   : train=0.512  val=0.372  gap=0.140

=================================================================
SUMMARY
=================================================================
  Consensus CV:     0.1316
  Consensus eff_dim:223.7
  Student v_cos:    0.612
  Student v_cv:     0.169
  Bank params:      2,918,784
  Bank geo_eff_dim: 33.7
  Bank geo_cv:      0.1790

=================================================================
DONE
=================================================================
```

# prototype 3; 
full anchor bank, 512 anchors, full procrustes whitening, dynamic ortho, full geometric statistics, even more improved accuracy

dynamic alignment, additional utility potential present from dynamic capacity than directly aligning to an expert.

```
=================================================================
PHASE 3: GEOMETRIC VERIFICATION
=================================================================
  Passthrough:     1.000000 (target: 1.000)
  Emb CV:          0.1592 (consensus: 0.1316)
  Geo context CV:  0.1817
  Geo eff_dim:     33.5 / 128
  Expert cos:      0.800 ± 0.004
  Anchor max cos:  0.656
  Cross-expert:    -0.107

=================================================================
PHASE 4: CLASSIFIER STABILITY TEST
=================================================================
  with_bank      : train=0.773  val=0.513  gap=0.260
  without_bank   : train=0.512  val=0.372  gap=0.140

=================================================================
SUMMARY
=================================================================
  Consensus CV:     0.1316
  Consensus eff_dim:223.7
  Student v_cos:    0.612
  Student v_cv:     0.169
  Bank params:      2,920,320
  Bank geo_eff_dim: 33.5
  Bank geo_cv:      0.1817

=================================================================
DONE
=================================================================
```

# prototype 4; 
full anchor bank, 512 anchors, full procrestes whitening, dynamic ortho, improved and robust relative geometric statistics, even more improved accuracy

with full dynamic alignment the system is now cross-expert positive with a dynamic series of ratios for loss and alignment

stability still present, additional complexity can be included.

```
=================================================================
PHASE 3: GEOMETRIC VERIFICATION
=================================================================
  Passthrough:     1.000000 (target: 1.000)
  Emb CV:          0.1557 (consensus: 0.1316)
  Geo context CV:  0.1872
  Geo eff_dim:     33.2 / 128
  Expert cos:      0.784 ± 0.005
  Anchor max cos:  0.655
  Disagreement:
    Cross-expert:  0.0463 ± 0.0154
    Ratio:         0.001912 (target: 0.000000)
    Norm spread:   0.1279

=================================================================
PHASE 4: CLASSIFIER STABILITY TEST
=================================================================
  with_bank      : train=0.775  val=0.508  gap=0.266
  without_bank   : train=0.513  val=0.392  gap=0.122

=================================================================
SUMMARY
=================================================================
  Consensus CV:     0.1316
  Consensus eff_dim:223.7
  Student v_cos:    0.612
  Student v_cv:     0.169
  Bank params:      2,921,088
  Bank geo_eff_dim: 33.2
  Bank geo_cv:      0.1872

=================================================================
DONE
=================================================================
```

# prototype 5

Attempted equidistance geodesic alignment protocol. Successful, perfect harmony to reach 0.2 context CV ratio possible.


```
=================================================================
PHASE 3: GEOMETRIC VERIFICATION
=================================================================
  Passthrough:     1.000000 (target: 1.000)
  Emb CV:          0.1660 (consensus: 0.1771)
  Geo context CV:  0.2053
  Geo eff_dim:     30.5 / 128
  Expert cos:      0.729 ± 0.005
  Anchor max cos:  0.668
  Disagreement:
    Cross-expert:  0.0826 ± 0.0021
    Ratio:         0.006963 (target: 0.000000)
    Norm spread:   0.0781

=================================================================
PHASE 4: CLASSIFIER STABILITY TEST
=================================================================
  with_bank      : train=0.746  val=0.500  gap=0.246
  without_bank   : train=0.490  val=0.363  gap=0.126

=================================================================
SUMMARY
=================================================================
  Consensus CV:     0.1771
  Consensus eff_dim:109.5
  Student v_cos:    0.686
  Student v_cv:     0.182
  Bank params:      2,921,088
  Bank geo_eff_dim: 30.5
  Bank geo_cv:      0.2053

=================================================================
DONE
=================================================================
```

# prototype 5.5 correct geodesic relational contextualization


```
=================================================================
PHASE 3: GEOMETRIC VERIFICATION
=================================================================
  Passthrough:     1.000000 (target: 1.000)
  Emb CV:          0.1635 (consensus: 0.1771)
  Geo context CV:  0.1892
  Geo eff_dim:     30.7 / 128
  Expert cos:      0.725 ± 0.005
  Anchor max cos:  0.668
  Disagreement:
    Cross-expert:  0.0776 ± 0.0023
    Ratio:         0.008188 (target: 0.000000)
    Norm spread:   0.0326

=================================================================
PHASE 4: CLASSIFIER STABILITY TEST
=================================================================

  Mode                    Dim   Train     Val     Gap
  --------------------------------------------------
  raw_768                1536   0.498   0.357   0.141
  raw+diff               3072   0.567   0.475   0.092
  bank_enriched          1792   0.766   0.532   0.235
  bank+diff              3584   0.722   0.670   0.052
  geo_explicit              6   0.326   0.363  -0.037

=================================================================
SUMMARY
=================================================================
  Consensus CV:     0.1771
  Consensus eff_dim:109.5
  Student v_cos:    0.686
  Student v_cv:     0.182
  Bank params:      2,921,088
  Bank geo_eff_dim: 30.7
  Bank geo_cv:      0.1892

=================================================================
DONE
=================================================================
```