AbstractPhil commited on
Commit
5f8027e
·
verified ·
1 Parent(s): 340870a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +150 -4
README.md CHANGED
@@ -2,10 +2,156 @@
2
  license: apache-2.0
3
  ---
4
 
5
- prototype 1; full anchor bank, minimal anchors, positive outcome from mini distilled expert student
 
6
 
7
- prototype 2; full anchor bank, 512 anchors, reduced training overfit, increased validation on distilled expert student
8
 
9
- prototype 3; full anchor bank, 512 anchors, full procrustes whitening, dynamic ortho, full geometric statistics, even more improved accuracy
10
 
11
- prototype 4; full anchor bank, 512 anchors, full procrestes whitening, dynamic ortho, improved and robust relative geometric statistics, even more improved accuracy
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  ---
4
 
5
+ # prototype 1;
6
+ full anchor bank, minimal anchors, positive outcome from mini distilled expert student
7
 
8
+ bert-base aligned.
9
 
10
+ Positive enough to continue refinement.
11
 
12
+ ```
13
+ =================================================================
14
+ PHASE 3: GEOMETRIC VERIFICATION
15
+ =================================================================
16
+ Passthrough integrity: 1.000000 (should be ~1.000)
17
+ Geo context CV: 0.1651
18
+ Geo context eff_dim: 21.5
19
+ Geo context shape: torch.Size([2000, 64])
20
+
21
+ =================================================================
22
+ PHASE 4: CLASSIFIER STABILITY TEST
23
+ =================================================================
24
+ with_bank : train_acc=0.481 val_acc=0.390 gap=0.091
25
+ without_bank : train_acc=0.443 val_acc=0.330 gap=0.113
26
+
27
+ =================================================================
28
+ DONE
29
+ =================================================================
30
+
31
+ Student: mini_student.pt
32
+ Bank: alignment_bank.pt
33
+ Consensus CV: 0.1316
34
+ Student v_cos: 0.612
35
+ ```
36
+
37
+ # prototype 2;
38
+ full anchor bank self aligned, 512 anchors, reduced training overfit, increased validation on distilled expert student
39
+
40
+ bert-base aligned.
41
+
42
+ ```
43
+ =================================================================
44
+ PHASE 3: GEOMETRIC VERIFICATION
45
+ =================================================================
46
+ Passthrough: 1.000000 (target: 1.000)
47
+ Emb CV: 0.1592 (consensus: 0.1316)
48
+ Geo context CV: 0.1790
49
+ Geo eff_dim: 33.7 / 128
50
+ Expert cos: 0.794 ± 0.052
51
+ Anchor max cos: 0.656
52
+ Cross-expert: 0.032
53
+
54
+ =================================================================
55
+ PHASE 4: CLASSIFIER STABILITY TEST
56
+ =================================================================
57
+ with_bank : train=0.782 val=0.505 gap=0.277
58
+ without_bank : train=0.512 val=0.372 gap=0.140
59
+
60
+ =================================================================
61
+ SUMMARY
62
+ =================================================================
63
+ Consensus CV: 0.1316
64
+ Consensus eff_dim:223.7
65
+ Student v_cos: 0.612
66
+ Student v_cv: 0.169
67
+ Bank params: 2,918,784
68
+ Bank geo_eff_dim: 33.7
69
+ Bank geo_cv: 0.1790
70
+
71
+ =================================================================
72
+ DONE
73
+ =================================================================
74
+ ```
75
+
76
+ # prototype 3;
77
+ full anchor bank, 512 anchors, full procrustes whitening, dynamic ortho, full geometric statistics, even more improved accuracy
78
+
79
+ dynamic alignment, additional utility potential present from dynamic capacity than directly aligning to an expert.
80
+
81
+ ```
82
+ =================================================================
83
+ PHASE 3: GEOMETRIC VERIFICATION
84
+ =================================================================
85
+ Passthrough: 1.000000 (target: 1.000)
86
+ Emb CV: 0.1592 (consensus: 0.1316)
87
+ Geo context CV: 0.1817
88
+ Geo eff_dim: 33.5 / 128
89
+ Expert cos: 0.800 ± 0.004
90
+ Anchor max cos: 0.656
91
+ Cross-expert: -0.107
92
+
93
+ =================================================================
94
+ PHASE 4: CLASSIFIER STABILITY TEST
95
+ =================================================================
96
+ with_bank : train=0.773 val=0.513 gap=0.260
97
+ without_bank : train=0.512 val=0.372 gap=0.140
98
+
99
+ =================================================================
100
+ SUMMARY
101
+ =================================================================
102
+ Consensus CV: 0.1316
103
+ Consensus eff_dim:223.7
104
+ Student v_cos: 0.612
105
+ Student v_cv: 0.169
106
+ Bank params: 2,920,320
107
+ Bank geo_eff_dim: 33.5
108
+ Bank geo_cv: 0.1817
109
+
110
+ =================================================================
111
+ DONE
112
+ =================================================================
113
+ ```
114
+
115
+ # prototype 4;
116
+ full anchor bank, 512 anchors, full procrestes whitening, dynamic ortho, improved and robust relative geometric statistics, even more improved accuracy
117
+
118
+ with full dynamic alignment the system is now cross-expert positive with a dynamic series of ratios for loss and alignment
119
+
120
+ stability still present, additional complexity can be included.
121
+
122
+ ```
123
+ =================================================================
124
+ PHASE 3: GEOMETRIC VERIFICATION
125
+ =================================================================
126
+ Passthrough: 1.000000 (target: 1.000)
127
+ Emb CV: 0.1557 (consensus: 0.1316)
128
+ Geo context CV: 0.1872
129
+ Geo eff_dim: 33.2 / 128
130
+ Expert cos: 0.784 �� 0.005
131
+ Anchor max cos: 0.655
132
+ Disagreement:
133
+ Cross-expert: 0.0463 ± 0.0154
134
+ Ratio: 0.001912 (target: 0.000000)
135
+ Norm spread: 0.1279
136
+
137
+ =================================================================
138
+ PHASE 4: CLASSIFIER STABILITY TEST
139
+ =================================================================
140
+ with_bank : train=0.775 val=0.508 gap=0.266
141
+ without_bank : train=0.513 val=0.392 gap=0.122
142
+
143
+ =================================================================
144
+ SUMMARY
145
+ =================================================================
146
+ Consensus CV: 0.1316
147
+ Consensus eff_dim:223.7
148
+ Student v_cos: 0.612
149
+ Student v_cv: 0.169
150
+ Bank params: 2,921,088
151
+ Bank geo_eff_dim: 33.2
152
+ Bank geo_cv: 0.1872
153
+
154
+ =================================================================
155
+ DONE
156
+ =================================================================
157
+ ```