AbstractPhil commited on
Commit
be11bf0
·
verified ·
1 Parent(s): 0eee32a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -0
README.md CHANGED
@@ -18,6 +18,46 @@ Until now this has proven valuable, and now that it's become a hinderance a new
18
  # Experiment 1: Retune AdamW Directly
19
  I'll attempt to tweak AdamW specifically to not destroy the geometric shape, disabling weight_decay from this point onward.
20
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  ---------------------------------------------------------------------------------------------------
23
  # Large Experimental Conclusion 1:
 
18
  # Experiment 1: Retune AdamW Directly
19
  I'll attempt to tweak AdamW specifically to not destroy the geometric shape, disabling weight_decay from this point onward.
20
 
21
+ The outcomes have shown there isn't much beyond tuning specifics to tweak this particular classifier. Introducing more anchors helps, and with that the dims can be reduced.
22
+ Essentially the anchors are capacity tuning forks in this variation rather than utilities, which is fine. We can allocate them to a student.
23
+
24
+ ```
25
+ =================================================================
26
+ SWEEP RESULTS
27
+ =================================================================
28
+
29
+ Config v_acc t_acc gap cv Δcv eq_std poly curve star struct
30
+ ------------------------------------------------------------------------------------------
31
+ raw_adam 0.617 0.681 +0.064 1.3917 +1.1917 0.4075 0.39 0.75 0.86 0.61
32
+ proven 0.722 0.706 -0.016 1.3629 +1.1629 0.4157 0.45 0.99 0.93 0.71
33
+ +spread 0.669 0.686 +0.017 1.4491 +1.2491 0.4212 0.41 0.98 0.71 0.72
34
+ +entropy 0.674 0.711 +0.037 1.4945 +1.2945 0.4237 0.42 0.97 0.70 0.74
35
+ +ortho 0.695 0.690 -0.005 1.3454 +1.1454 0.4171 0.40 0.99 0.85 0.72
36
+ +cluster 0.701 0.717 +0.016 1.3034 +1.1034 0.4131 0.44 0.93 0.91 0.70
37
+ +drift 0.709 0.698 -0.012 1.3480 +1.1480 0.4134 0.41 1.00 0.91 0.72
38
+ +spr+ort 0.723 0.698 -0.025 1.3881 +1.1881 0.4224 0.46 0.97 0.94 0.71
39
+ +all_micro 0.694 0.700 +0.007 1.5181 +1.3181 0.4077 0.40 0.97 0.85 0.72
40
+
41
+ Best accuracy: +spr+ort (val_acc=0.723)
42
+ Best structure: +entropy (struct=0.737)
43
+ Closest to CV=0.2: +cluster (cv=1.3034, Δ=+1.1034)
44
+ Most equidistant: raw_adam (equi_std=0.4075)
45
+ Most stable CV: raw_adam (cv_std=0.1734)
46
+
47
+ =================================================================
48
+ DONE
49
+ =================================================================
50
+ ```
51
+ The outcomes show that we can definitely impact the outcome and the deviation of the system will conform to an entirely new spectrum of CV currently unoccupied.
52
+
53
+ There's a lot to unpack here, and I think the biggest most critical piece to unpack is a hyperparameter about controlling where on the latent spectrum
54
+ you want your model's continuum to exist within.
55
+
56
+ # Experiment 2: Teacher/Student hierarchy
57
+ I've ran plenty of genetic experiments, a single student anchored from a teacher should provide a more robust sweep
58
+
59
+
60
+
61
 
62
  ---------------------------------------------------------------------------------------------------
63
  # Large Experimental Conclusion 1: