AbstractPhil commited on
Commit
150fa15
·
verified ·
1 Parent(s): 78debd3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -128,23 +128,35 @@ SVD information with the attention alignment spectrum.
128
 
129
  # What Needs To Work
130
  **Using MLP will reach fair accuracy and not use CONV or TRANSFORMERS.**
 
131
  I have seen **around 60% on cifar100** with no traditional encoders, but the system was crutching the M_path to fill the gaps after enough epochs of the SVD path.
132
  This structure is under the microscope now.
133
 
134
  Instability allows SGD optimization to heavily benefit some image tasks while it fails completely on text tasks.
135
 
136
  **Out Projection SUVt tokens are iffy**
 
137
  The out projection is an MLP multiscale projection that took a while to set up, and it produces approximate transformer QKV with useful SUVt tokens downstream.
138
 
139
  **Many activations corrupt geometry**
 
140
  They are in there for experimentation. Feel free to experiment.
141
 
142
  **without the expanded triton core spectrum larger systems suffer with triton**
 
143
  Claude code is having trouble with this one as a full task, I'll need to build it in pieces. I've had OpenClaw working on it but the outcome
144
  isn't looking good. The 4x4 and 5x4 won't converge, while the 6x6 crashes the system entirely instead of building it.
145
 
146
  I'll need to wait for a fix for claude code, this is a known issue apparently.
147
 
 
 
 
 
 
 
 
 
148
 
149
  ## Additionally
150
 
 
128
 
129
  # What Needs To Work
130
  **Using MLP will reach fair accuracy and not use CONV or TRANSFORMERS.**
131
+
132
  I have seen **around 60% on cifar100** with no traditional encoders, but the system was crutching the M_path to fill the gaps after enough epochs of the SVD path.
133
  This structure is under the microscope now.
134
 
135
  Instability allows SGD optimization to heavily benefit some image tasks while it fails completely on text tasks.
136
 
137
  **Out Projection SUVt tokens are iffy**
138
+
139
  The out projection is an MLP multiscale projection that took a while to set up, and it produces approximate transformer QKV with useful SUVt tokens downstream.
140
 
141
  **Many activations corrupt geometry**
142
+
143
  They are in there for experimentation. Feel free to experiment.
144
 
145
  **without the expanded triton core spectrum larger systems suffer with triton**
146
+
147
  Claude code is having trouble with this one as a full task, I'll need to build it in pieces. I've had OpenClaw working on it but the outcome
148
  isn't looking good. The 4x4 and 5x4 won't converge, while the 6x6 crashes the system entirely instead of building it.
149
 
150
  I'll need to wait for a fix for claude code, this is a known issue apparently.
151
 
152
+ **Magnitudes are WILDLY hard to control**
153
+
154
+ High magnitudes and complexity associations with those must be controlled. The vocabulary spectrum of a token is widely diverse, so the noise generated
155
+ from large and small magnitudes is very difficult to curate for. I'm debugging this task as I progress, and the dominant shape learning corruption is
156
+ magnitude differentiation currently.
157
+
158
+ I have some potentials and this is my current direction.
159
+
160
 
161
  ## Additionally
162