Update README.md
Browse files
README.md
CHANGED
|
@@ -34,6 +34,13 @@ enough to say the NLI head works yet.
|
|
| 34 |
|
| 35 |
HOWEVER, it's enough to say that it can with more training. This is enough to continue for me.
|
| 36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
# OKAY
|
| 39 |
|
|
|
|
| 34 |
|
| 35 |
HOWEVER, it's enough to say that it can with more training. This is enough to continue for me.
|
| 36 |
|
| 37 |
+
If I were to say unlock the model's weights and train ALL FIVE EXPERTS, this would be an arbitrary task. The system would learn it instantly.
|
| 38 |
+
|
| 39 |
+
However, this is an attempt to train WITHOUT the experts, as they are a large burden on time and effort. I need to test the system's capacity
|
| 40 |
+
to handle training it's own heads, without the experts forcing their geometric structure into the mix.
|
| 41 |
+
|
| 42 |
+
The geometric alignment helps, but it's not enough yet. It requires more.
|
| 43 |
+
|
| 44 |
|
| 45 |
# OKAY
|
| 46 |
|