Update README.md
Browse files
README.md
CHANGED
|
@@ -12,7 +12,7 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 12 |
|
| 13 |
Tried depth-upscaling Cosmo-1b by duplicating 6 layers, then LISA-training on a dataset reasonably similar to the original one in an attempt to 'self-repair'.
|
| 14 |
|
| 15 |
-
Not sure if it worked out exactly how I pictured but the nous eval's not overall worse than the original at least.
|
| 16 |
|
| 17 |
(Took I think about 8 hours for, I want to say, ~80 million tokens on one RTX 3090?)
|
| 18 |
|
|
|
|
| 12 |
|
| 13 |
Tried depth-upscaling Cosmo-1b by duplicating 6 layers, then LISA-training on a dataset reasonably similar to the original one in an attempt to 'self-repair'.
|
| 14 |
|
| 15 |
+
Not sure if it worked out exactly how I pictured but the nous eval's not overall much worse than the original at least.
|
| 16 |
|
| 17 |
(Took I think about 8 hours for, I want to say, ~80 million tokens on one RTX 3090?)
|
| 18 |
|