Lambent commited on
Commit
5e40d34
·
verified ·
1 Parent(s): fc320b0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -12,7 +12,7 @@ should probably proofread and complete it, then remove this comment. -->
12
 
13
  Tried depth-upscaling Cosmo-1b by duplicating 6 layers, then LISA-training on a dataset reasonably similar to the original one in an attempt to 'self-repair'.
14
 
15
- Not sure if it worked out exactly how I pictured but the nous eval's not overall worse than the original at least.
16
 
17
  (Took I think about 8 hours for, I want to say, ~80 million tokens on one RTX 3090?)
18
 
 
12
 
13
  Tried depth-upscaling Cosmo-1b by duplicating 6 layers, then LISA-training on a dataset reasonably similar to the original one in an attempt to 'self-repair'.
14
 
15
+ Not sure if it worked out exactly how I pictured but the nous eval's not overall much worse than the original at least.
16
 
17
  (Took I think about 8 hours for, I want to say, ~80 million tokens on one RTX 3090?)
18