something-else
/

rocm-rwkv

Model card Files Files and versions

something-else commited on Jan 16, 2024

Commit

73cf444

·

verified ·

1 Parent(s): 7dcd573

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -9,4 +9,5 @@ tags:
 ---
 3B rocm-rwkv pth record.
 - rwkv-final-chnk5.pth: 3B rocm-rwkv model trained with Slim pajama chunk1-5 and with a loss of 2.456.
-- rwkv-final-chnk17.pth: 3B rocm-rwkv model trained with Slim pajama chunk1-10 for the first epoch and an aditional training with chunk1-7 after the first epoch  and with a loss of 2.281

 ---
 3B rocm-rwkv pth record.
 - rwkv-final-chnk5.pth: 3B rocm-rwkv model trained with Slim pajama chunk1-5 and with a loss of 2.456.
+- rwkv-final-chnk17.pth: 3B rocm-rwkv model trained with Slim pajama chunk1-10 for the first epoch and an aditional training with chunk1-7 after the first epoch  and with a loss of 2.281
+- rwkv-code39-16012024.pth:  3B rocm-rwkv model trained with Slim pajama chunk1-10 for the first epoch and an aditional training with chunk1-8 after the first epoch; plus a little bit of code. This pth has a loss of 1.174.