Update README.md
Browse files
README.md
CHANGED
|
@@ -15,3 +15,5 @@ pipeline_tag: text-generation
|
|
| 15 |
I was initially going to upload a checkpoint that had been trained with 5,000 steps, but I made a mistake and had to train from scratch again, and I could only
|
| 16 |
save the 2,000-step checkpoint. As soon as I have access to an NVIDIA A100 again, I'm going to train a competent 100M model on a
|
| 17 |
larger scale and with more data, and then a 1B model.
|
|
|
|
|
|
|
|
|
| 15 |
I was initially going to upload a checkpoint that had been trained with 5,000 steps, but I made a mistake and had to train from scratch again, and I could only
|
| 16 |
save the 2,000-step checkpoint. As soon as I have access to an NVIDIA A100 again, I'm going to train a competent 100M model on a
|
| 17 |
larger scale and with more data, and then a 1B model.
|
| 18 |
+
|
| 19 |
+
It was only trained on 40960 tokens, and it is only to validate that the model implementation and training is correct.
|