Fr commited on
Commit
150b9c1
·
verified ·
1 Parent(s): d7ae0d3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -4
README.md CHANGED
@@ -12,8 +12,6 @@ pipeline_tag: text-generation
12
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63cb46191b705cc951e88e6c/2LZWEcQcoLIT4Fm1SHMCB.png)
13
 
14
 
15
- I was initially going to upload a checkpoint that had been trained with 5,000 steps, but I made a mistake and had to train from scratch again, and I could only
16
- save the 2,000-step checkpoint. As soon as I have access to an NVIDIA A100 again, I'm going to train a competent 100M model on a
17
- larger scale and with more data, and then a 1B model.
18
 
19
- It was only trained on 40960 tokens, and it is only to validate that the model implementation and training is correct.
 
12
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63cb46191b705cc951e88e6c/2LZWEcQcoLIT4Fm1SHMCB.png)
13
 
14
 
15
+ # DISCLAIMER: The uploaded model was a faulty model that was not uploaded in the correct format for use with `transformers`. It has been removed, and a new model will be retrained in the correct format.
 
 
16
 
17
+ ### For those who want to train and get the correct format to be able to load it with `transformers`, everything needed is in [`pre_train.py`](https://github.com/F4k3r22/LLaDA-from-scratch/blob/main/pre_train.py) of the project repo