Update README.md

README.md

@@ -22,7 +22,7 @@ tags:
 
 <!-- Provide a quick summary of what the model is/does. -->
 
-This model is based in [Muse](https://muse-model.github.io/) and trained using [`lucidrains/muse-maskgit-pytorch`](https://github.com/lucidrains/muse-maskgit-pytorch).
+This model is based in [Muse](https://muse-model.github.io/) and trained using the code hosted on [ZeroCool940711/muse-maskgit-pytorch](https://github.com/ZeroCool940711/muse-maskgit-pytorch), which is based on [`lucidrains/muse-maskgit-pytorch`](https://github.com/lucidrains/muse-maskgit-pytorch).
 
 # Model Details
 This model is a new model trained from scratch based on [Muse](https://muse-model.github.io/), trained on the [Imaginary Network Expanded Dataset](https://github.com/Sygil-Dev/INE-dataset), with the big advantage of allowing the use of multiple namespaces (labeled tags) to control various parts of the final generation.
@@ -40,7 +40,7 @@ This model is still in its infancy and it's meant to be constantly updated and t
   - #### Stable:
     - [vae.sygil_muse_v0.1.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/vae.sygil_muse_v0.1.pt): Trained from scratch for 3.0M steps with **dim: 128** and **vq_codebook_size: 256**.
   - #### Beta:
-    - [vae.442000.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/vae.442000.pt): Trained from scratch for 442K steps and higher **vq_codebook_size** than before.
+    - [vae.470500.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/vae.470500.pt): Trained from scratch for 470K steps and higher **vq_codebook_size** than before.
     - [maskgit.3461001.p](https://huggingface.co/Sygil/Sygil-Muse/blob/main/maskgit.3461001.pt): Maskgit trained from the VAE for 3.46M steps
   
   Note: Checkpoints under the Beta section are updated daily or at least 3-4 times a week. While the beta checkpoints can be used as they are only the latest version is kept on the repo and the older checkpoints are removed when a new one
@@ -57,15 +57,16 @@ The model was trained on the following dataset:
 - **Hours Trained:** NaN.
 - **Gradient Accumulations**: 1
 - **Batch:** 1
-- **Learning Rate:** 7e-04
+- **Learning Rate:** 1e-04
 - **Learning Rate Scheduler:** `cosine_with_restarts`
-- **Optimizer:** Lion
-- **Weight Decay:** 1e-2
+- **Optimizer:** Adam
+- **Weight Decay:** 1e-4
 - **Warmup Steps:** 10,000
+- **Number of Cycles:** 100
 - **Resolution/Image Size**: First trained at a resolution of 64x64, then increased to 256x256 and then to 512x512. Check the notes down below for more details on this.
 - **Dimension:** 128
 - **vq_codebook_size:** 8192
-- **Total Training Steps:** 442,000
+- **Total Training Steps:** 470,500
 
   Note: On Muse we can change the image_size or resolution at any time without having to train the model from scratch again, this allows us to first train the model at low resolution using the same `dim` and `vq_codebook_size` to train faster and then we can increase the `image_size` and use a higher resolution once the model has trained enough.