Update README.md

README.md

@@ -41,6 +41,7 @@ This model is still in its infancy and it's meant to be constantly updated and t
   - #### Stable:
     - [vae.sygil_muse_v0.1.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/vae.sygil_muse_v0.1.pt): Trained from scratch for 3.0M steps with **dim: 128** and **vq_codebook_size: 256**.
     - [maskgit.sygil_muse_v0.1.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/maskgit.sygil_muse_v0.1.pt): Maskgit trained from the VAE for 3.46M steps
+    - [vae.sygil_muse_v0.5.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/vae.sygil_muse_v0.5.pt): Trained from scratch for 1.99M steps with **dim: 128** and **vq_codebook_size: 8192**.
   - #### Beta:
     - [vae.1999500.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/vae.1999500.pt): Trained from scratch for 1.99M steps and higher **vq_codebook_size** than before.
     - [maskgit.39000.pt](https://huggingface.co/Sygil/Sygil-Muse/blob/main/maskgit.39000.pt): Maskgit trained from the VAE for 39K steps using the hyperparameters `heads 16` and `depth 22` for testing, these values have huge performance effects, the vram usage was also increased so it is just for testing, the quality on this checkpoint did increase a lot and requires a lot less training which is something we want but we need to find a balance between quality and performance.
@@ -57,18 +58,21 @@ The model was trained on the following dataset:
 **Hardware and others**
 - **Hardware:** 1 x Nvidia RTX 3050 GPU
 - **Hours Trained:** NaN.
-- **Gradient Accumulations**: 5
+- **Gradient Accumulations**: 10
 - **Batch:** 1
 - **Learning Rate:** 1e-4
-- **Learning Rate Scheduler:** `constant_with_warmup`
+- **Learning Rate Scheduler:** `cosine_with_restarts`
+- **Scheduler Power:** 0.5
 - **Optimizer:** Adam
 - **Warmup Steps:** 10,000
-- **Number of Cycles:** 100
+- **Number of Cycles:** 10,000
 - **Resolution/Image Size**: First trained at a resolution of 64x64, then increased to 256x256 and then to 512x512. Check the notes down below for more details on this.
 - **Dimension:** 128
+- **vq_codebook_dim:** 4096
 - **vq_codebook_size:** 8192
 - **heads:** 8
 - **depth:** 4
+- **Random Crop:** True
 - **Total Training Steps:** 1,999,500
 
   Note: On Muse we can change the image_size or resolution at any time without having to train the model from scratch again, this allows us to first train the model at low resolution using the same `dim` and `vq_codebook_size` to train faster and then we can increase the `image_size` and use a higher resolution once the model has trained enough.