Merge branch 'main' of https://huggingface.co/Crosstyan/BPModel

README.md

@@ -27,10 +27,28 @@ BPModel is an experimental Stable Diffusion model based on [ACertainty](https://
 Why is the Model even existing? There are loads of Stable Diffusion model out there, especially anime style models.
 Well, is there any models trained with resolution base resolution (`base_res`) 768 even 1024 before? Don't think so.
 Here it is, the BPModel, a Stable Diffusion model you may love or hate.
-Trained with 5k high quality images that suit my taste (not necessary yours unfortunately) from [Sankaku Complex](https://chan.sankakucomplex.com) with annotations. Not the best strategy since pure combination of tags may not be the optimal way to describe the image, but I don't need to do extra work. And no, I won't feed any AI generated image
-to the model even it might outlaw the model from being used in some countries. 
-
-The training of a high resolution model requires a significant amount of GPU hours and can be costly. In this particular case, 10 V100 GPU hours were spent on training a model with a resolution of 512, while 60 V100 GPU hours were spent on training a model with a resolution of 768. An additional 50 V100 GPU hours were also spent on training a model with a resolution of 1024, although only 10 epochs were run. The results of the training on the 1024 resolution model did not show a significant improvement compared to the 768 resolution model, and the resource demands, achieving a batch size of 1 on a V100 with 32G VRAM, were high. However, training on the 768 resolution did yield better results than training on the 512 resolution, and it is worth considering as an option. It is worth noting that Stable Diffusion 2.x also chose to train on a 768 resolution model. However, it may be more efficient to start with training on a 512 resolution model due to the slower training process and the need for additional prior knowledge to speed up the training process when working with a 768 resolution.
+Trained with 5k high quality images that suit my taste (not necessary yours unfortunately) from [Sankaku Complex](https://chan.sankakucomplex.com) with annotations. 
+The dataset is public in [Crosstyan/BPDataset](https://huggingface.co/datasets/Crosstyan/BPDataset) for the sake of full disclosure .
+Pure combination of tags may not be the optimal way to describe the image, 
+but I don't need to do extra work. 
+And no, I won't feed any AI generated image
+to the model even it might outlaw the model from being used in some countries.
+
+The training of a high resolution model requires a significant amount of GPU
+hours and can be costly. In this particular case, 10 V100 GPU hours were spent
+on training 30 epochs with a resolution of 512, while 60 V100 GPU hours were spent
+on training 30 epochs with a resolution of 768. An additional 100 V100 GPU hours
+were also spent on training a model with a resolution of 1024, although **ONLY** 10
+epochs were run. The results of the training on the 1024 resolution model did
+not show a significant improvement compared to the 768 resolution model, and the
+resource demands, achieving a batch size of 1 on a V100 with 32G VRAM, were
+high. However, training on the 768 resolution did yield better results than
+training on the 512 resolution, and it is worth considering as an option. It is
+worth noting that Stable Diffusion 2.x also chose to train on a 768 resolution
+model. However, it may be more efficient to start with training on a 512
+resolution model due to the slower training process and the need for additional
+prior knowledge to speed up the training process when working with a 768
+resolution.
 
 [Mikubill/naifu-diffusion](https://github.com/Mikubill/naifu-diffusion) is used as training script and I also recommend to
 checkout [CCRcmcpe/scal-sdt](https://github.com/CCRcmcpe/scal-sdt).
@@ -85,7 +103,7 @@ better than some artist style DreamBooth model which only train with a few
 hundred images or even less. I also oppose changing style by merging model since You
 could apply different style by training with proper captions and prompting.
 
-Besides some of images in my dataset has the artist name in the caption, however some artist name will
+Besides some of images in my dataset have the artist name in the caption, however some artist name will
 be misinterpreted by CLIP when tokenizing. For example, *as109* will be tokenized as `[as, 1, 0, 9]` and
 *fuzichoco* will become `[fu, z, ic, hoco]`. Romanized Japanese suffers from the problem a lot and
 I don't have a good solution to fix it other than changing the artist name in the caption, which is
@@ -101,6 +119,10 @@ I don't think anyone would like to do. (Could Unstable Diffusion give us surpris
 
 Here're some **cherry picked** samples.
 
+I were using [xformers](https://github.com/facebookresearch/xformers) when generating these sample 
+and it might yield slight different result even with the same seed (welcome to the non deterministic field).
+"`Upscale latent space image when doing hires. fix`" is enabled also.
+
 ![orange](images/00317-2017390109_20221220015645.png)
 
 ```txt
@@ -154,7 +176,9 @@ EMA weight is not included and it's fp16.
 If you want to continue training, use [`bp_1024_e10_ema.ckpt`](bp_1024_e10_ema.ckpt) which is the ema unet weight 
 and with fp32 precision.
 
-For better performance, it is strongly recommended to use Clip skip (CLIP stop at last layers) 2.
+For better performance, it is strongly recommended to use Clip skip (CLIP stop at last layers) 2. It's also recommended to use turn on
+"`Upscale latent space image when doing hires. fix`" in the settings of [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
+which adds intricate details when using `Highres. fix`. 
 
 ## About the Model Name