update readme with v5.1 model and laion training update
Browse files
README.md
CHANGED
|
@@ -4,6 +4,20 @@ license: creativeml-openrail-m
|
|
| 4 |
|
| 5 |
https://huggingface.co/spaces/CompVis/stable-diffusion-license
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
This is a finetuning of the compvis stable diffusion 1.4 ckpt. https://huggingface.co/CompVis/stable-diffusion
|
| 8 |
|
| 9 |
As an extension to the concept of "dreambooth" training, this fine tuning includes over a dozen concepts trained in over 1400 images with individual captions on each image.
|
|
|
|
| 4 |
|
| 5 |
https://huggingface.co/spaces/CompVis/stable-diffusion-license
|
| 6 |
|
| 7 |
+
# New v5.1 model
|
| 8 |
+
|
| 9 |
+
The new version is trained from a basis of the RunwayML 1.5 ckpt. This fine tuning sheds the last remnant of the concepts in original DreamBooth paper as regularization via generated images is dropped in favor of a mix a scrape of laion to protect the model's original qualities instead. 1636 training images, 1636 ground truth images from laion were trained for 19009 steps at LR 4e-7.
|
| 10 |
+
|
| 11 |
+
Results here (warning, huge image files)
|
| 12 |
+
|
| 13 |
+
[general model test](mega_test01.webp)
|
| 14 |
+
|
| 15 |
+
[new characters test](mega_test01_characters.webp)
|
| 16 |
+
|
| 17 |
+
There is some remaining impact to cartoon character, but there is little "bleed" of the video game context into non-video game subjects. There are also a number of images that show improved cropping behavior even from the base Runway 1.5 file, which I attribute to careful cropping of both training and the ground truth images scraped from laion.
|
| 18 |
+
|
| 19 |
+
# Prior info on 4.1 model
|
| 20 |
+
|
| 21 |
This is a finetuning of the compvis stable diffusion 1.4 ckpt. https://huggingface.co/CompVis/stable-diffusion
|
| 22 |
|
| 23 |
As an extension to the concept of "dreambooth" training, this fine tuning includes over a dozen concepts trained in over 1400 images with individual captions on each image.
|