philipp.haslbauer
commited on
Commit
·
19c40ce
1
Parent(s):
389a869
add current model
Browse files- .gitattributes +10 -0
- .gitignore +1 -0
- README.md +72 -3
- models/ukiyoe-all/v1/ema_0.9999_056000.pt +3 -0
- models/ukiyoe-all/v1/images/secondary_off_0.png +3 -0
- models/ukiyoe-all/v1/images/secondary_off_1.png +3 -0
- models/ukiyoe-all/v1/images/secondary_off_2.png +3 -0
- models/ukiyoe-all/v1/images/secondary_off_3.png +3 -0
- models/ukiyoe-all/v1/images/secondary_on_0.png +3 -0
- models/ukiyoe-all/v1/images/secondary_on_1.png +3 -0
- models/ukiyoe-all/v1/images/secondary_on_2.png +3 -0
- ukiyoe_diffusion_256_010000.pt → models/ukiyoe-few/v1/ukiyoe_diffusion_256_010000.pt +0 -0
- ukiyoe_diffusion_256_022000.pt → models/ukiyoe-few/v1/ukiyoe_diffusion_256_022000.pt +0 -0
.gitattributes
CHANGED
|
@@ -26,3 +26,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 26 |
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
| 27 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
ukiyoe-diffusion-256.pt filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
| 27 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
ukiyoe-diffusion-256.pt filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
models/ukiyoe-all/v1/ema_0.9999_056000.pt filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
models/ukiyoe-all/v1/images/secondary_off_3.png filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
models/ukiyoe-all/v1/images/secondary_on_0.png filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
models/ukiyoe-all/v1/images/secondary_on_1.png filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
models/ukiyoe-all/v1/images/secondary_on_2.png filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
models/ukiyoe-all/v1/images/secondary_off_0.png filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
models/ukiyoe-all/v1/images/secondary_off_1.png filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
models/ukiyoe-all/v1/images/secondary_off_2.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
models/ukiyoe-few/v1/ukiyoe_diffusion_256_010000.pt filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
models/ukiyoe-few/v1/ukiyoe_diffusion_256_022000.pt filter=lfs diff=lfs merge=lfs -text
|
.gitignore
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
**/.DS_Store
|
README.md
CHANGED
|
@@ -9,9 +9,78 @@ datasets:
|
|
| 9 |
---
|
| 10 |
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
Finetuned on 5224 images from Wikiart (1168) and ? ().
|
| 17 |
|
|
@@ -36,5 +105,5 @@ model_config = {
|
|
| 36 |
}
|
| 37 |
```
|
| 38 |
|
| 39 |
-
Trained using a fork of [guided-diffusion-sxela](https://github.com/thegenerativegeneration/guided-diffusion-sxela).
|
| 40 |
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
|
| 12 |
+
# Ukiyo-e Diffusion
|
| 13 |
|
| 14 |
+
If you make something using these models, you're welcome to mention me [@thegenerativegeneration](https://www.instagram.com/thegenerativegeneration/)
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
Named by dataset used. Current and best version is [models/ukiyoe-all/v1/ema_0.9999_056000.pt](models/ukiyoe-all/v1/ema_0.9999_056000.pt)
|
| 18 |
+
|
| 19 |
+
## Current Plans
|
| 20 |
+
|
| 21 |
+
* clean dataset
|
| 22 |
+
* remove borders
|
| 23 |
+
* remove some of the samples with text in them
|
| 24 |
+
|
| 25 |
+
## Models
|
| 26 |
+
|
| 27 |
+
### Ukiyoe-all
|
| 28 |
+
|
| 29 |
+
#### v1
|
| 30 |
+
|
| 31 |
+
[models/ukiyoe-all/v1/ema_0.9999_056000.pt](models/ukiyoe-all/v1/ema_0.9999_056000.pt)
|
| 32 |
+
|
| 33 |
+
Model configuration is:
|
| 34 |
+
|
| 35 |
+
```python
|
| 36 |
+
model_config = {
|
| 37 |
+
'attention_resolutions': '32, 16, 8',
|
| 38 |
+
'class_cond': False,
|
| 39 |
+
'image_size': 256,
|
| 40 |
+
'learn_sigma': True,
|
| 41 |
+
'rescale_timesteps': True,
|
| 42 |
+
'noise_schedule': 'linear',
|
| 43 |
+
'num_channels': 128,
|
| 44 |
+
'num_heads': 4,
|
| 45 |
+
'num_res_blocks': 2,
|
| 46 |
+
'resblock_updown': True,
|
| 47 |
+
'use_checkpoint': True,
|
| 48 |
+
'use_fp16': True,
|
| 49 |
+
'use_scale_shift_norm': True,
|
| 50 |
+
}
|
| 51 |
+
```
|
| 52 |
+
##### Tips
|
| 53 |
+
|
| 54 |
+
- Results closest to original training data are achieved by turning off the secondary model in Disco Diffusion.
|
| 55 |
+
- Turning secondary model on can lead to very creative results
|
| 56 |
+
- It is not necessary to specify Ukiyo-e as artstyle to get ukiyo-e-like images.
|
| 57 |
+
|
| 58 |
+
##### Examples
|
| 59 |
+
|
| 60 |
+
If you make something nice using these models, I would like to link your image.
|
| 61 |
+
|
| 62 |
+
###### Secondary Off
|
| 63 |
+
|
| 64 |
+

|
| 65 |
+

|
| 66 |
+

|
| 67 |
+

|
| 68 |
+
|
| 69 |
+
###### Secondary On
|
| 70 |
+
|
| 71 |
+

|
| 72 |
+

|
| 73 |
+

|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
##### About
|
| 77 |
+
Trained from scratch on a ~170000 images corpus of [ukiyo-e.org](https://ukiyo-e.org) filtered by [colorfulness](https://pyimagesearch.com/2017/06/05/computing-image-colorfulness-with-opencv-and-python/
|
| 78 |
+
) >= 5.
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
### (Deprecated) Ukiyoe-few
|
| 84 |
|
| 85 |
Finetuned on 5224 images from Wikiart (1168) and ? ().
|
| 86 |
|
|
|
|
| 105 |
}
|
| 106 |
```
|
| 107 |
|
| 108 |
+
Trained using a fork of [guided-diffusion-sxela](https://github.com/thegenerativegeneration/guided-diffusion-sxela). Added random crop which did not lead to good results.
|
| 109 |
|
models/ukiyoe-all/v1/ema_0.9999_056000.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5f9af55461783619f25dcbf8953f028c0768108801860afe8c947428dee7a991
|
| 3 |
+
size 553273895
|
models/ukiyoe-all/v1/images/secondary_off_0.png
ADDED
|
Git LFS Details
|
models/ukiyoe-all/v1/images/secondary_off_1.png
ADDED
|
Git LFS Details
|
models/ukiyoe-all/v1/images/secondary_off_2.png
ADDED
|
Git LFS Details
|
models/ukiyoe-all/v1/images/secondary_off_3.png
ADDED
|
Git LFS Details
|
models/ukiyoe-all/v1/images/secondary_on_0.png
ADDED
|
Git LFS Details
|
models/ukiyoe-all/v1/images/secondary_on_1.png
ADDED
|
Git LFS Details
|
models/ukiyoe-all/v1/images/secondary_on_2.png
ADDED
|
Git LFS Details
|
ukiyoe_diffusion_256_010000.pt → models/ukiyoe-few/v1/ukiyoe_diffusion_256_010000.pt
RENAMED
|
File without changes
|
ukiyoe_diffusion_256_022000.pt → models/ukiyoe-few/v1/ukiyoe_diffusion_256_022000.pt
RENAMED
|
File without changes
|