Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,112 @@
|
|
| 1 |
---
|
| 2 |
license: creativeml-openrail-m
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: creativeml-openrail-m
|
| 3 |
+
tags:
|
| 4 |
+
- text-to-image
|
| 5 |
+
- stable-diffusion
|
| 6 |
+
- anime
|
| 7 |
+
- aiart
|
| 8 |
---
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
This model is trained on 33 different concepts from Bofuri: I Don't Want to Get Hurt, so I'll Max Out My Defense (防振り: 痛いのは嫌なので防御力に極振りしたいと思います。).
|
| 15 |
+
Here are some examples generations.
|
| 16 |
+
|
| 17 |
+
### Example Generations
|
| 18 |
+
|
| 19 |
+
Prompt: `BoMaple uniform BoSally unfirom, yuri, in classroom, 4K wallpaper, beautiful eyes`
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
Prompt: `2girls, BoMay BoYui, yuri, half body, floating in the sky, cloud, sparkling eyes, 4K wallpaer, anime coloring, official art`
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
Prompt: `BoKanade casting magic, 4K wallpaper, outdoors`
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
(Negative is mostly variations of: `bad hands, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry`)
|
| 29 |
+
|
| 30 |
+
### Usage
|
| 31 |
+
|
| 32 |
+
The model is shared in both diffuser safetensors format. Intermediatet checkpoints are also shared in ckpt format in the directory `checkpoints`.
|
| 33 |
+
|
| 34 |
+
### Concepts
|
| 35 |
+
|
| 36 |
+
The 33 concepts are listed in `concept_list` and demonstrated below.
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
Expect bad results for `BoMaple sheep form` and non-human concepts. Espeically the model clearly does not understand the anatomy of syrup.
|
| 48 |
+
For `BoKasumi sarashi` adding `bandages` seems to help. For `BoMaple pajama` you can add `stripe` for more similarity to the ones appearing in anime.
|
| 49 |
+
The remaining concepts should go through smoothly.
|
| 50 |
+
|
| 51 |
+
#### Prompt format
|
| 52 |
+
During training the concept names are put at the beginning of the image separated only by spaces, but not doing so seems to work as well.
|
| 53 |
+
Put `aniscreen` after the concept names would reinfoce the anime style.
|
| 54 |
+
Mixint two concepts is fairly doable as demonstrated above.
|
| 55 |
+
However expect weird blending to happen most of the time starting from three concepts.
|
| 56 |
+
This is partially because this model is not trained too much on multi-concept scenes.
|
| 57 |
+
Below is roughly the best we can get after multiply tries (there is still clothe blending).
|
| 58 |
+
|
| 59 |
+
Prompt: `(BoMaple black armor) BoSally turtleneck BoKasumi, 3girls, 4K wallpaper, ahoge, black hair, brown hair, outdoors, long hair`
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
### More Generations
|
| 64 |
+
|
| 65 |
+
Prompt: BoMaple black armors aniscreen, 1girl solo, Hydra in the sky, light purple eyes, 4K wallpaper
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
Prompt: BoMaple black armors near small turtle syrup, sitting with knees up on rock looking at viewer, turtle shell, beautiful hand in glove, in front of trees , outdoors, close-up, 4K wallpaper
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
Prompt: BoMaple pajama stripe, sitting on bed with barefoot, in girl's room, detailed and fancy background, sparkling purple eyes, hand on bed, 4K wallpaper
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
Prompt: BoFrederica, cowboy shot, in rubble ruins, ((under blue sky)), cinematic angle, dynamic pose, oblique angle, 4K wallpaer, anime coloring, official art
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
Prompt: Turtle Syrup Fox Oboro next to each other simple background white background, animals
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
Failures are of course unavoidable
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
Finally, you can always get different styles via model merging
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
### Dataset Description
|
| 89 |
+
|
| 90 |
+
The dataset is prepared via the workflow detailed here: https://github.com/cyber-meow/anime_screenshot_pipeline
|
| 91 |
+
|
| 92 |
+
It contains 27031 images with the following composition
|
| 93 |
+
|
| 94 |
+
- 7752 bofuri images mainly composed of screenshots from the first season and of the first three episods of the second season
|
| 95 |
+
- 19279 regularization images which intend to be as various as possible while being in anime style (i.e. no photorealistic image is used)
|
| 96 |
+
|
| 97 |
+
Note that the model is trained with a specific weighting scheme to balance between different concepts so that every image does not weight equally.
|
| 98 |
+
After applying the per-image repeat we get around 20 images per epoch.
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
### Training
|
| 102 |
+
|
| 103 |
+
Training is done with [EveryDream2](https://github.com/victorchall/EveryDream2trainer) trainer using [JosephusCheung/ACertainty](https://huggingface.co/JosephusCheung/ACertainty) as base model.
|
| 104 |
+
I use the following configuration thanks to the suggestion of 金Goldkoron
|
| 105 |
+
|
| 106 |
+
- resolution 512
|
| 107 |
+
- cosine learning rate scheduler, lr 2.5e-6
|
| 108 |
+
- batch size 4
|
| 109 |
+
- conditional dropout 0.05
|
| 110 |
+
- change beta scheduler from `scaler_linear` to `linear` in `config.json` of the scheduler of the model
|
| 111 |
+
|
| 112 |
+
The released model is trained for 57751 steps, but among the provided checkpoints all the three starting from 34172 steps seem to work reasonably well.
|