| | --- |
| | --- |
| | license: creativeml-openrail-m |
| | tags: |
| | - stable-diffusion |
| | - text-to-image |
| | datasets: |
| | - ProGamerGov/StableDiffusion-v1-5-Regularization-Images |
| | --- |
| | # Ukeiyo-style Diffusion |
| |
|
| | This is the fine-tuned Stable Diffusion model trained on traditional Japanese Ukeiyo-style images. |
| | Use the tokens **_ukeiyoddim style_** in your prompts for the effect. |
| | The model repo also contains a ckpt file , so that you can use the model with your own implementation of |
| | stable diffusion. |
| |
|
| | ### 🧨 Diffusers |
| |
|
| | This model can be used just like any other Stable Diffusion model. For more information, |
| | please have a look at the [Stable Diffusion](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion). |
| |
|
| | You can also export the model to [ONNX](https://huggingface.co/docs/diffusers/optimization/onnx), [MPS](https://huggingface.co/docs/diffusers/optimization/mps) and/or [FLAX/JAX](). |
| |
|
| | ```python |
| | #!pip install diffusers transformers scipy torch |
| | from diffusers import StableDiffusionPipeline |
| | import torch |
| | model_id = "salmonhumorous/ukeiyo-style-diffusion" |
| | pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16) |
| | pipe = pipe.to("cuda") |
| | prompt = "illustration of ukeiyoddim style landscape" |
| | image = pipe(prompt).images[0] |
| | image.save("./ukeiyo_landscape.png") |
| | ``` |
| |
|
| | ## Training procedure and data |
| |
|
| | The training for this model was done using a RTX 3090. The training was completed in 28 minutes for a total of 2000 steps. A total of 33 instance images (Images of the style I was aiming for) and 1k Regularization images was used. Regularization images dataset used by [ProGamerGov](https://huggingface.co/datasets/ProGamerGov/StableDiffusion-v1-5-Regularization-Images). |
| |
|
| | Training notebook used by [Shivam Shrirao](https://colab.research.google.com/github/ShivamShrirao/diffusers/blob/main/examples/dreambooth/DreamBooth_Stable_Diffusion.ipynb). |
| |
|
| | ### Training hyperparameters |
| |
|
| | The following hyperparameters were used during training: |
| | - number of steps : 2000 |
| | - learning_rate: 1e-6 |
| | - train_batch_size: 1 |
| | - scheduler_type: DDIM |
| | - number of instance images : 33 |
| | - number of regularization images : 1000 |
| | - lr_scheduler : constant |
| | - gradient_checkpointing |
| |
|
| | ### Results |
| |
|
| | Below are the sample results for different training steps : |
| |  |
| |
|
| | ### Sample images by model trained for 2000 steps : |
| |
|
| | prompt = "landscape" |
| |  |
| | prompt = "ukeiyoddim style landscape" |
| |  |
| | prompt = " illustration of ukeiyoddim style landscape" |
| |  |
| |
|
| |  |
| |
|
| | ### Acknowledgement |
| |
|
| | Many thanks to [nitrosocke](https://huggingface.co/nitrosocke), for inspiration and for the [guide](https://github.com/nitrosocke/dreambooth-training-guide). Also thanks, to all the amazing people making stable diffusion easily accessible for everyone. |
| |
|
| |
|
