Text-to-Image
Diffusers
StableDiffusionPipeline
stable-diffusion
sygil-diffusion
sygil-devs
finetune
stable-diffusion-1.5
Instructions to use Sygil/Sygil-Diffusion with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Sygil/Sygil-Diffusion with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Sygil/Sygil-Diffusion", dtype=torch.bfloat16, device_map="cuda") prompt = "environment art, realistic" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps
- Draw Things
- DiffusionBee
Commit ·
6a8fdba
1
Parent(s): ad593b8
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,8 +11,8 @@ tags:
|
|
| 11 |
- sygil-devs
|
| 12 |
- finetune
|
| 13 |
- stable-diffusion-1.5
|
| 14 |
-
inference:
|
| 15 |
-
|
| 16 |
---
|
| 17 |
|
| 18 |
# About the model
|
|
@@ -24,18 +24,50 @@ on the final result including image composition.
|
|
| 24 |
|
| 25 |
**Note that the prompt engineering techniques is a bit different from other models and Stable Diffusion,
|
| 26 |
while you can still use normal prompts like in other Stable Diffusion modelsin order to get the best out of this model you will need to make use of tags and namespaces.
|
|
|
|
|
|
|
| 27 |
[More about it here](promptingGuide.md)** \
|
|
|
|
| 28 |
\
|
| 29 |
**If you find our work useful, please consider supporting us using one of the options below:**
|
| 30 |
- [OpenCollective](https://opencollective.com/sygil_dev)
|
| 31 |
-
|
| 32 |
-
-
|
| 33 |
**Join our Discord Server for supports and announcements**
|
| 34 |
[](https://discord.gg/fTtcufxyHQ)
|
| 35 |
|
| 36 |
# Showcase
|
| 37 |

|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
## Training
|
| 40 |
|
| 41 |
**Training Data**
|
|
@@ -43,6 +75,7 @@ The model was trained on the following dataset:
|
|
| 43 |
- [Imaginary Network Expanded Dataset](https://github.com/Sygil-Dev/INE-dataset) dataset.
|
| 44 |
|
| 45 |
- **Hardware:** 1 x Nvidia RTX 3050 8GB GPU
|
|
|
|
| 46 |
- **Optimizer:** AdamW
|
| 47 |
- **Gradient Accumulations**: 1
|
| 48 |
- **Batch:** 1
|
|
|
|
| 11 |
- sygil-devs
|
| 12 |
- finetune
|
| 13 |
- stable-diffusion-1.5
|
| 14 |
+
inference: true
|
| 15 |
+
pinned: true
|
| 16 |
---
|
| 17 |
|
| 18 |
# About the model
|
|
|
|
| 24 |
|
| 25 |
**Note that the prompt engineering techniques is a bit different from other models and Stable Diffusion,
|
| 26 |
while you can still use normal prompts like in other Stable Diffusion modelsin order to get the best out of this model you will need to make use of tags and namespaces.
|
| 27 |
+
More information about namespace will later be added.
|
| 28 |
+
<!---
|
| 29 |
[More about it here](promptingGuide.md)** \
|
| 30 |
+
-->
|
| 31 |
\
|
| 32 |
**If you find our work useful, please consider supporting us using one of the options below:**
|
| 33 |
- [OpenCollective](https://opencollective.com/sygil_dev)
|
| 34 |
+
|
|
|
|
| 35 |
**Join our Discord Server for supports and announcements**
|
| 36 |
[](https://discord.gg/fTtcufxyHQ)
|
| 37 |
|
| 38 |
# Showcase
|
| 39 |

|
| 40 |
|
| 41 |
+
## Examples
|
| 42 |
+
|
| 43 |
+
Using the [🤗's Diffusers library](https://github.com/huggingface/diffusers) to run Sygil Diffusion in a simple and efficient manner.
|
| 44 |
+
|
| 45 |
+
```bash
|
| 46 |
+
pip install diffusers transformers accelerate scipy safetensors
|
| 47 |
+
```
|
| 48 |
+
Running the pipeline (if you don't swap the scheduler it will run with the default DDIM, in this example we are swapping it to DPMSolverMultistepScheduler):
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler
|
| 52 |
+
|
| 53 |
+
model_id = "Sygil/Sygil-Diffusion"
|
| 54 |
+
|
| 55 |
+
# Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
|
| 56 |
+
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
|
| 57 |
+
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
|
| 58 |
+
pipe = pipe.to("cuda")
|
| 59 |
+
|
| 60 |
+
prompt = "a photo of an astronaut riding a horse on mars"
|
| 61 |
+
image = pipe(prompt).images[0]
|
| 62 |
+
|
| 63 |
+
image.save("astronaut_rides_horse.png")
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
**Notes**:
|
| 67 |
+
- Despite not being a dependency, we highly recommend you to install [xformers](https://github.com/facebookresearch/xformers) for memory efficient attention (better performance)
|
| 68 |
+
- If you have low GPU RAM available, make sure to add a `pipe.enable_attention_slicing()` after sending it to `cuda` for less VRAM usage (to the cost of speed).
|
| 69 |
+
|
| 70 |
+
|
| 71 |
## Training
|
| 72 |
|
| 73 |
**Training Data**
|
|
|
|
| 75 |
- [Imaginary Network Expanded Dataset](https://github.com/Sygil-Dev/INE-dataset) dataset.
|
| 76 |
|
| 77 |
- **Hardware:** 1 x Nvidia RTX 3050 8GB GPU
|
| 78 |
+
- **Hours Trained:** 384 approximately.
|
| 79 |
- **Optimizer:** AdamW
|
| 80 |
- **Gradient Accumulations**: 1
|
| 81 |
- **Batch:** 1
|