Commit
·
46b6a5b
1
Parent(s):
1099b93
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,12 +1,46 @@
|
|
| 1 |
---
|
| 2 |
license: openrail
|
| 3 |
datasets:
|
| 4 |
-
-
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
---
|
| 8 |
|
| 9 |
-
Based on https://github.com/lllyasviel/ControlNet/discussions/318
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
|
| 11 |
```
|
| 12 |
accelerate launch train_controlnet.py ^
|
|
@@ -25,6 +59,25 @@ accelerate launch train_controlnet.py ^
|
|
| 25 |
--seed=0
|
| 26 |
```
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
Images converted with https://github.com/shaojunluo/EDLinePython
|
| 29 |
|
| 30 |
Default settings are:
|
|
@@ -40,14 +93,16 @@ Default settings are:
|
|
| 40 |
}
|
| 41 |
```
|
| 42 |
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
additional argument: `--proportion_empty_prompts=0.5`
|
| 46 |
Trained for 40000 steps with default settings
|
| 47 |
-
=>
|
| 48 |
|
| 49 |
-
**control-edgedrawing-default-
|
| 50 |
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
Trained for 40000 steps with default settings
|
| 53 |
-
=>
|
|
|
|
| 1 |
---
|
| 2 |
license: openrail
|
| 3 |
datasets:
|
| 4 |
+
- ChristophSchuhmann/improved_aesthetics_6.5plus
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
---
|
| 8 |
|
| 9 |
+
Based on my GitHub monologs at [Edge Drawing - a Canny alternative](https://github.com/lllyasviel/ControlNet/discussions/318)
|
| 10 |
+
|
| 11 |
+
Controls image generation by edge maps generated with [EdgeDrawing Parameter-Free](https://github.com/CihanTopal/ED_Lib).
|
| 12 |
+
|
| 13 |
+
For usage see the model page on [Civitai.com]().
|
| 14 |
+
|
| 15 |
+
# Image dataset
|
| 16 |
+
|
| 17 |
+
* [laion2B-en aesthetics>=6.5 dataset](https://huggingface.co/datasets/ChristophSchuhmann/improved_aesthetics_6.5plus)
|
| 18 |
+
* `--min_image_size 512 --max_aspect_ratio 2 --resize_mode="center_crop" --image_size 512`
|
| 19 |
+
* resulting in 180k images
|
| 20 |
+
|
| 21 |
+
Original
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+

|
| 25 |
+
|
| 26 |
+
EdgeDrawing Parameter-Free
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+

|
| 30 |
+
|
| 31 |
+
Canndy Edge Detection (default in Automatic1111)
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+

|
| 35 |
+
|
| 36 |
+
Example
|
| 37 |
+
sampler=UniPC steps=20 cfg=7.5 seed=0 batch=9 model: v1-5-pruned-emaonly.safetensors cherry-picked: 1/9
|
| 38 |
+
prompt: _a detailed high-quality professional photo of swedish woman standing in front of a mirror, dark brown hair, white hat with purple feather_
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+

|
| 42 |
+
|
| 43 |
+
# Training
|
| 44 |
|
| 45 |
```
|
| 46 |
accelerate launch train_controlnet.py ^
|
|
|
|
| 59 |
--seed=0
|
| 60 |
```
|
| 61 |
|
| 62 |
+
# Versions
|
| 63 |
+
|
| 64 |
+
**v3 - control-edgedrawing-cv480edpf-drop0-fp16-checkpoint-45000**
|
| 65 |
+
|
| 66 |
+
Conditioning images generated with [edpf.py](https://gitlab.com/-/snippets/3601881) using [opencv-contrib-python::ximgproc::EdgeDrawing](https://docs.opencv.org/4.8.0/d1/d1c/classcv_1_1ximgproc_1_1EdgeDrawing.html).
|
| 67 |
+
|
| 68 |
+
```
|
| 69 |
+
ed = cv2.ximgproc.createEdgeDrawing()
|
| 70 |
+
params = cv2.ximgproc.EdgeDrawing.Params()
|
| 71 |
+
params.PFmode = True
|
| 72 |
+
ed.setParams(params)
|
| 73 |
+
edges = ed.detectEdges(image)
|
| 74 |
+
edge_map = ed.getEdgeImage(edges)
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
45000 steps
|
| 78 |
+
|
| 79 |
+
**v2 - control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000**
|
| 80 |
+
|
| 81 |
Images converted with https://github.com/shaojunluo/EDLinePython
|
| 82 |
|
| 83 |
Default settings are:
|
|
|
|
| 93 |
}
|
| 94 |
```
|
| 95 |
|
| 96 |
+
`smoothed=True`, but no empty prompts
|
|
|
|
|
|
|
| 97 |
Trained for 40000 steps with default settings
|
| 98 |
+
=> conditioning images are too noisy
|
| 99 |
|
| 100 |
+
**v1 - control-edgedrawing-default-drop50-fp16-checkpoint-40000**
|
| 101 |
|
| 102 |
+
Same as v1
|
| 103 |
+
|
| 104 |
+
Update: bug in algorithm produces too sparse images on default, see https://github.com/shaojunluo/EDLinePython/issues/4
|
| 105 |
+
|
| 106 |
+
additional arguments: `--proportion_empty_prompts=0.5`
|
| 107 |
Trained for 40000 steps with default settings
|
| 108 |
+
=> empty prompts were probably too excessive
|