Merge branch 'main' of hf.co:GeroldMeisinger/control-edgedrawing

README.md

@@ -1,20 +1,53 @@
 ---
 license: openrail
 datasets:
-- laion/laion2B-en-aesthetic
+- ChristophSchuhmann/improved_aesthetics_6.5plus
 language:
 - en
 ---
 
-Based on https://github.com/lllyasviel/ControlNet/discussions/318
+Based on my GitHub monologs at [Edge Drawing - a Canny alternative](https://github.com/lllyasviel/ControlNet/discussions/318)
+
+Controls image generation by edge maps generated with [EdgeDrawing Parameter-Free](https://github.com/CihanTopal/ED_Lib).
+
+For usage see the model page on [Civitai.com]().
+
+# Image dataset
+
+* [laion2B-en aesthetics>=6.5 dataset](https://huggingface.co/datasets/ChristophSchuhmann/improved_aesthetics_6.5plus)
+* `--min_image_size 512 --max_aspect_ratio 2 --resize_mode="center_crop" --image_size 512`
+* resulting in 180k images
+
+Original
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/RgXLEFqwSOzYQXvrs6pOr.png)
+
+EdgeDrawing Parameter-Free
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/jmdCGeMJx4dKFGo44cuEq.png)
+
+Canndy Edge Detection (default in Automatic1111)
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/JZTpa-HZfw0NUYnxZ52Iu.png)
+
+Example
+sampler=UniPC steps=20 cfg=7.5 seed=0 batch=9 model: v1-5-pruned-emaonly.safetensors cherry-picked: 1/9
+prompt: _a detailed high-quality professional photo of swedish woman standing in front of a mirror, dark brown hair, white hat with purple feather_
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/2PSWsmzLdHeVG-i67S7jF.png)
+
+# Training
 
 ```
 accelerate launch train_controlnet.py ^
   --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" ^
-  --output_dir="control-edgedrawing-default-drop50-fp16/" ^
+  --output_dir="control-edgedrawing-[version]-fp16/" ^
   --dataset_name="mydataset" ^
   --mixed_precision="fp16" ^
-  --proportion_empty_prompts=0.5 ^
   --resolution=512 ^
   --learning_rate=1e-5 ^
   --train_batch_size=1 ^
@@ -26,7 +59,30 @@ accelerate launch train_controlnet.py ^
   --seed=0
 ```
 
-Trained for 40000 steps on images converted with https://github.com/shaojunluo/EDLinePython using `smoothed = False` and default settings:
+# Versions
+
+**v3 - control-edgedrawing-cv480edpf-drop0-fp16-checkpoint-45000**
+
+Conditioning images generated with [edpf.py](https://gitlab.com/-/snippets/3601881) using [opencv-contrib-python::ximgproc::EdgeDrawing](https://docs.opencv.org/4.8.0/d1/d1c/classcv_1_1ximgproc_1_1EdgeDrawing.html).
+
+```
+ed     = cv2.ximgproc.createEdgeDrawing()
+params = cv2.ximgproc.EdgeDrawing.Params()
+params.PFmode = True
+ed.setParams(params)
+edges    = ed.detectEdges(image)
+edge_map = ed.getEdgeImage(edges)
+``` 
+
+45000 steps
+
+**v2 - control-edgedrawing-default-noisy-drop0-fp16-checkpoint-40000**
+
+Images converted with https://github.com/shaojunluo/EDLinePython
+
+Default settings are:
+
+`smoothed=False`
 
 ```
 { 'ksize'            :  5
@@ -37,13 +93,16 @@ Trained for 40000 steps on images converted with https://github.com/shaojunluo/E
 }
 ```
 
-**TODO**
+`smoothed=True`, but no empty prompts
+Trained for 40000 steps with default settings
+=> conditioning images are too noisy
+
+**v1 - control-edgedrawing-default-drop50-fp16-checkpoint-40000**
+
+Same as v1
 
-Results are not good so far:
+Update: bug in algorithm produces too sparse images on default, see https://github.com/shaojunluo/EDLinePython/issues/4
 
-* `--proportion_empty_prompts=0.5` me be too excessive for 40000 steps
-* Use `smoothed = True` next time, maybe control net doesn't pick up on single pixels
-* Find better parameter spread instead of default values, most images are very sparse
-* Train on more steps
-* Train on more diverse dataset
-* Train on higher-precision
+additional arguments: `--proportion_empty_prompts=0.5`
+Trained for 40000 steps with default settings
+=> empty prompts were probably too excessive