control-edgedrawing-cv480edpf-drop50-fp16-checkpoint-90000

README.md

@@ -4,9 +4,11 @@ datasets:
 - ChristophSchuhmann/improved_aesthetics_6.5plus
 language:
 - en
+tags:
+- controlnet
 ---
 
-Controls image generation by edge maps generated with [Edge Drawing](https://github.com/CihanTopal/ED_Lib). Edge Drawing comes in different flavors: original (_ed_), parameter-free (_edpf_), color (_edcolor_).
+Controls image generation by edge maps generated with [Edge Drawing](https://github.com/CihanTopal/ED_Lib). Edge Drawing comes in different flavors: original (_ed_), parameter free (_edpf_), color (_edcolor_).
 
 * Based on my monologs at [github.com - Edge Drawing](https://github.com/lllyasviel/ControlNet/discussions/318)
 * For usage see the model page on [civitai.com - Model](https://civitai.com/models/149740).
@@ -26,11 +28,13 @@ prompt: _a detailed high-quality professional photo of swedish woman standing in
 
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/2PSWsmzLdHeVG-i67S7jF.png)
 
+_Clear and pristine! Wooow!_
+
 **Canndy Edge for comparison (default in Automatic1111)**
 
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c0ec65a2ec8cb2f589233a/JZTpa-HZfw0NUYnxZ52Iu.png)
 
-_notice all the missing edges, the noise and artifacts. yuck! ugh!_
+_Noise, artifacts and missing edges. Yuck! Ugh!_
 
 # Image dataset
 
@@ -61,7 +65,7 @@ accelerate launch train_controlnet.py ^
 
 To evaluate the model it makes sense to compare it with the original Canny model. Original evaluations and comparisons are available at [ControlNet 1.0 repo](https://github.com/lllyasviel/ControlNet), [ControlNet 1.1 repo](https://github.com/lllyasviel/ControlNet-v1-1-nightly), [ControlNet paper v1](https://arxiv.org/abs/2302.05543v1), [ControlNet paper v2](https://arxiv.org/abs/2302.05543) and [Diffusers implementation](https://huggingface.co/takuma104/controlnet_dev/tree/main). Some points we have to keep in mind when comparing canny with edpf in order not to compare apples with oranges:
 * canny 1.0 model was trained on 3M images with fp32, canny 1.1 model on even more, while edpf model so far is only trained on a 180k-360k with fp16.
-* canny edge-detector requires parameter tuning while edpf is parameter-free.
+* canny edge-detector requires parameter tuning while edpf is parameter free.
 * Do we manually fine-tune canny to find the perfect input image or do we leave it at default? We could argue that "no fine-tuning required" is the usp of edpf and we want to compare in the default setting, whereas canny fine-tuning is subjective.
 * Would the canny model actually benefit from a edpf pre-processor and we might not even require a edpf model? (2023-09-25: see `eval_canny_edpf.zip` but it seems as if it doesn't work and the edpf model may be justified)
 * When evaluating human images we need to be aware of Stable Diffusion's inherent limits, like disformed faces and hands, and don't attribute them to the control net.
@@ -109,7 +113,7 @@ edges    = ed.detectEdges(image)
 edge_map = ed.getEdgeImage(edges)
 ``` 
 
-45000 steps => looks good. released as **version 0.1 on civitai**.
+45000 steps => looks good. resuming with left-right flipped images. released as **version 0.1 on civitai**.
 
 **Experiment 3.1 - 2023-09-24 - control-edgedrawing-cv480edpf-drop0-fp16-checkpoint-90000**
 
@@ -123,7 +127,9 @@ resumed with epoch 2 from 90000 using `--proportion_empty_prompts=0.5` => result
 
 see experiment 3.0. restarted from 0 with `--proportion_empty_prompts=0.5` => results are not good, 50% is probably too much for 45k steps. guessmode still doesn't work and tends to produces humans. resuming until 90k with right-left flipped in the hope it will get better with more images.
 
-**Experiment 4.1 - control-edgedrawing-cv480edpf-drop50-fp16-checkpoint-45000**
+**Experiment 4.1 - 2023-09-26 - control-edgedrawing-cv480edpf-drop50-fp16-checkpoint-90000**
+
+resumed from 45000 steps with left-right flipped images => results are still not good, 50% is probably too much for 45k steps. guessmode still doesn't work and tends to produces humans. abort.
 
 # Ideas
 

control-edgedrawing-cv480edpf-drop50-fp16-checkpoint-90000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a65184060f15e136ef5871c240d433a70bee6e61c1f9c1731d9e0c5783c40b2
+size 722598616

control-edgedrawing-cv480edpf-drop50-fp16-checkpoint-90000.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a27b63038afcde03e686d924d662bbd5d4c950b0019f6423c38e6854c36a79b
+size 5469226