SubMaroon commited on
Commit
022ee4c
·
verified ·
1 Parent(s): b0bca83

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -24
README.md CHANGED
@@ -26,19 +26,19 @@ This is a custom-trained **ControlNet** model designed to perform **automatic co
26
 
27
  The model takes in a **black-and-white anime styled pictures** (converted to RGB) as conditioning input and generates a **colorized version** using Stable Diffusion.
28
 
29
- It is trained to act as a ControlNet module and requires a compatible SD base model — such as `nsfw-anime-xl-v1-sdxl` or other anime/manga-focused SD models.
30
 
31
  ---
32
 
33
  ## Training details
34
 
35
  - **Base model:** `John6666/nsfw-anime-xl-v1-sdxl`
36
- - **Dataset:** Custom dataset of ~6,000 image pairs from **Danbooru-based manga scans**, manually cleaned and resized to `512x512`
37
  - **Inputs:**
38
  - `conditioning_image`: black-and-white manga scan (RGB)
39
- - `text prompt`: optional (e.g. "colorized version of this panel")
40
- - **Loss:** MSE with FP16, trained on 1×RTX3090, 4 epochs
41
- - **Resolution:** 512x512
42
  - **Scheduler:** default diffusers setup
43
  - **Optimizer:** LR: `1.4e-4`
44
 
@@ -76,7 +76,7 @@ image.save("colorized.png")
76
  - Place `diffusion_pytorch_model.safetensors` into your `ComfyUI/models/controlnet/` folder
77
  - Make sure to also include the `config.json`
78
  - Select this ControlNet in your workflow
79
- - Use grayscale images (512x512) as conditioning inputs
80
 
81
  ---
82
 
@@ -85,25 +85,25 @@ image.save("colorized.png")
85
  This version was trained using the SDXL-compatible ControlNet pipeline with the following CLI command:
86
 
87
  ```bash
88
- accelerate launch train_controlnet_sdxl.py \
89
- --pretrained_model_name_or_path John6666/nsfw-anime-xl-v1-sdxl \
90
- --dataset_name SubMaroon/danbooru-colored \
91
- --image_column image \
92
- --conditioning_image_column conditioning_image \
93
- --caption_column prompt \
94
- --output_dir ./controlnet-colorization \
95
- --resolution 512 \
96
- --train_batch_size 8 \
97
- --gradient_accumulation_steps 2 \
98
- --learning_rate 1.4e-4 \
99
- --num_train_epochs 4 \
100
- --mixed_precision fp16 \
101
  --gradient_checkpointing \
102
- --checkpointing_steps 1000 \
103
- --validation_steps 1000 \
104
- --report_to tensorboard \
105
- --tracker_project_name controlnet-colorization \
106
- --seed 42
107
  ```
108
 
109
  ---
 
26
 
27
  The model takes in a **black-and-white anime styled pictures** (converted to RGB) as conditioning input and generates a **colorized version** using Stable Diffusion.
28
 
29
+ It is trained to act as a ControlNet module and requires a compatible SDXL base model — such as `nsfw-anime-xl-v1-sdxl` or other anime/manga-focused SDXL models.
30
 
31
  ---
32
 
33
  ## Training details
34
 
35
  - **Base model:** `John6666/nsfw-anime-xl-v1-sdxl`
36
+ - **Dataset:** Custom dataset of ~6,000 image pairs from **Danbooru-based manga scans**, manually cleaned and resized to `768x768`
37
  - **Inputs:**
38
  - `conditioning_image`: black-and-white manga scan (RGB)
39
+ - `text prompt`: optional (e.g. "1girl, blue_eyes, blue_hair etc.")
40
+ - **Loss:** MSE with FP16, trained on 1×RTX3090, 12 epochs
41
+ - **Resolution:** 768x768
42
  - **Scheduler:** default diffusers setup
43
  - **Optimizer:** LR: `1.4e-4`
44
 
 
76
  - Place `diffusion_pytorch_model.safetensors` into your `ComfyUI/models/controlnet/` folder
77
  - Make sure to also include the `config.json`
78
  - Select this ControlNet in your workflow
79
+ - Use grayscale images as conditioning inputs
80
 
81
  ---
82
 
 
85
  This version was trained using the SDXL-compatible ControlNet pipeline with the following CLI command:
86
 
87
  ```bash
88
+ accelerate launch train_controlnet.py \
89
+ --pretrained_model_name_or_path="John6666/nsfw-anime-xl-v1-sdxl" \
90
+ --dataset_name="SubMaroon/danbooru-colored" \
91
+ --image_column="image" \
92
+ --conditioning_image_column="conditioning_image" \
93
+ --caption_column="prompt" \
94
+ --output_dir="./controlnet-colorization" \
95
+ --resolution=768 \
96
+ --train_batch_size=4 \
97
+ --gradient_accumulation_steps=4 \
98
+ --learning_rate=1.4e-4 \
99
+ --num_train_epochs=12 \
100
+ --mixed_precision="fp16" \
101
  --gradient_checkpointing \
102
+ --checkpointing_steps=1000 \
103
+ --validation_steps=1000 \
104
+ --report_to="tensorboard" \
105
+ --tracker_project_name="controlnet-colorization" \
106
+ --seed=42
107
  ```
108
 
109
  ---