Update README.md

README.md

@@ -1,3 +1,122 @@
----
-license: mit
----
+---
+license: creativeml-openrail-m
+datasets:
+- SubMaroon/danbooru-colored
+base_model:
+- John6666/nsfw-anime-xl-v1-sdxl
+tags:
+- controlnet
+- anime
+- stable
+- diffusion
+---
+# ControlNet for Manga Colorization
+
+**Model Name:** `SubMaroon/controlnet_mangaRecolor`  
+**Base model:** `John6666/nsfw-anime-xl-v1-sdxl`  
+**Task:** Conditional image generation — Colorization  
+**Conditioning:** Grayscale manga panel (lineart or filled)  
+**Trained with:** [Hugging Face diffusers](https://github.com/huggingface/diffusers) ControlNet training pipeline
+
+---
+
+## Description
+
+This is a custom-trained **ControlNet** model designed to perform **automatic colorization of grayscale manga** images.
+
+The model takes in a **black-and-white manga scan** (converted to RGB) as conditioning input and generates a **colorized version** using Stable Diffusion.
+
+It is trained to act as a ControlNet module and requires a compatible SD base model — such as `nsfw-anime-xl-v1-sdxl` or other anime/manga-focused SD models.
+
+---
+
+## Training details
+
+- **Base model:** `John6666/nsfw-anime-xl-v1-sdxl`
+- **Dataset:** Custom dataset of ~6,000 image pairs from **Danbooru-based manga scans**, manually cleaned and resized to `512x512`
+- **Inputs:**
+  - `conditioning_image`: black-and-white manga scan (RGB)
+  - `text prompt`: optional (e.g. "colorized version of this panel")
+- **Loss:** MSE with FP16, trained on 1×A100, 20 epochs
+- **Resolution:** 512x512
+- **Scheduler:** default diffusers setup
+- **Optimizer:** AdamW, LR: `1e-5`
+
+---
+
+## Usage (Diffusers)
+
+```python
+from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
+from diffusers.utils import load_image
+import torch
+
+# Load ControlNet
+controlnet = ControlNetModel.from_pretrained("SubMaroon/controlnet_mangaRecolor", torch_dtype=torch.float16)
+
+# Load base pipeline
+pipe = StableDiffusionControlNetPipeline.from_pretrained(
+    "John6666/nsfw-anime-xl-v1-sdxl", controlnet=controlnet, torch_dtype=torch.float16
+)
+
+pipe.to("cuda")
+
+# Load grayscale manga panel
+conditioning_image = load_image("bw_manga_panel.png").convert("RGB")
+
+# Generate
+image = pipe("manga colorization", image=conditioning_image, num_inference_steps=30).images[0]
+image.save("colorized.png")
+```
+
+---
+
+## Usage in ComfyUI / WebUI
+
+- Place `diffusion_pytorch_model.safetensors` into your `ComfyUI/models/controlnet/` folder
+- Make sure to also include the `config.json`
+- Select this ControlNet in your workflow
+- Use grayscale images (512x512) as conditioning inputs
+
+---
+
+## Alternative training run (SDXL version)
+
+This version was trained using the SDXL-compatible ControlNet pipeline with the following CLI command:
+
+```bash
+accelerate launch train_controlnet_sdxl.py \
+  --pretrained_model_name_or_path John6666/nsfw-anime-xl-v1-sdxl \
+  --dataset_name SubMaroon/danbooru-colored \
+  --image_column image \
+  --conditioning_image_column conditioning_image \
+  --caption_column prompt \
+  --output_dir ./controlnet-colorization \
+  --resolution 512 \
+  --train_batch_size 8 \
+  --gradient_accumulation_steps 2 \
+  --learning_rate 1.4e-4 \
+  --num_train_epochs 4 \
+  --mixed_precision fp16 \
+  --gradient_checkpointing \
+  --checkpointing_steps 1000 \
+  --validation_steps 1000 \
+  --report_to tensorboard \
+  --tracker_project_name controlnet-colorization \
+  --seed 42
+```
+
+---
+
+## License
+
+The model is released under the **CreativeML Open RAIL-M** license.  
+You are free to use it for non-commercial and research purposes. Commercial use may require additional permission.
+
+---
+
+## Credits
+
+Created by [SubMaroon](https://huggingface.co/SubMaroon)  
+Based on the Hugging Face [`diffusers`](https://github.com/huggingface/diffusers) ControlNet training example  
+Inspired by [lllyasviel's original ControlNet](https://github.com/lllyasviel/ControlNet)