README.md

---

license: apache-2.0
license_name: apache-2.0
tags:
- lora
- manga
- coloring
- anime
- qwen
- dataset
- diffusers
- image-to-image
viewer: false
---


<p align="center">
  <img src="images/logo.png" alt="PanelPainter Logo" width="400">
</p>

# PanelPainter-Project

**PanelPainter-Project** is an open-source initiative to automate black-and-white manga coloring using fine-tuned LoRAs.

This project is dedicated to training LoRAs to automate the coloring of black-and-white manga panels. I am releasing all the files here, including datasets, logs, and experimental versions, so others can see exactly how it was trained.

## Showcase

Here are some examples comparing the original panel, the base Qwen Image Edit model, and the result with the PanelPainter V3 LoRA.

> [!NOTE]
> **Showcase Generation Settings:**
> * **LoRAs:** PanelPainter V3 (Weight: 1.0) + 4-Step Lighting (Weight: 1.0)
> * **Steps:** 4
> * **Sampler:** Euler
> * **Scheduler:** Simple
> * **Seed:** 1000
> * **CFG:** 1.0

<p align="center">
  <img src="images/Sample_Image_1.png" alt="Chainsaw Man Showcase">
  <br>
  <em>Chainsaw Man</em>
</p>

<p align="center">
  <img src="images/Sample_Image_2.png" alt="Frieren Showcase">
  <br>
  <em>Frieren</em>
</p>

<p align="center">
  <img src="images/Sample_Image_3.png" alt="Komi Showcase">
  <br>
  <em>Komi Can't Communicate</em>
</p>

<p align="center">
  <img src="images/Sample_Image_4.png" alt="Oshi no Ko Showcase">
  <br>
  <em>Oshi no Ko</em>
</p>

## Project Structure

This repository contains everything used to create the models:

### 1. LoRA Models (`/loras`)
This directory contains the model weights for all iterations of the project:

> [!TIP]
> **Trigger Word:** `Color this panelpainter` (Applicable for both V2 and V3)

* **V3 (Latest Release):** `PanelPainter_v3_Qwen2511.safetensors`
    * **Base:** Qwen Image Edit 2511
    * **Note:** The latest model trained on the expanded 903-image dataset.
* **V2 (Stable):** `PanelPainter_v2_Qwen2509.safetensors`
    * **Base:** Qwen Image Edit 2509 (Compatible with 2511).
    * **Note:** Standard release (High quality, low variety).
* **V1 (Legacy):** `PanelPainter_v1_Legacy.safetensors`
    * **Base:** Qwen Image Edit 2509
    * **Note:** Archived experimental version (synthetic data).

### 2. Training Logs (`/logs`)
**Content:** Tensorboard logs and charts from my training runs. You can check these to see how the loss converged and how the model learned over time for each version.

### 3. Workflows (`/workflows`)
**Content:** ComfyUI workflow JSON files to help you get started with PanelPainter.

### 4. Training Dataset
The datasets used for this project are hosted separately:

* **PanelPainter-Dataset**

> [!NOTE]
> **Coming Soon:** The V3 dataset was a good learning step for captioning, but it was randomly picked without any streamlined curation roughly 50% doujin and 50% mainstream colored manga. We're refining it further. Expect handpicked panels, better captions, and reduced doujin content. Release coming once quality standards are met.

---

## Version History & Development Log

### Version 3.0 (Current Release)
* **Status:** Released.
* **Base Architecture:** Qwen 2511.
* **Strategy:** Scaling Up High-Quality Data.
* **Dataset:** Expanded to 903 images. Recreated from scratch, comprising 50% doujin and 50% SFW panels.
* **Summary:** This version combines the correct "real line art" training method discovered in V2 with a significantly larger dataset. This improves the model's ability to generalize across different manga styles while maintaining the color quality of V2.

### Version 2.0
* **Status:** Released / Stable.
* **Base Model:** Trained on Qwen Image Edit 2509, also it works on Qwen 2511 as well.
* **The Breakthrough:** After V1 failed, this version switched to training on real line art instead of synthetic grayscale.
* **Dataset:** A tiny, hyper-curated set of 150 images (70% Doujin / 30% SFW).
* **Outcome:** Despite the small size, it proved that high-quality real line art outperforms massive synthetic datasets. It produces good colors but lacks variety due to the small sample size.

### Version 1.0
* **Status:** Archived / Deprecated.
* **Base Model:** Qwen Image Edit 2509.
* **The Mistake:** Trained on 7,000 images generated by simply desaturating colored pages (synthetic grayscale).
* **Outcome:** The model learned to color "perfect gray" inputs but failed on real, imperfect ink lines.
* **Lesson:** Quantity does not matter if the data distribution doesn't match real usage.

---

## Training Configuration (V3)

**Hardware:** Trained on an A40 GPU on Runpod.

Below is the exact accelerate command used to train the V3 model on Musubi Tuner:

```bash

accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 \

  /workspace/musubi-tuner/src/musubi_tuner/qwen_image_train_network.py \

  --dataset_config dataset_edit.toml \

  --dit /workspace/Training_Models_Qwen/Qwen_Image_Edit_2511_BF16.safetensors \

  --vae /workspace/Training_Models_Qwen/qwen_train_vae.safetensors \

  --text_encoder /workspace/Training_Models_Qwen/qwen_2.5_vl_7b_bf16.safetensors \

  --model_version edit-2511 \

  --network_module networks.lora_qwen_image \

  --output_dir /workspace/output_panelpainter \

  --output_name panelpainter_v3_part1 \

  --mixed_precision bf16 \

  --max_data_loader_n_workers 0 \

  --learning_rate 3e-4 \

  --network_dim 128 \

  --network_alpha 128 \

  --optimizer_type adafactor \

  --optimizer_args "scale_parameter=False" "relative_step=False" "warmup_init=False" "weight_decay=0.01" \

  --lr_scheduler cosine \

  --lr_warmup_steps 150 \

  --timestep_sampling qinglong_qwen \

  --discrete_flow_shift 2.2 \

  --max_train_epochs 8 \

  --save_every_n_epochs 1 \

  --save_state \

  --gradient_checkpointing \

  --gradient_checkpointing_cpu_offload \

  --gradient_accumulation_steps 4 \

  --blocks_to_swap 20 \

  --sdpa

```

**Dataset Settings:** Use a resolution of **1328x1328** with bucketing enabled to handle varying aspect ratios (no upscaling). The training ran with a batch size of 1 and enabled `qwen_image_edit_no_resize_control` to preserve the original dimensions of the control images during processing.

## License

* **Project:** Apache 2.0
* **Dataset:** Hosted separately, contains copyrighted manga panels.
* **Copyright:** Original art belongs to the respective creators and publishers.

## Acknowledgements

Trained on Musubi Tuner. Thanks to kohya-ss.

**Dataset Contributors:** Thanks to @Rox_Jr & @lucifer_brine04 for their help with the dataset.

## External Links
* **Public Model Page:** [Civitai: PanelPainter](https://civitai.com/models/2103847/panelpainter-manga-coloring)