tellurion commited on
Commit
02fcdb4
·
verified ·
1 Parent(s): f75dbb1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +101 -0
README.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ tags:
4
+ - sketch-colorization
5
+ - diffusion
6
+ - stable-diffusion-xl
7
+ - image-to-image
8
+ - reference-based
9
+ ---
10
+
11
+ # ColorizeDiffusion XL - Model Weights
12
+
13
+ <div align="center">
14
+
15
+ [![arXiv Paper](https://img.shields.io/badge/arXiv-2407.15886%20(base)-B31B1B?style=flat&logo=arXiv)](https://arxiv.org/abs/2401.01456)
16
+ [![WACV 2025](https://img.shields.io/badge/WACV%202025-v1-0CA4A5?style=flat&logo=Semantic%20Web)](https://openaccess.thecvf.com/content/WACV2025/html/Yan_ColorizeDiffusion_Improving_Reference-Based_Sketch_Colorization_with_Latent_Diffusion_Model_WACV_2025_paper.html)
17
+ [![CVPR 2025](https://img.shields.io/badge/CVPR%202025-v1.5-0CA4A5?style=flat&logo=Semantic%20Web)](https://arxiv.org/abs/2502.19937)
18
+ [![arXiv v2 Paper](https://img.shields.io/badge/arXiv-2504.06895%20(v2)-B31B1B?style=flat&logo=arXiv)](https://arxiv.org/abs/2504.06895)
19
+ [![GitHub](https://img.shields.io/badge/GitHub-Code-181717?style=flat&logo=GitHub)](https://github.com/tellurion-kanata/ColorizeDiffusionXL)
20
+ [![Demo](https://img.shields.io/badge/Hugging%20Face-Demo-FF9D00?style=flat&logo=Hugging%20Face)](https://huggingface.co/spaces/tellurion/ColorizeDiffusion)
21
+
22
+ </div>
23
+
24
+ Model weights for [ColorizeDiffusion XL](https://github.com/tellurion-kanata/ColorizeDiffusionXL), an SDXL-based reference-based sketch colorization framework (1024px).
25
+ For the base SD2.1 implementation (512/768px), refer to the [original repository](https://github.com/tellurion-kanata/colorizeDiffusion) and [weights](https://huggingface.co/tellurion/ColorizeDiffusion).
26
+
27
+ ## Model Variants
28
+
29
+ | Filename prefix | Config | Description |
30
+ |:----------------|:-------|:------------|
31
+ | `sdxl-*` | `configs/inference/sdxl.yaml` | Base SDXL colorizer with embedding guidance |
32
+ | `xlv2-*` | `configs/inference/xlv2.yaml` | SDXL colorizer with ReferenceNet adapter for enhanced background/style |
33
+
34
+ ## Usage
35
+
36
+ ### Inference with UI
37
+
38
+ ```bash
39
+ git clone https://github.com/tellurion-kanata/ColorizeDiffusionXL.git
40
+ cd ColorizeDiffusionXL
41
+ conda env create -f environment.yml
42
+ conda activate hf
43
+ python -u app.py
44
+ ```
45
+
46
+ Select and load a checkpoint from the UI dropdown. Weights are downloaded automatically from this repository.
47
+
48
+ ### Online Demo
49
+
50
+ Try the model directly on [Hugging Face Spaces](https://huggingface.co/spaces/tellurion/ColorizeDiffusion).
51
+
52
+ ### Programmatic Download
53
+
54
+ ```python
55
+ from huggingface_hub import hf_hub_download
56
+
57
+ path = hf_hub_download(
58
+ repo_id="tellurion/ColorizeDiffusionXL",
59
+ filename="<checkpoint_name>.safetensors",
60
+ )
61
+ ```
62
+
63
+ ## License
64
+
65
+ These weights are released under [CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
66
+
67
+ ## Citation
68
+
69
+ ```bibtex
70
+ @article{2024arXiv240101456Y,
71
+ author = {{Yan}, Dingkun and {Yuan}, Liang and {Wu}, Erwin and {Nishioka}, Yuma and {Fujishiro}, Issei and {Saito}, Suguru},
72
+ title = "{ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text}",
73
+ journal = {arXiv e-prints},
74
+ year = {2024},
75
+ doi = {10.48550/arXiv.2401.01456},
76
+ }
77
+
78
+ @InProceedings{Yan_2025_WACV,
79
+ author = {Yan, Dingkun and Yuan, Liang and Wu, Erwin and Nishioka, Yuma and Fujishiro, Issei and Saito, Suguru},
80
+ title = {ColorizeDiffusion: Improving Reference-Based Sketch Colorization with Latent Diffusion Model},
81
+ booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)},
82
+ year = {2025},
83
+ pages = {5092-5102}
84
+ }
85
+
86
+ @article{2025arXiv250219937Y,
87
+ author = {{Yan}, Dingkun and {Wang}, Xinrui and {Li}, Zhuoru and {Saito}, Suguru and {Iwasawa}, Yusuke and {Matsuo}, Yutaka and {Guo}, Jiaxian},
88
+ title = "{Image Referenced Sketch Colorization Based on Animation Creation Workflow}",
89
+ journal = {arXiv e-prints},
90
+ year = {2025},
91
+ doi = {10.48550/arXiv.2502.19937},
92
+ }
93
+
94
+ @article{yan2025colorizediffusionv2enhancingreferencebased,
95
+ title={ColorizeDiffusion v2: Enhancing Reference-based Sketch Colorization Through Separating Utilities},
96
+ author={Dingkun Yan and Xinrui Wang and Yusuke Iwasawa and Yutaka Matsuo and Suguru Saito and Jiaxian Guo},
97
+ year={2025},
98
+ journal = {arXiv e-prints},
99
+ doi = {10.48550/arXiv.2504.06895},
100
+ }
101
+ ```