cyber commited on
Commit
66aa8e1
·
verified ·
1 Parent(s): ff62a56
cv_ddcolor_image-colorization/.mdl DELETED
Binary file (57 Bytes)
 
cv_ddcolor_image-colorization/.msc DELETED
Binary file (547 Bytes)
 
cv_ddcolor_image-colorization/.mv DELETED
@@ -1 +0,0 @@
1
- Revision:v1.02,CreatedAt:1678030594
 
 
cv_ddcolor_image-colorization/README.md DELETED
@@ -1,114 +0,0 @@
1
- ---
2
- tasks:
3
- - image-colorization
4
- widgets:
5
- - task: image-colorization
6
- inputs:
7
- - type: image
8
- examples:
9
- - name: 1
10
- inputs:
11
- - name: image
12
- data: git://resources/demo.jpg
13
- - name: 2
14
- inputs:
15
- - name: image
16
- data: git://resources/demo2.jpg
17
- - name: 3
18
- inputs:
19
- - name: image
20
- data: git://resources/demo3.jpg
21
- inferencespec:
22
- cpu: 4
23
- memory: 16000
24
- gpu: 1
25
- gpu_memory: 16000
26
- model-type:
27
- - ddcolor
28
- domain:
29
- - cv
30
- frameworks:
31
- - pytorch
32
- backbone:
33
- - unet
34
- metrics:
35
- - fid
36
- - colorfulness
37
- customized-quickstart: False
38
- finetune-support: False
39
- license: Apache License 2.0
40
- tags:
41
- - image colorization
42
- - old photo restoration
43
- - DDColor
44
- datasets:
45
- test:
46
- - modelscope/image-colorization-dataset
47
- ---
48
-
49
- # DDColor 图像上色模型
50
-
51
- 该模型为黑白图像上色模型,输入一张黑白图像,实现端到端的全图上色,返回上色处理后的彩色图像。
52
-
53
- ## 模型描述
54
-
55
- DDColor 是最新的图像上色算法,能够对输入的黑白图像生成自然生动的彩色结果。
56
-
57
- 算法整体流程如下图,使用 UNet 结构的骨干网络和图像解码器分别实现图像特征提取和特征图上采样,并利用 Transformer 结构的颜色解码器完成基于视觉语义的颜色查询,最终聚合输出彩色通道预测结果。
58
-
59
- ![ofa-image-caption](./resources/ddcolor_arch.jpg)
60
-
61
- ## 模型期望使用方式和适用范围
62
-
63
- 该模型适用于多种格式的图像输入,给定黑白图像,生成上色后的彩色图像;给定彩色图像,将自动提取灰度通道作为输入,生成重上色的图像。
64
-
65
- ### 如何使用
66
-
67
- 在 ModelScope 框架上,提供输入图片,即可以通过简单的 Pipeline 调用来使用图像上色模型。
68
-
69
- #### 代码范例
70
-
71
- ```python
72
- import cv2
73
- from modelscope.outputs import OutputKeys
74
- from modelscope.pipelines import pipeline
75
- from modelscope.utils.constant import Tasks
76
-
77
- img_colorization = pipeline(Tasks.image_colorization,
78
- model='damo/cv_ddcolor_image-colorization')
79
- img_path = 'https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/audrey_hepburn.jpg'
80
- result = img_colorization(img_path)
81
- cv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])
82
- ```
83
-
84
- ### 模型局限性以及可能的偏差
85
-
86
- - 本算法模型使用自然图像数据集进行训练,对于分布外场景(例如漫画等)可能产生不恰当的上色结果;
87
- - 对于低分辨率或包含明显噪声的图像,算法可能无法得到理想的生成效果。
88
-
89
- ## 训练数据介绍
90
-
91
- 模型使用公开数据集 [ImageNet](https://www.image-net.org/) 训练,其训练集包含 128 万张自然图像。
92
-
93
- ## 数据评估及结果
94
-
95
- 本算法主要在 [ImageNet](https://www.image-net.org/) 和 [COCO-Stuff](https://github.com/nightrome/cocostuff)上测试。
96
-
97
- | Val Name | FID | Colorfulness |
98
- |:-----------------:|:----:|:------------:|
99
- | ImageNet (val50k) | 3.92 | 38.26 |
100
- | ImageNet (val5k) | 0.96 | 38.65 |
101
- | COCO-Stuff | 5.18 | 38.48 |
102
-
103
- ## 引用
104
-
105
- 如果你觉得这个模型对你有所帮助,请考虑引用下面的相关论文:
106
-
107
- ```
108
- @article{kang2022ddcolor,
109
- title={DDColor: Towards Photo-Realistic and Semantic-Aware Image Colorization via Dual Decoders},
110
- author={Kang, Xiaoyang and Yang, Tao and Ouyang, Wenqi and Ren, Peiran and Li, Lingzhi and Xie, Xuansong},
111
- journal={arXiv preprint arXiv:2212.11613},
112
- year={2022}
113
- }
114
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
cv_ddcolor_image-colorization/configuration.json DELETED
@@ -1,67 +0,0 @@
1
- {
2
- "framework": "pytorch",
3
-
4
- "task": "image-colorization",
5
-
6
- "pipeline": {
7
- "type": "ddcolor-image-colorization"
8
- },
9
-
10
- "model": {
11
- "type": "ddcolor"
12
- },
13
-
14
- "dataset": {
15
- "name": "imagenet-val5k-image",
16
- "dataroot_gt": "val5k/",
17
- "filename_tmpl": "{}",
18
- "scale": 1,
19
- "gt_size": 256
20
- },
21
-
22
- "train": {
23
- "dataloader": {
24
- "batch_size_per_gpu": 4,
25
- "workers_per_gpu": 4,
26
- "shuffle": true
27
- },
28
- "optimizer": {
29
- "type": "AdamW",
30
- "lr": 1e-6,
31
- "weight_decay": 0.01,
32
- "betas": [0.9, 0.99]
33
- },
34
- "lr_scheduler": {
35
- "type": "CosineAnnealingLR",
36
- "T_max": 200000,
37
- "eta_min": 1e-7
38
- },
39
- "max_epochs": 2,
40
- "hooks": [{
41
- "type": "CheckpointHook",
42
- "interval": 1
43
- },
44
- {
45
- "type": "TextLoggerHook",
46
- "interval": 1
47
- },
48
- {
49
- "type": "IterTimerHook"
50
- },
51
- {
52
- "type": "EvaluationHook",
53
- "interval": 1
54
- }
55
- ]
56
- },
57
-
58
- "evaluation": {
59
- "dataloader": {
60
- "batch_size_per_gpu": 8,
61
- "workers_per_gpu": 1,
62
- "shuffle": false
63
- },
64
- "metrics": "image-colorization-metric"
65
- }
66
-
67
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
cv_ddcolor_image-colorization/pytorch_model.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:17c460d7e55b32a598370621d77173be59e03c24b0823f06821db23a50c263ce
3
- size 911950059
 
 
 
 
cv_ddcolor_image-colorization/resources/ddcolor_arch.jpg DELETED

Git LFS Details

  • SHA256: 56877be78eaadca79bccdf14bf1cc4e941a6817682c404ec63cdab1dd2f94ff6
  • Pointer size: 131 Bytes
  • Size of remote file: 240 kB
cv_ddcolor_image-colorization/resources/demo.jpg DELETED

Git LFS Details

  • SHA256: 137636b73f40742f1b25c9de630126fc216aef21459aeeda869b30bfc97843ab
  • Pointer size: 131 Bytes
  • Size of remote file: 204 kB
cv_ddcolor_image-colorization/resources/demo2.jpg DELETED
Binary file (97.2 kB)
 
cv_ddcolor_image-colorization/resources/demo3.jpg DELETED

Git LFS Details

  • SHA256: 6210433339fd74f268bff5af18492d16545f2207f2d5860ab84eb91bfb11bdc7
  • Pointer size: 131 Bytes
  • Size of remote file: 120 kB