tomz616 commited on
Commit
bb1f598
·
verified ·
1 Parent(s): 2e976cc
Files changed (9) hide show
  1. CONTRIBUTING.md +33 -0
  2. Dockerfile.cpu +30 -0
  3. Dockerfile.gpu +30 -0
  4. LICENSE +203 -0
  5. README.md +296 -0
  6. config.yml +83 -0
  7. pypi.sh +5 -0
  8. setup.cfg +2 -0
  9. setup.py +38 -0
CONTRIBUTING.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Contribution Guide
2
+
3
+ We welcome any contributions whether it's,
4
+
5
+ - Submitting feedback
6
+ - Fixing bugs
7
+ - Or implementing a new feature.
8
+
9
+ Please read this guide before making any contributions.
10
+
11
+ #### Submit Feedback
12
+ The feedback should be submitted by creating an issue at [GitHub issues](https://github.com/idealo/image-super-resolution/issues).
13
+ Select the related template (bug report, feature request, or custom) and add the corresponding labels.
14
+
15
+ #### Fix Bugs:
16
+ You may look through the [GitHub issues](https://github.com/idealo/image-super-resolution/issues) for bugs.
17
+
18
+ #### Implement Features
19
+ You may look through the [GitHub issues](https://github.com/idealo/image-super-resolution/issues) for feature requests.
20
+
21
+ ## Pull Requests (PR)
22
+ 1. Fork the repository and a create a new branch from the master branch.
23
+ 2. For bug fixes, add new tests and for new features please add changes to the documentation.
24
+ 3. Do a PR from your new branch to our `dev` branch of the original Image Super-Resolution repo.
25
+
26
+ ## Documentation
27
+ - Make sure any new function or class you introduce has proper docstrings.
28
+
29
+ ## Testing
30
+ - We use [pytest](https://docs.pytest.org/en/latest/) for our testing. Make sure to write tests for any new feature and/or bug fixes.
31
+
32
+ ## Main Contributor List
33
+ We maintain a list of main contributors to appreciate all the contributions.
Dockerfile.cpu ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM tensorflow/tensorflow:1.13.1-py3
2
+
3
+ # Install system packages
4
+ RUN apt-get update && apt-get install -y --no-install-recommends \
5
+ bzip2 \
6
+ g++ \
7
+ git \
8
+ graphviz \
9
+ libgl1-mesa-glx \
10
+ libhdf5-dev \
11
+ openmpi-bin \
12
+ screen \
13
+ wget && \
14
+ rm -rf /var/lib/apt/lists/* \
15
+ apt-get upgrade
16
+
17
+ ENV TENSOR_HOME /home/isr
18
+ WORKDIR $TENSOR_HOME
19
+
20
+ COPY ISR ./ISR
21
+ COPY scripts ./scripts
22
+ COPY weights ./weights
23
+ COPY config.yml ./
24
+ COPY setup.py ./
25
+
26
+ RUN pip install --upgrade pip
27
+ RUN pip install -e .
28
+
29
+ ENV PYTHONPATH ./ISR/:$PYTHONPATH
30
+ ENTRYPOINT ["sh", "./scripts/entrypoint.sh"]
Dockerfile.gpu ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM tensorflow/tensorflow:1.13.1-gpu-py3
2
+
3
+ # Install system packages
4
+ RUN apt-get update && apt-get install -y --no-install-recommends \
5
+ bzip2 \
6
+ g++ \
7
+ git \
8
+ graphviz \
9
+ libgl1-mesa-glx \
10
+ libhdf5-dev \
11
+ openmpi-bin \
12
+ screen \
13
+ wget && \
14
+ rm -rf /var/lib/apt/lists/* \
15
+ apt-get upgrade
16
+
17
+ ENV TENSOR_HOME /home/isr
18
+ WORKDIR $TENSOR_HOME
19
+
20
+ COPY ISR ./ISR
21
+ COPY scripts ./scripts
22
+ COPY weights ./weights
23
+ COPY config.yml ./
24
+ COPY setup.py ./
25
+
26
+ RUN pip install --upgrade pip
27
+ RUN pip install -e ".[gpu]" --ignore-installed
28
+
29
+ ENV PYTHONPATH ./ISR/:$PYTHONPATH
30
+ ENTRYPOINT ["sh", "./scripts/entrypoint.sh"]
LICENSE ADDED
@@ -0,0 +1,203 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright 2018 idealo internet GmbH. All rights reserved.
2
+
3
+ Apache License
4
+ Version 2.0, January 2004
5
+ http://www.apache.org/licenses/
6
+
7
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
8
+
9
+ 1. Definitions.
10
+
11
+ "License" shall mean the terms and conditions for use, reproduction,
12
+ and distribution as defined by Sections 1 through 9 of this document.
13
+
14
+ "Licensor" shall mean the copyright owner or entity authorized by
15
+ the copyright owner that is granting the License.
16
+
17
+ "Legal Entity" shall mean the union of the acting entity and all
18
+ other entities that control, are controlled by, or are under common
19
+ control with that entity. For the purposes of this definition,
20
+ "control" means (i) the power, direct or indirect, to cause the
21
+ direction or management of such entity, whether by contract or
22
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
23
+ outstanding shares, or (iii) beneficial ownership of such entity.
24
+
25
+ "You" (or "Your") shall mean an individual or Legal Entity
26
+ exercising permissions granted by this License.
27
+
28
+ "Source" form shall mean the preferred form for making modifications,
29
+ including but not limited to software source code, documentation
30
+ source, and configuration files.
31
+
32
+ "Object" form shall mean any form resulting from mechanical
33
+ transformation or translation of a Source form, including but
34
+ not limited to compiled object code, generated documentation,
35
+ and conversions to other media types.
36
+
37
+ "Work" shall mean the work of authorship, whether in Source or
38
+ Object form, made available under the License, as indicated by a
39
+ copyright notice that is included in or attached to the work
40
+ (an example is provided in the Appendix below).
41
+
42
+ "Derivative Works" shall mean any work, whether in Source or Object
43
+ form, that is based on (or derived from) the Work and for which the
44
+ editorial revisions, annotations, elaborations, or other modifications
45
+ represent, as a whole, an original work of authorship. For the purposes
46
+ of this License, Derivative Works shall not include works that remain
47
+ separable from, or merely link (or bind by name) to the interfaces of,
48
+ the Work and Derivative Works thereof.
49
+
50
+ "Contribution" shall mean any work of authorship, including
51
+ the original version of the Work and any modifications or additions
52
+ to that Work or Derivative Works thereof, that is intentionally
53
+ submitted to Licensor for inclusion in the Work by the copyright owner
54
+ or by an individual or Legal Entity authorized to submit on behalf of
55
+ the copyright owner. For the purposes of this definition, "submitted"
56
+ means any form of electronic, verbal, or written communication sent
57
+ to the Licensor or its representatives, including but not limited to
58
+ communication on electronic mailing lists, source code control systems,
59
+ and issue tracking systems that are managed by, or on behalf of, the
60
+ Licensor for the purpose of discussing and improving the Work, but
61
+ excluding communication that is conspicuously marked or otherwise
62
+ designated in writing by the copyright owner as "Not a Contribution."
63
+
64
+ "Contributor" shall mean Licensor and any individual or Legal Entity
65
+ on behalf of whom a Contribution has been received by Licensor and
66
+ subsequently incorporated within the Work.
67
+
68
+ 2. Grant of Copyright License. Subject to the terms and conditions of
69
+ this License, each Contributor hereby grants to You a perpetual,
70
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
71
+ copyright license to reproduce, prepare Derivative Works of,
72
+ publicly display, publicly perform, sublicense, and distribute the
73
+ Work and such Derivative Works in Source or Object form.
74
+
75
+ 3. Grant of Patent License. Subject to the terms and conditions of
76
+ this License, each Contributor hereby grants to You a perpetual,
77
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
78
+ (except as stated in this section) patent license to make, have made,
79
+ use, offer to sell, sell, import, and otherwise transfer the Work,
80
+ where such license applies only to those patent claims licensable
81
+ by such Contributor that are necessarily infringed by their
82
+ Contribution(s) alone or by combination of their Contribution(s)
83
+ with the Work to which such Contribution(s) was submitted. If You
84
+ institute patent litigation against any entity (including a
85
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
86
+ or a Contribution incorporated within the Work constitutes direct
87
+ or contributory patent infringement, then any patent licenses
88
+ granted to You under this License for that Work shall terminate
89
+ as of the date such litigation is filed.
90
+
91
+ 4. Redistribution. You may reproduce and distribute copies of the
92
+ Work or Derivative Works thereof in any medium, with or without
93
+ modifications, and in Source or Object form, provided that You
94
+ meet the following conditions:
95
+
96
+ (a) You must give any other recipients of the Work or
97
+ Derivative Works a copy of this License; and
98
+
99
+ (b) You must cause any modified files to carry prominent notices
100
+ stating that You changed the files; and
101
+
102
+ (c) You must retain, in the Source form of any Derivative Works
103
+ that You distribute, all copyright, patent, trademark, and
104
+ attribution notices from the Source form of the Work,
105
+ excluding those notices that do not pertain to any part of
106
+ the Derivative Works; and
107
+
108
+ (d) If the Work includes a "NOTICE" text file as part of its
109
+ distribution, then any Derivative Works that You distribute must
110
+ include a readable copy of the attribution notices contained
111
+ within such NOTICE file, excluding those notices that do not
112
+ pertain to any part of the Derivative Works, in at least one
113
+ of the following places: within a NOTICE text file distributed
114
+ as part of the Derivative Works; within the Source form or
115
+ documentation, if provided along with the Derivative Works; or,
116
+ within a display generated by the Derivative Works, if and
117
+ wherever such third-party notices normally appear. The contents
118
+ of the NOTICE file are for informational purposes only and
119
+ do not modify the License. You may add Your own attribution
120
+ notices within Derivative Works that You distribute, alongside
121
+ or as an addendum to the NOTICE text from the Work, provided
122
+ that such additional attribution notices cannot be construed
123
+ as modifying the License.
124
+
125
+ You may add Your own copyright statement to Your modifications and
126
+ may provide additional or different license terms and conditions
127
+ for use, reproduction, or distribution of Your modifications, or
128
+ for any such Derivative Works as a whole, provided Your use,
129
+ reproduction, and distribution of the Work otherwise complies with
130
+ the conditions stated in this License.
131
+
132
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
133
+ any Contribution intentionally submitted for inclusion in the Work
134
+ by You to the Licensor shall be under the terms and conditions of
135
+ this License, without any additional terms or conditions.
136
+ Notwithstanding the above, nothing herein shall supersede or modify
137
+ the terms of any separate license agreement you may have executed
138
+ with Licensor regarding such Contributions.
139
+
140
+ 6. Trademarks. This License does not grant permission to use the trade
141
+ names, trademarks, service marks, or product names of the Licensor,
142
+ except as required for reasonable and customary use in describing the
143
+ origin of the Work and reproducing the content of the NOTICE file.
144
+
145
+ 7. Disclaimer of Warranty. Unless required by applicable law or
146
+ agreed to in writing, Licensor provides the Work (and each
147
+ Contributor provides its Contributions) on an "AS IS" BASIS,
148
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
149
+ implied, including, without limitation, any warranties or conditions
150
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
151
+ PARTICULAR PURPOSE. You are solely responsible for determining the
152
+ appropriateness of using or redistributing the Work and assume any
153
+ risks associated with Your exercise of permissions under this License.
154
+
155
+ 8. Limitation of Liability. In no event and under no legal theory,
156
+ whether in tort (including negligence), contract, or otherwise,
157
+ unless required by applicable law (such as deliberate and grossly
158
+ negligent acts) or agreed to in writing, shall any Contributor be
159
+ liable to You for damages, including any direct, indirect, special,
160
+ incidental, or consequential damages of any character arising as a
161
+ result of this License or out of the use or inability to use the
162
+ Work (including but not limited to damages for loss of goodwill,
163
+ work stoppage, computer failure or malfunction, or any and all
164
+ other commercial damages or losses), even if such Contributor
165
+ has been advised of the possibility of such damages.
166
+
167
+ 9. Accepting Warranty or Additional Liability. While redistributing
168
+ the Work or Derivative Works thereof, You may choose to offer,
169
+ and charge a fee for, acceptance of support, warranty, indemnity,
170
+ or other liability obligations and/or rights consistent with this
171
+ License. However, in accepting such obligations, You may act only
172
+ on Your own behalf and on Your sole responsibility, not on behalf
173
+ of any other Contributor, and only if You agree to indemnify,
174
+ defend, and hold each Contributor harmless for any liability
175
+ incurred by, or claims asserted against, such Contributor by reason
176
+ of your accepting any such warranty or additional liability.
177
+
178
+ END OF TERMS AND CONDITIONS
179
+
180
+ APPENDIX: How to apply the Apache License to your work.
181
+
182
+ To apply the Apache License to your work, attach the following
183
+ boilerplate notice, with the fields enclosed by brackets "[]"
184
+ replaced with your own identifying information. (Don't include
185
+ the brackets!) The text should be enclosed in the appropriate
186
+ comment syntax for the file format. We also recommend that a
187
+ file or class name and description of purpose be included on the
188
+ same "printed page" as the copyright notice for easier
189
+ identification within third-party archives.
190
+
191
+ Copyright [yyyy] [name of copyright owner]
192
+
193
+ Licensed under the Apache License, Version 2.0 (the "License");
194
+ you may not use this file except in compliance with the License.
195
+ You may obtain a copy of the License at
196
+
197
+ http://www.apache.org/licenses/LICENSE-2.0
198
+
199
+ Unless required by applicable law or agreed to in writing, software
200
+ distributed under the License is distributed on an "AS IS" BASIS,
201
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
202
+ See the License for the specific language governing permissions and
203
+ limitations under the License.
README.md ADDED
@@ -0,0 +1,296 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Image Super-Resolution (ISR)
2
+
3
+ <img src="figures/butterfly.png">
4
+
5
+ [![Build Status](https://travis-ci.org/idealo/image-super-resolution.svg?branch=master)](https://travis-ci.org/idealo/image-super-resolution)
6
+ [![Docs](https://img.shields.io/badge/docs-online-brightgreen)](https://idealo.github.io/image-super-resolution/)
7
+ [![License](https://img.shields.io/badge/License-Apache%202.0-orange.svg)](https://github.com/idealo/image-super-resolution/blob/master/LICENSE)
8
+
9
+ The goal of this project is to upscale and improve the quality of low resolution images.
10
+
11
+ Since the code is no longer actively maintained, it will be archived on 2025-01-03.
12
+
13
+ This project contains Keras implementations of different Residual Dense Networks for Single Image Super-Resolution (ISR) as well as scripts to train these networks using content and adversarial loss components.
14
+
15
+ The implemented networks include:
16
+
17
+ - The super-scaling Residual Dense Network described in [Residual Dense Network for Image Super-Resolution](https://arxiv.org/abs/1802.08797) (Zhang et al. 2018)
18
+ - The super-scaling Residual in Residual Dense Network described in [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://arxiv.org/abs/1809.00219) (Wang et al. 2018)
19
+ - A multi-output version of the Keras VGG19 network for deep features extraction used in the perceptual loss
20
+ - A custom discriminator network based on the one described in [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://arxiv.org/abs/1609.04802) (SRGANS, Ledig et al. 2017)
21
+
22
+ Read the full documentation at: [https://idealo.github.io/image-super-resolution/](https://idealo.github.io/image-super-resolution/).
23
+
24
+ [Docker scripts](https://idealo.github.io/image-super-resolution/tutorials/docker/) and [Google Colab notebooks](https://github.com/idealo/image-super-resolution/tree/master/notebooks) are available to carry training and prediction. Also, we provide scripts to facilitate training on the cloud with AWS and [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) with only a few commands.
25
+
26
+ ISR is compatible with Python 3.6 and is distributed under the Apache 2.0 license. We welcome any kind of contribution. If you wish to contribute, please see the [Contribute](#contribute) section.
27
+
28
+ ## Contents
29
+ - [Pre-trained networks](#pre-trained-networks)
30
+ - [Installation](#installation)
31
+ - [Usage](#usage)
32
+ - [Additional Information](#additional-information)
33
+ - [Contribute](#contribute)
34
+ - [Citation](#citation)
35
+ - [Maintainers](#maintainers)
36
+ - [License](#copyright)
37
+
38
+ ## Troubleshooting
39
+ ### Training not delivering good/patchy results
40
+ When training your own model, start with only PSNR loss (50+ epochs, depending on the dataset) and only then introduce GANS and feature loss. This can be controlled by the loss weights argument.
41
+
42
+ This is just sample, you will need to tune these parameters.
43
+
44
+ PSNR only:
45
+ ```
46
+ loss_weights = {
47
+ 'generator': 1.0,
48
+ 'feature_extractor': 0.0,
49
+ 'discriminator': 0.00
50
+ }
51
+ ```
52
+
53
+ Later:
54
+ ```
55
+ loss_weights = {
56
+ 'generator': 0.0,
57
+ 'feature_extractor': 0.0833,
58
+ 'discriminator': 0.01
59
+ }
60
+ ```
61
+ ### Weights loading
62
+ If you are having trouble loading your own weights or the pre-trained weights (`AttributeError: 'str' object has no attribute 'decode'`), try:
63
+ ```bash
64
+ pip install 'h5py==2.10.0' --force-reinstall
65
+ ```
66
+ [Issue](https://github.com/idealo/image-super-resolution/issues/197#issue-877826405)
67
+
68
+ ## Pre-trained networks
69
+
70
+ The weights used to produced these images are available directly when creating the model object.
71
+
72
+ Currently 4 models are available:
73
+ - RDN: psnr-large, psnr-small, noise-cancel
74
+ - RRDN: gans
75
+
76
+ Example usage:
77
+
78
+ ```
79
+ model = RRDN(weights='gans')
80
+ ```
81
+
82
+ The network parameters will be automatically chosen.
83
+ (see [Additional Information](#additional-information)).
84
+
85
+ #### Basic model
86
+ RDN model, PSNR driven, choose the option ```weights='psnr-large'``` or ```weights='psnr-small'``` when creating a RDN model.
87
+
88
+ |![butterfly-sample](figures/butterfly_comparison_SR_baseline.png)|
89
+ |:--:|
90
+ | Low resolution image (left), ISR output (center), bicubic scaling (right). Click to zoom. |
91
+ #### GANS model
92
+ RRDN model, trained with Adversarial and VGG features losses, choose the option ```weights='gans'``` when creating a RRDN model.
93
+
94
+ |![baboon-comparison](figures/baboon-compare.png)|
95
+ |:--:|
96
+ | RRDN GANS model (left), bicubic upscaling (right). |
97
+ -> [more detailed comparison](http://www.framecompare.com/screenshotcomparison/PGZPNNNX)
98
+
99
+ #### Artefact Cancelling GANS model
100
+ RDN model, trained with Adversarial and VGG features losses, choose the option ```weights='noise-cancel'``` when creating a RDN model.
101
+
102
+ |![temple-comparison](figures/temple_comparison.png)|
103
+ |:--:|
104
+ | Standard vs GANS model. Click to zoom. |
105
+
106
+
107
+ |![sandal-comparison](figures/sandal-compare.png)|
108
+ |:--:|
109
+ | RDN GANS artefact cancelling model (left), RDN standard PSNR driven model (right). |
110
+ -> [more detailed comparison](http://www.framecompare.com/screenshotcomparison/2ECCNNNU)
111
+
112
+
113
+ ## Installation
114
+ There are two ways to install the Image Super-Resolution package:
115
+
116
+ - Install ISR from PyPI (recommended):
117
+ ```
118
+ pip install ISR
119
+ ```
120
+ - Install ISR from the GitHub source:
121
+ ```
122
+ git clone https://github.com/idealo/image-super-resolution
123
+ cd image-super-resolution
124
+ python setup.py install
125
+ ```
126
+
127
+ ## Usage
128
+
129
+ ### Prediction
130
+
131
+ Load image and prepare it
132
+ ```python
133
+ import numpy as np
134
+ from PIL import Image
135
+
136
+ img = Image.open('data/input/test_images/sample_image.jpg')
137
+ lr_img = np.array(img)
138
+ ```
139
+
140
+ Load a pre-trained model and run prediction (check the prediction tutorial under notebooks for more details)
141
+ ```python
142
+ from ISR.models import RDN
143
+
144
+ rdn = RDN(weights='psnr-small')
145
+ sr_img = rdn.predict(lr_img)
146
+ Image.fromarray(sr_img)
147
+ ```
148
+
149
+ #### Large image inference
150
+ To predict on large images and avoid memory allocation errors, use the `by_patch_of_size` option for the predict method, for instance
151
+ ```
152
+ sr_img = model.predict(image, by_patch_of_size=50)
153
+ ```
154
+ Check the documentation of the `ImageModel` class for further details.
155
+
156
+ ### Training
157
+
158
+ Create the models
159
+ ```python
160
+ from ISR.models import RRDN
161
+ from ISR.models import Discriminator
162
+ from ISR.models import Cut_VGG19
163
+
164
+ lr_train_patch_size = 40
165
+ layers_to_extract = [5, 9]
166
+ scale = 2
167
+ hr_train_patch_size = lr_train_patch_size * scale
168
+
169
+ rrdn = RRDN(arch_params={'C':4, 'D':3, 'G':64, 'G0':64, 'T':10, 'x':scale}, patch_size=lr_train_patch_size)
170
+ f_ext = Cut_VGG19(patch_size=hr_train_patch_size, layers_to_extract=layers_to_extract)
171
+ discr = Discriminator(patch_size=hr_train_patch_size, kernel_size=3)
172
+ ```
173
+
174
+ Create a Trainer object using the desired settings and give it the models (`f_ext` and `discr` are optional)
175
+ ```python
176
+ from ISR.train import Trainer
177
+ loss_weights = {
178
+ 'generator': 0.0,
179
+ 'feature_extractor': 0.0833,
180
+ 'discriminator': 0.01
181
+ }
182
+ losses = {
183
+ 'generator': 'mae',
184
+ 'feature_extractor': 'mse',
185
+ 'discriminator': 'binary_crossentropy'
186
+ }
187
+
188
+ log_dirs = {'logs': './logs', 'weights': './weights'}
189
+
190
+ learning_rate = {'initial_value': 0.0004, 'decay_factor': 0.5, 'decay_frequency': 30}
191
+
192
+ flatness = {'min': 0.0, 'max': 0.15, 'increase': 0.01, 'increase_frequency': 5}
193
+
194
+ trainer = Trainer(
195
+ generator=rrdn,
196
+ discriminator=discr,
197
+ feature_extractor=f_ext,
198
+ lr_train_dir='low_res/training/images',
199
+ hr_train_dir='high_res/training/images',
200
+ lr_valid_dir='low_res/validation/images',
201
+ hr_valid_dir='high_res/validation/images',
202
+ loss_weights=loss_weights,
203
+ learning_rate=learning_rate,
204
+ flatness=flatness,
205
+ dataname='image_dataset',
206
+ log_dirs=log_dirs,
207
+ weights_generator=None,
208
+ weights_discriminator=None,
209
+ n_validation=40,
210
+ )
211
+ ```
212
+
213
+ Start training
214
+ ```python
215
+ trainer.train(
216
+ epochs=80,
217
+ steps_per_epoch=500,
218
+ batch_size=16,
219
+ monitored_metrics={'val_PSNR_Y': 'max'}
220
+ )
221
+ ```
222
+
223
+ ## Additional Information
224
+ You can read about how we trained these network weights in our Medium posts:
225
+ - part 1: [A deep learning based magnifying glass](https://medium.com/idealo-tech-blog/a-deep-learning-based-magnifying-glass-dae1f565c359)
226
+ - part 2: [Zoom in... enhance](https://medium.com/idealo-tech-blog/zoom-in-enhance-a-deep-learning-based-magnifying-glass-part-2-c021f98ebede
227
+ )
228
+
229
+ ### RDN Pre-trained weights
230
+ The weights of the RDN network trained on the [DIV2K dataset](https://data.vision.ee.ethz.ch/cvl/DIV2K) are available in ```weights/sample_weights/rdn-C6-D20-G64-G064-x2/PSNR-driven/rdn-C6-D20-G64-G064-x2_PSNR_epoch086.hdf5```. <br>
231
+ The model was trained using ```C=6, D=20, G=64, G0=64``` as parameters (see architecture for details) for 86 epochs of 1000 batches of 8 32x32 augmented patches taken from LR images.
232
+
233
+ The artefact can cancelling weights obtained with a combination of different training sessions using different datasets and perceptual loss with VGG19 and GAN can be found at `weights/sample_weights/rdn-C6-D20-G64-G064-x2/ArtefactCancelling/rdn-C6-D20-G64-G064-x2_ArtefactCancelling_epoch219.hdf5`
234
+ We recommend using these weights only when cancelling compression artefacts is a desirable effect.
235
+
236
+ ### RDN Network architecture
237
+ The main parameters of the architecture structure are:
238
+ - D - number of Residual Dense Blocks (RDB)
239
+ - C - number of convolutional layers stacked inside a RDB
240
+ - G - number of feature maps of each convolutional layers inside the RDBs
241
+ - G0 - number of feature maps for convolutions outside of RDBs and of each RBD output
242
+
243
+ <img src="figures/RDN.png" width="600">
244
+ <br>
245
+
246
+ <img src="figures/RDB.png" width="600">
247
+
248
+ source: [Residual Dense Network for Image Super-Resolution](https://arxiv.org/abs/1802.08797)
249
+
250
+ ### RRDN Network architecture
251
+ The main parameters of the architecture structure are:
252
+ - T - number of Residual in Residual Dense Blocks (RRDB)
253
+ - D - number of Residual Dense Blocks (RDB) insider each RRDB
254
+ - C - number of convolutional layers stacked inside a RDB
255
+ - G - number of feature maps of each convolutional layers inside the RDBs
256
+ - G0 - number of feature maps for convolutions outside of RDBs and of each RBD output
257
+
258
+ <img src="figures/RRDN.jpg" width="600">
259
+ <br>
260
+
261
+ <img src="figures/RRDB.png" width="600">
262
+
263
+ source: [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://arxiv.org/abs/1809.00219)
264
+
265
+ ## Contribute
266
+ We welcome all kinds of contributions, models trained on different datasets, new model architectures and/or hyperparameters combinations that improve the performance of the currently published model.
267
+
268
+ Will publish the performances of new models in this repository.
269
+
270
+ See the [Contribution](CONTRIBUTING.md) guide for more details.
271
+
272
+ #### Bump version
273
+ To bump up the version, use
274
+ ```
275
+ bumpversion {part} setup.py
276
+ ```
277
+
278
+ ## Citation
279
+ Please cite our work in your publications if it helps your research.
280
+
281
+ ```BibTeX
282
+ @misc{cardinale2018isr,
283
+ title={ISR},
284
+ author={Francesco Cardinale et al.},
285
+ year={2018},
286
+ howpublished={\url{https://github.com/idealo/image-super-resolution}},
287
+ }
288
+ ```
289
+
290
+ ## Maintainers
291
+ * Francesco Cardinale, github: [cfrancesco](https://github.com/cfrancesco)
292
+ * Dat Tran, github: [datitran](https://github.com/datitran)
293
+
294
+ ## Copyright
295
+
296
+ See [LICENSE](LICENSE) for details.
config.yml ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ default:
3
+ generator: rdn
4
+ feature_extractor: false
5
+ discriminator: false
6
+ training_set: div2k
7
+ test_set: sample
8
+ log_dirs:
9
+ logs: ./logs
10
+ weights: ./weights
11
+ feature_extractor:
12
+ vgg19:
13
+ layers_to_extract:
14
+ - 5
15
+ - 9
16
+ generators:
17
+ rrdn:
18
+ C: 4
19
+ D: 3
20
+ G: 32
21
+ G0: 32
22
+ T: 4
23
+ x: 4
24
+ rdn:
25
+ C: 6
26
+ D: 20
27
+ G: 64
28
+ G0: 64
29
+ x: 2
30
+ loss_weights:
31
+ generator: 1.0
32
+ feature_extractor: 0.0833
33
+ discriminator: 0.01
34
+ losses:
35
+ generator: mae
36
+ discriminator: binary_crossentropy
37
+ feature_extractor: mse
38
+ session:
39
+ prediction:
40
+ patch_size:
41
+ training:
42
+ steps_per_epoch: 1000
43
+ patch_size: 32
44
+ batch_size: 16
45
+ epochs: 300
46
+ n_validation_samples: 100
47
+ learning_rate:
48
+ initial_value: 0.0004
49
+ decay_frequency: 50
50
+ decay_factor: 0.5
51
+ fallback_save_every_n_epochs: 2
52
+ flatness:
53
+ min: 0.0
54
+ increase_frequency: null
55
+ increase: 0.0
56
+ max: 0.0
57
+ metrics:
58
+ generator: PSNR_Y
59
+ monitored_metrics:
60
+ val_loss: min
61
+ val_PSNR_Y: max
62
+ adam_optimizer:
63
+ beta1: 0.9
64
+ beta2: 0.999
65
+ epsilon: null
66
+ test_sets:
67
+ sample: ./data/input/sample
68
+ training_sets:
69
+ custom data:
70
+ lr_train_dir: ./data/custom/lr/train
71
+ hr_train_dir: ./data/custom/hr/train
72
+ lr_valid_dir: ./data/custom/lr/validation
73
+ hr_valid_dir: ./data/custom/hr/validation
74
+ data_name: custom
75
+ div2k:
76
+ lr_train_dir: ./data/DIV2K/DIV2K_train_LR_bicubic/X2
77
+ hr_train_dir: ./data/DIV2K/DIV2K_train_HR
78
+ lr_valid_dir: ./data/DIV2K/DIV2K_valid_LR_bicubic/X2
79
+ hr_valid_dir: ./data/DIV2K/DIV2K_valid_HR
80
+ data_name: div2k
81
+ weights_paths:
82
+ discriminator:
83
+ generator:
pypi.sh ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+
3
+ rm -rf dist/
4
+ python setup.py sdist bdist_wheel
5
+ twine upload dist/*
setup.cfg ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ [metadata]
2
+ description-file = README.md
setup.py ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from setuptools import setup, find_packages
2
+
3
+ long_description = '''
4
+ ISR (Image Super-Resolution) is a library to upscale and improve the quality of low resolution images.
5
+
6
+ Read the documentation at: https://idealo.github.io/image-super-resolution/
7
+
8
+ ISR is compatible with Python 3.6 and is distributed under the Apache 2.0 license.
9
+ '''
10
+
11
+ setup(
12
+ name='ISR',
13
+ version='2.2.0',
14
+ author='Francesco Cardinale',
15
+ author_email='testadicardi@gmail.com',
16
+ description='Image Super Resolution',
17
+ long_description=long_description,
18
+ license='Apache 2.0',
19
+ install_requires=['imageio', 'numpy', 'tensorflow==2.*', 'tqdm', 'pyaml', 'h5py==2.10.0'],
20
+ extras_require={
21
+ 'tests': ['pytest==4.3.0', 'pytest-cov==2.6.1'],
22
+ 'docs': ['mkdocs==1.0.4', 'mkdocs-material==4.0.2'],
23
+ 'gpu': ['tensorflow-gpu==2.*'],
24
+ 'dev': ['bumpversion==0.5.3'],
25
+ },
26
+ classifiers=[
27
+ 'Development Status :: 4 - Beta',
28
+ 'Intended Audience :: Developers',
29
+ 'Intended Audience :: Education',
30
+ 'Intended Audience :: Science/Research',
31
+ 'License :: OSI Approved :: Apache Software License',
32
+ 'Programming Language :: Python :: 3',
33
+ 'Programming Language :: Python :: 3.6',
34
+ 'Topic :: Software Development :: Libraries',
35
+ 'Topic :: Software Development :: Libraries :: Python Modules',
36
+ ],
37
+ packages=find_packages(exclude=('tests',)),
38
+ )