Commit ·
72473b5
0
Parent(s):
Duplicate from OAOA/InvSR
Browse filesCo-authored-by: Zongsheng Yue <OAOA@users.noreply.huggingface.co>
- .gitattributes +35 -0
- LICENSE +35 -0
- README.md +54 -0
- discriminator_sd_turbo_v5.pth +3 -0
- noise_predictor_sd_turbo_v5.pth +3 -0
- noise_predictor_sd_turbo_v5_diftune.pth +3 -0
- vgg16_sdturbo_lpips.pth +3 -0
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
S-Lab License 1.0
|
| 2 |
+
|
| 3 |
+
Copyright 2024 S-Lab
|
| 4 |
+
|
| 5 |
+
Redistribution and use for non-commercial purpose in source and
|
| 6 |
+
binary forms, with or without modification, are permitted provided
|
| 7 |
+
that the following conditions are met:
|
| 8 |
+
|
| 9 |
+
1. Redistributions of source code must retain the above copyright
|
| 10 |
+
notice, this list of conditions and the following disclaimer.
|
| 11 |
+
|
| 12 |
+
2. Redistributions in binary form must reproduce the above copyright
|
| 13 |
+
notice, this list of conditions and the following disclaimer in
|
| 14 |
+
the documentation and/or other materials provided with the
|
| 15 |
+
distribution.
|
| 16 |
+
|
| 17 |
+
3. Neither the name of the copyright holder nor the names of its
|
| 18 |
+
contributors may be used to endorse or promote products derived
|
| 19 |
+
from this software without specific prior written permission.
|
| 20 |
+
|
| 21 |
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
| 22 |
+
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
| 23 |
+
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
| 24 |
+
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
| 25 |
+
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
| 26 |
+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
| 27 |
+
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
| 28 |
+
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
| 29 |
+
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
| 30 |
+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
| 31 |
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
| 32 |
+
|
| 33 |
+
In the event that redistribution and/or use for commercial purpose in
|
| 34 |
+
source or binary forms, with or without modification is required,
|
| 35 |
+
please contact the contributor(s) of the work.
|
README.md
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: license
|
| 4 |
+
license_link: LICENSE
|
| 5 |
+
pipeline_tag: image-to-image
|
| 6 |
+
tags:
|
| 7 |
+
- Image Super-resolution
|
| 8 |
+
- Diffusion Inversion
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# InvSR Model Card
|
| 12 |
+
This model card focuses on the models associated with the InvSR project, which is available [here](https://github.com/zsyOAOA/InvSR).
|
| 13 |
+
|
| 14 |
+
## Model Details
|
| 15 |
+
- **Developed by:** Zongsheng Yue
|
| 16 |
+
- **Model type:** Arbitrary-steps Image Super-resolution via Diffusion Inversion
|
| 17 |
+
- **Model Description:** This is the model used in [Paper](https://arxiv.org/abs/2412.09013).
|
| 18 |
+
- **Resources for more information:** [GitHub Repository](https://github.com/zsyOAOA/InvSR).
|
| 19 |
+
- **Cite as:**
|
| 20 |
+
|
| 21 |
+
@article{yue2024invSR,
|
| 22 |
+
author = {Zongsheng Yue, Kang Liao, Chen Change Loy},
|
| 23 |
+
title = {Arbitrary-steps Image Super-resolution via Diffusion Inversion},
|
| 24 |
+
journal = {arXiv preprint arXiv:2412.09013},
|
| 25 |
+
year = {2024},
|
| 26 |
+
}
|
| 27 |
+
|
| 28 |
+
## Limitations and Bias
|
| 29 |
+
|
| 30 |
+
### Limitations
|
| 31 |
+
|
| 32 |
+
- InvSR requires a tiled operation for generating a high-resolution image, which would largely increase the inference time.
|
| 33 |
+
- InvSR sometimes cannot keep 100% fidelity due to its generative nature.
|
| 34 |
+
- InvSR sometimes cannot generate perfect details under complex real-world scenarios.
|
| 35 |
+
|
| 36 |
+
### Bias
|
| 37 |
+
While our model is based on a pre-trained SD-Turbo model, currently we do not observe obvious bias in generated results.
|
| 38 |
+
|
| 39 |
+
## Training
|
| 40 |
+
|
| 41 |
+
**Training Data**
|
| 42 |
+
The model developer used the following dataset for training the model:
|
| 43 |
+
|
| 44 |
+
- Our model is finetuned on [LSDIR](https://data.vision.ee.ethz.ch/yawli/index.html) + 20K samples from FFHQ datasets.
|
| 45 |
+
|
| 46 |
+
**Training Procedure**
|
| 47 |
+
InvSR achieves the goal of image super-resolution via diffusion inversion technique on [SD-Turbo](https://huggingface.co/stabilityai/sd-turbo), detailed training pipelines can be found in our GitHub [repo](https://github.com/zsyOAOA/InvSR).
|
| 48 |
+
|
| 49 |
+
We currently provide the following checkpoints:
|
| 50 |
+
|
| 51 |
+
- [noise_predictor_sd_turbo_v5.pth](https://huggingface.co/OAOA/InvSR/blob/main/noise_predictor_sd_turbo_v5.pth): Noise estimation network trained for [SD-Turbo](https://huggingface.co/stabilityai/sd-turbo).
|
| 52 |
+
|
| 53 |
+
## Evaluation Results
|
| 54 |
+
See [Paper](https://arxiv.org/abs/2412.09013) for details.
|
discriminator_sd_turbo_v5.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fc37b3a5e0741d544038c55f9ef51c7603207fb779c273e5328d0af7626100ac
|
| 3 |
+
size 448540591
|
noise_predictor_sd_turbo_v5.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e1a03b0f447d6f2687815b0db45ff753c9fae87d6640bfadda1435b1f30f9415
|
| 3 |
+
size 135417810
|
noise_predictor_sd_turbo_v5_diftune.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:42ae029064e6c3199c820cff0306505878d938f9c6fe7e8b25dcc7d446849cc8
|
| 3 |
+
size 135417810
|
vgg16_sdturbo_lpips.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eb65c53886a3ca41f0fddecab142d79b98bc2840e0561b983d80d8034e83f8b3
|
| 3 |
+
size 58880967
|