amd/ryzenai-realesrgan

RealESRGAN for 4x Single Image Super Resolution

We provide 4x super-resolution models at multiple resolutions (128x128, 256x256, 512x512, 1024x1024). This is a lightweight version with reduced feature channels and fewer stacked blocks for improved efficiency.

It was introduced in the paper Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data by Xintao Wang et al. at ICCVW 2021.

We have developed a modified version optimized for AMD Ryzen AI.

Model description

RealESRGAN (Real-Enhanced Super-Resolution Generative Adversarial Network) is a deep learning model for 4x single image super-resolution.

Intended uses & limitations

You can use this model for single image super resolution tasks. See the model hub for all available realesrgan models.

How to use

Installation

# inference only
pip install -r requirements-infer.txt
# inference & evaluation
pip install -r requirements-eval.txt

Data Preparation (optional: for evaluation)

Run python download_edsr_benchmark.py. This script will download the EDSR benchmark dataset and automatically extract it into the datasets/ directory.

You can also download the DIV2K validation set with python download_div2k.py, which will download and extract DIV2K into datasets/ as well.

After downloading and extraction, the datasets/ directory should look like this:

datasets
  └──DIV2K_valid_HR
  └──DIV2K_valid_LR_bicubic/X4
  └──edsr_benchmark
      └── B100
            └── HR
              ├── 3096.png
              ├── ...
            └── LR_bicubic/X4
              ├── 3096x4.png
              ├── ...
      └── Set5
            └── HR
              ├── baby.png
              ├── ...
            └── LR_bicubic/X4
              ├── babyx4.png
              ├── ...

Test & Evaluation

• Run inference on images

python onnx_inference.py --onnx onnx-models/realesrgan_nchw_256x256_u8s8.onnx --input /Path/To/Image --out-dir outputs
python onnx_inference.py --onnx onnx-models/realesrgan_nchw_256x256_fp32.onnx --input /Path/To/Image --out-dir outputs

Arguments:

--input: Accepts either a single image file path or a directory path. If it's a file, the script will process that image only. If it's a directory, the script will recursively scan for .png, .jpg, and .jpeg files and process all of them.

--out-dir: Output directory where the restored images will be saved.

• Evaluate the quantized model

# eval on Set5, enable -clean option will remove generated sr images.
python onnx_eval.py \
  --onnx onnx-models/realesrgan_nchw_256x256_u8s8.onnx \
  --hq-dir datasets/edsr_benchmark/Set5/HR \
  --lq-dir datasets/edsr_benchmark/Set5/LR_bicubic/X4 \
  --out-dir outputs/u8s8-Set5 -clean

# eval on Set14
python onnx_eval.py \
  --onnx onnx-models/realesrgan_nchw_256x256_u8s8.onnx \
  --hq-dir datasets/edsr_benchmark/Set14/HR \
  --lq-dir datasets/edsr_benchmark/Set14/LR_bicubic/X4 \
  --out-dir outputs/u8s8-Set14 -clean

# eval on B100
python onnx_eval.py \
  --onnx onnx-models/realesrgan_nchw_256x256_u8s8.onnx \
  --hq-dir datasets/edsr_benchmark/B100/HR \
  --lq-dir datasets/edsr_benchmark/B100/LR_bicubic/X4 \
  --out-dir outputs/u8s8-B100 -clean

# eval on Urban100
python onnx_eval.py \
  --onnx onnx-models/realesrgan_nchw_256x256_u8s8.onnx \
  --hq-dir datasets/edsr_benchmark/Urban100/HR \
  --lq-dir datasets/edsr_benchmark/Urban100/LR_bicubic/X4 \
  --out-dir outputs/u8s8-Urban100 -clean

# eval on DIV2K
python onnx_eval.py \
  --onnx onnx-models/realesrgan_nchw_256x256_u8s8.onnx \
  --hq-dir datasets/DIV2K_valid_HR \
  --lq-dir datasets/DIV2K_valid_LR_bicubic/X4 \
  --out-dir outputs/u8s8-DIV2K -clean

Performance

Model		Set5			Set14			B100			Urban100			DIV2K
	PSNR(↑)	MS_SSIM(↑)	FID(↓)	PSNR(↑)	MS_SSIM(↑)	FID(↓)	PSNR(↑)	MS_SSIM (↑)	FID(↓)	PSNR(↑)	MS_SSIM(↑)	FID(↓)	PSNR(↑)	MS_SSIM(↑)	FID(↓)
128x128(fp32)	23.43	0.9346	114.31	22.38	0.8928	141.12	23.17	0.8804	134.00	20.02	0.8813	52.44	23.96	0.9096	29.79
128x128(int8)	23.99	0.9387	97.89	22.65	0.8942	137.35	23.37	0.8817	131.91	20.51	0.8861	49.88	24.26	0.9103	27.46
256x256(fp32)	23.44	0.9348	112.65	22.40	0.8932	139.71	23.21	0.8809	133.87	20.01	0.8815	52.09	23.96	0.9098	29.32
256x256(int8)	23.90	0.9386	101.03	22.62	0.8949	135.43	23.28	0.8821	128.82	20.44	0.8861	48.76	24.14	0.9099	27.33
512x512(fp32)	23.44	0.9348	112.65	22.40	0.8932	139.71	23.21	0.8809	133.87	20.01	0.8815	51.97	23.97	0.9099	29.02
512x512(int8)	23.37	0.9303	117.11	22.29	0.8921	138.18	23.05	0.8796	128.34	19.96	0.8773	49.70	23.79	0.9024	25.40
1024x1024(fp32)	23.44	0.9348	112.65	22.40	0.8932	139.71	23.21	0.8809	133.87	20.01	0.8815	51.97	23.97	0.9099	28.98
1024x1024(int8)	23.10	0.9249	113.23	22.10	0.8835	140.06	22.82	0.8692	130.24	19.80	0.8710	50.43	23.42	0.8932	27.59

---

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}