bconsolvo commited on
Commit
bad8fad
·
1 Parent(s): bcd7441

revert last 2 commits

Browse files
Files changed (1) hide show
  1. README.md +18 -176
README.md CHANGED
@@ -1,193 +1,35 @@
1
  ---
2
  license: apache-2.0
3
- datasets:
4
- - eugenesiow/Div2k
5
- language:
6
- - en
7
  tags:
8
- - RyzenAI
9
- - INT8
10
- - Single Image Super Resolution
11
- - SESR
12
- - ONNX
13
- - SISR
14
- - Computer Vision
15
  metrics:
16
- - PSNR
17
- - MS_SSIM
18
- - FID
19
- paper:
20
- - https://arxiv.org/abs/2103.09404
21
  ---
22
 
23
- # 🚀 SESR-S on AMD AI PC NPU
24
 
25
- [Bhardwaj et al. (2022)](https://arxiv.org/abs/2103.09404) introduced the Super-Efficient Super Resolution (SESR) model to solve a classic computer vision problem: to take a low-resolution input image and output a high-resolution image. The SESR model is based on a "linear overparameterization of CNNs and creates an efficient model architecture for [Single Image Super Resolution (SISR)]." The official code can be found on GitHub: https://github.com/ARM-software/sesr. One of the main ideas behind the model was to make it very computationally efficient.
26
 
27
- This version of the model is the SESR-S (Small) version; it has been converted from PyTorch format to ONNX, and then quantized to INT8 to run on an AMD AI PC NPU with Ryzen AI software. The model in its current form operates on tile sizes of 256x256; however, because tiles can overlap across the image, the model accepts input of almost any size and will create a 2x super-resolution image as output (Figure 1).
28
 
29
- | Input image | Output image |
30
- | --- | --- |
31
- | ![assets/input_ice_climber_0844.png](assets/input_ice_climber_0844.png) | ![assets/output_ice_climber_0844_x2.png](assets/output_ice_climber_0844_x2.png) |
32
 
33
- *Figure 1: Ice climber image upscaled by 2x with SESR-S model running on AMD AI PC NPU. Source: DIV2K dataset (DIV2K_valid_LR_bicubic\X4\0844x4.png).*
34
 
35
- | Model Details | Description |
36
- | ----------- | ----------- |
37
- | Person or organization developing model | [Yixuan Liu (AMD)](https://hf.co/yixliu1), [Hongwei Qin (AMD)](https://huggingface.co/hongwqin), [Benjamin Consolvo (AMD)](https://huggingface.co/bconsolvo) |
38
- | Model date | January 2026 |
39
- | Model version | 1 |
40
- | Model type | Super-Resolution (Image-to-Image) |
41
- | Information about training algorithms, parameters, fairness constraints or other applied approaches, and features | The \\(\times2\\) SESR was trained for "300 epochs using ADAM optimizer with a constant learning rate of \\(5 \times 10^{-4}\\) and a batch size of 32 on DIV2K training set." And the \\(\times4\\) SESR model starts with the pretrained \\(\times2\\) SESR model and replaces "the final layer of \\(5 \times 5 \times f \times 4\\) with a \\(5 \times 5 \times f \times 16\\) and then perform[s] the depth-to-space operation twice" ([Bhardwaj et al., 2022](https://arxiv.org/abs/2103.09404)). For more training details, refer to the paper.|
42
- | Paper or other resource for more information| [Bhardwaj, K., Milosavljevic, M., O'Neil, L., Gope, D., Matas, R., Chalfin, A., ... & Loh, D. (2022). Collapsible linear blocks for super-efficient super resolution. Proceedings of machine learning and systems, 4, 529-547](https://arxiv.org/abs/2103.09404) |
43
- | License | [Apache 2.0](LICENSE) |
44
- | Where to send questions or comments about the model | [Community Tab](https://huggingface.co/amd/sesr-s-256x256-tiles-ryzenai-npu/discussions) and [AMD Developer Community Discord](https://discord.gg/amd-dev)|
45
 
46
- ## Intended Use
47
 
48
- | Intended Use | Description |
49
- | ----------- | ----------- |
50
- | Primary intended uses | The model can be used to create high-resolution images from low-resolution images. The model has been converted to ONNX format and quantized for optimized performance on AMD AI PC NPUs. |
51
- | Primary intended users | Anyone using or evaluating super-resolution models on AMD AI PCs. |
52
- | Out-of-scope uses | This model is not intended for generating misinformation or disinformation, impersonating others, facilitating or inciting harassment or violence, any use that could lead to the violation of a human right. |
53
-
54
-
55
- ### How to Use
56
-
57
- #### 📐 Hardware Prerequisites
58
-
59
- Before getting started, make sure you meet the minimum hardware and OS requirements:
60
-
61
- | Series | Codename | Abbreviation | Launch Year | Windows 11 | Linux |
62
- |--------|----------|--------------|----------------|-------------|---------|
63
- | Ryzen AI Max PRO 300 Series | Strix Halo | STX | 2025 | ☑️ | |
64
- | Ryzen AI PRO 300 Series | Strix Point / Krackan Point | STX/KRK | 2025 | ☑️ | |
65
- | Ryzen AI Max 300 Series | Strix Halo | STX | 2025 | ☑️ | |
66
- | Ryzen AI 300 Series | Strix Point | STX | 2025 | ☑️ | |
67
- | Ryzen Pro 200 Series | Hawk Point | HPT | 2025 | ☑️ | |
68
- | Ryzen 200 Series | Hawk Point | HPT | 2025 | ☑️ | |
69
- | Ryzen PRO 8000 Series | Hawk Point | HPT | 2024 | ☑️ | |
70
- | Ryzen 8000 Series | Hawk Point | HPT | 2024 | ☑️ | |
71
- | Ryzen Pro 7000 Series | Phoenix | PHX | 2023 | ☑️ | |
72
- | Ryzen 7000 Series | Phoenix | PHX | 2023 | ☑️ | |
73
-
74
- #### Getting Started
75
-
76
- 1. Follow the instructions here to download necessary NPU drivers and Ryzen AI software: [Ryzen AI SW Installation Instructions](https://ryzenai.docs.amd.com/en/latest/inst.html). Please allow for around **30 minutes** to install all of the necessary components of Ryzen AI SW.
77
-
78
- 2. Activate the previously installed conda environment from Ryzen AI (RAI) SW, and set the RAI environment variable to your installation path:
79
- ```powershell
80
- conda activate ryzen-ai-1.7.1
81
- $Env:RYZEN_AI_INSTALLATION_PATH = 'C:/Program Files/RyzenAI/1.7.1/'
82
- ```
83
-
84
- 3. Clone the Hugging Face model repository:
85
-
86
- ```powershell
87
- git clone https://hf.co/amd/sesr-s-256x256-tiles-ryzenai-npu
88
- ```
89
-
90
- Alternatively, you can use the Hugging Face Hub API to download the files with Python:
91
-
92
- ```python
93
- from huggingface_hub import snapshot_download
94
- snapshot_download("amd/sesr-s-256x256-tiles-ryzenai-npu")
95
- ```
96
-
97
- 4. Install the necessary packages into the existing conda environment:
98
-
99
- ```powershell
100
- pip install -r requirements.txt
101
- ```
102
-
103
- 5. Data Preparation (optional: for evaluation).
104
- Download the EDSR benchmark dataset extract it into the `datasets/` directory. Note that you will need to run this script twice, as it seems to fail on first attempt.
105
-
106
- ```powershell
107
- python download_edsr_benchmark.py
108
- ```
109
-
110
- The `datasets/` directory should look like this:
111
-
112
- ```Plain
113
- datasets/edsr_benchmark
114
- └── B100
115
- └── HR
116
- ├── 3096.png
117
- ├── ...
118
- └── LR_bicubic/X2
119
- ├── 3096x4.png
120
- ├── ...
121
- └── Set5
122
- └── HR
123
- ├── baby.png
124
- ├── ...
125
- └── LR_bicubic/X2
126
- ├── babyx4.png
127
- ├── ...
128
- ```
129
-
130
- 6. Run inference on a single image or a folder of images. For example:
131
-
132
- ```bash
133
- python onnx_inference.py --onnx onnx-models/sesr_nchw_int8.onnx --input datasets/edsr_benchmark/B100/HR/3096.png --out-dir outputs --device npu
134
- ```
135
-
136
- _Arguments:_
137
-
138
- `--onnx`: The ONNX model file path.
139
-
140
- `--input`: Accepts either a single image file path or a directory path. If it's a file, the script will process that image only. If it's a directory, the script will recursively scan for .png, .jpg, and .jpeg files and process all of them.
141
-
142
- `--out-dir`: Output directory where the restored images will be saved.
143
-
144
- `--device`: Accepts "npu" or "cpu". The NPU will attempt to use the `VitisAIExecutionProvider`; the CPU will attempt to use the `CPUExecutionProvider`. Note that to use the NPU, the updated NPU drivers and Ryzen AI SW must first be installed.
145
-
146
- The model has already been compiled and cached under [modelcachekey_sesr_nchw_int8](modelcachekey_sesr_nchw_int8), but if this folder is not present, the model will be recompiled and then inference can be run.
147
-
148
- 7. Evaluate the accuracy of the model on benchmark datasets (optional).
149
-
150
- ```powershell
151
- # eval on Set5, enable -clean option will remove generated sr images.
152
- python onnx_eval.py --onnx onnx-models/realesrgan_nchw_128x128_u8s8.onnx --hq-dir datasets/edsr_benchmark/Set5/HR --lq-dir datasets/edsr_benchmark/Set5/LR_bicubic/X4 --out-dir outputs/u8s8-Set5 --device npu -clean
153
- ```
154
-
155
- The output will be a set of accuracy metrics: PSNR, MS_SSIM, and FID, as below.
156
- ```powershell
157
- ...
158
-
159
- Loading pretrained model Inception3 from C:\Users\bconsolvo\.cache\torch\hub\pyiqa\pt_inception-2015-12-05-6726825d.pth
160
- Computing FID between two folders
161
- Found 5 images in the folder outputs/u8s8-Set5/sr
162
- FID sr: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 10.70it/s]
163
- Found 5 images in the folder datasets/edsr_benchmark/Set5/HR
164
- FID HR: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████��█████████████████████████████| 5/5 [00:00<00:00, 13.99it/s]
165
- summary of Set5: PSNR | MS_SSIM | FID
166
- Set5: 24.22 | 0.9266 | 85.28
167
- result saved to outputs\u8s8-Set5\eval_realesrgan_nchw_1024x1024_u8s8_result.json
168
- ```
169
-
170
- The following are example scripts to run evaluation on the other datasets:
171
- ```powershell
172
- # eval on Set14
173
- python onnx_eval.py --onnx onnx-models/realesrgan_nchw_128x128_u8s8.onnx --hq-dir datasets/edsr_benchmark/Set14/HR --lq-dir datasets/edsr_benchmark/Set14/LR_bicubic/X4 --out-dir outputs/u8s8-Set14 --device npu -clean
174
- ```
175
-
176
- ```powershell
177
- # eval on B100
178
- python onnx_eval.py --onnx onnx-models/realesrgan_nchw_128x128_u8s8.onnx --hq-dir datasets/edsr_benchmark/B100/HR --lq-dir datasets/edsr_benchmark/B100/LR_bicubic/X4 --out-dir outputs/u8s8-B100 --device npu -clean
179
- ```
180
-
181
- ```powershell
182
- # eval on Urban100
183
- python onnx_eval.py --onnx onnx-models/realesrgan_nchw_128x128_u8s8.onnx --hq-dir datasets/edsr_benchmark/Urban100/HR --lq-dir datasets/edsr_benchmark/Urban100/LR_bicubic/X4 --out-dir outputs/u8s8-Urban100 --device npu -clean
184
- ```
185
-
186
- ```powershell
187
- # eval on DIV2K
188
- python onnx_eval.py --onnx onnx-models/realesrgan_nchw_128x128_u8s8.onnx --hq-dir datasets/DIV2K_valid_HR --lq-dir datasets/DIV2K_valid_LR_bicubic/X4 --out-dir outputs/u8s8-DIV2K --device npu -clean
189
- ```
190
 
 
191
 
192
  ### Installation
193
 
 
1
  ---
2
  license: apache-2.0
 
 
 
 
3
  tags:
4
+ - RyzenAI
5
+ - Int8 quantization
6
+ - Single Image Super Resolution
7
+ - SESR
8
+ - ONNX
9
+ - Computer Vision
 
10
  metrics:
11
+ - PSNR
12
+ - MS_SSIM
13
+ - FID
 
 
14
  ---
15
 
16
+ # SESR for 2x Single Image Super Resolution
17
 
18
+ We provide 2x super-resolution models at resolution 256x256.
19
 
20
+ It was introduced in the paper _Collapsible Linear Blocks for Super-Efficient Super Resolution_ by Bhardwaj. The official code for this work is available at [sesr](https://github.com/ARM-software/sesr).
21
 
22
+ We have developed a modified version optimized for [AMD Ryzen AI](https://onnxruntime.ai/docs/execution-providers/Vitis-AI-ExecutionProvider.html).
 
 
23
 
24
+ ## Model description
25
 
26
+ SESR is based on linear overparameterization of CNNs and creates an efficient model architecture for SISR.
 
 
 
 
 
 
 
 
 
27
 
28
+ ## Intended uses & limitations
29
 
30
+ You can use this model for single image super resolution tasks. See the [model hub](https://huggingface.co/models?search=amd/ryzenai-sesr) for all available models.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
+ ## How to use
33
 
34
  ### Installation
35