qaihm-bot commited on
Commit
52ad5ed
·
verified ·
1 Parent(s): 049ecfc

See https://github.com/quic/ai-hub-models/releases/v0.46.1 for changelog.

README.md CHANGED
@@ -9,273 +9,121 @@ pipeline_tag: image-to-image
9
 
10
  ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/web-assets/model_demo.png)
11
 
12
- # Real-ESRGAN-x4plus: Optimized for Mobile Deployment
13
- ## Upscale images and remove image noise
14
-
15
 
16
  Real-ESRGAN is a machine learning model that upscales an image with minimal loss in quality. The implementation is a derivative of the Real-ESRGAN-x4plus architecture, a larger and more powerful version compared to the Real-ESRGAN-general-x4v3 architecture.
17
 
18
- This model is an implementation of Real-ESRGAN-x4plus found [here](https://github.com/xinntao/Real-ESRGAN).
19
-
20
-
21
- This repository provides scripts to run Real-ESRGAN-x4plus on Qualcomm® devices.
22
- More details on model performance across various devices, can be found
23
- [here](https://aihub.qualcomm.com/models/real_esrgan_x4plus).
24
-
25
-
26
-
27
- ### Model Details
28
-
29
- - **Model Type:** Model_use_case.super_resolution
30
- - **Model Stats:**
31
- - Model checkpoint: RealESRGAN_x4plus
32
- - Input resolution: 128x128
33
- - Number of parameters: 16.7M
34
- - Model size (float): 63.9 MB
35
- - Model size (w8a8): 16.7 MB
36
-
37
- | Model | Precision | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit | Target Model
38
- |---|---|---|---|---|---|---|---|---|
39
- | Real-ESRGAN-x4plus | float | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | TFLITE | 449.244 ms | 0 - 319 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
40
- | Real-ESRGAN-x4plus | float | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_DLC | 448.83 ms | 0 - 277 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
41
- | Real-ESRGAN-x4plus | float | QCS8450 (Proxy) | Qualcomm® QCS8450 (Proxy) | TFLITE | 121.416 ms | 3 - 729 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
42
- | Real-ESRGAN-x4plus | float | QCS8450 (Proxy) | Qualcomm® QCS8450 (Proxy) | QNN_DLC | 117.402 ms | 0 - 694 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
43
- | Real-ESRGAN-x4plus | float | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | TFLITE | 66.756 ms | 3 - 6 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
44
- | Real-ESRGAN-x4plus | float | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_DLC | 63.296 ms | 0 - 2 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
45
- | Real-ESRGAN-x4plus | float | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | ONNX | 70.169 ms | 0 - 44 MB | NPU | [Real-ESRGAN-x4plus.onnx.zip](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.onnx.zip) |
46
- | Real-ESRGAN-x4plus | float | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | TFLITE | 105.934 ms | 3 - 322 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
47
- | Real-ESRGAN-x4plus | float | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_DLC | 105.363 ms | 0 - 277 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
48
- | Real-ESRGAN-x4plus | float | SA7255P ADP | Qualcomm® SA7255P | TFLITE | 449.244 ms | 0 - 319 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
49
- | Real-ESRGAN-x4plus | float | SA7255P ADP | Qualcomm® SA7255P | QNN_DLC | 448.83 ms | 0 - 277 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
50
- | Real-ESRGAN-x4plus | float | SA8295P ADP | Qualcomm® SA8295P | TFLITE | 112.035 ms | 3 - 331 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
51
- | Real-ESRGAN-x4plus | float | SA8295P ADP | Qualcomm® SA8295P | QNN_DLC | 111.251 ms | 0 - 296 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
52
- | Real-ESRGAN-x4plus | float | SA8775P ADP | Qualcomm® SA8775P | TFLITE | 105.934 ms | 3 - 322 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
53
- | Real-ESRGAN-x4plus | float | SA8775P ADP | Qualcomm® SA8775P | QNN_DLC | 105.363 ms | 0 - 277 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
54
- | Real-ESRGAN-x4plus | float | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | TFLITE | 49.001 ms | 3 - 729 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
55
- | Real-ESRGAN-x4plus | float | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_DLC | 48.877 ms | 0 - 685 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
56
- | Real-ESRGAN-x4plus | float | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | ONNX | 49.819 ms | 6 - 697 MB | NPU | [Real-ESRGAN-x4plus.onnx.zip](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.onnx.zip) |
57
- | Real-ESRGAN-x4plus | float | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | TFLITE | 38.135 ms | 1 - 300 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
58
- | Real-ESRGAN-x4plus | float | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_DLC | 37.926 ms | 0 - 270 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
59
- | Real-ESRGAN-x4plus | float | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | ONNX | 39.402 ms | 8 - 260 MB | NPU | [Real-ESRGAN-x4plus.onnx.zip](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.onnx.zip) |
60
- | Real-ESRGAN-x4plus | float | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | TFLITE | 29.872 ms | 3 - 310 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.tflite) |
61
- | Real-ESRGAN-x4plus | float | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | QNN_DLC | 24.855 ms | 0 - 276 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
62
- | Real-ESRGAN-x4plus | float | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | ONNX | 26.593 ms | 7 - 267 MB | NPU | [Real-ESRGAN-x4plus.onnx.zip](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.onnx.zip) |
63
- | Real-ESRGAN-x4plus | float | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_DLC | 64.886 ms | 0 - 0 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.dlc) |
64
- | Real-ESRGAN-x4plus | float | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 65.611 ms | 38 - 38 MB | NPU | [Real-ESRGAN-x4plus.onnx.zip](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus.onnx.zip) |
65
- | Real-ESRGAN-x4plus | w8a8 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | TFLITE | 376.467 ms | 1 - 422 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
66
- | Real-ESRGAN-x4plus | w8a8 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | QNN_DLC | 456.006 ms | 0 - 390 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
67
- | Real-ESRGAN-x4plus | w8a8 | Dragonwing RB3 Gen 2 Vision Kit | Qualcomm® QCS6490 | TFLITE | 91.91 ms | 1 - 26 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
68
- | Real-ESRGAN-x4plus | w8a8 | Dragonwing RB3 Gen 2 Vision Kit | Qualcomm® QCS6490 | QNN_DLC | 109.435 ms | 0 - 3 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
69
- | Real-ESRGAN-x4plus | w8a8 | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | TFLITE | 66.186 ms | 1 - 372 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
70
- | Real-ESRGAN-x4plus | w8a8 | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_DLC | 67.914 ms | 0 - 388 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
71
- | Real-ESRGAN-x4plus | w8a8 | QCS8450 (Proxy) | Qualcomm® QCS8450 (Proxy) | TFLITE | 38.263 ms | 1 - 604 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
72
- | Real-ESRGAN-x4plus | w8a8 | QCS8450 (Proxy) | Qualcomm® QCS8450 (Proxy) | QNN_DLC | 42.507 ms | 0 - 622 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
73
- | Real-ESRGAN-x4plus | w8a8 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | TFLITE | 18.649 ms | 1 - 4 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
74
- | Real-ESRGAN-x4plus | w8a8 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_DLC | 25.05 ms | 0 - 3 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
75
- | Real-ESRGAN-x4plus | w8a8 | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | TFLITE | 18.468 ms | 1 - 368 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
76
- | Real-ESRGAN-x4plus | w8a8 | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_DLC | 23.15 ms | 0 - 388 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
77
- | Real-ESRGAN-x4plus | w8a8 | SA7255P ADP | Qualcomm® SA7255P | TFLITE | 66.186 ms | 1 - 372 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
78
- | Real-ESRGAN-x4plus | w8a8 | SA7255P ADP | Qualcomm® SA7255P | QNN_DLC | 67.914 ms | 0 - 388 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
79
- | Real-ESRGAN-x4plus | w8a8 | SA8295P ADP | Qualcomm® SA8295P | TFLITE | 33.779 ms | 1 - 368 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
80
- | Real-ESRGAN-x4plus | w8a8 | SA8295P ADP | Qualcomm® SA8295P | QNN_DLC | 37.918 ms | 0 - 394 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
81
- | Real-ESRGAN-x4plus | w8a8 | SA8775P ADP | Qualcomm® SA8775P | TFLITE | 18.468 ms | 1 - 368 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
82
- | Real-ESRGAN-x4plus | w8a8 | SA8775P ADP | Qualcomm® SA8775P | QNN_DLC | 23.15 ms | 0 - 388 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
83
- | Real-ESRGAN-x4plus | w8a8 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | TFLITE | 14.328 ms | 1 - 603 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
84
- | Real-ESRGAN-x4plus | w8a8 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_DLC | 16.659 ms | 0 - 613 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
85
- | Real-ESRGAN-x4plus | w8a8 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | TFLITE | 11.844 ms | 1 - 388 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
86
- | Real-ESRGAN-x4plus | w8a8 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_DLC | 12.581 ms | 0 - 345 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
87
- | Real-ESRGAN-x4plus | w8a8 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | TFLITE | 35.75 ms | 3 - 390 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
88
- | Real-ESRGAN-x4plus | w8a8 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | QNN_DLC | 41.095 ms | 0 - 411 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
89
- | Real-ESRGAN-x4plus | w8a8 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | TFLITE | 7.465 ms | 1 - 418 MB | NPU | [Real-ESRGAN-x4plus.tflite](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.tflite) |
90
- | Real-ESRGAN-x4plus | w8a8 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | QNN_DLC | 9.112 ms | 0 - 357 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
91
- | Real-ESRGAN-x4plus | w8a8 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_DLC | 26.235 ms | 0 - 0 MB | NPU | [Real-ESRGAN-x4plus.dlc](https://huggingface.co/qualcomm/Real-ESRGAN-x4plus/blob/main/Real-ESRGAN-x4plus_w8a8.dlc) |
92
-
93
-
94
-
95
-
96
- ## Installation
97
-
98
-
99
- Install the package via pip:
100
- ```bash
101
- # NOTE: 3.10 <= PYTHON_VERSION < 3.14 is supported.
102
- pip install "qai-hub-models[real-esrgan-x4plus]"
103
- ```
104
-
105
-
106
- ## Configure Qualcomm® AI Hub Workbench to run this model on a cloud-hosted device
107
-
108
- Sign-in to [Qualcomm® AI Hub Workbench](https://workbench.aihub.qualcomm.com/) with your
109
- Qualcomm® ID. Once signed in navigate to `Account -> Settings -> API Token`.
110
-
111
- With this API token, you can configure your client to run models on the cloud
112
- hosted devices.
113
- ```bash
114
- qai-hub configure --api_token API_TOKEN
115
- ```
116
- Navigate to [docs](https://workbench.aihub.qualcomm.com/docs/) for more information.
117
-
118
-
119
-
120
- ## Demo off target
121
-
122
- The package contains a simple end-to-end demo that downloads pre-trained
123
- weights and runs this model on a sample input.
124
-
125
- ```bash
126
- python -m qai_hub_models.models.real_esrgan_x4plus.demo
127
- ```
128
-
129
- The above demo runs a reference implementation of pre-processing, model
130
- inference, and post processing.
131
-
132
- **NOTE**: If you want running in a Jupyter Notebook or Google Colab like
133
- environment, please add the following to your cell (instead of the above).
134
- ```
135
- %run -m qai_hub_models.models.real_esrgan_x4plus.demo
136
- ```
137
-
138
-
139
- ### Run model on a cloud-hosted device
140
-
141
- In addition to the demo, you can also run the model on a cloud-hosted Qualcomm®
142
- device. This script does the following:
143
- * Performance check on-device on a cloud-hosted device
144
- * Downloads compiled assets that can be deployed on-device for Android.
145
- * Accuracy check between PyTorch and on-device outputs.
146
-
147
- ```bash
148
- python -m qai_hub_models.models.real_esrgan_x4plus.export
149
- ```
150
-
151
-
152
-
153
- ## How does this work?
154
-
155
- This [export script](https://aihub.qualcomm.com/models/real_esrgan_x4plus/qai_hub_models/models/Real-ESRGAN-x4plus/export.py)
156
- leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
157
- on-device. Lets go through each step below in detail:
158
-
159
- Step 1: **Compile model for on-device deployment**
160
-
161
- To compile a PyTorch model for on-device deployment, we first trace the model
162
- in memory using the `jit.trace` and then call the `submit_compile_job` API.
163
-
164
- ```python
165
- import torch
166
-
167
- import qai_hub as hub
168
- from qai_hub_models.models.real_esrgan_x4plus import Model
169
-
170
- # Load the model
171
- torch_model = Model.from_pretrained()
172
-
173
- # Device
174
- device = hub.Device("Samsung Galaxy S25")
175
-
176
- # Trace model
177
- input_shape = torch_model.get_input_spec()
178
- sample_inputs = torch_model.sample_inputs()
179
-
180
- pt_model = torch.jit.trace(torch_model, [torch.tensor(data[0]) for _, data in sample_inputs.items()])
181
-
182
- # Compile model on a specific device
183
- compile_job = hub.submit_compile_job(
184
- model=pt_model,
185
- device=device,
186
- input_specs=torch_model.get_input_spec(),
187
- )
188
-
189
- # Get target model to run on-device
190
- target_model = compile_job.get_target_model()
191
-
192
- ```
193
-
194
-
195
- Step 2: **Performance profiling on cloud-hosted device**
196
-
197
- After compiling models from step 1. Models can be profiled model on-device using the
198
- `target_model`. Note that this scripts runs the model on a device automatically
199
- provisioned in the cloud. Once the job is submitted, you can navigate to a
200
- provided job URL to view a variety of on-device performance metrics.
201
- ```python
202
- profile_job = hub.submit_profile_job(
203
- model=target_model,
204
- device=device,
205
- )
206
-
207
- ```
208
-
209
- Step 3: **Verify on-device accuracy**
210
-
211
- To verify the accuracy of the model on-device, you can run on-device inference
212
- on sample input data on the same cloud hosted device.
213
- ```python
214
- input_data = torch_model.sample_inputs()
215
- inference_job = hub.submit_inference_job(
216
- model=target_model,
217
- device=device,
218
- inputs=input_data,
219
- )
220
- on_device_output = inference_job.download_output_data()
221
-
222
- ```
223
- With the output of the model, you can compute like PSNR, relative errors or
224
- spot check the output with expected output.
225
-
226
- **Note**: This on-device profiling and inference requires access to Qualcomm®
227
- AI Hub Workbench. [Sign up for access](https://myaccount.qualcomm.com/signup).
228
-
229
-
230
-
231
- ## Run demo on a cloud-hosted device
232
-
233
- You can also run the demo on-device.
234
-
235
- ```bash
236
- python -m qai_hub_models.models.real_esrgan_x4plus.demo --eval-mode on-device
237
- ```
238
-
239
- **NOTE**: If you want running in a Jupyter Notebook or Google Colab like
240
- environment, please add the following to your cell (instead of the above).
241
- ```
242
- %run -m qai_hub_models.models.real_esrgan_x4plus.demo -- --eval-mode on-device
243
- ```
244
-
245
-
246
- ## Deploying compiled model to Android
247
-
248
-
249
- The models can be deployed using multiple runtimes:
250
- - TensorFlow Lite (`.tflite` export): [This
251
- tutorial](https://www.tensorflow.org/lite/android/quickstart) provides a
252
- guide to deploy the .tflite model in an Android application.
253
-
254
-
255
- - QNN (`.so` export ): This [sample
256
- app](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/sample_app.html)
257
- provides instructions on how to use the `.so` shared library in an Android application.
258
-
259
-
260
- ## View on Qualcomm® AI Hub
261
- Get more details on Real-ESRGAN-x4plus's performance across various devices [here](https://aihub.qualcomm.com/models/real_esrgan_x4plus).
262
- Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
263
-
264
 
265
  ## License
266
  * The license for the original implementation of Real-ESRGAN-x4plus can be found
267
  [here](https://github.com/xinntao/Real-ESRGAN/blob/master/LICENSE).
268
 
269
-
270
-
271
  ## References
272
  * [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://arxiv.org/abs/2107.10833)
273
  * [Source Model Implementation](https://github.com/xinntao/Real-ESRGAN)
274
 
275
-
276
-
277
  ## Community
278
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
279
  * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
280
-
281
-
 
9
 
10
  ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/web-assets/model_demo.png)
11
 
12
+ # Real-ESRGAN-x4plus: Optimized for Qualcomm Devices
 
 
13
 
14
  Real-ESRGAN is a machine learning model that upscales an image with minimal loss in quality. The implementation is a derivative of the Real-ESRGAN-x4plus architecture, a larger and more powerful version compared to the Real-ESRGAN-general-x4v3 architecture.
15
 
16
+ This is based on the implementation of Real-ESRGAN-x4plus found [here](https://github.com/xinntao/Real-ESRGAN).
17
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/real_esrgan_x4plus) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
+
19
+ Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
+
21
+ ## Getting Started
22
+ There are two ways to deploy this model on your device:
23
+
24
+ ### Option 1: Download Pre-Exported Models
25
+
26
+ Below are pre-exported model assets ready for deployment.
27
+
28
+ | Runtime | Precision | Chipset | SDK Versions | Download |
29
+ |---|---|---|---|---|
30
+ | ONNX | float | Universal | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/releases/v0.46.1/real_esrgan_x4plus-onnx-float.zip)
31
+ | QNN_DLC | float | Universal | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/releases/v0.46.1/real_esrgan_x4plus-qnn_dlc-float.zip)
32
+ | QNN_DLC | w8a8 | Universal | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/releases/v0.46.1/real_esrgan_x4plus-qnn_dlc-w8a8.zip)
33
+ | TFLITE | float | Universal | QAIRT 2.42, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/releases/v0.46.1/real_esrgan_x4plus-tflite-float.zip)
34
+ | TFLITE | w8a8 | Universal | QAIRT 2.42, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/real_esrgan_x4plus/releases/v0.46.1/real_esrgan_x4plus-tflite-w8a8.zip)
35
+
36
+ For more device-specific assets and performance metrics, visit **[Real-ESRGAN-x4plus on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/real_esrgan_x4plus)**.
37
+
38
+
39
+ ### Option 2: Export with Custom Configurations
40
+
41
+ Use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/real_esrgan_x4plus) Python library to compile and export the model with your own:
42
+ - Custom weights (e.g., fine-tuned checkpoints)
43
+ - Custom input shapes
44
+ - Target device and runtime configurations
45
+
46
+ This option is ideal if you need to customize the model beyond the default configuration provided here.
47
+
48
+ See our repository for [Real-ESRGAN-x4plus on GitHub](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/real_esrgan_x4plus) for usage instructions.
49
+
50
+ ## Model Details
51
+
52
+ **Model Type:** Model_use_case.super_resolution
53
+
54
+ **Model Stats:**
55
+ - Model checkpoint: RealESRGAN_x4plus
56
+ - Input resolution: 128x128
57
+ - Number of parameters: 16.7M
58
+ - Model size (float): 63.9 MB
59
+ - Model size (w8a8): 16.7 MB
60
+
61
+ ## Performance Summary
62
+ | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
63
+ |---|---|---|---|---|---|---
64
+ | Real-ESRGAN-x4plus | ONNX | float | Snapdragon® X Elite | 65.564 ms | 37 - 37 MB | NPU
65
+ | Real-ESRGAN-x4plus | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 49.752 ms | 0 - 689 MB | NPU
66
+ | Real-ESRGAN-x4plus | ONNX | float | Qualcomm® QCS8550 (Proxy) | 64.138 ms | 0 - 559 MB | NPU
67
+ | Real-ESRGAN-x4plus | ONNX | float | Qualcomm® QCS9075 | 106.758 ms | 6 - 9 MB | NPU
68
+ | Real-ESRGAN-x4plus | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 38.817 ms | 7 - 258 MB | NPU
69
+ | Real-ESRGAN-x4plus | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 27.445 ms | 6 - 266 MB | NPU
70
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Snapdragon® X Elite | 65.048 ms | 0 - 0 MB | NPU
71
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 49.703 ms | 0 - 752 MB | NPU
72
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 452.116 ms | 0 - 348 MB | NPU
73
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 62.963 ms | 0 - 4 MB | NPU
74
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® SA8775P | 503.535 ms | 0 - 347 MB | NPU
75
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® QCS9075 | 108.847 ms | 0 - 5 MB | NPU
76
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 113.142 ms | 1 - 757 MB | NPU
77
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® SA7255P | 452.116 ms | 0 - 348 MB | NPU
78
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Qualcomm® SA8295P | 111.293 ms | 0 - 357 MB | NPU
79
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 37.922 ms | 0 - 332 MB | NPU
80
+ | Real-ESRGAN-x4plus | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 25.742 ms | 0 - 331 MB | NPU
81
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Snapdragon® X Elite | 25.884 ms | 0 - 0 MB | NPU
82
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Snapdragon® 8 Gen 3 Mobile | 16.79 ms | 0 - 691 MB | NPU
83
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® QCS6490 | 109.873 ms | 2 - 4 MB | NPU
84
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® QCS8275 (Proxy) | 69.033 ms | 0 - 478 MB | NPU
85
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® QCS8550 (Proxy) | 24.619 ms | 0 - 3 MB | NPU
86
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® SA8775P | 22.985 ms | 0 - 477 MB | NPU
87
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® QCS9075 | 29.524 ms | 0 - 2 MB | NPU
88
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® QCM6690 | 493.304 ms | 0 - 465 MB | NPU
89
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® QCS8450 (Proxy) | 37.494 ms | 0 - 713 MB | NPU
90
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® SA7255P | 69.033 ms | 0 - 478 MB | NPU
91
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Qualcomm® SA8295P | 36.364 ms | 0 - 484 MB | NPU
92
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 12.361 ms | 0 - 417 MB | NPU
93
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Snapdragon® 7 Gen 4 Mobile | 38.592 ms | 0 - 484 MB | NPU
94
+ | Real-ESRGAN-x4plus | QNN_DLC | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 9.126 ms | 0 - 431 MB | NPU
95
+ | Real-ESRGAN-x4plus | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 48.871 ms | 3 - 796 MB | NPU
96
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 452.332 ms | 3 - 395 MB | NPU
97
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 59.885 ms | 3 - 6 MB | NPU
98
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® SA8775P | 105.619 ms | 3 - 393 MB | NPU
99
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® QCS9075 | 109.28 ms | 1 - 45 MB | NPU
100
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 113.953 ms | 4 - 784 MB | NPU
101
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® SA7255P | 452.332 ms | 3 - 395 MB | NPU
102
+ | Real-ESRGAN-x4plus | TFLITE | float | Qualcomm® SA8295P | 110.425 ms | 3 - 398 MB | NPU
103
+ | Real-ESRGAN-x4plus | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 38.056 ms | 3 - 373 MB | NPU
104
+ | Real-ESRGAN-x4plus | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 28.131 ms | 3 - 377 MB | NPU
105
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Snapdragon® 8 Gen 3 Mobile | 13.828 ms | 1 - 674 MB | NPU
106
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® QCS6490 | 93.178 ms | 1 - 26 MB | NPU
107
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® QCS8275 (Proxy) | 64.674 ms | 1 - 450 MB | NPU
108
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® QCS8550 (Proxy) | 18.667 ms | 1 - 4 MB | NPU
109
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® SA8775P | 18.301 ms | 1 - 450 MB | NPU
110
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® QCS9075 | 25.716 ms | 1 - 27 MB | NPU
111
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® QCM6690 | 434.742 ms | 1 - 436 MB | NPU
112
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® QCS8450 (Proxy) | 37.707 ms | 1 - 684 MB | NPU
113
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® SA7255P | 64.674 ms | 1 - 450 MB | NPU
114
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Qualcomm® SA8295P | 33.678 ms | 1 - 450 MB | NPU
115
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 11.599 ms | 1 - 479 MB | NPU
116
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Snapdragon® 7 Gen 4 Mobile | 31.163 ms | 1 - 442 MB | NPU
117
+ | Real-ESRGAN-x4plus | TFLITE | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 7.474 ms | 1 - 508 MB | NPU
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
118
 
119
  ## License
120
  * The license for the original implementation of Real-ESRGAN-x4plus can be found
121
  [here](https://github.com/xinntao/Real-ESRGAN/blob/master/LICENSE).
122
 
 
 
123
  ## References
124
  * [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://arxiv.org/abs/2107.10833)
125
  * [Source Model Implementation](https://github.com/xinntao/Real-ESRGAN)
126
 
 
 
127
  ## Community
128
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
129
  * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
 
 
Real-ESRGAN-x4plus_float.dlc DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:aefddb96c6fcff88734361f3d1bb6ce7f5b15212514af885a8e378da303178fa
3
- size 67863468
 
 
 
 
Real-ESRGAN-x4plus_float.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:201366a7490f06850e986d826c387b012f5d40eb71a5ad84e1c2737b97c3ff13
3
- size 62328617
 
 
 
 
Real-ESRGAN-x4plus_float.tflite DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:656a422c544d8d5b3e2e59845b07393338a5d246fade1f99d05f318055cd5c6e
3
- size 67031032
 
 
 
 
Real-ESRGAN-x4plus_w8a8.dlc DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8b08380107bd07192d65891938d21383b378a25a78a0fa0c55ca23bdad4f74b1
3
- size 18689724
 
 
 
 
Real-ESRGAN-x4plus_w8a8.tflite DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:af8d5dbc5ea53f42cc601d30e5b8912fdbd849964884bcb641de685e172b64ae
3
- size 17567184
 
 
 
 
tool-versions.yaml DELETED
@@ -1,3 +0,0 @@
1
- tool_versions:
2
- qnn_dlc:
3
- qairt: 2.41.0.251128145156_191518