qaihm-bot commited on
Commit
3a966b1
·
verified ·
1 Parent(s): 3c29b83

See https://github.com/quic/ai-hub-models/releases/v0.39.0 for changelog.

.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ DEPLOYMENT_MODEL_LICENSE.pdf filter=lfs diff=lfs merge=lfs -text
37
+ RegNet-Y-800MF_float.dlc filter=lfs diff=lfs merge=lfs -text
DEPLOYMENT_MODEL_LICENSE.pdf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4409f93b0e82531303b3e10f52f1fdfb56467a25f05b7441c6bbd8bb8a64b42c
3
+ size 109629
LICENSE ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ The license of the original trained model can be found at https://github.com/pytorch/vision/blob/main/LICENSE.
2
+ The license for the deployable model files (.tflite, .onnx, .dlc, .bin, etc.) can be found in DEPLOYMENT_MODEL_LICENSE.pdf.
README.md ADDED
@@ -0,0 +1,233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: pytorch
3
+ license: other
4
+ tags:
5
+ - android
6
+ pipeline_tag: image-classification
7
+
8
+ ---
9
+
10
+ ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/regnet_y_800mf/web-assets/model_demo.png)
11
+
12
+ # RegNet-Y-800MF: Optimized for Mobile Deployment
13
+ ## Lightweight convolutional neural network for image classification
14
+
15
+ RegNet_Y_800MF is part of the RegNet family of models designed for efficient and scalable image classification. It uses a simple yet effective design space to balance performance and computational cost, making it suitable for mobile and edge devices.
16
+
17
+ This repository provides scripts to run RegNet-Y-800MF on Qualcomm® devices.
18
+ More details on model performance across various devices, can be found
19
+ [here](https://aihub.qualcomm.com/models/regnet_y_800mf).
20
+
21
+
22
+
23
+ ### Model Details
24
+
25
+ - **Model Type:** Model_use_case.image_classification
26
+ - **Model Stats:**
27
+ - Model checkpoint: regnet_y_800mf-1b27b58c.pth
28
+ - Input resolution: 1x3x224
29
+ - Model size: ~6.3 MB
30
+
31
+ | Model | Precision | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit | Target Model
32
+ |---|---|---|---|---|---|---|---|---|
33
+ | RegNet-Y-800MF | float | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | TFLITE | 3.94 ms | 0 - 36 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
34
+ | RegNet-Y-800MF | float | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_DLC | 3.809 ms | 1 - 28 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
35
+ | RegNet-Y-800MF | float | QCS8450 (Proxy) | Qualcomm® QCS8450 (Proxy) | TFLITE | 1.602 ms | 0 - 50 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
36
+ | RegNet-Y-800MF | float | QCS8450 (Proxy) | Qualcomm® QCS8450 (Proxy) | QNN_DLC | 2.047 ms | 0 - 41 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
37
+ | RegNet-Y-800MF | float | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | TFLITE | 1.329 ms | 0 - 86 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
38
+ | RegNet-Y-800MF | float | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_DLC | 1.305 ms | 1 - 14 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
39
+ | RegNet-Y-800MF | float | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | ONNX | 1.329 ms | 0 - 46 MB | NPU | [RegNet-Y-800MF.onnx.zip](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.onnx.zip) |
40
+ | RegNet-Y-800MF | float | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | TFLITE | 1.879 ms | 0 - 36 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
41
+ | RegNet-Y-800MF | float | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_DLC | 1.82 ms | 1 - 28 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
42
+ | RegNet-Y-800MF | float | SA7255P ADP | Qualcomm® SA7255P | TFLITE | 3.94 ms | 0 - 36 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
43
+ | RegNet-Y-800MF | float | SA7255P ADP | Qualcomm® SA7255P | QNN_DLC | 3.809 ms | 1 - 28 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
44
+ | RegNet-Y-800MF | float | SA8255 (Proxy) | Qualcomm® SA8255P (Proxy) | TFLITE | 1.329 ms | 0 - 86 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
45
+ | RegNet-Y-800MF | float | SA8255 (Proxy) | Qualcomm® SA8255P (Proxy) | QNN_DLC | 1.306 ms | 1 - 11 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
46
+ | RegNet-Y-800MF | float | SA8295P ADP | Qualcomm® SA8295P | TFLITE | 2.089 ms | 0 - 39 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
47
+ | RegNet-Y-800MF | float | SA8295P ADP | Qualcomm® SA8295P | QNN_DLC | 2.036 ms | 1 - 33 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
48
+ | RegNet-Y-800MF | float | SA8650 (Proxy) | Qualcomm® SA8650P (Proxy) | TFLITE | 1.328 ms | 0 - 85 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
49
+ | RegNet-Y-800MF | float | SA8650 (Proxy) | Qualcomm® SA8650P (Proxy) | QNN_DLC | 1.305 ms | 1 - 12 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
50
+ | RegNet-Y-800MF | float | SA8775P ADP | Qualcomm® SA8775P | TFLITE | 1.879 ms | 0 - 36 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
51
+ | RegNet-Y-800MF | float | SA8775P ADP | Qualcomm® SA8775P | QNN_DLC | 1.82 ms | 1 - 28 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
52
+ | RegNet-Y-800MF | float | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | TFLITE | 0.865 ms | 0 - 52 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
53
+ | RegNet-Y-800MF | float | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_DLC | 0.853 ms | 0 - 41 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
54
+ | RegNet-Y-800MF | float | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | ONNX | 0.846 ms | 0 - 48 MB | NPU | [RegNet-Y-800MF.onnx.zip](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.onnx.zip) |
55
+ | RegNet-Y-800MF | float | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | TFLITE | 0.691 ms | 0 - 41 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
56
+ | RegNet-Y-800MF | float | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_DLC | 0.663 ms | 1 - 35 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
57
+ | RegNet-Y-800MF | float | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | ONNX | 0.689 ms | 0 - 36 MB | NPU | [RegNet-Y-800MF.onnx.zip](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.onnx.zip) |
58
+ | RegNet-Y-800MF | float | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen5 Mobile | TFLITE | 0.596 ms | 0 - 39 MB | NPU | [RegNet-Y-800MF.tflite](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.tflite) |
59
+ | RegNet-Y-800MF | float | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen5 Mobile | QNN_DLC | 0.62 ms | 0 - 35 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
60
+ | RegNet-Y-800MF | float | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen5 Mobile | ONNX | 0.71 ms | 0 - 35 MB | NPU | [RegNet-Y-800MF.onnx.zip](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.onnx.zip) |
61
+ | RegNet-Y-800MF | float | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_DLC | 1.53 ms | 48 - 48 MB | NPU | [RegNet-Y-800MF.dlc](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.dlc) |
62
+ | RegNet-Y-800MF | float | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 1.261 ms | 14 - 14 MB | NPU | [RegNet-Y-800MF.onnx.zip](https://huggingface.co/qualcomm/RegNet-Y-800MF/blob/main/RegNet-Y-800MF.onnx.zip) |
63
+
64
+
65
+
66
+
67
+ ## Installation
68
+
69
+
70
+ Install the package via pip:
71
+ ```bash
72
+ pip install qai-hub-models
73
+ ```
74
+
75
+
76
+ ## Configure Qualcomm® AI Hub to run this model on a cloud-hosted device
77
+
78
+ Sign-in to [Qualcomm® AI Hub](https://app.aihub.qualcomm.com/) with your
79
+ Qualcomm® ID. Once signed in navigate to `Account -> Settings -> API Token`.
80
+
81
+ With this API token, you can configure your client to run models on the cloud
82
+ hosted devices.
83
+ ```bash
84
+ qai-hub configure --api_token API_TOKEN
85
+ ```
86
+ Navigate to [docs](https://app.aihub.qualcomm.com/docs/) for more information.
87
+
88
+
89
+
90
+ ## Demo off target
91
+
92
+ The package contains a simple end-to-end demo that downloads pre-trained
93
+ weights and runs this model on a sample input.
94
+
95
+ ```bash
96
+ python -m qai_hub_models.models.regnet_y_800mf.demo
97
+ ```
98
+
99
+ The above demo runs a reference implementation of pre-processing, model
100
+ inference, and post processing.
101
+
102
+ **NOTE**: If you want running in a Jupyter Notebook or Google Colab like
103
+ environment, please add the following to your cell (instead of the above).
104
+ ```
105
+ %run -m qai_hub_models.models.regnet_y_800mf.demo
106
+ ```
107
+
108
+
109
+ ### Run model on a cloud-hosted device
110
+
111
+ In addition to the demo, you can also run the model on a cloud-hosted Qualcomm®
112
+ device. This script does the following:
113
+ * Performance check on-device on a cloud-hosted device
114
+ * Downloads compiled assets that can be deployed on-device for Android.
115
+ * Accuracy check between PyTorch and on-device outputs.
116
+
117
+ ```bash
118
+ python -m qai_hub_models.models.regnet_y_800mf.export
119
+ ```
120
+
121
+
122
+
123
+ ## How does this work?
124
+
125
+ This [export script](https://aihub.qualcomm.com/models/regnet_y_800mf/qai_hub_models/models/RegNet-Y-800MF/export.py)
126
+ leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
127
+ on-device. Lets go through each step below in detail:
128
+
129
+ Step 1: **Compile model for on-device deployment**
130
+
131
+ To compile a PyTorch model for on-device deployment, we first trace the model
132
+ in memory using the `jit.trace` and then call the `submit_compile_job` API.
133
+
134
+ ```python
135
+ import torch
136
+
137
+ import qai_hub as hub
138
+ from qai_hub_models.models.regnet_y_800mf import Model
139
+
140
+ # Load the model
141
+ torch_model = Model.from_pretrained()
142
+
143
+ # Device
144
+ device = hub.Device("Samsung Galaxy S25")
145
+
146
+ # Trace model
147
+ input_shape = torch_model.get_input_spec()
148
+ sample_inputs = torch_model.sample_inputs()
149
+
150
+ pt_model = torch.jit.trace(torch_model, [torch.tensor(data[0]) for _, data in sample_inputs.items()])
151
+
152
+ # Compile model on a specific device
153
+ compile_job = hub.submit_compile_job(
154
+ model=pt_model,
155
+ device=device,
156
+ input_specs=torch_model.get_input_spec(),
157
+ )
158
+
159
+ # Get target model to run on-device
160
+ target_model = compile_job.get_target_model()
161
+
162
+ ```
163
+
164
+
165
+ Step 2: **Performance profiling on cloud-hosted device**
166
+
167
+ After compiling models from step 1. Models can be profiled model on-device using the
168
+ `target_model`. Note that this scripts runs the model on a device automatically
169
+ provisioned in the cloud. Once the job is submitted, you can navigate to a
170
+ provided job URL to view a variety of on-device performance metrics.
171
+ ```python
172
+ profile_job = hub.submit_profile_job(
173
+ model=target_model,
174
+ device=device,
175
+ )
176
+
177
+ ```
178
+
179
+ Step 3: **Verify on-device accuracy**
180
+
181
+ To verify the accuracy of the model on-device, you can run on-device inference
182
+ on sample input data on the same cloud hosted device.
183
+ ```python
184
+ input_data = torch_model.sample_inputs()
185
+ inference_job = hub.submit_inference_job(
186
+ model=target_model,
187
+ device=device,
188
+ inputs=input_data,
189
+ )
190
+ on_device_output = inference_job.download_output_data()
191
+
192
+ ```
193
+ With the output of the model, you can compute like PSNR, relative errors or
194
+ spot check the output with expected output.
195
+
196
+ **Note**: This on-device profiling and inference requires access to Qualcomm®
197
+ AI Hub. [Sign up for access](https://myaccount.qualcomm.com/signup).
198
+
199
+
200
+
201
+
202
+ ## Deploying compiled model to Android
203
+
204
+
205
+ The models can be deployed using multiple runtimes:
206
+ - TensorFlow Lite (`.tflite` export): [This
207
+ tutorial](https://www.tensorflow.org/lite/android/quickstart) provides a
208
+ guide to deploy the .tflite model in an Android application.
209
+
210
+
211
+ - QNN (`.so` export ): This [sample
212
+ app](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/sample_app.html)
213
+ provides instructions on how to use the `.so` shared library in an Android application.
214
+
215
+
216
+ ## View on Qualcomm® AI Hub
217
+ Get more details on RegNet-Y-800MF's performance across various devices [here](https://aihub.qualcomm.com/models/regnet_y_800mf).
218
+ Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
219
+
220
+
221
+ ## License
222
+ * The license for the original implementation of RegNet-Y-800MF can be found
223
+ [here](https://github.com/pytorch/vision/blob/main/LICENSE).
224
+ * The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
225
+
226
+
227
+
228
+
229
+ ## Community
230
+ * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
231
+ * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
232
+
233
+
RegNet-Y-800MF_float.dlc ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4186b0afb2aa0f14f27be0c3b776518a60c19771c9e4bdbb6d783cd7737a285
3
+ size 25915972
RegNet-Y-800MF_float.onnx.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:931a564fe81f176e39779d392fc6417fabddde9bb0954afc58df33d44d3d9043
3
+ size 24017835
RegNet-Y-800MF_float.tflite ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2de5d505e0d37a9c6a2b96d47bb1ebeee7faa3bbd1bfa54a7c2d4950155b6d2
3
+ size 25708524
tool-versions.yaml ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ tool_versions:
2
+ onnx:
3
+ qairt: 2.37.1.250807093845_124904
4
+ onnx_runtime: 1.22.2