Hugging Face's logo

qualcomm
/
ControlNet-Canny

Unconditional Image Generation
PyTorch
generative_ai
android
Model card Files
xet
Community
qaihm-bot commited on
Commit
9419161
·
verified ·
1 Parent(s): d876de4

v0.46.1

Browse files

See https://github.com/quic/ai-hub-models/releases/v0.46.1 for changelog.

This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +161 -185
  2. precompiled/qualcomm-qcm6690/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  3. precompiled/qualcomm-qcm6690/ControlNet-Canny_controlnet_w8a16.onnx.zip +0 -3
  4. precompiled/qualcomm-qcm6690/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  5. precompiled/qualcomm-qcm6690/ControlNet-Canny_text_encoder_w8a16.onnx.zip +0 -3
  6. precompiled/qualcomm-qcm6690/ControlNet-Canny_unet_w8a16.bin +0 -3
  7. precompiled/qualcomm-qcm6690/ControlNet-Canny_unet_w8a16.onnx.zip +0 -3
  8. precompiled/qualcomm-qcm6690/ControlNet-Canny_vae_w8a16.bin +0 -3
  9. precompiled/qualcomm-qcm6690/ControlNet-Canny_vae_w8a16.onnx.zip +0 -3
  10. precompiled/qualcomm-qcm6690/tool-versions.yaml +0 -4
  11. precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  12. precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  13. precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_unet_w8a16.bin +0 -3
  14. precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_vae_w8a16.bin +0 -3
  15. precompiled/qualcomm-qcs8275-proxy/tool-versions.yaml +0 -3
  16. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  17. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_controlnet_w8a16.onnx.zip +0 -3
  18. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  19. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_text_encoder_w8a16.onnx.zip +0 -3
  20. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_unet_w8a16.bin +0 -3
  21. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_unet_w8a16.onnx.zip +0 -3
  22. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_vae_w8a16.bin +0 -3
  23. precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_vae_w8a16.onnx.zip +0 -3
  24. precompiled/qualcomm-qcs8550-proxy/tool-versions.yaml +0 -4
  25. precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  26. precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  27. precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_unet_w8a16.bin +0 -3
  28. precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_vae_w8a16.bin +0 -3
  29. precompiled/qualcomm-qcs9075-proxy/tool-versions.yaml +0 -3
  30. precompiled/qualcomm-sa7255p/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  31. precompiled/qualcomm-sa7255p/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  32. precompiled/qualcomm-sa7255p/ControlNet-Canny_unet_w8a16.bin +0 -3
  33. precompiled/qualcomm-sa7255p/ControlNet-Canny_vae_w8a16.bin +0 -3
  34. precompiled/qualcomm-sa7255p/tool-versions.yaml +0 -3
  35. precompiled/qualcomm-sa8775p/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  36. precompiled/qualcomm-sa8775p/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  37. precompiled/qualcomm-sa8775p/ControlNet-Canny_unet_w8a16.bin +0 -3
  38. precompiled/qualcomm-sa8775p/ControlNet-Canny_vae_w8a16.bin +0 -3
  39. precompiled/qualcomm-sa8775p/tool-versions.yaml +0 -3
  40. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  41. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_controlnet_w8a16.onnx.zip +0 -3
  42. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_text_encoder_w8a16.bin +0 -3
  43. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_text_encoder_w8a16.onnx.zip +0 -3
  44. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_unet_w8a16.bin +0 -3
  45. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_unet_w8a16.onnx.zip +0 -3
  46. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_vae_w8a16.bin +0 -3
  47. precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_vae_w8a16.onnx.zip +0 -3
  48. precompiled/qualcomm-snapdragon-7gen4/tool-versions.yaml +0 -4
  49. precompiled/qualcomm-snapdragon-8-elite-for-galaxy/ControlNet-Canny_controlnet_w8a16.bin +0 -3
  50. precompiled/qualcomm-snapdragon-8-elite-for-galaxy/ControlNet-Canny_controlnet_w8a16.onnx.zip +0 -3
README.md CHANGED
@@ -10,204 +10,180 @@ pipeline_tag: unconditional-image-generation
10
 
11
  ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/web-assets/model_demo.png)
12
 
13
- # ControlNet-Canny: Optimized for Mobile Deployment
14
- ## Generating visual arts from text prompt and input guiding image
15
-
16
 
17
  On-device, high-resolution image synthesis from text and image prompts. ControlNet guides Stable-diffusion with provided input image to generate accurate images from given input prompt.
18
 
19
- This model is an implementation of ControlNet-Canny found [here](https://github.com/lllyasviel/ControlNet).
20
-
21
-
22
- This repository provides scripts to run ControlNet-Canny on Qualcomm® devices.
23
- More details on model performance across various devices, can be found
24
- [here](https://aihub.qualcomm.com/models/controlnet_canny).
25
-
26
-
27
-
28
- ### Model Details
29
-
30
- - **Model Type:** Model_use_case.image_generation
31
- - **Model Stats:**
32
- - Input: Text prompt and input image as a reference
33
- - Conditioning Input: Canny-Edge
34
- - Text Encoder Number of parameters: 340M
35
- - UNet Number of parameters: 865M
36
- - VAE Decoder Number of parameters: 83M
37
- - ControlNet Number of parameters: 361M
38
- - Model size: 1.4GB
39
-
40
- | Model | Precision | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit | Target Model
41
- |---|---|---|---|---|---|---|---|---|
42
- | text_encoder | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | QNN_CONTEXT_BINARY | 22.037 ms | 0 - 14 MB | NPU | Use Export Script |
43
- | text_encoder | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | PRECOMPILED_QNN_ONNX | 34.135 ms | 0 - 13 MB | NPU | Use Export Script |
44
- | text_encoder | w8a16 | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_CONTEXT_BINARY | 7.442 ms | 0 - 8 MB | NPU | Use Export Script |
45
- | text_encoder | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_CONTEXT_BINARY | 3.246 ms | 0 - 3 MB | NPU | Use Export Script |
46
- | text_encoder | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | PRECOMPILED_QNN_ONNX | 5.491 ms | 0 - 162 MB | NPU | Use Export Script |
47
- | text_encoder | w8a16 | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_CONTEXT_BINARY | 3.508 ms | 0 - 9 MB | NPU | Use Export Script |
48
- | text_encoder | w8a16 | SA7255P ADP | Qualcomm® SA7255P | QNN_CONTEXT_BINARY | 7.442 ms | 0 - 8 MB | NPU | Use Export Script |
49
- | text_encoder | w8a16 | SA8775P ADP | Qualcomm® SA8775P | QNN_CONTEXT_BINARY | 3.508 ms | 0 - 9 MB | NPU | Use Export Script |
50
- | text_encoder | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_CONTEXT_BINARY | 2.329 ms | 0 - 20 MB | NPU | Use Export Script |
51
- | text_encoder | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | PRECOMPILED_QNN_ONNX | 3.96 ms | 0 - 19 MB | NPU | Use Export Script |
52
- | text_encoder | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_CONTEXT_BINARY | 1.746 ms | 0 - 16 MB | NPU | Use Export Script |
53
- | text_encoder | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | PRECOMPILED_QNN_ONNX | 3.13 ms | 0 - 14 MB | NPU | Use Export Script |
54
- | text_encoder | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | QNN_CONTEXT_BINARY | 3.624 ms | 0 - 14 MB | NPU | Use Export Script |
55
- | text_encoder | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | PRECOMPILED_QNN_ONNX | 5.752 ms | 0 - 14 MB | NPU | Use Export Script |
56
- | text_encoder | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | QNN_CONTEXT_BINARY | 1.54 ms | 0 - 10 MB | NPU | Use Export Script |
57
- | text_encoder | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | PRECOMPILED_QNN_ONNX | 2.622 ms | 0 - 10 MB | NPU | Use Export Script |
58
- | text_encoder | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_CONTEXT_BINARY | 3.598 ms | 0 - 0 MB | NPU | Use Export Script |
59
- | text_encoder | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | PRECOMPILED_QNN_ONNX | 5.671 ms | 157 - 157 MB | NPU | Use Export Script |
60
- | unet | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | QNN_CONTEXT_BINARY | 2078.956 ms | 13 - 24 MB | NPU | Use Export Script |
61
- | unet | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | PRECOMPILED_QNN_ONNX | 2323.924 ms | 13 - 26 MB | NPU | Use Export Script |
62
- | unet | w8a16 | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_CONTEXT_BINARY | 265.894 ms | 11 - 19 MB | NPU | Use Export Script |
63
- | unet | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_CONTEXT_BINARY | 114.963 ms | 13 - 15 MB | NPU | Use Export Script |
64
- | unet | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | PRECOMPILED_QNN_ONNX | 116.648 ms | 0 - 882 MB | NPU | Use Export Script |
65
- | unet | w8a16 | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_CONTEXT_BINARY | 108.505 ms | 6 - 14 MB | NPU | Use Export Script |
66
- | unet | w8a16 | SA7255P ADP | Qualcomm® SA7255P | QNN_CONTEXT_BINARY | 265.894 ms | 11 - 19 MB | NPU | Use Export Script |
67
- | unet | w8a16 | SA8775P ADP | Qualcomm® SA8775P | QNN_CONTEXT_BINARY | 108.505 ms | 6 - 14 MB | NPU | Use Export Script |
68
- | unet | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_CONTEXT_BINARY | 80.67 ms | 13 - 29 MB | NPU | Use Export Script |
69
- | unet | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | PRECOMPILED_QNN_ONNX | 86.931 ms | 13 - 29 MB | NPU | Use Export Script |
70
- | unet | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_CONTEXT_BINARY | 61.336 ms | 5 - 21 MB | NPU | Use Export Script |
71
- | unet | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | PRECOMPILED_QNN_ONNX | 66.951 ms | 5 - 16 MB | NPU | Use Export Script |
72
- | unet | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | QNN_CONTEXT_BINARY | 176.43 ms | 13 - 26 MB | NPU | Use Export Script |
73
- | unet | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | PRECOMPILED_QNN_ONNX | 178.974 ms | 11 - 21 MB | NPU | Use Export Script |
74
- | unet | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | QNN_CONTEXT_BINARY | 47.318 ms | 13 - 24 MB | NPU | Use Export Script |
75
- | unet | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | PRECOMPILED_QNN_ONNX | 48.732 ms | 13 - 21 MB | NPU | Use Export Script |
76
- | unet | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_CONTEXT_BINARY | 116.519 ms | 13 - 13 MB | NPU | Use Export Script |
77
- | unet | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | PRECOMPILED_QNN_ONNX | 115.83 ms | 829 - 829 MB | NPU | Use Export Script |
78
- | vae | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | QNN_CONTEXT_BINARY | 6122.737 ms | 0 - 14 MB | NPU | Use Export Script |
79
- | vae | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | PRECOMPILED_QNN_ONNX | 5411.761 ms | 2 - 15 MB | NPU | Use Export Script |
80
- | vae | w8a16 | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_CONTEXT_BINARY | 718.637 ms | 0 - 9 MB | NPU | Use Export Script |
81
- | vae | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_CONTEXT_BINARY | 269.846 ms | 0 - 3 MB | NPU | Use Export Script |
82
- | vae | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | PRECOMPILED_QNN_ONNX | 226.709 ms | 0 - 66 MB | NPU | Use Export Script |
83
- | vae | w8a16 | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_CONTEXT_BINARY | 251.431 ms | 1 - 10 MB | NPU | Use Export Script |
84
- | vae | w8a16 | SA7255P ADP | Qualcomm® SA7255P | QNN_CONTEXT_BINARY | 718.637 ms | 0 - 9 MB | NPU | Use Export Script |
85
- | vae | w8a16 | SA8775P ADP | Qualcomm® SA8775P | QNN_CONTEXT_BINARY | 251.431 ms | 1 - 10 MB | NPU | Use Export Script |
86
- | vae | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_CONTEXT_BINARY | 203.734 ms | 0 - 18 MB | NPU | Use Export Script |
87
- | vae | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | PRECOMPILED_QNN_ONNX | 163.111 ms | 3 - 23 MB | NPU | Use Export Script |
88
- | vae | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_CONTEXT_BINARY | 178.105 ms | 0 - 13 MB | NPU | Use Export Script |
89
- | vae | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | PRECOMPILED_QNN_ONNX | 146.877 ms | 3 - 14 MB | NPU | Use Export Script |
90
- | vae | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | QNN_CONTEXT_BINARY | 514.198 ms | 0 - 14 MB | NPU | Use Export Script |
91
- | vae | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | PRECOMPILED_QNN_ONNX | 445.416 ms | 3 - 17 MB | NPU | Use Export Script |
92
- | vae | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | QNN_CONTEXT_BINARY | 116.031 ms | 2 - 13 MB | NPU | Use Export Script |
93
- | vae | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | PRECOMPILED_QNN_ONNX | 94.818 ms | 3 - 13 MB | NPU | Use Export Script |
94
- | vae | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_CONTEXT_BINARY | 266.234 ms | 0 - 0 MB | NPU | Use Export Script |
95
- | vae | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | PRECOMPILED_QNN_ONNX | 219.532 ms | 59 - 59 MB | NPU | Use Export Script |
96
- | controlnet | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | QNN_CONTEXT_BINARY | 1063.49 ms | 2 - 14 MB | NPU | Use Export Script |
97
- | controlnet | w8a16 | Dragonwing Q-6690 MTP | Qualcomm® QCM6690 | PRECOMPILED_QNN_ONNX | 882.494 ms | 2 - 17 MB | NPU | Use Export Script |
98
- | controlnet | w8a16 | QCS8275 (Proxy) | Qualcomm® QCS8275 (Proxy) | QNN_CONTEXT_BINARY | 130.285 ms | 2 - 10 MB | NPU | Use Export Script |
99
- | controlnet | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | QNN_CONTEXT_BINARY | 70.099 ms | 2 - 4 MB | NPU | Use Export Script |
100
- | controlnet | w8a16 | QCS8550 (Proxy) | Qualcomm® QCS8550 (Proxy) | PRECOMPILED_QNN_ONNX | 59.406 ms | 0 - 384 MB | NPU | Use Export Script |
101
- | controlnet | w8a16 | QCS9075 (Proxy) | Qualcomm® QCS9075 (Proxy) | QNN_CONTEXT_BINARY | 66.205 ms | 2 - 11 MB | NPU | Use Export Script |
102
- | controlnet | w8a16 | SA7255P ADP | Qualcomm® SA7255P | QNN_CONTEXT_BINARY | 130.285 ms | 2 - 10 MB | NPU | Use Export Script |
103
- | controlnet | w8a16 | SA8775P ADP | Qualcomm® SA8775P | QNN_CONTEXT_BINARY | 66.205 ms | 2 - 11 MB | NPU | Use Export Script |
104
- | controlnet | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | QNN_CONTEXT_BINARY | 53.524 ms | 2 - 21 MB | NPU | Use Export Script |
105
- | controlnet | w8a16 | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 Mobile | PRECOMPILED_QNN_ONNX | 45.398 ms | 32 - 50 MB | NPU | Use Export Script |
106
- | controlnet | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | QNN_CONTEXT_BINARY | 34.868 ms | 1 - 18 MB | NPU | Use Export Script |
107
- | controlnet | w8a16 | Samsung Galaxy S25 | Snapdragon® 8 Elite For Galaxy Mobile | PRECOMPILED_QNN_ONNX | 32.913 ms | 33 - 45 MB | NPU | Use Export Script |
108
- | controlnet | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | QNN_CONTEXT_BINARY | 117.593 ms | 0 - 14 MB | NPU | Use Export Script |
109
- | controlnet | w8a16 | Snapdragon 7 Gen 4 QRD | Snapdragon® 7 Gen 4 Mobile | PRECOMPILED_QNN_ONNX | 113.375 ms | 20 - 32 MB | NPU | Use Export Script |
110
- | controlnet | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | QNN_CONTEXT_BINARY | 28.041 ms | 2 - 12 MB | NPU | Use Export Script |
111
- | controlnet | w8a16 | Snapdragon 8 Elite Gen 5 QRD | Snapdragon® 8 Elite Gen 5 Mobile | PRECOMPILED_QNN_ONNX | 29.237 ms | 33 - 44 MB | NPU | Use Export Script |
112
- | controlnet | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN_CONTEXT_BINARY | 67.472 ms | 2 - 2 MB | NPU | Use Export Script |
113
- | controlnet | w8a16 | Snapdragon X Elite CRD | Snapdragon® X Elite | PRECOMPILED_QNN_ONNX | 59.051 ms | 351 - 351 MB | NPU | Use Export Script |
114
-
115
-
116
-
117
-
118
- ## Installation
119
-
120
-
121
- Install the package via pip:
122
- ```bash
123
- # NOTE: 3.10 <= PYTHON_VERSION < 3.14 is supported.
124
- pip install "qai-hub-models[controlnet-canny]"
125
- ```
126
-
127
-
128
- ## Configure Qualcomm® AI Hub Workbench to run this model on a cloud-hosted device
129
-
130
- Sign-in to [Qualcomm® AI Hub Workbench](https://workbench.aihub.qualcomm.com/) with your
131
- Qualcomm® ID. Once signed in navigate to `Account -> Settings -> API Token`.
132
-
133
- With this API token, you can configure your client to run models on the cloud
134
- hosted devices.
135
- ```bash
136
- qai-hub configure --api_token API_TOKEN
137
- ```
138
- Navigate to [docs](https://workbench.aihub.qualcomm.com/docs/) for more information.
139
-
140
-
141
-
142
- ## Demo off target
143
-
144
- The package contains a simple end-to-end demo that downloads pre-trained
145
- weights and runs this model on a sample input.
146
-
147
- ```bash
148
- python -m qai_hub_models.models.controlnet_canny.demo
149
- ```
150
-
151
- The above demo runs a reference implementation of pre-processing, model
152
- inference, and post processing.
153
-
154
- **NOTE**: If you want running in a Jupyter Notebook or Google Colab like
155
- environment, please add the following to your cell (instead of the above).
156
- ```
157
- %run -m qai_hub_models.models.controlnet_canny.demo
158
- ```
159
-
160
-
161
- ### Run model on a cloud-hosted device
162
-
163
- In addition to the demo, you can also run the model on a cloud-hosted Qualcomm®
164
- device. This script does the following:
165
- * Performance check on-device on a cloud-hosted device
166
- * Downloads compiled assets that can be deployed on-device for Android.
167
- * Accuracy check between PyTorch and on-device outputs.
168
-
169
- ```bash
170
- python -m qai_hub_models.models.controlnet_canny.export
171
- ```
172
-
173
-
174
-
175
-
176
-
177
-
178
- ## Deploying compiled model to Android
179
-
180
-
181
- The models can be deployed using multiple runtimes:
182
- - TensorFlow Lite (`.tflite` export): [This
183
- tutorial](https://www.tensorflow.org/lite/android/quickstart) provides a
184
- guide to deploy the .tflite model in an Android application.
185
-
186
-
187
- - QNN (`.so` export ): This [sample
188
- app](https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/sample_app.html)
189
- provides instructions on how to use the `.so` shared library in an Android application.
190
-
191
-
192
- ## View on Qualcomm® AI Hub
193
- Get more details on ControlNet-Canny's performance across various devices [here](https://aihub.qualcomm.com/models/controlnet_canny).
194
- Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
195
-
196
 
197
  ## License
198
  * The license for the original implementation of ControlNet-Canny can be found
199
  [here](https://github.com/lllyasviel/ControlNet/blob/main/LICENSE).
200
 
201
-
202
-
203
  ## References
204
  * [Adding Conditional Control to Text-to-Image Diffusion Models](https://arxiv.org/abs/2302.05543)
205
  * [Source Model Implementation](https://github.com/lllyasviel/ControlNet)
206
 
207
-
208
-
209
  ## Community
210
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
211
  * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
212
 
213
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
  ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/web-assets/model_demo.png)
12
 
13
+ # ControlNet-Canny: Optimized for Qualcomm Devices
 
 
14
 
15
  On-device, high-resolution image synthesis from text and image prompts. ControlNet guides Stable-diffusion with provided input image to generate accurate images from given input prompt.
16
 
17
+ This is based on the implementation of ControlNet-Canny found [here](https://github.com/lllyasviel/ControlNet).
18
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/controlnet_canny) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
19
+
20
+ Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
21
+
22
+ ## Getting Started
23
+ There are two ways to deploy this model on your device:
24
+
25
+ ### Option 1: Download Pre-Exported Models
26
+
27
+ Below are pre-exported model assets ready for deployment.
28
+
29
+ | Runtime | Precision | Chipset | SDK Versions | Download |
30
+ |---|---|---|---|---|
31
+ | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_x_elite.zip)
32
+ | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_8gen3.zip)
33
+ | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_qcs8550_proxy.zip)
34
+ | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_8_elite_for_galaxy.zip)
35
+ | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_7gen4.zip)
36
+ | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_snapdragon_8_elite_gen5.zip)
37
+ | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_qcm6690.zip)
38
+ | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-precompiled_qnn_onnx-w8a16-qualcomm_qcs9075.zip)
39
+ | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_snapdragon_x_elite.zip)
40
+ | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_snapdragon_8gen3.zip)
41
+ | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_qcs8275_proxy.zip)
42
+ | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_qcs8550_proxy.zip)
43
+ | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_sa8775p.zip)
44
+ | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_snapdragon_8_elite_for_galaxy.zip)
45
+ | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_snapdragon_7gen4.zip)
46
+ | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_snapdragon_8_elite_gen5.zip)
47
+ | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_sa7255p.zip)
48
+ | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_qcm6690.zip)
49
+ | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | QAIRT 2.42 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/controlnet_canny/releases/v0.46.1/controlnet_canny-qnn_context_binary-w8a16-qualcomm_qcs9075.zip)
50
+
51
+ For more device-specific assets and performance metrics, visit **[ControlNet-Canny on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/controlnet_canny)**.
52
+
53
+
54
+ ### Option 2: Export with Custom Configurations
55
+
56
+ Use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/controlnet_canny) Python library to compile and export the model with your own:
57
+ - Custom weights (e.g., fine-tuned checkpoints)
58
+ - Custom input shapes
59
+ - Target device and runtime configurations
60
+
61
+ This option is ideal if you need to customize the model beyond the default configuration provided here.
62
+
63
+ See our repository for [ControlNet-Canny on GitHub](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/controlnet_canny) for usage instructions.
64
+
65
+ ## Model Details
66
+
67
+ **Model Type:** Model_use_case.image_generation
68
+
69
+ **Model Stats:**
70
+ - Input: Text prompt and input image as a reference
71
+ - Conditioning Input: Canny-Edge
72
+ - Text Encoder Number of parameters: 340M
73
+ - UNet Number of parameters: 865M
74
+ - VAE Decoder Number of parameters: 83M
75
+ - ControlNet Number of parameters: 361M
76
+ - Model size: 1.4GB
77
+
78
+ ## Performance Summary
79
+ | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
80
+ |---|---|---|---|---|---|---
81
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 59.129 ms | 351 - 351 MB | NPU
82
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 45.568 ms | 33 - 45 MB | NPU
83
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 60.057 ms | 30 - 31 MB | NPU
84
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 58.171 ms | 30 - 34 MB | NPU
85
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 1040.403 ms | 0 - 10 MB | NPU
86
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 33.962 ms | 32 - 43 MB | NPU
87
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 113.204 ms | 33 - 39 MB | NPU
88
+ | controlnet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 28.884 ms | 31 - 42 MB | NPU
89
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 75.85 ms | 2 - 2 MB | NPU
90
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 59.518 ms | 2 - 10 MB | NPU
91
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 139.916 ms | 2 - 10 MB | NPU
92
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 83.656 ms | 2 - 10 MB | NPU
93
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 76.946 ms | 2 - 11 MB | NPU
94
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 75.284 ms | 0 - 16 MB | NPU
95
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 964.957 ms | 2 - 8 MB | NPU
96
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 139.916 ms | 2 - 10 MB | NPU
97
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 46.052 ms | 2 - 10 MB | NPU
98
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 124.414 ms | 2 - 8 MB | NPU
99
+ | controlnet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 28.773 ms | 2 - 11 MB | NPU
100
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 5.624 ms | 157 - 157 MB | NPU
101
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 3.96 ms | 0 - 7 MB | NPU
102
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 5.414 ms | 0 - 162 MB | NPU
103
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 5.438 ms | 0 - 3 MB | NPU
104
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 28.302 ms | 0 - 6 MB | NPU
105
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 3.167 ms | 0 - 7 MB | NPU
106
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 5.778 ms | 0 - 6 MB | NPU
107
+ | text_encoder | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 2.635 ms | 0 - 10 MB | NPU
108
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 3.608 ms | 0 - 0 MB | NPU
109
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 2.305 ms | 0 - 7 MB | NPU
110
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 7.549 ms | 0 - 8 MB | NPU
111
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 3.239 ms | 0 - 2 MB | NPU
112
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 3.505 ms | 0 - 9 MB | NPU
113
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 3.304 ms | 0 - 2 MB | NPU
114
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 22.025 ms | 0 - 6 MB | NPU
115
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 7.549 ms | 0 - 8 MB | NPU
116
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 1.749 ms | 0 - 13 MB | NPU
117
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 3.636 ms | 0 - 7 MB | NPU
118
+ | text_encoder | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 1.492 ms | 0 - 10 MB | NPU
119
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 115.766 ms | 829 - 829 MB | NPU
120
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 84.074 ms | 13 - 26 MB | NPU
121
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 117.53 ms | 0 - 883 MB | NPU
122
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 109.17 ms | 11 - 26 MB | NPU
123
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 2160.0 ms | 10 - 21 MB | NPU
124
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 67.33 ms | 6 - 17 MB | NPU
125
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 177.928 ms | 9 - 19 MB | NPU
126
+ | unet | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 46.028 ms | 13 - 23 MB | NPU
127
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 113.746 ms | 13 - 13 MB | NPU
128
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 80.577 ms | 13 - 20 MB | NPU
129
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 264.31 ms | 8 - 15 MB | NPU
130
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 112.347 ms | 10 - 12 MB | NPU
131
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 106.976 ms | 10 - 17 MB | NPU
132
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 106.531 ms | 11 - 25 MB | NPU
133
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 2039.478 ms | 13 - 24 MB | NPU
134
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 264.31 ms | 8 - 15 MB | NPU
135
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 61.075 ms | 6 - 18 MB | NPU
136
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 179.697 ms | 10 - 15 MB | NPU
137
+ | unet | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 47.215 ms | 13 - 22 MB | NPU
138
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® X Elite | 219.597 ms | 59 - 59 MB | NPU
139
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 163.642 ms | 4 - 11 MB | NPU
140
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS8550 (Proxy) | 219.435 ms | 0 - 66 MB | NPU
141
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCS9075 | 207.133 ms | 3 - 6 MB | NPU
142
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Qualcomm® QCM6690 | 5359.841 ms | 0 - 7 MB | NPU
143
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 147.795 ms | 3 - 16 MB | NPU
144
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 7 Gen 4 Mobile | 445.729 ms | 3 - 9 MB | NPU
145
+ | vae | PRECOMPILED_QNN_ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 89.515 ms | 3 - 14 MB | NPU
146
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® X Elite | 266.163 ms | 0 - 0 MB | NPU
147
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Gen 3 Mobile | 202.114 ms | 0 - 8 MB | NPU
148
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8275 (Proxy) | 725.838 ms | 0 - 9 MB | NPU
149
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS8550 (Proxy) | 273.289 ms | 1 - 5 MB | NPU
150
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA8775P | 1136.64 ms | 0 - 9 MB | NPU
151
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCS9075 | 249.997 ms | 0 - 3 MB | NPU
152
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® QCM6690 | 6374.38 ms | 0 - 6 MB | NPU
153
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Qualcomm® SA7255P | 725.838 ms | 0 - 9 MB | NPU
154
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite For Galaxy Mobile | 178.859 ms | 0 - 13 MB | NPU
155
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 7 Gen 4 Mobile | 512.932 ms | 0 - 6 MB | NPU
156
+ | vae | QNN_CONTEXT_BINARY | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 116.219 ms | 0 - 9 MB | NPU
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
157
 
158
  ## License
159
  * The license for the original implementation of ControlNet-Canny can be found
160
  [here](https://github.com/lllyasviel/ControlNet/blob/main/LICENSE).
161
 
 
 
162
  ## References
163
  * [Adding Conditional Control to Text-to-Image Diffusion Models](https://arxiv.org/abs/2302.05543)
164
  * [Source Model Implementation](https://github.com/lllyasviel/ControlNet)
165
 
 
 
166
  ## Community
167
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
168
  * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
169
 
170
+ ## Usage and Limitations
171
+
172
+ This model may not be used for or in connection with any of the following applications:
173
+
174
+ - Accessing essential private and public services and benefits;
175
+ - Administration of justice and democratic processes;
176
+ - Assessing or recognizing the emotional state of a person;
177
+ - Biometric and biometrics-based systems, including categorization of persons based on sensitive characteristics;
178
+ - Education and vocational training;
179
+ - Employment and workers management;
180
+ - Exploitation of the vulnerabilities of persons resulting in harmful behavior;
181
+ - General purpose social scoring;
182
+ - Law enforcement;
183
+ - Management and operation of critical infrastructure;
184
+ - Migration, asylum and border control management;
185
+ - Predictive policing;
186
+ - Real-time remote biometric identification in public spaces;
187
+ - Recommender systems of social media platforms;
188
+ - Scraping of facial images (from the internet or otherwise); and/or
189
+ - Subliminal manipulation
precompiled/qualcomm-qcm6690/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:179918d2ab35caf027536729a0739d52eb887c2f59b73df31840d80374c3d3c1
3
- size 373108736
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_controlnet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:a8cff0d189ade8b7f6f718312f84270ce90748b3e691d717ea92df93d7e6bf2b
3
- size 249250861
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:5b172c7bf7e17e6e433e40bd2979c7a860af6e2f27eee414a88d1c4a8090b5d7
3
- size 163041280
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_text_encoder_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:505fb5faeecb3145fcc9b2691f618339f0bfe608f8b919e3962435b89e9bd0d2
3
- size 127279652
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:889fbf823e5faeb464195a9e671644e4ae84b4b946750e36237131a40fa3099f
3
- size 891428864
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_unet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b7c815069f65483e06a4ead7abd6fa1c83aff8229359b41bfb93496f1ee4d3d8
3
- size 569625186
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:07851b86b75c6a5c7ef654d93fde833533dc7a21399f708c9501a14c74ef745b
3
- size 91009024
 
 
 
 
precompiled/qualcomm-qcm6690/ControlNet-Canny_vae_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:29cb59ca0332448bad03bdcb7f1fae41072fab210224d6b7a3a001cbeb40f400
3
- size 45491599
 
 
 
 
precompiled/qualcomm-qcm6690/tool-versions.yaml DELETED
@@ -1,4 +0,0 @@
1
- tool_versions:
2
- precompiled_qnn_onnx:
3
- qairt: 2.37.1.250807093845_124904
4
- onnx_runtime: 1.23.0
 
 
 
 
 
precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:60c08024bbbde337dc73d04ff61388559a689bcc3b7a06856ac75e71f42bf724
3
- size 368775168
 
 
 
 
precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:72c2f9814ff484db9bb100e6fcc8d7708f0de665706a9406409e650159928ab6
3
- size 163045376
 
 
 
 
precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3ad964c7e42ea1b564f38b7add6ee6c7e5403f53c4dd6299fe406c0c7c88b184
3
- size 881262592
 
 
 
 
precompiled/qualcomm-qcs8275-proxy/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b3b3962e1ae97a849fe2a23e23641239ff23ca7b940ce95334749243ab867243
3
- size 64618496
 
 
 
 
precompiled/qualcomm-qcs8275-proxy/tool-versions.yaml DELETED
@@ -1,3 +0,0 @@
1
- tool_versions:
2
- qnn_context_binary:
3
- qairt: 2.41.0.251128145156_191518-auto
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6127546fea0ed67db55ca362d3eedc15925db01bd4afe887f522cae6d6c8cc6c
3
- size 369344512
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_controlnet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:526a58c70379c4188858e96ed1ec6614fdb3d85915f56edfe3d36ab1125b602a
3
- size 247559619
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:ca522a9eed29ac7c2416365787b64010d2c71ee63dafa47af6472261a868de70
3
- size 163049472
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_text_encoder_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bed8956f74d1590a42713219a859e1815ec7f5281ae28d5f6354a4b1a6075c54
3
- size 127297231
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3e83f0d59e88a8b681d370b7ca933f8c0ad2624cab58383584d1567d4ba2d63c
3
- size 879382528
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_unet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:308a1bcdf8d6a0417d94f96a348904647bb908e62be3089709296623e45d86aa
3
- size 566892078
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e6f643c8e3c0bfaafb2954f0320e453f58548c840f2426011001e56fc3c2e95
3
- size 64618496
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/ControlNet-Canny_vae_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6b503b8013e3f9ee1b452689cc3f8b187365772c3b0615a851903fdb985d56bd
3
- size 40357078
 
 
 
 
precompiled/qualcomm-qcs8550-proxy/tool-versions.yaml DELETED
@@ -1,4 +0,0 @@
1
- tool_versions:
2
- precompiled_qnn_onnx:
3
- qairt: 2.37.1.250807093845_124904
4
- onnx_runtime: 1.23.0
 
 
 
 
 
precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bcbd437f479db10fac4bd573c5010c8a4f793de7b1d8ab09ba5b93e861adec30
3
- size 369352704
 
 
 
 
precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d02a190fcee33d338b6ae82a7c7bf7e553b9cf0e70057c386e4cd217fb58e10d
3
- size 163049472
 
 
 
 
precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:19c5304d410c029e1651d3c6fcab31507a1ae217a46feab82672ad42e84e68ef
3
- size 881311744
 
 
 
 
precompiled/qualcomm-qcs9075-proxy/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6d199ebc76256020321dc5c4e0f4dc9ddc581f87856efde9b9554c3d4b7ad422
3
- size 64634880
 
 
 
 
precompiled/qualcomm-qcs9075-proxy/tool-versions.yaml DELETED
@@ -1,3 +0,0 @@
1
- tool_versions:
2
- qnn_context_binary:
3
- qairt: 2.41.0.251128145156_191518-auto
 
 
 
 
precompiled/qualcomm-sa7255p/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:60c08024bbbde337dc73d04ff61388559a689bcc3b7a06856ac75e71f42bf724
3
- size 368775168
 
 
 
 
precompiled/qualcomm-sa7255p/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:72c2f9814ff484db9bb100e6fcc8d7708f0de665706a9406409e650159928ab6
3
- size 163045376
 
 
 
 
precompiled/qualcomm-sa7255p/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3ad964c7e42ea1b564f38b7add6ee6c7e5403f53c4dd6299fe406c0c7c88b184
3
- size 881262592
 
 
 
 
precompiled/qualcomm-sa7255p/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b3b3962e1ae97a849fe2a23e23641239ff23ca7b940ce95334749243ab867243
3
- size 64618496
 
 
 
 
precompiled/qualcomm-sa7255p/tool-versions.yaml DELETED
@@ -1,3 +0,0 @@
1
- tool_versions:
2
- qnn_context_binary:
3
- qairt: 2.41.0.251128145156_191518-auto
 
 
 
 
precompiled/qualcomm-sa8775p/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bcbd437f479db10fac4bd573c5010c8a4f793de7b1d8ab09ba5b93e861adec30
3
- size 369352704
 
 
 
 
precompiled/qualcomm-sa8775p/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d02a190fcee33d338b6ae82a7c7bf7e553b9cf0e70057c386e4cd217fb58e10d
3
- size 163049472
 
 
 
 
precompiled/qualcomm-sa8775p/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:19c5304d410c029e1651d3c6fcab31507a1ae217a46feab82672ad42e84e68ef
3
- size 881311744
 
 
 
 
precompiled/qualcomm-sa8775p/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6d199ebc76256020321dc5c4e0f4dc9ddc581f87856efde9b9554c3d4b7ad422
3
- size 64634880
 
 
 
 
precompiled/qualcomm-sa8775p/tool-versions.yaml DELETED
@@ -1,3 +0,0 @@
1
- tool_versions:
2
- qnn_context_binary:
3
- qairt: 2.41.0.251128145156_191518-auto
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:19707c0ec15029fe94464deee1215be955f6457ccaf874db7204c1bc7773529a
3
- size 370434048
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_controlnet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:dc3f7c2d40b1b6933c213bc3628a7b94df3acbd184e8252d9280122750bcfab4
3
- size 247965655
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_text_encoder_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c66ef8bb8149c700b919d51ff0f4b2b501e43df386ac3e4deecabcfde154d292
3
- size 163016704
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_text_encoder_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:601a271c13d23b507f424c8e3603eeaadfc0fddde294bf573fb825ad1e7f52ac
3
- size 127214412
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_unet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:101af8d1526aff764579ef3985038c10d6c6528dc6e6cf29131bf5c5df57c167
3
- size 882741248
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_unet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:51d4168fd6d53745a85cc3647d920d6bbfc47004d8b513c90490f181a4c90600
3
- size 566869017
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_vae_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0041e2114e01f524939a5fe72f69e28d7963f4e78daa91137dd5053578fe5e7c
3
- size 72445952
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/ControlNet-Canny_vae_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:77de814f7fe28bdc922fdd56ca9d96665a7b30cbcc64fd9a590a754c0d0ce8c3
3
- size 41993021
 
 
 
 
precompiled/qualcomm-snapdragon-7gen4/tool-versions.yaml DELETED
@@ -1,4 +0,0 @@
1
- tool_versions:
2
- precompiled_qnn_onnx:
3
- qairt: 2.37.1.250807093845_124904
4
- onnx_runtime: 1.23.0
 
 
 
 
 
precompiled/qualcomm-snapdragon-8-elite-for-galaxy/ControlNet-Canny_controlnet_w8a16.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:00c52b54b3f926e8692f39436ddb1c2a178427e0fbe248da50198f710d372aec
3
- size 367906816
 
 
 
 
precompiled/qualcomm-snapdragon-8-elite-for-galaxy/ControlNet-Canny_controlnet_w8a16.onnx.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:127618577683ae87da27e45051c7f05e485e648b95e1aef3ae0f1cac32441ad8
3
- size 247643958