Hugging Face's logo

qualcomm
/
WeDetect

Object Detection
PyTorch
real_time
android
Model card Files
xet
 Community
WeDetect
File size: 16,311 Bytes
08e0b16
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
---
library_name: pytorch
license: gpl-3.0
tags:
- real_time
- android
pipeline_tag: object-detection

---

![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/wedetect/web-assets/model_demo.png)

# WeDetect: Optimized for Qualcomm Devices

WeDetect is a text-conditioned object detection model that uses a prompt-then-detect strategy. Text class names are reparameterized into the model weights before export, producing a fixed-vocabulary detector that runs efficiently on-device. The model uses LTRB (Left-Top-Right-Bottom) regression with multi-scale feature maps at strides 8, 16, and 32.

This is based on the implementation of WeDetect found [here](https://github.com/WeChatCV/WeDetect).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.61.0/src/qai_hub_models/models/wedetect) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

## Getting Started
Due to licensing restrictions, we cannot distribute pre-exported model assets for this model.
Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.61.0/src/qai_hub_models/models/wedetect) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations

See our repository for [WeDetect on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.61.0/src/qai_hub_models/models/wedetect) for usage instructions.


## Model Details

**Model Type:** Model_use_case.object_detection

**Model Stats:**
- Input resolution: 640x640
- Model checkpoint: wedetect_tiny
- Model size (detector) (float): 143 MB
- Model size (detector) (mixed_with_float): 35.9 MB
- Model size (text_encoder) (float): 592 B
- Model size (text_encoder) (mixed_with_float): 13.3 KB
- Number of parameters (detector): 37.3M
- Number of parameters (text_encoder): 3.00

## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| detector | ONNX | float | Snapdragon® X2 Elite | 10.426 ms | 5 - 5 MB | NPU
| detector | ONNX | float | Snapdragon® X Elite | 21.594 ms | 71 - 71 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 15.848 ms | 0 - 331 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 82.452 ms | 1 - 325 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 35.18 ms | 5 - 13 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 21.019 ms | 0 - 77 MB | NPU
| detector | ONNX | float | Qualcomm® QCS8450 | 82.452 ms | 1 - 325 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 33.506 ms | 5 - 12 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 21.594 ms | 71 - 71 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 11.352 ms | 3 - 260 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Elite Mobile | 11.352 ms | 3 - 260 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 8.252 ms | 1 - 262 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® X2 Elite | 10.674 ms | 9 - 9 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® X Elite | 16.478 ms | 35 - 35 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 11.362 ms | 2 - 365 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Gen 1 Mobile | 25.631 ms | 2 - 363 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 15.652 ms | 6 - 14 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 16.014 ms | 8 - 11 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® QCS8450 | 25.631 ms | 2 - 363 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 17.237 ms | 6 - 14 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-X7181 | 16.478 ms | 35 - 35 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 11.243 ms | 8 - 314 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Elite Mobile | 11.243 ms | 8 - 314 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Elite Gen 5 Mobile | 9.44 ms | 4 - 316 MB | NPU
| detector | QNN_DLC | float | Snapdragon® X2 Elite | 10.354 ms | 5 - 5 MB | NPU
| detector | QNN_DLC | float | Snapdragon® X Elite | 21.052 ms | 5 - 5 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 15.376 ms | 5 - 367 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 81.716 ms | 5 - 377 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 34.453 ms | 5 - 12 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 116.247 ms | 1 - 292 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.178 ms | 5 - 7 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8775P | 30.364 ms | 2 - 295 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8650P | 30.364 ms | 2 - 295 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8255P | 30.364 ms | 2 - 295 MB | NPU
| detector | QNN_DLC | float | Qualcomm® QCS8450 | 81.716 ms | 5 - 377 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 30.784 ms | 7 - 14 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 21.052 ms | 5 - 5 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 10.918 ms | 0 - 270 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA7255P | 116.247 ms | 1 - 292 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8295P | 86.75 ms | 0 - 284 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 10.918 ms | 0 - 270 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 8.078 ms | 5 - 284 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® X2 Elite | 10.056 ms | 5 - 5 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 13.033 ms | 0 - 360 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 18.18 ms | 5 - 12 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 43.62 ms | 1 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 18.008 ms | 5 - 7 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® SA8775P | 18.231 ms | 0 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® SA8650P | 18.231 ms | 0 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® SA8255P | 18.231 ms | 0 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 18.814 ms | 7 - 13 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 11.04 ms | 5 - 308 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® SA7255P | 43.62 ms | 1 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Mobile | 11.04 ms | 5 - 308 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Gen 5 Mobile | 8.058 ms | 5 - 326 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 16.057 ms | 0 - 343 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 94.737 ms | 1 - 337 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 35.505 ms | 0 - 83 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 117.868 ms | 1 - 266 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 21.216 ms | 0 - 7 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8775P | 31.581 ms | 0 - 268 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8650P | 31.581 ms | 0 - 268 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8255P | 31.581 ms | 0 - 268 MB | NPU
| detector | TFLITE | float | Qualcomm® QCS8450 | 94.737 ms | 1 - 337 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 31.63 ms | 0 - 83 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 11.797 ms | 0 - 271 MB | NPU
| detector | TFLITE | float | Qualcomm® SA7255P | 117.868 ms | 1 - 266 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8295P | 83.191 ms | 0 - 265 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Elite Mobile | 11.797 ms | 0 - 271 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 9.883 ms | 0 - 273 MB | NPU
| text_encoder | ONNX | float | Snapdragon® X2 Elite | 42.773 ms | 1 - 1 MB | NPU
| text_encoder | ONNX | float | Snapdragon® X Elite | 97.059 ms | 550 - 550 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 65.375 ms | 1 - 2266 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 146.223 ms | 2 - 1863 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 102.452 ms | 1 - 4 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 97.003 ms | 0 - 644 MB | NPU
| text_encoder | ONNX | float | Qualcomm® QCS8450 | 146.223 ms | 2 - 1863 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 99.394 ms | 1 - 3 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 97.059 ms | 550 - 550 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 46.642 ms | 1 - 2019 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Elite Mobile | 46.642 ms | 1 - 2019 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 40.332 ms | 1 - 2264 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® X2 Elite | 42.617 ms | 1 - 1 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® X Elite | 97.181 ms | 550 - 550 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 65.815 ms | 1 - 2268 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Gen 1 Mobile | 145.25 ms | 2 - 1865 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 103.16 ms | 1 - 4 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 96.752 ms | 0 - 643 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® QCS8450 | 145.25 ms | 2 - 1865 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 99.51 ms | 1 - 3 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-X7181 | 97.181 ms | 550 - 550 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 48.085 ms | 1 - 2019 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Elite Mobile | 48.085 ms | 1 - 2019 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Elite Gen 5 Mobile | 39.855 ms | 1 - 2265 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® X2 Elite | 43.277 ms | 0 - 0 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® X Elite | 86.461 ms | 0 - 0 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 62.016 ms | 0 - 2450 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 140.009 ms | 0 - 3013 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 95.48 ms | 0 - 3 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 260.992 ms | 1 - 2036 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 85.173 ms | 0 - 3 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8775P | 95.567 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8650P | 95.567 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8255P | 95.567 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® QCS8450 | 140.009 ms | 0 - 3013 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 93.955 ms | 0 - 2 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 86.461 ms | 0 - 0 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 46.578 ms | 0 - 1973 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA7255P | 260.992 ms | 1 - 2036 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8295P | 107.476 ms | 0 - 1902 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 46.578 ms | 0 - 1973 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 34.527 ms | 0 - 1983 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® X2 Elite | 37.173 ms | 0 - 0 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 61.867 ms | 0 - 2451 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 95.226 ms | 0 - 3 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 261.037 ms | 0 - 2035 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 85.295 ms | 0 - 4 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8775P | 95.609 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8650P | 95.609 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8255P | 95.609 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 93.359 ms | 0 - 2 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 47.44 ms | 1 - 1975 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA7255P | 261.037 ms | 0 - 2035 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Mobile | 47.44 ms | 1 - 1975 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Gen 5 Mobile | 34.504 ms | 0 - 1983 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 72.829 ms | 1 - 3683 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 159.932 ms | 1 - 3564 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 109.605 ms | 0 - 553 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 289.737 ms | 1 - 2144 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 99.966 ms | 1 - 4 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8775P | 111.295 ms | 1 - 2144 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8650P | 111.295 ms | 1 - 2144 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8255P | 111.295 ms | 1 - 2144 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® QCS8450 | 159.932 ms | 1 - 3564 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 108.948 ms | 0 - 553 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 55.49 ms | 1 - 2073 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA7255P | 289.737 ms | 1 - 2144 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8295P | 112.85 ms | 1 - 2099 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Elite Mobile | 55.49 ms | 1 - 2073 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 41.511 ms | 0 - 2220 MB | NPU

## License
* The license for the original implementation of WeDetect can be found
  [here](https://github.com/WeChatCV/WeDetect/blob/main/README.md).

## References
* [WeDetect: Fast Open-Vocabulary Object Detection as Retrieval](https://arxiv.org/abs/2512.12309)
* [Source Model Implementation](https://github.com/WeChatCV/WeDetect)

## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).