Hugging Face's logo

qualcomm
/
Track-Anything

PyTorch
android
video-object-tracking
Model card Files
xet
Community
Track-Anything
File size: 10,817 Bytes 
6fc807c
 
 
 
 
 
 
 
 
 
 
d9297e5
6fc807c
 
 
d9297e5
 
 
 
 
 
 
 
 
 
 
 
 
 
7d1eab6
 
d9297e5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6fc807c
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
 
---
library_name: pytorch
license: other
tags:
- android
pipeline_tag: video-object-tracking

---

![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/track_anything/web-assets/model_demo.png)

# Track-Anything: Optimized for Qualcomm Devices

Track-Anything is a video based machine learning model to track an object in a video.

This is based on the implementation of Track-Anything found [here](https://github.com/gaomingqi/Track-Anything).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/track_anything) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

## Getting Started
There are two ways to deploy this model on your device:

### Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.37, ONNX Runtime 1.23.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/track_anything/releases/v0.46.0/track_anything-onnx-float.zip)
| TFLITE | float | Universal | QAIRT 2.42, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/track_anything/releases/v0.46.0/track_anything-tflite-float.zip)

For more device-specific assets and performance metrics, visit **[Track-Anything on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/track_anything)**.


### Option 2: Export with Custom Configurations

Use the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/track_anything) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for [Track-Anything on GitHub](https://github.com/quic/ai-hub-models/blob/main/qai_hub_models/models/track_anything) for usage instructions.

## Model Details

**Model Type:** Model_use_case.video_object_tracking

**Model Stats:**
- Model checkpoint: xmem
- Input resolution: 320x568
- Number of parameters (TrackAnythingEncodeKeyWithShrinkage): 9.72M
- Model size (TrackAnythingEncodeKeyWithShrinkage) (float): 37.1 MB
- Number of parameters (TrackAnythingEncodeValue): 23.3M
- Model size (TrackAnythingEncodeValue) (float): 88.8 MB
- Number of parameters (TrackAnythingEncodeKeyWithoutShrinkage): 9.71M
- Model size (TrackAnythingEncodeKeyWithoutShrinkage) (float): 37.1 MB
- Number of parameters (TrackAnythingSegment): 30.1M
- Model size (TrackAnythingSegment) (float): 115 MB

## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| TrackAnythingEncodeKeyWithShrinkage | ONNX | float | Snapdragon® X Elite | 5.562 ms | 19 - 19 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 3.893 ms | 10 - 152 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | ONNX | float | Qualcomm® QCS8550 (Proxy) | 5.415 ms | 0 - 20 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | ONNX | float | Qualcomm® QCS9075 | 9.595 ms | 10 - 13 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 3.175 ms | 0 - 115 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.572 ms | 0 - 117 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 4.098 ms | 0 - 217 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 31.289 ms | 3 - 181 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 5.889 ms | 3 - 34 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® SA8775P | 41.8 ms | 3 - 181 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® QCS9075 | 11.161 ms | 2 - 29 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 11.69 ms | 3 - 201 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® SA7255P | 31.289 ms | 3 - 181 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Qualcomm® SA8295P | 10.203 ms | 3 - 164 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 3.32 ms | 0 - 187 MB | NPU
| TrackAnythingEncodeKeyWithShrinkage | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.45 ms | 0 - 191 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | ONNX | float | Snapdragon® X Elite | 6.877 ms | 66 - 66 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 5.172 ms | 64 - 212 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | ONNX | float | Qualcomm® QCS8550 (Proxy) | 7.005 ms | 58 - 78 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | ONNX | float | Qualcomm® QCS9075 | 11.772 ms | 62 - 64 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.611 ms | 61 - 208 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.76 ms | 0 - 155 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 5.302 ms | 20 - 239 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 33.117 ms | 20 - 205 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 7.511 ms | 18 - 20 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® SA8775P | 11.533 ms | 20 - 206 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® QCS9075 | 12.674 ms | 20 - 63 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 14.205 ms | 20 - 221 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® SA7255P | 33.117 ms | 20 - 205 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Qualcomm® SA8295P | 12.482 ms | 20 - 184 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.413 ms | 19 - 205 MB | NPU
| TrackAnythingEncodeKeyWithoutShrinkage | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.729 ms | 0 - 229 MB | NPU
| TrackAnythingEncodeValue | ONNX | float | Snapdragon® X Elite | 12.251 ms | 38 - 38 MB | NPU
| TrackAnythingEncodeValue | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 10.039 ms | 0 - 150 MB | NPU
| TrackAnythingEncodeValue | ONNX | float | Qualcomm® QCS8550 (Proxy) | 15.664 ms | 0 - 52 MB | NPU
| TrackAnythingEncodeValue | ONNX | float | Qualcomm® QCS9075 | 19.384 ms | 6 - 14 MB | NPU
| TrackAnythingEncodeValue | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 8.45 ms | 7 - 127 MB | NPU
| TrackAnythingEncodeValue | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 6.443 ms | 4 - 125 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 8.276 ms | 2 - 290 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 51.6 ms | 3 - 224 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 11.859 ms | 3 - 6 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® SA8775P | 17.544 ms | 3 - 218 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® QCS9075 | 18.977 ms | 0 - 55 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 22.669 ms | 3 - 285 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® SA7255P | 51.6 ms | 3 - 224 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Qualcomm® SA8295P | 19.664 ms | 0 - 223 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.178 ms | 2 - 197 MB | NPU
| TrackAnythingEncodeValue | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 6.542 ms | 2 - 213 MB | NPU
| TrackAnythingSegment | ONNX | float | Snapdragon® X Elite | 23.784 ms | 38 - 38 MB | NPU
| TrackAnythingSegment | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 17.9 ms | 4 - 191 MB | NPU
| TrackAnythingSegment | ONNX | float | Qualcomm® QCS8550 (Proxy) | 24.373 ms | 17 - 28 MB | NPU
| TrackAnythingSegment | ONNX | float | Qualcomm® QCS9075 | 39.271 ms | 21 - 45 MB | NPU
| TrackAnythingSegment | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 14.036 ms | 5 - 166 MB | NPU
| TrackAnythingSegment | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 10.769 ms | 6 - 170 MB | NPU
| TrackAnythingSegment | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 17.532 ms | 0 - 283 MB | NPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 521.551 ms | 38 - 56 MB | GPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 23.497 ms | 0 - 239 MB | NPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® SA8775P | 33.249 ms | 0 - 196 MB | NPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® QCS9075 | 38.367 ms | 0 - 83 MB | NPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 54.61 ms | 2 - 285 MB | NPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® SA7255P | 521.551 ms | 38 - 56 MB | GPU
| TrackAnythingSegment | TFLITE | float | Qualcomm® SA8295P | 37.307 ms | 2 - 209 MB | NPU
| TrackAnythingSegment | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 13.587 ms | 1 - 204 MB | NPU
| TrackAnythingSegment | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 10.892 ms | 2 - 227 MB | NPU

## License
* The license for the original implementation of Track-Anything can be found
  [here](https://github.com/gaomingqi/Track-Anything/blob/master/LICENSE).

## References
* [Track Anything: Segment Anything Meets Videos](https://arxiv.org/abs/2304.11968)
* [Source Model Implementation](https://github.com/gaomingqi/Track-Anything)

## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).