Hugging Face's logo

qualcomm
/
ACT

Robotics
PyTorch
android
Model card Files
xet
 Community
qaihm-bot commited on
Commit
ac143f0
·
verified ·
1 Parent(s): 818f731

v0.58.0

Browse files

See https://github.com/qualcomm/ai-hub-models/releases/v0.58.0 for changelog.

Files changed (2) hide show
  1. README.md +48 -45
  2. release_assets.json +4 -4
README.md CHANGED
@@ -14,7 +14,7 @@ pipeline_tag: robotics
14
  ACT (Action Chunking with Transformers) is a robotic policy model that is trained to predict the next chunk of actions that the robotic hand is expected to perform.
15
 
16
  This is based on the implementation of ACT found [here](https://github.com/tonyzhaozh/act).
17
- This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.57.3/src/qai_hub_models/models/act) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
 
19
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
 
@@ -27,23 +27,23 @@ Below are pre-exported model assets ready for deployment.
27
 
28
  | Runtime | Precision | Chipset | SDK Versions | Download |
29
  |---|---|---|---|---|
30
- | ONNX | float | Universal | QAIRT 2.45, ONNX Runtime 1.25.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.57.3/act-onnx-float.zip)
31
- | QNN_DLC | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.57.3/act-qnn_dlc-float.zip)
32
- | TFLITE | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.57.3/act-tflite-float.zip)
33
 
34
  For more device-specific assets and performance metrics, visit **[ACT on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/act)**.
35
 
36
 
37
  ### Option 2: Export with Custom Configurations
38
 
39
- Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.57.3/src/qai_hub_models/models/act) Python library to compile and export the model with your own:
40
  - Custom weights (e.g., fine-tuned checkpoints)
41
  - Custom input shapes
42
  - Target device and runtime configurations
43
 
44
  This option is ideal if you need to customize the model beyond the default configuration provided here.
45
 
46
- See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.57.3/src/qai_hub_models/models/act) for usage instructions.
47
 
48
  ## Model Details
49
 
@@ -58,45 +58,48 @@ See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models
58
  ## Performance Summary
59
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
60
  |---|---|---|---|---|---|---
61
- | ACT | ONNX | float | Snapdragon® X2 Elite | 6.721 ms | 209 - 209 MB | NPU
62
- | ACT | ONNX | float | Snapdragon® X Elite | 12.758 ms | 149 - 149 MB | NPU
63
- | ACT | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.59 ms | 4 - 501 MB | NPU
64
- | ACT | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 19.508 ms | 4 - 309 MB | NPU
65
- | ACT | ONNX | float | Qualcomm® QCS8550 (Proxy) | 12.422 ms | 0 - 80 MB | NPU
66
- | ACT | ONNX | float | Qualcomm® QCS8450 | 19.508 ms | 4 - 309 MB | NPU
67
- | ACT | ONNX | float | Snapdragon® 8 Elite Mobile | 6.84 ms | 0 - 412 MB | NPU
68
- | ACT | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.977 ms | 2 - 442 MB | NPU
69
- | ACT | ONNX | float | Qualcomm® QCS9075 | 20.199 ms | 4 - 49 MB | NPU
70
- | ACT | ONNX | float | Qualcomm® QCS8750 | 6.84 ms | 0 - 412 MB | NPU
71
- | ACT | ONNX | float | Qualcomm® QCS7181 | 12.758 ms | 149 - 149 MB | NPU
72
- | ACT | QNN_DLC | float | Snapdragon® X2 Elite | 4.896 ms | 4 - 4 MB | NPU
73
- | ACT | QNN_DLC | float | Snapdragon® X Elite | 9.178 ms | 4 - 4 MB | NPU
74
- | ACT | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 6.066 ms | 0 - 338 MB | NPU
75
- | ACT | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 16.407 ms | 4 - 253 MB | NPU
76
- | ACT | QNN_DLC | float | Qualcomm® QCS8275 | 44.737 ms | 0 - 284 MB | NPU
77
- | ACT | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 8.548 ms | 4 - 26 MB | NPU
78
- | ACT | QNN_DLC | float | Qualcomm® QCS8450 | 16.407 ms | 4 - 253 MB | NPU
79
- | ACT | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 4.696 ms | 4 - 272 MB | NPU
80
- | ACT | QNN_DLC | float | Qualcomm® SA8295P | 15.267 ms | 2 - 206 MB | NPU
81
- | ACT | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.751 ms | 3 - 299 MB | NPU
82
- | ACT | QNN_DLC | float | Qualcomm® SA7255P | 44.737 ms | 0 - 284 MB | NPU
83
- | ACT | QNN_DLC | float | Qualcomm® QCS9075 | 15.789 ms | 4 - 9 MB | NPU
84
- | ACT | QNN_DLC | float | Qualcomm® QCS8750 | 4.696 ms | 4 - 272 MB | NPU
85
- | ACT | QNN_DLC | float | Qualcomm® QCS7181 | 9.178 ms | 4 - 4 MB | NPU
86
- | ACT | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 6.106 ms | 0 - 415 MB | NPU
87
- | ACT | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 16.399 ms | 0 - 271 MB | NPU
88
- | ACT | TFLITE | float | Qualcomm® QCS8275 | 44.759 ms | 0 - 330 MB | NPU
89
- | ACT | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 8.581 ms | 0 - 4 MB | NPU
90
- | ACT | TFLITE | float | Qualcomm® SA8775P | 323.031 ms | 3 - 13 MB | CPU
91
- | ACT | TFLITE | float | Qualcomm® SA8650P | 323.031 ms | 3 - 13 MB | CPU
92
- | ACT | TFLITE | float | Qualcomm® SA8255P | 323.031 ms | 3 - 13 MB | CPU
93
- | ACT | TFLITE | float | Qualcomm® QCS8450 | 16.399 ms | 0 - 271 MB | NPU
94
- | ACT | TFLITE | float | Snapdragon® 8 Elite Mobile | 4.728 ms | 0 - 284 MB | NPU
95
- | ACT | TFLITE | float | Qualcomm® SA8295P | 15.759 ms | 0 - 209 MB | NPU
96
- | ACT | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.754 ms | 0 - 324 MB | NPU
97
- | ACT | TFLITE | float | Qualcomm® SA7255P | 44.759 ms | 0 - 330 MB | NPU
98
- | ACT | TFLITE | float | Qualcomm® QCS9075 | 15.175 ms | 0 - 71 MB | NPU
99
- | ACT | TFLITE | float | Qualcomm® QCS8750 | 4.728 ms | 0 - 284 MB | NPU
 
 
 
100
 
101
  ## License
102
  * The license for the original implementation of ACT can be found
 
14
  ACT (Action Chunking with Transformers) is a robotic policy model that is trained to predict the next chunk of actions that the robotic hand is expected to perform.
15
 
16
  This is based on the implementation of ACT found [here](https://github.com/tonyzhaozh/act).
17
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.58.0/src/qai_hub_models/models/act) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
 
19
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
 
 
27
 
28
  | Runtime | Precision | Chipset | SDK Versions | Download |
29
  |---|---|---|---|---|
30
+ | ONNX | float | Universal | QAIRT 2.45, ONNX Runtime 1.25.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.58.0/act-onnx-float.zip)
31
+ | QNN_DLC | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.58.0/act-qnn_dlc-float.zip)
32
+ | TFLITE | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.58.0/act-tflite-float.zip)
33
 
34
  For more device-specific assets and performance metrics, visit **[ACT on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/act)**.
35
 
36
 
37
  ### Option 2: Export with Custom Configurations
38
 
39
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.58.0/src/qai_hub_models/models/act) Python library to compile and export the model with your own:
40
  - Custom weights (e.g., fine-tuned checkpoints)
41
  - Custom input shapes
42
  - Target device and runtime configurations
43
 
44
  This option is ideal if you need to customize the model beyond the default configuration provided here.
45
 
46
+ See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.58.0/src/qai_hub_models/models/act) for usage instructions.
47
 
48
  ## Model Details
49
 
 
58
  ## Performance Summary
59
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
60
  |---|---|---|---|---|---|---
61
+ | ACT | ONNX | float | Snapdragon® X2 Elite | 6.716 ms | 4 - 4 MB | NPU
62
+ | ACT | ONNX | float | Snapdragon® X Elite | 12.675 ms | 63 - 63 MB | NPU
63
+ | ACT | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.627 ms | 4 - 511 MB | NPU
64
+ | ACT | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 19.486 ms | 4 - 320 MB | NPU
65
+ | ACT | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 12.151 ms | 0 - 79 MB | NPU
66
+ | ACT | ONNX | float | Qualcomm® QCS8450 | 19.486 ms | 4 - 320 MB | NPU
67
+ | ACT | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 18.022 ms | 4 - 10 MB | NPU
68
+ | ACT | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.984 ms | 1 - 444 MB | NPU
69
+ | ACT | ONNX | float | Snapdragon® 8 Elite Mobile | 6.868 ms | 0 - 408 MB | NPU
70
+ | ACT | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 6.868 ms | 0 - 408 MB | NPU
71
+ | ACT | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 12.675 ms | 63 - 63 MB | NPU
72
+ | ACT | QNN_DLC | float | Snapdragon® X2 Elite | 4.804 ms | 4 - 4 MB | NPU
73
+ | ACT | QNN_DLC | float | Snapdragon® X Elite | 9.198 ms | 4 - 4 MB | NPU
74
+ | ACT | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 6.074 ms | 0 - 336 MB | NPU
75
+ | ACT | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 16.419 ms | 3 - 253 MB | NPU
76
+ | ACT | QNN_DLC | float | Qualcomm® QCS8275 | 44.71 ms | 1 - 285 MB | NPU
77
+ | ACT | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 8.531 ms | 4 - 5 MB | NPU
78
+ | ACT | QNN_DLC | float | Qualcomm® SA8775P | 13.792 ms | 1 - 269 MB | NPU
79
+ | ACT | QNN_DLC | float | Qualcomm® SA8650P | 13.792 ms | 1 - 269 MB | NPU
80
+ | ACT | QNN_DLC | float | Qualcomm® SA8255P | 13.792 ms | 1 - 269 MB | NPU
81
+ | ACT | QNN_DLC | float | Qualcomm® QCS8450 | 16.419 ms | 3 - 253 MB | NPU
82
+ | ACT | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 13.728 ms | 4 - 9 MB | NPU
83
+ | ACT | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.745 ms | 4 - 300 MB | NPU
84
+ | ACT | QNN_DLC | float | Qualcomm® SA7255P | 44.71 ms | 1 - 285 MB | NPU
85
+ | ACT | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 4.717 ms | 4 - 273 MB | NPU
86
+ | ACT | QNN_DLC | float | Qualcomm® SA8295P | 15.254 ms | 0 - 204 MB | NPU
87
+ | ACT | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 4.717 ms | 4 - 273 MB | NPU
88
+ | ACT | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 9.198 ms | 4 - 4 MB | NPU
89
+ | ACT | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 6.105 ms | 0 - 409 MB | NPU
90
+ | ACT | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 16.37 ms | 0 - 271 MB | NPU
91
+ | ACT | TFLITE | float | Qualcomm® QCS8275 | 44.735 ms | 0 - 326 MB | NPU
92
+ | ACT | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 8.448 ms | 0 - 3 MB | NPU
93
+ | ACT | TFLITE | float | Qualcomm® SA8775P | 13.83 ms | 0 - 279 MB | NPU
94
+ | ACT | TFLITE | float | Qualcomm® SA8650P | 13.83 ms | 0 - 279 MB | NPU
95
+ | ACT | TFLITE | float | Qualcomm® SA8255P | 13.83 ms | 0 - 279 MB | NPU
96
+ | ACT | TFLITE | float | Qualcomm® QCS8450 | 16.37 ms | 0 - 271 MB | NPU
97
+ | ACT | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 13.582 ms | 0 - 71 MB | NPU
98
+ | ACT | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.791 ms | 0 - 324 MB | NPU
99
+ | ACT | TFLITE | float | Qualcomm® SA7255P | 44.735 ms | 0 - 326 MB | NPU
100
+ | ACT | TFLITE | float | Snapdragon�� 8 Elite Mobile | 4.733 ms | 0 - 283 MB | NPU
101
+ | ACT | TFLITE | float | Qualcomm® SA8295P | 15.739 ms | 0 - 208 MB | NPU
102
+ | ACT | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 4.733 ms | 0 - 283 MB | NPU
103
 
104
  ## License
105
  * The license for the original implementation of ACT can be found
release_assets.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "version": "0.57.3",
3
  "precisions": {
4
  "float": {
5
  "universal_assets": {
@@ -8,20 +8,20 @@
8
  "qairt": "2.45.0.260326154327",
9
  "litert": "1.4.4"
10
  },
11
- "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.57.3/act-tflite-float.zip"
12
  },
13
  "qnn_dlc": {
14
  "tool_versions": {
15
  "qairt": "2.45.0.260326154327"
16
  },
17
- "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.57.3/act-qnn_dlc-float.zip"
18
  },
19
  "onnx": {
20
  "tool_versions": {
21
  "qairt": "2.45.0.260326154327",
22
  "onnx_runtime": "1.25.0"
23
  },
24
- "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.57.3/act-onnx-float.zip"
25
  }
26
  }
27
  }
 
1
  {
2
+ "version": "0.58.0",
3
  "precisions": {
4
  "float": {
5
  "universal_assets": {
 
8
  "qairt": "2.45.0.260326154327",
9
  "litert": "1.4.4"
10
  },
11
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.58.0/act-tflite-float.zip"
12
  },
13
  "qnn_dlc": {
14
  "tool_versions": {
15
  "qairt": "2.45.0.260326154327"
16
  },
17
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.58.0/act-qnn_dlc-float.zip"
18
  },
19
  "onnx": {
20
  "tool_versions": {
21
  "qairt": "2.45.0.260326154327",
22
  "onnx_runtime": "1.25.0"
23
  },
24
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.58.0/act-onnx-float.zip"
25
  }
26
  }
27
  }