Hugging Face's logo

qualcomm
/
Distil-Whisper

Automatic Speech Recognition
PyTorch
android
Model card Files
xet
 Community
Distil-Whisper
File size: 10,105 Bytes
3686ec8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3b7bbad
 
 
3686ec8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ee4059d
 
 
 
 
 
 
 
3b7bbad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ee4059d
 
 
 
 
 
 
 
 
 
3b7bbad
 
ee4059d
 
 
 
 
 
 
 
3b7bbad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ee4059d
 
 
 
 
 
 
 
 
 
3b7bbad
 
3686ec8
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
---
library_name: pytorch
license: other
tags:
- android
pipeline_tag: automatic-speech-recognition

---

![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/distil_whisper/web-assets/model_demo.png)

# Distil-Whisper: Optimized for Qualcomm Devices

Distil-Whisper Small English is a distilled version of Whisper Small, optimized for fast and efficient automatic speech recognition.

This is based on the implementation of Distil-Whisper found [here](https://github.com/huggingface/distil-whisper).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/distil_whisper) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).

Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.

## Getting Started
There are two ways to deploy this model on your device:

### Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.3 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/distil_whisper/releases/v0.54.0/distil_whisper-onnx-float.zip)
| QNN_DLC | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/distil_whisper/releases/v0.54.0/distil_whisper-qnn_dlc-float.zip)
| TFLITE | float | Universal |  | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/distil_whisper/releases/v0.54.0/distil_whisper-tflite-float.zip)

For more device-specific assets and performance metrics, visit **[Distil-Whisper on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/distil_whisper)**.


### Option 2: Export with Custom Configurations

Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/distil_whisper) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for [Distil-Whisper on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/distil_whisper) for usage instructions.

## Model Details

**Model Type:** Model_use_case.speech_recognition

**Model Stats:**
- Model checkpoint: distil-whisper/distil-small.en
- Input resolution: 80x3000 (30 seconds audio)
- Max decoded sequence length: 200 tokens
- Number of parameters (encoder): 166M
- Model size (encoder) (float): 332 MB
- Number of parameters (decoder): 211M
- Model size (decoder) (float): 450MB

## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| decoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.546 ms | 52 - 375 MB | NPU
| decoder | ONNX | float | Snapdragon® 8 Elite Mobile | 7.178 ms | 16 - 472 MB | NPU
| decoder | ONNX | float | Snapdragon® X2 Elite | 5.083 ms | 178 - 178 MB | NPU
| decoder | ONNX | float | Snapdragon® X Elite | 11.41 ms | 178 - 178 MB | NPU
| decoder | ONNX | float | Snapdragon® X Elite | 11.41 ms | 178 - 178 MB | NPU
| decoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.67 ms | 52 - 426 MB | NPU
| decoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 11.746 ms | 0 - 184 MB | NPU
| decoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.178 ms | 16 - 472 MB | NPU
| decoder | ONNX | float | Qualcomm® QCS9075 | 13.188 ms | 40 - 82 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.934 ms | 39 - 538 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 7.351 ms | 4 - 548 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® X2 Elite | 5.954 ms | 40 - 40 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® X Elite | 10.825 ms | 40 - 40 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® X Elite | 10.825 ms | 40 - 40 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 8.656 ms | 0 - 600 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 11.398 ms | 40 - 634 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® SA8775P | 12.874 ms | 15 - 507 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® SA8775P | 12.874 ms | 15 - 507 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® SA8775P | 12.874 ms | 15 - 507 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® SA7255P | 19.195 ms | 31 - 528 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® SA8295P | 13.997 ms | 21 - 263 MB | NPU
| decoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.351 ms | 4 - 548 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® QCS9075 | 16.524 ms | 40 - 86 MB | NPU
| decoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 18.145 ms | 36 - 339 MB | NPU
| decoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.848 ms | 4 - 570 MB | NPU
| decoder | TFLITE | float | Snapdragon® 8 Elite Mobile | 7.215 ms | 4 - 572 MB | NPU
| decoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 8.619 ms | 4 - 749 MB | NPU
| decoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 11.713 ms | 5 - 7 MB | NPU
| decoder | TFLITE | float | Qualcomm® SA8775P | 13.037 ms | 5 - 537 MB | NPU
| decoder | TFLITE | float | Qualcomm® SA8775P | 13.037 ms | 5 - 537 MB | NPU
| decoder | TFLITE | float | Qualcomm® SA8775P | 13.037 ms | 5 - 537 MB | NPU
| decoder | TFLITE | float | Qualcomm® SA7255P | 19.275 ms | 4 - 537 MB | NPU
| decoder | TFLITE | float | Qualcomm® SA8295P | 14.231 ms | 5 - 297 MB | NPU
| decoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 7.215 ms | 4 - 572 MB | NPU
| decoder | TFLITE | float | Qualcomm® QCS9075 | 16.188 ms | 0 - 265 MB | NPU
| decoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 18.522 ms | 5 - 466 MB | NPU
| encoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 49.731 ms | 80 - 839 MB | NPU
| encoder | ONNX | float | Snapdragon® 8 Elite Mobile | 60.575 ms | 80 - 734 MB | NPU
| encoder | ONNX | float | Snapdragon® X2 Elite | 50.437 ms | 183 - 183 MB | NPU
| encoder | ONNX | float | Snapdragon® X Elite | 123.445 ms | 182 - 182 MB | NPU
| encoder | ONNX | float | Snapdragon® X Elite | 123.445 ms | 182 - 182 MB | NPU
| encoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 82.214 ms | 80 - 1239 MB | NPU
| encoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 118.816 ms | 0 - 202 MB | NPU
| encoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 60.575 ms | 80 - 734 MB | NPU
| encoder | ONNX | float | Qualcomm® QCS9075 | 150.604 ms | 79 - 83 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 57.546 ms | 1 - 712 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 71.556 ms | 1 - 691 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® X2 Elite | 60.179 ms | 1 - 1 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® X Elite | 139.254 ms | 1 - 1 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® X Elite | 139.254 ms | 1 - 1 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 96.947 ms | 0 - 963 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 135.243 ms | 1 - 4 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® SA8775P | 153.197 ms | 1 - 687 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® SA8775P | 153.197 ms | 1 - 687 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® SA8775P | 153.197 ms | 1 - 687 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® SA7255P | 437.44 ms | 1 - 696 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® SA8295P | 192.943 ms | 1 - 610 MB | NPU
| encoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 71.556 ms | 1 - 691 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® QCS9075 | 170.231 ms | 1 - 39 MB | NPU
| encoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 267.063 ms | 5 - 828 MB | NPU
| encoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 403.997 ms | 42 - 85 MB | GPU
| encoder | TFLITE | float | Snapdragon® 8 Elite Mobile | 409.903 ms | 40 - 80 MB | GPU
| encoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 475.908 ms | 42 - 184 MB | GPU
| encoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 657.568 ms | 0 - 318 MB | GPU
| encoder | TFLITE | float | Qualcomm® SA8775P | 1316.653 ms | 20 - 64 MB | GPU
| encoder | TFLITE | float | Qualcomm® SA8775P | 1316.653 ms | 20 - 64 MB | GPU
| encoder | TFLITE | float | Qualcomm® SA8775P | 1316.653 ms | 20 - 64 MB | GPU
| encoder | TFLITE | float | Qualcomm® SA7255P | 3135.306 ms | 24 - 69 MB | GPU
| encoder | TFLITE | float | Qualcomm® SA8295P | 671.062 ms | 38 - 81 MB | GPU
| encoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 409.903 ms | 40 - 80 MB | GPU
| encoder | TFLITE | float | Qualcomm® QCS9075 | 1271.896 ms | 0 - 40 MB | GPU
| encoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 852.126 ms | 39 - 193 MB | GPU

## License
* The license for the original implementation of Distil-Whisper can be found
  [here](https://github.com/huggingface/distil-whisper/blob/main/LICENSE).

## References
* [Distil-Whisper - Robust Knowledge Distillation via Large-Scale Pseudo Labelling](https://arxiv.org/abs/2311.00430)
* [Source Model Implementation](https://github.com/huggingface/distil-whisper)

## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).