---
license: apache-2.0
language:
  - en
  - de
  - fr
  - es
  - it
  - pt
  - nl
  - pl
  - ta
  - hi
  - te
  - kn
  - ml
tags:
  - automatic-speech-recognition
  - whisper
  - sherpa-onnx
  - onnx
  - on-device
  - quantized
library_name: sherpa-onnx
pipeline_tag: automatic-speech-recognition
base_model:
  - openai/whisper-tiny
  - vasista22/whisper-tamil-small
  - vasista22/whisper-hindi-small
  - vasista22/whisper-telugu-tiny
  - vasista22/whisper-kannada-tiny
  - kavyamanohar/whisper-small-malayalam
---

# indic-asr-onnx

INT8-quantized Whisper models in [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)
format, packaged for on-device speech recognition.

## Packs

Each pack contains `encoder.int8.onnx`, `decoder.int8.onnx`, and `tokens.txt`,
and is downloaded independently at runtime.

| Pack | Languages | Size | Source model |
|---|---|---|---|
| `whisper-tiny` | en, de, fr, es, it, pt, nl, pl (+90 more) | 99 MB | [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) |
| `whisper-small-ta` | Tamil | 358 MB | [vasista22/whisper-tamil-small](https://huggingface.co/vasista22/whisper-tamil-small) |
| `whisper-small-hi` | Hindi | 358 MB | [vasista22/whisper-hindi-small](https://huggingface.co/vasista22/whisper-hindi-small) |
| `whisper-small-ml` | Malayalam | 358 MB | [kavyamanohar/whisper-small-malayalam](https://huggingface.co/kavyamanohar/whisper-small-malayalam) |
| `whisper-tiny-te` | Telugu | 99 MB | [vasista22/whisper-telugu-tiny](https://huggingface.co/vasista22/whisper-telugu-tiny) |
| `whisper-tiny-kn` | Kannada | 99 MB | [vasista22/whisper-kannada-tiny](https://huggingface.co/vasista22/whisper-kannada-tiny) |

`whisper-tiny` is the multilingual base model — it serves all European languages
from a single download. The Indic packs are language-specific fine-tunes, which
substantially outperform stock Whisper on those languages.

## Structure

```
models/
├── whisper-tiny/
│   ├── encoder.int8.onnx
│   ├── decoder.int8.onnx
│   └── tokens.txt
├── whisper-small-ta/
├── whisper-small-hi/
├── whisper-small-ml/
├── whisper-tiny-te/
└── whisper-tiny-kn/
```

## Usage

### Python (sherpa-onnx)

```python
import sherpa_onnx

recognizer = sherpa_onnx.OfflineRecognizer.from_whisper(
    encoder="models/whisper-small-ta/encoder.int8.onnx",
    decoder="models/whisper-small-ta/decoder.int8.onnx",
    tokens="models/whisper-small-ta/tokens.txt",
    language="ta",
    task="transcribe",
    num_threads=2,
)

stream = recognizer.create_stream()
stream.accept_waveform(16000, samples)   # float32 PCM, 16 kHz mono
recognizer.decode_stream(stream)
print(stream.result.text)
```

### Android (Kotlin)

```kotlin
val config = OfflineRecognizerConfig(
    featConfig = FeatureConfig(sampleRate = 16000, featureDim = 80),
    modelConfig = OfflineModelConfig(
        whisper = OfflineWhisperModelConfig(
            encoder = "$packDir/encoder.int8.onnx",
            decoder = "$packDir/decoder.int8.onnx",
            language = "ta",
            task = "transcribe",
        ),
        tokens = "$packDir/tokens.txt",
        numThreads = 2,
    )
)
val recognizer = OfflineRecognizer(config)
```

## Export

Exported with sherpa-onnx's
[`scripts/whisper/export-onnx.py`](https://github.com/k2-fsa/sherpa-onnx/blob/master/scripts/whisper/export-onnx.py),
then dynamically quantized to INT8.

HuggingFace fine-tunes were first converted to OpenAI Whisper checkpoint format
(`{"dims": ..., "model_state_dict": ...}`) so the export script could load them.

Note: on PyTorch ≥ 2.9 the export script requires `dynamo=False` to be passed to
`torch.onnx.export` — the dynamo-based exporter fails on Whisper's
data-dependent positional embedding indexing.

## Credits

- Fine-tuned Indic models by **[vasista22](https://huggingface.co/vasista22)**
  (Speech Lab, IIT Madras — funded by Bhashini, MeitY, Govt. of India) and
  **[kavyamanohar](https://huggingface.co/kavyamanohar)**
- Base models by **OpenAI** ([Whisper](https://github.com/openai/whisper))
- Export tooling by **[k2-fsa](https://github.com/k2-fsa)** (sherpa-onnx)

## License

Apache 2.0, inherited from the upstream models.