| --- |
| license: mit |
| library_name: sherpa-onnx |
| tags: |
| - automatic-speech-recognition |
| - indic |
| - onnx |
| - sherpa-onnx |
| - nemo |
| language: |
| - gu |
| - pa |
| - bn |
| - mr |
| - ml |
| - te |
| - ta |
| - kn |
| --- |
| |
| # Betterflow — IndicConformer CTC, ONNX int8 (8 Indic languages) |
|
|
| ONNX int8 exports of **AI4Bharat's per-language IndicConformer** hybrid CTC/RNNT checkpoints, |
| prepared for on-device offline ASR through `sherpa-onnx`'s `OfflineRecognizer.from_nemo_ctc`. |
|
|
| **These are derived artefacts. We are not the authors of the weights.** |
|
|
| ## Provenance and licence |
|
|
| | | | |
| |---|---| |
| | Upstream | `ai4bharat/indicconformer_stt_<lang>_hybrid_ctc_rnnt_large` | |
| | Upstream licence | **MIT** (verified on each source model card, not inferred) | |
| | This repo's licence | **MIT**, inherited | |
| | What changed | CTC head exported to ONNX; dynamic int8 quantization; sherpa metadata stamped | |
| | What did NOT change | the weights themselves — no fine-tuning, no distillation | |
|
|
| Each language directory carries `export-meta.json` with the **sha256 of the source `.nemo`**, so |
| provenance is checkable rather than asserted. |
|
|
| Please cite AI4Bharat for the underlying models. |
|
|
| ## Contents |
|
|
| Eight languages: `gu` `pa` `bn` `mr` `ml` `te` `ta` `kn`. |
|
|
| ``` |
| <lang>/model.int8.onnx 140,337,395 B (identical size across languages, DISTINCT weights) |
| <lang>/tokens.txt 67,605 B 5,633 entries |
| <lang>/export-meta.json source sha256, checksums, export settings |
| ``` |
|
|
| ### int8 sha256 — all eight distinct |
|
|
| | lang | sha256 (first 16) | lang | sha256 (first 16) | |
| |---|---|---|---| |
| | `gu` | `9f8031996be1a4d7` | `ml` | `81c172f6ee3796f5` | |
| | `pa` | `1ad447f477040bbb` | `te` | `b0b3d335f2505ecd` | |
| | `bn` | `8b1bd509bb36ba86` | `ta` | `08b85596996429e1` | |
| | `mr` | `af22de1897679e4a` | `kn` | `16d950f36cfdb01e` | |
|
|
| The eight files are byte-identical **in size** because they share an architecture. They are |
| **different models** — the checksums above are the check that matters. |
|
|
| ## ⚠️ Two things that will silently break this |
|
|
| **1. `normalize_type` must be `per_feature`.** It is stamped on every graph. If it is stripped or |
| set to `""`, the model **loads, runs, and returns EMPTY with no error**. Verify by decoding, never |
| by a successful load. |
|
|
| Required ONNX metadata: `vocab_size=5633`, `subsampling_factor=4`, |
| `normalize_type=per_feature`, `feature_dim=80`. |
|
|
| **2. `tokens.txt` opens with Bengali tokens in every language directory — this is correct.** The |
| vocabulary is AI4Bharat's shared 22-language tokenizer (5,633 = 22 × 256 + 1) and the CTC head is |
| multi-softmax. It looks like a packaging error and is not. The file is byte-identical across all |
| eight (`ee60967630213f31…`); it is duplicated per directory so each bundle installs independently. |
|
|
| Unlike the 600m multilingual checkpoint, these per-language models **did** learn to suppress the |
| other 21 blocks — measured 100% own-script output on all eight, 0/400 wrong-alphabet characters — |
| so **no `language_id` masking is required**. |
| |
| ## Measured quality |
| |
| Scored on held-out `ARTPARK-IISc/Vaani`, 50 clips × 50 speakers per language, against the |
| general-purpose multilingual Whisper bundle these replace. |
| |
| | lang | incumbent pooled WER | **this export** | script purity | catastrophic | |
| |---|---|---|---|---| |
| | `pa` | 106.2% | **20.2%** | 0% → **100%** | 13 → **0** | |
| | `gu` | 106.6% | **22.6%** | 0% → **100%** | 16 → **0** | |
| | `bn` | 135.1% | **30.6%** | 0% → **100%** | 21 → **0** | |
| | `mr` | 100.5% | **36.6%** | 83% → **100%** | 14 → **0** | |
| | `ml` | 111.9% | **37.2%** | 0% → **100%** | 16 → **1** | |
| | `te` | 115.5% | **45.0%** | 6% → **100%** | 8 → **1** | |
| | `ta` | 87.0% | **52.1%** | 100% → **100%** | 3 → **1** | |
| | `kn` | 100.3% | **59.5%** | 100% → **100%** | 2 → **1** | |
| |
| **Script purity is the headline, not WER.** A 0% score means the incumbent was not producing the |
| language's alphabet at all. Catastrophic failures fell from **93/400 to 5/400**. |
| |
| `te` and `kn` remain insertion-bound and are **not** recommended without further evaluation. |
| |
| ## Runtime cost |
| |
| ### ⚠️ Peak RSS scales with utterance length — it is not a constant |
| |
| Conformer self-attention is **O(T²)** in sequence length, so memory is a function of how long the |
| speaker talks. Measured in-process (Android arm64, sherpa-onnx 1.13.4, int8, 2 threads, one process |
| per utterance, ±1 MB reproducible): |
| |
| | utterance | peak RSS | | utterance | peak RSS | |
| |---|---|---|---|---| |
| | 5 s | **285 MB** | | 45 s | 868 MB | |
| | 10 s | **350 MB** | | 60 s | 986 MB | |
| | 20 s | **482 MB** | | 90 s | 1,628 MB | |
| | 30 s | 606 MB | | 134 s | **2,758 MB** | |
| |
| **Budget accordingly**: a typical 3–10 s dictation utterance costs under 350 MB, but a two-minute |
| monologue costs 2.7 GB. If you have a memory ceiling, cap utterance length — that is the cheapest |
| control and it belongs in your app, not in the model. |
| |
| | | | |
| |---|---| |
| | RTF | 0.062–0.064 (~16× real time on an M4 laptop) | |
| | RTF, 2× ARM Cortex-A55 @1.25 GHz | **1.33** (~13 s for a 10 s utterance) | |
| | RTF, 4× Cortex-A78 | **0.255** | |
| |
| The int8 build quantizes **all 54 `Conv` nodes** to `ConvInteger`. Leaving them in fp32 — as some |
| other re-exports do — makes ONNX Runtime allocate an im2col workspace that scales with clip length, |
| pushing peak RSS to **2,099–2,635 MB** from the same nominal precision. If you are getting 4× the |
| memory you expect from an int8 Conformer, check your `Conv` node dtypes. |
| |
| Cost of that choice: `ConvInteger` is ~3.6× slower than fp32 `Conv` on CPU. |
| |
| ## Not evaluated here |
| |
| Device-side accuracy, languages outside the eight, streaming/partial decoding (these are **offline** |
| models), and the RNNT head (not exported — it needs numba/CUDA). |
| |
