Betterflow — SraVaani streaming, ONNX int8 for sherpa-onnx
An ONNX export of ARTPARK-IISc/SraVaani-0.5-live, prepared so it loads in
sherpa_onnx.OnlineRecognizer and produces live partials for nine Indian languages.
We are not the authors of the weights. Upstream is ARTPARK-IISc; this repo is a format conversion plus quantization.
Provenance and licence
| Upstream | ARTPARK-IISc/SraVaani-0.5-live |
| Upstream licence | MIT (read off the model card, not inferred) |
| This repo | MIT, inherited |
| What changed | fp16 TorchScript → ONNX (opset 17) · MatMul-only int8 · per-window normalization folded into the graph · sherpa metadata stamped |
| What did NOT change | the weights — no fine-tuning, no distillation |
Please cite ARTPARK-IISc for the underlying model.
Why this export exists
Upstream ships fp16 TorchScript with trust_remote_code, which sherpa-onnx cannot load.
torch.jit.load turns out to expose a literal NeMo ConformerEncoder (5-in/5-out) plus a
ctc_decoder, and the shipped graph already carries sherpa's cache contract — so a single
scripted fusing wrapper was enough.
Contents
model-la13.onnx 658,699,885 B MatMul-only int8, att_context_size [70,13]
tokens.txt 68,907 B 5,001 entries = the model's output dim
export-la13.json export settings and provenance
Measured
vaani-{hi,gu}, n=50 each, through sherpa with a padded tail, same normalizer as our other
evaluations:
| WER pooled | RTF | peak RSS | empty | |
|---|---|---|---|---|
hi |
29.6% | 0.059 | 1,536 MB | 0/50 |
gu |
30.3% | 0.059 | 1,535 MB | 0/50 |
The TorchScript parent scores 29.2 / 28.8 on the same clips, so export + sherpa costs under 1.5 points. Partials arrive incrementally and monotonically — they never rewrite:
'बहुत बड़े' → 'बहुत बड़े नहर' → 'बहुत बड़े नहर जो बह रही' → 'बहुत बड़े नहर जो बह रही है'
⚠️ Four things that will bite you
1. Pad the tail, or lose 14 WER points. sherpa never decodes the trailing partial chunk. After
InputFinished(), push ~1,300 ms of silence before reading the final result.
Unpadded hi scores 43.7%; padded, 29.7%.
2. Use [70,13], not [70,6]. The shorter lookahead is worse, not merely faster —
hi 29.7 → 33.2, gu 30.2 → 36.3.
3. Normalization is folded in — do not strip metadata. With the wrong normalization this model loads, runs, and returns EMPTY with no error. Acceptance must be a decode producing the right script, never a successful load.
4. It CODE-SWITCHES, and a naive script-purity gate will reject it for that. Own-script share is
75.6–85.7%, but ~90% of the Latin on Hindi is real English words (bank, computer). If you
gate on script purity, gate on the non-word fraction, or you will reject the behaviour that makes
it useful for code-switched speech.
Cost, and one structural advantage
Peak RSS is FLAT in utterance length — 1,531 MB at 5 s, 1,575 MB at 134 s. A streaming model has bounded context and therefore bounded memory, unlike an offline Conformer whose attention is O(T²). No utterance-length cap is needed.
int8 is close to free here: 1.79 GB → 659 MB on disk for 29.7 → 29.6 WER.
Not evaluated
Device/Android verification · the [70,1] and [70,0] arms · languages beyond hi/gu through
this export (the parent was scored on all nine) · dictation-register audio — everything above is
read/prompted speech.