File size: 3,895 Bytes
4f3ff9e d8ca24c 4f3ff9e d8ca24c 4f3ff9e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | ---
license: cc-by-4.0
library_name: sherpa-onnx
tags:
- automatic-speech-recognition
- streaming
- cache-aware
- onnx
- nemo
- transducer
language: [en]
---
# Betterflow — English streaming FastConformer, ONNX for sherpa-onnx
An ONNX export of **`nvidia/stt_en_fastconformer_hybrid_large_streaming_multi`**, prepared so it
loads in **`sherpa_onnx.OnlineRecognizer`** and produces **live partials** for English.
**We are not the authors of the weights.** Upstream is NVIDIA; this repo is a format conversion
plus quantization.
## Provenance and licence
| | |
|---|---|
| Upstream | [`nvidia/stt_en_fastconformer_hybrid_large_streaming_multi`](https://huggingface.co/nvidia/stt_en_fastconformer_hybrid_large_streaming_multi) |
| Upstream licence | **CC-BY-4.0** (read off the model card, not inferred) |
| This repo | **CC-BY-4.0**, inherited — **attribution required** |
| What changed | `.nemo` → ONNX via k2-fsa's own export path · `set_default_att_context_size([70,13])` · int8 quantization |
| What did NOT change | the weights — no fine-tuning |
Please cite NVIDIA for the underlying model.
## Contents — a transducer bundle (three graphs)
```
encoder.int8.onnx 131,507,640 B encoder.onnx 456,772,215 B
decoder.int8.onnx 3,955,863 B decoder.onnx 15,753,087 B
joiner.int8.onnx 1,408,183 B joiner.onnx 5,584,035 B
tokens.txt 11,896 B
```
int8 total ≈ **137 MB**.
## Measured
`librispeech-en`, n=50, through sherpa with a padded tail:
| | |
|---|---|
| WER | **7.7% pooled · 5.3% median** |
| RTF | **0.021** |
| peak RSS | **662 MB** |
| empty | **0/50** |
| script | **100% Latin** |
Sample decode (int8, 2 s tail pad):
```
'concord returned to its place amidst the tents'
'congratulations were poured in upon the princess everywhere during her journey'
```
## ⚠️ Three things worth knowing
**1. Pad the tail — and this bundle tells you exactly how much.** sherpa's online recogniser only
decodes when `num_frames_ready - num_processed >= window_size`, and `input_finished()` does **not**
pad to a whole window, so up to `window_size - 1` frames of every utterance are never decoded.
This encoder declares **`window_size = 121`, `chunk_shift = 112`, `subsampling_factor = 8`** in its
ONNX metadata. At a 10 ms hop that is **1.21 s**, so **pad ≥ ~1.3 s**; we use 2,000 ms. Shorter pads
lose words as *deletions*, which read as poor model quality rather than as a configuration error.
> ‼️ **Read `window_size` off the graph rather than copying a number.** An earlier version of this
> card quoted a `0 → 20.7% · 500 → 10.4% · 2,000 → 5.4%` sweep as if it were measured on this bundle.
> **It was not** — those are third-party figures from a *streaming zipformer* on Android, a different
> architecture whose chunk length we never read. The advice was right; the numbers were not ours.
**2. `downloadMb` is not `peakRssMb`.** 137 MB on disk, **662 MB resident** — a 4.9× gap. Budget on
the resident figure.
**3. Peak RSS is FLAT in utterance length** — **671.2 MB at 5 s, 671.5 MB at 240 s**. **No
utterance-length cap is needed** for this bundle.
The reason is the **cache-aware architecture** — bounded left context plus a fixed cache — not the
fact that it streams. ‼️ **"Streaming ⇒ bounded memory" is false as a general rule**: we measured a
streaming decoder-only model whose peak RSS scales **T^1.49** and walls at ~10.6 s of audio. Flat
memory is a property of *this family* (cache-aware Conformer), and an offline Conformer's attention
is O(T²). **Check the scaling; do not infer it from the word "streaming."**
## Not evaluated
Device/Android verification · lookaheads other than `[70,13]` (`0`/`80`/`480` ms are exportable via
the same script) · languages other than English · dictation-register audio — the numbers above are
read speech.
|