betterflow — Bengali streaming transducer (sherpa-onnx)
Streaming zipformer2 transducer exported to ONNX for on-device dictation. Emits partials DURING speech, so first-partial latency is constant rather than proportional to utterance length — the property batch whisper structurally lacks.
- encoder.int8.onnx (24 MB) + decoder.onnx (2.0 MB) + joiner.onnx (0.98 MB) + tokens.txt (500 BPE tokens) + bpe.model — 27 MB total
- Load with sherpa-onnx
OnlineRecognizer.from_transducer,model_type="zipformer2" - Source weights: alphacep/vosk-model-small-streaming-bn, Apache-2.0, trained by Alphacephei (Vosk) with k2-fsa/icefall. Weights unmodified but for int8 quantization of the encoder.
Mirrored so betterflow can pin an immutable revision and checksum.
‼️ This is a TRANSDUCER, not CTC — four files, not two
Our Hindi bundle is a Zipformer2-CTC: encoder.int8.onnx + tokens.txt. This one needs
encoder + decoder + joiner. ModelHandle.kindOf resolves by inspecting the file list, so the
manifest partial.files array has four entries.
‼️ Only the ENCODER is int8-quantized. That is deliberate.
Quantizing decoder and joiner as well produces degenerate repetitive output. They are ~1–2 MB
networks with almost no weights to quantize, and they sit in the autoregressive loop where error
compounds token over token. The encoder is 86 MB of feed-forward weights and quantizes cleanly
(86.78 MB → 24.08 MB, 3.6×). This matches the sherpa-onnx zoo convention.
‼️ The upstream fp32 encoder does not decode correctly; this int8 one does
The raw encoder.onnx returns near-empty output under sherpa-onnx 1.13.4 ("", আমার, ফুল),
while the int8 graph built from it returns full coherent sentences. Upstream ships am-onnx/opt.py
precisely because the raw export needs ONNX graph optimization; quantize_dynamic's rewriting does
the equivalent. Do not "fix" this by reverting to fp32 — and do not read a fp32-vs-int8
comparison as evidence that quantization degraded anything.
Verified
Decoded through sherpa-onnx 1.13.4 (the version vendored in the betterflow client, so this exercises the engine that actually ships):
- 100% Bengali script across 3 clips, coherent rather than merely Bengali-shaped — one test clip returns the JFK line in full: তোমার দেশ তোমার জন্য কি করতে পারে তা জিজ্ঞেস করো না…
- Deterministic: byte-identical inputs produce byte-identical output.
‼️ Not verified
- WER is UNMEASURED. No Bengali eval set exists. Do not assume it matches our Hindi model.
- First-partial latency unmeasured.
- No punctuation.
tokens.txtcontains 500 tokens and zero punctuation — no danda (।), no comma, no question mark. Output is an unbroken run of words. This is not specific to Bengali: our shipped Hindi streaming model has zero punctuation too. Punctuation is a text-cleanup responsibility, not a model one.