| --- |
| license: cc-by-4.0 |
| language: |
| - en |
| library_name: voxrt |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - speech-recognition |
| - streaming |
| - asr |
| - fastconformer |
| - nemo |
| - nvidia |
| - on-device |
| - edge |
| - arm |
| - neon |
| - voxrt |
| base_model: |
| - nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc |
| --- |
| |
| # NeMo FastConformer streaming-medium-pc on the VoxRT runtime |
|
|
| The `stt_en_fastconformer_hybrid_medium_streaming_80ms_pc` model |
| from NVIDIA NeMo, packaged as a **`.vxrt`** file for the VoxRT |
| on-device inference runtime. Same weights, repackaged so the |
| 80 ms cache-aware streaming ASR runs on Android or iOS in |
| real-time — with punctuation and capitalisation output straight |
| from the model, no post-processing layer required. |
|
|
| **This is not our model.** The weights are NVIDIA's |
| `nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc` |
| checkpoint, released under CC-BY-4.0. What we ship is the runtime |
| that makes it fast on cheap ARM hardware plus the `.vxrt` |
| container that the runtime consumes. |
|
|
| ## Runtime performance |
|
|
| Cumulative RTF on a single CPU core, `arm64` release builds, |
| post-warmup. Live-mic figures are the production-realistic ones |
| (scheduler jitter + capture overhead included): |
|
|
| | Device | CPU | Decoder | Mode | RTF | |
| |----------------------------|------------|---------|-------------|-----------:| |
| | Xiaomi Redmi 9C (Android) | Cortex-A73 | RNN-T | file replay | **0.302** | |
| | Xiaomi Redmi 9C (Android) | Cortex-A73 | RNN-T | live mic | **0.353** | |
| | iPhone 13 Pro Max (iOS) | Apple A15 | RNN-T | live mic | **0.08–0.10** | |
|
|
| For the same weights, RNN-T decoding costs ~50 ms of CPU per |
| 1.12 s chunk on SD662; the CTC head is ~5 ms per chunk with a |
| minor WER hit. The SDK exposes both decoders — pick per your |
| battery / accuracy trade-off. |
|
|
| Chunked streaming granularity is **80 ms** cache-aware |
| look-ahead. Inherent end-to-end buffering is one chunk |
| (≈ 1.12 s at chunk_size=112) before text emission begins. |
| |
| ## Model quality |
| |
| Empirically validated on LibriSpeech test-clean (500-utterance |
| subset, matches the SDK repos' reported numbers): |
| |
| | Decoder | WER | Notes | |
| |-----------------|-------:|--------------------------------------------------| |
| | **RNN-T** ★ | **3.267 %** | Recommended default. Higher accuracy. | |
| | CTC | 4.895 % | ~15 % cheaper per chunk; long-session friendly. | |
| |
| Model architecture, training data, and topline WER claims are |
| NVIDIA's — see the upstream checkpoint at |
| [huggingface.co/nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc](https://huggingface.co/nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc). |
| |
| ## Download & use |
| |
| The `.vxrt` file on this HF repo is byte-identical to the one at |
| [github.com/VoxRT/voxrt-asr-models/releases](https://github.com/VoxRT/voxrt-asr-models/releases). |
| Either source is fine. |
| |
| `.vxrt` files cannot be loaded with `transformers`, `nemo_toolkit`, |
| or any standard HF library — they are a proprietary container |
| the VoxRT runtime reads. Use one of our SDKs: |
|
|
| - Android — [`voxrt-asr-android`](https://github.com/VoxRT/voxrt-asr-android) (JitPack) |
| - iOS — [`voxrt-asr-ios`](https://github.com/VoxRT/voxrt-asr-ios) (Swift Package) |
| - Linux aarch64 — available on request (contact `help@voxrt.com`) |
|
|
| ## Kotlin example |
|
|
| ```kotlin |
| import com.voxrt.asr.VoxrtAsrNative |
| import com.voxrt.asr.VoxrtAsrStreamingEngine |
| |
| val engine = VoxrtAsrStreamingEngine.fromAssetFd(modelFd) |
| // Or explicitly pick CTC: |
| // val engine = VoxrtAsrStreamingEngine.fromAssetFd(modelFd, VoxrtAsrNative.DECODE_CTC) |
| |
| val delta = engine.processPcm(pcmFloatArray) // text emitted this call |
| val tail = engine.stop() // drain remaining text |
| engine.close() |
| ``` |
|
|
| `engine.processPcm` / `stop` / `reset` / `close` are |
| **synchronous and stateful** — the engine doesn't own a worker |
| thread. Drive it from your own capture / IO thread; marshal text |
| deltas back to UI via `runOnUiThread` / a Flow / your preferred |
| concurrency. RNN-T (default) survives chunk boundaries via its |
| LSTM state; CTC dedupes across chunks internally. |
|
|
| ## Licensing |
|
|
| - **Model weights** are derived from |
| `nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc`, |
| © NVIDIA Corporation, CC-BY-4.0 licensed. |
| - **Repackaging** into the `.vxrt` container preserves the CC-BY-4.0 |
| obligations attached to the weights. Full notice lives at |
| [github.com/VoxRT/voxrt-asr-models/blob/main/LICENSE](https://github.com/VoxRT/voxrt-asr-models/blob/main/LICENSE). |
| - **The VoxRT runtime and `.vxrt` container format** are proprietary |
| Elephant Enterprises LLC IP. Redistribution allowed as an |
| unmodified part of the VoxRT SDKs above. |
|
|
| Attribution required by CC-BY-4.0: |
|
|
| > Speech recognition powered by NVIDIA NeMo FastConformer |
| > (streaming, medium, 80 ms look-ahead, P&C), |
| > © NVIDIA Corporation, licensed under CC-BY-4.0. |
|
|
| Include this line in your product's UI, docs, or credits when you |
| ship a product that runs this model. |
|
|
| ## About VoxRT |
|
|
| VoxRT is a from-scratch on-device inference runtime tuned for |
| streaming audio on commodity ARM CPUs — no GPU, no NPU, no vendor |
| accelerator required. Sister products on the same runtime: |
|
|
| - Wake-word: **["Hey Assistant" model + custom-phrase tier](https://huggingface.co/VoxRT/wake-word-hey-assistant-vxrt)** |
| - Voice activity detection: **[Silero VAD in `.vxrt`](https://huggingface.co/VoxRT/silero-vad-vxrt)** |
|
|
| Commercial integration / custom-model packaging: `help@voxrt.com` |
| · [voxrt.com](https://voxrt.com) |
|
|
