Automatic Speech Recognition
NeMo
English
voxrt
speech-recognition
streaming
asr
fastconformer
nvidia
on-device
edge
arm
neon
Instructions to use VoxRT/streaming-medium-pc-vxrt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NeMo
How to use VoxRT/streaming-medium-pc-vxrt with NeMo:
import nemo.collections.asr as nemo_asr asr_model = nemo_asr.models.ASRModel.from_pretrained("VoxRT/streaming-medium-pc-vxrt") transcriptions = asr_model.transcribe(["file.wav"]) - Notebooks
- Google Colab
- Kaggle
Add initial model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-4.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
library_name: voxrt
|
| 6 |
+
pipeline_tag: automatic-speech-recognition
|
| 7 |
+
tags:
|
| 8 |
+
- speech-recognition
|
| 9 |
+
- streaming
|
| 10 |
+
- asr
|
| 11 |
+
- fastconformer
|
| 12 |
+
- nemo
|
| 13 |
+
- nvidia
|
| 14 |
+
- on-device
|
| 15 |
+
- edge
|
| 16 |
+
- arm
|
| 17 |
+
- neon
|
| 18 |
+
- voxrt
|
| 19 |
+
base_model:
|
| 20 |
+
- nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
# NeMo FastConformer streaming-medium-pc on the VoxRT runtime
|
| 24 |
+
|
| 25 |
+
The `stt_en_fastconformer_hybrid_medium_streaming_80ms_pc` model
|
| 26 |
+
from NVIDIA NeMo, packaged as a **`.vxrt`** file for the VoxRT
|
| 27 |
+
on-device inference runtime. Same weights, repackaged so the
|
| 28 |
+
80 ms cache-aware streaming ASR runs on Android or iOS in
|
| 29 |
+
real-time β with punctuation and capitalisation output straight
|
| 30 |
+
from the model, no post-processing layer required.
|
| 31 |
+
|
| 32 |
+
**This is not our model.** The weights are NVIDIA's
|
| 33 |
+
`nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc`
|
| 34 |
+
checkpoint, released under CC-BY-4.0. What we ship is the runtime
|
| 35 |
+
that makes it fast on cheap ARM hardware plus the `.vxrt`
|
| 36 |
+
container that the runtime consumes.
|
| 37 |
+
|
| 38 |
+
## Runtime performance
|
| 39 |
+
|
| 40 |
+
Cumulative RTF on a single CPU core, `arm64` release builds,
|
| 41 |
+
post-warmup. Live-mic figures are the production-realistic ones
|
| 42 |
+
(scheduler jitter + capture overhead included):
|
| 43 |
+
|
| 44 |
+
| Device | CPU | Decoder | Mode | RTF |
|
| 45 |
+
|----------------------------|------------|---------|-------------|-----------:|
|
| 46 |
+
| Xiaomi Redmi 9C (Android) | Cortex-A73 | RNN-T | file replay | **0.302** |
|
| 47 |
+
| Xiaomi Redmi 9C (Android) | Cortex-A73 | RNN-T | live mic | **0.353** |
|
| 48 |
+
| iPhone 13 Pro Max (iOS) | Apple A15 | RNN-T | live mic | **0.08β0.10** |
|
| 49 |
+
|
| 50 |
+
For the same weights, RNN-T decoding costs ~50 ms of CPU per
|
| 51 |
+
1.12 s chunk on SD662; the CTC head is ~5 ms per chunk with a
|
| 52 |
+
minor WER hit. The SDK exposes both decoders β pick per your
|
| 53 |
+
battery / accuracy trade-off.
|
| 54 |
+
|
| 55 |
+
Chunked streaming granularity is **80 ms** cache-aware
|
| 56 |
+
look-ahead. Inherent end-to-end buffering is one chunk
|
| 57 |
+
(β 1.12 s at chunk_size=112) before text emission begins.
|
| 58 |
+
|
| 59 |
+
## Model quality
|
| 60 |
+
|
| 61 |
+
Empirically validated on LibriSpeech test-clean (500-utterance
|
| 62 |
+
subset, matches the SDK repos' reported numbers):
|
| 63 |
+
|
| 64 |
+
| Decoder | WER | Notes |
|
| 65 |
+
|-----------------|-------:|--------------------------------------------------|
|
| 66 |
+
| **RNN-T** β
| **3.267 %** | Recommended default. Higher accuracy. |
|
| 67 |
+
| CTC | 4.895 % | ~15 % cheaper per chunk; long-session friendly. |
|
| 68 |
+
|
| 69 |
+
Model architecture, training data, and topline WER claims are
|
| 70 |
+
NVIDIA's β see the upstream checkpoint at
|
| 71 |
+
[huggingface.co/nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc](https://huggingface.co/nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc).
|
| 72 |
+
|
| 73 |
+
## Download & use
|
| 74 |
+
|
| 75 |
+
The `.vxrt` file on this HF repo is byte-identical to the one at
|
| 76 |
+
[github.com/VoxRT/voxrt-asr-models/releases](https://github.com/VoxRT/voxrt-asr-models/releases).
|
| 77 |
+
Either source is fine.
|
| 78 |
+
|
| 79 |
+
`.vxrt` files cannot be loaded with `transformers`, `nemo_toolkit`,
|
| 80 |
+
or any standard HF library β they are a proprietary container
|
| 81 |
+
the VoxRT runtime reads. Use one of our SDKs:
|
| 82 |
+
|
| 83 |
+
- Android β [`voxrt-asr-android`](https://github.com/VoxRT/voxrt-asr-android) (JitPack)
|
| 84 |
+
- iOS β [`voxrt-asr-ios`](https://github.com/VoxRT/voxrt-asr-ios) (Swift Package)
|
| 85 |
+
- Linux aarch64 β available on request (contact `help@voxrt.com`)
|
| 86 |
+
|
| 87 |
+
## Kotlin example
|
| 88 |
+
|
| 89 |
+
```kotlin
|
| 90 |
+
import com.voxrt.asr.VoxrtAsrNative
|
| 91 |
+
import com.voxrt.asr.VoxrtAsrStreamingEngine
|
| 92 |
+
|
| 93 |
+
val engine = VoxrtAsrStreamingEngine.fromAssetFd(modelFd)
|
| 94 |
+
// Or explicitly pick CTC:
|
| 95 |
+
// val engine = VoxrtAsrStreamingEngine.fromAssetFd(modelFd, VoxrtAsrNative.DECODE_CTC)
|
| 96 |
+
|
| 97 |
+
val delta = engine.processPcm(pcmFloatArray) // text emitted this call
|
| 98 |
+
val tail = engine.stop() // drain remaining text
|
| 99 |
+
engine.close()
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
`engine.processPcm` / `stop` / `reset` / `close` are
|
| 103 |
+
**synchronous and stateful** β the engine doesn't own a worker
|
| 104 |
+
thread. Drive it from your own capture / IO thread; marshal text
|
| 105 |
+
deltas back to UI via `runOnUiThread` / a Flow / your preferred
|
| 106 |
+
concurrency. RNN-T (default) survives chunk boundaries via its
|
| 107 |
+
LSTM state; CTC dedupes across chunks internally.
|
| 108 |
+
|
| 109 |
+
## Licensing
|
| 110 |
+
|
| 111 |
+
- **Model weights** are derived from
|
| 112 |
+
`nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc`,
|
| 113 |
+
Β© NVIDIA Corporation, CC-BY-4.0 licensed.
|
| 114 |
+
- **Repackaging** into the `.vxrt` container preserves the CC-BY-4.0
|
| 115 |
+
obligations attached to the weights. Full notice lives at
|
| 116 |
+
[github.com/VoxRT/voxrt-asr-models/blob/main/LICENSE](https://github.com/VoxRT/voxrt-asr-models/blob/main/LICENSE).
|
| 117 |
+
- **The VoxRT runtime and `.vxrt` container format** are proprietary
|
| 118 |
+
Elephant Enterprises LLC IP. Redistribution allowed as an
|
| 119 |
+
unmodified part of the VoxRT SDKs above.
|
| 120 |
+
|
| 121 |
+
Attribution required by CC-BY-4.0:
|
| 122 |
+
|
| 123 |
+
> Speech recognition powered by NVIDIA NeMo FastConformer
|
| 124 |
+
> (streaming, medium, 80 ms look-ahead, P&C),
|
| 125 |
+
> Β© NVIDIA Corporation, licensed under CC-BY-4.0.
|
| 126 |
+
|
| 127 |
+
Include this line in your product's UI, docs, or credits when you
|
| 128 |
+
ship a product that runs this model.
|
| 129 |
+
|
| 130 |
+
## About VoxRT
|
| 131 |
+
|
| 132 |
+
VoxRT is a from-scratch on-device inference runtime tuned for
|
| 133 |
+
streaming audio on commodity ARM CPUs β no GPU, no NPU, no vendor
|
| 134 |
+
accelerator required. Sister products on the same runtime:
|
| 135 |
+
|
| 136 |
+
- Wake-word: **["Hey Assistant" model + custom-phrase tier](https://huggingface.co/VoxRT/wake-word-hey-assistant-vxrt)**
|
| 137 |
+
- Voice activity detection: **[Silero VAD in `.vxrt`](https://huggingface.co/VoxRT/silero-vad-vxrt)**
|
| 138 |
+
|
| 139 |
+
Commercial integration / custom-model packaging: `help@voxrt.com`
|
| 140 |
+
Β· [voxrt.com](https://voxrt.com)
|