maratos commited on
Commit
37fae26
Β·
verified Β·
1 Parent(s): fa8aaa4

Add initial model card

Browse files
Files changed (1) hide show
  1. README.md +140 -0
README.md ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ language:
4
+ - en
5
+ library_name: voxrt
6
+ pipeline_tag: automatic-speech-recognition
7
+ tags:
8
+ - speech-recognition
9
+ - streaming
10
+ - asr
11
+ - fastconformer
12
+ - nemo
13
+ - nvidia
14
+ - on-device
15
+ - edge
16
+ - arm
17
+ - neon
18
+ - voxrt
19
+ base_model:
20
+ - nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc
21
+ ---
22
+
23
+ # NeMo FastConformer streaming-medium-pc on the VoxRT runtime
24
+
25
+ The `stt_en_fastconformer_hybrid_medium_streaming_80ms_pc` model
26
+ from NVIDIA NeMo, packaged as a **`.vxrt`** file for the VoxRT
27
+ on-device inference runtime. Same weights, repackaged so the
28
+ 80 ms cache-aware streaming ASR runs on Android or iOS in
29
+ real-time β€” with punctuation and capitalisation output straight
30
+ from the model, no post-processing layer required.
31
+
32
+ **This is not our model.** The weights are NVIDIA's
33
+ `nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc`
34
+ checkpoint, released under CC-BY-4.0. What we ship is the runtime
35
+ that makes it fast on cheap ARM hardware plus the `.vxrt`
36
+ container that the runtime consumes.
37
+
38
+ ## Runtime performance
39
+
40
+ Cumulative RTF on a single CPU core, `arm64` release builds,
41
+ post-warmup. Live-mic figures are the production-realistic ones
42
+ (scheduler jitter + capture overhead included):
43
+
44
+ | Device | CPU | Decoder | Mode | RTF |
45
+ |----------------------------|------------|---------|-------------|-----------:|
46
+ | Xiaomi Redmi 9C (Android) | Cortex-A73 | RNN-T | file replay | **0.302** |
47
+ | Xiaomi Redmi 9C (Android) | Cortex-A73 | RNN-T | live mic | **0.353** |
48
+ | iPhone 13 Pro Max (iOS) | Apple A15 | RNN-T | live mic | **0.08–0.10** |
49
+
50
+ For the same weights, RNN-T decoding costs ~50 ms of CPU per
51
+ 1.12 s chunk on SD662; the CTC head is ~5 ms per chunk with a
52
+ minor WER hit. The SDK exposes both decoders β€” pick per your
53
+ battery / accuracy trade-off.
54
+
55
+ Chunked streaming granularity is **80 ms** cache-aware
56
+ look-ahead. Inherent end-to-end buffering is one chunk
57
+ (β‰ˆ 1.12 s at chunk_size=112) before text emission begins.
58
+
59
+ ## Model quality
60
+
61
+ Empirically validated on LibriSpeech test-clean (500-utterance
62
+ subset, matches the SDK repos' reported numbers):
63
+
64
+ | Decoder | WER | Notes |
65
+ |-----------------|-------:|--------------------------------------------------|
66
+ | **RNN-T** β˜… | **3.267 %** | Recommended default. Higher accuracy. |
67
+ | CTC | 4.895 % | ~15 % cheaper per chunk; long-session friendly. |
68
+
69
+ Model architecture, training data, and topline WER claims are
70
+ NVIDIA's β€” see the upstream checkpoint at
71
+ [huggingface.co/nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc](https://huggingface.co/nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc).
72
+
73
+ ## Download & use
74
+
75
+ The `.vxrt` file on this HF repo is byte-identical to the one at
76
+ [github.com/VoxRT/voxrt-asr-models/releases](https://github.com/VoxRT/voxrt-asr-models/releases).
77
+ Either source is fine.
78
+
79
+ `.vxrt` files cannot be loaded with `transformers`, `nemo_toolkit`,
80
+ or any standard HF library β€” they are a proprietary container
81
+ the VoxRT runtime reads. Use one of our SDKs:
82
+
83
+ - Android β€” [`voxrt-asr-android`](https://github.com/VoxRT/voxrt-asr-android) (JitPack)
84
+ - iOS β€” [`voxrt-asr-ios`](https://github.com/VoxRT/voxrt-asr-ios) (Swift Package)
85
+ - Linux aarch64 β€” available on request (contact `help@voxrt.com`)
86
+
87
+ ## Kotlin example
88
+
89
+ ```kotlin
90
+ import com.voxrt.asr.VoxrtAsrNative
91
+ import com.voxrt.asr.VoxrtAsrStreamingEngine
92
+
93
+ val engine = VoxrtAsrStreamingEngine.fromAssetFd(modelFd)
94
+ // Or explicitly pick CTC:
95
+ // val engine = VoxrtAsrStreamingEngine.fromAssetFd(modelFd, VoxrtAsrNative.DECODE_CTC)
96
+
97
+ val delta = engine.processPcm(pcmFloatArray) // text emitted this call
98
+ val tail = engine.stop() // drain remaining text
99
+ engine.close()
100
+ ```
101
+
102
+ `engine.processPcm` / `stop` / `reset` / `close` are
103
+ **synchronous and stateful** β€” the engine doesn't own a worker
104
+ thread. Drive it from your own capture / IO thread; marshal text
105
+ deltas back to UI via `runOnUiThread` / a Flow / your preferred
106
+ concurrency. RNN-T (default) survives chunk boundaries via its
107
+ LSTM state; CTC dedupes across chunks internally.
108
+
109
+ ## Licensing
110
+
111
+ - **Model weights** are derived from
112
+ `nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms_pc`,
113
+ Β© NVIDIA Corporation, CC-BY-4.0 licensed.
114
+ - **Repackaging** into the `.vxrt` container preserves the CC-BY-4.0
115
+ obligations attached to the weights. Full notice lives at
116
+ [github.com/VoxRT/voxrt-asr-models/blob/main/LICENSE](https://github.com/VoxRT/voxrt-asr-models/blob/main/LICENSE).
117
+ - **The VoxRT runtime and `.vxrt` container format** are proprietary
118
+ Elephant Enterprises LLC IP. Redistribution allowed as an
119
+ unmodified part of the VoxRT SDKs above.
120
+
121
+ Attribution required by CC-BY-4.0:
122
+
123
+ > Speech recognition powered by NVIDIA NeMo FastConformer
124
+ > (streaming, medium, 80 ms look-ahead, P&C),
125
+ > Β© NVIDIA Corporation, licensed under CC-BY-4.0.
126
+
127
+ Include this line in your product's UI, docs, or credits when you
128
+ ship a product that runs this model.
129
+
130
+ ## About VoxRT
131
+
132
+ VoxRT is a from-scratch on-device inference runtime tuned for
133
+ streaming audio on commodity ARM CPUs β€” no GPU, no NPU, no vendor
134
+ accelerator required. Sister products on the same runtime:
135
+
136
+ - Wake-word: **["Hey Assistant" model + custom-phrase tier](https://huggingface.co/VoxRT/wake-word-hey-assistant-vxrt)**
137
+ - Voice activity detection: **[Silero VAD in `.vxrt`](https://huggingface.co/VoxRT/silero-vad-vxrt)**
138
+
139
+ Commercial integration / custom-model packaging: `help@voxrt.com`
140
+ Β· [voxrt.com](https://voxrt.com)