Reza2kn commited on
Commit
bb3648e
·
verified ·
1 Parent(s): 1576a56

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +89 -0
README.md ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - fa
4
+ license: cc-by-4.0
5
+ library_name: nemo
6
+ pipeline_tag: automatic-speech-recognition
7
+ tags:
8
+ - automatic-speech-recognition
9
+ - fastconformer
10
+ - persian
11
+ - fa
12
+ - streaming
13
+ - cache-aware
14
+ - on-device
15
+ - webgpu
16
+ - rnnt
17
+ base_model: nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms
18
+ metrics:
19
+ - wer
20
+ - cer
21
+ ---
22
+
23
+ # Shenava-Rizeh 0.9 — Persian Cache-Aware Streaming ASR (32M)
24
+
25
+ A tiny (**32M-param**) **cache-aware, multi-latency streaming** Persian (Farsi) ASR model — FastConformer-Hybrid (RNNT + CTC), 16 kHz. Built for **fully offline, on-device, real-time** captioning (WebGPU / WASM / NeMo), part of the [VisualEars](https://huggingface.co/Reza2kn) project (SLT 2026).
26
+
27
+ **One model serves the entire latency–accuracy curve** (0 / 80 / 480 / 1040 ms) — pick your operating point at inference time, no re-training. Its larger sibling is [`Shenava-Koochik-0.9`](https://huggingface.co/Reza2kn/Shenava-Koochik-0.9) (114M).
28
+
29
+ ## 📊 Results — Golden6669 (held-out gold Persian eval)
30
+
31
+ Evaluated on [`Reza2kn/visualears-golden-6669`](https://huggingface.co/datasets/Reza2kn/visualears-golden-6669) (6,669 clips, official Persian normalizer), RNNT head:
32
+
33
+ | `att_context_size` | **Latency** | **WER** | **CER** | WER_bf |
34
+ |---|---|---|---|---|
35
+ | `[70, 0]` | **0 ms** (real-time) | **11.08%** | **3.17%** | 9.68% |
36
+ | `[70, 1]` | 80 ms | 10.85% | 3.08% | 9.43% |
37
+ | `[70, 6]` | 480 ms | 10.56% | 2.93% | 9.14% |
38
+ | `[70, 13]` | 1040 ms | **10.46%** | **2.89%** | 9.03% |
39
+
40
+ **The curve is nearly flat — only 0.62 pp WER from 0 → 1040 ms.** You get near-best accuracy at **zero lookahead**: a 32M model doing **11.08% WER / 3.17% CER at true real-time**, fully on-device. For reference, the previous-generation single-latency `fa32M` scored 17.40% (and could not run below its trained latency).
41
+
42
+ ### Flatness holds per-condition
43
+
44
+ The latency penalty is uniform across acoustic conditions — low-latency does **not** fray on the hard far-field/obstructed tail (Golden6669 is deliberately ~94% non-clean):
45
+
46
+ | Condition | n | 0 ms WER | 1040 ms WER | Δ |
47
+ |---|---|---|---|---|
48
+ | clean | 400 | 13.38 | 12.62 | +0.76 |
49
+ | obstructed | 4,335 | 10.87 | 10.20 | +0.67 |
50
+ | far-field | 1,934 | 11.24 | 10.77 | **+0.47** |
51
+
52
+ Far-field has the *smallest* 0→1040 ms gap. (`clean` scores worst here — a quirk of Golden6669's small 400-clip clean slice, not a streaming effect.)
53
+
54
+ `WER_bf` = boundary-forgiven WER (utterances perfect modulo Persian word-spacing conventions counted correct).
55
+
56
+ ## How it was trained
57
+
58
+ - **Base:** `nvidia/stt_en_fastconformer_hybrid_medium_streaming_80ms` (English cache-aware streaming), persianized by swapping in a Persian **BPE-1024** tokenizer and reinitializing the decoder + joint (encoder kept).
59
+ - **Multi-latency:** `att_context_size = [[70,13],[70,6],[70,1],[70,0]]` (chunked_limited) — one checkpoint covers 0 / 80 / 480 / 1040 ms.
60
+ - **Phase A:** ~7,386 h / 3.66M clips of cleaned, teacher-pseudo-labeled Persian ([`Reza2kn/visualears-persian-pseudo-asr`](https://huggingface.co/datasets/Reza2kn/visualears-persian-pseudo-asr)).
61
+ - **Phase B:** gold-anchor fine-tune on 355 human-verified gold + active-learning corrections.
62
+ - Trajectory: random-init decoder **→ Phase A 17.94% → Phase B 10.46%** (@1040 ms).
63
+
64
+ ## Usage (NeMo)
65
+
66
+ ```python
67
+ from nemo.collections.asr.models import ASRModel
68
+ m = ASRModel.restore_from("shenava-rizeh-0.9.nemo").cuda().eval()
69
+ m.encoder.set_default_att_context_size([70, 0]) # 0 ms (real-time); or [70,13] for best WER
70
+ print(m.transcribe(["clip.wav"])[0].text)
71
+ ```
72
+
73
+ `[70,0]`=0 ms · `[70,1]`=80 ms · `[70,6]`=480 ms · `[70,13]`=1040 ms (1 encoder frame = 80 ms, FastConformer subsampling 8).
74
+
75
+ ## Notes
76
+ - **Version 0.9** — Phase B on 355 human-verified gold. **v1.0** follows a larger active-learning gold round (the 6K worst-disagreement clips now under review on Argilla).
77
+ - The CTC head is the deployment head for real-time/WebGPU; RNNT is the higher-accuracy offline/rescorer head.
78
+ - Larger streaming sibling: [`Shenava-Koochik-0.9`](https://huggingface.co/Reza2kn/Shenava-Koochik-0.9) (114M). Offline flagship: [`shenava-fa-fastconformer-115m`](https://huggingface.co/Reza2kn/shenava-fa-fastconformer-115m) (7.29%).
79
+
80
+ ## Citation
81
+ ```bibtex
82
+ @misc{shenava_rizeh_2026,
83
+ title = {Shenava-Rizeh: Persian Cache-Aware Streaming ASR (32M)},
84
+ author = {Sayar, Reza},
85
+ year = {2026},
86
+ howpublished = {Hugging Face},
87
+ url = {https://huggingface.co/Reza2kn/Shenava-Rizeh-0.9}
88
+ }
89
+ ```