eval: add VisualEars golden test set results (3 datasets)
Browse files
README.md
CHANGED
|
@@ -61,9 +61,10 @@ Measured by Rade with greedy CTC decoding (fp16) on two standard Persian test se
|
|
| 61 |
| Test set | Clips | WER | CER |
|
| 62 |
|---|---|---|---|
|
| 63 |
| **FLEURS** `fa_ir` β read speech | 871 | 19.6 % | **4.4 %** |
|
|
|
|
| 64 |
| **Common Voice 17.0** `fa` β crowd-sourced | 10,355 | 21.8 % | 17.8 % |
|
| 65 |
|
| 66 |
-
> **CER is the more faithful metric for Persian.** Persian **WER** is inflated by orthographic/spacing variation (ΩΫΩ
βΩΨ§Ψ΅ΩΩ/ZWNJ, affix spacing, compound spelling) that doesn't reflect actual mis-recognition β note FLEURS sits at **19.6 % WER but only 4.4 % CER**, i.e. most "word errors" are one-character spelling differences. On clean
|
| 67 |
|
| 68 |
## Speed & hardware
|
| 69 |
|
|
|
|
| 61 |
| Test set | Clips | WER | CER |
|
| 62 |
|---|---|---|---|
|
| 63 |
| **FLEURS** `fa_ir` β read speech | 871 | 19.6 % | **4.4 %** |
|
| 64 |
+
| **VisualEars golden** `fa` β curated (clean/farfield/obstructed) | 6,669 | 22.9 % | **4.3 %** |
|
| 65 |
| **Common Voice 17.0** `fa` β crowd-sourced | 10,355 | 21.8 % | 17.8 % |
|
| 66 |
|
| 67 |
+
> **CER is the more faithful metric for Persian.** Persian **WER** is inflated by orthographic/spacing variation (ΩΫΩ
βΩΨ§Ψ΅ΩΩ/ZWNJ, affix spacing, compound spelling) that doesn't reflect actual mis-recognition β note FLEURS sits at **19.6 % WER but only 4.4 % CER**, i.e. most "word errors" are one-character spelling differences. On clean, well-curated speech (FLEURS, VisualEars) the model reaches **CER β 4 %**, and it stays robust across recording conditions (VisualEars far-field 4.5 % / obstructed 4.2 % CER). On noisier crowd-sourced audio (Common Voice β spontaneous speech, varied mics/accents, loan words) CER rises to **β 18 %**.
|
| 68 |
|
| 69 |
## Speed & hardware
|
| 70 |
|