eval: add Common Voice 17 results + consistent normalization, lead with CER

README.md

@@ -36,7 +36,7 @@ metrics:
   <a href="https://huggingface.co/RadeAI/Rade-ASR-CTC-3B-fa/colab"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"></a>
 </div>
 
-> **TL;DR** — A fast, non-autoregressive (CTC) speech-to-text model specialized for **Persian**, built on top of Meta's 3-billion-parameter Omnilingual ASR encoder. It transcribes Persian audio clips (≤ 40 s) and runs **~199× faster than real time** in fp16 on a single RTX 4090. On FLEURS Persian it reaches **WER ≈ 11% / CER ≈ 3.4%** (normalized).
+> **TL;DR** — A fast, non-autoregressive (CTC) speech-to-text model specialized for **Persian**, built on top of Meta's 3-billion-parameter Omnilingual ASR encoder. It transcribes Persian audio clips (≤ 40 s) and runs **~199× faster than real time** in fp16 on a single RTX 4090. On Persian test sets it reaches **CER ≈ 4 %** on FLEURS (clean read speech) and **≈ 18 %** on Common Voice (noisier, crowd-sourced) — normalized.
 
 ---
 
@@ -56,14 +56,14 @@ This checkpoint takes the strong multilingual representations of Omnilingual ASR
 
 ## Evaluation
 
-Measured by Rade on the **FLEURS `fa_ir` test set** (871 read-speech clips, ~222 min), greedy CTC decoding, fp16:
+Measured by Rade with greedy CTC decoding (fp16) on two standard Persian test sets. Both reference and hypothesis are normalized before scoring — unify `ك→ک` / `ي→ی`, convert ZWNJ (نیم‌فاصله) to space, strip punctuation and diacritics, collapse whitespace — so that orthography-only differences don't count as errors.
 
-| Metric | With Persian normalization (hazm) | Raw |
-|---|---|---|
-| **WER** | **10.8 %** | 19.6 % |
-| **CER** | **3.4 %** | 4.5 % |
+| Test set | Clips | WER | CER |
+|---|---|---|---|
+| **FLEURS** `fa_ir` — read speech | 871 | 19.6 % | **4.4 %** |
+| **Common Voice 17.0** `fa` — crowd-sourced | 10,355 | 21.8 % | 17.8 % |
 
-> The **normalized** column is the meaningful one: it unifies Persian orthographic conventions (نیم‌فاصله / ZWNJ, `ه‌ی` vs `ه`, Arabic vs Persian `ك/ي`) on both reference and hypothesis before scoring. Most of the raw WER gap is orthography, not misrecognition. **WER ≈ 11% / CER ≈ 3.4%** is the model's real accuracy on Persian read speech.
+> **CER is the more faithful metric for Persian.** Persian **WER** is inflated by orthographic/spacing variation (نیم‌فاصله/ZWNJ, affix spacing, compound spelling) that doesn't reflect actual mis-recognition — note FLEURS sits at **19.6 % WER but only 4.4 % CER**, i.e. most "word errors" are one-character spelling differences. On clean read speech (FLEURS) the model reaches **CER ≈ 4 %**; on noisier crowd-sourced audio (Common Voice — spontaneous speech, varied mics/accents, loan words) **CER ≈ 18 %**.
 
 ## Speed & hardware
 
@@ -183,7 +183,7 @@ Built and maintained by **Rade AI**. For questions, collaboration, or custom Per
 - گفتار فارسی را به متن تبدیل می‌کند (کلیپ‌های کوتاه‌تر از ۴۰ ثانیه).
 - معماری **CTC** (غیر-اتورگرسیو) دارد، برای همین خیلی سریع است — در fp16 حدود **۱۹۹ برابر سریع‌تر از زمان واقعی** روی یک RTX 4090.
 - در fp16 فقط **۶.۴ گیگابایت VRAM** می‌خواهد (یک GPU ۱۶ گیگ کافی است).
-- روی دیتاستِ FLEURS فارسی (با نرمال‌سازیِ hazm): **WER ۱۰.۸٪** و **CER ۳.۴٪**.
+- دقت (با نرمال‌سازیِ متن): روی **FLEURS** فارسی (گفتارِ تمیز) **CER حدود ۴٪** (WER ۱۹.۶٪)، و روی **Common Voice 17** فارسی (داده‌ی محاوره‌ایِ نویزی، ۱۰٬۳۵۵ کلیپ) **CER حدود ۱۸٪** (WER ۲۱.۸٪). در فارسی CER معیارِ معتبرتریه چون WER با اختلافِ املایی/نیم‌فاصله متورم می‌شه.
 
 نحوه‌ی استفاده در بخش انگلیسیِ بالا آمده. برای تستِ سریع، دکمه‌ی **Open in Colab** (بالای همین صفحه، منوی «Use this model») یا نوت‌بوکِ `notebook.ipynb` رو باز کن.