NghiMe/NghiASR

Vietnamese Zipformer ASR

Model description

Model type: Automatic speech recognition (ASR) — streaming-capable transducer (Zipformer encoder + stateless decoder + joiner), exported as ONNX and quantized to INT8 for deployment.

Languages: Vietnamese (primary); handles Vietnamese–English code-switched speech.

Training framework: Icefall (k2 + Lhotse); pruned RNN-T loss; optional CTC; ScaledAdam + Eden scheduler; mixed precision (FP16/BF16).

Evaluation (results-4): WER reported on VietCasual test sets (combined A+B), normalized transcripts, jiwer-based WER.

---

Model variants and performance

Variant	Checkpoint	Overall WER (%)	Notes
exp7	epoch-4-avg-4.int8	8.22	Best WER in results-4
exp7	epoch-10-avg-10.int8	8.43
exp7	epoch-15-avg-15.int8	8.43
exp7	epoch-1-avg-1.int8	9.80	Early epoch
zipformer-6000h	epoch-20-avg-10.int8	21.93	~6k-hour pretrain; different data/domain
viet_iter	epoch-12-avg-8.int8	25.79	Different training setup

• Recommended for VietCasual: exp7 with epoch-4-avg-4.int8 (8.22% WER).
• Test set: VietCasual (vietcasual_test_A_1000 + vietcasual_test_B_1000), casual Vietnamese and code-switched English; transcripts normalized (e.g. soe_vinorm, punctuation cleaning).

---

Intended use

• Primary: Vietnamese ASR and Vietnamese–English code-switched speech (e.g. podcasts, interviews, casual dialogue).
• Deployment: On-device or server with sherpa-onnx (INT8 ONNX); supports streaming.
• Out-of-scope: Formal written text read-aloud only; other languages without adaptation; high-noise or heavily accented speech without further fine-tuning.

---

Training and evaluation data

• Training (representative): Vietnamese cuts (e.g. vi_cuts_train), optionally VietCasualSpeech, VietCetera, VietSuccess; LibriSpeech or mixed data for pretraining/fine-tuning.
• Evaluation (this card): VietCasual test A+B; WER computed with jiwer on normalized reference vs hypothesis.

---

Evaluation setup

• Metric: Word error rate (WER), case-sensitive on normalized text.
• Tooling: sherpa-onnx transducer decoder; run_wer_casual.py (or equivalent); combined wer_results.txt / wer_errors.txt per checkpoint.
• Outputs: Per-utterance dataset_tag:wav_name | reference | hypothesis | wer_pct; overall WER reported at top of wer_results.txt.

---

Limitations and bias

• Performance depends on domain and normalization; code-switched English segments can have higher error rates.
• INT8 quantization may slightly affect accuracy vs FP32.
• Not tested for robustness to strong accents, dialects, or very noisy environments in this card.

---

How to use (inference)

Example layout for sherpa-onnx (e.g. under exp7/epoch-4-avg-4.int8/):

• encoder-<name>.onnx
• decoder-<name>.onnx
• joiner-<name>.onnx
• tokens.txt

Use sherpa-onnx Transducer API with the above files and your 16 kHz (or model-native) audio. See project scripts (e.g. utils/WER/run_wer_casual.py) for batch WER.

---

Citation

If you use these models or results-4 metrics, please cite the training framework (e.g. Icefall/k2, Lhotse) and this card’s WER table and evaluation setup.

---

Summary

Field	Value
Best WER (results-4)	8.22% (exp7, epoch-4-avg-4.int8)
Test set	VietCasual A+B
Model form	ONNX INT8 transducer (Zipformer)
Languages	Vietnamese, Vietnamese–English code-switch