# Vietnamese Zipformer ASR

## Model description

**Model type:** Automatic speech recognition (ASR) — streaming-capable transducer (Zipformer encoder + stateless decoder + joiner), exported as ONNX and quantized to INT8 for deployment.

**Languages:** Vietnamese (primary); handles Vietnamese–English code-switched speech.

**Training framework:** Icefall (k2 + Lhotse); pruned RNN-T loss; optional CTC; ScaledAdam + Eden scheduler; mixed precision (FP16/BF16).

**Evaluation (results-4):** WER reported on VietCasual test sets (combined A+B), normalized transcripts, jiwer-based WER.

---

## Model variants and performance

| Variant | Checkpoint | Overall WER (%) | Notes |
|--------|------------|------------------|-------|
| **exp7** | epoch-4-avg-4.int8 | **8.22** | Best WER in results-4 |
| **exp7** | epoch-10-avg-10.int8 | 8.43 | |
| **exp7** | epoch-15-avg-15.int8 | 8.43 | |
| **exp7** | epoch-1-avg-1.int8 | 9.80 | Early epoch |
| **zipformer-6000h** | epoch-20-avg-10.int8 | 21.93 | ~6k-hour pretrain; different data/domain |
| **viet_iter** | epoch-12-avg-8.int8 | 25.79 | Different training setup |

- **Recommended for VietCasual:** **exp7** with **epoch-4-avg-4.int8** (8.22% WER).
- Test set: VietCasual (vietcasual_test_A_1000 + vietcasual_test_B_1000), casual Vietnamese and code-switched English; transcripts normalized (e.g. soe_vinorm, punctuation cleaning).

---

## Intended use

- **Primary:** Vietnamese ASR and Vietnamese–English code-switched speech (e.g. podcasts, interviews, casual dialogue).
- **Deployment:** On-device or server with sherpa-onnx (INT8 ONNX); supports streaming.
- **Out-of-scope:** Formal written text read-aloud only; other languages without adaptation; high-noise or heavily accented speech without further fine-tuning.

---

## Training and evaluation data

- **Training (representative):** Vietnamese cuts (e.g. vi_cuts_train), optionally VietCasualSpeech, VietCetera, VietSuccess; LibriSpeech or mixed data for pretraining/fine-tuning.
- **Evaluation (this card):** VietCasual test A+B; WER computed with jiwer on normalized reference vs hypothesis.

---

## Evaluation setup

- **Metric:** Word error rate (WER), case-sensitive on normalized text.
- **Tooling:** sherpa-onnx transducer decoder; `run_wer_casual.py` (or equivalent); combined `wer_results.txt` / `wer_errors.txt` per checkpoint.
- **Outputs:** Per-utterance `dataset_tag:wav_name | reference | hypothesis | wer_pct`; overall WER reported at top of `wer_results.txt`.

---

## Limitations and bias

- Performance depends on domain and normalization; code-switched English segments can have higher error rates.
- INT8 quantization may slightly affect accuracy vs FP32.
- Not tested for robustness to strong accents, dialects, or very noisy environments in this card.

---

## How to use (inference)

Example layout for sherpa-onnx (e.g. under `exp7/epoch-4-avg-4.int8/`):

- `encoder-<name>.onnx`
- `decoder-<name>.onnx`
- `joiner-<name>.onnx`
- `tokens.txt`

Use sherpa-onnx Transducer API with the above files and your 16 kHz (or model-native) audio. See project scripts (e.g. `utils/WER/run_wer_casual.py`) for batch WER.

---

## Citation

If you use these models or results-4 metrics, please cite the training framework (e.g. Icefall/k2, Lhotse) and this card’s WER table and evaluation setup.

---

## Summary

| Field | Value |
|-------|--------|
| Best WER (results-4) | **8.22%** (exp7, epoch-4-avg-4.int8) |
| Test set | VietCasual A+B |
| Model form | ONNX INT8 transducer (Zipformer) |
| Languages | Vietnamese, Vietnamese–English code-switch |