OpenVoiceOS/whisper-small-cantonese-onnx

whisper-small-cantonese — ONNX

ONNX export of alvanlii/whisper-small-cantonese (Whisper small fine-tuned for Cantonese/Yue Chinese by alvanlii) for onnx-asr (standard whisper model type — works with stock onnx-asr, no patches needed). fp32 and int8 variants included.

License: apache-2.0, inherited from the source model.

First specialized ONNX ASR model for Yue Chinese (Cantonese) in this collection.

Usage

Whisper has no dedicated yue language token; use language="zh" (the base Whisper tokenizer's Chinese token) — the Cantonese behavior comes from the fine-tune itself.

import onnx_asr
model = onnx_asr.load_model("whisper", "path/to/this/repo")  # or quantization="int8"
print(model.recognize("audio_16khz.wav", language="zh"))

Verified on a FLEURS Cantonese (yue_hant_hk) test clip:

• Reference: 在短短兩週內美軍和自由法國軍就解放了南法並轉向德國
• fp32: 在短短兩週內美軍和自由法國軍就解放了南法並轉向德國 (exact match, RTF 0.24)
• int8: 在短端兩州內美軍和自由法國軍就解放了南法並轉向德國 (RTF 0.11)

fp32 is an exact transcript match. int8 shows minor character-substitution errors ("短端"/"兩州" vs "短短"/"兩週") — this small model appears more sensitive to dynamic quantization than the medium/large exports in this collection; still largely intelligible, but prefer fp32 where accuracy matters. RTF measured on an AMD Ryzen 5 7600 (CPU, 4 OMP threads, shared/loaded box — not a clean benchmark number).

Int8 decoder was produced by quantizing the pre-merge decoders separately and re-merging (merge_decoders(..., strict=False)); direct quantization of the merged decoder graph does not shrink it (its If subgraphs are skipped by onnxruntime's dynamic quantizer).