POWSM encoder + CTC β ONNX int8
An adapted copy of espnet/powsm by Chin-Jou Li, Kalvin Chang, Shikhar Bharadwaj, Eunjung Yeo, Kwanghee Choi, Jian Zhu, David R. Mortensen and Shinji Watanabe (POWSM: A Phonetic Open Whisper-Style Speech Foundation Model). Licensed, like the original, under CC BY 4.0.
Changes made: only the encoder and the CTC head are exported (the attention
decoder is dropped, so no language/task prompt is needed); the front end
(STFT β 80 log-mel β global MVN) is left out of the graph and described in
frontend.json; weights dynamically quantised to int8 with onnxruntime. Export script:
speech-to-ipa/eval/export_powsm_onnx.py.
Files
model.int8.onnxβ inputsfeatsfloat32[1, T, 80](normalised log-mel),feats_lensint64[1]; outputlogits[1, frames, 40002].frontend.jsonβ window, mel filters and normalisation statistics.vocab.jsonβ{"blank": 0, "vocab": {phone: id}}, phone tokens only (6,405 of 40,002). Greedy CTC: argmax per frame, merge repeats, keep phone ids.
Accuracy
On 760 VoxAngeles utterances (8 from each of 95 languages), narrow scoring, no
language hint: CTC-only PER 53.2 / PFER 14.7 (this int8 copy 53.9 / 15.1), vs
56.2 / 17.6 for the full model's attention decoder with <unk> language.
Model tree for ThePowerOfHebrew/powsm-ctc-onnx
Base model
espnet/powsm