parakeet-ctc-110m-yiddish

nvidia/parakeet-tdt_ctc-110m (110M, FastConformer hybrid TDT-CTC) fine-tuned for Yiddish speech recognition, with a new 1024-token Yiddish+English SentencePiece BPE vocabulary.

Trained on ~146h: ivrit-ai crowd corpora (recital + whatsapp), the YakovW curated set family, ~10k single-speaker TTS clips, LibriSpeech English retention, and noise/no-speech clips — the same data as whisper-small-yiddish v3.

Use the CTC decoder branch. The TDT branch did not converge in this run and its outputs are unusable — decode with decoder_type="ctc" (see below).

Results

Dev WER (547 held-out real-Yiddish clips; normalization: punctuation stripped, niqqud folded, ligatures expanded — same metric as the whisper-small-yiddish evals; greedy decode):

model	dev WER (yi)
whisper-small-yiddish v3 (244M)	36.5%
this model, CTC (110M)	25.6%
this model, ONNX fp32	25.5%
this model, ONNX int8	26.5%

Per-source (n / whisper v3 / this model):

source	n	whisper v3	this model
ivrit_recital	200	46.4%	31.1%
ivrit_whatsapp	100	37.3%	23.6%
km	21	19.1%	18.2%
mozilla	58	15.3%	16.6%
multispk	56	21.2%	21.9%
seg1	29	22.4%	20.2%
seghq	83	13.8%	14.6%

On 30 noise-only clips this model emits no text at all (whisper hallucinated on ~100% of them). Tail-hallucination (3s appended silence): 0.14 extra words/clip, same as whisper v3.

Files

• parakeet-ctc-110m-yiddish.nemo — full NeMo checkpoint (both branches)
• model.onnx / model.int8.onnx — CTC branch (encoder + CTC head) for CPU inference, input: log-mel features (B, 80, T) + int64 lengths, output: token logprobs (blank id = 1024, 8x subsampling)
• tokenizer.model — SentencePiece BPE, 1024 tokens

Usage (NeMo)

import nemo.collections.asr as nemo_asr
m = nemo_asr.models.ASRModel.restore_from("parakeet-ctc-110m-yiddish.nemo")
m.change_decoding_strategy(decoder_type="ctc")  # important — do not use the TDT branch
print(m.transcribe(["audio_16k_mono.wav"])[0].text)

Usage (ONNX, CPU)

Works with plain onnxruntime (NeMo preprocessor or equivalent 80-dim log-mel front end, greedy CTC collapse, sentencepiece decode) and is compatible with sherpa-onnx-style NeMo CTC runtimes.

Text output is normalized: no punctuation, no niqqud, ligatures expanded (וו / וי / יי).

Language model (KenLM shallow fusion)

language_model/ contains a word-level KenLM for decode-time shallow fusion — no retraining of the acoustic model needed:

• yi_en_5gram.q8.trie.bin — 5-gram, 8-bit quantized trie, 38.7MB. 390k vocab (~75% Yiddish / 25% English tokens), trained on 33.9M words of Hasidic Yiddish forum/news/print text + English (for the bilingual use case). Pruning 0 1 2 2 2.
• unigrams.txt — vocabulary list for pyctcdecode (keep next to the .bin).
• yi_en_5gram.arpa.gz — ARPA source, if you want to re-binarize with other flags.

Normalization contract: the LM expects this model's output convention — no punctuation, no niqqud, ligatures expanded (וו/וי/יי), charset [א-ת a-z 0-9]. Text going into WER references must pass the same normalizer (see examples/eval_wer_sweep.py :: norm() — note it uses NFKD, which also folds Hebrew presentation forms U+FB1D–FB4F).

Usage

# pip install onnxruntime sentencepiece soundfile librosa pyctcdecode kenlm
python examples/fusion_decode.py --model model.onnx --tokenizer tokenizer.model \
    --lm language_model/yi_en_5gram.q8.trie.bin --alpha 0.6 --beta 0.5 audio.wav

Wiring: CTC log-probs (1024 BPE + blank id 1024) → pyctcdecode beam search → words scored by the 5-gram. Long audio: add --chunk-sec 30.

Tune alpha/beta on your own dev set (examples/eval_wer_sweep.py does the sweep with an honest dev/test split). Starting point: alpha 0.5–0.7, beta 0.5–1.5.

Code-switch control (--lang-bias)

--lang-bias B softly steers decoding toward the locally dominant script. The bias is local, not per-file: the frame-aligned greedy pass measures the Hebrew/Latin ratio in a sliding window (--bias-window, default 10 s) around each frame, and where one script has ≥70% of the letters, the other script's BPE pieces are penalized by B (log space) at those frames before the final decode (beam search, or a second greedy pass when no LM is given). A 20-minute dictation that alternates languages therefore gets the right bias in each region. Windows with a 30–70% mix are untouched, and the penalty is soft — a clearly-articulated minority-language word still wins on acoustics. --bias-window 0 falls back to judging each utterance/chunk as a whole.

Default 0 (off). Try B 0.5–2.0, tuned on a dev set that contains English and code-switched clips — the 547-clip Yiddish dev set cannot measure this failure mode. Overhead is negligible (~ms even for 20-minute audio).