parakeet-es — Spanish-locked decoding for Parakeet TDT 0.6B v3
Problem. NVIDIA Parakeet TDT 0.6B v3 is multilingual
(25 European languages) and detects the language on its own — there is no language input. On short or
ambiguous Spanish audio it sometimes drifts to another language:
| Audio (Spanish) | Parakeet v3 | parakeet-es |
|---|---|---|
| "Dale." (es-MX voice) | Далее. (Russian) |
Dale. |
| "Ey, Paimón." | Эй, пойман! (Russian) |
Spanish output only |
| "Oye." | Oh yeah. (English) |
penalized: Spanish pieces preferred |
Fix (no retraining, no new weights). A thin layer over onnx-asr that changes how tokens are chosen in the TDT joint network:
- Alphabet lock — 2,696 of 8,193 vocabulary pieces (Cyrillic, Greek and Latin letters not used in
Spanish such as
ł š ø ç ß) can never be emitted. - Spanish language prior — pieces Parakeet never emits when transcribing real Spanish speech get a penalty (−4 logits by default), so ambiguous audio prefers Spanish pieces over English/Portuguese ones. The set of 2,336 "Spanish pieces" was measured on FLEURS es_419 test[150:900] and MLS Spanish test[150:1000] — held out from the evaluation below.
- Hotwords — names, brands and jargon you pass get a token-level prefix-trie boost (same idea as sherpa-onnx / NeMo word boosting).
- Regional profiles
es-MX/es-ES(experimental) — word lists of places, institutions and regional vocabulary for Mexico and Spain, applied as light hotwords.
Usage
pip install "onnx-asr[cpu,hub]" numpy
# copy the parakeet_es/ folder next to your code (or pip install .)
import parakeet_es
asr = parakeet_es.load("es", hotwords=["Paimón", "OpenClaw"]) # "es" | "es-MX" | "es-ES"
print(asr.recognize(waveform_16k_float32))
Options: prior_penalty (default 4.0), hotword_boost (5.0), lexicon_boost (3.0), lock (True),
quantization ("int8"). Runs on CPU; on a Ryzen 7 5700X3D a 12 s clip takes ~0.45 s (int8).
Results (reproducible: python eval/evaluar.py --datos <data>)
WER = word error rate (lowercase, punctuation removed). Drift = % of outputs containing non-Spanish letters.
| Config | FLEURS es_419 (150, human) | MLS Spanish (150, human) | Short phrases WER (180) | All phrases WER (270) | Drift |
|---|---|---|---|---|---|
| Parakeet v3 original | 3.24 % | 4.70 % | 24.44 % | 13.45 % | 0.37 % |
parakeet-es es |
3.22 % | 4.67 % | 23.81 % | 13.55 % | 0 % |
parakeet-es es-MX |
3.27 % | 4.65 % | 23.49 % | 13.35 % | 0 % |
parakeet-es es-ES |
3.22 % | 4.70 % | 23.17 % | 12.96 % | 0 % |
The phrase set = 30 phrases (20 very short ones like "Oye.", "Dale.", "¿Qué onda?") × 9 voices
(Kokoro ×3, Piper es_MX ×2, Edge es-MX ×2, Edge es-ES ×2). With hotwords ["Paimón", "OpenClaw"]
these names went from 22 % to 44 % correct.
Honest takeaways.
- On normal human Spanish, accuracy is unchanged (differences ≤ 0.05 pt) — the lock costs nothing.
- Language drift goes to 0 %. The original drifts rarely, but when it does the output is unusable.
- Very short single words (< 0.8 s) remain hard for every model tested (Whisper small "es" scores the same).
- The regional profiles did not clearly improve regional-word accuracy on this synthetic set (es-MX words: 84.6 % → 82.1 %, a 3-word difference). They are experimental; contributions with real es-MX / es-ES recordings are welcome.
Limitations / next steps
- This is decoding-time specialization. A true Spanish fine-tune (e.g. NeMo on Common Voice es, CIEMPIESS for es-MX) needs an NVIDIA GPU and is the natural next step.
- English words said inside Spanish speech are penalized (not blocked); add them as hotwords if needed.
Credits & licenses
Code and word lists: Apache-2.0. Base model: NVIDIA Parakeet TDT 0.6B v3, CC-BY-4.0. ONNX export: istupakov/parakeet-tdt-0.6b-v3-onnx. Evaluation data: FLEURS (CC-BY-4.0), Multilingual LibriSpeech (CC-BY-4.0). This repo does not redistribute model weights.
En español
El problema: Parakeet v3 entiende 25 idiomas y adivina el idioma solo. Con frases cortas en español a
veces escribe en ruso ("Dale." → Далее.) o en inglés ("Oye." → Oh yeah.).
La solución: parakeet-es fija el decodificador al español, sin reentrenar:
- bloquea los fragmentos de otros alfabetos;
- penaliza los fragmentos que Parakeet nunca usa para escribir español;
- refuerza tus palabras propias (nombres, marcas);
- trae perfiles es-MX y es-ES (experimentales).
Resultado: la deriva a otro idioma baja a 0 % y la precisión en español real queda igual (FLEURS 3.22 %, MLS 4.67 %). Ver la tabla de arriba.
Model tree for solucionesft/parakeet-es
Base model
nvidia/parakeet-tdt-0.6b-v3