parakeet-indic-v1

Speech recognition for 9 languages: Hindi, Marathi, Bengali, Assamese, Tamil, Telugu, Kannada, Malayalam and Indian English, with automatic language identification. One model, no language setting needed.

FastConformer encoder + TDT decoder, 0.6B parameters. Built from parakeet-hi-v3 (Hindi) with a new multilingual vocabulary; base: nvidia/parakeet-tdt-0.6b-v3.

Status: stage-2 checkpoint, step 141,000 (best mean WER; ~45K h of audio seen over both stages). Internal use only.

How language works

Every transcript starts with a language tag (<hi>, <mr>, <bn>, <as>, <ta>, <te>, <kn>, <ml>, <en>) that the model predicts itself from the audio. Long audio is cut into pieces of at most 20 s and every piece gets its own tag, so a recording that switches language is followed piece by piece. transcribe.py removes the tags from the text and reports the language of each piece.

With NeMo, the language can also be forced by priming the decoder with the tag (see below). On Hindi, forcing <hi> lowers WER by up to ~2 points on the hardest sets (Lahaja 19.1 → 16.7, noisy-phone 23.9 → 22.1).

Results

All scores on held-out clips that were never trained on, WER / CER (%), punctuation ignored. Language is not given to the model (automatic language ID).

Per language

Mean over each language's test sets; language ID = share of clips whose predicted tag is the correct language.

language	sets	clips	WER	CER	language ID
Hindi (hi)	5	8,417	16.1	6.4	96.2%
Marathi (mr)	6	2,962	31.8	11.7	95.2%
Bengali (bn)	7	3,334	29.4	11.0	97.4%
Assamese (as)	5	2,500	33.4	12.4	92.5%
Tamil (ta)	7	3,500	41.1	13.5	96.0%
Telugu (te)	5	2,328	34.6	10.7	95.3%
Kannada (kn)	7	3,310	39.7	13.4	96.7%
Malayalam (ml)	6	3,000	44.3	13.9	97.2%
English (Indian) (en)	2	1,000	11.3	5.8	98.9%

Tamil, Malayalam, Kannada and Telugu words are long (many morphemes per word), so one wrong letter costs a whole word: compare these languages by CER rather than WER.

Per test set

language	FLEURS test	IndicVoices valid	Vaani test	SPRING-INX test	Rasa test	NPTEL (held-out lectures)	SUBAK.KO test	IISc-MILE test
mr	27.3 / 8.4	34.0 / 11.8	40.4 / 14.0	49.0 / 25.1	16.9 / 3.6	23.2 / 7.2	–	–
bn	24.5 / 7.3	28.6 / 10.1	27.2 / 8.8	37.2 / 19.1	24.0 / 6.2	32.8 / 15.4	31.3 / 10.3	–
as	33.6 / 10.6	30.2 / 11.8	25.7 / 7.6	51.1 / 24.5	26.2 / 7.5	–	–	–
ta	37.7 / 12.2	48.9 / 16.9	47.2 / 15.8	55.6 / 25.4	36.2 / 10.0	26.7 / 7.2	–	35.3 / 7.2
te	32.7 / 10.2	41.2 / 14.6	37.0 / 11.9	–	32.9 / 8.8	29.1 / 7.9	–	–
kn	24.8 / 6.9	57.5 / 17.4	45.5 / 16.4	69.8 / 34.6	28.0 / 6.0	25.4 / 7.8	–	27.0 / 4.8
ml	29.7 / 7.5	61.1 / 20.6	43.5 / 12.2	62.7 / 26.6	36.4 / 6.7	32.2 / 10.1	–	–
en	–	–	13.4 / 6.6	–	–	9.2 / 5.1	–	–

IndicVoices, Vaani and SPRING-INX are spontaneous, conversational and often noisy; FLEURS, Rasa, IISc-MILE and NPTEL are read or lecture speech.

Hindi: compared with parakeet-hi-v3

Same clips, same scoring, automatic language ID.

set	clips	parakeet-hi-v3	parakeet-indic-v1
FLEURS test	417	12.37 / 4.24	13.39 / 4.86
SVQ	2,000	13.30 / 4.61	13.85 / 5.03
Vaani test	2,000	14.40 / 5.89	15.08 / 6.40
Lahaja	2,000	15.28 / 4.98	18.33 / 7.13
IndicVoices valid	2,000	18.12 / 7.37	19.64 / 8.38
low-volume (speech 25–35 dB quieter)	500	15.97 / 6.20	17.27 / 7.00
numbers (written as digits)	710	16.31 / 6.49	18.89 / 7.53
noisy-phone (8 kHz phone + noise + cross-talk)	800	23.45 / 11.40	24.45 / 12.29
no speech (noise / near-silence): clips where text is produced	150	0%	0%

Hindi is 0.5–3 WER points behind the Hindi-only v3 in automatic mode. For Hindi-only traffic, v3 is still the better choice; use this model where other languages occur.

Files

file	size	use
parakeet-indic-v1.nemo	2.4 GB	NeMo checkpoint: GPU inference, forced language, further fine-tuning
onnx/encoder.onnx + onnx/encoder.weights	2.4 GB	sherpa-onnx, fp32
onnx/encoder.int8.onnx	0.85 GB	sherpa-onnx, int8 (recommended on CPU)
onnx/decoder*.onnx, onnx/joiner*.onnx	small	TDT prediction network and joint, fp32 and int8
onnx/tokens.txt		12,288 SentencePiece tokens (incl. 9 language tags) + blank
transcribe.py		CPU transcription (sherpa-onnx; no PyTorch or NeMo needed)

Quick start: CPU with sherpa-onnx

pip install sherpa-onnx soundfile numpy scipy huggingface_hub
hf auth login                    # the repo is private
# int8 only (~0.9 GB download). Repeat --include for every pattern:
hf download amn-raw/parakeet-indic-v1 --local-dir parakeet-indic-v1 \
    --include "onnx/*.int8.onnx" --include "onnx/tokens.txt" --include "transcribe.py"
cd parakeet-indic-v1
python transcribe.py call.wav                  # [hi,en] full text
python transcribe.py --segments call.wav       # one line per piece: start-end  <lang>  text
python transcribe.py --precision fp32 --threads 8 call.wav

    0.0-    9.1  <ml>  പാലത്തിന്റെ താഴെയുള്ള കുത്തനെയുള്ള ഉയരം പതിനഞ്ച് മീറ്റർ ...
    9.1-   23.6  <ml>  രണ്ടായിരത്തി പതിനൊന്ന് ഓഗസ്റ്റിൽ പൂർത്തിയാക്കിയതാണ്

transcribe.py takes audio of any length, sample rate and channel count:

• Resampling to 16 kHz uses a polyphase filter, which works well for 8 kHz telephone audio.
• Levelling: quiet recordings are lifted (up to +30 dB), and each piece is loudness-normalised.
• Long audio is cut at pauses into pieces of at most 20 s and decoded in batches of 16. --vad uses Silero VAD instead, which helps on recordings with long silences.
• Stereo files are transcribed per channel; --mix merges them.

From Python:

import soundfile as sf
from transcribe import load_recognizer, transcribe

rec = load_recognizer("onnx", precision="int8", threads=4)
audio, sr = sf.read("call.wav", dtype="float32")     # mono, 16 kHz
for start, end, lang, text in transcribe(rec, [audio])[0]:
    print(f"{start:.1f}-{end:.1f} {lang}: {text}")

Which precision?

600 held-out clips (150 each: FLEURS Hindi, FLEURS Tamil, Vaani Bengali, Vaani English; 1.0 h of audio), 16 CPU threads, AMD EPYC Genoa, WER / CER (%):

set	sherpa-onnx fp32	sherpa-onnx int8
FLEURS Hindi	11.92 / 3.96	12.10 / 4.01
FLEURS Tamil	40.86 / 13.35	42.46 / 13.70
Vaani Bengali	28.27 / 9.46	28.90 / 9.31
Vaani English	12.04 / 5.70	12.62 / 6.26
all	22.23 / 8.64	22.93 / 8.85
speed	63× real time	70× real time

int8 costs ~0.7 WER / 0.2 CER for a 3× smaller encoder; use fp32 where accuracy matters more than size.

GPU: NeMo

pip install "nemo_toolkit[asr]>=2.4" "numba>=0.61,<0.63"   # numba 0.68 breaks the TDT loss kernel

import re
import nemo.collections.asr as nemo_asr
from omegaconf import open_dict

model = nemo_asr.models.ASRModel.restore_from("parakeet-indic-v1.nemo").cuda().eval()
cfg = model.cfg.decoding
with open_dict(cfg):
    cfg.greedy.use_cuda_graph_decoder = False   # the CUDA-graph decoder crashed in our runs
model.change_decoding_strategy(cfg)
raw = model.transcribe(["clip.wav"])[0].text      # "<ta> ..." ; clips up to ~30 s
lang = re.match(r"\s*<([a-z]{2})>", raw).group(1)
text = re.sub(r"<[a-z]{2}>", "", raw).strip()

To force a language, prime the prediction network with its tag before the first frame (greedy TDT loop); optionally restrict the output to that language's script using allowed_ids from the tokenizer's token_scripts.json.

Output conventions

• Each language in its own script (Assamese with ৰ / ৱ). English words inside Indic speech are usually in lowercase Latin (payment, account), sometimes in the native script.
• Numbers as digits for Hindi and English (25 तारीख, 2500, 9876543210). The other languages mostly keep number words (രണ്ടായിരത്തി പതിനൊന്ന്); per-language number conversion is planned.
• Light punctuation (। , ? .). All scores ignore punctuation.

Evaluation details

Text is normalised the same way on both sides: NFC, zero-width characters removed, punctuation removed, Latin lowercased, all-script digits converted to 0-9, number words to digits for Hindi and English. Vowel signs are kept. Scores are corpus-level.

Test sets come from held-out splits of the training corpora (IndicVoices valid, FLEURS / Vaani / SPRING-INX / Rasa / SUBAK.KO / IISc-MILE test) and 0.5% of NPTEL lectures held out by lecture. Up to 500 clips per set; clips whose transcript text also appears in training were removed.

Limitations

• Hindi is slightly behind parakeet-hi-v3 in automatic mode (table above).
• Language ID is lowest on Assamese (92.5% of clips; 79.8% on SPRING-INX) and Marathi (95.2%), the two languages that share a script with a larger one (Bengali, Hindi). Very short clips are the hardest.
• Spontaneous speech in the Dravidian languages (IndicVoices, SPRING-INX) is the hardest case: 45–70 WER, 17–35 CER.
• Numbers are digits only for Hindi and English.
• No timestamps in transcribe.py. Word timings are available through NeMo (timestamps=True).

Training

Init	parakeet-hi-v3 with a new 12,288-token SentencePiece vocabulary (9 languages, script-separated pieces, digits split, 9 language tags). 1,648 tokens shared with v3 kept their weights; new Devanagari / Latin tokens started as the mean of their v3 sub-pieces; tokens of new scripts started neutral.
Data	kapturecx/bolAIndia, streamed (no local copy). Stage 1 (45k steps): 42.8K h of human-transcribed, read-prompt and high-confidence data; every transcript checked to be in its language's script. Stage 2 (141k steps): + ~10K h of aligned and yodas3 YouTube data, kept only where the stage-1 model agrees with the label (per-language CER thresholds 0.14–0.30). Hindi: the v3 selection.
Mix	Hindi 50% (stage 2); other languages ∝ hours^0.4 so smaller languages (Assamese) are not drowned out.
Labels	per-language normalisation; number words → digits for Hindi and English; language tag prepended
Augmentation	v3 set (8 kHz phone channel with random resamplers and codecs, noise, cross-talk, quiet speech, noise-only clips with empty transcripts) + resampling round trips (8 / 11.025 / 12 kHz) on ~12% of clips + real call-centre background noise on ~5%
Recipe	TDT loss. AdamW (0.9, 0.98), weight decay 1e-3. LR 2e-4 decoder/joint, 5e-5 encoder, cosine schedule; stage 1 froze the encoder for the first 1k steps. bf16, ~1,000 s of audio per batch. 1× H200 (shared).

Base model: NVIDIA parakeet-tdt-0.6b-v3, CC-BY-4.0.