You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Matcha-TTS Nepali (Chatterbox, single speaker, English+Nepali mixed)

Fine-tuned Matcha-TTS acoustic model for Nepali (with mixed English words in Latin script) built on the chatterbox-nepali dataset, warm-started from the pretrained English (LJSpeech) base checkpoint.

Model summary

Property	Value
Architecture	Matcha-TTS (CFM decoder, RoPE encoder)
Vocab	union (n_vocab=264): English IPA (178) + real_nepali G2P phones preserved at 0..177 so the English base loads 100%
G2P	`real_nepali` profile `real_nepali_v0.2` (Devanagari) + espeak-ng en-us IPA (Latin)
Speakers	1 (single speaker)
Sample rate	22,050 Hz
Mel	n_feats=80, n_fft=1024, hop=256, f_max=8000; mel_mean=-5.980818, mel_std=2.582703
Warm start	`pretrained/matcha_tts_ljspeech_english.ckpt` via `extend_vocab.py`
Optimisation	Adam lr=1e-4 constant, batch 32 x grad-accum 2 (eff. 64), 16-mixed precision, grad-clip 5.0
Training	150-epoch target, early-stopped (see Re-generation)

Files

Path	Description
`checkpoints/matcha_nepali_final.ckpt`	Final acoustic model (epoch 999 of the original run, picked by listening)
`configs/nepali_real_fresh.yaml`	Experiment config (hydra) used for training
`configs/nepali_real.yaml`	Data config (filelists, batch, mel stats)
`configs/matcha.yaml`	Model config (architecture, optimizer, scheduler)
`scripts/synthesize.py`	Inference script (union symbols, denoiser, mel-stats matching)
`scripts/extend_vocab.py`	Extends the English base ckpt to the 264 union vocab

Usage

.venv/bin/python scripts/synthesize.py \
  --ckpt checkpoints/matcha_nepali_final.ckpt \
  --file texts.txt --out outputs --symbols union \
  --steps 25 --denoiser_strength 0.02

One utterance per line in texts.txt. Mixed Devanagari + Latin works in the same sentence, e.g. नमस्ते, welcome to our bank.

Recommended synthesis settings (cleanest output):

--steps 25 (more ODE steps than the 10 default = less noise)
--denoiser_strength 0.02 (vs default 0.00025: removes high-frequency hiss)

Vocoder: universal HiFi-GAN hifigan_univ_v1 (g_02500000).

Regeneration

Prepare the dataset (split + phonemize with real_nepali G2P), loudness-normalize to data/wavs_norm/, set mel stats in the data config.

Extend the English base to the union vocab:

.venv/bin/python scripts/extend_vocab.py \
  --src pretrained/matcha_tts_ljspeech_english.ckpt \
  --out checkpoints/matcha_tts_english_plus_nepali.ckpt

Train:
```
bash scripts/train.sh fresh
```

10-15k steps on a 1.5k-utterance single-speaker set is enough; longer runs overfit (val loss rises again ~epoch 126) which adds noise and degrades the English-prosody carried from the base model.

Notes

The English base checkpoint is MIT (Shivam Mehta et al., ICASSP 2024); training data comes from the chatterbox-nepali dataset (CC-BY-SA-4.0).
The real_nepali G2P frontend (CC-BY-SA-4.0, Ampixa Labs) is required at train/inference time for Devanagari text.
This upload (weights + docs) is CC-BY-4.0.

Downloads last month: -; Downloads are not tracked for this model. How to track