kokoro-ne

Nepali text-to-speech that also speaks Hindi and English, including mixed sentences ("आज मेरो office मा meeting छ"). 82M parameters, 24 kHz, fast on a 2-core CPU. A fine-tune of goonj-1-82M (itself a Hindi fine-tune of Kokoro-82M) on about 4 hours of multi-speaker Nepali.

Code, usage and any-voice cloning: github.com/Nayanbst/kokoro-ne

Files

File What it is
kokoro_ne.pth KModel weights (load with the kokoro package's KModel)
config.json Model config and phoneme vocabulary (goonj's, unchanged)
voices/openslr_0546.pt, voices/openslr_2099.pt Voicepacks of two OpenSLR 43 speakers (CC BY-SA 4.0)

Use

import torch
from huggingface_hub import hf_hub_download
from kokoro import KModel

repo = "nayanbastakoti/kokoro-ne"
model = KModel(repo_id="hexgrad/Kokoro-82M", config=hf_hub_download(repo, "config.json"),
               model=hf_hub_download(repo, "kokoro_ne.pth")).eval()
voice = torch.load(hf_hub_download(repo, "voices/openslr_0546.pt"), weights_only=True)
# phonemes: misaki EspeakG2P("ne") for Nepali, EspeakG2P("hi") for Hindi, misaki en.G2P for English
audio = model(phonemes, voice[len(phonemes) - 1], 1.0)

The repository's kokoro_ne package (TTS, CloneTTS) handles phonemisation, language mixing and long text for you.

Training

Data Hours Speakers
OpenSLR 43 Nepali TTS 2.6 18 (all female)
FLEURS Nepali, cleanest clips 1.3 many
indic-audio replay (Hindi, English) 1.0 15

kikiri-tts StyleTTS 2 recipe on one T4: Stage 1, 2 epochs (validation mel loss 0.342 → 0.322); Stage 2, 1 epoch.

Limitations

The Nepali speech is mostly female, so voices lean female; for male voices use the repository's kokoro-ne-clone (Kanade voice conversion). Numbers are best written as words. No listening tests or ASR scores yet.

Licences

Weights: Apache-2.0, like the base models. They were trained on OpenSLR 43 (CC BY-SA 4.0), FLEURS (CC BY 4.0) and indic-audio, whose synthetic audio follows the terms of the services that generated it; review these before commercial use. The two voicepacks are derived from OpenSLR 43 recordings and are shared under CC BY-SA 4.0. Not legal advice.

Downloads last month
10
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for nayanbastakoti/kokoro-ne

Finetuned
(1)
this model

Datasets used to train nayanbastakoti/kokoro-ne