| --- |
| language: |
| - sa |
| - hi |
| - kn |
| - ta |
| - te |
| - ml |
| license: apache-2.0 |
| library_name: onnx |
| pipeline_tag: text-to-speech |
| datasets: |
| - ai4bharat/Rasa |
| tags: |
| - text-to-speech |
| - streaming-tts |
| - ninada |
| - sanskrit |
| - vedic |
| - indic |
| - kannada |
| - styletts2 |
| - istftnet |
| - onnxruntime-web |
| - wasm |
| --- |
| |
| # `Ninada` (निनाद) v3.0 — 24,000 Hz Studio Classical Sanskrit & Pan-Indic Neural TTS |
|
|
| `Ninada` (*Nināda* — Sanskrit `निनाद`, deep resonant sound) v3.0 is a **24,000 Hz Studio Neural TTS** engine (**`88.0 MB` unified Q8F16 ONNX + `510 KB` per-language studio voice banks, `361.0 ms` native CPU median TTFA, `0.091x` RTF**) supporting **6 Indic languages** (`Sanskrit [sa]`, `Kannada [kn]`, `Hindi [hi]`, `Tamil [ta]`, `Telugu [te]`, `Malayalam [ml]`) across **9 Brahmic scripts + ISO-15919/IAST**. |
|
|
| - **Live Off-Main-Thread Browser `AudioWorklet` Studio**: [https://hemanth.github.io/ninada/](https://hemanth.github.io/ninada/) |
| - **GitHub Repository & Streaming SDKs (JS + Python)**: [https://github.com/hemanth/ninada](https://github.com/hemanth/ninada) |
| - **Curated 24 kHz Studio Split Manifests (`2400` Clips)**: [https://huggingface.co/datasets/gnumanth/ninada-sanskrit-indic](https://huggingface.co/datasets/gnumanth/ninada-sanskrit-indic) |
|
|
| ## Architecture & GCP Cloud GPU Studio Fine-Tuning |
| - **24,000 Hz Harmonic-Source Fourier Synthesis**: Built on **`StyleTTS2 + Contextual PL-BERT + Explicit 80 Hz F0/Energy ProsodyPredictor + Frozen Harmonic-Source iSTFTNet`** (`SAMPLE_RATE = 24,000 Hz`, `HOP_LENGTH = 600` [`40 Hz`], `F0_HOP_LENGTH = 300` [`80 Hz`]), warm-started from Apache-2.0 `hexgrad/Kokoro-82M` + Indic Male studio priors with dedicated slot `79 = 'ɭ'` for Vedic & Dravidian retroflex laterals (`ळ / ಳ / ள / ళ / ള`). |
| - **GCP Cloud GPU Adaptation + Validation `WiSE-FT` (`alpha = 1.0`)**: Trained on Google Cloud GPU across `1920` Viterbi-aligned 24 kHz studio Male clips (`400` per language across `sa`, `hi`, `kn`, `ta`, `te`, `ml`: `1920` train / `240` val / `240` held-out test from `ai4bharat/Rasa`), keeping `SineGen + CustomSTFT` 100% frozen to preserve crisp `8–12 kHz` studio sibilance and zero phase buzz while optimizing per-language voice banks, slot `79 = 'ɭ'`, `PL-BERT`, and `ProsodyPredictor` (`duration_proj`, `F0`, `N`). |
| - **Unified `< 90 MB` Browser ONNX + Offline `CacheStorage`**: Exports a single `88.0 MB` `onnx/ninada_24k_q8f16.onnx` (`2.71 dB` MCD parity vs PyTorch FP32) shared across all 6 languages plus `510 KB` `voices/{sa,hi,kn,ta,te,ml}.bin` style banks, streamed through a dedicated `24 kHz` `AudioWorklet` ring buffer with `0` buffer underruns and `0` network refetches after initial cache load. |
|
|
| ## Held-Out 24 kHz Studio Test Leaderboard (`N = 240` Utterances; `B = 1,000` Bootstrap 95% CIs) |
|
|
| | System | Active Footprint (↓) | Sample Rate | Coverage (↑) | TTFA p50 (ms, ↓) | Mean RTF (↓) | MCD 0–8k (dB, ↓) | Studio 8–12k HF Err (dB, ↓) | Log-F0 RMSE (c, ↓) | Dur MAE (s, ↓) | |
| |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| |
| | Base `facebook/mms-tts-hin` (Raw) | 138.8 MB | 16 kHz | 35.0% | 0.3 | 0.032x | `7.98 [7.70, 8.24]` | `66.39 [65.65, 67.02]` | `1015.51 [983.33, 1047.17]` | `3.26 [2.94, 3.61]` | |
| | Meta MMS 5-Specialist Fleet (`5x` VITS) | 694.0 MB | 16 kHz | 100.0% | 425.0 | 0.084x | `5.10 [5.04, 5.16]` | `65.94 [65.51, 66.36]` | `673.92 [659.62, 688.25]` | `1.27 [1.13, 1.41]` | |
| | `Ninada` Tier-2 Zero-Dep Klatt Fallback | 0.0 MB | 24 kHz | 100.0% | 37.9 | 0.0110x | `10.38 [10.26, 10.50]` | `41.94 [41.27, 42.56]` | `613.77 [597.33, 630.07]` | `0.98 [0.88, 1.08]` | |
| | `Ninada` v3.0 Untrained Warm-Start (`alpha=0`) | 312.0 MB | 24 kHz | 100.0% | 237.3 | 0.091x | `5.26 [5.21, 5.31]` | `17.88 [17.34, 18.42]` | `714.12 [702.17, 725.85]` | `1.70 [1.59, 1.81]` | |
| | **`Ninada` v3.0 GCP Trained PyTorch (FP32)** | 312.0 MB | **24 kHz** | **100.0%** | **174.7** | **0.093x** | **`4.80 [4.76, 4.85]`** | **`9.45 [9.21, 9.71]`** | **`825.50 [814.05, 835.81]`** | **`0.58 [0.51, 0.65]`** | |
| | **`Ninada` v3.0 Unified Q8F16 ONNX (Browser)** | **88.0 MB** | **24 kHz** | **100.0%** | **361.0** | **0.091x** | **`5.05 [5.00, 5.10]`** | **`10.87 [10.57, 11.16]`** | **`828.06 [817.71, 838.24]`** | **`0.57 [0.51, 0.64]`** | |
|
|
| ## 24 kHz Streaming Quickstart |
|
|
| ```js |
| import ninada from 'ninada'; |
| |
| const voice = await ninada('kannada', { |
| model: 'https://huggingface.co/gnumanth/ninada/resolve/main/onnx/ninada_24k_q8f16.onnx', |
| voiceUrl: 'https://huggingface.co/gnumanth/ninada/resolve/main/voices/kn.bin', |
| }); |
| for await (const chunk of voice.stream('ನಮಸ್ಕಾರ, ಇಂದು ಬೆಂಗಳೂರಿನಲ್ಲಿ ಹವಾಮಾನ ತುಂಬಾ ಚೆನ್ನಾಗಿದೆ.')) { |
| playPcm24k(chunk.pcm, chunk.sampleRate); // 24,000 Hz studio stream! |
| } |
| ``` |
|
|
| ```python |
| from ninada import Ninada |
| |
| tts = Ninada(lang="sa") |
| for chunk in tts.stream("प्रज्ञापूर्वकव्यापारवत्त्वं जीवत्वम् । यथार्थज्ञानं प्रमाणम् ॥"): |
| play(chunk["pcm"], chunk["sample_rate"]) # 24,000 Hz float32 PCM |
| ``` |
|
|
