README.md

metadata

language:
  - sa
  - hi
  - kn
  - ta
  - te
  - ml
license: apache-2.0
library_name: onnx
pipeline_tag: text-to-speech
datasets:
  - ai4bharat/Rasa
tags:
  - text-to-speech
  - streaming-tts
  - ninada
  - sanskrit
  - vedic
  - indic
  - kannada
  - styletts2
  - istftnet
  - onnxruntime-web
  - wasm

Ninada (निनाद) v3.0 — 24,000 Hz Studio Classical Sanskrit & Pan-Indic Neural TTS

Ninada (Nināda — Sanskrit निनाद, deep resonant sound) v3.0 is a 24,000 Hz Studio Neural TTS engine (88.0 MB unified Q8F16 ONNX + 510 KB per-language studio voice banks, 361.0 ms native CPU median TTFA, 0.091x RTF) supporting 6 Indic languages (Sanskrit [sa], Kannada [kn], Hindi [hi], Tamil [ta], Telugu [te], Malayalam [ml]) across 9 Brahmic scripts + ISO-15919/IAST.

• Live Off-Main-Thread Browser AudioWorklet Studio: https://hemanth.github.io/ninada/
• GitHub Repository & Streaming SDKs (JS + Python): https://github.com/hemanth/ninada
• Curated 24 kHz Studio Split Manifests (2400 Clips): https://huggingface.co/datasets/gnumanth/ninada-sanskrit-indic

Architecture & GCP Cloud GPU Studio Fine-Tuning

• 24,000 Hz Harmonic-Source Fourier Synthesis: Built on StyleTTS2 + Contextual PL-BERT + Explicit 80 Hz F0/Energy ProsodyPredictor + Frozen Harmonic-Source iSTFTNet (SAMPLE_RATE = 24,000 Hz, HOP_LENGTH = 600 [40 Hz], F0_HOP_LENGTH = 300 [80 Hz]), warm-started from Apache-2.0 hexgrad/Kokoro-82M + Indic Male studio priors with dedicated slot 79 = 'ɭ' for Vedic & Dravidian retroflex laterals (ळ / ಳ / ள / ళ / ള).
• GCP Cloud GPU Adaptation + Validation WiSE-FT (alpha = 1.0): Trained on Google Cloud GPU across 1920 Viterbi-aligned 24 kHz studio Male clips (400 per language across sa, hi, kn, ta, te, ml: 1920 train / 240 val / 240 held-out test from ai4bharat/Rasa), keeping SineGen + CustomSTFT 100% frozen to preserve crisp 8–12 kHz studio sibilance and zero phase buzz while optimizing per-language voice banks, slot 79 = 'ɭ', PL-BERT, and ProsodyPredictor (duration_proj, F0, N).
• Unified < 90 MB Browser ONNX + Offline CacheStorage: Exports a single 88.0 MB onnx/ninada_24k_q8f16.onnx (2.71 dB MCD parity vs PyTorch FP32) shared across all 6 languages plus 510 KB voices/{sa,hi,kn,ta,te,ml}.bin style banks, streamed through a dedicated 24 kHz AudioWorklet ring buffer with 0 buffer underruns and 0 network refetches after initial cache load.

Held-Out 24 kHz Studio Test Leaderboard (N = 240 Utterances; B = 1,000 Bootstrap 95% CIs)

System	Active Footprint (↓)	Sample Rate	Coverage (↑)	TTFA p50 (ms, ↓)	Mean RTF (↓)	MCD 0–8k (dB, ↓)	Studio 8–12k HF Err (dB, ↓)	Log-F0 RMSE (c, ↓)	Dur MAE (s, ↓)
Base facebook/mms-tts-hin (Raw)	138.8 MB	16 kHz	35.0%	0.3	0.032x	7.98 [7.70, 8.24]	66.39 [65.65, 67.02]	1015.51 [983.33, 1047.17]	3.26 [2.94, 3.61]
Meta MMS 5-Specialist Fleet (5x VITS)	694.0 MB	16 kHz	100.0%	425.0	0.084x	5.10 [5.04, 5.16]	65.94 [65.51, 66.36]	673.92 [659.62, 688.25]	1.27 [1.13, 1.41]
Ninada Tier-2 Zero-Dep Klatt Fallback	0.0 MB	24 kHz	100.0%	37.9	0.0110x	10.38 [10.26, 10.50]	41.94 [41.27, 42.56]	613.77 [597.33, 630.07]	0.98 [0.88, 1.08]
Ninada v3.0 Untrained Warm-Start (alpha=0)	312.0 MB	24 kHz	100.0%	237.3	0.091x	5.26 [5.21, 5.31]	17.88 [17.34, 18.42]	714.12 [702.17, 725.85]	1.70 [1.59, 1.81]
Ninada v3.0 GCP Trained PyTorch (FP32)	312.0 MB	24 kHz	100.0%	174.7	0.093x	4.80 [4.76, 4.85]	9.45 [9.21, 9.71]	825.50 [814.05, 835.81]	0.58 [0.51, 0.65]
Ninada v3.0 Unified Q8F16 ONNX (Browser)	88.0 MB	24 kHz	100.0%	361.0	0.091x	5.05 [5.00, 5.10]	10.87 [10.57, 11.16]	828.06 [817.71, 838.24]	0.57 [0.51, 0.64]

24 kHz Streaming Quickstart

import ninada from 'ninada';

const voice = await ninada('kannada', {
  model: 'https://huggingface.co/gnumanth/ninada/resolve/main/onnx/ninada_24k_q8f16.onnx',
  voiceUrl: 'https://huggingface.co/gnumanth/ninada/resolve/main/voices/kn.bin',
});
for await (const chunk of voice.stream('ನಮಸ್ಕಾರ, ಇಂದು ಬೆಂಗಳೂರಿನಲ್ಲಿ ಹವಾಮಾನ ತುಂಬಾ ಚೆನ್ನಾಗಿದೆ.')) {
  playPcm24k(chunk.pcm, chunk.sampleRate); // 24,000 Hz studio stream!
}

from ninada import Ninada

tts = Ninada(lang="sa")
for chunk in tts.stream("प्रज्ञापूर्वकव्यापारवत्त्वं जीवत्वम् । यथार्थज्ञानं प्रमाणम् ॥"):
    play(chunk["pcm"], chunk["sample_rate"])  # 24,000 Hz float32 PCM