AksharaVP-17M — Telugu, Hindi and Indian English text to speech in 18.0M parameters, animated banner

Live demo Parameters Release v2 Languages Zero-shot cloning CPU speed GGUF License

AksharaVP-17M

Telugu, Hindi and Indian English speech from one 18.0M-parameter checkpoint.
Built-in voices, zero-shot voice cloning, and it runs on a laptop CPU.

17,988,627 6.20% · 4.17% · 4.79% 12 + clone 36.0 MB 2.8× 1
parameters, decoder included character error rate · te / hi / en (FLEURS dev) built-in voices · zero-shot cloning f16 GGUF (72.0 MB fp32) faster than real time on 1 CPU thread checkpoint for all three languages

AksharaVP-17M reads Telugu, Hindi (Devanagari) and Indian English aloud at 24 kHz. A small transformer writes continuous audio latents one 80 ms frame at a time, a 4-step flow-map head samples each frame, and Kyutai's 2.6M-parameter Mimi decoder turns them into a waveform. One file holds everything the speaking path needs: no phonemiser, no external vocoder, no GPU.

Animated waveforms of three real AksharaVP-17M outputs: Telugu, Hindi and Indian English

⚠️ Naturalness is the weak point. Speech is clear (low error rates below) but flatter and less expressive than large TTS systems: UTMOS is 2.65 (te), 2.84 (hi) and 3.19 (en) on a 1–5 scale. See the Comparison section for how that sits next to bigger models.

What's new in v2 (round 5)

v2 is a drop-in update of the same model: 17,988,627 parameters (+345,601, +2.0%), same files, same API. v1 copied a voice only from the prompt latents. v2 adds a 172,993-parameter speaker-fingerprint module (SpkPool) that pools the voice prompt into a 192-d speaker vector, which steers the backbone and the flow head (345,601 parameters for the whole speaker path). It was fine-tuned with many more real speakers (LibriTTS-R plus FLEURS-train Hindi and Telugu recordings) as cloning prompts.

v1 vs v2 zero-shot cloning on FLEURS dev: higher speaker similarity, far fewer dropouts and long pauses in every language

Zero-shot cloning, v1 → v2: the same 30 FLEURS dev reference clips and sentences per language for both versions, paired bootstrap (10,000 resamples):

Metric Telugu Hindi Indian English
Speaker similarity, ECAPA cosine (higher = closer) 0.295 → 0.402 p<0.001 0.259 → 0.363 p<0.001 0.179 → 0.302 p<0.001
Speaker similarity, WavLM-SV cosine 0.879 → 0.913 p=0.002 0.861 → 0.904 p=0.003 0.860 → 0.889 p=0.218
Dropouts per clip (brief energy dips inside speech) 11.2 → 1.1 p<0.001 10.8 → 2.2 p<0.001 7.9 → 1.8 p<0.001
Pauses ≥ 300 ms per clip 1.23 → 0.23 p<0.001 1.33 → 0.33 p<0.001 0.67 → 0.63 p=0.993
Pauses ≥ 700 ms per clip 0.10 → 0.00 p=0.085 0.20 → 0.00 p=0.002 0.07 → 0.23 p=0.255
Total pause time per clip (s) 0.76 → 0.15 p<0.001 0.87 → 0.23 p<0.001 0.51 → 0.76 p=0.565
Clone UTMOS (1–5) 2.38 → 1.89 p<0.001 2.54 → 2.15 p<0.001 3.02 → 2.44 p<0.001
Clone CER % 7.87 → 10.42 p=0.026 6.40 → 6.72 p=0.723 4.39 → 9.75 p=0.073
Built-in voices, CER % 6.04 → 6.20 p=0.577 4.57 → 4.17 p=0.135 4.42 → 4.79 p=0.315
Built-in voices, WER % 28.94 → 30.38 p=0.218 13.97 → 12.66 p=0.076 9.50 → 9.65 p=0.896
Built-in voices, UTMOS 2.64 → 2.65 p=0.489 2.73 → 2.84 p=0.002 3.19 → 3.19 p=0.933
  • Better: clones sound clearly more like the reference speaker in all three languages, and brief dropouts fell from about 8–11 per clip to 1–2. Long pauses (≥ 300 ms) fell from about 1.2–1.3 to 0.2–0.3 per clip in Telugu and Hindi; English pauses showed no significant change.
  • Worse, stated plainly: clone UTMOS fell by 0.39–0.58, and Telugu clone CER rose by 2.55 points (significant). English clone CER also rose (+5.36, p=0.073, not significant). v2's clone UTMOS (1.89 te, 2.15 hi, 2.44 en) is close to (te, hi) or above (en) the UTMOS of the reference recordings themselves (1.88, 2.15, 1.99), so v2 may be copying the reference's microphone and room along with the voice. That is a hypothesis, not a measured fact. Clone from a clean recording for the best result.
  • Built-in voices: no significant change in CER or WER; Hindi UTMOS is slightly higher.

Listen

All clips are raw model output from the files in this repository (published sampler: guidance 1.25, temperature 0.3, 4 flow steps, seed 0) — nothing hand-picked after listening. Texts were written for this card (they are not FLEURS evaluation sentences).

Language Voice Text Audio
Hindi Meera नमस्ते! मैं अक्षरा हूँ, और मैं हिंदी, तेलुगु और अंग्रेज़ी बोल सकती हूँ।
Hindi Siddharth कल सुबह दस बजे हमारी मीटिंग है, कृपया समय पर पहुँचिए।
Hindi Nandini बारिश के बाद हवा में मिट्टी की खुशबू कितनी अच्छी लगती है।
Telugu Lakshmi నమస్కారం! నేను అక్షర, తెలుగు, హిందీ మరియు ఇంగ్లీష్ మాట్లాడగలను.
Telugu Srinivas రేపు ఉదయం పది గంటలకు మన సమావేశం ఉంది, దయచేసి సమయానికి రండి.
Telugu Sravani వర్షం తర్వాత మట్టి వాసన ఎంతో హాయిగా ఉంటుంది.
Indian English Meera Hello! I'm a tiny text to speech model that runs on an ordinary laptop processor.
Indian English Ishaan Your train to Hyderabad leaves from platform four at nine fifteen.
Indian English Sneha Small models can still speak clearly, if they are trained with care.

Zero-shot voice cloning

A few seconds of someone's speech in, the same voice reading new text out. The reference clips are FLEURS dev recordings (CC-BY-4.0); the English row clones a Hindi speaker (cross-lingual). Cloning needs the optional clone_encoder.safetensors (9,749,024 parameters); the speaker-fingerprint module is already inside model.safetensors.

Language Reference clip Text AksharaVP-17M
Hindi यह आवाज़ सिर्फ़ कुछ सेकंड की रिकॉर्डिंग से बनाई गई है।
Telugu ఈ గొంతు కేవలం కొన్ని సెకన్ల రికార్డింగ్ నుండి తయారైంది.
Indian English This voice was copied from a few seconds of recorded speech.

What it achieves

# Claim The evidence
1 Three languages, one 18.0M checkpoint. 17,988,627 parameters: 15,371,394 for text → latents (including the 345,601-parameter speaker path) and 2,617,233 for the decoder, counted from model.safetensors. Telugu, Hindi and Indian English share every weight.
2 Smallest open TTS we found that speaks Telugu + Hindi + English from one checkpoint and clones voices. 27,737,651 parameters with the clone encoder. The next smallest we found: Pravega Pocket TTS 121.8M, DhVaani-0.5 122.8M, IndicF5 350.7M (Hub safetensors metadata, 17 Nov 2026). Smaller single-language Indic TTS exist (e.g. 8.9M-parameter per-language VITS models), but without cloning or a shared checkpoint.
3 Low character error rates on all three languages. FLEURS dev, 100 sentences each, ASR by facebook/mms-1b-all: CER 6.20% te, 4.17% hi, 4.79% en.
4 Faster than real time on a CPU. RTF 0.36 or better on one thread of a cloud Xeon, 0.19 or better on four (CPU speed). No GPU needed.
5 Bit-exact GGUF. The f32 GGUF reproduces the safetensors output sample-for-sample (max abs difference 0.0); the f16 GGUF is 36.0 MB with the same frame counts and max sample difference 0.087.
6 Runs in a browser tab. The demo runs the full model client-side with ONNX Runtime Web (WebAssembly), no server: same frame counts as PyTorch, audio error at or below -71 dB relative to the signal (details).

Parameter counts of open TTS that speak Telugu, Hindi and English from one checkpoint and clone voices: AksharaVP-17M with clone encoder 27.7M, Pravega Pocket TTS 121.8M, DhVaani-0.5 122.8M, IndicF5 350.7M

Stated plainly: "smallest" here means the smallest we found in that specific category, by parameter count, on the date above — not a claim about every TTS model in existence, and not a quality ranking. The other models in that chart were not evaluated by us.


How it works

Architecture: text and voice prompt into an 8-layer d320 transformer, a 4-step flow-map head producing 32-d latent frames at 12.5 Hz, Kyutai Mimi decoder to 24 kHz audio

  • Text: characters, not phonemes. Telugu and Devanagari share one 128-slot table (same Unicode offsets), plus ASCII — a 256-symbol vocabulary with a rank-64 factorised embedding. Numbers, dates and symbols are expanded by the bundled normaliser.
  • Backbone: 8 pre-LN transformer layers, width 320, grouped-query attention (5 query heads, 1 KV head), RoPE, causal. It reads [text][voice prompt][frames so far] and predicts the next frame's conditioning vector and an end-of-speech logit.
  • Flow-map head: an MLP-AdaLN (512 wide, 6 blocks, two time inputs) that jumps from noise to a 32-d latent frame in 4 steps, with classifier-free guidance on its outputs.
  • Decoder: Kyutai Pocket TTS's Mimi decoder (english_7m_2026-10), unchanged: 32-d latents at 12.5 Hz → 24 kHz audio.
  • Voices: a voice is a prompt of up to 62 latent frames (~5 s). 12 are built in; cloning encodes your clip into the same space.
  • Speaker fingerprint (new in v2): SpkPool (172,993 parameters, attentive mean + s.d. pooling) turns the prompt into a unit 192-d speaker vector. It is added to the prompt positions and to the flow head's conditioning (345,601 parameters in all). It runs for built-in voices and clones alike.

Parameter budget: flow-map head 53%, transformer layers 29%, speaker path 2%, embeddings and projections 1%, Mimi decoder 15%


Benchmarks

FLEURS dev results per language: CER, WER and UTMOS with 95% intervals

FLEURS dev, first 100 sentences per language, built-in voices (cycled), published sampler through the Python API (AksharaVP.say, automatic retakes on). Brackets: 95% bootstrap intervals. English uses FLEURS en_us sentences spoken with the built-in Indian voices.

Language CER % WER % UTMOS (1–5)
Telugu 6.20 [5.06, 7.53] 30.38 [27.16, 33.63] 2.65 [2.59, 2.72]
Hindi 4.17 [3.46, 4.96] 12.66 [10.89, 14.55] 2.84 [2.75, 2.91]
Indian English 4.79 [3.26, 6.41] 9.65 [7.47, 11.93] 3.19 [3.12, 3.26]
  • ASR: facebook/mms-1b-all (CTC, greedy, language adapters hin/tel/eng); CER/WER after Unicode NFC and punctuation removal. Telugu WER is high for every system we measured because Telugu words are long and agglutinative — one wrong character costs a whole word.
  • Naturalness: UTMOS22-strong (SpeechMOS). It is an automatic predictor trained mostly on English; treat absolute values for Telugu/Hindi with care, but differences between systems on the same sentences are informative.
  • Dev, not test. Every choice that shaped this release (checkpoint, guidance, temperature, v2 over other round-5 variants) was made on the dev split or on training-validation loss. The FLEURS test split has not been used for this model.
  • The Python API retakes automatically when a take runs on or goes silent, and these numbers include that, as a user gets them. The v1 card reported v1 without retakes (CER 6.04 / 4.57 / 4.37); measured the same way as here, v1 scores 6.04 / 4.57 / 4.42.
  • Zero-shot cloning, first 30 sentences per language, each cloned from a FLEURS dev reference clip: speaker similarity (ECAPA) 0.40 te, 0.36 hi, 0.30 en; clone CER 10.42 / 6.72 / 9.75% (see What's new).

CPU speed

Real-time factor = synthesis wall time ÷ audio duration (lower is faster), including decoding; first 20 FLEURS dev sentences per language, PyTorch 2.15.0+cpu on an 8-vCPU Intel Xeon cloud machine, measured with no other heavy job running.

Language 1 thread 4 threads clone prompt encode (4 threads)
Telugu 0.357 (2.8× real time) 0.188 (5.3× real time) 0.27 s
Hindi 0.356 (2.8× real time) 0.193 (5.2× real time) 0.28 s
Indian English 0.345 (2.9× real time) 0.182 (5.5× real time) 0.22 s

In the browser (demo, ONNX Runtime Web on WebAssembly, headless Chrome on an 8-vCPU cloud machine) the full-precision model ran at RTF 0.30–0.32 on 1 thread, 0.18 on 4 threads.


Comparison

Same benchmark for every row: FLEURS dev (first 100 sentences per language, same sentence ids), facebook/mms-1b-all CER / WER (%) and UTMOS22-strong, each system with its own built-in voices and default sampler, all measured by us on the same machine with the same scripts. Parameters = the speaking path (text → audio) excluding any clone-only encoder; fp32 size = parameters × 4 bytes.

System Parameters fp32 size Languages Telugu CER / WER / UTMOS Hindi CER / WER / UTMOS English CER / WER / UTMOS
AksharaVP-17M v2 (this model) 17,988,627 72.0 MB te · hi · en, one checkpoint 6.20 / 30.38 / 2.65 4.17 / 12.66 / 2.84 4.79 / 9.65 / 3.19
AksharaVP-17M v1 (previous release) 17,643,026 70.6 MB te · hi · en, one checkpoint 6.04 / 28.94 / 2.64 4.57 / 13.97 / 2.73 4.42 / 9.50 / 3.19
Hima TTS nano (hi + te checkpoints) 72,160,162 288.6 MB te+en / hi+en, two checkpoints 6.71 / 31.23 / 3.50 4.70 / 14.80 / 3.61 5.29 / 11.47 / 4.03
Kyutai Pocket TTS (english) 99,752,802 399.0 MB en only — — 2.35 / 6.01 / 3.99
Kyutai Pocket TTS english_7m_2026-10 6,862,578 27.5 MB en only — — 1.99 / 6.37 / 4.10
Kyutai Pocket TTS english_2m_2026-10 1,922,846 7.7 MB en only — — 3.89 / 9.70 / 3.49
Meta MMS-TTS tel (VITS) 36,289,776 145.2 MB te only 10.48 / 41.67 / 3.28 — —
Meta MMS-TTS hin (VITS) 36,291,120 145.2 MB hi only — 11.26 / 28.63 / 3.49 —
Meta MMS-TTS eng (VITS) 36,284,592 145.1 MB en only — — 6.46 / 17.33 / 4.10

Notes: the AksharaVP and Hima rows were generated through each model's published Python API (sentence split, automatic retake when a take runs on or goes silent); the Pocket TTS and MMS-TTS rows use those models' own default generation. Hima TTS nano ships separate Telugu and Hindi checkpoints of 72,160,162 parameters each (English is spoken by the Hindi one; its English row uses that checkpoint). Pocket TTS rows are English-only models speaking with their own (non-Indian) built-in voices, so English accent differs from the Indian voices of the other rows. Meta MMS-TTS rows are three separate single-language VITS checkpoints (16 kHz output, no cloning, CC-BY-NC-4.0), given the same normalised input text as AksharaVP; their ASR judge (mms-1b-all) comes from the same MMS project. UTMOS is an English-centric predictor; compare systems within a language column, not across columns.

Paired bootstrap, AksharaVP-17M v2 − Hima TTS nano (same 100 sentences per language, 10,000 resamples, seed 1; negative CER/WER = fewer errors, negative UTMOS = less natural):

Language ΔCER (points) ΔWER (points) ΔUTMOS
Telugu -0.51 [-1.16, +0.11] (p=0.117) -0.85 [-3.13, +1.45] (p=0.492) -0.84 [-0.91, -0.77]
Hindi -0.52 [-1.25, +0.18] (p=0.155) -2.14 [-3.99, -0.31] (p=0.024) -0.77 [-0.86, -0.69]
Indian English -0.50 [-1.78, +0.78] (p=0.429) -1.82 [-4.59, +0.56] (p=0.149) -0.84 [-0.92, -0.76]

Read honestly: AksharaVP-17M v2's error rates are as low as or lower than Hima TTS nano's (lower WER is significant only in Hindi; every CER difference is within noise), at a quarter of the parameters — but it is clearly less natural (UTMOS 0.77–0.84 lower, significant in every language). Kyutai's 99.8M and 6.9M English-only Pocket TTS models beat it on English on every metric; the 1.9M one has lower CER and higher UTMOS, with WER about equal. Meta's single-language MMS-TTS models (36.3M parameters each) make more ASR errors than AksharaVP-17M in all three languages on this benchmark, but score higher on UTMOS.

Zero-shot cloning (the same 30 FLEURS dev reference clips per language; speaker similarity = ECAPA cosine to the reference, higher is closer):

Language v2 similarity v1 similarity Hima TTS nano similarity v2 clone UTMOS Hima clone UTMOS v2 pauses ≥ 300 ms / clip Hima pauses ≥ 300 ms / clip
Telugu 0.40 0.30 0.56 1.89 2.99 0.23 1.50
Hindi 0.36 0.26 0.49 2.15 3.07 0.33 1.27
Indian English 0.30 0.18 0.42 2.44 3.38 0.63 0.90

v2 closes about 40–50% of v1's speaker-similarity gap to the 4× larger Hima TTS nano, but Hima's clones are still closer to the speaker and score higher on UTMOS. v2's clones have fewer long pauses than Hima's in Telugu and Hindi (significant) and about as many in English.


Quick start

pip install -U huggingface_hub
hf download sourojitdotcom/AksharaVP-17M --local-dir AksharaVP-17M
cd AksharaVP-17M
pip install -r requirements.txt
from aksharavp_tts import AksharaVP

tts = AksharaVP(".")                      # loads model.safetensors (+ clone_encoder.safetensors if present)
audio = tts.say("నమస్కారం! మీరు ఎలా ఉన్నారు?", lang="te", voice="Lakshmi")
tts.save(audio, "telugu.wav")

audio = tts.say("कल सुबह दस बजे मिलते हैं।", lang="hi", voice="Siddharth")
audio = tts.say("Your table is ready.", lang="en", clone="my_voice.wav")   # zero-shot cloning, 3–10 s clip
python -m aksharavp_tts "नमस्ते, आप कैसे हैं?" --lang hi --voice Meera -o hello.wav
python -m aksharavp_tts --list-voices

Built-in voices — Hindi set: Meera, Sneha, Nandini (female), Siddharth, Ishaan, Nikhil (male); Telugu set: Lakshmi, Sravani, Bhavana (female), Srinivas, Karthik, Pranay (male). Any voice can speak any of the three languages. Options: temperature (0.3), cfg (guidance, 1.25), steps (4), seed.

Run it in your browser

The live demo runs the whole model client-side — text normaliser (Pyodide), transformer, flow head and streaming decoder (ONNX Runtime Web, WebAssembly) — with no inference server. The ONNX graphs and the JavaScript sampler are in the Space files.

Weights Download Same frame count as PyTorch Max frame difference Audio error vs PyTorch Clone-prompt latents, max difference
Full precision (default) 73.4 MB (+39.1 MB clone encoder) yes (6/6 sentences) 6.8e-04 -95 to -71 dB 1.0e-05
Lite (fp16 storage, transformer kept in fp32) 48.4 MB (+19.7 MB) yes (6/6) 2.0e-01 -46 to -20 dB 1.5e-05

Measured in headless Chrome by feeding the browser sampler the same noise as PyTorch (retakes off): 4 built-in-voice and 2 cloned-voice sentences, compared with AksharaVP.say(). The lite copy keeps the transformer in fp32 because fp16 storage there drifted from PyTorch. Its audio differs most on one English sentence (−20 dB), with the same frame counts. Runtime files (ONNX Runtime Web + Pyodide) add about 29 MB on first load; everything is cached afterwards.

GGUF

File Size Storage Parity with model.safetensors
gguf/aksharavp-17m-f32.gguf 71.98 MB F32, all 300 tensors tensors identical (max abs diff 0.0); audio bit-identical on te/hi/en test sentences
gguf/aksharavp-17m-f16.gguf 36.01 MB F16, upcast on load tensor max abs diff 0.0072; same frame counts; waveform max abs diff ≤ 0.087

GGUF v3 containers written with the gguf Python package; metadata holds the full config, parameter counts and sampler defaults. llama.cpp has no runtime for this architecture — these are portable weight files. Load them back into PyTorch with:

from gguf_load import load_gguf
tts = load_gguf("gguf/aksharavp-17m-f16.gguf")
tts.save(tts.say("నమస్కారం", lang="te"), "out.wav")

Files

File What
model.safetensors the 17,988,627-parameter speaking path (student.* incl. the speaker-fingerprint module + mimi.* decoder), fp32, 72.0 MB
clone_encoder.safetensors optional audio encoder for zero-shot cloning (9,749,024 params)
voices.safetensors 12 built-in voice prompts (62 × 32 latents each)
config.json architecture, parameter counts, sampler defaults, voices
aksharavp_tts/ PyTorch inference code (AksharaVP.say) and the text normaliser
gguf/, gguf_load.py GGUF containers and loader
samples/ the audio on this page
evaluation/ measured numbers behind every table (benchmarks, comparison, GGUF parity, speed)

Training

Knowledge distillation into a small student: latent-level L2 on the teacher's hidden states plus sample-level distillation of the flow head, from a larger in-house teacher TTS (used with its authors' written permission; the flow-map head is initialised from its head, and the built-in voice prompts and clone encoder come from it). Teacher speech was generated for IndicCorp v2 sentences (CC0) in Telugu, Hindi and English, with FLEURS dev/test sentences removed from the training text; LibriTTS-R (CC-BY-4.0) clips served as cloning prompts. v1: 100,026 steps on one Kaggle T4 GPU (about 10.8 h), initialised from an earlier distilled checkpoint of the same architecture. v2 (round 5): initialised from v1; SpkPool was first trained separately to predict a speaker embedding (ECAPA) from prompt latents, then the model was distilled again with the speaker path added, using LibriTTS-R and FLEURS-train Hindi/Telugu recordings (CC-BY-4.0) as real-speaker prompts: 98,419 steps on one Kaggle T4 GPU (about 9.8 h); the released weights are the step-98,000 checkpoint, picked by validation loss. FLEURS dev was used only for evaluation and the test split was not used.

Limitations and responsible use

  • Naturalness is the main gap: prosody is flatter than large models. Expect clear, slightly monotone speech.
  • Very short inputs (one or two words) can run on or come out oddly; the API retakes automatically, but short phrases are the least reliable.
  • Code-mixed text (English words inside Hindi/Telugu) works only as well as the character table allows; it was not specifically evaluated.
  • Cloning quality depends on a clean 3–10 s clip. v2 copies the speaker much better than v1 but is still below larger models, and its clones score lower on UTMOS than v1's (possibly by copying the reference's recording conditions). Telugu clones have more character errors than v1's.
  • Do not clone a voice without the speaker's consent, and label generated speech as synthetic.

Licence & attribution

  • AksharaVP weights and code: CC-BY-NC-4.0 — non-commercial; credit Sourojit Dhua (sourojitdotcom) and link this page.
  • Mimi decoder tensors (mimi.*): Kyutai Pocket TTS, unchanged, CC-BY-4.0; they keep that licence.
  • Flow-head code derives from MAR (MIT) via Kyutai Pocket TTS (MIT). Full list in NOTICE.
AksharaVP-17M by Sourojit Dhua (sourojitdotcom) — https://huggingface.co/sourojitdotcom/AksharaVP-17M
@misc{dhua2026aksharavp,
  title  = {AksharaVP-17M: Telugu, Hindi and Indian-English text to speech in 18.0M parameters},
  author = {Dhua, Sourojit},
  year   = {2026},
  url    = {https://huggingface.co/sourojitdotcom/AksharaVP-17M}
}
Downloads last month
49
Safetensors
Model size
18M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train sourojitdotcom/AksharaVP-17M

Space using sourojitdotcom/AksharaVP-17M 1

Evaluation results