Instructions to use mlboydaisuke/Matcha-TTS-LiteRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use mlboydaisuke/Matcha-TTS-LiteRT with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- text-to-speech
|
| 5 |
+
- tts
|
| 6 |
+
- litert
|
| 7 |
+
- tflite
|
| 8 |
+
- on-device
|
| 9 |
+
- matcha-tts
|
| 10 |
+
- hifigan
|
| 11 |
+
language:
|
| 12 |
+
- en
|
| 13 |
+
library_name: litert
|
| 14 |
+
pipeline_tag: text-to-speech
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# Matcha-TTS — LiteRT (on-device, FFT-free, GPU)
|
| 18 |
+
|
| 19 |
+
On-device English text-to-speech for Android via LiteRT `CompiledModel`. This is the
|
| 20 |
+
**FFT-free** TTS lane: [Matcha-TTS](https://github.com/shivammehta25/Matcha-TTS) pairs a
|
| 21 |
+
conditional flow-matching (CFM) acoustic model with a **HiFi-GAN time-domain vocoder**, so
|
| 22 |
+
there is **no FFT/iSTFT anywhere** in the synthesis path. 22.05 kHz, LJSpeech voice.
|
| 23 |
+
|
| 24 |
+
Converted from the official `matcha_ljspeech` + `hifigan_T2_v1` checkpoints with
|
| 25 |
+
[litert-torch](https://github.com/google-ai-edge/litert), re-authored to be ML-Drift-GPU-clean
|
| 26 |
+
(per-graph tflite-vs-torch corr **1.000000**; end-to-end waveform corr ≥0.99). fp16 weights.
|
| 27 |
+
|
| 28 |
+
## Files
|
| 29 |
+
|
| 30 |
+
| File | Size | In → Out | Delegate (Pixel 8a) |
|
| 31 |
+
|---|---|---|---|
|
| 32 |
+
| `matcha_textenc_fp16.tflite` | 15 MB | emb[1,256,192] + mask[1,1,256] → mu[1,80,256], logw[1,1,256] | GPU |
|
| 33 |
+
| `matcha_decoder_fp16.tflite` | 23 MB | x,mu[1,80,512] + t_sin[1,160] + mask[1,1,512] → v[1,80,512] | CPU¹ |
|
| 34 |
+
| `matcha_vocoder_fp16.tflite` | 29 MB | mel[1,80,512] → wav[1,1,131072] | GPU |
|
| 35 |
+
| `dp_g2p_matcha_fp16.tflite` | 26 MB | text[1,96] (char ids) → logits[1,96,64] (IPA) | CPU |
|
| 36 |
+
| `emb.bin` | 0.1 MB | phoneme embedding table (178×192 f32, host lookup) | host |
|
| 37 |
+
| `g2p_dict.txt.gz` | 1.8 MB | 275k-entry espeak-IPA dictionary (primary G2P) | host |
|
| 38 |
+
| `config.json`, `g2p_meta.json` | — | symbols, shapes, mel stats, G2P tokenizer tables | host |
|
| 39 |
+
|
| 40 |
+
¹ The CFM decoder runs on the **CompiledModel CPU** delegate. It converts GPU-clean and is
|
| 41 |
+
correct on CPU, but the Mali ML Drift GPU delegate **mis-fuses the decoder's transformer blocks
|
| 42 |
+
at large activation magnitude** (the same block is correct as a standalone GPU graph, corr 0.984,
|
| 43 |
+
but collapses to corr 0.006 fused — a graph-fusion bug, not a bad op). text encoder + vocoder run
|
| 44 |
+
on the GPU; the GPU vocoder dominates wall time so the pipeline stays **realtime (RTF ~0.8)**.
|
| 45 |
+
|
| 46 |
+
## Pipeline (host orchestration)
|
| 47 |
+
|
| 48 |
+
```
|
| 49 |
+
text --G2P(CPU dict+neural)--> phoneme ids
|
| 50 |
+
--host: embed + intersperse + pad--> text_encoder(GPU) -> mu, logw
|
| 51 |
+
--host: durations + length-regulator--> mu_y[1,80,T]
|
| 52 |
+
--host: Euler ODE loop (N steps)--> decoder(CPU) x N -> v
|
| 53 |
+
--host: denormalize--> vocoder(GPU) -> waveform
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
Fixed shapes (256 phonemes, 512 mel frames ≈ 5.9 s); a runtime float mask makes padded positions
|
| 57 |
+
a no-op so one compiled graph handles any length.
|
| 58 |
+
|
| 59 |
+
## G2P (espeak-free)
|
| 60 |
+
|
| 61 |
+
Matcha-LJSpeech is trained on espeak en-us IPA, but espeak is GPL. The clean replacement is a
|
| 62 |
+
275k-entry espeak-IPA dictionary (from [OpenPhonemizer](https://github.com/NeuralVox/OpenPhonemizer),
|
| 63 |
+
Clear BSD) as primary + [DeepPhonemizer](https://github.com/as-ideas/DeepPhonemizer) (MIT) on
|
| 64 |
+
LiteRT CPU for out-of-dictionary words. Output IPA maps 1:1 onto the keithito 178-symbol set.
|
| 65 |
+
|
| 66 |
+
## Android sample
|
| 67 |
+
|
| 68 |
+
Full sample app + conversion scripts: [LiteRT-Models / matcha](https://github.com/john-rocky/LiteRT-Models/tree/main/matcha).
|
| 69 |
+
|
| 70 |
+
## License
|
| 71 |
+
|
| 72 |
+
Model: MIT (Matcha-TTS / HiFi-GAN). G2P dict: Clear BSD (OpenPhonemizer) + MIT (DeepPhonemizer).
|