mlboydaisuke commited on
Commit
cac734c
·
verified ·
1 Parent(s): 61fdd88

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - text-to-speech
5
+ - tts
6
+ - litert
7
+ - tflite
8
+ - on-device
9
+ - matcha-tts
10
+ - hifigan
11
+ language:
12
+ - en
13
+ library_name: litert
14
+ pipeline_tag: text-to-speech
15
+ ---
16
+
17
+ # Matcha-TTS — LiteRT (on-device, FFT-free, GPU)
18
+
19
+ On-device English text-to-speech for Android via LiteRT `CompiledModel`. This is the
20
+ **FFT-free** TTS lane: [Matcha-TTS](https://github.com/shivammehta25/Matcha-TTS) pairs a
21
+ conditional flow-matching (CFM) acoustic model with a **HiFi-GAN time-domain vocoder**, so
22
+ there is **no FFT/iSTFT anywhere** in the synthesis path. 22.05 kHz, LJSpeech voice.
23
+
24
+ Converted from the official `matcha_ljspeech` + `hifigan_T2_v1` checkpoints with
25
+ [litert-torch](https://github.com/google-ai-edge/litert), re-authored to be ML-Drift-GPU-clean
26
+ (per-graph tflite-vs-torch corr **1.000000**; end-to-end waveform corr ≥0.99). fp16 weights.
27
+
28
+ ## Files
29
+
30
+ | File | Size | In → Out | Delegate (Pixel 8a) |
31
+ |---|---|---|---|
32
+ | `matcha_textenc_fp16.tflite` | 15 MB | emb[1,256,192] + mask[1,1,256] → mu[1,80,256], logw[1,1,256] | GPU |
33
+ | `matcha_decoder_fp16.tflite` | 23 MB | x,mu[1,80,512] + t_sin[1,160] + mask[1,1,512] → v[1,80,512] | CPU¹ |
34
+ | `matcha_vocoder_fp16.tflite` | 29 MB | mel[1,80,512] → wav[1,1,131072] | GPU |
35
+ | `dp_g2p_matcha_fp16.tflite` | 26 MB | text[1,96] (char ids) → logits[1,96,64] (IPA) | CPU |
36
+ | `emb.bin` | 0.1 MB | phoneme embedding table (178×192 f32, host lookup) | host |
37
+ | `g2p_dict.txt.gz` | 1.8 MB | 275k-entry espeak-IPA dictionary (primary G2P) | host |
38
+ | `config.json`, `g2p_meta.json` | — | symbols, shapes, mel stats, G2P tokenizer tables | host |
39
+
40
+ ¹ The CFM decoder runs on the **CompiledModel CPU** delegate. It converts GPU-clean and is
41
+ correct on CPU, but the Mali ML Drift GPU delegate **mis-fuses the decoder's transformer blocks
42
+ at large activation magnitude** (the same block is correct as a standalone GPU graph, corr 0.984,
43
+ but collapses to corr 0.006 fused — a graph-fusion bug, not a bad op). text encoder + vocoder run
44
+ on the GPU; the GPU vocoder dominates wall time so the pipeline stays **realtime (RTF ~0.8)**.
45
+
46
+ ## Pipeline (host orchestration)
47
+
48
+ ```
49
+ text --G2P(CPU dict+neural)--> phoneme ids
50
+ --host: embed + intersperse + pad--> text_encoder(GPU) -> mu, logw
51
+ --host: durations + length-regulator--> mu_y[1,80,T]
52
+ --host: Euler ODE loop (N steps)--> decoder(CPU) x N -> v
53
+ --host: denormalize--> vocoder(GPU) -> waveform
54
+ ```
55
+
56
+ Fixed shapes (256 phonemes, 512 mel frames ≈ 5.9 s); a runtime float mask makes padded positions
57
+ a no-op so one compiled graph handles any length.
58
+
59
+ ## G2P (espeak-free)
60
+
61
+ Matcha-LJSpeech is trained on espeak en-us IPA, but espeak is GPL. The clean replacement is a
62
+ 275k-entry espeak-IPA dictionary (from [OpenPhonemizer](https://github.com/NeuralVox/OpenPhonemizer),
63
+ Clear BSD) as primary + [DeepPhonemizer](https://github.com/as-ideas/DeepPhonemizer) (MIT) on
64
+ LiteRT CPU for out-of-dictionary words. Output IPA maps 1:1 onto the keithito 178-symbol set.
65
+
66
+ ## Android sample
67
+
68
+ Full sample app + conversion scripts: [LiteRT-Models / matcha](https://github.com/john-rocky/LiteRT-Models/tree/main/matcha).
69
+
70
+ ## License
71
+
72
+ Model: MIT (Matcha-TTS / HiFi-GAN). G2P dict: Clear BSD (OpenPhonemizer) + MIT (DeepPhonemizer).