Restore Kokoro-82M LiteRT (free-text 3-bundle + fixed-length sample build)

README.md

@@ -0,0 +1,105 @@
+---
+license: apache-2.0
+library_name: litert
+pipeline_tag: text-to-speech
+base_model: hexgrad/Kokoro-82M
+tags:
+  - litert
+  - tflite
+  - tts
+  - text-to-speech
+  - on-device
+  - kokoro
+  - free-text
+  - preview
+---
+
+# Kokoro-82M — LiteRT (free-text, bucketed)
+
+> ⚠️ **Labeled preview — FP32, CPU.** Arbitrary free text → speech (not a single baked
+> sentence). The neural graphs are LiteRT `.tflite`; two small steps run host-side (the
+> hn-NSF source STFT and the final iSTFT overlap-add). GPU + quantization are the next steps.
+
+A [LiteRT](https://ai.google.dev/edge/litert) (`.tflite`) conversion of
+[hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) (StyleTTS2 + ISTFTNet) for
+on-device **free-text** text-to-speech (arbitrary text in, 24 kHz speech out). **Audio fidelity
+≈ 0.9994 magnitude-spectrogram correlation to the PyTorch reference** (verified across multiple
+held-out sentences, not just the export sample).
+
+Kokoro has one data-dependent length (the duration→alignment expansion `L = sum(pred_dur)`),
+which `litert_torch` cannot keep dynamic (the LSTM sequence axis specializes). It is therefore
+split into **three fixed-bucket bundles** with host steps between them, so arbitrary text works
+by left-padding to the bucket and trimming the output (longer text is split into sentences
+host-side, each ≤ the bucket):
+
+```
+text --(G2P, host)--> phoneme ids
+1. kokoro_predictor.tflite : ids[1,128], ref_s[1,256], attn[1,128] -> duration, d, t_en
+   host: pred_dur = round(duration); alignment one-hot aln[1,128,512]; frame_mask[1,512]
+2. kokoro_prosody.tflite   : d, t_en, aln, ref_s, frame_mask       -> asr, F0, N
+   host: har = STFT(SineGen(f0_upsamp(F0)))     (the hn-NSF excitation)
+3. kokoro_vocoder.tflite   : asr, F0, N, har, ref_s, frame_mask    -> spec, phase
+   host: iSTFT overlap-add(spec, phase) -> 24 kHz waveform; trim to L*600
+```
+
+## Files
+
+| File | Precision | Size | Role |
+|---|---|---|---|
+| `kokoro_predictor.tflite` | fp32 | ~91 MB | PL-BERT + duration/text encoders (masked unrolled bi-LSTMs) |
+| `kokoro_prosody.tflite` | fp32 | ~37 MB | shared prosody LSTM + F0/N (masked) |
+| `kokoro_vocoder.tflite` | fp32 | ~236 MB | iSTFTNet decoder → magnitude/phase spectrogram |
+| `istft_Wr_f32.bin`, `istft_Wi_f32.bin` | fp32 | 880 B each | inverse-DFT bases for the host-side iSTFT |
+| `kokoro_82m_fixedlen_fp32.tflite` | fp32 | ~338 MB | single-graph **fixed-length demo build** used by the litert-samples `text_to_speech` sample (see below) |
+
+Token bucket `T = 128`, frame bucket `L = 512` (≈ 12.8 s of audio per chunk at 24 kHz).
+Bundles are **voice-independent** — the voice is the `ref_s` input (a `voices/*.pt` from the
+base repo, indexed by token-sequence length).
+
+## Fixed-length demo build (litert-samples)
+
+`kokoro_82m_fixedlen_fp32.tflite` is the earlier single-graph build (fixed-length input, baked
+demo sentence length, spec+phase out, host-side iSTFT) kept for the official
+[litert-samples `text_to_speech` sample](https://github.com/google-ai-edge/litert-samples/pull/159),
+which downloads it by this name. Measured on Pixel 8a (fp32, CPU, 4 threads): **RTF ≈ 1.8**
+(~6.6 s to synthesize 3.7 s of audio) — quantization is the path to realtime on-device.
+
+## Specs
+
+| | |
+|---|---|
+| Task | Text-to-speech (English), free text, 24 kHz mono |
+| Source | [hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) (StyleTTS2 + ISTFTNet) |
+| Fidelity | **magspec-corr 0.9994** vs PyTorch (waveform corr ≈ 0.98 — the bounded bucket pad-boundary effect; the spectrum is what's perceived) |
+| Runtime | CPU (LiteRT CompiledModel API) |
+
+## How it was converted
+- Stock official converter (`litert_torch`), **general path** — Kokoro is StyleTTS2/ISTFTNet, not a transformer LLM, so the Generative-API re-authoring path does not apply.
+- **Three fixed-bucket bundles** because the dynamic alignment length can't stay symbolic through the converter (the dynamic-LSTM wall). Every workaround is load-bearing and numerically faithful: the 6 bidirectional LSTMs are **unrolled as masked bi-LSTMs** that carry state through right-padding (a fused `nn.LSTM` leaks pad tokens into the backward pass and wrecks prosody); the 58 AdaIN InstanceNorms **normalize over real frames only** so bucket pad frames don't poison the statistics; the hn-NSF **source STFT runs host-side** (its atan2 phase flips at the F0→0 pad boundary on-device).
+- **iSTFT runs host-side** (vocoder emits spec+phase): the in-graph conv-transpose iSTFT hits a converter weight-dedup bug that fuses the cos/sin DFT bases. The host overlap-add (no learned weights) is numerically exact.
+
+## Training data
+Inherited from [hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M): a few hundred hours of **permissive / non-copyrighted audio** — public-domain audio, audio under permissive licenses (e.g. Koniwa `tnc` CC BY 3.0, SIWIS CC BY 4.0), and synthetic audio from large-provider TTS — paired with IPA phoneme labels. No proprietary or scraped personal recordings, no custom voice clones. This LiteRT artifact is a format conversion of the released weights and introduces no additional training data.
+
+## PII
+No personally identifiable information is included. Per the base model's disclosure the training audio is permissive / public-domain / synthetic rather than scraped personal recordings; to the best of our knowledge the released weights contain no PII, and the conversion adds none.
+
+## Front-end (free-text G2P)
+For arbitrary text with no dropped words (names, brands, numbers), pair with the neural grapheme-to-phoneme front-end: [litert-community/Kokoro-G2P-en-US](https://huggingface.co/litert-community/Kokoro-G2P-en-US).
+
+## Sample / code
+- Official Android sample (Interpreter API, fixed-length demo build): [google-ai-edge/litert-samples #159](https://github.com/google-ai-edge/litert-samples/pull/159)
+- Conversion / pipeline code: https://github.com/john-rocky/LiteRT-Models
+
+## Roadmap
+- **GPU**: the attention's fused-QKV >4-D layout + mask `EQUAL`/`SELECT` keep it on CPU; decomposing attention to ≤4-D would let it ride the GPU delegate. In-graph iSTFT on GPU additionally needs an FFT kernel in the LiteRT GPU delegate (ML Drift).
+- **Quantization** (int8/int4) is the obvious next step.
+
+## Status
+Labeled preview — the converted model is parity-verified. The official litert-samples
+`text_to_speech` sample (fixed-length demo build) is in review at
+[google-ai-edge/litert-samples #159](https://github.com/google-ai-edge/litert-samples/pull/159);
+a free-text sample (3-bundle + G2P front-end) is in progress.
+
+## License
+Apache-2.0, inherited from [hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M).

istft_Wi_f32.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72414803f6e0acad9f459cfacb4511c6f0bc5bbf239a6db8a7b78880d06b2cad
+size 880

istft_Wr_f32.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89ac7e7cd20ef63055b80cbf8864ff7c1ea214069e97d5039a4f5cc3df87bf44
+size 880

kokoro_82m_fixedlen_fp32.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5a066787eeffd73c0f10d5abf54813f3baa9020bbf1b32ef25473906e411d3c
+size 337596032

kokoro_predictor.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:696847964d2537fe8d72fd71b77c1fde9ba49f33cbc0ba09fa0329680cf42241
+size 90741756

kokoro_prosody.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60c3154a4afc4626db51d7f9382ec8b87cfe2ea7979a098c52739454fdd9fe43
+size 37393104

kokoro_vocoder.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:acf5043fea1fa4c522095e81f2f7c5892eb178fb1bd154ae34fb21cb60948dff
+size 236130056