Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,186 @@
+---
+license: apache-2.0
+tags:
+- executorch
+- xnnpack
+- pte
+- on-device
+- text-to-speech
+base_model:
+- hexgrad/Kokoro-82M
+---
+# Kokoro-82M — ExecuTorch (text to speech, 54 voices)
+
+Phonemes in, a waveform out, in one `.pte` with two methods. StyleTTS2-shaped: a
+12-layer phoneme BERT and a duration predictor decide how long each sound lasts, and
+an iSTFTNet vocoder turns the stretched features into 24 kHz audio. A single
+256-dimensional style vector picks the voice and colours the prosody.
+
+```
+predict  input_ids (1, N) int64, ref_s (1, 256) fp32, speed (1) fp32
+             ->  d (1, 640, N), t_en (1, 512, N), duration (N) int64
+vocode   d, t_en, aln (1, N, F) fp32, ref_s
+             ->  waveform (F * 600) fp32 @ 24 kHz
+```
+
+- **File**: `kokoro_82m_xnnpack_fp32.pte` — **325.4 MB**, two methods
+- **Source**: [hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) — 81.8M parameters
+- **License**: apache-2.0
+- **Voices**: 54, shipped separately as `voices/*.pt` in the source repo
+
+Both axes — phonemes and frames — are **dynamic**, and both are **exact**. Nothing is
+padded, nothing is stretched, and there is no ladder of fixed-size methods. That took
+some doing; see below.
+
+## Running it
+
+**1. Text to phonemes, outside the graph.** Kokoro's vocabulary is IPA, and getting
+there is [`misaki`](https://github.com/hexgrad/misaki) (or espeak-ng) — a lexicon and a
+G2P model, not arithmetic. Same line this shelf takes with E5's prefix and Whisper's
+mel: the recipe is here, the graph takes what the recipe produces.
+
+```python
+from misaki import en
+ps, _ = en.G2P(trf=False, british=False)(text)
+ids = [vocab[c] for c in ps if c in vocab]          # config.json's 178-entry vocab
+input_ids = torch.LongTensor([[0, *ids, 0]])        # wrapped in the boundary token
+```
+
+**2. Pick the style row by phoneme count.** The voice pack is 510 rows and the row is
+`pack[len(ps) - 1]` — indexed by the length of the phoneme **string**, which is what
+upstream's own pipeline does, and not by the number of ids, which is smaller whenever
+the string holds a character the vocabulary does not have.
+
+**3. Run `predict`,** which returns the features and one duration per phoneme.
+
+**4. Build the alignment matrix yourself, at exactly `sum(duration)` frames.**
+Upstream builds it with `repeat_interleave`, whose width is the sum of the durations —
+the model's own output deciding the shape of its next input, which no single graph can
+express. That is the only reason this is two methods rather than one. The same matrix
+is comparisons only:
+
+```python
+ends = torch.cumsum(duration, 0)
+starts = ends - duration
+frame = torch.arange(int(ends[-1]))                 # exactly sum(duration)
+aln = ((frame[None, :] >= starts[:, None]) &
+       (frame[None, :] < ends[:, None])).float()[None]
+```
+
+**Give it exactly `sum(duration)` frames.** Not more — see the next section.
+
+**5. Run `vocode`.** Out comes `600 * F` samples at 24 kHz. `speed` above 1 speaks
+faster; it divides the durations before they are rounded.
+
+## Do not pad either axis
+
+Both axes are dynamic, so there is no window to pad into — but it is worth saying why
+the file is built that way, because the obvious fixed-window design does not work here
+and the damage does not show up in a transcript.
+
+| axis | what forbids padding | measured |
+|---|---|---|
+| phonemes | five bidirectional LSTMs — state flows in from the padding | speaking rate moves up to **19%** |
+| frames | a bidirectional LSTM **and** `AdaIN1d`, which is `InstanceNorm` over time | log-mel **0.18–0.86** against a 0.04 noise floor |
+
+The frame axis is the surprising one. `AdaIN1d` normalises over **time**, so one extra
+frame changes the statistics the entire signal is divided by. Padding to the next
+16-frame rung, appending 256 frames, and padding out to 1024 all land far outside what
+the model does to itself, and it is not a level change — taking out one global gain
+factor leaves the distance where it was.
+
+Padding with spaces rather than zeros roughly halves the damage on the phoneme axis,
+and a recogniser transcribes **every** padded arm correctly. That is exactly why the
+gate here is not a recogniser alone.
+
+## The LSTMs are rolled, not unrolled
+
+`nn.LSTM` will not export with a dynamic sequence axis: `torch.export` pins it to
+whatever it was traced at. The reason is that `to_edge` **unrolls** the recurrence —
+this model's `predict` graph is 1238 ATen nodes at any length, and
+`1651 + 108 per phoneme` in edge dialect.
+
+That makes a ladder of fixed-length methods look like the only option, and then makes
+the ladder impossible. The XNNPACK partitioner is superlinear in node count and cuts an
+unrolled LSTM into hundreds of tiny delegates — 383 partitions at 32 phonemes — so
+lowering one method costs:
+
+| phonemes | edge nodes | lowering |
+|---|---|---|
+| 8 | 1651 | 33 s |
+| 16 | 2515 | 63 s |
+| 32 | 4243 | 153 s |
+| 128 | 14611 | ~19 min, extrapolated |
+
+A rung per phoneme count from 8 to 128 is upwards of **16 hours**, and the frame axis
+would need its own ladder on top of that.
+
+A `scan` higher-order op keeps the loop rolled. ExecuTorch lowers it, the runtime runs
+it, and the sequence axis stays dynamic. On this model's own LSTM shape — 640 in, 256
+hidden each way, 128 steps:
+
+| | build | edge nodes | delegates | 128 steps |
+|---|---|---|---|---|
+| `nn.LSTM`, one fixed length | 59.2 s | 3366 | 131 | 5.72 ms |
+| rolled `scan`, any length | **3.1 s** | **49** | 3 | 16.65 ms |
+
+Three times the runtime for one LSTM, against a build that finishes and a file that
+takes any length. Kokoro has six of them. The whole file now builds in about two
+minutes.
+
+## Verification
+
+Five English sentences through `misaki`, each one synthesised by the `.pte` and by the
+unmodified upstream model in eager. Three gates, because no single one works here:
+
+**Durations must match exactly** — they are integers, they decide the rhythm, and they
+come out of the half of the model that has no noise in it. **5 of 5 exact.**
+
+**Waveform correlation is not usable.** The vocoder's excitation carries Gaussian noise
+and a random initial phase, so the eager model does not reproduce itself: two runs of
+the same input correlate 0.9948. A correlation gate here measures the noise.
+
+**Log-mel distance against the model's own floor** is the gate that works. Measure the
+distance between two eager runs, then between eager and the `.pte`, and ask whether the
+second is the first:
+
+| | log-mel vs eager | eager's own floor | ratio |
+|---|---|---|---|
+| worst of five | 0.0427 | 0.0412 | **1.04x** |
+| best of five | 0.0442 | 0.0449 | 0.98x |
+
+Below and above 1.0 across the five, which is what "indistinguishable from running it
+again" looks like. For scale, one extra frame of padding shows up at 4.4x.
+
+**Transcripts**, through Qwen3-ASR: **CER 0.0000** on all five. Synthetic speech read
+by a recogniser is a control, not a benchmark — it says the file still says the words.
+
+## Speed
+
+3.25 s of audio (50 phonemes, 130 frames) on an M-series laptop, host CPU:
+
+| | ms |
+|---|---|
+| `predict` | 56.6 |
+| `vocode` | 443.7 |
+| **total** | **500.3** — 6.5x faster than real time |
+| the same utterance in eager PyTorch | 227.5 |
+
+Measured on a machine that was busy, so read these as a floor rather than a number to
+quote. Eager is faster here and that is expected: it reaches Accelerate's LSTM and
+convolution kernels, while about half the graph runs on portable kernels
+(`predict` 49.6% of ops delegated, `vocode` 54.8%). No device numbers yet.
+
+## What is not in this file
+
+**No fp16 or int8 build.** The int8 decision is measured, not assumed: dynamic int8
+quantises `nn.Linear` only, and Kokoro is **66.9% Conv1d** with just **15.9%** of its
+weights in Linear layers, so it would touch a sixth of the file. Static int8 would
+reach the convolutions, but a vocoder's quality under quantisation has to be measured
+per task rather than declared, and that has not been done here.
+
+**The vocoder is partitioned without XNNPACK's `PermuteConfig`.** A permute inside the
+decoder feeds two consumers outside its partition, and the delegate's output list then
+carries that one node twice, which XNNPACK rejects with "Output node ... is already in
+the inputs ... pass through arguments". It is the partition boundary that is wrong, not
+the graph: the same graph lowers the moment permutes are not eligible to be one.