NERDDISCO's picture
kale-tts-1
f21d6eb
|
Raw History Blame Contribute Delete
4.56 kB
<!-- SPDX-License-Identifier: Apache-2.0 | Copyright 2026 Kale -->
# recipe/render: Maya-1 training takes for each voice
Renders each designed voice's training data with Maya-1 on vLLM: one take per text, original 24 kHz audio, uncut, no processing.
This is the path that rendered the release data (batch render, then `../filter/`). The recommended streaming shape (render and every
check in one warm container) is `../filter/stream_filter.py`; as written it screens voice designs, not whole training sets (see
`../filter/README.md`).
Inputs: Project Gutenberg books as `pg<id>.txt` (ids listed in `make_text.py`; download them yourself), ban files (your bench and test
lines, e.g. `../bench/script.jsonl`), the voice designs in `config/voices.json`.
Outputs: `work/jobs-v7.jsonl` (text pool), `work/singles.jsonl`, `work/jobs-<voice>.jsonl`; on the data volume
`/data/onevoice/<voice>/<sub>/<clip>.wav` (float32 24 kHz) + `<clip>.snac.npy` (raw Maya codes, uint16 frames x 7); locally
`work/render-<group>/chunk-*.json` (clip, eos, seconds, seed, raw_peak, text, path per take).
`work/ids_<voice>_all.json` = `{"rows": {clip: [Kokoro ids]}}` (`make_ids.py`; sentences joined by the space id 16, en-GB for d03 / d13;
`--text-map` also writes `{text: ids}` for `../teacher/make_rows.py`).
```
python3 -I make_text.py <gutenberg dir> "<v7 description>" --ban ../bench/script.jsonl --ban my_test_lines.txt
python3 -I make_singles.py work/jobs-v7.jsonl work/singles.jsonl "<m3 description>"
python3 -I make_jobs.py f06 12028 9000 "<f06 description from config/voices.json>"
modal run render.py::main --jobs work/jobs-f06.jsonl --sub f06 --group f06 --per-call 10000 --seed0 8000000
python3 -I make_ids.py f06 # token ids per take via ../../python/frontend_cli.mjs -> work/ids_f06_all.json
```
Per voice: `n_mix`, `n_single`, `start`, `seed0` and the render group are in `config/voices.json`. The release rendered five voices
(t071, s057b, w602, w165, x328) in one group `td10` (their jobs files concatenated, seed0 9100000). `KALE_TTS_ROOT` sets the folder
that holds `work/` (default: the current folder).
Config: `config/render.json` (Maya-1 and SNAC revisions, vLLM settings, sampling: temperature 0.4, top-p 0.9, repetition penalty 1.1,
min 28 / max min(2048, 220 + 60 x words) tokens, one seed per take = seed0 + row index; GPU cap; text mix 40/20/20/12/8 %),
`config/voices.json` (the 10 release voices in model order with public names and Maya-1 descriptions, plus v7, whose renders are only
teacher data), `../config/modal.json` (app `render_app`, volume `data_volume`).
Files and sources (our research scripts, copied and trimmed):
- `render.py` <- the 16 Nov 10-voice build's `vrender.py` (identical to the copy in the earlier two-voice build; v7 and m3 were
rendered with the same prompt and sampling). Trimmed: app, volume
and GPU cap read from config instead of environment variables; sampling numbers read from config (same values).
- `make_text.py` <- the storyteller text-pool builder. Trimmed: ban sources are `--ban` files instead of fixed paths into our bench,
test-set and level-guard files; output path and voice id are arguments; comments about our earlier voices removed.
- `make_singles.py` <- the single-sentence pool builder. Trimmed: it also rebuilt each sentence's token ids through our earlier front
end's character table and dropped sentences it couldn't map; now it only extracts sentences (ids come from `frontend/` in the train
step), so the pool can be slightly larger than ours.
- `make_ids.py`: written for this recipe (same role as our build's id step, which used an earlier Python copy of the same lexicon +
g2p): ids through the release front end. Spot check against the release build's id files: identical on most takes, a few differ
by an id or two.
- `make_jobs.py` <- the per-voice job builder (with the start-offset variant merged in). Trimmed: pool paths are arguments; copying
token ids from the pool is optional (`--ids-pool`, `--ids-singles`, writes `ids_<voice>_all.json`).
Known gaps: the Gutenberg texts are not shipped. Our pool came from Gutenberg books only (no LJSpeech, although the guide mentions it).
`make_jobs.py` seeds its sample from Python's per-process string hash of the voice id (as the release run did): set `PYTHONHASHSEED`
for repeatable draws; the release's exact draws are only in its jobs files. m3's first batches (3.9 h, two-voice build) came from an
earlier job builder on the same pool and the same `render.py`; that builder isn't included.