NERDDISCO's picture
kale-tts-1
f21d6eb
|
Raw History Blame Contribute Delete
4.56 kB

recipe/render: Maya-1 training takes for each voice

Renders each designed voice's training data with Maya-1 on vLLM: one take per text, original 24 kHz audio, uncut, no processing. This is the path that rendered the release data (batch render, then ../filter/). The recommended streaming shape (render and every check in one warm container) is ../filter/stream_filter.py; as written it screens voice designs, not whole training sets (see ../filter/README.md).

Inputs: Project Gutenberg books as pg<id>.txt (ids listed in make_text.py; download them yourself), ban files (your bench and test lines, e.g. ../bench/script.jsonl), the voice designs in config/voices.json. Outputs: work/jobs-v7.jsonl (text pool), work/singles.jsonl, work/jobs-<voice>.jsonl; on the data volume /data/onevoice/<voice>/<sub>/<clip>.wav (float32 24 kHz) + <clip>.snac.npy (raw Maya codes, uint16 frames x 7); locally work/render-<group>/chunk-*.json (clip, eos, seconds, seed, raw_peak, text, path per take). work/ids_<voice>_all.json = {"rows": {clip: [Kokoro ids]}} (make_ids.py; sentences joined by the space id 16, en-GB for d03 / d13; --text-map also writes {text: ids} for ../teacher/make_rows.py).

python3 -I make_text.py <gutenberg dir> "<v7 description>" --ban ../bench/script.jsonl --ban my_test_lines.txt
python3 -I make_singles.py work/jobs-v7.jsonl work/singles.jsonl "<m3 description>"
python3 -I make_jobs.py f06 12028 9000 "<f06 description from config/voices.json>"
modal run render.py::main --jobs work/jobs-f06.jsonl --sub f06 --group f06 --per-call 10000 --seed0 8000000
python3 -I make_ids.py f06                      # token ids per take via ../../python/frontend_cli.mjs -> work/ids_f06_all.json

Per voice: n_mix, n_single, start, seed0 and the render group are in config/voices.json. The release rendered five voices (t071, s057b, w602, w165, x328) in one group td10 (their jobs files concatenated, seed0 9100000). KALE_TTS_ROOT sets the folder that holds work/ (default: the current folder).

Config: config/render.json (Maya-1 and SNAC revisions, vLLM settings, sampling: temperature 0.4, top-p 0.9, repetition penalty 1.1, min 28 / max min(2048, 220 + 60 x words) tokens, one seed per take = seed0 + row index; GPU cap; text mix 40/20/20/12/8 %), config/voices.json (the 10 release voices in model order with public names and Maya-1 descriptions, plus v7, whose renders are only teacher data), ../config/modal.json (app render_app, volume data_volume).

Files and sources (our research scripts, copied and trimmed):

  • render.py <- the 16 Nov 10-voice build's vrender.py (identical to the copy in the earlier two-voice build; v7 and m3 were rendered with the same prompt and sampling). Trimmed: app, volume and GPU cap read from config instead of environment variables; sampling numbers read from config (same values).
  • make_text.py <- the storyteller text-pool builder. Trimmed: ban sources are --ban files instead of fixed paths into our bench, test-set and level-guard files; output path and voice id are arguments; comments about our earlier voices removed.
  • make_singles.py <- the single-sentence pool builder. Trimmed: it also rebuilt each sentence's token ids through our earlier front end's character table and dropped sentences it couldn't map; now it only extracts sentences (ids come from frontend/ in the train step), so the pool can be slightly larger than ours.
  • make_ids.py: written for this recipe (same role as our build's id step, which used an earlier Python copy of the same lexicon + g2p): ids through the release front end. Spot check against the release build's id files: identical on most takes, a few differ by an id or two.
  • make_jobs.py <- the per-voice job builder (with the start-offset variant merged in). Trimmed: pool paths are arguments; copying token ids from the pool is optional (--ids-pool, --ids-singles, writes ids_<voice>_all.json).

Known gaps: the Gutenberg texts are not shipped. Our pool came from Gutenberg books only (no LJSpeech, although the guide mentions it). make_jobs.py seeds its sample from Python's per-process string hash of the voice id (as the release run did): set PYTHONHASHSEED for repeatable draws; the release's exact draws are only in its jobs files. m3's first batches (3.9 h, two-voice build) came from an earlier job builder on the same pool and the same render.py; that builder isn't included.