|
Download README.md from KaleComputer/kale-tts-1-tiny: direct link, hf CLI and curl.
- Browser
- Download file 24.6 kB
-
https://huggingface.co/KaleComputer/kale-tts-1-tiny/resolve/main/README.md
- Command line
-
hf download hf://KaleComputer/kale-tts-1-tiny/README.md
-
curl -L -o README.md https://huggingface.co/KaleComputer/kale-tts-1-tiny/resolve/main/README.md
24.6 kB
| # DRAFT, NOT PUBLISHED. Credit: based on Maya (Maya-1). | |
| license: apache-2.0 | |
| language: [en] | |
| pipeline_tag: text-to-speech | |
| library_name: onnx | |
| tags: [text-to-speech, tts, small, cpu, on-device, synthetic-voices, based-on-maya] | |
| base_model: KaleComputer/kale-tts-1-base | |
| base_model_relation: finetune | |
| datasets: [KaleComputer/kale-tts-1-data] | |
|  | |
| # kale-tts-1-tiny | |
| kale-tts-1-tiny is a small English text-to-speech model with 10 voices designed with [Maya-1](https://huggingface.co/maya-research/maya1), that runs on CPU or GPU on low-end devices. It stays as close as possible to Maya-1's melody, which already sounds great, while running on weak hardware. | |
| | measure | value | | |
| |---|---| | |
| | size | 79.9 MB on the CPU (int8, `kale-tts-1-tiny.onnx`); 183.2 MB for WebGPU in the browser (float32, `kale-tts-1-tiny.fp32.onnx`) | | |
| | parameters | 45.7M (16.8M of them the frozen Ettin word encoder) | | |
| | unclear words | +1.4 points vs Maya-1's own recordings (Maya-1: 5.82 %) | | |
| | melody gap | +7.48 points vs Maya-1's own recordings (Maya-1: 0) | | |
| | speed, Samsung Galaxy Tab S10 Lite CPU | 24x real time, first sentence after 0.16 s | | |
| | speed, Mac browser | 87x (WebGPU), 48x (WASM) | | |
| Unclear words: a speech recognizer listens word by word, and "unclear" is the share of words it isn't sure about; Maya-1's own recordings | |
| score 5.82 % on the same lines. Melody gap: how far the voice's pitch melody is from Maya-1's own recordings of the same lines, in | |
| percentile points; Maya-1 scores 0. Lower is better for both. | |
| Want something smaller and faster, with slightly less quality? Try [kale-tts-1-nano](https://huggingface.co/KaleComputer/kale-tts-1-nano). | |
| Trained on [kale-tts-1-data](https://huggingface.co/datasets/KaleComputer/kale-tts-1-data): 204.7 hours (99,783 takes) of the 10 voices' speech with its text, CC BY 4.0, open for anyone to train their own. | |
| ## Voices | |
| | name | voice | pronunciation | | |
| |---|---|---| | |
| | Kenzie | esports hype woman | US English | | |
| | Simone | deep and confident woman | US English | | |
| | Amy | crisp news-style woman | British English | | |
| | Frankie | hard-boiled detective's partner | US English | | |
| | Darlene | rough low-voiced old trucker woman | US English | | |
| | Owen | bright young host | US English | | |
| | Toby | mischievous prankster | US English | | |
| | Jack | crisp news-style man | British English | | |
| | Earl | African American jazz granddad | US English | | |
| | Magnus | deep dramatic trailer narrator | US English | | |
| Every voice was designed from a one-line text description; none is a real person. | |
| ## Listen | |
| [](https://huggingface.co/spaces/KaleComputer/kale-tts-1) | |
| Every voice, in your browser: [the kale-tts-1 demo](https://huggingface.co/spaces/KaleComputer/kale-tts-1). | |
| ## Why we made it | |
| We build a privacy-focused computer for kids. Our goal is to run all the AI locally on the device, so privacy is as high as it can be. | |
| Audio models were the quick first win, because they're easy to do with AI right now. | |
| We made our own voice for Kale (Hazel), and we're keeping that one. But we share the whole concept, so you can recreate any voice from [Maya-1](https://huggingface.co/maya-research/maya1). And here are 10 voices to try right away. | |
| ## How it works | |
| ### What it's made of | |
|  | |
| 45.7M parameters in one 79.9 MB file (the CPU file; 183.2 MB for WebGPU): the dictionary in front, the Ettin word reader (16.8M, frozen), the part that reads the text and plans timing and melody (24.8M: a 3.0M phoneme reader + the 21.8M planner of kale-tts-1-base, frozen), a sound generator (4.1M), and a voice pack per voice. | |
| ### How it was made | |
|  | |
| 1. **Maya-1** designs the 10 voices from a line of text and records their training audio: 204.7 hours. | |
| 2. **[kale-tts-1-base](https://huggingface.co/KaleComputer/kale-tts-1-base)**, our Kokoro-82M fine-tuned on our 10 voices (made with Maya-1), is the teacher: its timing-and-melody planner taught this model; it's too big and slow for the tablet. kale-tts-1-tiny is distilled from it; the teacher is its own model repo. | |
| 3. **kale-tts-1-tiny** keeps the timing-and-melody planner of kale-tts-1-base (21.8M, frozen) and puts a small phoneme reader (3.0M) and the frozen Ettin word reader (16.8M), which understands whole sentences, in front of it. The part that makes the sound (4.1M, a small harmonic-plus-noise generator) learns straight from Maya-1's audio. kale-tts-1-nano is a smaller student distilled from the same teacher. | |
| 4. In front of it all, **the dictionary** turns words into sounds; every word is checked. | |
| No flow matching and no diffusion anywhere: one pass per sentence. | |
| ### Turns words into sounds | |
|  | |
| Tech terms first ("Node.js" -> "Node J S"), then numbers, dates, money and units as words (dizione rules), then every word through a | |
| CMUdict-based dictionary, a hand-checked override list and a small guesser for unknown words. No espeak. Files: `frontend/`. | |
| ### A checked dictionary | |
| Small TTS models live or die by their front end: the wrong sounds make the best voice say the wrong word. We checked every word the front | |
| end can meet (334,000 words, American and British) and fixed 3,800+ readings in `frontend/words/override.public.tsv`. Example: "details" | |
| came out of the base dictionary as dih-TAILS (`dɪtˈeɪlz`); one line in the override list makes it DEE-tails (`dˈiːtˌeɪlz`). | |
| **Add a word:** one line in `frontend/words/override.public.tsv` (British: `frontend/words/en-gb/override.public.tsv`): the word in | |
| lower case, a tab, its IPA (stress mark right before the vowel). It wins over the dictionary and the guesser. Prompt for a coding agent: | |
| ``` | |
| In this repository, add the word "<word>" to frontend/words/override.public.tsv with the pronunciation <IPA> (one line: word, a tab, | |
| the IPA in the same style as the other lines). Check that the word is not a person's name and is not offensive; if it is, stop. Run the | |
| front end on a sentence with the word (python/kale_tts.py: QuickJSFrontEnd) and confirm the new IPA. Then render it with kale-tts-1-tiny and listen. | |
| ``` | |
| ### Reads the text, plans timing and melody | |
| A small phoneme reader (3.0M) and the frozen Ettin word reader (jhu-clsp/ettin-encoder-17m, MIT, 16.8M, inside the graph), which understands whole sentences, feed the teacher's own timing-and-melody planner (21.8M, kept frozen). The word inputs (`ettin_ids`, `ettin_mask`, `wavg`, `widx`) come from `frontend/ettin.js` + `ettin-tokenizer.json`. | |
| ### Makes the sound | |
| A small generator (4.1M): a harmonic source plus noise through a learned filter, then an inverse STFT to 24 kHz. No separate vocoder, no flow matching: one pass per sentence. At the end of a sentence a per-voice pitch floor keeps the last words from creaking. | |
| ### Voices | |
|  | |
| Each voice starts as a one-line description given to Maya-1. Maya-1 reads thousands of sentences in that voice; a filter keeps the takes | |
| that read back right, match the voice and end cleanly. Each voice ships as a style table and a 16-number sound row (`voices/`). | |
| ### Training data | |
|  | |
| 99,783 original Maya-1 takes, 204.7 hours, 10 voices, texts from public-domain books: [kale-tts-1-data](https://huggingface.co/datasets/KaleComputer/kale-tts-1-data) (CC BY 4.0). | |
| About 25 hours of clean takes per voice is where clarity levels off; 5 hours already sounds right but is less clear. | |
| ### How we picked the base model | |
|  | |
| We tried open-weight models on the same 192 test lines against Maya-1's own recordings. Fine-tuned on Maya-1's audio, Kokoro-82M was the | |
| clearest and the closest to Maya-1's melody, but it ran at about real time on the tablet, far too slow, so it became the teacher: | |
| [kale-tts-1-base](https://huggingface.co/KaleComputer/kale-tts-1-base). The published kale-tts-1-base keeps that fine-tune's text side and planner with stock Kokoro-82M's | |
| sound decoder; on the same bench it is less clear (+3.9 points) than tiny and nano. | |
| ### How we measure | |
|  | |
| 192 English lines (stories, explanations, numbers, questions), all 10 voices, sentence by sentence as an app speaks; each voice against | |
| Maya-1's own recordings of the same lines. Unclear words: word spans from CTC forced alignment (wav2vec2-large-960h-lv60-self); kale-tts-1-tiny | |
| reads 7.23 % unclear, Maya-1's recordings 5.82 %. Per voice: `results/bench/bench-10voices.json`. | |
| Export check: every release file against a float32 build of the same weights on 200 sentences; the same length on each (int8: at most one | |
| sentence one frame off) and a log-mel distance of at most 0.20. tiny: the same length on all 200 sentences, log-mel distance 0.077; Whisper word errors 0.49 % on the 10-voice bench. | |
| ### Extend it | |
|  | |
| - **A word:** one line in the override list (above). | |
| - **A voice:** a one-line description, a few thousand Maya-1 takes (`recipe/render/`, `recipe/filter/`), then teacher, heads and a new sound row (`recipe/train/`, `recipe/export/`). | |
| - **A language:** its own dictionary, guesser and number rules, takes in the same voices, then the text reader and planner fine-tuned on both languages. | |
| ## Where it runs | |
| | where | runs | speed and notes | | |
| |---|---|---| | |
| | Browser, WebGPU | yes | 87x; downloads the 183.2 MB float32 file (onnxruntime-web 1.31 or newer) | | |
| | Browser, WASM | yes | 48x with 4 threads (the 79.9 MB CPU file; threads need COOP/COEP headers) | | |
| | Android (onnxruntime) | yes | 24x on the Galaxy Tab S10 Lite CPU, first sentence after 0.16 s | | |
| | iOS | not tested | onnxruntime runs there; we have no device numbers | | |
| | Python | yes | 114x (`python/kale_tts.py`; needs `pip install quickjs`) | | |
| | Node.js | yes | 117x (`js/kale-tts.mjs`) | | |
| | HTTP server (`server/server.mjs`) | yes | 82x for whole lines, text in, WAV out (one process; WAV or streamed) | | |
| | transformers.js | yes | in Node, with our text-prep helper for the inputs (see Use it) | | |
| | sherpa-onnx, kokoro-onnx, plain C / Go / Rust / Swift | no | these pass phoneme ids only; tiny also needs word inputs from its Ettin encoder, which our JavaScript and Python code builds from the text | | |
| Speed = times faster than real time, median over 70 test lines in Kenzie's voice (18 Nov): the model one sentence at a time; the server row whole lines, text in, WAV out. Machines: Apple MacBook Pro, M5 Max (18 CPU cores, 128 GB RAM) for the browser, Python, Node and server rows; | |
| Samsung Galaxy Tab S10 Lite (Exynos 1380, 8 CPU cores: 4 × Cortex-A78 + 4 × Cortex-A55) for Android (10x on 1 thread). Speed was measured on a 73.9 MB variant of this graph (its phoneme encoder in fp16); the 79.9 MB file keeps that encoder in float32 and may be slightly slower. | |
| ## Use it | |
| **transformers.js** | |
| ```bash | |
| npm i @huggingface/transformers | |
| npm pkg set type=module | |
| # our text-prep helper: the js/ and frontend/ folders of this repo | |
| pip install -U huggingface_hub | |
| hf download KaleComputer/kale-tts-1-tiny --local-dir . --include "js/*" --include "frontend/*" | |
| ``` | |
| Save as `tts.mjs`, then run `node tts.mjs`: | |
| ```js | |
| import { writeFileSync } from "node:fs"; | |
| import { StyleTextToSpeech2Model, Tensor } from "@huggingface/transformers"; | |
| import { KaleTTS, toWav } from "./js/kale-tts.mjs"; | |
| const model = await StyleTextToSpeech2Model.from_pretrained("KaleComputer/kale-tts-1-tiny", { subfolder: "", dtype: "fp32" }); | |
| // text -> the model's inputs, sentence by sentence | |
| const prep = await KaleTTS.prepare("https://huggingface.co/KaleComputer/kale-tts-1-tiny/resolve/main/"); | |
| const parts = []; | |
| for await (const { inputs } of prep.inputs("Hello there! It is 9:30 on March 3.", { voice: "kenzie" })) { | |
| const feeds = {}; | |
| for (const [name, t] of Object.entries(inputs)) feeds[name] = new Tensor(t.type, t.data, t.dims); | |
| // waveform.data: Float32Array, 24 kHz | |
| const { waveform } = await model(feeds); | |
| parts.push(waveform.data); | |
| } | |
| const audio = new Float32Array(parts.reduce((n, p) => n + p.length, 0)); | |
| let at = 0; | |
| for (const p of parts) { audio.set(p, at); at += p.length; } | |
| writeFileSync("hello.wav", Buffer.from(toWav(audio, 24000))); | |
| ``` | |
| `KaleTTS.prepare()` is our small text-prep helper: it turns text into this model's inputs (numbers and dates as words, the checked | |
| dictionary, the voice's style and voice inputs, and for tiny the Ettin word inputs). transformers.js has no tokenizer for these inputs, | |
| so the helper builds them and transformers.js runs the model. | |
| On WebGPU (transformers.js 4.3 or newer), load the float32 file and keep the six small LSTM layers on the CPU for full speed, the same | |
| settings our runtime uses: | |
| ```js | |
| const LSTM_NODES = ["lstms.0", "lstms.2", "lstms.4", "lstm", "shared", "lstm_1"].flatMap((n) => [`/g/${n}/LSTM`, `/${n}/LSTM`]); | |
| const model = await StyleTextToSpeech2Model.from_pretrained("KaleComputer/kale-tts-1-tiny", { | |
| subfolder: "", | |
| dtype: "fp32", | |
| model_file_name: "model.fp32", | |
| device: "webgpu", | |
| session_options: { executionProviders: [{ name: "webgpu", forceCpuNodeNames: LSTM_NODES }] }, | |
| }); | |
| ``` | |
| **This is temporary.** | |
| - CPU and WASM: Transformers.js is 11–19 % slower than our own runtime today, because it pins an older onnxruntime. The gap closes by | |
| itself when [transformers.js PR #1879](https://github.com/huggingface/transformers.js/pull/1879) (onnxruntime 1.31.1) is merged; | |
| nothing changes in your code. | |
| - WebGPU: the two extra lines (`LSTM_NODES` and `session_options`) keep six small LSTM layers on the CPU. They stay until an | |
| onnxruntime release with the LSTM fix ships in Transformers.js; then you can remove them. | |
| **Without transformers.js: our runtime** | |
| Download the model with our runtime. It turns any text into speech, numbers, dates and tech terms included: | |
| ```bash | |
| # the hf command comes with huggingface_hub | |
| pip install -U huggingface_hub | |
| # only the files the runtime needs | |
| hf download KaleComputer/kale-tts-1-tiny --local-dir kale-tts-1-tiny \ | |
| --include "kale-tts-1-tiny.onnx" --include "*tokenizer.json" --include "voices.json" --include "config.json" \ | |
| --include "voices/*" --include "js/*" --include "frontend/*" --include "python/*" --include "server/*" | |
| cd kale-tts-1-tiny | |
| ``` | |
| **Node.js** | |
| ```bash | |
| npm i onnxruntime-node | |
| npm pkg set type=module | |
| ``` | |
| Save as `hello.mjs`, then run `node hello.mjs`: | |
| ```js | |
| import { writeFileSync } from "node:fs"; | |
| import { KaleTTS, toWav } from "./js/kale-tts.mjs"; | |
| const tts = await KaleTTS.load("./"); | |
| // audio: Float32Array, 24 kHz | |
| const { audio } = await tts.speak("Hello there! It is 9:30 on March 3.", { voice: "kenzie" }); | |
| writeFileSync("hello.wav", Buffer.from(toWav(audio, 24000))); | |
| // for long text, sentence by sentence: | |
| // for await (const { audio } of tts.stream(longText, { voice: "amy" })) play(audio); | |
| ``` | |
| **Python** | |
| ```bash | |
| pip install onnxruntime numpy quickjs soundfile | |
| ``` | |
| Save as `hello.py` in the same folder, then run `python hello.py`: | |
| ```python | |
| import sys | |
| sys.path.insert(0, "python") | |
| import soundfile as sf | |
| from kale_tts import KaleTTS | |
| tts = KaleTTS(".") | |
| # float32 audio, 24 kHz | |
| audio = tts.speak("Hello there!", voice="amy") | |
| sf.write("hello.wav", audio, 24000) | |
| ``` | |
| **HTTP server** | |
| Run it locally and call it from any language or app over HTTP (an OpenAI-shaped `POST /v1/audio/speech`, WAV or streamed PCM): | |
| ```bash | |
| npm i onnxruntime-node | |
| # without a token it only listens on 127.0.0.1 | |
| TTS_API_TOKEN=choose-a-secret-of-20-characters PORT=8000 node server/server.mjs | |
| ``` | |
| ```bash | |
| curl -s http://127.0.0.1:8000/v1/audio/speech \ | |
| -H "Authorization: Bearer choose-a-secret-of-20-characters" \ | |
| -d '{"model": "kale-tts-1-tiny", "input": "Hello there!", "voice": "kenzie"}' \ | |
| -o hello.wav | |
| ``` | |
| You pass the voice by its name. | |
| ## How we made it | |
| Research agents ran the experiments, on Claude Opus 5.5 and Claude Fable 5.2. We told them what to find out, and an automation checked | |
| every 60 minutes whether any agent was stuck and got it moving again. | |
| What we tried, one line each: | |
| **Teacher and data** | |
| - Maya-1 voice designer + SNAC codec, served on vLLM: worked. It made all the training audio, about 12-14x faster than plain generation. | |
| - Maya-1 itself on the device, with speculative decoding: didn't work, about 0.1x real time. | |
| - Kokoro-82M fine-tuned StyleTTS2-style (kikiri-tts) as the teacher: worked. The clearest in our model comparison (before its decoder swap), but about 1x real time on the tablet. | |
| - Training on whole multi-sentence passages, speaking sentence by sentence: worked, the main melody lever. | |
| - More data (a 114 h multi-voice pretrain, more hours per voice): didn't help much; about 25 h per voice is enough. | |
| - Speed multipliers and stretched durations: didn't work, less clear. Pace now comes from per-voice duration heads. | |
| **Reading the text, planning, the dictionary** | |
| - Our own ~8 M model from scratch: misread too many words. Its shape became nano's student. | |
| - A small student that reads the text and plans timing, pitch and loudness, distilled from the teacher, plus a sound generator trained on real audio: worked, that's kale-tts-1-nano. | |
| - Ettin-encoder-17m word features instead of ALBERT: worked on tiny (about 1.6x faster on the tablet, same clarity); on nano bigger, not faster. | |
| - Fewer ALBERT passes, dynamic int8 ALBERT: didn't work, broke the melody or failed to load on Android. | |
| - Pitch fixes (pitch LoRA, contour discriminator, flow-matching pitch sampler, prosody label heads, a GRPO duration reward): no melody gain; GRPO learned to talk faster. | |
| - misaki dictionaries: best melody, but ruled out (they derive from GPL espeak output). CMUdict + our own guesser recovers about half of it. | |
| - A British front end (CMUdict rules + Britfone): only works with British-trained weights, so it ships off. | |
| **Making the sound** | |
| - Matcha-TTS + Vocos: larger, slower, dull above about 5 kHz. | |
| - Other vocoders and codec adapters (WaveNeXt, BigVGAN, codec adapters): no gain on the device, much slower, or worse sound. | |
| - DiT flow-matching decoder, REPA, one-step decoders, an adversarial mel post-filter: too slow on the tablet or no gain by ear. | |
| - A small autoregressive codec-token TTS: about 1.3-1.6x real time, too slow. | |
| - A source-filter decoder (sine bank + shaped noise + one inverse STFT, no separate vocoder): worked, it ships in both models. | |
| - A per-voice pitch floor at sentence ends: worked, it removed most end-of-sentence creak. | |
| **Export** | |
| - ONNX with int8 weights where safe, float32 timing and phase paths, the LSTMs on the CPU in browsers: worked. int8 on the timing path changed | |
| durations; CoreML and GPU serving didn't help. | |
| **Earlier directions: voices and data** | |
| - VoxCPM2 and other voice-design models (Qwen3-TTS VoiceDesign, MOSS-VoiceGenerator, Ming-TTS, Parler-TTS) as the voice maker: dropped. Their takes sounded flatter, and the Maya-1 designs won the listening rounds. | |
| - Moods and sound tags from Maya-1 (calm, whisper, laugh, sigh) as extra speaker ids: worked at no speed cost, but Maya-1 often skipped the tag sounds. kale-tts-1 has one style per voice and no tag tokens. | |
| **Earlier directions: Matcha-TTS fine-tuning** | |
| - Our own Matcha-TTS base trained from scratch on LJSpeech: worked, as good as the original. | |
| - Per-voice adapters: LoRA rank 32 came close to a full fine-tune, rank 8 was worse; a decoder-only fine-tune was far less clear (13.3 % vs 4.8 % unclear words). | |
| - Sampler settings (2-10 decoder steps, solvers, noise, guidance, checkpoint averaging): no gain; 2 steps sounded like 3. | |
| **Earlier directions: other models as the on-device voice** | |
| - Kokoro-82M on the tablet, full and pruned to 37-44 M: too slow (about 1x and 6x real time), so it became the teacher only. | |
| - Piper (VITS) medium fine-tunes: as fast as Matcha but scored lower; the high size was too slow. | |
| - Supertonic 3: 2.4x real time on the tablet, and its OpenRAIL-M licence also covers models trained on its output. | |
| - Kitten TTS nano with a searched style: no closer to the target voice than its preset, 2.5x on the tablet. | |
| - Zonos 3 Mini: below real time on the tablet and a 1.1 GB download. | |
| - Voice cloning (a 12 M ZipVoice from scratch, Matcha-P with a voice prompt, the 123 M ZipVoice): dropped for a fixed voice set; ZipVoice cloned well but was 15-25x too slow on the tablet. | |
| - An MB-iSTFT-VITS-style end-to-end model (8.2 M, 4.7 M): very fast, but quality stalled below the bar. | |
| - Mach-Boom (Phonon 31M): fast, but no published audio encoder, so it can't learn a new voice. | |
| - Not tested because their licences don't allow commercial use: F5-TTS, XTTS-v2, Fish Speech, Spark-TTS, MaskGCT. | |
| **Earlier directions: prosody, speed and serving** | |
| - A sampled prosody plan (flow-matching durations, pitch, loudness): closest pauses, but less clear (6.7-7.8 % unclear words). | |
| - A sentence-level delivery latent with context from the previous sentence: no melody gain. | |
| - A distilled 5-6 M Vocos vocoder: about 2x faster, but below the full vocoder. | |
| - The tablet GPU (our own WebGPU kernels) and XNNPACK: both slower than plain onnxruntime on 4 CPU cores. | |
| - Streaming part of a sentence: left seams on our decoder and hurt the melody; whole sentences are already fast. | |
| - vLLM / SGLang serving and CoreML: don't fit a one-pass model, or 2-4x slower than the CPU. | |
| **Earlier directions: yardsticks and other languages** | |
| - kitten-tts-2 as a yardstick: kitten mini matched our small model on clarity but was 20x slower on the tablet. | |
| - A German teacher scout (Qwen3-TTS, Chatterbox, VoxCPM2, CosyVoice3, MOSS-TTS): Qwen3-TTS read best; no German model trained yet. | |
| ## Make it faster or better | |
| - **WebGPU:** keep the six small LSTM layers on the CPU (`forceCpuNodeNames`; our JS API does it by default): about 2.2x faster on onnxruntime-web 1.27 and newer (29x -> 63x through transformers.js, 18 Nov). | |
| - **Pick the file per backend:** the int8 `kale-tts-1-tiny.onnx` on CPU and WASM, `kale-tts-1-tiny.fp32.onnx` on WebGPU. | |
| - **Threads:** 4 threads on a phone or tablet CPU; on a server 1 thread per process and one process per core. | |
| - **Stream sentence by sentence:** the first sentence plays while the rest renders. | |
| - **A smaller text reader:** most of tiny's extra time is the Ettin word reader; nano's own small reader is about 2x faster, with a melody further from Maya-1's. | |
| - **Train it better:** more clean recordings per voice (about 25 h each is where clarity levels off) and a cleaner teacher help more than a bigger model. | |
| ## Limits | |
| - Kenzie's melody is the furthest from her Maya-1 recordings (melody gap 9.19, against 7.48 on average). | |
| - The two British voices are the least clear: Amy 9.0 %, Jack 9.1 % unclear (their Maya-1 recordings: 5.8 / 6.0 %). | |
| - English only. Rare names and heavy numbers can be misread; the guesser for unknown words is right on about two thirds of them. | |
| - Needs our text-prep code (JavaScript or Python) for the Ettin word inputs; tools that pass phoneme ids only can run nano instead. | |
| - Not for impersonating real people or passing the output off as human speech. | |
| ## Files | |
| `kale-tts-1-tiny.onnx` (79.90 MB, int8 / fp16 weights where safe, phoneme encoder and durations float32), `kale-tts-1-tiny.fp32.onnx` (183.25 MB, float32, for WebGPU), `voices/` + `voices.json`, `tokenizer.json`, `ettin-tokenizer.json` (MIT). Inputs: `input_ids`, `style`, `speed`, `ettin_ids`, `ettin_mask`, `wavg`, `widx`, then the voice inputs listed in `voices.json`. PyTorch weights in `pytorch/` (see `pytorch/README.md`). The training recipe is in `recipe/`. | |
| ## Licence and credits | |
| Weights and code Apache-2.0; dataset CC BY 4.0. | |
| AI-generated speech: everything this model says is synthetic speech made by an AI model, in voices that belong to no real person. Tell | |
| listeners it is AI-generated, never present it as a human recording, and never use it to imitate a real person. | |
| Credits: based on Maya-1: all training audio is original speech from Maya-1 (Maya Research, Apache-2.0), decoded with SNAC (MIT). The | |
| teacher is Kokoro-82M (hexgrad, Apache-2.0), fine-tuned with StyleTTS2 (MIT) and kikiri-tts (Apache-2.0) code; kale-tts-1-tiny keeps | |
| its timing-and-melody planner and contains the Ettin 17M encoder (JHU CLSP, MIT). Pronunciations: CMUdict (BSD-2-Clause), Britfone | |
| (MIT); the number, date and unit rules come from dizione and the homograph tables from tiny-voca, both by n4ze3m (Apache-2.0), | |
| the tables trained on Google Research's Wikipedia Homograph Data (Apache-2.0). Training texts: public-domain books made available by Project Gutenberg. Dataset: kale-tts-1-data (CC BY 4.0). | |