|
Download README.md from KaleComputer/kale-tts-1-base: direct link, hf CLI and curl.
- Browser
- Download file 5.75 kB
-
https://huggingface.co/KaleComputer/kale-tts-1-base/resolve/main/README.md
- Command line
-
hf download hf://KaleComputer/kale-tts-1-base/README.md
-
curl -L -o README.md https://huggingface.co/KaleComputer/kale-tts-1-base/resolve/main/README.md
5.75 kB
| # DRAFT, NOT PUBLISHED. Credit: based on Maya (Maya-1). | |
| license: apache-2.0 | |
| language: [en] | |
| pipeline_tag: text-to-speech | |
| library_name: kokoro | |
| tags: [text-to-speech, tts, kokoro, synthetic-voices, based-on-maya] | |
| base_model: hexgrad/Kokoro-82M | |
| base_model_relation: finetune | |
| datasets: [KaleComputer/kale-tts-1-data] | |
|  | |
| # kale-tts-1-base | |
| kale-tts-1-base is [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) with its text side and its timing-and-melody planner | |
| fine-tuned on our 10 voices designed with [Maya-1](https://huggingface.co/maya-research/maya1); its sound decoder is stock Kokoro-82M. | |
| It's the teacher of [kale-tts-1-tiny](https://huggingface.co/KaleComputer/kale-tts-1-tiny) and | |
| [kale-tts-1-nano](https://huggingface.co/KaleComputer/kale-tts-1-nano): both learned their timing and melody from it and have their own small | |
| sound generators, so they run on a low-end tablet. kale-tts-1-base itself is too big and slow for that; use it on a desktop or GPU, or to | |
| train and distill your own small voices. | |
| ## Distilled into | |
| | model | parameters | CPU file (int8) | WebGPU file (float32) | Galaxy Tab S10 Lite | unclear words vs Maya-1 | | |
| |---|---|---|---|---|---| | |
| | [kale-tts-1-tiny](https://huggingface.co/KaleComputer/kale-tts-1-tiny) | 45.7M | 79.9 MB | 183.2 MB | 24x real time | +1.4 points | | |
| | [kale-tts-1-nano](https://huggingface.co/KaleComputer/kale-tts-1-nano) | 7.5M | 14.8 MB | 30.6 MB | 44x real time | +1.6 points | | |
| tiny keeps this model's timing-and-melody planner (frozen) and adds the Ettin word reader; nano is a smaller student distilled from it. | |
| Both have their own small sound generator and run on a low-end tablet. | |
| | measure | value | | |
| |---|---| | |
| | parameters | 81.8M (28.5M fine-tuned on our voices: text side + planner; the rest is stock Kokoro-82M's sound decoder) | | |
| | size | 327 MB (`kale-tts-1-base.pth`, PyTorch) | | |
| | voices | 10, as voice packs in `voices/` | | |
| | trained on | [kale-tts-1-data](https://huggingface.co/datasets/KaleComputer/kale-tts-1-data): 1,300 takes per voice, 26.9 hours | | |
| | clarity and melody (our release bench, 10 voices, 18 Nov) | unclear words +3.9 points vs Maya-1's own recordings (Maya-1: 5.82 %; tiny +1.4, nano +1.6); melody gap 12.2 (tiny 7.48, nano 7.46). Not built for the tablet | | |
| ## Voices | |
| | name | voice | pronunciation | | |
| |---|---|---| | |
| | Kenzie | esports hype woman | US English | | |
| | Simone | deep and confident woman | US English | | |
| | Amy | crisp news-style woman | British English | | |
| | Frankie | hard-boiled detective's partner | US English | | |
| | Darlene | rough low-voiced old trucker woman | US English | | |
| | Owen | bright young host | US English | | |
| | Toby | mischievous prankster | US English | | |
| | Jack | crisp news-style man | British English | | |
| | Earl | African American jazz granddad | US English | | |
| | Magnus | deep dramatic trailer narrator | US English | | |
| Every voice was designed from a one-line text description; none is a real person. | |
| ## Use it | |
| ```bash | |
| pip install kokoro soundfile | |
| pip install -U huggingface_hub | |
| hf download KaleComputer/kale-tts-1-base --local-dir kale-tts-1-base | |
| cd kale-tts-1-base | |
| ``` | |
| Save as `speak.py` in that folder, then run `python speak.py`: | |
| ```python | |
| import json, torch, soundfile as sf | |
| from kokoro import KModel, KPipeline | |
| model = KModel(config="config.json", model="kale-tts-1-base.pth").eval() | |
| voices = {v["name"]: v for v in json.load(open("voices.json"))} | |
| # American voices use lang_code "a", the two British ones (Amy, Jack) "b" | |
| v = voices["Kenzie"] | |
| pipe = KPipeline(lang_code=v["lang_code"], model=model) | |
| pack = torch.load(v["file"], weights_only=True) | |
| audio = torch.cat([a for _, _, a in pipe("Hello there! A volcano is a place where very hot, melted rock comes up out of the ground.", voice=pack)]) | |
| sf.write("hello.wav", audio.numpy(), 24000) | |
| ``` | |
| `KPipeline` turns text into sounds with Kokoro's own text front end (misaki). Our checked dictionary and text rules are in the | |
| kale-tts-1-tiny and kale-tts-1-nano repos (`frontend/`). | |
| ## How it was made | |
|  | |
| 1. **Maya-1** designs each of the 10 voices from a voice prompt and reads thousands of sentences with it; a filter keeps the clean takes | |
| ([kale-tts-1-data](https://huggingface.co/datasets/KaleComputer/kale-tts-1-data)). | |
| 2. We trained our own aligner on 1,300 takes per voice (26.9 hours, 30 epochs on an NVIDIA L40S), then fine-tuned Kokoro-82M on them | |
| in two stages on an NVIDIA H100 (three epochs, then one more with the adversarial stage). | |
| 3. The voice packs come from the fine-tuned style encoder, one per voice. | |
| 4. kale-tts-1-tiny keeps this planner (frozen) and adds the Ettin word reader; kale-tts-1-nano is a smaller student distilled from it. | |
| Both learn their sound straight from Maya-1's recordings. | |
| ## Limits | |
| - English only. The two British voices read best with `lang_code="b"`. | |
| - Too big and slow for a low-end tablet: that's what kale-tts-1-tiny and kale-tts-1-nano are for. | |
| - Not for impersonating real people or passing the output off as human speech. | |
| ## Licence and credits | |
| Apache-2.0. | |
| AI-generated speech: everything this model says is synthetic speech made by an AI model, in voices that belong to no real person. Tell | |
| listeners it is AI-generated, never present it as a human recording, and never use it to imitate a real person. | |
| Credits: based on Maya-1: all training audio is original speech from Maya-1 (Maya Research, Apache-2.0), decoded with SNAC (MIT). Fine-tuned | |
| from Kokoro-82M (hexgrad, Apache-2.0), whose sound decoder this model keeps unchanged, with StyleTTS2 (MIT) and kikiri-tts (Apache-2.0) | |
| code. Training texts: public-domain books made available by Project Gutenberg. Dataset: kale-tts-1-data (CC BY 4.0). | |