--- # DRAFT, NOT PUBLISHED. Credit: based on Maya (Maya-1). license: apache-2.0 language: [en] pipeline_tag: text-to-speech library_name: kokoro tags: [text-to-speech, tts, kokoro, synthetic-voices, based-on-maya] base_model: hexgrad/Kokoro-82M base_model_relation: finetune datasets: [KaleComputer/kale-tts-1-data] --- ![kale-tts-1-base](diagrams/header-base.png) # kale-tts-1-base kale-tts-1-base is [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) with its text side and its timing-and-melody planner fine-tuned on our 10 voices designed with [Maya-1](https://huggingface.co/maya-research/maya1); its sound decoder is stock Kokoro-82M. It's the teacher of [kale-tts-1-tiny](https://huggingface.co/KaleComputer/kale-tts-1-tiny) and [kale-tts-1-nano](https://huggingface.co/KaleComputer/kale-tts-1-nano): both learned their timing and melody from it and have their own small sound generators, so they run on a low-end tablet. kale-tts-1-base itself is too big and slow for that; use it on a desktop or GPU, or to train and distill your own small voices. ## Distilled into | model | parameters | CPU file (int8) | WebGPU file (float32) | Galaxy Tab S10 Lite | unclear words vs Maya-1 | |---|---|---|---|---|---| | [kale-tts-1-tiny](https://huggingface.co/KaleComputer/kale-tts-1-tiny) | 45.7M | 79.9 MB | 183.2 MB | 24x real time | +1.4 points | | [kale-tts-1-nano](https://huggingface.co/KaleComputer/kale-tts-1-nano) | 7.5M | 14.8 MB | 30.6 MB | 44x real time | +1.6 points | tiny keeps this model's timing-and-melody planner (frozen) and adds the Ettin word reader; nano is a smaller student distilled from it. Both have their own small sound generator and run on a low-end tablet. | measure | value | |---|---| | parameters | 81.8M (28.5M fine-tuned on our voices: text side + planner; the rest is stock Kokoro-82M's sound decoder) | | size | 327 MB (`kale-tts-1-base.pth`, PyTorch) | | voices | 10, as voice packs in `voices/` | | trained on | [kale-tts-1-data](https://huggingface.co/datasets/KaleComputer/kale-tts-1-data): 1,300 takes per voice, 26.9 hours | | clarity and melody (our release bench, 10 voices, 18 Nov) | unclear words +3.9 points vs Maya-1's own recordings (Maya-1: 5.82 %; tiny +1.4, nano +1.6); melody gap 12.2 (tiny 7.48, nano 7.46). Not built for the tablet | ## Voices | name | voice | pronunciation | |---|---|---| | Kenzie | esports hype woman | US English | | Simone | deep and confident woman | US English | | Amy | crisp news-style woman | British English | | Frankie | hard-boiled detective's partner | US English | | Darlene | rough low-voiced old trucker woman | US English | | Owen | bright young host | US English | | Toby | mischievous prankster | US English | | Jack | crisp news-style man | British English | | Earl | African American jazz granddad | US English | | Magnus | deep dramatic trailer narrator | US English | Every voice was designed from a one-line text description; none is a real person. ## Use it ```bash pip install kokoro soundfile pip install -U huggingface_hub hf download KaleComputer/kale-tts-1-base --local-dir kale-tts-1-base cd kale-tts-1-base ``` Save as `speak.py` in that folder, then run `python speak.py`: ```python import json, torch, soundfile as sf from kokoro import KModel, KPipeline model = KModel(config="config.json", model="kale-tts-1-base.pth").eval() voices = {v["name"]: v for v in json.load(open("voices.json"))} # American voices use lang_code "a", the two British ones (Amy, Jack) "b" v = voices["Kenzie"] pipe = KPipeline(lang_code=v["lang_code"], model=model) pack = torch.load(v["file"], weights_only=True) audio = torch.cat([a for _, _, a in pipe("Hello there! A volcano is a place where very hot, melted rock comes up out of the ground.", voice=pack)]) sf.write("hello.wav", audio.numpy(), 24000) ``` `KPipeline` turns text into sounds with Kokoro's own text front end (misaki). Our checked dictionary and text rules are in the kale-tts-1-tiny and kale-tts-1-nano repos (`frontend/`). ## How it was made ![How we picked the base model, at selection](diagrams/birth.png) 1. **Maya-1** designs each of the 10 voices from a voice prompt and reads thousands of sentences with it; a filter keeps the clean takes ([kale-tts-1-data](https://huggingface.co/datasets/KaleComputer/kale-tts-1-data)). 2. We trained our own aligner on 1,300 takes per voice (26.9 hours, 30 epochs on an NVIDIA L40S), then fine-tuned Kokoro-82M on them in two stages on an NVIDIA H100 (three epochs, then one more with the adversarial stage). 3. The voice packs come from the fine-tuned style encoder, one per voice. 4. kale-tts-1-tiny keeps this planner (frozen) and adds the Ettin word reader; kale-tts-1-nano is a smaller student distilled from it. Both learn their sound straight from Maya-1's recordings. ## Limits - English only. The two British voices read best with `lang_code="b"`. - Too big and slow for a low-end tablet: that's what kale-tts-1-tiny and kale-tts-1-nano are for. - Not for impersonating real people or passing the output off as human speech. ## Licence and credits Apache-2.0. AI-generated speech: everything this model says is synthetic speech made by an AI model, in voices that belong to no real person. Tell listeners it is AI-generated, never present it as a human recording, and never use it to imitate a real person. Credits: based on Maya-1: all training audio is original speech from Maya-1 (Maya Research, Apache-2.0), decoded with SNAC (MIT). Fine-tuned from Kokoro-82M (hexgrad, Apache-2.0), whose sound decoder this model keeps unchanged, with StyleTTS2 (MIT) and kikiri-tts (Apache-2.0) code. Training texts: public-domain books made available by Project Gutenberg. Dataset: kale-tts-1-data (CC BY 4.0).