**KokoClone** is a fast, real-time compatible multilingual voice cloning system built on top of **Kokoro-ONNX**, one of the fastest open-source neural TTS engines available today.
It allows you to:
* **Text → Clone:** Type text in multiple languages, provide a short reference audio clip, and instantly generate speech in that same voice.
* **Audio → Clone:** Re-voice an existing audio recording to sound like any reference speaker — *no transcription needed*.
## Features
### Multilingual Speech Generation
Generate native speech in English (`en`), Hindi (`hi`), French (`fr`), Japanese (`ja`), Chinese (`zh`), Italian (`it`), Portuguese (`pt`), and Spanish (`es`).
### Zero-Shot Voice Cloning
Upload a 3–10 second voice sample and KokoClone instantly transfers its vocal characteristics to the generated speech.
### Audio-to-Audio Voice Conversion
Upload any existing speech recording and re-voice it to sound like a reference speaker. The pipeline skips TTS entirely and runs purely through the Kanade voice-conversion model. Works on recordings of any length thanks to automatic VRAM-aware chunking!
### Automatic Model Handling
On the first run, the required model weights (`.onnx` and `.bin` files) are automatically downloaded from Hugging Face and placed in the correct directories.
### Real-Time Friendly
Built on Kokoro's efficient ONNX runtime pipeline, KokoClone detects your hardware and runs smoothly on both standard laptops (CPU) and workstations (GPU).
## Live Demo
Try it instantly without installing anything:
👉 **[KokoClone on Hugging Face Spaces](https://huggingface.co/spaces/PatnaikAshish/kokoclone)**
## Installation
You can set up KokoClone using either **Conda** (Recommended) or **uv**.
### 1. Clone the Repository
```bash
git clone https://github.com/Ashish-Patnaik/kokoclone.git
cd kokoclone
```
### 2. Set Up the Environment & Install Dependencies
#### Option A: Using Conda (Recommended)
```bash
conda create -n kokoclone python=3.12.12 -y
conda activate kokoclone
```
**For CPU Users (Mac / Standard Laptops):**
```bash
pip install torch torchaudio --index-url [https://download.pytorch.org/whl/cpu](https://download.pytorch.org/whl/cpu)
pip install -r requirements.txt
```
**For GPU Users (Nvidia GPUs):**
```bash
pip install -r requirements.txt
pip install kokoro-onnx[gpu]
```
#### Option B: Using `uv`
If you prefer [uv](https://docs.astral.sh/uv/) for fast package management:
```bash
# For CPU Users
uv sync
# For GPU Users (Nvidia)
uv sync --extra gpu
# Activate the environment
source .venv/bin/activate # Linux/macOS
.venv\Scripts\activate # Windows
```
## Usage
KokoClone is highly flexible and can be used via Web UI, CLI, or Python API.
### 1. Web Interface (Gradio)
Launch the interactive web app:
```bash
python app.py
```
* **Tab 1 (Text → Clone):** Enter text, pick a language, upload a reference voice, and generate.
* **Tab 2 (Audio → Clone):** Upload source audio and a reference voice, and get back re-voiced audio.
### 2. Command Line Interface (CLI)
Generate speech directly from your terminal.
**Text to cloned speech (default mode):**
```bash
python cli.py --text "Hello from KokoClone" --lang en --ref reference.wav --out output.wav
```
**Audio to re-voiced speech:**
```bash
python cli.py --mode convert --source original_speech.wav --ref target_voice.wav --out revoiced.wav
```
| Argument | Default | Description |
| --- | --- | --- |
| `--mode` | `tts` | `tts` (text → speech) or `convert` (audio → re-voiced audio) |
| `--text` | — | Text to synthesize *(required for `tts` mode)* |
| `--lang` | `en` | Language code: `en hi fr ja zh it es pt` |
| `--source` | — | Path to source audio *(required for `convert` mode)* |
| `--ref` | — | Path to reference voice audio *(always required)* |
| `--out` | `output.wav` | Output file path |
### 3. Python API
Integrate KokoClone into your own Python applications.
**Text to Cloned Speech:**
```python
from core.cloner import KokoClone
cloner = KokoClone()
cloner.generate(
text="This voice is cloned using KokoClone.",
lang="en",
reference_audio="reference.wav",
output_path="output.wav"
)
```
**Audio-to-Audio Voice Conversion:**
```python
import soundfile as sf
from kanade_tokenizer import load_audio
from core.cloner import KokoClone
from core.chunked_convert import chunked_voice_conversion
cloner = KokoClone()
# Load audio tensors
source_wav = load_audio("source_speech.wav", sample_rate=cloner.sample_rate).to(cloner.device)
ref_wav = load_audio("target_voice.wav", sample_rate=cloner.sample_rate).to(cloner.device)
# Convert using VRAM-aware chunking
converted = chunked_voice_conversion(
kanade=cloner.kanade,
vocoder_model=cloner.vocoder,
source_wav=source_wav,
ref_wav=ref_wav,
sample_rate=cloner.sample_rate,
)
sf.write("revoiced_output.wav", converted.numpy(), cloner.sample_rate)
```
## Memory Management for Long Audio
The `chunked_voice_conversion` function in `core/chunked_convert.py` handles memory automatically when converting long audio recordings:
* **VRAM Budget:** On CUDA, chunks are sized so each forward pass uses at most 50% of total GPU memory (configurable via the `vram_fraction` parameter).
* **RoPE Ceiling:** The Kanade `mel_decoder` Transformer has positional embeddings precomputed for 1,024 mel frames. Chunk windows are hard-capped below this limit (≈ 8.9s of source audio per chunk) with a 10% safety margin to prevent recomputation and quality degradation.
* **Overlap Smoothing:** Each chunk includes a 0.5s overlap on both sides to suppress boundary artifacts.
* **Single-Pass Vocoding:** The full reassembled mel spectrogram is passed to the vocoder in one shot for clean waveform reconstruction.
## Project Structure
```text
app.py → Gradio Web Interface (two-tab UI)
cli.py → Command-line tool (tts and convert modes)
inference.py → Example API usage script
core/
├── cloner.py → Core TTS + voice cloning engine
└── chunked_convert.py → VRAM-aware chunked audio conversion
model/ → Downloaded Kokoro model weights (Auto-populates)
voice/ → Downloaded Kokoro voice bins (Auto-populates)
```
## Star History
## Acknowledgments
This project builds upon the incredible open-source work of:
* **[Kokoro-ONNX](https://github.com/thewh1teagle/kokoro-onnx)** — for fast and efficient neural speech synthesis.
* **[Kanade Tokenizer](https://github.com/frothywater/kanade-tokenizer)** — for the brilliant zero-shot voice conversion architecture.
## License
Licensed under the [Apache 2.0 License](https://www.google.com/search?q=LICENSE).
```
