README.md

---
license: apache-2.0
language:
- zh
pipeline_tag: text-to-speech
tags:
- cosyvoice
- cosyvoice3
- multi-accent
- chinese
---

# CosyVoice3 Multi-Accent

Instruction-controlled Chinese multi-accent TTS based on
`Fun-CosyVoice3-0.5B`. The fine-tuned LLM supports nine accents:

`north`, `Sichuan`, `Guangdong`, `south`, `Henan`, `Shanghai`, `Wuhan`,
`Tianjin`, and `singapore`.

The repository is self-contained: it includes the CosyVoice inference source,
Matcha-TTS source, tokenizer, flow model, vocoder, ONNX components, and the
fine-tuned `llm.pt`. A separate CosyVoice checkout is not required.

## Install and run

```bash
git clone https://huggingface.co/walston/cosyvoice3-multiaccent
cd cosyvoice3-multiaccent
pip install -r requirements.txt
python inference.py \
  --text "今天的天气很好，我们一起去吃饭吧。" \
  --accent Sichuan \
  --output sichuan.wav
```

You can also download without Git LFS:

```python
from huggingface_hub import snapshot_download

model_dir = snapshot_download("walston/cosyvoice3-multiaccent")
```

Then run `inference.py` from `model_dir`, or import the bundled CosyVoice code.
CUDA is recommended. CPU inference is possible but slow.

## Hugging Face Inference Endpoint

The repository includes `handler.py`, which implements a custom Endpoint
handler. Request format:

```json
{
  "inputs": "今天的天气很好，我们一起去吃饭吧。",
  "parameters": {
    "accent": "Sichuan",
    "speed": 1.0
  }
}
```

The response contains `audio_base64`, `sample_rate`, and `accent`. The bundled
reference voice is used by default. Pass a WAV file as
`parameters.prompt_audio_base64` to select another reference voice.

## Fine-tuning

Training used 135k original Chinese transcriptions with accent instructions.
The published LLM is the average of the five checkpoints with the lowest
validation loss (epochs 0 through 4). Training was stopped after epoch 31 when
validation loss showed sustained overfitting.

Accent IDs used during training:

| ID | Accent |
|---:|---|
| 0 | north |
| 1 | Sichuan |
| 2 | Guangdong |
| 3 | south |
| 4 | Henan |
| 5 | Shanghai |
| 6 | Wuhan |
| 7 | Tianjin |
| 8 | singapore |

## Limitations

Accent strength and speaker similarity depend on the reference voice and the
amount of training data available for each accent. The training distribution is
imbalanced. Evaluate generated speech before production use.