Add files using upload-large-folder tool

a960dea verified 3 days ago

2.41 kB

license: apache-2.0
language:
  - zh
pipeline_tag: text-to-speech
tags:
  - cosyvoice
  - cosyvoice3
  - multi-accent
  - chinese

CosyVoice3 Multi-Accent

Instruction-controlled Chinese multi-accent TTS based on Fun-CosyVoice3-0.5B. The fine-tuned LLM supports nine accents:

north, Sichuan, Guangdong, south, Henan, Shanghai, Wuhan, Tianjin, and singapore.

The repository is self-contained: it includes the CosyVoice inference source, Matcha-TTS source, tokenizer, flow model, vocoder, ONNX components, and the fine-tuned llm.pt. A separate CosyVoice checkout is not required.

Install and run

git clone https://huggingface.co/walston/cosyvoice3-multiaccent
cd cosyvoice3-multiaccent
pip install -r requirements.txt
python inference.py \
  --text "今天的天气很好，我们一起去吃饭吧。" \
  --accent Sichuan \
  --output sichuan.wav

You can also download without Git LFS:

from huggingface_hub import snapshot_download

model_dir = snapshot_download("walston/cosyvoice3-multiaccent")

Then run inference.py from model_dir, or import the bundled CosyVoice code. CUDA is recommended. CPU inference is possible but slow.

Hugging Face Inference Endpoint

The repository includes handler.py, which implements a custom Endpoint handler. Request format:

{
  "inputs": "今天的天气很好，我们一起去吃饭吧。",
  "parameters": {
    "accent": "Sichuan",
    "speed": 1.0
  }
}

The response contains audio_base64, sample_rate, and accent. The bundled reference voice is used by default. Pass a WAV file as parameters.prompt_audio_base64 to select another reference voice.

Fine-tuning

Training used 135k original Chinese transcriptions with accent instructions. The published LLM is the average of the five checkpoints with the lowest validation loss (epochs 0 through 4). Training was stopped after epoch 31 when validation loss showed sustained overfitting.

Accent IDs used during training:

ID	Accent
0	north
1	Sichuan
2	Guangdong
3	south
4	Henan
5	Shanghai
6	Wuhan
7	Tianjin
8	singapore

Limitations

Accent strength and speaker similarity depend on the reference voice and the amount of training data available for each accent. The training distribution is imbalanced. Evaluate generated speech before production use.