File size: 3,084 Bytes
7017c4c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | ---
base_model:
- FunAudioLLM/Fun-CosyVoice3-0.5B-2512
datasets:
- XXH333/WordVoice-5A
language:
- zh
- en
license: apache-2.0
pipeline_tag: text-to-speech
---
# WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS
WordVoice is a novel speech generation framework that transforms traditional implicit end-to-end TTS generation into an explicit, highly controllable paradigm. Built on top of the CosyVoice3 framework, it enables precise and decoupled word-level control over five acoustic dimensions.
- **Paper:** [WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS](https://huggingface.co/papers/2607.06461)
- **GitHub Repository:** [XXH333/WordVoice-main](https://github.com/XXH333/WordVoice-main)
- **Project Page & Demo:** [WordVoice Demo](https://xxh333.github.io/wordvoice-demo/)
---
## ✨ Features
### 🎯 Explicit Word-Level Control
Supports independent and decoupled control of five acoustic attributes for each input word:
- ⏱️ **Duration**: Word-level pronunciation duration.
- ⏸️ **Boundary**: 5-level pause classification (`b0`–`b4`).
- 🔊 **Energy**: Word-level volume/loudness (`0`–`1`).
- 🎵 **Pitch**: Word-level core fundamental frequency (`-1`–`1`).
- 📈 **Tone**: 7 categories of prosodic morphologies (flat, rise, strong rise, fall, strong fall, peak, valley).
### 🧠 "Acoustic Thinking" Mechanism via Bound-Token
Employs a `bound-token` (`<b>`) mechanism within the autoregressive (AR) language model. Before generating the speech tokens for a specific word, the model explicitly predicts its acoustic attributes, realizing an intelligent process of "planning prosody first, then generating sound."
---
## 🛠️ Quick Start
### Installation
We recommend using **Conda** to manage your Python environment.
```bash
conda create -n wordvoice python=3.10 -y
conda activate wordvoice
git clone https://github.com/XXH333/WordVoice-main.git
cd WordVoice-main
pip install -e .
pip install num2words==0.5.14 x_transformers==2.11.24
```
### Download Model Weights
Run the following script to automatically download the pre-trained weights and dependencies (such as CosyVoice3, MMS-FA, etc.):
```bash
bash download_models.sh
```
### Inference
You can run the out-of-the-box inference script to experience both the **Free Mode** and **Control Mode** of WordVoice:
```bash
python wordvoice_infer.py
```
For custom prompts and detailed control parameters, refer to `wordvoice_infer.py` and the [GitHub repository](https://github.com/XXH333/WordVoice-main).
---
## 📝 Citation
If you find this work or the models useful, please cite:
```bibtex
@misc{nie2026wordvoice,
title={WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS},
author={Sihang Nie and Jinxin Ji and Xiaofen Xing and Deyi Tuo and Chengbin Jin and Jialong Mai and Xiangmin Xu},
year={2026},
eprint={2607.06461},
archivePrefix={arXiv},
primaryClass={eess.AS},
url={https://arxiv.org/abs/2607.06461},
}
``` |