Update README.md
Browse files
README.md
CHANGED
|
@@ -48,7 +48,7 @@ Choose the right model for your application.
|
|
| 48 |
|
| 49 |
| Model | Size | Languages | Key Features | Best For | 🤗 | Examples |
|
| 50 |
|:----------------------------------------------------------------------------------------------------------------| :--- | :--- |:--------------------------------------------------------|:---------------------------------------------|:--------------------------------------------------------------------------| :--- |
|
| 51 |
-
| **Chatterbox-Nano** | **110M** | **English** | Paralinguistic Tags (`[laugh]`), 3x faster than realtime on
|
| 52 |
| Chatterbox-Turbo | 350M | English | Paralinguistic Tags (`[laugh]`), Lower Compute and VRAM | Zero-shot voice agents, Production | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-turbo-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
|
| 53 |
| Chatterbox-Multilingual [(Language list)](#supported-languages) | 500M | 23+ | Zero-shot cloning, Multiple Languages | Global applications, Localization | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
|
| 54 |
| Chatterbox [(Tips and Tricks)](#original-chatterbox-tips) | 500M | English | CFG & Exaggeration tuning | General zero-shot TTS with creative controls | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
|
|
@@ -71,6 +71,27 @@ We developed and tested Chatterbox on Python 3.11 on Debian 11 OS; the versions
|
|
| 71 |
|
| 72 |
## Usage
|
| 73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
##### Chatterbox-Turbo
|
| 75 |
|
| 76 |
```python
|
|
@@ -98,8 +119,10 @@ import torchaudio as ta
|
|
| 98 |
from chatterbox.tts import ChatterboxTTS
|
| 99 |
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
|
| 100 |
|
|
|
|
|
|
|
| 101 |
# English example
|
| 102 |
-
model = ChatterboxTTS.from_pretrained(device=
|
| 103 |
|
| 104 |
text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill."
|
| 105 |
wav = model.generate(text)
|
|
@@ -109,19 +132,19 @@ ta.save("test-english.wav", wav, model.sr)
|
|
| 109 |
multilingual_model = ChatterboxMultilingualTTS.from_pretrained(device=device)
|
| 110 |
|
| 111 |
french_text = "Bonjour, comment ça va? Ceci est le modèle de synthèse vocale multilingue Chatterbox, il prend en charge 23 langues."
|
| 112 |
-
wav_french = multilingual_model.generate(
|
| 113 |
-
ta.save("test-french.wav", wav_french,
|
| 114 |
|
| 115 |
chinese_text = "你好,今天天气真不错,希望你有一个愉快的周末。"
|
| 116 |
wav_chinese = multilingual_model.generate(chinese_text, language_id="zh")
|
| 117 |
-
ta.save("test-chinese.wav", wav_chinese,
|
| 118 |
|
| 119 |
# If you want to synthesize with a different voice, specify the audio prompt
|
| 120 |
AUDIO_PROMPT_PATH = "YOUR_FILE.wav"
|
| 121 |
wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
|
| 122 |
ta.save("test-2.wav", wav, model.sr)
|
| 123 |
```
|
| 124 |
-
See `example_tts.py` and `example_vc.py` for more examples.
|
| 125 |
|
| 126 |
## Supported Languages
|
| 127 |
Arabic (ar) • Danish (da) • German (de) • Greek (el) • English (en) • Spanish (es) • Finnish (fi) • French (fr) • Hebrew (he) • Hindi (hi) • Italian (it) • Japanese (ja) • Korean (ko) • Malay (ms) • Dutch (nl) • Norwegian (no) • Polish (pl) • Portuguese (pt) • Russian (ru) • Swedish (sv) • Swahili (sw) • Turkish (tr) • Chinese (zh)
|
|
@@ -188,4 +211,4 @@ If you find this model useful, please consider citing.
|
|
| 188 |
}
|
| 189 |
```
|
| 190 |
## Disclaimer
|
| 191 |
-
Don't use this model to do bad things. Prompts are sourced from freely available data on the internet.
|
|
|
|
| 48 |
|
| 49 |
| Model | Size | Languages | Key Features | Best For | 🤗 | Examples |
|
| 50 |
|:----------------------------------------------------------------------------------------------------------------| :--- | :--- |:--------------------------------------------------------|:---------------------------------------------|:--------------------------------------------------------------------------| :--- |
|
| 51 |
+
| **Chatterbox-Nano** | **110M** | **English** | Paralinguistic Tags (`[laugh]`), 3x faster than realtime on 8-core CPU | On-device / CPU inference, tight latency & memory budgets | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-nano-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
|
| 52 |
| Chatterbox-Turbo | 350M | English | Paralinguistic Tags (`[laugh]`), Lower Compute and VRAM | Zero-shot voice agents, Production | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-turbo-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
|
| 53 |
| Chatterbox-Multilingual [(Language list)](#supported-languages) | 500M | 23+ | Zero-shot cloning, Multiple Languages | Global applications, Localization | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
|
| 54 |
| Chatterbox [(Tips and Tricks)](#original-chatterbox-tips) | 500M | English | CFG & Exaggeration tuning | General zero-shot TTS with creative controls | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
|
|
|
|
| 71 |
|
| 72 |
## Usage
|
| 73 |
|
| 74 |
+
##### Chatterbox-Nano
|
| 75 |
+
|
| 76 |
+
Nano shares Turbo's architecture and is loaded through the same `ChatterboxTurboTTS` class by passing `nano=True`:
|
| 77 |
+
|
| 78 |
+
```python
|
| 79 |
+
import torchaudio as ta
|
| 80 |
+
import torch
|
| 81 |
+
from chatterbox.tts_turbo import ChatterboxTurboTTS
|
| 82 |
+
|
| 83 |
+
# Load the Nano model (also runs on CPU: device="cpu")
|
| 84 |
+
model = ChatterboxTurboTTS.from_pretrained(device="cuda", nano=True)
|
| 85 |
+
|
| 86 |
+
# Generate with Paralinguistic Tags
|
| 87 |
+
text = "Hi there, Sarah here from MochaFone calling you back [chuckle], have you got one minute to chat about the billing issue?"
|
| 88 |
+
|
| 89 |
+
# Generate audio (requires a reference clip for voice cloning)
|
| 90 |
+
wav = model.generate(text, audio_prompt_path="your_10s_ref_clip.wav")
|
| 91 |
+
|
| 92 |
+
ta.save("test-nano.wav", wav, model.sr)
|
| 93 |
+
```
|
| 94 |
+
|
| 95 |
##### Chatterbox-Turbo
|
| 96 |
|
| 97 |
```python
|
|
|
|
| 119 |
from chatterbox.tts import ChatterboxTTS
|
| 120 |
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
|
| 121 |
|
| 122 |
+
device = "cuda" # or "cpu" / "mps"
|
| 123 |
+
|
| 124 |
# English example
|
| 125 |
+
model = ChatterboxTTS.from_pretrained(device=device)
|
| 126 |
|
| 127 |
text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill."
|
| 128 |
wav = model.generate(text)
|
|
|
|
| 132 |
multilingual_model = ChatterboxMultilingualTTS.from_pretrained(device=device)
|
| 133 |
|
| 134 |
french_text = "Bonjour, comment ça va? Ceci est le modèle de synthèse vocale multilingue Chatterbox, il prend en charge 23 langues."
|
| 135 |
+
wav_french = multilingual_model.generate(french_text, language_id="fr")
|
| 136 |
+
ta.save("test-french.wav", wav_french, multilingual_model.sr)
|
| 137 |
|
| 138 |
chinese_text = "你好,今天天气真不错,希望你有一个愉快的周末。"
|
| 139 |
wav_chinese = multilingual_model.generate(chinese_text, language_id="zh")
|
| 140 |
+
ta.save("test-chinese.wav", wav_chinese, multilingual_model.sr)
|
| 141 |
|
| 142 |
# If you want to synthesize with a different voice, specify the audio prompt
|
| 143 |
AUDIO_PROMPT_PATH = "YOUR_FILE.wav"
|
| 144 |
wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
|
| 145 |
ta.save("test-2.wav", wav, model.sr)
|
| 146 |
```
|
| 147 |
+
See `example_tts.py`, `example_tts_turbo.py`, `example_tts_nano.py`, and `example_vc.py` for more examples.
|
| 148 |
|
| 149 |
## Supported Languages
|
| 150 |
Arabic (ar) • Danish (da) • German (de) • Greek (el) • English (en) • Spanish (es) • Finnish (fi) • French (fr) • Hebrew (he) • Hindi (hi) • Italian (it) • Japanese (ja) • Korean (ko) • Malay (ms) • Dutch (nl) • Norwegian (no) • Polish (pl) • Portuguese (pt) • Russian (ru) • Swedish (sv) • Swahili (sw) • Turkish (tr) • Chinese (zh)
|
|
|
|
| 211 |
}
|
| 212 |
```
|
| 213 |
## Disclaimer
|
| 214 |
+
Don't use this model to do bad things. Prompts are sourced from freely available data on the internet.
|