jin9581 commited on
Commit
71ccd1d
·
verified ·
1 Parent(s): 4933170

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -7
README.md CHANGED
@@ -48,7 +48,7 @@ Choose the right model for your application.
48
 
49
  | Model | Size | Languages | Key Features | Best For | 🤗 | Examples |
50
  |:----------------------------------------------------------------------------------------------------------------| :--- | :--- |:--------------------------------------------------------|:---------------------------------------------|:--------------------------------------------------------------------------| :--- |
51
- | **Chatterbox-Nano** | **110M** | **English** | Paralinguistic Tags (`[laugh]`), 3x faster than realtime on 8core cpu | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-nano-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
52
  | Chatterbox-Turbo | 350M | English | Paralinguistic Tags (`[laugh]`), Lower Compute and VRAM | Zero-shot voice agents, Production | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-turbo-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
53
  | Chatterbox-Multilingual [(Language list)](#supported-languages) | 500M | 23+ | Zero-shot cloning, Multiple Languages | Global applications, Localization | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
54
  | Chatterbox [(Tips and Tricks)](#original-chatterbox-tips) | 500M | English | CFG & Exaggeration tuning | General zero-shot TTS with creative controls | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
@@ -71,6 +71,27 @@ We developed and tested Chatterbox on Python 3.11 on Debian 11 OS; the versions
71
 
72
  ## Usage
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
  ##### Chatterbox-Turbo
75
 
76
  ```python
@@ -98,8 +119,10 @@ import torchaudio as ta
98
  from chatterbox.tts import ChatterboxTTS
99
  from chatterbox.mtl_tts import ChatterboxMultilingualTTS
100
 
 
 
101
  # English example
102
- model = ChatterboxTTS.from_pretrained(device="cuda")
103
 
104
  text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill."
105
  wav = model.generate(text)
@@ -109,19 +132,19 @@ ta.save("test-english.wav", wav, model.sr)
109
  multilingual_model = ChatterboxMultilingualTTS.from_pretrained(device=device)
110
 
111
  french_text = "Bonjour, comment ça va? Ceci est le modèle de synthèse vocale multilingue Chatterbox, il prend en charge 23 langues."
112
- wav_french = multilingual_model.generate(spanish_text, language_id="fr")
113
- ta.save("test-french.wav", wav_french, model.sr)
114
 
115
  chinese_text = "你好,今天天气真不错,希望你有一个愉快的周末。"
116
  wav_chinese = multilingual_model.generate(chinese_text, language_id="zh")
117
- ta.save("test-chinese.wav", wav_chinese, model.sr)
118
 
119
  # If you want to synthesize with a different voice, specify the audio prompt
120
  AUDIO_PROMPT_PATH = "YOUR_FILE.wav"
121
  wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
122
  ta.save("test-2.wav", wav, model.sr)
123
  ```
124
- See `example_tts.py` and `example_vc.py` for more examples.
125
 
126
  ## Supported Languages
127
  Arabic (ar) • Danish (da) • German (de) • Greek (el) • English (en) • Spanish (es) • Finnish (fi) • French (fr) • Hebrew (he) • Hindi (hi) • Italian (it) • Japanese (ja) • Korean (ko) • Malay (ms) • Dutch (nl) • Norwegian (no) • Polish (pl) • Portuguese (pt) • Russian (ru) • Swedish (sv) • Swahili (sw) • Turkish (tr) • Chinese (zh)
@@ -188,4 +211,4 @@ If you find this model useful, please consider citing.
188
  }
189
  ```
190
  ## Disclaimer
191
- Don't use this model to do bad things. Prompts are sourced from freely available data on the internet.
 
48
 
49
  | Model | Size | Languages | Key Features | Best For | 🤗 | Examples |
50
  |:----------------------------------------------------------------------------------------------------------------| :--- | :--- |:--------------------------------------------------------|:---------------------------------------------|:--------------------------------------------------------------------------| :--- |
51
+ | **Chatterbox-Nano** | **110M** | **English** | Paralinguistic Tags (`[laugh]`), 3x faster than realtime on 8-core CPU | On-device / CPU inference, tight latency & memory budgets | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-nano-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
52
  | Chatterbox-Turbo | 350M | English | Paralinguistic Tags (`[laugh]`), Lower Compute and VRAM | Zero-shot voice agents, Production | [Demo](https://huggingface.co/spaces/ResembleAI/chatterbox-turbo-demo) | [Listen](https://resemble-ai.github.io/chatterbox_turbo_demopage/) |
53
  | Chatterbox-Multilingual [(Language list)](#supported-languages) | 500M | 23+ | Zero-shot cloning, Multiple Languages | Global applications, Localization | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
54
  | Chatterbox [(Tips and Tricks)](#original-chatterbox-tips) | 500M | English | CFG & Exaggeration tuning | General zero-shot TTS with creative controls | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox) | [Listen](https://resemble-ai.github.io/chatterbox_demopage/) |
 
71
 
72
  ## Usage
73
 
74
+ ##### Chatterbox-Nano
75
+
76
+ Nano shares Turbo's architecture and is loaded through the same `ChatterboxTurboTTS` class by passing `nano=True`:
77
+
78
+ ```python
79
+ import torchaudio as ta
80
+ import torch
81
+ from chatterbox.tts_turbo import ChatterboxTurboTTS
82
+
83
+ # Load the Nano model (also runs on CPU: device="cpu")
84
+ model = ChatterboxTurboTTS.from_pretrained(device="cuda", nano=True)
85
+
86
+ # Generate with Paralinguistic Tags
87
+ text = "Hi there, Sarah here from MochaFone calling you back [chuckle], have you got one minute to chat about the billing issue?"
88
+
89
+ # Generate audio (requires a reference clip for voice cloning)
90
+ wav = model.generate(text, audio_prompt_path="your_10s_ref_clip.wav")
91
+
92
+ ta.save("test-nano.wav", wav, model.sr)
93
+ ```
94
+
95
  ##### Chatterbox-Turbo
96
 
97
  ```python
 
119
  from chatterbox.tts import ChatterboxTTS
120
  from chatterbox.mtl_tts import ChatterboxMultilingualTTS
121
 
122
+ device = "cuda" # or "cpu" / "mps"
123
+
124
  # English example
125
+ model = ChatterboxTTS.from_pretrained(device=device)
126
 
127
  text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill."
128
  wav = model.generate(text)
 
132
  multilingual_model = ChatterboxMultilingualTTS.from_pretrained(device=device)
133
 
134
  french_text = "Bonjour, comment ça va? Ceci est le modèle de synthèse vocale multilingue Chatterbox, il prend en charge 23 langues."
135
+ wav_french = multilingual_model.generate(french_text, language_id="fr")
136
+ ta.save("test-french.wav", wav_french, multilingual_model.sr)
137
 
138
  chinese_text = "你好,今天天气真不错,希望你有一个愉快的周末。"
139
  wav_chinese = multilingual_model.generate(chinese_text, language_id="zh")
140
+ ta.save("test-chinese.wav", wav_chinese, multilingual_model.sr)
141
 
142
  # If you want to synthesize with a different voice, specify the audio prompt
143
  AUDIO_PROMPT_PATH = "YOUR_FILE.wav"
144
  wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
145
  ta.save("test-2.wav", wav, model.sr)
146
  ```
147
+ See `example_tts.py`, `example_tts_turbo.py`, `example_tts_nano.py`, and `example_vc.py` for more examples.
148
 
149
  ## Supported Languages
150
  Arabic (ar) • Danish (da) • German (de) • Greek (el) • English (en) • Spanish (es) • Finnish (fi) • French (fr) • Hebrew (he) • Hindi (hi) • Italian (it) • Japanese (ja) • Korean (ko) • Malay (ms) • Dutch (nl) • Norwegian (no) • Polish (pl) • Portuguese (pt) • Russian (ru) • Swedish (sv) • Swahili (sw) • Turkish (tr) • Chinese (zh)
 
211
  }
212
  ```
213
  ## Disclaimer
214
+ Don't use this model to do bad things. Prompts are sourced from freely available data on the internet.