| --- |
| license: mit |
| language: |
| - ar |
| - bg |
| - cs |
| - da |
| - de |
| - el |
| - en |
| - es |
| - fi |
| - fr |
| - he |
| - hi |
| - hu |
| - it |
| - ja |
| - ko |
| - ms |
| - nl |
| - "no" |
| - pl |
| - pt |
| - ro |
| - ru |
| - sk |
| - sv |
| - sw |
| - ta |
| - tr |
| - vi |
| - zh |
| library_name: phoonnx |
| pipeline_tag: text-to-speech |
| tags: |
| - tts |
| - chatterbox |
| - onnx |
| - phoonnx |
| - offline |
| - voice-cloning |
| - codec-lm |
| - multilingual |
| --- |
| |
| # phoonnx-chatterbox-multilingual |
|
|
| Mirror of the multilingual variant of [Chatterbox](https://github.com/resemble-ai/chatterbox) |
| by Resemble AI, exported to ONNX for |
| [phoonnx](https://github.com/TigreGotico/phoonnx), OpenVoiceOS's offline TTS |
| library, covering 30 languages from Arabic to Vietnamese. Like the base |
| model, this is an autoregressive codec-LM (Llama backbone) with zero-shot |
| cloning from an untranscribed reference clip and an `exaggeration` control. |
| See |
| [docs/training/engines/chatterbox.md](https://github.com/TigreGotico/phoonnx/blob/dev/docs/training/engines/chatterbox.md) |
| and [docs/cloning.md](https://github.com/TigreGotico/phoonnx/blob/dev/docs/cloning.md). |
|
|
| The multilingual variant uses its own `ChatterboxMTLTokenizer`. Japanese and |
| Chinese script normalization need |
| `pip install phoonnx[chatterbox-multilingual]` (pulls in `pykakasi` and |
| `spacy-pkuseg`); Korean normalization is pure-Python and always available; |
| Hebrew and Russian degrade gracefully when their optional |
| stressors/diacritizers are absent. |
|
|
| Related repositories: [phoonnx-chatterbox](https://huggingface.co/OpenVoiceOS/phoonnx-chatterbox) |
| (base English), [phoonnx-chatterbox-turbo](https://huggingface.co/OpenVoiceOS/phoonnx-chatterbox-turbo) |
| (faster GPT-2-based variant), [phoonnx-chatterbox-lahgtna](https://huggingface.co/OpenVoiceOS/phoonnx-chatterbox-lahgtna) |
| (Arabic dialects, fine-tuned from this model). |
|
|
| ## Usage |
|
|
| Cloning support requires `pip install phoonnx[cloning]`. |
|
|
| ```python |
| import wave |
| from phoonnx.model_manager import TTSModelManager |
| |
| manager = TTSModelManager() |
| manager.load() |
| manager.merge_default_voices() |
| |
| voice = manager.voices["chatterbox/multilingual/pt"].load() |
| with wave.open("out.wav", "wb") as wav_file: |
| voice.synthesize_wav( |
| "Olá, isto é um teste.", |
| wav_file, |
| speaker_reference="reference.wav", # optional: clone this speaker |
| ) |
| ``` |
|
|
| ## License |
|
|
| Resemble AI released Chatterbox Multilingual under the MIT license (see |
| [`onnx-community/chatterbox-multilingual-ONNX`](https://huggingface.co/onnx-community/chatterbox-multilingual-ONNX)). |
| `phoonnx` itself is Apache-2.0. |
|
|
