Salupala commited on
Commit
719e258
ยท
verified ยท
1 Parent(s): 140ddd3

update readme

Browse files
Files changed (1) hide show
  1. README.md +58 -31
README.md CHANGED
@@ -1,58 +1,85 @@
1
  ---
2
  license: mit
3
  language:
4
- - pt
5
  tags:
6
- - chatterbox
7
  - text-to-speech
8
  - tts
9
  - multilingual
10
- - single-language-tts
11
  - voice-cloning
 
 
12
  - chatterbox-v3
13
  pipeline_tag: text-to-speech
14
  base_model: ResembleAI/chatterbox
15
- base_model_relation: finetune
16
  ---
17
 
18
- <!-- chatterbox-space-link -->
19
- > ๐ŸŽ™๏ธ **Live demo:** Try this model in the [`ResembleAI/Chatterbox-Multilingual-TTS-pt-br`](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-pt-br) Space.
20
- <!-- chatterbox-space-link -->
 
 
 
 
 
 
 
21
 
22
- # Chatterbox Multilingual: Brazilian Portuguese
 
 
 
 
 
 
23
 
24
- Chatterbox Multilingual: Brazilian Portuguese is a dedicated single-language finetune in the **Chatterbox Multilingual V3 Single Language Pack**. It is optimized for Portuguese as spoken in Brazil, with language- and region-specific behavior for expressive text-to-speech and voice cloning.
25
 
26
- Use this model when you want tighter Brazilian Portuguese quality control than the broad multilingual checkpoint. For a single model that covers all supported languages, use [`ResembleAI/chatterbox`](https://huggingface.co/ResembleAI/chatterbox).
27
 
28
- ## Demo
 
29
 
30
- Try the hosted demo Space: [`ResembleAI/Chatterbox-Multilingual-TTS-pt-br`](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-pt-br).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
- ## Files
 
33
 
34
- - `t3_pt_br.safetensors`: T3 state dict in safetensors format.
35
- - `s3gen_v3.pt` / `s3gen_v3.safetensors`: V3 S3Gen speech decoder checkpoint.
36
- - `grapheme_mtl_merged_expanded_v1.json`: multilingual tokenizer config.
37
 
38
- ## Language
39
 
40
- - Locale: `pt-BR`
41
- - Chatterbox language ID: `pt`
42
 
43
- ## Checkpoint Metadata
 
 
44
 
45
- - Source step: `137400`
46
- - Source checkpoint: `t3_137400.pth.tar`
47
- - Tensor count: `292`
48
- - Dtype: `float32`
49
- - Text embedding shape: `(2454, 1024)`
50
- - Speech embedding shape: `(8194, 1024)`
51
- - Size: `2143990296` bytes
52
- - SHA256: `074aaf65255eb9cb960288f7cc72e09d3b5008f6e0b14868c0d4e5b0bd7cbb6c`
53
 
54
- ## Loader Notes
55
 
56
- This repository contains Chatterbox Multilingual V3 single-language assets used by the linked demo Space. The T3 checkpoint is loaded with multilingual vocabulary shape `2454` and S3 speech vocabulary shape `8194`.
 
 
 
 
57
 
58
- The demo combines these model-specific assets with the shared Chatterbox inference code and companion assets needed for end-to-end speech generation.
 
1
  ---
2
  license: mit
3
  language:
4
+ - multilingual
5
  tags:
 
6
  - text-to-speech
7
  - tts
8
  - multilingual
 
9
  - voice-cloning
10
+ - speech-synthesis
11
+ - chatterbox
12
  - chatterbox-v3
13
  pipeline_tag: text-to-speech
14
  base_model: ResembleAI/chatterbox
 
15
  ---
16
 
17
+ # SonicVox-Multilingual
18
+
19
+ ๐ŸŽ™๏ธ **SonicVox-Multilingual** is a multilingual Text-to-Speech (TTS) model
20
+ built on top of **Chatterbox Multilingual** by Resemble AI.
21
+
22
+ It is designed to generate natural, expressive speech across multiple
23
+ languages with support for **zero-shot voice cloning** using a reference
24
+ audio sample.
25
+
26
+ ## โœจ Features
27
 
28
+ - ๐ŸŒ Multilingual Text-to-Speech
29
+ - ๐ŸŽ™๏ธ Natural and expressive speech generation
30
+ - ๐Ÿ—ฃ๏ธ Zero-shot voice cloning
31
+ - ๐Ÿ”Š High-quality speech synthesis
32
+ - ๐ŸŽš๏ธ Voice conditioning using reference audio
33
+ - โšก GPU-accelerated inference
34
+ - ๐Ÿงฉ Based on Chatterbox Multilingual V3
35
 
36
+ ## ๐ŸŒŽ Supported Languages
37
 
38
+ SonicVox-Multilingual is based on the multilingual Chatterbox model.
39
 
40
+ Supported languages depend on the underlying checkpoint and include
41
+ multiple languages such as:
42
 
43
+ - English
44
+ - Spanish
45
+ - French
46
+ - German
47
+ - Italian
48
+ - Portuguese
49
+ - Hindi
50
+ - Telugu
51
+ - Tamil
52
+ - Kannada
53
+ - Malayalam
54
+ - Chinese
55
+ - Japanese
56
+ - Korean
57
+ - Arabic
58
+ - Russian
59
+ - Turkish
60
+ - Polish
61
 
62
+ > Always verify the supported language IDs with the specific Chatterbox
63
+ > checkpoint being used.
64
 
65
+ ## ๐ŸŽค Voice Cloning
 
 
66
 
67
+ SonicVox-Multilingual supports voice cloning using a reference audio file.
68
 
69
+ Example:
 
70
 
71
+ ```python
72
+ import torchaudio as ta
73
+ from chatterbox.mtl_tts import ChatterboxMultilingualTTS
74
 
75
+ model = ChatterboxMultilingualTTS.from_pretrained(device="cuda")
 
 
 
 
 
 
 
76
 
77
+ text = "Welcome to SonicVox Multilingual."
78
 
79
+ wav = model.generate(
80
+ text,
81
+ language_id="en",
82
+ audio_prompt_path="reference.wav"
83
+ )
84
 
85
+ ta.save("output.wav", wav, model.sr)