prince-canuma commited on
Commit
0d2cf05
Β·
verified Β·
1 Parent(s): bf2c7f6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +81 -26
README.md CHANGED
@@ -1,49 +1,104 @@
1
  ---
2
- library_name: mlx-audio
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  base_model:
4
  - ResembleAI/chatterbox
 
 
5
  tags:
 
 
 
 
6
  - mlx
7
- pipeline_tag: text-to-speech
8
  ---
9
 
10
- # litmudoc/Chatterbox-Multilingual-MLX-v2-fp16
11
-
12
- This model was converted to MLX format from [ResembleAI/chatterbox](https://huggingface.co/ResembleAI/chatterbox) using [mlx-audio](https://github.com/Blaizzy/mlx-audio) version **0.2.10**.
13
 
14
- **Note:** This model requires the S3Tokenizer weights from [mlx-community/S3TokenizerV2](https://huggingface.co/mlx-community/S3TokenizerV2), which will be downloaded automatically.
15
 
16
- ## Use with mlx-audio
 
 
17
 
 
18
  ```bash
19
- pip install -U git+https://github.com/litmudoc/mlx-audio.git@main mlx-audio
 
 
 
 
 
 
 
 
 
 
20
  ```
21
 
22
- ### Command line
 
 
23
 
24
- ```bash
25
- curl -L -o ko.wav https://huggingface.co/litmudoc/Chatterbox-Multilingual-MLX-v2-fp16/blob/main/ko.wav
26
-
27
- mlx_audio.tts.generate \
28
- --model litmudoc/Chatterbox-Multilingual-MLX-v2-fp16 \
29
- --text ", μ§€λ‚œλ‹¬ μš°λ¦¬λŠ” 유튜브 μ±„λ„μ—μ„œ 이십얡 μ‘°νšŒμˆ˜λΌλŠ” μƒˆλ‘œμš΄ μ΄μ •ν‘œμ— λ„λ‹¬ν–ˆμŠ΅λ‹ˆλ‹€." \
30
- --lang_code ko \
31
- --ref_audio ko.wav \
32
- --ref_text "μš°λ¦¬λŠ” μ •λ§λ‘œ ν—ˆλ¦„ν•œ ν˜Έν…”μ— λ¬΅μ—ˆμ§€λ§Œ, κ·Έλž˜λ„ ν–‰λ³΅ν–ˆλ‹€." \
33
- --verbose --play
34
  ```
35
 
36
- ### Python
 
 
37
 
38
  ```python
39
  from mlx_audio.tts.generate import generate_audio
40
 
41
  generate_audio(
42
- text=", μ§€λ‚œλ‹¬ μš°λ¦¬λŠ” 유튜브 μ±„λ„μ—μ„œ 이십얡 μ‘°νšŒμˆ˜λΌλŠ” μƒˆλ‘œμš΄ μ΄μ •ν‘œμ— λ„λ‹¬ν–ˆμŠ΅λ‹ˆλ‹€.",
43
- model="litmudoc/Chatterbox-Multilingual-MLX-v2-fp16",
44
- lang_code="ko",
45
- ref_audio="ko.wav",
46
- ref_text="μš°λ¦¬λŠ” μ •λ§λ‘œ ν—ˆλ¦„ν•œ ν˜Έν…”μ— λ¬΅μ—ˆμ§€λ§Œ, κ·Έλž˜λ„ ν–‰λ³΅ν–ˆλ‹€.",
47
- file_prefix="output",
48
  )
49
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: apache-2.0
3
+ language:
4
+ - ar
5
+ - da
6
+ - de
7
+ - el
8
+ - en
9
+ - es
10
+ - fi
11
+ - fr
12
+ - he
13
+ - hi
14
+ - it
15
+ - ja
16
+ - ko
17
+ - ms
18
+ - nl
19
+ - no
20
+ - pl
21
+ - pt
22
+ - ru
23
+ - sv
24
+ - sw
25
+ - tr
26
+ - zh
27
  base_model:
28
  - ResembleAI/chatterbox
29
+ pipeline_tag: text-to-speech
30
+ library_name: mlx-audio
31
  tags:
32
+ - text-to-speech
33
+ - speech
34
+ - speech generation
35
+ - voice cloning
36
  - mlx
37
+ - tts
38
  ---
39
 
40
+ # mlx-community/chatterbox-6bit
41
+ This model was converted to MLX format from [`ResembleAI/chatterbox`](https://huggingface.co/ResembleAI/chatterbox) using mlx-audio version **0.2.7**.
42
+ Refer to the [original model card](https://huggingface.co/ResembleAI/chatterbox) for more details on the model.
43
 
44
+ ## Use with mlx
45
 
46
+ ```bash
47
+ pip install -U mlx-audio
48
+ ```
49
 
50
+ ### CLI Example:
51
  ```bash
52
+ python -m mlx_audio.tts.generate --model mlx-community/chatterbox-6bit --text "Hello, this is a test."
53
+ ```
54
+ ### Python Example:
55
+ ```python
56
+ from mlx_audio.tts.utils import load_model
57
+ from mlx_audio.tts.generate import generate_audio
58
+ model = load_model("mlx-community/chatterbox-6bit")
59
+ generate_audio(
60
+ model=model, text="Hello, this is a test.",
61
+ file_prefix="test_audio",
62
+ )
63
  ```
64
 
65
+ ### Voice Cloning:
66
+ ```python
67
+ from mlx_audio.tts.generate import generate_audio
68
 
69
+ generate_audio(
70
+ text="Hello, this is a test of Chatterbox TTS.",
71
+ model="mlx-community/chatterbox-6bit",
72
+ ref_audio="path_to_file.wav",
73
+ ref_text="Text transcript", # Optional
74
+ file_prefix="vc_output",
75
+ )
 
 
 
76
  ```
77
 
78
+ ## Multilingual Quickstart
79
+
80
+ Generate speech in 23 languages by specifying the `lang_code` parameter. The model automatically handles pronunciation, intonation, and language-specific phonemes.
81
 
82
  ```python
83
  from mlx_audio.tts.generate import generate_audio
84
 
85
  generate_audio(
86
+ text="OlΓ‘, tudo bem?",
87
+ model="mlx-community/chatterbox-6bit",
88
+ lang_code="pt",
89
+ file_prefix="multilingual_output",
 
 
90
  )
91
  ```
92
+
93
+ ### Supported Languages
94
+
95
+ | Code | Language | Code | Language | Code | Language |
96
+ |------|------------|------|------------|------|------------|
97
+ | `ar` | Arabic | `he` | Hebrew | `no` | Norwegian |
98
+ | `da` | Danish | `hi` | Hindi | `pl` | Polish |
99
+ | `de` | German | `it` | Italian | `pt` | Portuguese |
100
+ | `el` | Greek | `ja` | Japanese | `ru` | Russian |
101
+ | `en` | English | `ko` | Korean | `sv` | Swedish |
102
+ | `es` | Spanish | `ms` | Malay | `sw` | Swahili |
103
+ | `fi` | Finnish | `nl` | Dutch | `tr` | Turkish |
104
+ | `fr` | French | `zh` | Chinese | | |