Joshuant commited on
Commit
3e018bc
·
verified ·
1 Parent(s): 27eff36

improve model card: discoverability tags, benefits, quickstart

Browse files
Files changed (1) hide show
  1. README.md +61 -55
README.md CHANGED
@@ -5,13 +5,21 @@ language:
5
  library_name: transformers
6
  pipeline_tag: text-to-speech
7
  base_model: OpenMOSS-Team/MOSS-TTS-Local-Transformer
 
 
8
  tags:
9
  - text-to-speech
10
  - tts
11
- - moss-tts
12
  - indian-english
13
- - lora
14
  - voice-agent
 
 
 
 
 
 
15
  model-index:
16
  - name: roxi-tts-pro
17
  results:
@@ -27,52 +35,46 @@ model-index:
27
  value: 0.18
28
  ---
29
 
30
- # Roxi-TTS Pro (1.7B), Indian-English voice
 
 
 
 
 
31
 
32
- A larger, higher-fidelity member of the Roxi-TTS line. This is a LoRA fine-tune of the
33
- 1.7B MOSS-TTS-Local-Transformer on about 4 hours of a single studio speaker, producing a
34
- clear Indian-English voice for the VozVox voice-agent platform.
35
 
36
- Compared with the 0.1B roxi-tts-v3.1, this 1.7B version is more intelligible (lower WER)
37
- and more stable (fewer under-generations), at the cost of generation speed. See the
38
- Performance section before deploying.
 
 
 
 
39
 
40
- ## Model at a glance
41
 
42
  | Field | Value |
43
  |---|---|
44
  | Base model | OpenMOSS-Team/MOSS-TTS-Local-Transformer (1.7B, Apache-2.0) |
45
- | Audio tokenizer | OpenMOSS-Team/MOSS-Audio-Tokenizer (32 codebooks, Apache-2.0) |
46
  | Method | LoRA (PEFT), r=32, alpha=64, merged into the base weights |
47
- | LoRA targets | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
48
- | Training data | about 4 hours, single IndicTTS-English speaker, 2371 clips |
49
  | Output | 24 kHz mono |
50
  | Speaker similarity | 0.97 (WavLM-SV cosine to held-out target) |
51
  | Intelligibility WER | 0.18 (Whisper-base.en on generated audio) |
 
52
 
53
- ## Performance and deployability
54
-
55
- Measured on a single 16 GB GPU (bf16, SDPA attention):
56
-
57
- - Real-time factor about 2.5, that is roughly 13 seconds of compute per 5 seconds of audio.
58
- - Peak GPU memory about 13.4 GB during generation.
59
-
60
- This model is well suited to offline or pre-rendered speech and to a premium quality tier.
61
- For live, low-latency turn taking, prefer the 0.1B roxi-tts-v3.1, or optimize this model
62
- with quantization, torch.compile, a faster GPU, or by caching common phrases.
63
 
64
- ## Requirements
65
-
66
- This model uses MOSS-TTS custom code and is built for transformers 4.57.1. Newer versions
67
- may break generation. Pin the version and install the MOSS-TTS repository so the model class
68
- is importable.
69
 
70
  ```bash
71
  pip install "transformers==4.57.1" torch torchaudio soundfile librosa peft
72
  git clone https://github.com/OpenMOSS/MOSS-TTS.git
73
  ```
74
 
75
- ## Usage
76
 
77
  ```python
78
  import sys, torch, soundfile as sf
@@ -105,43 +107,47 @@ audio = processor.decode(out)[0].audio_codes_list[0]
105
  sf.write("out.wav", audio.float().cpu().numpy(), processor.model_config.sampling_rate)
106
  ```
107
 
108
- Generation is autoregressive and occasionally under-generates. If a clip is short, generate
109
- two or three times and keep the longest, then trim leading and trailing silence.
110
-
111
- Tips: write numbers as words, spell brand names phonetically (for example Voz Vox), avoid raw
112
- abbreviations, and keep sentences to about twelve words for reliability. Do not raise
113
- max_new_tokens far above the default, since the codec decode grows quadratically in memory.
114
 
115
- ## The Roxi line
116
 
117
- | Model | Base | Speaker sim | WER | Notes |
118
  |---|---|---|---|---|
119
- | roxi-tts-v2 | MOSS-TTS-Nano 0.1B | 0.96 | 0.26 | speaker A, about 50 min |
120
- | roxi-tts-v3 | MOSS-TTS-Nano 0.1B | 0.96 | 0.29 | speaker B, about 70 min |
121
- | roxi-tts-v3.1 | MOSS-TTS-Nano 0.1B | 0.96 | 0.33 | speaker B, about 4 hours |
122
- | roxi-tts-pro (this) | MOSS-TTS-Local 1.7B | 0.97 | 0.18 | speaker B, about 4 hours, most intelligible |
123
-
124
- ## Training
125
-
126
- - Data: a single expressive speaker isolated from SPRINGLab/IndicTTS-English via WavLM-SV
127
- speaker clustering across dataset shards, about 2371 clips, about 4 hours, studio quality,
128
- proper-case transcripts.
129
- - LoRA r=32 alpha=64 on attention and MLP projections, bf16, learning rate 1e-4 with cosine
130
- schedule, gradient accumulation 8, gradient checkpointing, 3 epochs. The epoch-1 checkpoint
131
- was selected because later epochs overfit the reading style and raised WER.
132
- - Evaluation on held-out prompts: speaker similarity with microsoft/wavlm-base-plus-sv,
133
- intelligibility with openai/whisper-base.en on the actual generated audio.
 
 
 
134
 
135
  ## Limitations
136
 
137
  - The training data is read speech, so delivery is somewhat formal rather than fully
138
  conversational.
139
- - Not real-time on a single consumer GPU. See the Performance section.
140
  - Stochastic under-generation. Use the retry approach and keep sentences short.
141
- - Style and emotion control are not reliable. The voice is neutral.
 
142
  - Requires transformers 4.57.1.
143
 
144
- ## Attribution and license
145
 
146
  Released under Apache-2.0. Built on MOSS-TTS-Local-Transformer (Apache-2.0) and its audio
147
  tokenizer (Apache-2.0). Training data is the IIT-Madras Indic TTS English set accessed via
 
5
  library_name: transformers
6
  pipeline_tag: text-to-speech
7
  base_model: OpenMOSS-Team/MOSS-TTS-Local-Transformer
8
+ datasets:
9
+ - SPRINGLab/IndicTTS-English
10
  tags:
11
  - text-to-speech
12
  - tts
13
+ - speech-synthesis
14
  - indian-english
15
+ - indian-accent
16
  - voice-agent
17
+ - voice-assistant
18
+ - customer-support
19
+ - conversational
20
+ - moss-tts
21
+ - lora
22
+ - audio
23
  model-index:
24
  - name: roxi-tts-pro
25
  results:
 
35
  value: 0.18
36
  ---
37
 
38
+ # Roxi-TTS Pro (1.7B): Indian-English text-to-speech
39
+
40
+ Roxi-TTS Pro is a 1.7B text-to-speech model that speaks in a clear, natural Indian-English
41
+ accent. It is built for customer-support calls and website voice assistants, and it is the
42
+ highest-quality voice in the Roxi line. If you need an Indian-English voice that sounds
43
+ warm, professional, and telephony-ready, start here.
44
 
45
+ ## Why Roxi-TTS Pro
 
 
46
 
47
+ - Natural Indian-English accent, not a generic English voice with an accent bolted on.
48
+ - Highest intelligibility in the Roxi line: word error rate 0.18 (Whisper-base.en), and
49
+ strong speaker consistency 0.97 (WavLM-SV).
50
+ - Stable generation with fewer cut-offs than the smaller models, so most lines are usable
51
+ on the first try.
52
+ - 24 kHz output, single consistent branded voice.
53
+ - Apache-2.0 base models, so it is commercially permissive end to end.
54
 
55
+ ## Quick facts
56
 
57
  | Field | Value |
58
  |---|---|
59
  | Base model | OpenMOSS-Team/MOSS-TTS-Local-Transformer (1.7B, Apache-2.0) |
60
+ | Audio tokenizer | OpenMOSS-Team/MOSS-Audio-Tokenizer (Apache-2.0) |
61
  | Method | LoRA (PEFT), r=32, alpha=64, merged into the base weights |
62
+ | Training data | About 4 hours, single IndicTTS-English speaker, 2371 clips |
 
63
  | Output | 24 kHz mono |
64
  | Speaker similarity | 0.97 (WavLM-SV cosine to held-out target) |
65
  | Intelligibility WER | 0.18 (Whisper-base.en on generated audio) |
66
+ | Speed | Real-time factor about 2.5 on a 16 GB GPU (best for offline or premium audio) |
67
 
68
+ ## Install
 
 
 
 
 
 
 
 
 
69
 
70
+ Built for transformers 4.57.1. Install the MOSS-TTS repository so the model class is importable.
 
 
 
 
71
 
72
  ```bash
73
  pip install "transformers==4.57.1" torch torchaudio soundfile librosa peft
74
  git clone https://github.com/OpenMOSS/MOSS-TTS.git
75
  ```
76
 
77
+ ## Quick start
78
 
79
  ```python
80
  import sys, torch, soundfile as sf
 
107
  sf.write("out.wav", audio.float().cpu().numpy(), processor.model_config.sampling_rate)
108
  ```
109
 
110
+ Tips for reliable output: write numbers as words, spell brand names phonetically (for
111
+ example Voz Vox), avoid raw abbreviations, and keep sentences to about twelve words.
112
+ Generation is autoregressive and can occasionally under-generate, so if a clip is short,
113
+ generate two or three times and keep the longest, then trim leading and trailing silence.
114
+ Do not raise max_new_tokens far above the default, since the codec decode grows
115
+ quadratically in memory.
116
 
117
+ ## Which Roxi voice should I use
118
 
119
+ | Model | Base | Best for | Speaker sim | WER |
120
  |---|---|---|---|---|
121
+ | roxi-tts-pro (this) | MOSS-TTS-Local 1.7B | Highest quality, offline or premium audio | 0.97 | 0.18 |
122
+ | roxi-tts-v3.1 | MOSS-TTS-Nano 0.1B | Real-time, live voice agents | 0.96 | 0.33 |
123
+
124
+ Use Roxi-TTS Pro when quality matters most and you can pre-render or afford a GPU. Use the
125
+ smaller 0.1B voice when you need real-time, low-latency speech for a live agent.
126
+
127
+ ## Performance and deployability
128
+
129
+ Measured on a single 16 GB GPU (bf16, SDPA attention): real-time factor about 2.5, that is
130
+ roughly 13 seconds of compute per 5 seconds of audio, with peak GPU memory about 13.4 GB.
131
+ This makes Roxi-TTS Pro well suited to offline or pre-rendered speech and to a premium
132
+ quality tier. For live, low-latency turn taking, prefer the 0.1B roxi-tts-v3.1, or optimize
133
+ this model with quantization, torch.compile, a faster GPU, or by caching common phrases.
134
+
135
+ ## Intended use
136
+
137
+ Indian-English text to speech for customer-support calls and website voice assistants:
138
+ natural, warm or professional, and telephony aware. Single-speaker branded voice.
139
 
140
  ## Limitations
141
 
142
  - The training data is read speech, so delivery is somewhat formal rather than fully
143
  conversational.
144
+ - Not real-time on a single consumer GPU. See Performance.
145
  - Stochastic under-generation. Use the retry approach and keep sentences short.
146
+ - Style and emotion control are not reliable. The voice is neutral. For emotion, see
147
+ roxi-tts-emotion.
148
  - Requires transformers 4.57.1.
149
 
150
+ ## License and attribution
151
 
152
  Released under Apache-2.0. Built on MOSS-TTS-Local-Transformer (Apache-2.0) and its audio
153
  tokenizer (Apache-2.0). Training data is the IIT-Madras Indic TTS English set accessed via