fdemelo commited on
Commit
06d804f
·
verified ·
1 Parent(s): ba0271a

x-vits Expresso ex02 voice (ONNX), model card and samples

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ samples/enunciated_recording.mp3 filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,160 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ language:
4
+ - en
5
+ pipeline_tag: text-to-speech
6
+ library_name: onnx
7
+ tags:
8
+ - text-to-speech
9
+ - tts
10
+ - piper
11
+ - vits
12
+ - onnx
13
+ - prosody
14
+ - expressive-speech
15
+ - emotional-tts
16
+ - flow-matching
17
+ datasets:
18
+ - ylacombe/expresso
19
+ ---
20
+
21
+ # x-vits · Expresso ex02 (English, expressive)
22
+
23
+ An expressive **[Piper](https://github.com/rhasspy/piper)/VITS** voice that can speak in **six styles** and
24
+ **stress the words you mark**, or the words it thinks matter. Made with
25
+ **[x-vits](https://github.com/femelo/x-vits)**, a recipe that adds prosody modelling (intonation, rhythm,
26
+ emphasis) to small, fast VITS voices. It's about 25M parameters in all, exported to ONNX, and runs on a CPU.
27
+
28
+ ## What's different from a plain Piper voice
29
+
30
+ - 🎭 **Styles:** `default`, `happy`, `sad`, `confused`, `enunciated`, `laughing`.
31
+ - 💥 **Emphasis:** write `*word*` to stress a word. Without marks, a small model predicts which words to stress.
32
+ - 🌊 **Sampled intonation:** a flow-matching model picks a plausible pitch, voicing and energy contour per phoneme, instead of averaging every sentence into the same melody.
33
+ - 🎚️ **Style-aware timing:** sad slows down, enunciated takes its pauses.
34
+
35
+ ## 🎧 Samples
36
+
37
+ Held-out sentences, never seen in training. **Piper** is the same speaker fine-tuned with plain
38
+ Piper/VITS; **x-vits** adds the full recipe, synthesised by the ONNX files in this repository. The
39
+ **recording** is the real take in that style, for reference. All samples use the same random seed
40
+ and none were cherry-picked.
41
+
42
+ <table>
43
+ <tr><th>style</th><th>text</th><th>Piper</th><th>x-vits</th><th>recording</th></tr>
44
+ <tr><td><b>default</b></td><td>I'm more of an all terrain vehicle type.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/default_piper.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/default_xvits.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/default_recording.mp3"></audio></td></tr>
45
+ <tr><td><b>happy</b></td><td>We didn't get the weather right, guys.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/happy_piper.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/happy_xvits.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/happy_recording.mp3"></audio></td></tr>
46
+ <tr><td><b>sad</b></td><td>No, in Salt Lake City, today, it won't snow.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/sad_piper.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/sad_xvits.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/sad_recording.mp3"></audio></td></tr>
47
+ <tr><td><b>confused</b></td><td>Is young adult fiction very popular?</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/confused_piper.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/confused_xvits.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/confused_recording.mp3"></audio></td></tr>
48
+ <tr><td><b>enunciated</b></td><td>One PM to nine PM, it'll be cloudy, and in the low fifties.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/enunciated_piper.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/enunciated_xvits.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/enunciated_recording.mp3"></audio></td></tr>
49
+ <tr><td><b>laughing</b></td><td>Their moment had come, or so they said.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/laughing_piper.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/laughing_xvits.mp3"></audio></td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/laughing_recording.mp3"></audio></td></tr>
50
+ </table>
51
+
52
+ Emphasis, by moving the `*mark*` (style `default`):
53
+
54
+ <table>
55
+ <tr><th>text</th><th>x-vits</th></tr>
56
+ <tr><td>I never said she stole my money.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/emphasis_0.mp3"></audio></td></tr>
57
+ <tr><td><b>*I*</b> never said she stole my money.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/emphasis_1.mp3"></audio></td></tr>
58
+ <tr><td>I never said <b>*she*</b> stole my money.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/emphasis_2.mp3"></audio></td></tr>
59
+ <tr><td>I never said she stole my <b>*money*</b>.</td><td><audio controls preload="none" src="https://huggingface.co/fdemelo/x-vits/resolve/main/samples/emphasis_3.mp3"></audio></td></tr>
60
+ </table>
61
+
62
+ ## 🚀 Usage
63
+
64
+ ```bash
65
+ sudo apt install libespeak-ng1 # phonemizer backend (macOS: brew install espeak-ng)
66
+ pip install "x-vits @ git+https://github.com/femelo/x-vits"
67
+ ```
68
+
69
+ ```python
70
+ import soundfile as sf
71
+ from xvits.runtime import XVits
72
+
73
+ voice = XVits.from_pretrained("fdemelo/x-vits")
74
+
75
+ audio = voice.synthesize("We didn't get the weather right, guys.", style="happy")
76
+ sf.write("happy.wav", audio, voice.sample_rate)
77
+
78
+ # explicit emphasis, a different style, a fixed seed and a faster CPU setting
79
+ audio = voice.synthesize("I never said *she* stole my money.", style="confused", seed=7, steps=4)
80
+ sf.write("confused.wav", audio, voice.sample_rate)
81
+ ```
82
+
83
+ Or from the command line:
84
+
85
+ ```bash
86
+ xvits-speak --model fdemelo/x-vits --text "Is it *really* you?" --style sad --out sad.wav
87
+ ```
88
+
89
+ | option | values | |
90
+ |---|---|---|
91
+ | `style` | `default` `happy` `sad` `confused` `enunciated` `laughing` | none gives a neutral mix |
92
+ | `emphasis` | `auto` · `marked` · `none` | `auto`: marks if present, otherwise predicted |
93
+ | `emphasis_scale` | default 1.5 | 1.0 = average emphasis in the recordings |
94
+ | `length_scale` | default 1.08 | larger is slower |
95
+ | `steps` | default 16 | 4 is about 2.5× faster on CPU and changes pitch by about 0.1 semitone |
96
+ | `seed` | int | reproducible output |
97
+
98
+ Speed with onnxruntime on 4 CPU threads: real-time factor 0.60 (16 steps) or 0.24 (4 steps).
99
+
100
+ ## 📦 Files
101
+
102
+ | file | role | size |
103
+ |---|---|---|
104
+ | `encoder.onnx` | phonemes → text states | 26 MB |
105
+ | `predictor.onnx` | one step of the flow-matching prosody predictor (style-conditioned) | 31 MB |
106
+ | `durations.onnx` | style-aware stochastic duration predictor | 2.5 MB |
107
+ | `acoustic.onnx` | prosody-conditioned prior, length regulation, flow, HiFi-GAN | 36 MB |
108
+ | `prominence.onnx` | predicts which words to stress | 4.8 MB |
109
+ | `dp_cond.npz`, `emphasis_template.json` | style vectors for the duration predictor; the emphasis shape | small |
110
+ | `runtime.json`, `config.json` | runtime settings; Piper phoneme map (eSpeak `en`) | |
111
+
112
+ These graphs are driven by `xvits.runtime`, so they aren't drop-in files for Piper's own runtime.
113
+ The export was verified against PyTorch end to end (maximum difference about 1e-4 at several sentence lengths).
114
+
115
+ ## 🧪 Training
116
+
117
+ - **Base:** Piper's LJSpeech `medium` checkpoint (LJSpeech is public domain).
118
+ - **Data:** [Expresso](https://speechbot.github.io/expresso/) speaker **ex02**, read speech in six styles,
119
+ 2.2 h after leaving out whisper, singing, long-form and `<laugh>` rows. 245 utterances held out.
120
+ - **Recipe** ([x-vits](https://github.com/femelo/x-vits)):
121
+ 1. fine-tune the base checkpoint on ex02;
122
+ 2. extract per-phoneme duration, pitch and energy targets;
123
+ 3. fine-tune with a zero-initialised prosody input to the prior, including frame-level interpolation;
124
+ 4. train an OT-CFM prosody predictor with a style input;
125
+ 5. train a style-conditioned duration predictor;
126
+ 6. train a word-prominence student on acoustic labels;
127
+ 7. fit an emphasis template anchored on the stressed vowel, from Expresso's `*emphasis*` marks.
128
+
129
+ **Held-out results** (pitch match is the correlation of the pitch contour with the recording of the same sentence):
130
+
131
+ | | pitch match | WER (Whisper) | UTMOS |
132
+ |---|---|---|---|
133
+ | Piper, fine-tuned | 0.35 | 1.2 % | 4.08 |
134
+ | **x-vits** | **0.44** | **0.6 %** | 4.04 |
135
+
136
+ On marked words, the pitch peak rises from +2.3 to +4.6 semitones above the sentence median (the recordings: +5.3).
137
+ The prominence student finds the recording's main stress 42 % of the time, against 34 % for "last word" and 15 % for a random word.
138
+
139
+ ## ⚠️ License and limitations
140
+
141
+ - **CC BY-NC 4.0.** Expresso is licensed for non-commercial use only, and so is this voice. The x-vits code is MIT.
142
+ - English, one female speaker, read speech. Styles are as recorded in Expresso; there are no other emotions.
143
+ - Laughing is rendered through intonation and timing; the voice doesn't produce actual laughter.
144
+ - Questions sometimes end with a smaller rise than a natural reading.
145
+ - Don't use this voice to impersonate real people or to mislead listeners.
146
+
147
+ ## Citation
148
+
149
+ If you use this voice, please cite Expresso:
150
+
151
+ ```bibtex
152
+ @inproceedings{nguyen2023expresso,
153
+ title = {EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis},
154
+ author = {Nguyen, Tu Anh and Hsu, Wei-Ning and D'Avirro, Antony and Shi, Bowen and Gat, Itai and
155
+ Fazel-Zarani, Maryam and Remez, Tal and Copet, Jade and Synnaeve, Gabriel and Hassid, Michael and
156
+ Kreuk, Felix and Adi, Yossi and Dupoux, Emmanuel},
157
+ booktitle = {Interspeech},
158
+ year = {2023}
159
+ }
160
+ ```
acoustic.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ec55c955c671a7ca81f3df0c6d6c07a278f8154849ef0bce3cf32677e629382
3
+ size 36045993
config.json ADDED
@@ -0,0 +1,497 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "dataset": "ljspeech",
3
+ "audio": {
4
+ "sample_rate": 22050,
5
+ "quality": "medium"
6
+ },
7
+ "espeak": {
8
+ "voice": "en"
9
+ },
10
+ "language": {
11
+ "code": "en_US"
12
+ },
13
+ "inference": {
14
+ "noise_scale": 0.667,
15
+ "length_scale": 1,
16
+ "noise_w": 0.8
17
+ },
18
+ "phoneme_type": "espeak",
19
+ "phoneme_map": {},
20
+ "phoneme_id_map": {
21
+ " ": [
22
+ 3
23
+ ],
24
+ "!": [
25
+ 4
26
+ ],
27
+ "\"": [
28
+ 150
29
+ ],
30
+ "#": [
31
+ 149
32
+ ],
33
+ "$": [
34
+ 2
35
+ ],
36
+ "'": [
37
+ 5
38
+ ],
39
+ "(": [
40
+ 6
41
+ ],
42
+ ")": [
43
+ 7
44
+ ],
45
+ ",": [
46
+ 8
47
+ ],
48
+ "-": [
49
+ 9
50
+ ],
51
+ ".": [
52
+ 10
53
+ ],
54
+ "0": [
55
+ 130
56
+ ],
57
+ "1": [
58
+ 131
59
+ ],
60
+ "2": [
61
+ 132
62
+ ],
63
+ "3": [
64
+ 133
65
+ ],
66
+ "4": [
67
+ 134
68
+ ],
69
+ "5": [
70
+ 135
71
+ ],
72
+ "6": [
73
+ 136
74
+ ],
75
+ "7": [
76
+ 137
77
+ ],
78
+ "8": [
79
+ 138
80
+ ],
81
+ "9": [
82
+ 139
83
+ ],
84
+ ":": [
85
+ 11
86
+ ],
87
+ ";": [
88
+ 12
89
+ ],
90
+ "?": [
91
+ 13
92
+ ],
93
+ "X": [
94
+ 156
95
+ ],
96
+ "^": [
97
+ 1
98
+ ],
99
+ "_": [
100
+ 0
101
+ ],
102
+ "a": [
103
+ 14
104
+ ],
105
+ "b": [
106
+ 15
107
+ ],
108
+ "c": [
109
+ 16
110
+ ],
111
+ "d": [
112
+ 17
113
+ ],
114
+ "e": [
115
+ 18
116
+ ],
117
+ "f": [
118
+ 19
119
+ ],
120
+ "g": [
121
+ 154
122
+ ],
123
+ "h": [
124
+ 20
125
+ ],
126
+ "i": [
127
+ 21
128
+ ],
129
+ "j": [
130
+ 22
131
+ ],
132
+ "k": [
133
+ 23
134
+ ],
135
+ "l": [
136
+ 24
137
+ ],
138
+ "m": [
139
+ 25
140
+ ],
141
+ "n": [
142
+ 26
143
+ ],
144
+ "o": [
145
+ 27
146
+ ],
147
+ "p": [
148
+ 28
149
+ ],
150
+ "q": [
151
+ 29
152
+ ],
153
+ "r": [
154
+ 30
155
+ ],
156
+ "s": [
157
+ 31
158
+ ],
159
+ "t": [
160
+ 32
161
+ ],
162
+ "u": [
163
+ 33
164
+ ],
165
+ "v": [
166
+ 34
167
+ ],
168
+ "w": [
169
+ 35
170
+ ],
171
+ "x": [
172
+ 36
173
+ ],
174
+ "y": [
175
+ 37
176
+ ],
177
+ "z": [
178
+ 38
179
+ ],
180
+ "æ": [
181
+ 39
182
+ ],
183
+ "ç": [
184
+ 40
185
+ ],
186
+ "ð": [
187
+ 41
188
+ ],
189
+ "ø": [
190
+ 42
191
+ ],
192
+ "ħ": [
193
+ 43
194
+ ],
195
+ "ŋ": [
196
+ 44
197
+ ],
198
+ "œ": [
199
+ 45
200
+ ],
201
+ "ǀ": [
202
+ 46
203
+ ],
204
+ "ǁ": [
205
+ 47
206
+ ],
207
+ "ǂ": [
208
+ 48
209
+ ],
210
+ "ǃ": [
211
+ 49
212
+ ],
213
+ "ɐ": [
214
+ 50
215
+ ],
216
+ "ɑ": [
217
+ 51
218
+ ],
219
+ "ɒ": [
220
+ 52
221
+ ],
222
+ "ɓ": [
223
+ 53
224
+ ],
225
+ "ɔ": [
226
+ 54
227
+ ],
228
+ "ɕ": [
229
+ 55
230
+ ],
231
+ "ɖ": [
232
+ 56
233
+ ],
234
+ "ɗ": [
235
+ 57
236
+ ],
237
+ "ɘ": [
238
+ 58
239
+ ],
240
+ "ə": [
241
+ 59
242
+ ],
243
+ "ɚ": [
244
+ 60
245
+ ],
246
+ "ɛ": [
247
+ 61
248
+ ],
249
+ "ɜ": [
250
+ 62
251
+ ],
252
+ "ɞ": [
253
+ 63
254
+ ],
255
+ "ɟ": [
256
+ 64
257
+ ],
258
+ "ɠ": [
259
+ 65
260
+ ],
261
+ "ɡ": [
262
+ 66
263
+ ],
264
+ "ɢ": [
265
+ 67
266
+ ],
267
+ "ɣ": [
268
+ 68
269
+ ],
270
+ "ɤ": [
271
+ 69
272
+ ],
273
+ "ɥ": [
274
+ 70
275
+ ],
276
+ "ɦ": [
277
+ 71
278
+ ],
279
+ "ɧ": [
280
+ 72
281
+ ],
282
+ "ɨ": [
283
+ 73
284
+ ],
285
+ "ɪ": [
286
+ 74
287
+ ],
288
+ "ɫ": [
289
+ 75
290
+ ],
291
+ "ɬ": [
292
+ 76
293
+ ],
294
+ "ɭ": [
295
+ 77
296
+ ],
297
+ "ɮ": [
298
+ 78
299
+ ],
300
+ "ɯ": [
301
+ 79
302
+ ],
303
+ "ɰ": [
304
+ 80
305
+ ],
306
+ "ɱ": [
307
+ 81
308
+ ],
309
+ "ɲ": [
310
+ 82
311
+ ],
312
+ "ɳ": [
313
+ 83
314
+ ],
315
+ "ɴ": [
316
+ 84
317
+ ],
318
+ "ɵ": [
319
+ 85
320
+ ],
321
+ "ɶ": [
322
+ 86
323
+ ],
324
+ "ɸ": [
325
+ 87
326
+ ],
327
+ "ɹ": [
328
+ 88
329
+ ],
330
+ "ɺ": [
331
+ 89
332
+ ],
333
+ "ɻ": [
334
+ 90
335
+ ],
336
+ "ɽ": [
337
+ 91
338
+ ],
339
+ "ɾ": [
340
+ 92
341
+ ],
342
+ "ʀ": [
343
+ 93
344
+ ],
345
+ "ʁ": [
346
+ 94
347
+ ],
348
+ "ʂ": [
349
+ 95
350
+ ],
351
+ "ʃ": [
352
+ 96
353
+ ],
354
+ "ʄ": [
355
+ 97
356
+ ],
357
+ "ʈ": [
358
+ 98
359
+ ],
360
+ "ʉ": [
361
+ 99
362
+ ],
363
+ "ʊ": [
364
+ 100
365
+ ],
366
+ "ʋ": [
367
+ 101
368
+ ],
369
+ "ʌ": [
370
+ 102
371
+ ],
372
+ "ʍ": [
373
+ 103
374
+ ],
375
+ "ʎ": [
376
+ 104
377
+ ],
378
+ "ʏ": [
379
+ 105
380
+ ],
381
+ "ʐ": [
382
+ 106
383
+ ],
384
+ "ʑ": [
385
+ 107
386
+ ],
387
+ "ʒ": [
388
+ 108
389
+ ],
390
+ "ʔ": [
391
+ 109
392
+ ],
393
+ "ʕ": [
394
+ 110
395
+ ],
396
+ "ʘ": [
397
+ 111
398
+ ],
399
+ "ʙ": [
400
+ 112
401
+ ],
402
+ "ʛ": [
403
+ 113
404
+ ],
405
+ "ʜ": [
406
+ 114
407
+ ],
408
+ "ʝ": [
409
+ 115
410
+ ],
411
+ "ʟ": [
412
+ 116
413
+ ],
414
+ "ʡ": [
415
+ 117
416
+ ],
417
+ "ʢ": [
418
+ 118
419
+ ],
420
+ "ʦ": [
421
+ 155
422
+ ],
423
+ "ʰ": [
424
+ 145
425
+ ],
426
+ "ʲ": [
427
+ 119
428
+ ],
429
+ "ˈ": [
430
+ 120
431
+ ],
432
+ "ˌ": [
433
+ 121
434
+ ],
435
+ "ː": [
436
+ 122
437
+ ],
438
+ "ˑ": [
439
+ 123
440
+ ],
441
+ "˞": [
442
+ 124
443
+ ],
444
+ "ˤ": [
445
+ 146
446
+ ],
447
+ "̃": [
448
+ 141
449
+ ],
450
+ "̧": [
451
+ 140
452
+ ],
453
+ "̩": [
454
+ 144
455
+ ],
456
+ "̪": [
457
+ 142
458
+ ],
459
+ "̯": [
460
+ 143
461
+ ],
462
+ "̺": [
463
+ 152
464
+ ],
465
+ "̻": [
466
+ 153
467
+ ],
468
+ "β": [
469
+ 125
470
+ ],
471
+ "ε": [
472
+ 147
473
+ ],
474
+ "θ": [
475
+ 126
476
+ ],
477
+ "χ": [
478
+ 127
479
+ ],
480
+ "ᵻ": [
481
+ 128
482
+ ],
483
+ "↑": [
484
+ 151
485
+ ],
486
+ "↓": [
487
+ 148
488
+ ],
489
+ "ⱱ": [
490
+ 129
491
+ ]
492
+ },
493
+ "num_symbols": 256,
494
+ "num_speakers": 1,
495
+ "speaker_id_map": {},
496
+ "piper_version": "1.0.0"
497
+ }
dp_cond.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbdfeb499ce3d163709ae660cd8bc8d48aa452852bb543e069c4d4b7826d1081
3
+ size 5640
durations.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:097154c898fec9c766a01402ebca7540a3d6cade67c912a4c83b24bd37dce192
3
+ size 2506044
emphasis_template.json ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "anchor": "stressed vowel, token offsets",
3
+ "fit_range": [
4
+ -8,
5
+ 14
6
+ ],
7
+ "styles": {
8
+ "confused": {
9
+ "pitch": {
10
+ "A": 3.559,
11
+ "mu": 1.518,
12
+ "s_l": 2.118,
13
+ "s_r": 2.271,
14
+ "R": 0.991,
15
+ "k_r": 1.259,
16
+ "tau": 1.5
17
+ },
18
+ "dur": {
19
+ "C": 0.074,
20
+ "B": 0.311,
21
+ "mu": -1.408,
22
+ "s_l": 2.5,
23
+ "s_r": 3.415
24
+ },
25
+ "n_words": 63,
26
+ "rmse_pitch_st": 1.06,
27
+ "rmse_logdur": 0.087,
28
+ "pitch_from": "pooled (style shape does not fit)"
29
+ },
30
+ "default": {
31
+ "pitch": {
32
+ "A": 6.073,
33
+ "mu": 1.518,
34
+ "s_l": 2.118,
35
+ "s_r": 2.271,
36
+ "R": 1.192,
37
+ "k_r": 1.259,
38
+ "tau": 1.5
39
+ },
40
+ "dur": {
41
+ "C": 0.13,
42
+ "B": 0.215,
43
+ "mu": -1.408,
44
+ "s_l": 2.5,
45
+ "s_r": 3.415
46
+ },
47
+ "n_words": 176,
48
+ "rmse_pitch_st": 0.56,
49
+ "rmse_logdur": 0.077
50
+ },
51
+ "enunciated": {
52
+ "pitch": {
53
+ "A": 3.291,
54
+ "mu": 1.518,
55
+ "s_l": 2.118,
56
+ "s_r": 2.271,
57
+ "R": 2.31,
58
+ "k_r": 1.259,
59
+ "tau": 1.5
60
+ },
61
+ "dur": {
62
+ "C": 0.074,
63
+ "B": 0.395,
64
+ "mu": -1.408,
65
+ "s_l": 2.5,
66
+ "s_r": 3.415
67
+ },
68
+ "n_words": 48,
69
+ "rmse_pitch_st": 0.63,
70
+ "rmse_logdur": 0.125
71
+ },
72
+ "happy": {
73
+ "pitch": {
74
+ "A": 3.605,
75
+ "mu": 1.518,
76
+ "s_l": 2.118,
77
+ "s_r": 2.271,
78
+ "R": 0.643,
79
+ "k_r": 1.259,
80
+ "tau": 1.5
81
+ },
82
+ "dur": {
83
+ "C": -0.051,
84
+ "B": 0.432,
85
+ "mu": -1.408,
86
+ "s_l": 2.5,
87
+ "s_r": 3.415
88
+ },
89
+ "n_words": 56,
90
+ "rmse_pitch_st": 0.78,
91
+ "rmse_logdur": 0.11
92
+ },
93
+ "laughing": {
94
+ "pitch": {
95
+ "A": 3.559,
96
+ "mu": 1.518,
97
+ "s_l": 2.118,
98
+ "s_r": 2.271,
99
+ "R": 0.991,
100
+ "k_r": 1.259,
101
+ "tau": 1.5
102
+ },
103
+ "dur": {
104
+ "C": 0.11,
105
+ "B": 0.309,
106
+ "mu": -1.408,
107
+ "s_l": 2.5,
108
+ "s_r": 3.415
109
+ },
110
+ "n_words": 49,
111
+ "rmse_pitch_st": 1.16,
112
+ "rmse_logdur": 0.142,
113
+ "pitch_from": "pooled (style shape does not fit)"
114
+ },
115
+ "sad": {
116
+ "pitch": {
117
+ "A": 2.056,
118
+ "mu": 1.518,
119
+ "s_l": 2.118,
120
+ "s_r": 2.271,
121
+ "R": 0.731,
122
+ "k_r": 1.259,
123
+ "tau": 1.5
124
+ },
125
+ "dur": {
126
+ "C": 0.017,
127
+ "B": 0.325,
128
+ "mu": -1.408,
129
+ "s_l": 2.5,
130
+ "s_r": 3.415
131
+ },
132
+ "n_words": 52,
133
+ "rmse_pitch_st": 0.43,
134
+ "rmse_logdur": 0.104
135
+ }
136
+ },
137
+ "pitch": {
138
+ "A": 3.559,
139
+ "mu": 1.518,
140
+ "s_l": 2.118,
141
+ "s_r": 2.271,
142
+ "R": 0.991,
143
+ "k_r": 1.259,
144
+ "tau": 1.5
145
+ },
146
+ "dur": {
147
+ "C": 0.076,
148
+ "B": 0.302,
149
+ "mu": -1.408,
150
+ "s_l": 2.5,
151
+ "s_r": 3.415
152
+ },
153
+ "n_words": 444,
154
+ "rmse_pitch_st": 0.53,
155
+ "rmse_logdur": 0.081
156
+ }
encoder.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01a952e28b9f45f773d26f5c7e5c3ceb0f39402b9ee079d48379cdce1fa576de
3
+ size 25515134
predictor.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:242f7f482da7feac31b79852f22796b0eac8f6a2c6a10c90c8faeffd86988511
3
+ size 31049429
prominence.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e125b3e46cceea2d9f6a4018ac42fa1280c07b0be8e7c3847cdbc9b8735a87b9
3
+ size 4769869
runtime.json ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "x-vits-expresso-ex02",
3
+ "files": {
4
+ "encoder": "encoder.onnx",
5
+ "durations": "durations.onnx",
6
+ "acoustic": "acoustic.onnx",
7
+ "predictor": "predictor.onnx",
8
+ "prominence": "prominence.onnx",
9
+ "dp_cond": "dp_cond.npz",
10
+ "template": "emphasis_template.json"
11
+ },
12
+ "config": "config.json",
13
+ "sample_rate": 22050,
14
+ "hop_length": 256,
15
+ "inter_channels": 192,
16
+ "text_channels": 192,
17
+ "predictor": {
18
+ "stats": {
19
+ "lf0_mean": 5.490487575531006,
20
+ "lf0_std": 0.23599191009998322,
21
+ "energy_mean": 2.2885968685150146,
22
+ "energy_std": 1.3675802946090698,
23
+ "f0_range": [
24
+ 114.04395363547584,
25
+ 476.70372619628904
26
+ ],
27
+ "sample_rate": 22050,
28
+ "hop": 256,
29
+ "n_utts": 2254,
30
+ "logdur_mean": 1.2397095361668065,
31
+ "logdur_std": 0.5578987053044226
32
+ },
33
+ "styles": [
34
+ "confused",
35
+ "default",
36
+ "enunciated",
37
+ "happy",
38
+ "laughing",
39
+ "sad"
40
+ ],
41
+ "token_feats": 0
42
+ },
43
+ "durations_styles": [
44
+ "confused",
45
+ "default",
46
+ "enunciated",
47
+ "happy",
48
+ "laughing",
49
+ "sad"
50
+ ],
51
+ "frame_prosody": true,
52
+ "defaults": {
53
+ "noise_scale": 0.667,
54
+ "noise_scale_w": 0.4,
55
+ "length_scale": 1.08,
56
+ "pp_temperature": "0.8,0.7",
57
+ "pp_guidance": "1.0,2.0",
58
+ "style_temperature": {
59
+ "happy": "0.8,0.9"
60
+ },
61
+ "template_scale": 1.5,
62
+ "prominence_threshold": 0.5,
63
+ "pp_steps": 16
64
+ }
65
+ }
samples/confused_piper.mp3 ADDED
Binary file (33.9 kB). View file
 
samples/confused_recording.mp3 ADDED
Binary file (44.5 kB). View file
 
samples/confused_xvits.mp3 ADDED
Binary file (40.8 kB). View file
 
samples/default_piper.mp3 ADDED
Binary file (25.4 kB). View file
 
samples/default_recording.mp3 ADDED
Binary file (31.9 kB). View file
 
samples/default_xvits.mp3 ADDED
Binary file (25.4 kB). View file
 
samples/emphasis_0.mp3 ADDED
Binary file (26.4 kB). View file
 
samples/emphasis_1.mp3 ADDED
Binary file (27.3 kB). View file
 
samples/emphasis_2.mp3 ADDED
Binary file (27 kB). View file
 
samples/emphasis_3.mp3 ADDED
Binary file (28.6 kB). View file
 
samples/enunciated_piper.mp3 ADDED
Binary file (62.1 kB). View file
 
samples/enunciated_recording.mp3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a33634ed08174e4db689f16b580364e4569c0e1adc096f9bb68e09a25b0f22c
3
+ size 101502
samples/enunciated_xvits.mp3 ADDED
Binary file (80 kB). View file
 
samples/happy_piper.mp3 ADDED
Binary file (32 kB). View file
 
samples/happy_recording.mp3 ADDED
Binary file (26.9 kB). View file
 
samples/happy_xvits.mp3 ADDED
Binary file (26.1 kB). View file
 
samples/laughing_piper.mp3 ADDED
Binary file (35.5 kB). View file
 
samples/laughing_recording.mp3 ADDED
Binary file (41.6 kB). View file
 
samples/laughing_xvits.mp3 ADDED
Binary file (38.9 kB). View file
 
samples/sad_piper.mp3 ADDED
Binary file (55.8 kB). View file
 
samples/sad_recording.mp3 ADDED
Binary file (48.5 kB). View file
 
samples/sad_xvits.mp3 ADDED
Binary file (52.4 kB). View file