Dev1nW-webAI commited on
Commit
2db1567
·
verified ·
1 Parent(s): c0f1d9a

Upload Phonon-31-en-Air

Browse files
.gitattributes CHANGED
@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ figures/benchmarks.png filter=lfs diff=lfs merge=lfs -text
37
+ figures/latent_error.png filter=lfs diff=lfs merge=lfs -text
38
+ figures/similarity.png filter=lfs diff=lfs merge=lfs -text
39
+ figures/wer.png filter=lfs diff=lfs merge=lfs -text
40
+ model.q4k.gguf filter=lfs diff=lfs merge=lfs -text
LICENSE.md ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # webAI Non-Commercial License ver. 1.0
2
+
3
+ ## 1. Definitions
4
+
5
+ “Licensor” means any person or entity that distributes its Work. “Work” means (a) the original work of authorship made available under this license, which may include software, documentation, or other files, and (b) any additions to or derivative works thereof that are made available under this license. The terms “reproduce,” “reproduction,” “derivative works,” and “distribution” have the meaning as provided under U.S. copyright law; provided, however, that for the purposes of this license, derivative works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work. Works are “made available” under this license by including in or with the Work either (a) a copyright notice referencing the applicability of this license to the Work, or (b) a copy of this license.
6
+
7
+ ## 2. License Grant
8
+
9
+ 2.1 Copyright Grant. Subject to the terms and conditions of this license, each Licensor grants to you a perpetual, worldwide, non-exclusive, royalty-free, copyright license to use, reproduce, prepare derivative works of, publicly display, publicly perform, sublicense and distribute its Work and any resulting derivative works in any form.
10
+
11
+ ## 3. Limitations
12
+
13
+ 3.1 Redistribution. You may reproduce or distribute the Work only if (a) you do so under this license, (b) you include a complete copy of this license with your distribution, and (c) you retain without modification any copyright, patent, trademark, or attribution notices that are present in the Work.
14
+
15
+ 3.2 Derivative Works. You may specify that additional or different terms apply to the use, reproduction, and distribution of your derivative works of the Work (“Your Terms”) only if (a) Your Terms provide that the use limitation in Section 3.3 applies to your derivative works, and (b) you identify the specific derivative works that are subject to Your Terms. Notwithstanding Your Terms, this license (including the redistribution requirements in Section 3.1) will continue to apply to the Work itself.
16
+
17
+ 3.3 Use Limitation. The Work and any derivative works thereof only may be used or intended for use non-commercially. As used herein, “non-commercially” means for non-commercial research and educational purposes only.
18
+
19
+ 3.4 Patent Claims. If you bring or threaten to bring a patent claim against any Licensor (including any claim, cross-claim or counterclaim in a lawsuit) to enforce any patents that you allege are infringed by any Work, then your rights under this license from such Licensor (including the grant in Section 2.1) will terminate immediately.
20
+
21
+ 3.5 Trademarks. This license does not grant any rights to use any Licensor's or its affiliates' names, logos, or trademarks, except as necessary to reproduce the notices described in this license. Nothing in this license shall be construed as permission to use the trade names, trademarks, service marks, or product names of webAI, Inc. or its affiliates to endorse, promote, or imply association with any derivative work, product, service, or entity, without prior written consent from webAI, Inc. Use of the Work does not imply endorsement by webAI, Inc. of any derivative work, product, service, or entity.
22
+
23
+ 3.6 Termination. If you violate any term of this license, then your rights under this license (including the grant in Section 2.1) will terminate immediately.
24
+
25
+ ## 4. Disclaimer of Warranty.
26
+
27
+ THE WORK IS PROVIDED “AS IS” WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR NON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER THIS LICENSE.
28
+
29
+ ## 5. Limitation of Liability.
30
+
31
+ EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF OR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK (INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION, LOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
README.md ADDED
@@ -0,0 +1,280 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: webai-non-commercial-license-ver.-1.0
4
+ license_link: https://huggingface.co/webAI-Official/Gradium-Air/blob/main/LICENSE.md
5
+ base_model: gradium/phonon-31m-en
6
+ base_model_relation: quantized
7
+ pipeline_tag: text-to-speech
8
+ library_name: gguf
9
+ language:
10
+ - en
11
+ tags:
12
+ - gguf
13
+ - text-to-speech
14
+ - tts
15
+ - phonon
16
+ - ptts
17
+ - on-device
18
+ - cpu
19
+ ---
20
+
21
+ # Gradium-Air
22
+
23
+ A 4-bit build of Gradium's [phonon-31m-en](https://huggingface.co/gradium/phonon-31m-en) English text-to-speech model.
24
+ It is a drop-in replacement for Gradium's own `model.q4k.gguf`: the same 55,075,168 bytes, the same file name, and
25
+ it loads in Gradium's unmodified `ptts` runtime on CPU. In a pre-registered test on 2,576 synthesized clips, its
26
+ speech was **as intelligible as the BF16 model's** (word error rate within 0.5 points) and **closer to BF16's audio
27
+ than Gradium's Q4_K file**. It also scored higher predicted quality than that file.
28
+
29
+ > **Non-commercial use only.** Released under the webAI Non-Commercial License ver. 1.0 (research and education).
30
+
31
+ ### At a glance
32
+
33
+ Passes all four rules of the pre-registered test.
34
+
35
+ | | BF16 (Gradium) | Gradium Q4_K | **Gradium-Air** |
36
+ |---|---:|---:|---:|
37
+ | File size | 102,823,920 B | 55,075,168 B | **55,075,168 B** |
38
+ | Word error rate, all 1,273 sentences | 1.83% | 2.02% | **1.91%** |
39
+ | Word error rate, 200 hard sentences (numbers, dates, codes) | 5.90% | 6.30% | **5.98%** |
40
+ | Speaker similarity to BF16's audio of the same sentence | 1 (itself) | 0.9859 | **0.9866** |
41
+ | Predicted quality (UTMOS) | 4.333 | 4.281 | **4.314** |
42
+ | Latent error vs BF16, teacher-forced screen (400 items, separate protocol) | 0 (itself) | 0.01608 | **0.01284** |
43
+
44
+ <sub>Audio rows: stock ptts 0.6.0 on CPU, temperature 0.3, one seed per clip shared by every file, 1,273 sentences ×
45
+ 2 voices, one run per file. Word error rate from Whisper large-v3. See [Evaluation protocol](#evaluation-protocol).</sub>
46
+
47
+ ![Word error rate, distance from BF16's audio and UTMOS for BF16, Gradium's Q4_K and Gradium-Air](figures/benchmarks.png)
48
+
49
+ ## Highlights
50
+
51
+ * **As intelligible as BF16.** Word error rate 1.91% vs BF16's 1.83%, a difference of +0.08 points (95% CI −0.02 to
52
+ +0.18), inside the pre-registered +0.5-point margin. No suite is significantly worse. Gradium's Q4_K is
53
+ significantly worse than BF16 on the same test: +0.19 points (+0.06 to +0.32).
54
+ * **Closer to BF16's voice than Gradium's Q4_K.** Speaker similarity to BF16's clip of the same sentence is 0.9866 vs
55
+ 0.9859, +0.0007 (95% CI +0.0002 to +0.0011), and higher on both suites.
56
+ * **Higher predicted quality than Gradium's Q4_K.** UTMOS 4.314 vs 4.281, +0.033 (95% CI +0.027 to +0.039).
57
+ * **Same size, same runtime.** A byte-for-byte drop-in for `model.q4k.gguf` in Gradium's repository; nothing else
58
+ changes.
59
+
60
+ It is **not identical to BF16**: its audio is still slightly further from BF16's than BF16 resampled with a second
61
+ seed, and its UTMOS is 0.019 lower than BF16's. See [Limitations](#limitations).
62
+
63
+ ## Model Details
64
+
65
+ | Property | Value |
66
+ |---|---|
67
+ | Model ID | `webAI-Official/Gradium-Air` (proposed; not yet published) |
68
+ | Base model | [`gradium/phonon-31m-en`](https://huggingface.co/gradium/phonon-31m-en) at revision `92741c5b`, 31M parameters plus the Mimi audio decoder |
69
+ | Model file | `model.q4k.gguf`, 55,075,168 bytes (sha256 `ef1b486d7072d3db…`) |
70
+ | Format | GGUF. The 24 transformer matrices of the flow LM are 4-bit Q4_K, the same tensor types and layout as Gradium's `model.q4k.gguf`; every other tensor is unchanged |
71
+ | Runs on | Gradium's `ptts` runtime, CPU backend (`device="cpu"`, `quant="q4k"`). Tested with ptts 0.6.0 on Linux x86-64 and on macOS (Apple silicon) |
72
+ | Input / output | English text and a voice / mono 24 kHz audio |
73
+ | Voices | Gradium's [phonon-voices](https://huggingface.co/gradium/phonon-voices) catalog and the included `default-voice.safetensors` |
74
+ | License | webAI Non-Commercial License ver. 1.0 |
75
+
76
+ ## Results
77
+
78
+ Every audio result comes from one test, run once: each file synthesized the same 2,576 clips (1,273 distinct
79
+ sentences in 2 voices) with stock ptts 0.6.0 on CPU, temperature 0.3, and the same seed per clip for every file.
80
+ Verdicts pair sentences (each sentence's 2 voices are averaged first) and use a bootstrap 95% interval. "BF16" is
81
+ Gradium's `model.safetensors`; "Gradium Q8_0" and "Gradium Q4_K" are the two GGUF files in Gradium's repository;
82
+ "llama.cpp Q4_K" is a second same-size file, the same matrices quantized to Q4_K with llama.cpp's standard quantizer;
83
+ the two "BF16" control rows rerun BF16 with the same seeds (runtime noise) and with a second seed (how much BF16
84
+ differs from itself when resampled).
85
+
86
+ ### Pre-registered test
87
+
88
+ Gradium-Air passes all four rules, which were written down before the test ran.
89
+
90
+ | Check | Rule | Result (95% CI) | Pass |
91
+ |---|---|---|---|
92
+ | Primary: Gradium-Air vs BF16, word error rate | upper end of the 95% interval below +0.005, and no suite significantly worse | 0.0183 → 0.0191, +0.0008 (−0.0002 to +0.0018) | yes |
93
+ | Primary: Gradium-Air vs Gradium Q4_K, similarity to BF16 | lower end of the 95% interval above 0 | 0.9859 → 0.9866, +0.0007 (+0.0002 to +0.0011) | yes |
94
+ | Guardrail: Gradium-Air vs Gradium Q4_K, word error rate | not significantly worse | 0.0202 → 0.0191, −0.0011 (−0.0024 to +0.0002) | yes |
95
+ | Guardrail: Gradium-Air vs Gradium Q4_K, UTMOS | lower end of the 95% interval above −0.05 | 4.2814 → 4.3141, +0.0327 (+0.0267 to +0.0388) | yes |
96
+
97
+ These four results are confirmatory. Everything else on this page (per-suite rows outside the rules, the control
98
+ rows, and the screen) is supporting detail and should be read as preliminary.
99
+
100
+ ### Word error rate
101
+
102
+ Gradium-Air shows no significant change from BF16, while both other 4-bit files are significantly worse than BF16.
103
+
104
+ | File | Size | All (1,273) | Seed-TTS English (1,073) | Hard (200) | Corpus WER | vs BF16, all, points (95% CI) | Verdict |
105
+ |---|---:|---:|---:|---:|---:|---|---|
106
+ | BF16 (baseline) | 102,823,920 B | 1.83% | 1.07% | 5.90% | 1.82% | baseline | |
107
+ | BF16, rerun with the same seeds | same file | 1.83% | 1.07% | 5.90% | 1.81% | −0.00 (−0.01 to +0.00) | no significant change |
108
+ | BF16, second seed | same file | 1.82% | 1.10% | 5.67% | 1.81% | −0.01 (−0.13 to +0.09) | no significant change |
109
+ | Gradium Q8_0 | 64,512,352 B | 1.82% | 1.04% | 6.05% | 1.81% | −0.01 (−0.08 to +0.06) | no significant change |
110
+ | Gradium Q4_K | 55,075,168 B | 2.02% | 1.23% | 6.30% | 1.98% | +0.19 (+0.06 to +0.32) | higher (worse) |
111
+ | llama.cpp Q4_K | 55,075,168 B | 1.96% | 1.14% | 6.31% | 1.93% | +0.13 (+0.03 to +0.22) | higher (worse) |
112
+ | **Gradium-Air** | 55,075,168 B | **1.91%** | **1.16%** | **5.98%** | **1.88%** | +0.08 (−0.02 to +0.18) | no significant change |
113
+
114
+ The columns give the mean per-sentence word error rate (each clip capped at 1; no clip reached the cap), with the
115
+ number of sentences in brackets; "Corpus WER" is total errors over total words. Per suite against BF16, Gradium-Air is
116
+ +0.09 points on Seed-TTS English (95% CI −0.00 to +0.18) and +0.08 on the hard sentences (−0.37 to +0.50), neither
117
+ significant; the Seed-TTS interval just touches zero. Against Gradium's Q4_K, Gradium-Air is −0.11 points (−0.24 to
118
+ +0.02), no significant change. The two BF16 controls show the test's own noise: the same-seed rerun differs only by
119
+ transcription noise, and a second seed moves the score by −0.01 points.
120
+
121
+ ![Word error rate per suite for every file](figures/wer.png)
122
+
123
+ ### Speaker similarity to BF16's audio
124
+
125
+ Gradium-Air is closer to BF16's audio than both other 4-bit files, but not as close as BF16 resampled.
126
+
127
+ | File | All | Seed-TTS English | Hard | vs Gradium Q4_K, all (95% CI) | Verdict |
128
+ |---|---:|---:|---:|---|---|
129
+ | BF16, rerun with the same seeds | 1.0000 | 1.0000 | 1.0000 | +0.0141 (+0.0136 to +0.0145) | higher |
130
+ | Gradium Q8_0 | 0.9956 | 0.9956 | 0.9959 | +0.0097 (+0.0093 to +0.0101) | higher |
131
+ | BF16, second seed | 0.9872 | 0.9866 | 0.9906 | +0.0013 (+0.0009 to +0.0018) | higher |
132
+ | Gradium Q4_K (baseline) | 0.9859 | 0.9855 | 0.9880 | baseline | |
133
+ | llama.cpp Q4_K | 0.9859 | 0.9853 | 0.9887 | −0.0000 (−0.0005 to +0.0004) | no significant change |
134
+ | **Gradium-Air** | **0.9866** | **0.9861** | **0.9890** | +0.0007 (+0.0002 to +0.0011) | higher |
135
+
136
+ Similarity is the cosine between speaker-verification embeddings of a file's clip and BF16's clip of the same
137
+ sentence, voice and seed. Gradium-Air beats Gradium's Q4_K on both suites: Seed-TTS English +0.0006 (+0.0001 to
138
+ +0.0011) and hard +0.0010 (+0.0001 to +0.0020). Against BF16 resampled with a second seed it is −0.0007 (−0.0011 to
139
+ −0.0002), significantly lower.
140
+
141
+ ![Similarity to BF16's audio per file, and each file's difference from Gradium's Q4_K](figures/similarity.png)
142
+
143
+ ### Predicted quality (UTMOS)
144
+
145
+ Gradium-Air scores higher than Gradium's Q4_K but lower than BF16.
146
+
147
+ | File | All | Seed-TTS English | Hard | vs Gradium Q4_K, all (95% CI) | Verdict |
148
+ |---|---:|---:|---:|---|---|
149
+ | BF16 | 4.333 | 4.328 | 4.361 | +0.052 (+0.046 to +0.058) | higher |
150
+ | BF16, rerun with the same seeds | 4.333 | 4.328 | 4.361 | +0.052 (+0.046 to +0.058) | higher |
151
+ | BF16, second seed | 4.330 | 4.324 | 4.360 | +0.049 (+0.043 to +0.055) | higher |
152
+ | Gradium Q8_0 | 4.332 | 4.327 | 4.359 | +0.050 (+0.045 to +0.056) | higher |
153
+ | Gradium Q4_K (baseline) | 4.281 | 4.278 | 4.300 | baseline | |
154
+ | llama.cpp Q4_K | 4.283 | 4.277 | 4.311 | +0.001 (−0.005 to +0.007) | no significant change |
155
+ | **Gradium-Air** | **4.314** | **4.309** | **4.343** | +0.033 (+0.027 to +0.039) | higher |
156
+
157
+ Against BF16, Gradium-Air is −0.019 (−0.025 to −0.014), significantly lower.
158
+
159
+ ### Against Gradium's Q4_K on the teacher-forced screen (separate protocol)
160
+
161
+ Gradium-Air's predictions are 20% closer to BF16's than Gradium's Q4_K's at the identical file size.
162
+
163
+ | File (55,075,168 B each) | Latent error | vs Gradium Q4_K (95% CI) | Verdict | End-of-speech logit error | Stop frame changed |
164
+ |---|---:|---|---|---:|---:|
165
+ | Gradium Q4_K (baseline) | 0.01608 | baseline | | 0.224 | 23 / 400 |
166
+ | llama.cpp Q4_K | 0.01390 | −0.00218 (−0.00249 to −0.00189) | lower | 0.195 | 16 / 400 |
167
+ | **Gradium-Air** | **0.01284** | −0.00324 (−0.00357 to −0.00293) | lower | 0.209 | 8 / 400 |
168
+
169
+ This screen does not synthesize audio. Each file's model predicts every next audio latent from BF16's own previous
170
+ frames, with the same text, voice and noise, and the error is the mean squared difference from BF16's prediction. It
171
+ covers 400 items (200 sentences × 2 voices), was run once, and uses transformers on CPU in f32, not ptts. Gradium-Air is
172
+ lower on both sentence sources: WikiText-2 (320 items) 0.01640 → 0.01318, −0.00322 (−0.00356 to −0.00290), and
173
+ hand-written numbers and dates (80 items) 0.01478 → 0.01148, −0.00331 (−0.00406 to −0.00261). Against llama.cpp's Q4_K
174
+ it is also lower, −0.00106 (−0.00130 to −0.00082), but its end-of-speech logit error is higher than that file's (0.209
175
+ vs 0.195). Against Gradium's Q4_K that error is lower, −0.0151 (−0.0176 to −0.0126), and Gradium-Air changes the stop
176
+ frame least often.
177
+
178
+ ![Latent error against BF16 per sentence source](figures/latent_error.png)
179
+
180
+ ## Files
181
+
182
+ | File | What it is |
183
+ |---|---|
184
+ | `model.q4k.gguf` | Gradium-Air (55,075,168 bytes, sha256 `ef1b486d7072d3dbd4634b6f846cd818e6fa129a819e928da23e7287b0920fba`); modified from Gradium's `model.q4k.gguf` |
185
+ | `default-voice.safetensors` | Gradium's default voice, unchanged (256,136 bytes, sha256 `dc537478eaadc1a5257fcda2e3501ab81e14c111a5d95a5838f350db2afd0f1a`) |
186
+ | `config.json`, `generation_config.json`, `processor_config.json`, `chat_template.jinja`, `tokenizer.json`, `tokenizer.5.0.0.json`, `tokenizer.model`, `tokenizer_config.json` | Gradium's configuration and tokenizer files, unchanged from revision `92741c5b`, so the folder runs on its own |
187
+ | `LICENSE.md` | webAI Non-Commercial License ver. 1.0 |
188
+ | `apache-2.0-LICENSE.txt` | Apache License 2.0, the license of Gradium's phonon-31m-en |
189
+ | `figures/` | The charts in this card |
190
+
191
+ Check the downloads with `shasum -a 256 model.q4k.gguf default-voice.safetensors` against the hashes above.
192
+
193
+ ## Quick start
194
+
195
+ ```bash
196
+ pip install ptts==0.6.0
197
+ hf download webAI-Official/Gradium-Air --local-dir Gradium-Air && cd Gradium-Air
198
+ ptts -m . -l en -q q4k -d cpu -v Easton "Gradium-Air read this sentence on a laptop CPU at 6:20 PM on November 8th, 2026." -o speech.wav
199
+ ```
200
+
201
+ In Python:
202
+
203
+ ```python
204
+ import ptts
205
+
206
+ tts = ptts.TTS(".", lang="en", quant="q4k", device="cpu")
207
+ tts.save("speech.wav", "Hello from Gradium-Air.", voice="Easton")
208
+ ```
209
+
210
+ * The `ptts` command was run from inside this folder with ptts 0.6.0; `quant="q4k"` selects `model.q4k.gguf`, and
211
+ `device="cpu"` is the backend this file was tested on.
212
+ * The evaluation used temperature 0.3 (the model's default) and a fixed seed per sentence; pass `seed=` (Python) or
213
+ `-s` (command line) for reproducible audio.
214
+ * Voices other than the default come from Gradium's voice catalog and are downloaded on first use; every voice in
215
+ [gradium/phonon-voices](https://huggingface.co/gradium/phonon-voices) works by name or ID.
216
+ * Streaming (`tts.stream(...)`) and the other runtimes in [Gradium's guide](https://huggingface.co/gradium/phonon-31m-en)
217
+ work as with Gradium's files; only the CPU backend was tested with this file.
218
+
219
+ ## Limitations
220
+
221
+ * **Not identical to BF16.** Its audio is less similar to BF16's (0.9866) than BF16 resampled with a second seed
222
+ (0.9872), a significant difference of −0.0007, and its UTMOS is 0.019 lower than BF16's (significant). If the
223
+ closest possible match to BF16 matters more than size, Gradium's Q8_0 (64.5 MB) is closer to BF16 on every measure
224
+ here.
225
+ * **Small effects.** The similarity gain over Gradium's Q4_K is +0.0007 on a 0–1 scale; it is significant over 1,273
226
+ sentences but small. The word-error-rate improvement over Gradium's Q4_K (−0.11 points) is not significant.
227
+ * **One run.** Each file was measured once on one test set, with one seed per clip; only BF16 was also resampled.
228
+ The screen was also run once.
229
+ * **Scope.** English only, two voices (Easton and Master Aldric), sentences of up to about 10 seconds of audio. Long-form
230
+ reading, other voices, voice cloning and listening tests were not evaluated. Measures are automatic (speech
231
+ recognition, speaker embeddings, predicted quality), not human ratings.
232
+ * **CPU only.** Results hold for ptts's CPU backend. ptts's other backends (Core ML, CUDA, Metal, Vulkan and the
233
+ browser build) were not tested with this file.
234
+ * **Speed was not measured** under controlled conditions. The file has the same size and tensor types as Gradium's
235
+ Q4_K. One example on an Apple silicon Mac: the quick-start command produced 6.3 s of audio in 1.8 s, model loading
236
+ included.
237
+ * **Which file.** Results hold for the included `model.q4k.gguf` (sha256 above), byte-identical to the file that was
238
+ evaluated.
239
+
240
+ ## Evaluation protocol
241
+
242
+ * **Synthesis:** stock ptts 0.6.0 (Linux wheel), CPU backend, 128 single-thread processes on an AMD EPYC 9355 machine
243
+ (container limited to 13.6 CPUs);
244
+ temperature 0.3; one seed per clip, derived from the clip's id and shared by every file; the "second seed" control
245
+ uses a different seed per clip. Voice catalog `gradium/phonon-voices` at revision `0ee17205`. BF16 ran from
246
+ `model.safetensors` in f32, Gradium Q8_0 as `q8_0`, and the three Q4_K files as `q4k`. Every file rendered all 2,576
247
+ clips on CPU. Clips with unusually long audio (≥ 29.5 s, or > 1.5 s per word) numbered 0–4 per file.
248
+ * **Sentences:** the 1,088 English target sentences of the Seed-TTS evaluation set (Hugging Face mirror
249
+ `zhaochenyang20/seed-tts-eval` at revision `8f5e1aa2`; 1,073 distinct) and 200 templated sentences full of numbers,
250
+ dates, money, codes and units (the "hard" suite), each in the voices Easton and Master Aldric: 2,576 clips, 1,273
251
+ distinct sentences.
252
+ * **Word error rate:** Whisper large-v3 (fp16, greedy, English), Whisper's English text normaliser on both reference
253
+ and transcript, then digit groups split at "." or ":" and "a m"/"p m" joined (so "3:51 p.m." and "3 51 pm" match);
254
+ word errors counted with jiwer, each clip capped at 1, then averaged over a sentence's 2 voices. Not comparable with
255
+ Gradium's published LLM-assisted word error rates.
256
+ * **Speaker similarity:** cosine between `microsoft/wavlm-base-plus-sv` speaker-verification embeddings of a file's clip
257
+ and BF16's clip of the same sentence, voice and seed, at 16 kHz.
258
+ * **Predicted quality:** UTMOS (`utmos22_strong` from `tarepan/SpeechMOS` v1.2.0), at 16 kHz.
259
+ * **Statistics:** every comparison pairs the same sentences (n = 1,273; 1,073 Seed-TTS and 200 hard); 95% intervals
260
+ from 2,000 bootstrap resamples of the per-sentence differences. "No significant change" means the interval includes
261
+ zero.
262
+ * **Pre-registration:** the question, primary measures, margins (+0.005 word error rate against BF16; UTMOS within
263
+ 0.05 of Gradium's Q4_K) and pass rules were written down before the test ran.
264
+ * **Screen:** teacher-forced, 160 WikiText-2 test sentences and 40 hand-written sentences of numbers and dates in the
265
+ voices Hadley and Zoey (400 items); BF16 reference latents from transformers in f32 on CPU, temperature 0.3, one flow
266
+ step; latent error = mean squared difference of the predicted normalised latents; stop frame = first frame whose
267
+ end-of-speech logit exceeds 4.0.
268
+
269
+ ## License and attribution
270
+
271
+ Released under the **webAI Non-Commercial License ver. 1.0** (`LICENSE.md`): non-commercial research and educational use
272
+ only. `model.q4k.gguf` is a modified version of the weights of
273
+ [`gradium/phonon-31m-en`](https://huggingface.co/gradium/phonon-31m-en), released by Gradium under the Apache License
274
+ 2.0 (text kept in `apache-2.0-LICENSE.txt`). The unchanged files from Gradium's repository (configuration, tokenizer
275
+ and default voice) remain under Apache 2.0. All credit for the base model goes to Gradium; as its card notes, Phonon
276
+ builds on [Pocket TTS](https://github.com/kyutai-labs/pocket-tts) and the Mimi codec, developed by Kyutai.
277
+
278
+ Please follow Gradium's responsible-use guidance: do not use the model to impersonate a real person, to deceive
279
+ listeners about who is speaking, or to clone a voice without the speaker's consent, and tell listeners when speech is
280
+ synthetic where that matters to them.
apache-2.0-LICENSE.txt ADDED
@@ -0,0 +1,201 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
chat_template.jinja ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- set training = messages | length == 2 -%}
2
+ {%- if messages | length > 2 or messages[0]['role'] != 'user' or (training and messages[1]['role'] != 'assistant') -%}
3
+ {{- raise_exception('Gradium Phonon TTS takes one `user` message per conversation, followed by the `assistant` target audio to train on.') -}}
4
+ {%- endif -%}
5
+ {%- if training -%}
6
+ {%- set target = messages[1]['content'] -%}
7
+ {%- if target is string or target | length != 1 or target[0]['type'] != 'audio' -%}
8
+ {{- raise_exception('The `assistant` turn holds one `audio` item, the target.') -}}
9
+ {%- endif -%}
10
+ {%- endif -%}
11
+ {%- set content = messages[0]['content'] -%}
12
+ {%- if content is string -%}
13
+ {%- set texts = [content] -%}
14
+ {%- set num_audio = 0 -%}
15
+ {%- else -%}
16
+ {%- set texts = content | selectattr('type', 'equalto', 'text') | map(attribute='text') | list -%}
17
+ {%- set num_audio = content | selectattr('type', 'equalto', 'audio') | list | length -%}
18
+ {%- endif -%}
19
+ {%- if texts | length > 1 or (texts | length == 0 and not training) or num_audio > 1 -%}
20
+ {{- raise_exception('The `user` turn holds one `text` item and at most one `audio` item (the voice).') -}}
21
+ {%- endif -%}
22
+ {%- set voice = '<|audio_prompt|>' if num_audio else '' -%}
23
+ {%- set text = (texts[0] if texts else '') | trim -%}
24
+ {%- if training -%}
25
+ {{- voice ~ text ~ '<|audio_bos|><|audio_prompt|>' -}}
26
+ {%- else -%}
27
+ {{- voice ~ text -}}
28
+ {%- if add_generation_prompt -%}
29
+ {{- '<|audio_bos|>' -}}
30
+ {%- endif -%}
31
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,204 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "languages": [
3
+ "en"
4
+ ],
5
+ "embeddings_ext": [
6
+ "ba7a5a45@300"
7
+ ],
8
+ "flow_lm": {
9
+ "d_model": 512,
10
+ "num_heads": 8,
11
+ "num_layers": 6,
12
+ "dim_feedforward": 2048,
13
+ "max_period": 10000.0,
14
+ "n_bins": 4000,
15
+ "lut_dim": 512,
16
+ "flow_dim": 512,
17
+ "flow_depth": 6,
18
+ "ldim": 32,
19
+ "time_rms_norm": "rms",
20
+ "gelu": "erf"
21
+ },
22
+ "mimi": {
23
+ "channels": 1,
24
+ "sample_rate": 24000,
25
+ "frame_rate": 12.5,
26
+ "dimension": 512,
27
+ "quantizer_dimension": 32,
28
+ "quantizer_output_dimension": 512,
29
+ "n_filters": 64,
30
+ "n_residual_layers": 1,
31
+ "ratios": [
32
+ 6,
33
+ 5,
34
+ 4
35
+ ],
36
+ "kernel_size": 7,
37
+ "last_kernel_size": 3,
38
+ "residual_kernel_size": 3,
39
+ "dilation_base": 2,
40
+ "compress": 2,
41
+ "transformer_d_model": 512,
42
+ "transformer_num_heads": 8,
43
+ "transformer_num_layers": 2,
44
+ "transformer_layer_scale": 0.01,
45
+ "transformer_context": 250,
46
+ "transformer_max_period": 10000.0,
47
+ "transformer_dim_feedforward": 2048,
48
+ "gelu": "erf"
49
+ },
50
+ "temp": 0.8,
51
+ "lsd_decode_steps": 1,
52
+ "eos_threshold": 4.0,
53
+ "model_id": {
54
+ "sig": "5f0a2aab",
55
+ "epoch": 200,
56
+ "mimi_sig": "b77c7ffa",
57
+ "mimi_epoch": 150
58
+ },
59
+ "weights_name": "model.safetensors",
60
+ "tokenizer_name": "tokenizer.model",
61
+ "conditioners": [
62
+ {
63
+ "type": "lut",
64
+ "lut": {
65
+ "n_bins": 31,
66
+ "dim": 16,
67
+ "tokenizer": "noop",
68
+ "possible_values": [
69
+ "0",
70
+ "1",
71
+ "2"
72
+ ]
73
+ },
74
+ "name": "num_speakers"
75
+ },
76
+ {
77
+ "type": "continuous",
78
+ "continuous": {
79
+ "scale_factor": 1000.0,
80
+ "dim": 128,
81
+ "max_period": 10000.0
82
+ },
83
+ "name": "duration_delta"
84
+ }
85
+ ],
86
+ "fuser": {
87
+ "cross_attention_pos_emb": false,
88
+ "cross_attention_pos_emb_scale": 1,
89
+ "sum": [
90
+ "num_speakers",
91
+ "duration_delta"
92
+ ],
93
+ "streaming_sum": [],
94
+ "prepend": [
95
+ "transcript_in_segment",
96
+ "speaker_wavs"
97
+ ],
98
+ "cross": [],
99
+ "pad_prepend": false
100
+ },
101
+ "speaker_mimi": {
102
+ "prefix": "mimi_speaker",
103
+ "mimi": {
104
+ "channels": 1,
105
+ "sample_rate": 24000,
106
+ "frame_rate": 12.5,
107
+ "dimension": 512,
108
+ "quantizer_dimension": 256,
109
+ "quantizer_output_dimension": 512,
110
+ "n_filters": 64,
111
+ "n_residual_layers": 1,
112
+ "ratios": [
113
+ 8,
114
+ 6,
115
+ 5,
116
+ 4
117
+ ],
118
+ "kernel_size": 7,
119
+ "last_kernel_size": 3,
120
+ "residual_kernel_size": 3,
121
+ "dilation_base": 2,
122
+ "compress": 2,
123
+ "transformer_d_model": 512,
124
+ "transformer_num_heads": 8,
125
+ "transformer_num_layers": 8,
126
+ "transformer_layer_scale": 0.01,
127
+ "transformer_context": 250,
128
+ "transformer_max_period": 10000.0,
129
+ "transformer_dim_feedforward": 2048,
130
+ "gelu": "erf"
131
+ }
132
+ },
133
+ "audio_prompt_min_duration": 2.5,
134
+ "architectures": [
135
+ "GradiumPhononTTSForConditionalGeneration"
136
+ ],
137
+ "attention_bias": false,
138
+ "attention_dropout": 0.0,
139
+ "audio_bos_token_id": 4,
140
+ "audio_latent_token_id": 6,
141
+ "audio_prompt_dim": 512,
142
+ "audio_prompt_token_id": 5,
143
+ "bos_token_id": null,
144
+ "dtype": "bfloat16",
145
+ "duration_delta": 0.0,
146
+ "duration_delta_embedding_dim": 128,
147
+ "duration_delta_max_period": 10000.0,
148
+ "duration_delta_scale": 1000.0,
149
+ "eos_token_id": null,
150
+ "flow_head_config": {
151
+ "_name_or_path": "",
152
+ "architectures": null,
153
+ "chunk_size_feed_forward": 0,
154
+ "condition_dim": 512,
155
+ "dtype": null,
156
+ "flow_type": "lsd",
157
+ "frequency_embedding_size": 256,
158
+ "hidden_act": "silu",
159
+ "hidden_size": 512,
160
+ "id2label": {
161
+ "0": "LABEL_0",
162
+ "1": "LABEL_1"
163
+ },
164
+ "is_encoder_decoder": false,
165
+ "label2id": {
166
+ "LABEL_0": 0,
167
+ "LABEL_1": 1
168
+ },
169
+ "latent_dim": 32,
170
+ "layer_norm_eps": 1e-06,
171
+ "max_period": 10000.0,
172
+ "model_type": "",
173
+ "num_hidden_layers": 6,
174
+ "output_attentions": false,
175
+ "output_hidden_states": false,
176
+ "problem_type": null,
177
+ "return_dict": true,
178
+ "rms_norm_eps": 1e-05
179
+ },
180
+ "frame_rate": 12.5,
181
+ "head_dim": 64,
182
+ "hidden_act": "gelu",
183
+ "hidden_size": 512,
184
+ "initializer_range": 0.02,
185
+ "intermediate_size": 2048,
186
+ "latent_dim": 32,
187
+ "layer_norm_eps": 1e-05,
188
+ "max_position_embeddings": 8192,
189
+ "model_type": "gradium_phonon_tts",
190
+ "num_attention_heads": 8,
191
+ "num_hidden_layers": 6,
192
+ "num_key_value_heads": 8,
193
+ "num_speakers": 1,
194
+ "num_speakers_embedding_dim": 16,
195
+ "num_speakers_vocab_size": 32,
196
+ "pad_token_id": 3,
197
+ "rope_parameters": {
198
+ "rope_theta": 10000.0,
199
+ "rope_type": "default"
200
+ },
201
+ "transformers_version": "5.21.0.dev0",
202
+ "use_cache": true,
203
+ "vocab_size": 4001
204
+ }
default-voice.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc537478eaadc1a5257fcda2e3501ab81e14c111a5d95a5838f350db2afd0f1a
3
+ size 256136
figures/benchmarks.png ADDED

Git LFS Details

  • SHA256: 5a7289b4dbd3f95ca4d08e5e73fa5c272dec8ebce124532e3b5294752662ac79
  • Pointer size: 131 Bytes
  • Size of remote file: 171 kB
figures/latent_error.png ADDED

Git LFS Details

  • SHA256: c66ad9a02e6a6600d5726a38238f42ef19168d505e12339d98065fe944dd3ed0
  • Pointer size: 131 Bytes
  • Size of remote file: 155 kB
figures/similarity.png ADDED

Git LFS Details

  • SHA256: adf882228084c464b3e8fa32a85f214067ac87e150284997f0569b08271e135b
  • Pointer size: 131 Bytes
  • Size of remote file: 200 kB
figures/wer.png ADDED

Git LFS Details

  • SHA256: ec96a25d7e7cfc725c167c1778c537fded5b81486318275d4fe44425cb193af0
  • Pointer size: 131 Bytes
  • Size of remote file: 178 kB
generation_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_threshold": 4.0,
4
+ "min_frames_before_eos": 0,
5
+ "num_flow_steps": 1,
6
+ "pad_token_id": 3,
7
+ "return_dict_in_generate": true,
8
+ "temperature": 0.3,
9
+ "transformers_version": "5.21.0.dev0"
10
+ }
model.q4k.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef1b486d7072d3dbd4634b6f846cd818e6fa129a819e928da23e7287b0920fba
3
+ size 55075168
processor_config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "audio_tokenizer": {
3
+ "audio_tokenizer_class": "PocketTTSAudioTokenizerModel",
4
+ "audio_tokenizer_name_or_path": "gradium/phonon-mimi"
5
+ },
6
+ "fade_in_duration": 0.0,
7
+ "feature_extractor": {
8
+ "analysis_frame_duration": 0.02,
9
+ "end_on_pause": true,
10
+ "fade_out_duration": 0.02,
11
+ "feature_extractor_type": "PocketTTSFeatureExtractor",
12
+ "feature_size": 1,
13
+ "frame_size": 1920,
14
+ "pad_to_multiple_of": 1920,
15
+ "padding_side": "right",
16
+ "padding_value": 0.0,
17
+ "pause_duration": 0.08,
18
+ "return_attention_mask": true,
19
+ "sampling_rate": 24000,
20
+ "silence_floor_db": 35.0
21
+ },
22
+ "frames_after_eos": 1,
23
+ "frames_after_eos_short_text": 3,
24
+ "processor_class": "PocketTTSProcessor",
25
+ "short_text_max_words": 4
26
+ }
tokenizer.5.0.0.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82e4572006d64d483317074c22c368e87779640b6f3a9dcd07b7caae32b030ce
3
+ size 65045
tokenizer_config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "tokenizer_class": "TokenizersBackend",
3
+ "fast_tokenizer_files": [
4
+ "tokenizer.5.0.0.json"
5
+ ],
6
+ "unk_token": "\u0000<unk>",
7
+ "bos_token": "\u0000<s>",
8
+ "eos_token": "\u0000</s>",
9
+ "pad_token": "\u0000<pad>",
10
+ "padding": true,
11
+ "padding_side": "left",
12
+ "return_tensors": "pt",
13
+ "model_input_names": [
14
+ "input_ids",
15
+ "attention_mask"
16
+ ],
17
+ "clean_up_tokenization_spaces": false,
18
+ "model_specific_special_tokens": {
19
+ "audio_prompt_token": "<|audio_prompt|>",
20
+ "audio_latent_token": "<|audio_latent|>",
21
+ "audio_bos_token": "<|audio_bos|>"
22
+ }
23
+ }