Upload Phonon-31-en-Air
Browse files- .gitattributes +5 -0
- LICENSE.md +31 -0
- README.md +280 -0
- apache-2.0-LICENSE.txt +201 -0
- chat_template.jinja +31 -0
- config.json +204 -0
- default-voice.safetensors +3 -0
- figures/benchmarks.png +3 -0
- figures/latent_error.png +3 -0
- figures/similarity.png +3 -0
- figures/wer.png +3 -0
- generation_config.json +10 -0
- model.q4k.gguf +3 -0
- processor_config.json +26 -0
- tokenizer.5.0.0.json +0 -0
- tokenizer.json +0 -0
- tokenizer.model +3 -0
- tokenizer_config.json +23 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
figures/benchmarks.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
figures/latent_error.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
figures/similarity.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
figures/wer.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
model.q4k.gguf filter=lfs diff=lfs merge=lfs -text
|
LICENSE.md
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# webAI Non-Commercial License ver. 1.0
|
| 2 |
+
|
| 3 |
+
## 1. Definitions
|
| 4 |
+
|
| 5 |
+
“Licensor” means any person or entity that distributes its Work. “Work” means (a) the original work of authorship made available under this license, which may include software, documentation, or other files, and (b) any additions to or derivative works thereof that are made available under this license. The terms “reproduce,” “reproduction,” “derivative works,” and “distribution” have the meaning as provided under U.S. copyright law; provided, however, that for the purposes of this license, derivative works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work. Works are “made available” under this license by including in or with the Work either (a) a copyright notice referencing the applicability of this license to the Work, or (b) a copy of this license.
|
| 6 |
+
|
| 7 |
+
## 2. License Grant
|
| 8 |
+
|
| 9 |
+
2.1 Copyright Grant. Subject to the terms and conditions of this license, each Licensor grants to you a perpetual, worldwide, non-exclusive, royalty-free, copyright license to use, reproduce, prepare derivative works of, publicly display, publicly perform, sublicense and distribute its Work and any resulting derivative works in any form.
|
| 10 |
+
|
| 11 |
+
## 3. Limitations
|
| 12 |
+
|
| 13 |
+
3.1 Redistribution. You may reproduce or distribute the Work only if (a) you do so under this license, (b) you include a complete copy of this license with your distribution, and (c) you retain without modification any copyright, patent, trademark, or attribution notices that are present in the Work.
|
| 14 |
+
|
| 15 |
+
3.2 Derivative Works. You may specify that additional or different terms apply to the use, reproduction, and distribution of your derivative works of the Work (“Your Terms”) only if (a) Your Terms provide that the use limitation in Section 3.3 applies to your derivative works, and (b) you identify the specific derivative works that are subject to Your Terms. Notwithstanding Your Terms, this license (including the redistribution requirements in Section 3.1) will continue to apply to the Work itself.
|
| 16 |
+
|
| 17 |
+
3.3 Use Limitation. The Work and any derivative works thereof only may be used or intended for use non-commercially. As used herein, “non-commercially” means for non-commercial research and educational purposes only.
|
| 18 |
+
|
| 19 |
+
3.4 Patent Claims. If you bring or threaten to bring a patent claim against any Licensor (including any claim, cross-claim or counterclaim in a lawsuit) to enforce any patents that you allege are infringed by any Work, then your rights under this license from such Licensor (including the grant in Section 2.1) will terminate immediately.
|
| 20 |
+
|
| 21 |
+
3.5 Trademarks. This license does not grant any rights to use any Licensor's or its affiliates' names, logos, or trademarks, except as necessary to reproduce the notices described in this license. Nothing in this license shall be construed as permission to use the trade names, trademarks, service marks, or product names of webAI, Inc. or its affiliates to endorse, promote, or imply association with any derivative work, product, service, or entity, without prior written consent from webAI, Inc. Use of the Work does not imply endorsement by webAI, Inc. of any derivative work, product, service, or entity.
|
| 22 |
+
|
| 23 |
+
3.6 Termination. If you violate any term of this license, then your rights under this license (including the grant in Section 2.1) will terminate immediately.
|
| 24 |
+
|
| 25 |
+
## 4. Disclaimer of Warranty.
|
| 26 |
+
|
| 27 |
+
THE WORK IS PROVIDED “AS IS” WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR NON-INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER THIS LICENSE.
|
| 28 |
+
|
| 29 |
+
## 5. Limitation of Liability.
|
| 30 |
+
|
| 31 |
+
EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF OR RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK (INCLUDING BUT NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION, LOST PROFITS OR DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER DAMAGES OR LOSSES), EVEN IF THE LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
|
README.md
ADDED
|
@@ -0,0 +1,280 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: webai-non-commercial-license-ver.-1.0
|
| 4 |
+
license_link: https://huggingface.co/webAI-Official/Gradium-Air/blob/main/LICENSE.md
|
| 5 |
+
base_model: gradium/phonon-31m-en
|
| 6 |
+
base_model_relation: quantized
|
| 7 |
+
pipeline_tag: text-to-speech
|
| 8 |
+
library_name: gguf
|
| 9 |
+
language:
|
| 10 |
+
- en
|
| 11 |
+
tags:
|
| 12 |
+
- gguf
|
| 13 |
+
- text-to-speech
|
| 14 |
+
- tts
|
| 15 |
+
- phonon
|
| 16 |
+
- ptts
|
| 17 |
+
- on-device
|
| 18 |
+
- cpu
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# Gradium-Air
|
| 22 |
+
|
| 23 |
+
A 4-bit build of Gradium's [phonon-31m-en](https://huggingface.co/gradium/phonon-31m-en) English text-to-speech model.
|
| 24 |
+
It is a drop-in replacement for Gradium's own `model.q4k.gguf`: the same 55,075,168 bytes, the same file name, and
|
| 25 |
+
it loads in Gradium's unmodified `ptts` runtime on CPU. In a pre-registered test on 2,576 synthesized clips, its
|
| 26 |
+
speech was **as intelligible as the BF16 model's** (word error rate within 0.5 points) and **closer to BF16's audio
|
| 27 |
+
than Gradium's Q4_K file**. It also scored higher predicted quality than that file.
|
| 28 |
+
|
| 29 |
+
> **Non-commercial use only.** Released under the webAI Non-Commercial License ver. 1.0 (research and education).
|
| 30 |
+
|
| 31 |
+
### At a glance
|
| 32 |
+
|
| 33 |
+
Passes all four rules of the pre-registered test.
|
| 34 |
+
|
| 35 |
+
| | BF16 (Gradium) | Gradium Q4_K | **Gradium-Air** |
|
| 36 |
+
|---|---:|---:|---:|
|
| 37 |
+
| File size | 102,823,920 B | 55,075,168 B | **55,075,168 B** |
|
| 38 |
+
| Word error rate, all 1,273 sentences | 1.83% | 2.02% | **1.91%** |
|
| 39 |
+
| Word error rate, 200 hard sentences (numbers, dates, codes) | 5.90% | 6.30% | **5.98%** |
|
| 40 |
+
| Speaker similarity to BF16's audio of the same sentence | 1 (itself) | 0.9859 | **0.9866** |
|
| 41 |
+
| Predicted quality (UTMOS) | 4.333 | 4.281 | **4.314** |
|
| 42 |
+
| Latent error vs BF16, teacher-forced screen (400 items, separate protocol) | 0 (itself) | 0.01608 | **0.01284** |
|
| 43 |
+
|
| 44 |
+
<sub>Audio rows: stock ptts 0.6.0 on CPU, temperature 0.3, one seed per clip shared by every file, 1,273 sentences ×
|
| 45 |
+
2 voices, one run per file. Word error rate from Whisper large-v3. See [Evaluation protocol](#evaluation-protocol).</sub>
|
| 46 |
+
|
| 47 |
+

|
| 48 |
+
|
| 49 |
+
## Highlights
|
| 50 |
+
|
| 51 |
+
* **As intelligible as BF16.** Word error rate 1.91% vs BF16's 1.83%, a difference of +0.08 points (95% CI −0.02 to
|
| 52 |
+
+0.18), inside the pre-registered +0.5-point margin. No suite is significantly worse. Gradium's Q4_K is
|
| 53 |
+
significantly worse than BF16 on the same test: +0.19 points (+0.06 to +0.32).
|
| 54 |
+
* **Closer to BF16's voice than Gradium's Q4_K.** Speaker similarity to BF16's clip of the same sentence is 0.9866 vs
|
| 55 |
+
0.9859, +0.0007 (95% CI +0.0002 to +0.0011), and higher on both suites.
|
| 56 |
+
* **Higher predicted quality than Gradium's Q4_K.** UTMOS 4.314 vs 4.281, +0.033 (95% CI +0.027 to +0.039).
|
| 57 |
+
* **Same size, same runtime.** A byte-for-byte drop-in for `model.q4k.gguf` in Gradium's repository; nothing else
|
| 58 |
+
changes.
|
| 59 |
+
|
| 60 |
+
It is **not identical to BF16**: its audio is still slightly further from BF16's than BF16 resampled with a second
|
| 61 |
+
seed, and its UTMOS is 0.019 lower than BF16's. See [Limitations](#limitations).
|
| 62 |
+
|
| 63 |
+
## Model Details
|
| 64 |
+
|
| 65 |
+
| Property | Value |
|
| 66 |
+
|---|---|
|
| 67 |
+
| Model ID | `webAI-Official/Gradium-Air` (proposed; not yet published) |
|
| 68 |
+
| Base model | [`gradium/phonon-31m-en`](https://huggingface.co/gradium/phonon-31m-en) at revision `92741c5b`, 31M parameters plus the Mimi audio decoder |
|
| 69 |
+
| Model file | `model.q4k.gguf`, 55,075,168 bytes (sha256 `ef1b486d7072d3db…`) |
|
| 70 |
+
| Format | GGUF. The 24 transformer matrices of the flow LM are 4-bit Q4_K, the same tensor types and layout as Gradium's `model.q4k.gguf`; every other tensor is unchanged |
|
| 71 |
+
| Runs on | Gradium's `ptts` runtime, CPU backend (`device="cpu"`, `quant="q4k"`). Tested with ptts 0.6.0 on Linux x86-64 and on macOS (Apple silicon) |
|
| 72 |
+
| Input / output | English text and a voice / mono 24 kHz audio |
|
| 73 |
+
| Voices | Gradium's [phonon-voices](https://huggingface.co/gradium/phonon-voices) catalog and the included `default-voice.safetensors` |
|
| 74 |
+
| License | webAI Non-Commercial License ver. 1.0 |
|
| 75 |
+
|
| 76 |
+
## Results
|
| 77 |
+
|
| 78 |
+
Every audio result comes from one test, run once: each file synthesized the same 2,576 clips (1,273 distinct
|
| 79 |
+
sentences in 2 voices) with stock ptts 0.6.0 on CPU, temperature 0.3, and the same seed per clip for every file.
|
| 80 |
+
Verdicts pair sentences (each sentence's 2 voices are averaged first) and use a bootstrap 95% interval. "BF16" is
|
| 81 |
+
Gradium's `model.safetensors`; "Gradium Q8_0" and "Gradium Q4_K" are the two GGUF files in Gradium's repository;
|
| 82 |
+
"llama.cpp Q4_K" is a second same-size file, the same matrices quantized to Q4_K with llama.cpp's standard quantizer;
|
| 83 |
+
the two "BF16" control rows rerun BF16 with the same seeds (runtime noise) and with a second seed (how much BF16
|
| 84 |
+
differs from itself when resampled).
|
| 85 |
+
|
| 86 |
+
### Pre-registered test
|
| 87 |
+
|
| 88 |
+
Gradium-Air passes all four rules, which were written down before the test ran.
|
| 89 |
+
|
| 90 |
+
| Check | Rule | Result (95% CI) | Pass |
|
| 91 |
+
|---|---|---|---|
|
| 92 |
+
| Primary: Gradium-Air vs BF16, word error rate | upper end of the 95% interval below +0.005, and no suite significantly worse | 0.0183 → 0.0191, +0.0008 (−0.0002 to +0.0018) | yes |
|
| 93 |
+
| Primary: Gradium-Air vs Gradium Q4_K, similarity to BF16 | lower end of the 95% interval above 0 | 0.9859 → 0.9866, +0.0007 (+0.0002 to +0.0011) | yes |
|
| 94 |
+
| Guardrail: Gradium-Air vs Gradium Q4_K, word error rate | not significantly worse | 0.0202 → 0.0191, −0.0011 (−0.0024 to +0.0002) | yes |
|
| 95 |
+
| Guardrail: Gradium-Air vs Gradium Q4_K, UTMOS | lower end of the 95% interval above −0.05 | 4.2814 → 4.3141, +0.0327 (+0.0267 to +0.0388) | yes |
|
| 96 |
+
|
| 97 |
+
These four results are confirmatory. Everything else on this page (per-suite rows outside the rules, the control
|
| 98 |
+
rows, and the screen) is supporting detail and should be read as preliminary.
|
| 99 |
+
|
| 100 |
+
### Word error rate
|
| 101 |
+
|
| 102 |
+
Gradium-Air shows no significant change from BF16, while both other 4-bit files are significantly worse than BF16.
|
| 103 |
+
|
| 104 |
+
| File | Size | All (1,273) | Seed-TTS English (1,073) | Hard (200) | Corpus WER | vs BF16, all, points (95% CI) | Verdict |
|
| 105 |
+
|---|---:|---:|---:|---:|---:|---|---|
|
| 106 |
+
| BF16 (baseline) | 102,823,920 B | 1.83% | 1.07% | 5.90% | 1.82% | baseline | |
|
| 107 |
+
| BF16, rerun with the same seeds | same file | 1.83% | 1.07% | 5.90% | 1.81% | −0.00 (−0.01 to +0.00) | no significant change |
|
| 108 |
+
| BF16, second seed | same file | 1.82% | 1.10% | 5.67% | 1.81% | −0.01 (−0.13 to +0.09) | no significant change |
|
| 109 |
+
| Gradium Q8_0 | 64,512,352 B | 1.82% | 1.04% | 6.05% | 1.81% | −0.01 (−0.08 to +0.06) | no significant change |
|
| 110 |
+
| Gradium Q4_K | 55,075,168 B | 2.02% | 1.23% | 6.30% | 1.98% | +0.19 (+0.06 to +0.32) | higher (worse) |
|
| 111 |
+
| llama.cpp Q4_K | 55,075,168 B | 1.96% | 1.14% | 6.31% | 1.93% | +0.13 (+0.03 to +0.22) | higher (worse) |
|
| 112 |
+
| **Gradium-Air** | 55,075,168 B | **1.91%** | **1.16%** | **5.98%** | **1.88%** | +0.08 (−0.02 to +0.18) | no significant change |
|
| 113 |
+
|
| 114 |
+
The columns give the mean per-sentence word error rate (each clip capped at 1; no clip reached the cap), with the
|
| 115 |
+
number of sentences in brackets; "Corpus WER" is total errors over total words. Per suite against BF16, Gradium-Air is
|
| 116 |
+
+0.09 points on Seed-TTS English (95% CI −0.00 to +0.18) and +0.08 on the hard sentences (−0.37 to +0.50), neither
|
| 117 |
+
significant; the Seed-TTS interval just touches zero. Against Gradium's Q4_K, Gradium-Air is −0.11 points (−0.24 to
|
| 118 |
+
+0.02), no significant change. The two BF16 controls show the test's own noise: the same-seed rerun differs only by
|
| 119 |
+
transcription noise, and a second seed moves the score by −0.01 points.
|
| 120 |
+
|
| 121 |
+

|
| 122 |
+
|
| 123 |
+
### Speaker similarity to BF16's audio
|
| 124 |
+
|
| 125 |
+
Gradium-Air is closer to BF16's audio than both other 4-bit files, but not as close as BF16 resampled.
|
| 126 |
+
|
| 127 |
+
| File | All | Seed-TTS English | Hard | vs Gradium Q4_K, all (95% CI) | Verdict |
|
| 128 |
+
|---|---:|---:|---:|---|---|
|
| 129 |
+
| BF16, rerun with the same seeds | 1.0000 | 1.0000 | 1.0000 | +0.0141 (+0.0136 to +0.0145) | higher |
|
| 130 |
+
| Gradium Q8_0 | 0.9956 | 0.9956 | 0.9959 | +0.0097 (+0.0093 to +0.0101) | higher |
|
| 131 |
+
| BF16, second seed | 0.9872 | 0.9866 | 0.9906 | +0.0013 (+0.0009 to +0.0018) | higher |
|
| 132 |
+
| Gradium Q4_K (baseline) | 0.9859 | 0.9855 | 0.9880 | baseline | |
|
| 133 |
+
| llama.cpp Q4_K | 0.9859 | 0.9853 | 0.9887 | −0.0000 (−0.0005 to +0.0004) | no significant change |
|
| 134 |
+
| **Gradium-Air** | **0.9866** | **0.9861** | **0.9890** | +0.0007 (+0.0002 to +0.0011) | higher |
|
| 135 |
+
|
| 136 |
+
Similarity is the cosine between speaker-verification embeddings of a file's clip and BF16's clip of the same
|
| 137 |
+
sentence, voice and seed. Gradium-Air beats Gradium's Q4_K on both suites: Seed-TTS English +0.0006 (+0.0001 to
|
| 138 |
+
+0.0011) and hard +0.0010 (+0.0001 to +0.0020). Against BF16 resampled with a second seed it is −0.0007 (−0.0011 to
|
| 139 |
+
−0.0002), significantly lower.
|
| 140 |
+
|
| 141 |
+

|
| 142 |
+
|
| 143 |
+
### Predicted quality (UTMOS)
|
| 144 |
+
|
| 145 |
+
Gradium-Air scores higher than Gradium's Q4_K but lower than BF16.
|
| 146 |
+
|
| 147 |
+
| File | All | Seed-TTS English | Hard | vs Gradium Q4_K, all (95% CI) | Verdict |
|
| 148 |
+
|---|---:|---:|---:|---|---|
|
| 149 |
+
| BF16 | 4.333 | 4.328 | 4.361 | +0.052 (+0.046 to +0.058) | higher |
|
| 150 |
+
| BF16, rerun with the same seeds | 4.333 | 4.328 | 4.361 | +0.052 (+0.046 to +0.058) | higher |
|
| 151 |
+
| BF16, second seed | 4.330 | 4.324 | 4.360 | +0.049 (+0.043 to +0.055) | higher |
|
| 152 |
+
| Gradium Q8_0 | 4.332 | 4.327 | 4.359 | +0.050 (+0.045 to +0.056) | higher |
|
| 153 |
+
| Gradium Q4_K (baseline) | 4.281 | 4.278 | 4.300 | baseline | |
|
| 154 |
+
| llama.cpp Q4_K | 4.283 | 4.277 | 4.311 | +0.001 (−0.005 to +0.007) | no significant change |
|
| 155 |
+
| **Gradium-Air** | **4.314** | **4.309** | **4.343** | +0.033 (+0.027 to +0.039) | higher |
|
| 156 |
+
|
| 157 |
+
Against BF16, Gradium-Air is −0.019 (−0.025 to −0.014), significantly lower.
|
| 158 |
+
|
| 159 |
+
### Against Gradium's Q4_K on the teacher-forced screen (separate protocol)
|
| 160 |
+
|
| 161 |
+
Gradium-Air's predictions are 20% closer to BF16's than Gradium's Q4_K's at the identical file size.
|
| 162 |
+
|
| 163 |
+
| File (55,075,168 B each) | Latent error | vs Gradium Q4_K (95% CI) | Verdict | End-of-speech logit error | Stop frame changed |
|
| 164 |
+
|---|---:|---|---|---:|---:|
|
| 165 |
+
| Gradium Q4_K (baseline) | 0.01608 | baseline | | 0.224 | 23 / 400 |
|
| 166 |
+
| llama.cpp Q4_K | 0.01390 | −0.00218 (−0.00249 to −0.00189) | lower | 0.195 | 16 / 400 |
|
| 167 |
+
| **Gradium-Air** | **0.01284** | −0.00324 (−0.00357 to −0.00293) | lower | 0.209 | 8 / 400 |
|
| 168 |
+
|
| 169 |
+
This screen does not synthesize audio. Each file's model predicts every next audio latent from BF16's own previous
|
| 170 |
+
frames, with the same text, voice and noise, and the error is the mean squared difference from BF16's prediction. It
|
| 171 |
+
covers 400 items (200 sentences × 2 voices), was run once, and uses transformers on CPU in f32, not ptts. Gradium-Air is
|
| 172 |
+
lower on both sentence sources: WikiText-2 (320 items) 0.01640 → 0.01318, −0.00322 (−0.00356 to −0.00290), and
|
| 173 |
+
hand-written numbers and dates (80 items) 0.01478 → 0.01148, −0.00331 (−0.00406 to −0.00261). Against llama.cpp's Q4_K
|
| 174 |
+
it is also lower, −0.00106 (−0.00130 to −0.00082), but its end-of-speech logit error is higher than that file's (0.209
|
| 175 |
+
vs 0.195). Against Gradium's Q4_K that error is lower, −0.0151 (−0.0176 to −0.0126), and Gradium-Air changes the stop
|
| 176 |
+
frame least often.
|
| 177 |
+
|
| 178 |
+

|
| 179 |
+
|
| 180 |
+
## Files
|
| 181 |
+
|
| 182 |
+
| File | What it is |
|
| 183 |
+
|---|---|
|
| 184 |
+
| `model.q4k.gguf` | Gradium-Air (55,075,168 bytes, sha256 `ef1b486d7072d3dbd4634b6f846cd818e6fa129a819e928da23e7287b0920fba`); modified from Gradium's `model.q4k.gguf` |
|
| 185 |
+
| `default-voice.safetensors` | Gradium's default voice, unchanged (256,136 bytes, sha256 `dc537478eaadc1a5257fcda2e3501ab81e14c111a5d95a5838f350db2afd0f1a`) |
|
| 186 |
+
| `config.json`, `generation_config.json`, `processor_config.json`, `chat_template.jinja`, `tokenizer.json`, `tokenizer.5.0.0.json`, `tokenizer.model`, `tokenizer_config.json` | Gradium's configuration and tokenizer files, unchanged from revision `92741c5b`, so the folder runs on its own |
|
| 187 |
+
| `LICENSE.md` | webAI Non-Commercial License ver. 1.0 |
|
| 188 |
+
| `apache-2.0-LICENSE.txt` | Apache License 2.0, the license of Gradium's phonon-31m-en |
|
| 189 |
+
| `figures/` | The charts in this card |
|
| 190 |
+
|
| 191 |
+
Check the downloads with `shasum -a 256 model.q4k.gguf default-voice.safetensors` against the hashes above.
|
| 192 |
+
|
| 193 |
+
## Quick start
|
| 194 |
+
|
| 195 |
+
```bash
|
| 196 |
+
pip install ptts==0.6.0
|
| 197 |
+
hf download webAI-Official/Gradium-Air --local-dir Gradium-Air && cd Gradium-Air
|
| 198 |
+
ptts -m . -l en -q q4k -d cpu -v Easton "Gradium-Air read this sentence on a laptop CPU at 6:20 PM on November 8th, 2026." -o speech.wav
|
| 199 |
+
```
|
| 200 |
+
|
| 201 |
+
In Python:
|
| 202 |
+
|
| 203 |
+
```python
|
| 204 |
+
import ptts
|
| 205 |
+
|
| 206 |
+
tts = ptts.TTS(".", lang="en", quant="q4k", device="cpu")
|
| 207 |
+
tts.save("speech.wav", "Hello from Gradium-Air.", voice="Easton")
|
| 208 |
+
```
|
| 209 |
+
|
| 210 |
+
* The `ptts` command was run from inside this folder with ptts 0.6.0; `quant="q4k"` selects `model.q4k.gguf`, and
|
| 211 |
+
`device="cpu"` is the backend this file was tested on.
|
| 212 |
+
* The evaluation used temperature 0.3 (the model's default) and a fixed seed per sentence; pass `seed=` (Python) or
|
| 213 |
+
`-s` (command line) for reproducible audio.
|
| 214 |
+
* Voices other than the default come from Gradium's voice catalog and are downloaded on first use; every voice in
|
| 215 |
+
[gradium/phonon-voices](https://huggingface.co/gradium/phonon-voices) works by name or ID.
|
| 216 |
+
* Streaming (`tts.stream(...)`) and the other runtimes in [Gradium's guide](https://huggingface.co/gradium/phonon-31m-en)
|
| 217 |
+
work as with Gradium's files; only the CPU backend was tested with this file.
|
| 218 |
+
|
| 219 |
+
## Limitations
|
| 220 |
+
|
| 221 |
+
* **Not identical to BF16.** Its audio is less similar to BF16's (0.9866) than BF16 resampled with a second seed
|
| 222 |
+
(0.9872), a significant difference of −0.0007, and its UTMOS is 0.019 lower than BF16's (significant). If the
|
| 223 |
+
closest possible match to BF16 matters more than size, Gradium's Q8_0 (64.5 MB) is closer to BF16 on every measure
|
| 224 |
+
here.
|
| 225 |
+
* **Small effects.** The similarity gain over Gradium's Q4_K is +0.0007 on a 0–1 scale; it is significant over 1,273
|
| 226 |
+
sentences but small. The word-error-rate improvement over Gradium's Q4_K (−0.11 points) is not significant.
|
| 227 |
+
* **One run.** Each file was measured once on one test set, with one seed per clip; only BF16 was also resampled.
|
| 228 |
+
The screen was also run once.
|
| 229 |
+
* **Scope.** English only, two voices (Easton and Master Aldric), sentences of up to about 10 seconds of audio. Long-form
|
| 230 |
+
reading, other voices, voice cloning and listening tests were not evaluated. Measures are automatic (speech
|
| 231 |
+
recognition, speaker embeddings, predicted quality), not human ratings.
|
| 232 |
+
* **CPU only.** Results hold for ptts's CPU backend. ptts's other backends (Core ML, CUDA, Metal, Vulkan and the
|
| 233 |
+
browser build) were not tested with this file.
|
| 234 |
+
* **Speed was not measured** under controlled conditions. The file has the same size and tensor types as Gradium's
|
| 235 |
+
Q4_K. One example on an Apple silicon Mac: the quick-start command produced 6.3 s of audio in 1.8 s, model loading
|
| 236 |
+
included.
|
| 237 |
+
* **Which file.** Results hold for the included `model.q4k.gguf` (sha256 above), byte-identical to the file that was
|
| 238 |
+
evaluated.
|
| 239 |
+
|
| 240 |
+
## Evaluation protocol
|
| 241 |
+
|
| 242 |
+
* **Synthesis:** stock ptts 0.6.0 (Linux wheel), CPU backend, 128 single-thread processes on an AMD EPYC 9355 machine
|
| 243 |
+
(container limited to 13.6 CPUs);
|
| 244 |
+
temperature 0.3; one seed per clip, derived from the clip's id and shared by every file; the "second seed" control
|
| 245 |
+
uses a different seed per clip. Voice catalog `gradium/phonon-voices` at revision `0ee17205`. BF16 ran from
|
| 246 |
+
`model.safetensors` in f32, Gradium Q8_0 as `q8_0`, and the three Q4_K files as `q4k`. Every file rendered all 2,576
|
| 247 |
+
clips on CPU. Clips with unusually long audio (≥ 29.5 s, or > 1.5 s per word) numbered 0–4 per file.
|
| 248 |
+
* **Sentences:** the 1,088 English target sentences of the Seed-TTS evaluation set (Hugging Face mirror
|
| 249 |
+
`zhaochenyang20/seed-tts-eval` at revision `8f5e1aa2`; 1,073 distinct) and 200 templated sentences full of numbers,
|
| 250 |
+
dates, money, codes and units (the "hard" suite), each in the voices Easton and Master Aldric: 2,576 clips, 1,273
|
| 251 |
+
distinct sentences.
|
| 252 |
+
* **Word error rate:** Whisper large-v3 (fp16, greedy, English), Whisper's English text normaliser on both reference
|
| 253 |
+
and transcript, then digit groups split at "." or ":" and "a m"/"p m" joined (so "3:51 p.m." and "3 51 pm" match);
|
| 254 |
+
word errors counted with jiwer, each clip capped at 1, then averaged over a sentence's 2 voices. Not comparable with
|
| 255 |
+
Gradium's published LLM-assisted word error rates.
|
| 256 |
+
* **Speaker similarity:** cosine between `microsoft/wavlm-base-plus-sv` speaker-verification embeddings of a file's clip
|
| 257 |
+
and BF16's clip of the same sentence, voice and seed, at 16 kHz.
|
| 258 |
+
* **Predicted quality:** UTMOS (`utmos22_strong` from `tarepan/SpeechMOS` v1.2.0), at 16 kHz.
|
| 259 |
+
* **Statistics:** every comparison pairs the same sentences (n = 1,273; 1,073 Seed-TTS and 200 hard); 95% intervals
|
| 260 |
+
from 2,000 bootstrap resamples of the per-sentence differences. "No significant change" means the interval includes
|
| 261 |
+
zero.
|
| 262 |
+
* **Pre-registration:** the question, primary measures, margins (+0.005 word error rate against BF16; UTMOS within
|
| 263 |
+
0.05 of Gradium's Q4_K) and pass rules were written down before the test ran.
|
| 264 |
+
* **Screen:** teacher-forced, 160 WikiText-2 test sentences and 40 hand-written sentences of numbers and dates in the
|
| 265 |
+
voices Hadley and Zoey (400 items); BF16 reference latents from transformers in f32 on CPU, temperature 0.3, one flow
|
| 266 |
+
step; latent error = mean squared difference of the predicted normalised latents; stop frame = first frame whose
|
| 267 |
+
end-of-speech logit exceeds 4.0.
|
| 268 |
+
|
| 269 |
+
## License and attribution
|
| 270 |
+
|
| 271 |
+
Released under the **webAI Non-Commercial License ver. 1.0** (`LICENSE.md`): non-commercial research and educational use
|
| 272 |
+
only. `model.q4k.gguf` is a modified version of the weights of
|
| 273 |
+
[`gradium/phonon-31m-en`](https://huggingface.co/gradium/phonon-31m-en), released by Gradium under the Apache License
|
| 274 |
+
2.0 (text kept in `apache-2.0-LICENSE.txt`). The unchanged files from Gradium's repository (configuration, tokenizer
|
| 275 |
+
and default voice) remain under Apache 2.0. All credit for the base model goes to Gradium; as its card notes, Phonon
|
| 276 |
+
builds on [Pocket TTS](https://github.com/kyutai-labs/pocket-tts) and the Mimi codec, developed by Kyutai.
|
| 277 |
+
|
| 278 |
+
Please follow Gradium's responsible-use guidance: do not use the model to impersonate a real person, to deceive
|
| 279 |
+
listeners about who is speaking, or to clone a voice without the speaker's consent, and tell listeners when speech is
|
| 280 |
+
synthetic where that matters to them.
|
apache-2.0-LICENSE.txt
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 6 |
+
|
| 7 |
+
1. Definitions.
|
| 8 |
+
|
| 9 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 10 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 11 |
+
|
| 12 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 13 |
+
the copyright owner that is granting the License.
|
| 14 |
+
|
| 15 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 16 |
+
other entities that control, are controlled by, or are under common
|
| 17 |
+
control with that entity. For the purposes of this definition,
|
| 18 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 19 |
+
direction or management of such entity, whether by contract or
|
| 20 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 21 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 22 |
+
|
| 23 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 24 |
+
exercising permissions granted by this License.
|
| 25 |
+
|
| 26 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 27 |
+
including but not limited to software source code, documentation
|
| 28 |
+
source, and configuration files.
|
| 29 |
+
|
| 30 |
+
"Object" form shall mean any form resulting from mechanical
|
| 31 |
+
transformation or translation of a Source form, including but
|
| 32 |
+
not limited to compiled object code, generated documentation,
|
| 33 |
+
and conversions to other media types.
|
| 34 |
+
|
| 35 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 36 |
+
Object form, made available under the License, as indicated by a
|
| 37 |
+
copyright notice that is included in or attached to the work
|
| 38 |
+
(an example is provided in the Appendix below).
|
| 39 |
+
|
| 40 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 41 |
+
form, that is based on (or derived from) the Work and for which the
|
| 42 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 43 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 44 |
+
of this License, Derivative Works shall not include works that remain
|
| 45 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 46 |
+
the Work and Derivative Works thereof.
|
| 47 |
+
|
| 48 |
+
"Contribution" shall mean any work of authorship, including
|
| 49 |
+
the original version of the Work and any modifications or additions
|
| 50 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 51 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 52 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 53 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 54 |
+
means any form of electronic, verbal, or written communication sent
|
| 55 |
+
to the Licensor or its representatives, including but not limited to
|
| 56 |
+
communication on electronic mailing lists, source code control systems,
|
| 57 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 58 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 59 |
+
excluding communication that is conspicuously marked or otherwise
|
| 60 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 61 |
+
|
| 62 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 63 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 64 |
+
subsequently incorporated within the Work.
|
| 65 |
+
|
| 66 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 67 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 68 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 69 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 70 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 71 |
+
Work and such Derivative Works in Source or Object form.
|
| 72 |
+
|
| 73 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 74 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 75 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 76 |
+
(except as stated in this section) patent license to make, have made,
|
| 77 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 78 |
+
where such license applies only to those patent claims licensable
|
| 79 |
+
by such Contributor that are necessarily infringed by their
|
| 80 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 81 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 82 |
+
institute patent litigation against any entity (including a
|
| 83 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 84 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 85 |
+
or contributory patent infringement, then any patent licenses
|
| 86 |
+
granted to You under this License for that Work shall terminate
|
| 87 |
+
as of the date such litigation is filed.
|
| 88 |
+
|
| 89 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 90 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 91 |
+
modifications, and in Source or Object form, provided that You
|
| 92 |
+
meet the following conditions:
|
| 93 |
+
|
| 94 |
+
(a) You must give any other recipients of the Work or
|
| 95 |
+
Derivative Works a copy of this License; and
|
| 96 |
+
|
| 97 |
+
(b) You must cause any modified files to carry prominent notices
|
| 98 |
+
stating that You changed the files; and
|
| 99 |
+
|
| 100 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 101 |
+
that You distribute, all copyright, patent, trademark, and
|
| 102 |
+
attribution notices from the Source form of the Work,
|
| 103 |
+
excluding those notices that do not pertain to any part of
|
| 104 |
+
the Derivative Works; and
|
| 105 |
+
|
| 106 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 107 |
+
distribution, then any Derivative Works that You distribute must
|
| 108 |
+
include a readable copy of the attribution notices contained
|
| 109 |
+
within such NOTICE file, excluding those notices that do not
|
| 110 |
+
pertain to any part of the Derivative Works, in at least one
|
| 111 |
+
of the following places: within a NOTICE text file distributed
|
| 112 |
+
as part of the Derivative Works; within the Source form or
|
| 113 |
+
documentation, if provided along with the Derivative Works; or,
|
| 114 |
+
within a display generated by the Derivative Works, if and
|
| 115 |
+
wherever such third-party notices normally appear. The contents
|
| 116 |
+
of the NOTICE file are for informational purposes only and
|
| 117 |
+
do not modify the License. You may add Your own attribution
|
| 118 |
+
notices within Derivative Works that You distribute, alongside
|
| 119 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 120 |
+
that such additional attribution notices cannot be construed
|
| 121 |
+
as modifying the License.
|
| 122 |
+
|
| 123 |
+
You may add Your own copyright statement to Your modifications and
|
| 124 |
+
may provide additional or different license terms and conditions
|
| 125 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 126 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 127 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 128 |
+
the conditions stated in this License.
|
| 129 |
+
|
| 130 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 131 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 132 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 133 |
+
this License, without any additional terms or conditions.
|
| 134 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 135 |
+
the terms of any separate license agreement you may have executed
|
| 136 |
+
with Licensor regarding such Contributions.
|
| 137 |
+
|
| 138 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 139 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 140 |
+
except as required for reasonable and customary use in describing the
|
| 141 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 142 |
+
|
| 143 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 144 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 145 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 146 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 147 |
+
implied, including, without limitation, any warranties or conditions
|
| 148 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 149 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 150 |
+
appropriateness of using or redistributing the Work and assume any
|
| 151 |
+
risks associated with Your exercise of permissions under this License.
|
| 152 |
+
|
| 153 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 154 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 155 |
+
unless required by applicable law (such as deliberate and grossly
|
| 156 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 157 |
+
liable to You for damages, including any direct, indirect, special,
|
| 158 |
+
incidental, or consequential damages of any character arising as a
|
| 159 |
+
result of this License or out of the use or inability to use the
|
| 160 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 161 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 162 |
+
other commercial damages or losses), even if such Contributor
|
| 163 |
+
has been advised of the possibility of such damages.
|
| 164 |
+
|
| 165 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 166 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 167 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 168 |
+
or other liability obligations and/or rights consistent with this
|
| 169 |
+
License. However, in accepting such obligations, You may act only
|
| 170 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 171 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 172 |
+
defend, and hold each Contributor harmless for any liability
|
| 173 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 174 |
+
of your accepting any such warranty or additional liability.
|
| 175 |
+
|
| 176 |
+
END OF TERMS AND CONDITIONS
|
| 177 |
+
|
| 178 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 179 |
+
|
| 180 |
+
To apply the Apache License to your work, attach the following
|
| 181 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 182 |
+
replaced with your own identifying information. (Don't include
|
| 183 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 184 |
+
comment syntax for the file format. We also recommend that a
|
| 185 |
+
file or class name and description of purpose be included on the
|
| 186 |
+
same "printed page" as the copyright notice for easier
|
| 187 |
+
identification within third-party archives.
|
| 188 |
+
|
| 189 |
+
Copyright [yyyy] [name of copyright owner]
|
| 190 |
+
|
| 191 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 192 |
+
you may not use this file except in compliance with the License.
|
| 193 |
+
You may obtain a copy of the License at
|
| 194 |
+
|
| 195 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 196 |
+
|
| 197 |
+
Unless required by applicable law or agreed to in writing, software
|
| 198 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 199 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 200 |
+
See the License for the specific language governing permissions and
|
| 201 |
+
limitations under the License.
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set training = messages | length == 2 -%}
|
| 2 |
+
{%- if messages | length > 2 or messages[0]['role'] != 'user' or (training and messages[1]['role'] != 'assistant') -%}
|
| 3 |
+
{{- raise_exception('Gradium Phonon TTS takes one `user` message per conversation, followed by the `assistant` target audio to train on.') -}}
|
| 4 |
+
{%- endif -%}
|
| 5 |
+
{%- if training -%}
|
| 6 |
+
{%- set target = messages[1]['content'] -%}
|
| 7 |
+
{%- if target is string or target | length != 1 or target[0]['type'] != 'audio' -%}
|
| 8 |
+
{{- raise_exception('The `assistant` turn holds one `audio` item, the target.') -}}
|
| 9 |
+
{%- endif -%}
|
| 10 |
+
{%- endif -%}
|
| 11 |
+
{%- set content = messages[0]['content'] -%}
|
| 12 |
+
{%- if content is string -%}
|
| 13 |
+
{%- set texts = [content] -%}
|
| 14 |
+
{%- set num_audio = 0 -%}
|
| 15 |
+
{%- else -%}
|
| 16 |
+
{%- set texts = content | selectattr('type', 'equalto', 'text') | map(attribute='text') | list -%}
|
| 17 |
+
{%- set num_audio = content | selectattr('type', 'equalto', 'audio') | list | length -%}
|
| 18 |
+
{%- endif -%}
|
| 19 |
+
{%- if texts | length > 1 or (texts | length == 0 and not training) or num_audio > 1 -%}
|
| 20 |
+
{{- raise_exception('The `user` turn holds one `text` item and at most one `audio` item (the voice).') -}}
|
| 21 |
+
{%- endif -%}
|
| 22 |
+
{%- set voice = '<|audio_prompt|>' if num_audio else '' -%}
|
| 23 |
+
{%- set text = (texts[0] if texts else '') | trim -%}
|
| 24 |
+
{%- if training -%}
|
| 25 |
+
{{- voice ~ text ~ '<|audio_bos|><|audio_prompt|>' -}}
|
| 26 |
+
{%- else -%}
|
| 27 |
+
{{- voice ~ text -}}
|
| 28 |
+
{%- if add_generation_prompt -%}
|
| 29 |
+
{{- '<|audio_bos|>' -}}
|
| 30 |
+
{%- endif -%}
|
| 31 |
+
{%- endif -%}
|
config.json
ADDED
|
@@ -0,0 +1,204 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"languages": [
|
| 3 |
+
"en"
|
| 4 |
+
],
|
| 5 |
+
"embeddings_ext": [
|
| 6 |
+
"ba7a5a45@300"
|
| 7 |
+
],
|
| 8 |
+
"flow_lm": {
|
| 9 |
+
"d_model": 512,
|
| 10 |
+
"num_heads": 8,
|
| 11 |
+
"num_layers": 6,
|
| 12 |
+
"dim_feedforward": 2048,
|
| 13 |
+
"max_period": 10000.0,
|
| 14 |
+
"n_bins": 4000,
|
| 15 |
+
"lut_dim": 512,
|
| 16 |
+
"flow_dim": 512,
|
| 17 |
+
"flow_depth": 6,
|
| 18 |
+
"ldim": 32,
|
| 19 |
+
"time_rms_norm": "rms",
|
| 20 |
+
"gelu": "erf"
|
| 21 |
+
},
|
| 22 |
+
"mimi": {
|
| 23 |
+
"channels": 1,
|
| 24 |
+
"sample_rate": 24000,
|
| 25 |
+
"frame_rate": 12.5,
|
| 26 |
+
"dimension": 512,
|
| 27 |
+
"quantizer_dimension": 32,
|
| 28 |
+
"quantizer_output_dimension": 512,
|
| 29 |
+
"n_filters": 64,
|
| 30 |
+
"n_residual_layers": 1,
|
| 31 |
+
"ratios": [
|
| 32 |
+
6,
|
| 33 |
+
5,
|
| 34 |
+
4
|
| 35 |
+
],
|
| 36 |
+
"kernel_size": 7,
|
| 37 |
+
"last_kernel_size": 3,
|
| 38 |
+
"residual_kernel_size": 3,
|
| 39 |
+
"dilation_base": 2,
|
| 40 |
+
"compress": 2,
|
| 41 |
+
"transformer_d_model": 512,
|
| 42 |
+
"transformer_num_heads": 8,
|
| 43 |
+
"transformer_num_layers": 2,
|
| 44 |
+
"transformer_layer_scale": 0.01,
|
| 45 |
+
"transformer_context": 250,
|
| 46 |
+
"transformer_max_period": 10000.0,
|
| 47 |
+
"transformer_dim_feedforward": 2048,
|
| 48 |
+
"gelu": "erf"
|
| 49 |
+
},
|
| 50 |
+
"temp": 0.8,
|
| 51 |
+
"lsd_decode_steps": 1,
|
| 52 |
+
"eos_threshold": 4.0,
|
| 53 |
+
"model_id": {
|
| 54 |
+
"sig": "5f0a2aab",
|
| 55 |
+
"epoch": 200,
|
| 56 |
+
"mimi_sig": "b77c7ffa",
|
| 57 |
+
"mimi_epoch": 150
|
| 58 |
+
},
|
| 59 |
+
"weights_name": "model.safetensors",
|
| 60 |
+
"tokenizer_name": "tokenizer.model",
|
| 61 |
+
"conditioners": [
|
| 62 |
+
{
|
| 63 |
+
"type": "lut",
|
| 64 |
+
"lut": {
|
| 65 |
+
"n_bins": 31,
|
| 66 |
+
"dim": 16,
|
| 67 |
+
"tokenizer": "noop",
|
| 68 |
+
"possible_values": [
|
| 69 |
+
"0",
|
| 70 |
+
"1",
|
| 71 |
+
"2"
|
| 72 |
+
]
|
| 73 |
+
},
|
| 74 |
+
"name": "num_speakers"
|
| 75 |
+
},
|
| 76 |
+
{
|
| 77 |
+
"type": "continuous",
|
| 78 |
+
"continuous": {
|
| 79 |
+
"scale_factor": 1000.0,
|
| 80 |
+
"dim": 128,
|
| 81 |
+
"max_period": 10000.0
|
| 82 |
+
},
|
| 83 |
+
"name": "duration_delta"
|
| 84 |
+
}
|
| 85 |
+
],
|
| 86 |
+
"fuser": {
|
| 87 |
+
"cross_attention_pos_emb": false,
|
| 88 |
+
"cross_attention_pos_emb_scale": 1,
|
| 89 |
+
"sum": [
|
| 90 |
+
"num_speakers",
|
| 91 |
+
"duration_delta"
|
| 92 |
+
],
|
| 93 |
+
"streaming_sum": [],
|
| 94 |
+
"prepend": [
|
| 95 |
+
"transcript_in_segment",
|
| 96 |
+
"speaker_wavs"
|
| 97 |
+
],
|
| 98 |
+
"cross": [],
|
| 99 |
+
"pad_prepend": false
|
| 100 |
+
},
|
| 101 |
+
"speaker_mimi": {
|
| 102 |
+
"prefix": "mimi_speaker",
|
| 103 |
+
"mimi": {
|
| 104 |
+
"channels": 1,
|
| 105 |
+
"sample_rate": 24000,
|
| 106 |
+
"frame_rate": 12.5,
|
| 107 |
+
"dimension": 512,
|
| 108 |
+
"quantizer_dimension": 256,
|
| 109 |
+
"quantizer_output_dimension": 512,
|
| 110 |
+
"n_filters": 64,
|
| 111 |
+
"n_residual_layers": 1,
|
| 112 |
+
"ratios": [
|
| 113 |
+
8,
|
| 114 |
+
6,
|
| 115 |
+
5,
|
| 116 |
+
4
|
| 117 |
+
],
|
| 118 |
+
"kernel_size": 7,
|
| 119 |
+
"last_kernel_size": 3,
|
| 120 |
+
"residual_kernel_size": 3,
|
| 121 |
+
"dilation_base": 2,
|
| 122 |
+
"compress": 2,
|
| 123 |
+
"transformer_d_model": 512,
|
| 124 |
+
"transformer_num_heads": 8,
|
| 125 |
+
"transformer_num_layers": 8,
|
| 126 |
+
"transformer_layer_scale": 0.01,
|
| 127 |
+
"transformer_context": 250,
|
| 128 |
+
"transformer_max_period": 10000.0,
|
| 129 |
+
"transformer_dim_feedforward": 2048,
|
| 130 |
+
"gelu": "erf"
|
| 131 |
+
}
|
| 132 |
+
},
|
| 133 |
+
"audio_prompt_min_duration": 2.5,
|
| 134 |
+
"architectures": [
|
| 135 |
+
"GradiumPhononTTSForConditionalGeneration"
|
| 136 |
+
],
|
| 137 |
+
"attention_bias": false,
|
| 138 |
+
"attention_dropout": 0.0,
|
| 139 |
+
"audio_bos_token_id": 4,
|
| 140 |
+
"audio_latent_token_id": 6,
|
| 141 |
+
"audio_prompt_dim": 512,
|
| 142 |
+
"audio_prompt_token_id": 5,
|
| 143 |
+
"bos_token_id": null,
|
| 144 |
+
"dtype": "bfloat16",
|
| 145 |
+
"duration_delta": 0.0,
|
| 146 |
+
"duration_delta_embedding_dim": 128,
|
| 147 |
+
"duration_delta_max_period": 10000.0,
|
| 148 |
+
"duration_delta_scale": 1000.0,
|
| 149 |
+
"eos_token_id": null,
|
| 150 |
+
"flow_head_config": {
|
| 151 |
+
"_name_or_path": "",
|
| 152 |
+
"architectures": null,
|
| 153 |
+
"chunk_size_feed_forward": 0,
|
| 154 |
+
"condition_dim": 512,
|
| 155 |
+
"dtype": null,
|
| 156 |
+
"flow_type": "lsd",
|
| 157 |
+
"frequency_embedding_size": 256,
|
| 158 |
+
"hidden_act": "silu",
|
| 159 |
+
"hidden_size": 512,
|
| 160 |
+
"id2label": {
|
| 161 |
+
"0": "LABEL_0",
|
| 162 |
+
"1": "LABEL_1"
|
| 163 |
+
},
|
| 164 |
+
"is_encoder_decoder": false,
|
| 165 |
+
"label2id": {
|
| 166 |
+
"LABEL_0": 0,
|
| 167 |
+
"LABEL_1": 1
|
| 168 |
+
},
|
| 169 |
+
"latent_dim": 32,
|
| 170 |
+
"layer_norm_eps": 1e-06,
|
| 171 |
+
"max_period": 10000.0,
|
| 172 |
+
"model_type": "",
|
| 173 |
+
"num_hidden_layers": 6,
|
| 174 |
+
"output_attentions": false,
|
| 175 |
+
"output_hidden_states": false,
|
| 176 |
+
"problem_type": null,
|
| 177 |
+
"return_dict": true,
|
| 178 |
+
"rms_norm_eps": 1e-05
|
| 179 |
+
},
|
| 180 |
+
"frame_rate": 12.5,
|
| 181 |
+
"head_dim": 64,
|
| 182 |
+
"hidden_act": "gelu",
|
| 183 |
+
"hidden_size": 512,
|
| 184 |
+
"initializer_range": 0.02,
|
| 185 |
+
"intermediate_size": 2048,
|
| 186 |
+
"latent_dim": 32,
|
| 187 |
+
"layer_norm_eps": 1e-05,
|
| 188 |
+
"max_position_embeddings": 8192,
|
| 189 |
+
"model_type": "gradium_phonon_tts",
|
| 190 |
+
"num_attention_heads": 8,
|
| 191 |
+
"num_hidden_layers": 6,
|
| 192 |
+
"num_key_value_heads": 8,
|
| 193 |
+
"num_speakers": 1,
|
| 194 |
+
"num_speakers_embedding_dim": 16,
|
| 195 |
+
"num_speakers_vocab_size": 32,
|
| 196 |
+
"pad_token_id": 3,
|
| 197 |
+
"rope_parameters": {
|
| 198 |
+
"rope_theta": 10000.0,
|
| 199 |
+
"rope_type": "default"
|
| 200 |
+
},
|
| 201 |
+
"transformers_version": "5.21.0.dev0",
|
| 202 |
+
"use_cache": true,
|
| 203 |
+
"vocab_size": 4001
|
| 204 |
+
}
|
default-voice.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dc537478eaadc1a5257fcda2e3501ab81e14c111a5d95a5838f350db2afd0f1a
|
| 3 |
+
size 256136
|
figures/benchmarks.png
ADDED
|
Git LFS Details
|
figures/latent_error.png
ADDED
|
Git LFS Details
|
figures/similarity.png
ADDED
|
Git LFS Details
|
figures/wer.png
ADDED
|
Git LFS Details
|
generation_config.json
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_sample": true,
|
| 3 |
+
"eos_threshold": 4.0,
|
| 4 |
+
"min_frames_before_eos": 0,
|
| 5 |
+
"num_flow_steps": 1,
|
| 6 |
+
"pad_token_id": 3,
|
| 7 |
+
"return_dict_in_generate": true,
|
| 8 |
+
"temperature": 0.3,
|
| 9 |
+
"transformers_version": "5.21.0.dev0"
|
| 10 |
+
}
|
model.q4k.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef1b486d7072d3dbd4634b6f846cd818e6fa129a819e928da23e7287b0920fba
|
| 3 |
+
size 55075168
|
processor_config.json
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audio_tokenizer": {
|
| 3 |
+
"audio_tokenizer_class": "PocketTTSAudioTokenizerModel",
|
| 4 |
+
"audio_tokenizer_name_or_path": "gradium/phonon-mimi"
|
| 5 |
+
},
|
| 6 |
+
"fade_in_duration": 0.0,
|
| 7 |
+
"feature_extractor": {
|
| 8 |
+
"analysis_frame_duration": 0.02,
|
| 9 |
+
"end_on_pause": true,
|
| 10 |
+
"fade_out_duration": 0.02,
|
| 11 |
+
"feature_extractor_type": "PocketTTSFeatureExtractor",
|
| 12 |
+
"feature_size": 1,
|
| 13 |
+
"frame_size": 1920,
|
| 14 |
+
"pad_to_multiple_of": 1920,
|
| 15 |
+
"padding_side": "right",
|
| 16 |
+
"padding_value": 0.0,
|
| 17 |
+
"pause_duration": 0.08,
|
| 18 |
+
"return_attention_mask": true,
|
| 19 |
+
"sampling_rate": 24000,
|
| 20 |
+
"silence_floor_db": 35.0
|
| 21 |
+
},
|
| 22 |
+
"frames_after_eos": 1,
|
| 23 |
+
"frames_after_eos_short_text": 3,
|
| 24 |
+
"processor_class": "PocketTTSProcessor",
|
| 25 |
+
"short_text_max_words": 4
|
| 26 |
+
}
|
tokenizer.5.0.0.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82e4572006d64d483317074c22c368e87779640b6f3a9dcd07b7caae32b030ce
|
| 3 |
+
size 65045
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"tokenizer_class": "TokenizersBackend",
|
| 3 |
+
"fast_tokenizer_files": [
|
| 4 |
+
"tokenizer.5.0.0.json"
|
| 5 |
+
],
|
| 6 |
+
"unk_token": "\u0000<unk>",
|
| 7 |
+
"bos_token": "\u0000<s>",
|
| 8 |
+
"eos_token": "\u0000</s>",
|
| 9 |
+
"pad_token": "\u0000<pad>",
|
| 10 |
+
"padding": true,
|
| 11 |
+
"padding_side": "left",
|
| 12 |
+
"return_tensors": "pt",
|
| 13 |
+
"model_input_names": [
|
| 14 |
+
"input_ids",
|
| 15 |
+
"attention_mask"
|
| 16 |
+
],
|
| 17 |
+
"clean_up_tokenization_spaces": false,
|
| 18 |
+
"model_specific_special_tokens": {
|
| 19 |
+
"audio_prompt_token": "<|audio_prompt|>",
|
| 20 |
+
"audio_latent_token": "<|audio_latent|>",
|
| 21 |
+
"audio_bos_token": "<|audio_bos|>"
|
| 22 |
+
}
|
| 23 |
+
}
|