Preview: user-facing v0.1 model card

#2
by jer3mi - opened
README.md CHANGED
@@ -18,259 +18,204 @@ language:
18
  - nl
19
  - sv
20
  - da
21
- base_model: ResembleAI/chatterbox
22
- base_model_relation: finetune
23
  ---
24
 
25
  # loudr-1
26
 
27
- **loudr-1** is a text-to-speech model that runs on your own machine. Twenty
28
- voices across ten languages, a new voice cloned from ten seconds of audio, and
29
- the same speech tokens from five implementations of the engine. No account, no
30
- network, no telemetry.
31
 
32
- It is the checkpoint for [`loudkit`](https://github.com/loudreader/loudkit):
33
- one file, mixed precision by module, with the algorithm's own settings recorded
34
- inside it so no backend has to guess them.
35
 
36
- ## Listen first
 
 
 
37
 
38
- Every voice has a playable sample on the
39
- [demo page](https://loudreader.github.io/loudkit/demo/). `joe` and `joe`
40
- read the identical sentence from *Alice in Wonderland*, so they are a real A/B
41
- of the model rather than of the material. Sample provenance, down to the seed,
42
- is recorded per voice in
43
- [provenance.json](https://github.com/loudreader/loudkit/blob/main/docs/voices/roster/provenance.json).
44
 
45
- ## Links
46
 
47
- - [GitHub](https://github.com/loudreader/loudkit): the engine, five
48
- implementations
49
- - [Documentation](https://loudreader.github.io/loudkit/): guides and reference
50
- - [Demo](https://loudreader.github.io/loudkit/demo/): hear the voices
51
- - [Voices](https://github.com/loudreader/loudkit/blob/main/VOICES.md): the full
52
- roster, with source and licence per voice
53
- - [Measured performance](https://loudreader.github.io/loudkit/benchmarks/):
54
- speed per device and per execution provider, with the command that reproduces
55
- each number
56
 
57
- ## Usage
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
  ```bash
60
  pip install "loudkit[torch,audio,hub]"
61
  ```
62
 
63
- The package is published alongside each release. If pip cannot find it yet,
64
- follow the [install instructions on
65
- GitHub](https://github.com/loudreader/loudkit#hear-it).
66
-
67
  ```python
68
  import loudkit as lk
69
 
70
  engine = lk.load("loudreader/loudr-1")
71
- narrator = lk.voice("joe", repo="loudreader/loudr-1")
72
 
73
- engine.synthesize("Hello from loudkit.", narrator, seed=7).save("hello.wav")
74
  ```
75
 
76
- Or from the shell:
 
77
 
78
  ```bash
79
- loudkit speak --checkpoint loudreader/loudr-1 --voice joe "Hello from loudkit."
 
80
  ```
81
 
82
- The first synthesis downloads this checkpoint (1.27 GB); everything after that
83
- runs from the cache. Cloning a voice takes ten seconds of clean audio and
84
- produces a ~300 KB file of tensors:
85
 
86
  ```python
87
  mine = lk.enroll("my-recording.wav", "loudreader/loudr-1", name="my-voice")
88
  mine.save("voices/my-voice.safetensors")
89
  ```
90
 
91
- ## What it is
92
-
93
- Two stages. A **token generator**, a Llama-architecture decoder (16 layers,
94
- hidden 1024, GQA with 4 KV heads, 145M parameters), writes discrete speech
95
- tokens at 25 Hz, autoregressively. A **renderer**, a conditional flow-matching
96
- mel decoder plus a HiFiGAN vocoder, turns those tokens into a 24 kHz waveform
97
- in one parallel pass.
98
-
99
- The two have opposite shapes, so every backend treats them differently. On
100
- Apple silicon the generator is faster on the CPU and the renderer is faster on
101
- the GPU.
102
-
103
- ## How it was made
104
-
105
- Distilled from [Chatterbox](https://github.com/resemble-ai/chatterbox) (MIT,
106
- Resemble AI) in three steps, each measured rather than assumed:
107
-
108
- 1. **Guidance distillation.** Classifier-free guidance runs the flow estimator
109
- twice per step. The student learns to emit the combined velocity directly,
110
- in one pass.
111
- 2. **Step distillation.** Six Euler integration steps to two, by training the
112
- student's two-step rollout endpoint against the teacher's six-step endpoint.
113
- 3. **Quantisation-aware training** for the estimator, after post-training
114
- quantisation failed: int8 PTQ dropped velocity-field correlation from 0.980
115
- to 0.924, and QAT with straight-through estimators recovered it.
116
-
117
- Distillation targeted seven product languages. The release ran against a frozen
118
- evaluation set of 11 languages × 50 sentences = 550 items, cut
119
- deterministically from the corpus (hash-sorted, stratified by length) with
120
- voices rotated so a gate cannot be passed by one speaker. Four gates, applied
121
- per language rather than in aggregate, because an average hides the one
122
- language that broke: teacher/student KL, length ratio, early end-of-speech
123
- rate, and word error rate against an ASR model. The shipped student cleared all
124
- four in every product language. Thresholds and per-language values are internal
125
- training records; this card states only the verdict.
126
-
127
- **Those four gates are metrics, not ears.** Clearing them means the student
128
- tracks the teacher and does not truncate or ramble. It does not mean a language
129
- sounds good. Listening quality has been evaluated for **English only**.
130
-
131
- ### The five language counts, and why they differ
132
-
133
- This project carries five counts. Conflating any two of them produces a false
134
- claim:
135
-
136
- | count | what it describes |
137
- | --- | --- |
138
- | **23** | the teacher model's language coverage |
139
- | **7** | the languages distillation optimised for |
140
- | **11** | the languages the release gates were measured on (× 50 sentences) |
141
- | **12** | the languages the text frontend verbalises numbers, dates and times in |
142
- | **10** | the languages voices actually ship for |
143
-
144
- Separately, the tokenizer inherits **31** two-letter language tags. That is a
145
- vocabulary fact, not a capability: the frontend refuses anything outside the 12
146
- it has grammars for.
147
-
148
- ## Precision, per module
149
-
150
- Mixed on purpose, and each decision has a measurement behind it:
151
-
152
- | module | precision | why |
153
- |---|---|---|
154
- | token generator | fp16 | teacher-forced median KL 1.29e-06 against fp32, top-1 agreement 99.9%. A sampling decision boundary annihilates error below it. |
155
- | flow estimator | fp16 | mel correlation 0.999999, waveform 0.9987 |
156
- | flow encoder | **fp32** | fp16 here gives mel correlation 0.619 and +22 dB of high-frequency energy |
157
- | vocoder | **fp32** | fp16 produces an audible tone at Nyquist: the source module accumulates phase to ~1400 cycles, where fp16 resolution is coarser than the per-sample increment |
158
-
159
- The same numerical change that is free for a discrete-token decoder is
160
- destructive for a continuous-output regressor, because there is no decision
161
- boundary downstream to absorb it.
162
-
163
- ## Determinism
164
-
165
- Same text, voice and seed give a **bit-identical waveform** on a given build,
166
- device and backend. Every backend implements one counter-based sampling law, so
167
- the same seed makes the same token decisions within the envelope documented in
168
- [the identity
169
- contract](https://github.com/loudreader/loudkit/blob/main/docs/reference/IDENTITY-CONTRACT.md).
170
- That is not free: `torch.multinomial` returns different samples for an
171
- identical probability vector on x86 and arm64.
172
-
173
- **Not** bit-identical across backends or across releases. Different hardware
174
- sums floating-point numbers in different orders, and engine changes re-base the
175
- goldens under a versioned contract. Token identity across backends holds *at
176
- matched precision*: fp16 in the generator flips roughly one token in a
177
- thousand, and one flip re-routes everything after it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
178
 
179
  ## Intended use
180
 
181
- Narration, accessibility, localisation, research: any application that speaks
182
- text. Local by design, with no account, no network and no telemetry.
183
-
184
- ## Limitations
185
-
186
- - **Model quality validated for English only.** The architecture is
187
- multilingual and the training data was not restricted to English. The text
188
- frontend can *spell out* other languages graphemically, including Polish,
189
- which has a full numbers→words/anglicism/respelling pipeline. Only English
190
- has been listening-evaluated. Token parity across the five implementations is
191
- measured for every language; how good the result sounds is not. A language
192
- nobody has listened to will be read, not spoken well. Listen before you ship
193
- it.
194
- - **~10 seconds per window.** Longer passages are split at sentence boundaries.
195
- The library does this, and where the breaks fall is audible.
196
- - **Marking is metadata only; there is no audio watermark.** From 2026-08-02
197
- the EU AI Act (Article 50) requires synthetic audio to carry a
198
- machine-readable marking. loudkit ships the **C2PA claim-only manifest**
199
- written into every saved WAV and every server response
200
- ([provenance](https://github.com/loudreader/loudkit/blob/main/docs/reference/provenance.md)).
201
- It is *unsigned*, so it is a disclosure aid, not proof of origin. An in-audio
202
- watermark was evaluated and not shipped: its detection statistic was not
203
- separable from the null. Any re-encode strips the metadata and a re-synthesis
204
- defeats it entirely, so disclosure remains the user's responsibility. See
205
- [RESPONSIBLE_USE.md](https://huggingface.co/loudreader/loudr-1/blob/main/RESPONSIBLE_USE.md).
206
- - **The generator dominates on Apple hardware.** The renderer is exported to
207
- CoreML and fast. The generator is not, and runs natively.
208
- - Prosody on unusual input (heavy punctuation, numbers, abbreviations) is where
209
- this model is weakest, as is common for the family.
210
-
211
- ## Voices
212
-
213
- Twenty shipped voice profiles across ten languages: English, Spanish, French,
214
- German, Italian, Polish, Portuguese, Dutch, Swedish, Danish. Two per language,
215
- one of each gender. Each is an enrollment of a recording made or released for
216
- speech-technology use, enrolled by this project's own pipeline. None are
217
- Resemble AI artefacts, and none are cloned from scraped audio.
218
-
219
- The release policy is that **every voice carries its licence and its consent
220
- basis**. Ten of the twenty are personal donations recorded expressly for
221
- building TTS voices (CC0, via the OHF-Voice / NabuCasa voice-datasets
222
- collection). The rest come from CC0 speech databases (NST Swedish and Danish)
223
- or CC-BY-4.0 corpora (Kyutai tts-voices, CML-TTS, Multilingual LibriSpeech,
224
- which are LibriVox-derived recordings whose terms credit the readers).
225
-
226
- Donor names, sources, licences, consent basis, reference construction and
227
- sha256 for every profile live in
228
- [provenance.json](https://github.com/loudreader/loudkit/blob/main/docs/voices/roster/provenance.json),
229
- alongside speaker similarity wherever it was recorded (0.89–0.96 across the
230
- profiles that carry the measurement).
231
-
232
- **No cloned voice ships with this project.** Enrollment code is included so you
233
- can create one from a voice you have the right to use.
234
-
235
- ## Risks, stated plainly
236
-
237
- This clones a voice from a few seconds of audio and reads arbitrary text in it,
238
- locally, with no gatekeeping. Those properties are the point, and they also
239
- make misuse cheap. Read
240
- [RESPONSIBLE_USE.md](https://huggingface.co/loudreader/loudr-1/blob/main/RESPONSIBLE_USE.md).
241
- Consent is not implied by a recording being public.
242
-
243
- ## Training data and compute
244
-
245
- The distillation used speech generated by the teacher model, plus recordings
246
- made or released for speech-technology use for the reference voices (see
247
- [Voices](#voices)). The teacher's own training data is Resemble AI's and is not
248
- documented here. Anyone relying on provenance should read that project's terms
249
- rather than ours.
250
-
251
- Hardware: two consumer RTX 3090 GPUs. The full campaign (corpus generation,
252
- scaling probes, final training, evaluation) ran over roughly two weeks at about
253
- 435–440 GPU-hours. Around half of that budget was corpus generation, which is
254
- teacher inference: producing the distillation data cost more than training the
255
- shipped student (~15 GPU-hours). The recipe involved more than distillation
256
- alone, and this card does not try to compress it. **A technical report covering
257
- everything that was done is pending.**
258
-
259
- ## Citing
260
-
261
- If you use the model or the toolkit:
262
-
263
- ```bibtex
264
- @software{loudkit2026,
265
- title = {loudkit: a portable text-to-speech engine},
266
- author = {LoudReader},
267
- year = {2026},
268
- url = {https://github.com/loudreader/loudkit}
269
- }
270
- ```
271
 
272
  ## Licence
273
 
274
  [Apache-2.0](https://huggingface.co/loudreader/loudr-1/blob/main/LICENSE).
275
- Component licences:
276
  [NOTICE](https://huggingface.co/loudreader/loudr-1/blob/main/NOTICE).
 
18
  - nl
19
  - sv
20
  - da
 
 
21
  ---
22
 
23
  # loudr-1
24
 
25
+ **Natural-sounding text-to-speech with twenty voices, ten languages and voice
26
+ cloning.**
 
 
27
 
28
+ loudr-1 runs on your own hardware through
29
+ [loudkit](https://github.com/pepinu/loudkit). Download it once and work offline
30
+ from Python, Swift, Go, Rust or TypeScript with PyTorch, ONNX Runtime or CoreML.
31
 
32
+ [**Hear all 20 voices**](https://pepinu.github.io/loudkit/demo/) |
33
+ [**Open in Colab**](https://colab.research.google.com/github/pepinu/loudkit/blob/main/notebooks/loudkit_quickstart.ipynb) |
34
+ [**GitHub**](https://github.com/pepinu/loudkit) |
35
+ [**Documentation**](https://pepinu.github.io/loudkit/)
36
 
37
+ ## Listen
 
 
 
 
 
38
 
39
+ **Joe**
40
 
41
+ <audio controls src="https://huggingface.co/loudreader/loudr-1/resolve/main/samples/joe.opus"></audio>
 
 
 
 
 
 
 
 
42
 
43
+ **Kathleen**
44
+
45
+ <audio controls src="https://huggingface.co/loudreader/loudr-1/resolve/main/samples/kathleen.opus"></audio>
46
+
47
+ Both voices read the same passage from *Alice's Adventures in Wonderland*.
48
+ [Open the gallery](https://pepinu.github.io/loudkit/demo/) to compare every
49
+ shipped voice with the enrollment reference used to create its profile.
50
+
51
+ > English is the only language we could evaluate ourselves by ear. We do not
52
+ > speak the other nine languages well enough to judge their naturalness
53
+ > reliably. If you do, please listen and share what sounds good or wrong.
54
+ > Feedback from native speakers is very welcome.
55
+
56
+ ## Start in Python
57
 
58
  ```bash
59
  pip install "loudkit[torch,audio,hub]"
60
  ```
61
 
 
 
 
 
62
  ```python
63
  import loudkit as lk
64
 
65
  engine = lk.load("loudreader/loudr-1")
66
+ voice = lk.voice("joe", repo="loudreader/loudr-1")
67
 
68
+ engine.synthesize("Hello from loudkit.", voice, seed=7).save("hello.wav")
69
  ```
70
 
71
+ The first run downloads the 747 MB synthesis checkpoint and the voices. Later
72
+ runs use the local cache. The same path from a shell is:
73
 
74
  ```bash
75
+ loudkit speak --checkpoint loudreader/loudr-1 --voice joe \
76
+ "Hello from loudkit." -o hello.wav
77
  ```
78
 
79
+ To enroll a voice that you own or have permission to use:
 
 
80
 
81
  ```python
82
  mine = lk.enroll("my-recording.wav", "loudreader/loudr-1", name="my-voice")
83
  mine.save("voices/my-voice.safetensors")
84
  ```
85
 
86
+ The reusable profile is about 150 KB. Install
87
+ `loudkit[torch,audio,enroll,hub]` for enrollment.
88
+
89
+ ## Choose your runtime
90
+
91
+ The repository contains all supported formats, but the downloader fetches only
92
+ the runtime you select. Add `--with-cloning` when the installation also needs
93
+ enrollment.
94
+
95
+ | path | command | download |
96
+ |---|---|---:|
97
+ | Python, synthesis | `loudkit download loudreader/loudr-1 --for torch` | 750 MB |
98
+ | Python, with cloning | `--for torch --with-cloning` | 1.28 GB |
99
+ | JS, Go or Rust with ONNX | `--for onnx` | 2.60 GB |
100
+ | ONNX, with cloning | `--for onnx --with-cloning` | 3.13 GB |
101
+ | Swift or Python with CoreML | `--for coreml` | 1.16 GB |
102
+ | CoreML, with cloning | `--for coreml --with-cloning` | 1.69 GB |
103
+
104
+ Add `--local-dir loudr-1` to create a portable directory instead of using the
105
+ shared cache. The synthesis checkpoint will be at
106
+ `loudr-1/loudr-1.safetensors`.
107
+
108
+ ### Measured speed
109
+
110
+ | path | hardware | real-time factor |
111
+ |---|---|---:|
112
+ | PyTorch with CUDA graphs | RTX 3090 | 7.47x |
113
+ | PyTorch with CUDA graphs | Jetson Orin Nano | 1.83x |
114
+ | split PyTorch engine | Apple M3 Pro | 3.43x |
115
+ | ONNX Runtime, CPU provider | Apple M3 Pro | 1.21x |
116
+ | PyTorch CPU reference | Apple M3 Pro | 0.33x |
117
+
118
+ Higher is faster, and 1.0x means real time. ONNX Runtime on the measured M3 Pro
119
+ CPU is faster than real time. The PyTorch CPU reference path on the same machine
120
+ is not.
121
+
122
+ For batched workloads, the token generator reaches 20.1x aggregate throughput
123
+ at batch 1 and 153.1x at batch 64 on the RTX 3090. The highest measured result
124
+ is 170.8x on an A100 at batch 64. These are generator-only throughput numbers,
125
+ not single-request latency or end-to-end RTF. See the
126
+ [benchmark report](https://pepinu.github.io/loudkit/benchmarks/) for commands,
127
+ hardware and caveats.
128
+
129
+ ## What ships
130
+
131
+ | artefact | size | used by |
132
+ |---|---:|---|
133
+ | `loudr-1.safetensors` | 747 MB | synthesis |
134
+ | `loudr-1-enrollment.safetensors` | 523 MB | PyTorch enrollment |
135
+ | `ve.safetensors` | 5.7 MB | PyTorch enrollment |
136
+ | `onnx/` | 2.38 GB | nine graphs: six synthesis, three enrollment |
137
+ | `coreml/` | 941 MB | six packages: three synthesis, three enrollment |
138
+ | `voices/` | 3.1 MB | twenty voice profiles |
139
+ | `samples/` | 108 KB | the two players above |
140
+ | `tokenizer.json` | 70 KB | text processing |
141
+
142
+ Synthesis and enrollment are separate so users who only need speech generation
143
+ do not download the enrollment weights. ONNX and CoreML use their own enrollment
144
+ graphs. loudkit also verifies that paired model files came from the same source
145
+ checkpoint.
146
+
147
+ ## Voices and consent
148
+
149
+ The release includes two profiles for each of these languages: English,
150
+ Spanish, French, German, Italian, Polish, Portuguese, Dutch, Swedish and Danish.
151
+
152
+ The profiles were built from recordings donated for speech technology or from
153
+ CC0 and CC-BY speech corpora. No scraped celebrity voices ship with the model.
154
+ [The full roster](https://github.com/pepinu/loudkit/blob/main/VOICES.md) records
155
+ the source, licence and consent basis for every profile. The
156
+ [voice gallery](https://pepinu.github.io/loudkit/demo/) provides a generated
157
+ sample and enrollment preview for all twenty.
158
+
159
+ The source enrollment WAVs are not redistributed in the model repository. Their
160
+ digests, construction notes and the digests of every shipped profile and sample
161
+ are recorded in
162
+ [provenance.json](https://github.com/pepinu/loudkit/blob/main/docs/voices/roster/provenance.json).
163
+
164
+ ## Model lineage
165
+
166
+ loudr-1 is derived from
167
+ [Chatterbox](https://github.com/resemble-ai/chatterbox), released by Resemble AI
168
+ under the MIT licence. We optimized it for faster local inference by profiling
169
+ the full synthesis path, changing the signal flow, separating synthesis from
170
+ enrollment, and adjusting graph boundaries and device placement for PyTorch,
171
+ ONNX Runtime and CoreML.
172
+
173
+ Release gates compare the implementations, check output length and early end of
174
+ speech, and run ASR-based checks per measured language. These checks catch
175
+ mechanical regressions. They do not replace listening by native speakers.
176
+
177
+ ## Reproducibility
178
+
179
+ For a fixed build, device and backend, the same text, voice and seed produce the
180
+ same waveform. Across devices or backends, loudkit checks the token stream and
181
+ keeps waveform differences inside measured correlation bands. Floating-point
182
+ execution means that waveforms are not promised to be byte-identical across
183
+ different runtimes.
184
+
185
+ The exact contract and current measurements are in the
186
+ [identity contract](https://github.com/pepinu/loudkit/blob/main/docs/reference/IDENTITY-CONTRACT.md)
187
+ and [measured parity report](https://pepinu.github.io/loudkit/parity-measured/).
188
+
189
+ ## Before you ship
190
+
191
+ - Long passages are rendered in windows of about ten seconds. Sentence joins
192
+ can occasionally be audible.
193
+ - Difficult punctuation, numbers and abbreviations can change pronunciation or
194
+ prosody.
195
+ - Voice cloning requires consent. A recording being public does not grant
196
+ permission to clone the speaker.
197
+ - Saved WAVs and server responses include an unsigned C2PA Content Credentials
198
+ manifest by default. It records the model, voice, seed, backend and audio
199
+ digest in a machine-readable form.
200
+
201
+ Read [Responsible use](https://huggingface.co/loudreader/loudr-1/blob/main/RESPONSIBLE_USE.md)
202
+ before exposing enrollment to other people.
203
 
204
  ## Intended use
205
 
206
+ loudr-1 is intended for local narration, accessibility, localisation, games,
207
+ prototyping and speech research. It is not a voice-authentication system and
208
+ must not be used for deceptive impersonation.
209
+
210
+ ## Training data
211
+
212
+ The original Chatterbox training data is controlled by Resemble AI and is not
213
+ documented by this project. The shipped voice profiles use recordings made or
214
+ released for speech-technology use; their sources and licences are listed in
215
+ the public roster.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
216
 
217
  ## Licence
218
 
219
  [Apache-2.0](https://huggingface.co/loudreader/loudr-1/blob/main/LICENSE).
220
+ Upstream attributions and component licences are listed in
221
  [NOTICE](https://huggingface.co/loudreader/loudr-1/blob/main/NOTICE).
RESPONSIBLE_USE.md CHANGED
@@ -60,13 +60,18 @@ fingerprint, the seed, and the SHA-256 binding it to the audio bytes. See
60
  because signing needs a certificate and that is the deployer's choice. It says
61
  what made the file, not who vouches for it.
62
 
63
- **That is the whole of the machine-readable marking. There is no audio
64
- watermark.**
65
 
66
  ## What we ship, and what we will not
67
 
68
- We ship no voice cloned by us, from anyone, including the voices in our own
69
- product. The enrollment code is included so you can make your own, from a voice
 
 
 
 
 
70
  you have the right to use.
71
 
72
  Issues and pull requests asking for help with undisclosed impersonation, voice
 
60
  because signing needs a certificate and that is the deployer's choice. It says
61
  what made the file, not who vouches for it.
62
 
63
+ This is the machine-readable marking loudkit provides by default. Deployers can
64
+ add their own signing policy and disclosure around it.
65
 
66
  ## What we ship, and what we will not
67
 
68
+ We ship twenty voice profiles enrolled from recordings donated for speech
69
+ technology or released under terms that permit this use. Every profile has a
70
+ named source, licence and consent basis in the public roster. We do not ship
71
+ profiles made from private recordings or recordings published for an unrelated
72
+ purpose without the speaker's permission.
73
+
74
+ The enrollment code is included so you can make your own profile from a voice
75
  you have the right to use.
76
 
77
  Issues and pull requests asking for help with undisclosed impersonation, voice
SHA256SUMS CHANGED
@@ -1,5 +1,6 @@
1
- 26075de97f0dd09e56abf49cfede42828aafc90d025eae303102bb643a5d3d1c loudr-1.safetensors
2
- 0688ea295583be6497ec2883a7e55e3fa3463ae4cc3e4aa33fccce3249bf8e25 manifest.json
 
3
  69632f47220a788a52ce2661d096453c5655e9bf25289d89a8d832c46ee07dbf tokenizer.json
4
  f0921cab452fa278bc25cd23ffd59d36f816d7dc5181dd1bef9751a7fb61f63c ve.safetensors
5
  78988a6769f90c209240f9f285eb9cdd760d1ba471cfa7d6e67db632599cfa1b voices/darkman.safetensors
@@ -49,8 +50,10 @@ e7f2f756511fe047d0cd1997cdf2d42ba5c0c9fc8273c4450d25587e9e7aa77e coreml/camp.ml
49
  759103649d9bf8d388b523b915e1eb9b285fb81e58fdef6d420d4c7e21053765 coreml/voice_encoder.mlpackage/Data/com.apple.CoreML/model.mlmodel
50
  95b1a8b8c6b558fafee4a1c784076c8e099070c45958526c9a187c15de55d2d3 coreml/voice_encoder.mlpackage/Data/com.apple.CoreML/weights/weight.bin
51
  fbd810d98add4c3349f01891e1b7c6dc975070bfa83e78f8f8a115f5dcea3c55 coreml/voice_encoder.mlpackage/Manifest.json
52
- c399332def766b45b9c121515a72121aae648c214398df6effd11a376fb12fde README.md
53
  5d5671b95e9797856175dd08d5d2f20d21d72e8785aba077ee38eb378d432528 LICENSE
54
  fda3657fa4d457f9dc379c1ca8550d665eb972ac1c7a260fbe94821ef418ff28 NOTICE
55
- c1fe707141d6fda3f4daf53ca259eaef79067564847738843e91aa3bf9b90d18 RESPONSIBLE_USE.md
56
- 6e0959d98112be647fc2f9c992de1e9977cf06ba3cfa623a6484095dfa5fe5d4 release.json
 
 
 
1
+ 688e87cddfc95a37b6705bda05ad95a7ddb219a4c0c232fa4b93ccadc42efe2c loudr-1.safetensors
2
+ dc825dcdec843cc0fb40ad3a0f051e869db97689a33712d1d11c009ce16af022 loudr-1-enrollment.safetensors
3
+ 795d019797197a0f01c722af516ac6521cda59c6a27d8d07828489175659a99f manifest.json
4
  69632f47220a788a52ce2661d096453c5655e9bf25289d89a8d832c46ee07dbf tokenizer.json
5
  f0921cab452fa278bc25cd23ffd59d36f816d7dc5181dd1bef9751a7fb61f63c ve.safetensors
6
  78988a6769f90c209240f9f285eb9cdd760d1ba471cfa7d6e67db632599cfa1b voices/darkman.safetensors
 
50
  759103649d9bf8d388b523b915e1eb9b285fb81e58fdef6d420d4c7e21053765 coreml/voice_encoder.mlpackage/Data/com.apple.CoreML/model.mlmodel
51
  95b1a8b8c6b558fafee4a1c784076c8e099070c45958526c9a187c15de55d2d3 coreml/voice_encoder.mlpackage/Data/com.apple.CoreML/weights/weight.bin
52
  fbd810d98add4c3349f01891e1b7c6dc975070bfa83e78f8f8a115f5dcea3c55 coreml/voice_encoder.mlpackage/Manifest.json
53
+ dac906a2310982bb236b258db60f81141224087d54a3a66259332a74cb786c4a README.md
54
  5d5671b95e9797856175dd08d5d2f20d21d72e8785aba077ee38eb378d432528 LICENSE
55
  fda3657fa4d457f9dc379c1ca8550d665eb972ac1c7a260fbe94821ef418ff28 NOTICE
56
+ 87c35454a03b026a5a13093566ace245c339dc6af68c7f794f6ed322a731f753 RESPONSIBLE_USE.md
57
+ 83e0bf0a1f6b03e047de66871536c3ff77291041d0a3f2907b6990e580519fea samples/joe.opus
58
+ 48615517c1d1d2d55dbb7f74f37982918236e33f079947459d56c116cd70b7a3 samples/kathleen.opus
59
+ 4058b467cd4a585bc1f39f1d236793181fef0b2e31dacdfb4cc8e31fb22f0aff release.json
loudr-1-enrollment.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc825dcdec843cc0fb40ad3a0f051e869db97689a33712d1d11c009ce16af022
3
+ size 522832312
loudr-1.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:26075de97f0dd09e56abf49cfede42828aafc90d025eae303102bb643a5d3d1c
3
- size 1269460068
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:688e87cddfc95a37b6705bda05ad95a7ddb219a4c0c232fa4b93ccadc42efe2c
3
+ size 746630956
manifest.json CHANGED
@@ -1,18 +1,27 @@
1
  {
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  "dtype_map": {
3
  "s3gen.flow": "float32",
4
  "s3gen.flow.decoder.estimator": "float16",
5
  "s3gen.mel2wav": "float32",
6
- "s3gen.speaker_encoder": "float32",
7
- "s3gen.tokenizer": "float32",
8
  "t3": "float16"
9
  },
10
  "dtype_rationale": {
11
  "s3gen.flow": "EXP-011: whole-flow fp16 destroys output (mel corr 0.619, +22 dB HF); damage is in the encoder",
12
  "s3gen.flow.decoder.estimator": "EXP-011: fp16 mel corr 0.999999, wave corr 0.9987, no new HF energy",
13
  "s3gen.mel2wav": "EXP-003: fp16 produces audible ~12 kHz tone (NSF cumulative phase accumulator). Non-negotiable fp32.",
14
- "s3gen.speaker_encoder": "enrollment support; never measured under reduced precision -> fp32",
15
- "s3gen.tokenizer": "enrollment support; never measured under reduced precision -> fp32",
16
  "t3": "EXP-010: fp16 teacher-forced median KL 1.29e-06 vs fp32, top-1 99.9%; bf16 rejected (no ANE datapath)"
17
  },
18
  "eos_floor": {
@@ -56,6 +65,27 @@
56
  },
57
  "n_cfm_timesteps": 2,
58
  "name": "loudkit-v0.1",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  "recipe_version": "loudkit-1",
60
  "runtime_requirements": {
61
  "allow_tf32_cudnn": false,
@@ -151,6 +181,15 @@
151
  "stop": 6562
152
  },
153
  "speech_vocab_size": 8194,
 
 
 
 
 
 
 
 
 
154
  "t3_config": "multilingual",
155
  "t3_keep_layers": [
156
  0,
@@ -171,7 +210,9 @@
171
  29
172
  ],
173
  "t3_llama_config_registered_as": "Llama_520M",
174
- "tensor_payload_sha256": "09957204231efbdf99e73220fdb6abfa2cf313f124184e5bcf36bfc4feba17eb",
 
 
175
  "torch_version_at_pack": "2.6.0",
176
  "weight_norm": {
177
  "folded": true,
 
1
  {
2
+ "artifact_role": "synthesis",
3
+ "chunking": {
4
+ "enabled": true,
5
+ "max_tokens": 255,
6
+ "prefix_tokens": 6,
7
+ "split_on": [
8
+ ". ",
9
+ "! ",
10
+ "? ",
11
+ "; ",
12
+ ", "
13
+ ]
14
+ },
15
  "dtype_map": {
16
  "s3gen.flow": "float32",
17
  "s3gen.flow.decoder.estimator": "float16",
18
  "s3gen.mel2wav": "float32",
 
 
19
  "t3": "float16"
20
  },
21
  "dtype_rationale": {
22
  "s3gen.flow": "EXP-011: whole-flow fp16 destroys output (mel corr 0.619, +22 dB HF); damage is in the encoder",
23
  "s3gen.flow.decoder.estimator": "EXP-011: fp16 mel corr 0.999999, wave corr 0.9987, no new HF energy",
24
  "s3gen.mel2wav": "EXP-003: fp16 produces audible ~12 kHz tone (NSF cumulative phase accumulator). Non-negotiable fp32.",
 
 
25
  "t3": "EXP-010: fp16 teacher-forced median KL 1.29e-06 vs fp32, top-1 99.9%; bf16 rejected (no ANE datapath)"
26
  },
27
  "eos_floor": {
 
65
  },
66
  "n_cfm_timesteps": 2,
67
  "name": "loudkit-v0.1",
68
+ "postprocess": {
69
+ "ceiling_slack_tokens": 40,
70
+ "ceiling_speech_per_text_token": 4.0,
71
+ "desperation_min_text_tokens": 10,
72
+ "desperation_speech_per_text_token": 4.5,
73
+ "echo_strong_eos_probability": 0.1,
74
+ "echo_strong_max_tail": 30,
75
+ "echo_strong_min_position_pct": 68,
76
+ "echo_weak_eos_probability": 0.003,
77
+ "echo_weak_max_tail": 16,
78
+ "echo_weak_min_position_pct": 85,
79
+ "ended_tail_blip_max": 2,
80
+ "ended_tail_keep": 5,
81
+ "ended_tail_silence_run": 6,
82
+ "ended_tail_word_max": 10,
83
+ "filler_max_speech_after_run": 10,
84
+ "filler_min_eos_probability": 0.05,
85
+ "mode": "trim",
86
+ "trailing_filler_threshold": 0.7,
87
+ "trailing_silence_run_tokens": 12
88
+ },
89
  "recipe_version": "loudkit-1",
90
  "runtime_requirements": {
91
  "allow_tf32_cudnn": false,
 
181
  "stop": 6562
182
  },
183
  "speech_vocab_size": 8194,
184
+ "split": {
185
+ "roles": {
186
+ "enrollment": "loudr-1-enrollment.safetensors",
187
+ "synthesis": "loudr-1.safetensors"
188
+ },
189
+ "source_payload_sha256": "09957204231efbdf99e73220fdb6abfa2cf313f124184e5bcf36bfc4feba17eb",
190
+ "source_tensor_count": 2574,
191
+ "source_tensor_names_sha256": "d0f4888a6b123444a3ff5084bb1b6da8574af04c25af80cd06342ade33c05118"
192
+ },
193
  "t3_config": "multilingual",
194
  "t3_keep_layers": [
195
  0,
 
210
  29
211
  ],
212
  "t3_llama_config_registered_as": "Llama_520M",
213
+ "tensor_count": 1533,
214
+ "tensor_payload_sha256": "9972afd402cd7349fb5320f30855fab1d2d01b201cfb46ae20868193bf211ec5",
215
+ "tokenizer_sha256": "69632f47220a788a52ce2661d096453c5655e9bf25289d89a8d832c46ee07dbf",
216
  "torch_version_at_pack": "2.6.0",
217
  "weight_norm": {
218
  "folded": true,
release.json CHANGED
@@ -2,13 +2,18 @@
2
  "profile": "full-0.1",
3
  "checkpoint": {
4
  "path": "loudr-1.safetensors",
5
- "sha256": "26075de97f0dd09e56abf49cfede42828aafc90d025eae303102bb643a5d3d1c",
6
- "bytes": 1269460068
 
 
 
 
 
7
  },
8
  "manifest": {
9
  "path": "manifest.json",
10
- "sha256": "0688ea295583be6497ec2883a7e55e3fa3463ae4cc3e4aa33fccce3249bf8e25",
11
- "bytes": 4599
12
  },
13
  "tokenizer": {
14
  "path": "tokenizer.json",
@@ -263,8 +268,8 @@
263
  ],
264
  "readme": {
265
  "path": "README.md",
266
- "sha256": "c399332def766b45b9c121515a72121aae648c214398df6effd11a376fb12fde",
267
- "bytes": 11908
268
  },
269
  "license": {
270
  "path": "LICENSE",
@@ -278,8 +283,20 @@
278
  },
279
  "responsible_use": {
280
  "path": "RESPONSIBLE_USE.md",
281
- "sha256": "c1fe707141d6fda3f4daf53ca259eaef79067564847738843e91aa3bf9b90d18",
282
- "bytes": 3699
283
  },
 
 
 
 
 
 
 
 
 
 
 
 
284
  "verified": true
285
  }
 
2
  "profile": "full-0.1",
3
  "checkpoint": {
4
  "path": "loudr-1.safetensors",
5
+ "sha256": "688e87cddfc95a37b6705bda05ad95a7ddb219a4c0c232fa4b93ccadc42efe2c",
6
+ "bytes": 746630956
7
+ },
8
+ "enrollment_checkpoint": {
9
+ "path": "loudr-1-enrollment.safetensors",
10
+ "sha256": "dc825dcdec843cc0fb40ad3a0f051e869db97689a33712d1d11c009ce16af022",
11
+ "bytes": 522832312
12
  },
13
  "manifest": {
14
  "path": "manifest.json",
15
+ "sha256": "795d019797197a0f01c722af516ac6521cda59c6a27d8d07828489175659a99f",
16
+ "bytes": 5635
17
  },
18
  "tokenizer": {
19
  "path": "tokenizer.json",
 
268
  ],
269
  "readme": {
270
  "path": "README.md",
271
+ "sha256": "dac906a2310982bb236b258db60f81141224087d54a3a66259332a74cb786c4a",
272
+ "bytes": 8274
273
  },
274
  "license": {
275
  "path": "LICENSE",
 
283
  },
284
  "responsible_use": {
285
  "path": "RESPONSIBLE_USE.md",
286
+ "sha256": "87c35454a03b026a5a13093566ace245c339dc6af68c7f794f6ed322a731f753",
287
+ "bytes": 4022
288
  },
289
+ "samples": [
290
+ {
291
+ "path": "samples/joe.opus",
292
+ "sha256": "83e0bf0a1f6b03e047de66871536c3ff77291041d0a3f2907b6990e580519fea",
293
+ "bytes": 52187
294
+ },
295
+ {
296
+ "path": "samples/kathleen.opus",
297
+ "sha256": "48615517c1d1d2d55dbb7f74f37982918236e33f079947459d56c116cd70b7a3",
298
+ "bytes": 55798
299
+ }
300
+ ],
301
  "verified": true
302
  }
samples/joe.opus ADDED
Binary file (52.2 kB). View file
 
samples/kathleen.opus ADDED
Binary file (55.8 kB). View file