jer3mi commited on
Commit
a454062
·
verified ·
1 Parent(s): 998a84c

loudkit demo: listen, speak, clone

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
README.md CHANGED
@@ -1,15 +1,77 @@
1
  ---
2
- title: Loudkit
3
- emoji: 📚
4
- colorFrom: pink
5
- colorTo: green
6
  sdk: gradio
7
- sdk_version: 6.26.0
8
- python_version: '3.12'
9
  app_file: app.py
10
  pinned: false
11
  license: apache-2.0
12
- short_description: showcase of the loudkit TTS library
 
 
 
 
13
  ---
14
 
15
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: loudkit
3
+ emoji: 🔊
4
+ colorFrom: gray
5
+ colorTo: red
6
  sdk: gradio
7
+ sdk_version: 5.50.0
8
+ python_version: "3.12.12"
9
  app_file: app.py
10
  pinned: false
11
  license: apache-2.0
12
+ short_description: On-device TTS. Twenty voices, ten languages, one engine.
13
+ models:
14
+ - loudreader/loudr-1
15
+ preload_from_hub:
16
+ - loudreader/loudr-1 loudr-1.safetensors,loudr-1-enrollment.safetensors,ve.safetensors,manifest.json,release.json,SHA256SUMS,voices/carmen.safetensors,voices/colette.safetensors,voices/dante.safetensors,voices/darkman.safetensors,voices/dave.safetensors,voices/freja.safetensors,voices/gosia.safetensors,voices/henri.safetensors,voices/ines.safetensors,voices/joe.safetensors,voices/kathleen.safetensors,voices/kerstin.safetensors,voices/nathalie.safetensors,voices/nils.safetensors,voices/paola.safetensors,voices/pim.safetensors,voices/selma.safetensors,voices/soren.safetensors,voices/thorsten.safetensors,voices/tugao.safetensors
17
  ---
18
 
19
+ # loudkit
20
+
21
+ Twenty voices across ten languages, from [loudreader/loudr-1](https://huggingface.co/loudreader/loudr-1),
22
+ running the [loudkit](https://github.com/loudreader/loudkit) engine on ZeroGPU.
23
+
24
+ ## Three tabs
25
+
26
+ - **Listen.** Twenty voices, each beside the reference recording it was enrolled
27
+ from. These files were rendered ahead of time and ship in this repo. This tab
28
+ uses no GPU.
29
+ - **Speak.** Your text, in any of the twenty voices. Up to 1,000 characters.
30
+ - **Clone.** Your own voice, from about ten seconds of audio.
31
+
32
+ ZeroGPU bills GPU time to the visitor, not to the owner. An anonymous visitor
33
+ gets about two minutes a day. A signed-in free account gets about five. Listening
34
+ costs none of it.
35
+
36
+ ## Cloning and consent
37
+
38
+ Clone your own voice, or a voice you have permission to use.
39
+
40
+ - The microphone is the default path.
41
+ - An upload is secondary, and needs an explicit confirmation.
42
+ - Recordings are deleted when the request ends. Nothing is kept.
43
+
44
+ See [RESPONSIBLE_USE](https://github.com/loudreader/loudkit/blob/main/RESPONSIBLE_USE.md).
45
+
46
+ ## Determinism
47
+
48
+ The Speak tab has a determinism check. It renders the same text twice at the same
49
+ seed and prints the SHA-256 of both waveforms. They match.
50
+
51
+ That holds within this build and this device. loudkit promises a bit-identical
52
+ waveform for the same seed, build, backend and input. It does not promise that
53
+ your machine matches this GPU. See the
54
+ [identity contract](https://github.com/loudreader/loudkit/blob/main/docs/reference/IDENTITY-CONTRACT.md).
55
+
56
+ ## Run it locally
57
+
58
+ ```bash
59
+ pip install "loudkit[torch,audio,enroll,hub]"
60
+ ```
61
+
62
+ ```python
63
+ import loudkit as lk
64
+
65
+ engine = lk.load("loudreader/loudr-1")
66
+ voice = lk.voice("kathleen", repo="loudreader/loudr-1")
67
+ engine.synthesize_long("Hello from loudkit.", voice, seed=7).save("hello.wav")
68
+ ```
69
+
70
+ Audio in this Space, and from `Result.save`, carries C2PA provenance: the
71
+ algorithm fingerprint, the recipe and the seed.
72
+
73
+ ## Voice sources
74
+
75
+ Every voice is enrolled from a public-domain or openly licensed recording.
76
+ `voices.json` in this repo carries the full record for each one: donor, source,
77
+ licence, consent, and the SHA-256 of both the reference and the sample.
app.py ADDED
@@ -0,0 +1,541 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The loudkit demo Space: hear twenty voices, speak your own text, clone your own.
2
+
3
+ ZeroGPU bills GPU time to the *visitor*, not to the owner: an anonymous visitor
4
+ gets about two minutes a day, a signed-in free account about five. A demo whose
5
+ first click spends that budget is one most people bounce off before they have
6
+ heard anything at all. So the Listen tab is twenty pre-rendered files served
7
+ straight out of this repo — no GPU, no queue, no quota — and the GPU is spent
8
+ only on what a visitor types or records.
9
+
10
+ The engine and the enroller are both built on `cuda` at module level, which is
11
+ what ZeroGPU asks for: CUDA transfers are optimised for start-up placement, and
12
+ lazy-loading inside a `@spaces.GPU` function is explicitly discouraged. Each
13
+ decorated call then runs in a freshly forked, short-lived process, which is also
14
+ why there is no `torch.compile` and no CUDA graph capture here: both pay their
15
+ cost once per process and would never amortise.
16
+
17
+ Cloning is exposed, which the CPU scaffold this replaces deliberately did not do.
18
+ The reasoning that kept it out was about consent, not about capability, so the
19
+ consent is built into the shape of the tab rather than written beside it: the
20
+ microphone is the default path, an upload is secondary and gated on an explicit
21
+ confirmation, and neither recording outlives the request that carried it.
22
+ """
23
+
24
+ from __future__ import annotations
25
+
26
+ import contextlib
27
+ import dataclasses
28
+ import hashlib
29
+ import json
30
+ import os
31
+ import tempfile
32
+ from pathlib import Path
33
+
34
+ # Before torch, and before anything that imports torch. The module installs the
35
+ # CUDA emulation that lets a module-level `.to("cuda")` succeed on a machine
36
+ # that has no GPU attached yet.
37
+ import spaces
38
+
39
+ import gradio as gr
40
+ import numpy as np
41
+
42
+ import loudkit as lk
43
+ from loudkit.backends.torch_backend import build_torch_enroller
44
+ from loudkit.hub import resolve_enrollment_checkpoint, resolve_voice_encoder
45
+
46
+ REPO = "loudreader/loudr-1"
47
+ DEVICE = "cuda"
48
+ HERE = Path(__file__).parent
49
+
50
+ # The CPU scaffold capped text at 300 characters because CPU synthesis ran at
51
+ # roughly a tenth of real time. On a GPU the cap is about the visitor's daily
52
+ # quota instead, which is a far looser bound: 1000 characters is ~70 s of speech.
53
+ MAX_CHARS = 1_000
54
+ MAX_CLONE_CHARS = 400
55
+ MAX_PROBE_CHARS = 200
56
+
57
+ # The enroller refuses anything over 30 s and wants 5 to 10. The prompt is built
58
+ # from the first 10 s; the speaker embedding reads whatever else is there, so a
59
+ # little past the prompt window is useful and 20 s stays clear of the refusal.
60
+ ENROLL_SECONDS = 20.0
61
+
62
+ DOCS = "https://github.com/loudreader/loudkit"
63
+ IDENTITY_CONTRACT = f"{DOCS}/blob/main/docs/reference/IDENTITY-CONTRACT.md"
64
+ RESPONSIBLE_USE = f"{DOCS}/blob/main/RESPONSIBLE_USE.md"
65
+
66
+ ROSTER = json.loads((HERE / "voices.json").read_text(encoding="utf-8"))
67
+ BY_NAME = {entry["name"]: entry for entry in ROSTER}
68
+ ORDERED = sorted(ROSTER, key=lambda e: (e["language"], e["name"]))
69
+ VOICE_CHOICES = [(f"{e['name']} · {e['language']} ({e['gender']})", e["name"]) for e in ORDERED]
70
+
71
+ # --------------------------------------------------------------------------
72
+ # Module-level model placement, per the ZeroGPU contract.
73
+ # --------------------------------------------------------------------------
74
+
75
+ engine = lk.load(REPO, device=DEVICE)
76
+
77
+ # Voice profiles are numpy, not torch, so they are device-agnostic and cost a
78
+ # few hundred kilobytes each. Loading all twenty up front means switching voice
79
+ # in the Speak tab never blocks on a download.
80
+ PROFILES = {entry["name"]: lk.voice(entry["name"], repo=REPO) for entry in ROSTER}
81
+
82
+ # Enrollment reads the other half of the release: the speech tokenizer and the
83
+ # speaker encoder, which synthesis never touches, plus the utterance voice
84
+ # encoder that sits beside both. `lk.enroll()` builds this per call by design;
85
+ # a Space would pay the load on every clone, so it is built once here instead.
86
+ enroller = build_torch_enroller(
87
+ str(resolve_enrollment_checkpoint(REPO)),
88
+ device=DEVICE,
89
+ voice_encoder_weights=str(resolve_voice_encoder(REPO)),
90
+ )
91
+
92
+ FINGERPRINT = engine.algorithm.fingerprint()
93
+
94
+ _LANGUAGE_NAMES = {e["language_id"]: e["language"] for e in ROSTER}
95
+ LANGUAGE_CHOICES = [("Follow the voice", "")] + [
96
+ (f"{_LANGUAGE_NAMES.get(code, code)} ({code})", code) for code in lk.languages()
97
+ ]
98
+
99
+ # --------------------------------------------------------------------------
100
+ # Helpers
101
+ # --------------------------------------------------------------------------
102
+
103
+
104
+ def _sha256_audio(audio: np.ndarray) -> str:
105
+ """Hash the waveform, not the file.
106
+
107
+ `Result.save` appends a C2PA manifest carrying a wall-clock creation time,
108
+ which the library itself calls the one byte range in which two identical
109
+ renders may legitimately differ. Hashing the saved WAV would therefore print
110
+ two different digests for two identical renders and read as a determinism
111
+ failure. The waveform is what the identity contract makes its promise about,
112
+ so the waveform is what gets hashed.
113
+ """
114
+ return hashlib.sha256(np.ascontiguousarray(audio, dtype=np.float32).tobytes()).hexdigest()
115
+
116
+
117
+ def _write(result: lk.Result, *, voice: str, language: str) -> str:
118
+ out = tempfile.NamedTemporaryFile(suffix=".wav", delete=False)
119
+ out.close()
120
+ # Provenance on: the manifest carries the fingerprint, the recipe and the
121
+ # seed, which is the machine-readable marking a synthetic-speech demo should
122
+ # be handing out by default.
123
+ result.save(out.name, voice=voice, language=language)
124
+ return out.name
125
+
126
+
127
+ def _stats(result: lk.Result) -> str:
128
+ seconds = len(result.audio) / result.sample_rate
129
+ return (
130
+ f"**{seconds:.1f} s of audio.** {result.timings.describe(seconds)}\n\n"
131
+ f"`algo[{result.algorithm_fingerprint}]` · seed `{result.seed}` · "
132
+ f"speed `{result.speed:g}x` · {result.sample_rate} Hz"
133
+ )
134
+
135
+
136
+ def _estimate(text: str, *, passes: int = 1, overhead: float = 15.0) -> int:
137
+ """Seconds of GPU to ask for.
138
+
139
+ Speech runs at roughly 14 characters a second, and the render is asked to
140
+ keep up with better than real time; the overhead covers the process fork and
141
+ the first real CUDA touch. Asking for too much costs queue priority but not
142
+ quota, which is charged on effective duration, so this leans generous.
143
+ """
144
+ audio_seconds = len((text or "").strip()) / 14.0
145
+ return int(min(180.0, overhead + passes * max(4.0, audio_seconds * 0.9)))
146
+
147
+
148
+ def _check(text: str, limit: int) -> str:
149
+ text = (text or "").strip()
150
+ if not text:
151
+ raise gr.Error("Type something to say.")
152
+ if len(text) > limit:
153
+ raise gr.Error(f"Keep it under {limit:,} characters here. The library itself takes 10,000.")
154
+ return text
155
+
156
+
157
+ # --------------------------------------------------------------------------
158
+ # Listen. No GPU: these files were rendered ahead of time and ship in the repo.
159
+ # --------------------------------------------------------------------------
160
+
161
+
162
+ def listen(name: str):
163
+ entry = BY_NAME[name]
164
+ sample, reference, source = entry["sample"], entry["reference"], entry["source"]
165
+
166
+ lines = [
167
+ f"### {entry['name']}. {entry['language']} ({entry['gender']}).",
168
+ "",
169
+ f"> {sample['text']}",
170
+ "",
171
+ f"From *{sample['work']}*, seed `{sample['seed']}`.",
172
+ "",
173
+ f"- Reference recording: {reference['duration_s']:.1f} s, {reference['construction']}.",
174
+ f"- Source: [{source['name']}]({source['url']}), {source['license']}.",
175
+ f"- Consent: {source['consent']}.",
176
+ ]
177
+ similarity = entry.get("speaker_similarity")
178
+ if similarity is not None:
179
+ lines.append(f"- Speaker similarity to the reference: {similarity:.3f}.")
180
+ lines.append(f"- Voice profile: `{entry['profile']['hf_path']}`.")
181
+
182
+ return (
183
+ str(HERE / sample["audio"]),
184
+ str(HERE / reference["public_preview"]),
185
+ "\n".join(lines),
186
+ )
187
+
188
+
189
+ ROSTER_TABLE = [
190
+ [
191
+ entry["name"],
192
+ entry["language"],
193
+ entry["gender"],
194
+ entry["source"]["license"],
195
+ f"{entry['speaker_similarity']:.3f}" if entry.get("speaker_similarity") is not None else "",
196
+ ]
197
+ for entry in ORDERED
198
+ ]
199
+
200
+
201
+ # --------------------------------------------------------------------------
202
+ # Speak. GPU.
203
+ # --------------------------------------------------------------------------
204
+
205
+
206
+ def _speak_duration(text, name, language, seed, speed):
207
+ return _estimate(text, overhead=15.0)
208
+
209
+
210
+ @spaces.GPU(duration=_speak_duration)
211
+ def speak(text: str, name: str, language: str, seed: float, speed: float):
212
+ text = _check(text, MAX_CHARS)
213
+ result = engine.synthesize_long(
214
+ text,
215
+ PROFILES[name],
216
+ seed=int(seed),
217
+ language=language or None,
218
+ speed=float(speed),
219
+ )
220
+ label = language or BY_NAME[name]["language_id"]
221
+ return _write(result, voice=name, language=label), _stats(result)
222
+
223
+
224
+ # --------------------------------------------------------------------------
225
+ # Clone. GPU. The microphone is the default path; an upload is gated.
226
+ # --------------------------------------------------------------------------
227
+
228
+
229
+ def _clone_duration(mic, upload, consent, text, language, seed, speed):
230
+ # Enrollment is a fixed cost on top of the render: two encoders and a
231
+ # tokenizer over at most 20 s of audio.
232
+ return _estimate(text, overhead=30.0)
233
+
234
+
235
+ @spaces.GPU(duration=_clone_duration)
236
+ def clone(mic, upload, consent: bool, text: str, language: str, seed: float, speed: float):
237
+ source = mic or upload
238
+ if not source:
239
+ raise gr.Error("Record yourself first, or upload a clip you are allowed to use.")
240
+ if upload and not mic and not consent:
241
+ raise gr.Error("Confirm the uploaded voice is yours, or that you have permission to use it.")
242
+ text = _check(text, MAX_CLONE_CHARS)
243
+
244
+ try:
245
+ import librosa
246
+
247
+ samples, _ = librosa.load(source, sr=24_000, mono=True)
248
+ limit = int(ENROLL_SECONDS * 24_000)
249
+ if samples.size > limit:
250
+ samples = samples[:limit]
251
+
252
+ try:
253
+ profile = enroller.enroll(samples, 24_000, name="your voice")
254
+ except ValueError as exc:
255
+ # The library's own messages name the bound and describe a good
256
+ # input, which is more useful than anything restated here.
257
+ raise gr.Error(str(exc)) from exc
258
+
259
+ # `enroll` writes no language, so every cloned voice would claim English
260
+ # and read its text through the English funnel.
261
+ profile = dataclasses.replace(profile, language=language or "en")
262
+
263
+ result = engine.synthesize_long(
264
+ text, profile, seed=int(seed), language=language or None, speed=float(speed)
265
+ )
266
+ return _write(result, voice="cloned", language=profile.language), _stats(result)
267
+ finally:
268
+ # Nothing the visitor recorded outlives the request that carried it.
269
+ with contextlib.suppress(OSError):
270
+ os.unlink(source)
271
+
272
+
273
+ # --------------------------------------------------------------------------
274
+ # Determinism probe. GPU. Renders the same text twice at the same seed.
275
+ # --------------------------------------------------------------------------
276
+
277
+
278
+ def _probe_duration(text, name, seed):
279
+ return _estimate(text, passes=2, overhead=20.0)
280
+
281
+
282
+ @spaces.GPU(duration=_probe_duration)
283
+ def probe(text: str, name: str, seed: float):
284
+ text = _check(text, MAX_PROBE_CHARS)
285
+ profile = PROFILES[name]
286
+ first = engine.synthesize_long(text, profile, seed=int(seed))
287
+ second = engine.synthesize_long(text, profile, seed=int(seed))
288
+
289
+ left, right = _sha256_audio(first.audio), _sha256_audio(second.audio)
290
+ verdict = "Identical." if left == right else "Different. Please report this."
291
+
292
+ return "\n".join(
293
+ [
294
+ f"**{verdict}**",
295
+ "",
296
+ "```",
297
+ f"render 1 sha256 {left}",
298
+ f"render 2 sha256 {right}",
299
+ f" algo[{first.algorithm_fingerprint}] seed {int(seed)}",
300
+ "```",
301
+ "",
302
+ "Identical within this build and this device. loudkit promises a "
303
+ "bit-identical waveform for the same seed, build, backend and input. "
304
+ "It does not promise that your laptop matches this GPU. "
305
+ f"[Read the identity contract]({IDENTITY_CONTRACT}).",
306
+ ]
307
+ )
308
+
309
+
310
+ # --------------------------------------------------------------------------
311
+ # Interface
312
+ # --------------------------------------------------------------------------
313
+
314
+ # loudreader.io: cream ground, ink text, black pill buttons at 14px.
315
+ CSS = """
316
+ #lk-head h1 { font-size: 2.15rem; margin-bottom: .25rem; letter-spacing: -.02em; }
317
+ #lk-head p { margin-top: 0; }
318
+ .lk-pill {
319
+ display: inline-block; padding: .2rem .75rem; margin: .15rem .35rem .15rem 0;
320
+ border: 1px solid #ded8ce; border-radius: 999px; font-size: .8rem;
321
+ color: #374151; background: #fffdfa;
322
+ }
323
+ .lk-card { background: #fffdfa; border: 1px solid #e7e1d7; border-radius: 14px; padding: .35rem 1rem; }
324
+ footer { display: none !important; }
325
+ """
326
+
327
+ # Gradio follows the visitor's system theme unless told otherwise, and this
328
+ # palette is light-first. Without this the ink-on-cream tokens below land under
329
+ # a dark stylesheet and the text turns near-white on a cream ground.
330
+ FORCE_LIGHT = """
331
+ () => {
332
+ const url = new URL(window.location);
333
+ if (url.searchParams.get('__theme') !== 'light') {
334
+ url.searchParams.set('__theme', 'light');
335
+ window.location.replace(url.href);
336
+ }
337
+ }
338
+ """
339
+
340
+ THEME = gr.themes.Soft(
341
+ primary_hue=gr.themes.colors.gray,
342
+ neutral_hue=gr.themes.colors.stone,
343
+ font=[gr.themes.GoogleFont("Inter"), "system-ui", "sans-serif"],
344
+ ).set(
345
+ body_background_fill="#f7f5f2",
346
+ body_text_color="#111827",
347
+ body_text_color_subdued="#4b5563",
348
+ block_background_fill="#fffdfa",
349
+ block_border_color="#e7e1d7",
350
+ border_color_primary="#e7e1d7",
351
+ input_background_fill="#ffffff",
352
+ button_primary_background_fill="#111827",
353
+ button_primary_background_fill_hover="#374151",
354
+ button_primary_text_color="#ffffff",
355
+ button_large_radius="14px",
356
+ button_small_radius="14px",
357
+ )
358
+
359
+ with gr.Blocks(title="loudkit", theme=THEME, css=CSS, js=FORCE_LIGHT, fill_width=False) as demo:
360
+ gr.Markdown(
361
+ f"""
362
+ # Twenty voices. Ten languages. One engine.
363
+
364
+ On-device text to speech, running here on ZeroGPU.
365
+ [Model](https://huggingface.co/{REPO}) · [Code]({DOCS}) · [Responsible use]({RESPONSIBLE_USE})
366
+
367
+ <span class="lk-pill">Listening costs no GPU</span>
368
+ <span class="lk-pill">Speaking and cloning spend your daily quota</span>
369
+ <span class="lk-pill">algo[{FINGERPRINT}]</span>
370
+ """,
371
+ elem_id="lk-head",
372
+ )
373
+
374
+ with gr.Tabs():
375
+ # ---------------- Listen ----------------
376
+ with gr.Tab("Listen"):
377
+ gr.Markdown(
378
+ "Twenty voices, rendered ahead of time and served as files. "
379
+ "This tab uses no GPU and spends none of your quota. "
380
+ "Each voice is paired with the reference recording it was enrolled from."
381
+ )
382
+ with gr.Row():
383
+ with gr.Column(scale=1):
384
+ pick = gr.Dropdown(
385
+ VOICE_CHOICES, value=ORDERED[0]["name"], label="Voice", filterable=True
386
+ )
387
+ made = gr.Audio(label="loudkit", type="filepath", interactive=False)
388
+ ref = gr.Audio(label="Reference recording", type="filepath", interactive=False)
389
+ with gr.Column(scale=1):
390
+ card = gr.Markdown(elem_classes="lk-card")
391
+
392
+ with gr.Accordion("The whole roster", open=False):
393
+ gr.Dataframe(
394
+ value=ROSTER_TABLE,
395
+ headers=["Voice", "Language", "Gender", "Licence", "Similarity"],
396
+ interactive=False,
397
+ wrap=True,
398
+ )
399
+
400
+ pick.change(listen, pick, [made, ref, card])
401
+ demo.load(listen, pick, [made, ref, card])
402
+
403
+ # ---------------- Speak ----------------
404
+ with gr.Tab("Speak"):
405
+ gr.Markdown(
406
+ f"Your text, in one of the twenty voices. "
407
+ f"Up to {MAX_CHARS:,} characters here. The library itself takes 10,000. "
408
+ "This tab spends your ZeroGPU quota."
409
+ )
410
+ with gr.Row():
411
+ with gr.Column(scale=3):
412
+ say = gr.Textbox(
413
+ label="Text",
414
+ placeholder="Hello from loudkit.",
415
+ lines=4,
416
+ max_length=MAX_CHARS,
417
+ )
418
+ with gr.Column(scale=2):
419
+ say_voice = gr.Dropdown(
420
+ VOICE_CHOICES, value=ORDERED[0]["name"], label="Voice", filterable=True
421
+ )
422
+ say_lang = gr.Dropdown(
423
+ LANGUAGE_CHOICES, value="", label="Read the text as"
424
+ )
425
+ with gr.Row():
426
+ say_seed = gr.Number(value=7, precision=0, label="Seed")
427
+ say_speed = gr.Slider(
428
+ lk.MIN_SPEED, lk.MAX_SPEED, value=1.0, step=0.05, label="Speed"
429
+ )
430
+ say_go = gr.Button("Speak", variant="primary")
431
+ say_out = gr.Audio(label="Speech", type="filepath")
432
+ say_stats = gr.Markdown()
433
+
434
+ say_go.click(
435
+ speak,
436
+ [say, say_voice, say_lang, say_seed, say_speed],
437
+ [say_out, say_stats],
438
+ )
439
+
440
+ with gr.Accordion("Determinism check", open=False):
441
+ gr.Markdown(
442
+ "This renders the same text twice at the same seed and hashes "
443
+ "both waveforms. The digests must match."
444
+ )
445
+ with gr.Row():
446
+ probe_text = gr.Textbox(
447
+ value="The same seed gives the same audio.",
448
+ label="Text",
449
+ max_length=MAX_PROBE_CHARS,
450
+ scale=3,
451
+ )
452
+ probe_voice = gr.Dropdown(
453
+ VOICE_CHOICES, value=ORDERED[0]["name"], label="Voice", scale=2
454
+ )
455
+ probe_seed = gr.Number(value=7, precision=0, label="Seed", scale=1)
456
+ probe_go = gr.Button("Render twice")
457
+ probe_out = gr.Markdown()
458
+ probe_go.click(probe, [probe_text, probe_voice, probe_seed], probe_out)
459
+
460
+ # ---------------- Clone ----------------
461
+ with gr.Tab("Clone"):
462
+ gr.Markdown(
463
+ f"""
464
+ Clone a voice from a short recording, then speak with it.
465
+
466
+ - Record 5 to 10 seconds. Read anything. Speak normally.
467
+ - Clone only your own voice, or a voice you have permission to use.
468
+ - Nothing you record is kept. The recording is deleted when the request ends.
469
+ - See [Responsible use]({RESPONSIBLE_USE}).
470
+ """
471
+ )
472
+ with gr.Row():
473
+ with gr.Column(scale=1):
474
+ mic = gr.Audio(
475
+ sources=["microphone"],
476
+ type="filepath",
477
+ label="Record yourself",
478
+ )
479
+ with gr.Accordion("Upload a file instead", open=False):
480
+ upload = gr.Audio(
481
+ sources=["upload"], type="filepath", label="Audio file"
482
+ )
483
+ consent = gr.Checkbox(
484
+ value=False,
485
+ label=(
486
+ "This is my own voice, or I have permission from the "
487
+ "person who owns it."
488
+ ),
489
+ )
490
+ with gr.Column(scale=1):
491
+ clone_text = gr.Textbox(
492
+ label="Text to speak",
493
+ placeholder="Now in my own voice.",
494
+ lines=3,
495
+ max_length=MAX_CLONE_CHARS,
496
+ )
497
+ clone_lang = gr.Dropdown(
498
+ LANGUAGE_CHOICES[1:], value="en", label="Language of the text"
499
+ )
500
+ with gr.Row():
501
+ clone_seed = gr.Number(value=7, precision=0, label="Seed")
502
+ clone_speed = gr.Slider(
503
+ lk.MIN_SPEED, lk.MAX_SPEED, value=1.0, step=0.05, label="Speed"
504
+ )
505
+ clone_go = gr.Button("Clone and speak", variant="primary")
506
+ clone_out = gr.Audio(label="Speech", type="filepath")
507
+ clone_stats = gr.Markdown()
508
+
509
+ clone_go.click(
510
+ clone,
511
+ [mic, upload, consent, clone_text, clone_lang, clone_seed, clone_speed],
512
+ [clone_out, clone_stats],
513
+ )
514
+
515
+ gr.Markdown(
516
+ f"""
517
+ ---
518
+ Run the same engine locally, where nothing is queued and nothing is metered.
519
+
520
+ ```bash
521
+ pip install "loudkit[torch,audio,enroll,hub]"
522
+ ```
523
+
524
+ ```python
525
+ import loudkit as lk
526
+
527
+ engine = lk.load("{REPO}")
528
+ voice = lk.voice("kathleen", repo="{REPO}")
529
+ engine.synthesize_long("Hello from loudkit.", voice, seed=7).save("hello.wav")
530
+ ```
531
+
532
+ Output files carry C2PA provenance: the fingerprint, the recipe and the seed.
533
+ """
534
+ )
535
+
536
+ # The engine holds one set of weights and renders with an internal producer
537
+ # thread. One render at a time keeps two requests off the same buffers.
538
+ demo.queue(default_concurrency_limit=1, max_size=24)
539
+
540
+ if __name__ == "__main__":
541
+ demo.launch()
audio/carmen.opus ADDED
Binary file (40.8 kB). View file
 
audio/colette.opus ADDED
Binary file (69.7 kB). View file
 
audio/dante.opus ADDED
Binary file (44.7 kB). View file
 
audio/darkman.opus ADDED
Binary file (32.9 kB). View file
 
audio/dave.opus ADDED
Binary file (36.3 kB). View file
 
audio/freja.opus ADDED
Binary file (36.3 kB). View file
 
audio/gosia.opus ADDED
Binary file (31.3 kB). View file
 
audio/henri.opus ADDED
Binary file (62.7 kB). View file
 
audio/ines.opus ADDED
Binary file (57 kB). View file
 
audio/joe.opus ADDED
Binary file (52.2 kB). View file
 
audio/kathleen.opus ADDED
Binary file (55.8 kB). View file
 
audio/kerstin.opus ADDED
Binary file (48.9 kB). View file
 
audio/nathalie.opus ADDED
Binary file (42.3 kB). View file
 
audio/nils.opus ADDED
Binary file (36.7 kB). View file
 
audio/paola.opus ADDED
Binary file (50.2 kB). View file
 
audio/pim.opus ADDED
Binary file (42.2 kB). View file
 
audio/refs/carmen.opus ADDED
Binary file (84.3 kB). View file
 
audio/refs/colette.opus ADDED
Binary file (42.6 kB). View file
 
audio/refs/dante.opus ADDED
Binary file (76.4 kB). View file
 
audio/refs/darkman.opus ADDED
Binary file (57.8 kB). View file
 
audio/refs/dave.opus ADDED
Binary file (67.2 kB). View file
 
audio/refs/freja.opus ADDED
Binary file (52.5 kB). View file
 
audio/refs/gosia.opus ADDED
Binary file (62.2 kB). View file
 
audio/refs/henri.opus ADDED
Binary file (40.1 kB). View file
 
audio/refs/ines.opus ADDED
Binary file (61.2 kB). View file
 
audio/refs/joe.opus ADDED
Binary file (62.2 kB). View file
 
audio/refs/kathleen.opus ADDED
Binary file (66.4 kB). View file
 
audio/refs/kerstin.opus ADDED
Binary file (57 kB). View file
 
audio/refs/nathalie.opus ADDED
Binary file (59.3 kB). View file
 
audio/refs/nils.opus ADDED
Binary file (64.7 kB). View file
 
audio/refs/paola.opus ADDED
Binary file (67 kB). View file
 
audio/refs/pim.opus ADDED
Binary file (63.6 kB). View file
 
audio/refs/selma.opus ADDED
Binary file (50.5 kB). View file
 
audio/refs/soren.opus ADDED
Binary file (43.1 kB). View file
 
audio/refs/thorsten.opus ADDED
Binary file (39.7 kB). View file
 
audio/refs/tugao.opus ADDED
Binary file (54.5 kB). View file
 
audio/selma.opus ADDED
Binary file (32.6 kB). View file
 
audio/soren.opus ADDED
Binary file (42.3 kB). View file
 
audio/thorsten.opus ADDED
Binary file (49.2 kB). View file
 
audio/tugao.opus ADDED
Binary file (55.8 kB). View file
 
hf-loudkit/.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
hf-loudkit/README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: loudkit
3
+ emoji: 🔊
4
+ colorFrom: gray
5
+ colorTo: red
6
+ sdk: gradio
7
+ sdk_version: 5.50.0
8
+ python_version: "3.12.12"
9
+ app_file: app.py
10
+ pinned: false
11
+ license: apache-2.0
12
+ short_description: On-device TTS. Twenty voices, ten languages, one engine.
13
+ models:
14
+ - loudreader/loudr-1
15
+ preload_from_hub:
16
+ - loudreader/loudr-1 loudr-1.safetensors,loudr-1-enrollment.safetensors,ve.safetensors,manifest.json,release.json,SHA256SUMS,voices/carmen.safetensors,voices/colette.safetensors,voices/dante.safetensors,voices/darkman.safetensors,voices/dave.safetensors,voices/freja.safetensors,voices/gosia.safetensors,voices/henri.safetensors,voices/ines.safetensors,voices/joe.safetensors,voices/kathleen.safetensors,voices/kerstin.safetensors,voices/nathalie.safetensors,voices/nils.safetensors,voices/paola.safetensors,voices/pim.safetensors,voices/selma.safetensors,voices/soren.safetensors,voices/thorsten.safetensors,voices/tugao.safetensors
17
+ ---
18
+
19
+ # loudkit
20
+
21
+ Twenty voices across ten languages, from [loudreader/loudr-1](https://huggingface.co/loudreader/loudr-1),
22
+ running the [loudkit](https://github.com/loudreader/loudkit) engine on ZeroGPU.
23
+
24
+ ## Three tabs
25
+
26
+ - **Listen.** Twenty voices, each beside the reference recording it was enrolled
27
+ from. These files were rendered ahead of time and ship in this repo. This tab
28
+ uses no GPU.
29
+ - **Speak.** Your text, in any of the twenty voices. Up to 1,000 characters.
30
+ - **Clone.** Your own voice, from about ten seconds of audio.
31
+
32
+ ZeroGPU bills GPU time to the visitor, not to the owner. An anonymous visitor
33
+ gets about two minutes a day. A signed-in free account gets about five. Listening
34
+ costs none of it.
35
+
36
+ ## Cloning and consent
37
+
38
+ Clone your own voice, or a voice you have permission to use.
39
+
40
+ - The microphone is the default path.
41
+ - An upload is secondary, and needs an explicit confirmation.
42
+ - Recordings are deleted when the request ends. Nothing is kept.
43
+
44
+ See [RESPONSIBLE_USE](https://github.com/loudreader/loudkit/blob/main/RESPONSIBLE_USE.md).
45
+
46
+ ## Determinism
47
+
48
+ The Speak tab has a determinism check. It renders the same text twice at the same
49
+ seed and prints the SHA-256 of both waveforms. They match.
50
+
51
+ That holds within this build and this device. loudkit promises a bit-identical
52
+ waveform for the same seed, build, backend and input. It does not promise that
53
+ your machine matches this GPU. See the
54
+ [identity contract](https://github.com/loudreader/loudkit/blob/main/docs/reference/IDENTITY-CONTRACT.md).
55
+
56
+ ## Run it locally
57
+
58
+ ```bash
59
+ pip install "loudkit[torch,audio,enroll,hub]"
60
+ ```
61
+
62
+ ```python
63
+ import loudkit as lk
64
+
65
+ engine = lk.load("loudreader/loudr-1")
66
+ voice = lk.voice("kathleen", repo="loudreader/loudr-1")
67
+ engine.synthesize_long("Hello from loudkit.", voice, seed=7).save("hello.wav")
68
+ ```
69
+
70
+ Audio in this Space, and from `Result.save`, carries C2PA provenance: the
71
+ algorithm fingerprint, the recipe and the seed.
72
+
73
+ ## Voice sources
74
+
75
+ Every voice is enrolled from a public-domain or openly licensed recording.
76
+ `voices.json` in this repo carries the full record for each one: donor, source,
77
+ licence, consent, and the SHA-256 of both the reference and the sample.
hf-loudkit/app.py ADDED
@@ -0,0 +1,541 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The loudkit demo Space: hear twenty voices, speak your own text, clone your own.
2
+
3
+ ZeroGPU bills GPU time to the *visitor*, not to the owner: an anonymous visitor
4
+ gets about two minutes a day, a signed-in free account about five. A demo whose
5
+ first click spends that budget is one most people bounce off before they have
6
+ heard anything at all. So the Listen tab is twenty pre-rendered files served
7
+ straight out of this repo — no GPU, no queue, no quota — and the GPU is spent
8
+ only on what a visitor types or records.
9
+
10
+ The engine and the enroller are both built on `cuda` at module level, which is
11
+ what ZeroGPU asks for: CUDA transfers are optimised for start-up placement, and
12
+ lazy-loading inside a `@spaces.GPU` function is explicitly discouraged. Each
13
+ decorated call then runs in a freshly forked, short-lived process, which is also
14
+ why there is no `torch.compile` and no CUDA graph capture here: both pay their
15
+ cost once per process and would never amortise.
16
+
17
+ Cloning is exposed, which the CPU scaffold this replaces deliberately did not do.
18
+ The reasoning that kept it out was about consent, not about capability, so the
19
+ consent is built into the shape of the tab rather than written beside it: the
20
+ microphone is the default path, an upload is secondary and gated on an explicit
21
+ confirmation, and neither recording outlives the request that carried it.
22
+ """
23
+
24
+ from __future__ import annotations
25
+
26
+ import contextlib
27
+ import dataclasses
28
+ import hashlib
29
+ import json
30
+ import os
31
+ import tempfile
32
+ from pathlib import Path
33
+
34
+ # Before torch, and before anything that imports torch. The module installs the
35
+ # CUDA emulation that lets a module-level `.to("cuda")` succeed on a machine
36
+ # that has no GPU attached yet.
37
+ import spaces
38
+
39
+ import gradio as gr
40
+ import numpy as np
41
+
42
+ import loudkit as lk
43
+ from loudkit.backends.torch_backend import build_torch_enroller
44
+ from loudkit.hub import resolve_enrollment_checkpoint, resolve_voice_encoder
45
+
46
+ REPO = "loudreader/loudr-1"
47
+ DEVICE = "cuda"
48
+ HERE = Path(__file__).parent
49
+
50
+ # The CPU scaffold capped text at 300 characters because CPU synthesis ran at
51
+ # roughly a tenth of real time. On a GPU the cap is about the visitor's daily
52
+ # quota instead, which is a far looser bound: 1000 characters is ~70 s of speech.
53
+ MAX_CHARS = 1_000
54
+ MAX_CLONE_CHARS = 400
55
+ MAX_PROBE_CHARS = 200
56
+
57
+ # The enroller refuses anything over 30 s and wants 5 to 10. The prompt is built
58
+ # from the first 10 s; the speaker embedding reads whatever else is there, so a
59
+ # little past the prompt window is useful and 20 s stays clear of the refusal.
60
+ ENROLL_SECONDS = 20.0
61
+
62
+ DOCS = "https://github.com/loudreader/loudkit"
63
+ IDENTITY_CONTRACT = f"{DOCS}/blob/main/docs/reference/IDENTITY-CONTRACT.md"
64
+ RESPONSIBLE_USE = f"{DOCS}/blob/main/RESPONSIBLE_USE.md"
65
+
66
+ ROSTER = json.loads((HERE / "voices.json").read_text(encoding="utf-8"))
67
+ BY_NAME = {entry["name"]: entry for entry in ROSTER}
68
+ ORDERED = sorted(ROSTER, key=lambda e: (e["language"], e["name"]))
69
+ VOICE_CHOICES = [(f"{e['name']} · {e['language']} ({e['gender']})", e["name"]) for e in ORDERED]
70
+
71
+ # --------------------------------------------------------------------------
72
+ # Module-level model placement, per the ZeroGPU contract.
73
+ # --------------------------------------------------------------------------
74
+
75
+ engine = lk.load(REPO, device=DEVICE)
76
+
77
+ # Voice profiles are numpy, not torch, so they are device-agnostic and cost a
78
+ # few hundred kilobytes each. Loading all twenty up front means switching voice
79
+ # in the Speak tab never blocks on a download.
80
+ PROFILES = {entry["name"]: lk.voice(entry["name"], repo=REPO) for entry in ROSTER}
81
+
82
+ # Enrollment reads the other half of the release: the speech tokenizer and the
83
+ # speaker encoder, which synthesis never touches, plus the utterance voice
84
+ # encoder that sits beside both. `lk.enroll()` builds this per call by design;
85
+ # a Space would pay the load on every clone, so it is built once here instead.
86
+ enroller = build_torch_enroller(
87
+ str(resolve_enrollment_checkpoint(REPO)),
88
+ device=DEVICE,
89
+ voice_encoder_weights=str(resolve_voice_encoder(REPO)),
90
+ )
91
+
92
+ FINGERPRINT = engine.algorithm.fingerprint()
93
+
94
+ _LANGUAGE_NAMES = {e["language_id"]: e["language"] for e in ROSTER}
95
+ LANGUAGE_CHOICES = [("Follow the voice", "")] + [
96
+ (f"{_LANGUAGE_NAMES.get(code, code)} ({code})", code) for code in lk.languages()
97
+ ]
98
+
99
+ # --------------------------------------------------------------------------
100
+ # Helpers
101
+ # --------------------------------------------------------------------------
102
+
103
+
104
+ def _sha256_audio(audio: np.ndarray) -> str:
105
+ """Hash the waveform, not the file.
106
+
107
+ `Result.save` appends a C2PA manifest carrying a wall-clock creation time,
108
+ which the library itself calls the one byte range in which two identical
109
+ renders may legitimately differ. Hashing the saved WAV would therefore print
110
+ two different digests for two identical renders and read as a determinism
111
+ failure. The waveform is what the identity contract makes its promise about,
112
+ so the waveform is what gets hashed.
113
+ """
114
+ return hashlib.sha256(np.ascontiguousarray(audio, dtype=np.float32).tobytes()).hexdigest()
115
+
116
+
117
+ def _write(result: lk.Result, *, voice: str, language: str) -> str:
118
+ out = tempfile.NamedTemporaryFile(suffix=".wav", delete=False)
119
+ out.close()
120
+ # Provenance on: the manifest carries the fingerprint, the recipe and the
121
+ # seed, which is the machine-readable marking a synthetic-speech demo should
122
+ # be handing out by default.
123
+ result.save(out.name, voice=voice, language=language)
124
+ return out.name
125
+
126
+
127
+ def _stats(result: lk.Result) -> str:
128
+ seconds = len(result.audio) / result.sample_rate
129
+ return (
130
+ f"**{seconds:.1f} s of audio.** {result.timings.describe(seconds)}\n\n"
131
+ f"`algo[{result.algorithm_fingerprint}]` · seed `{result.seed}` · "
132
+ f"speed `{result.speed:g}x` · {result.sample_rate} Hz"
133
+ )
134
+
135
+
136
+ def _estimate(text: str, *, passes: int = 1, overhead: float = 15.0) -> int:
137
+ """Seconds of GPU to ask for.
138
+
139
+ Speech runs at roughly 14 characters a second, and the render is asked to
140
+ keep up with better than real time; the overhead covers the process fork and
141
+ the first real CUDA touch. Asking for too much costs queue priority but not
142
+ quota, which is charged on effective duration, so this leans generous.
143
+ """
144
+ audio_seconds = len((text or "").strip()) / 14.0
145
+ return int(min(180.0, overhead + passes * max(4.0, audio_seconds * 0.9)))
146
+
147
+
148
+ def _check(text: str, limit: int) -> str:
149
+ text = (text or "").strip()
150
+ if not text:
151
+ raise gr.Error("Type something to say.")
152
+ if len(text) > limit:
153
+ raise gr.Error(f"Keep it under {limit:,} characters here. The library itself takes 10,000.")
154
+ return text
155
+
156
+
157
+ # --------------------------------------------------------------------------
158
+ # Listen. No GPU: these files were rendered ahead of time and ship in the repo.
159
+ # --------------------------------------------------------------------------
160
+
161
+
162
+ def listen(name: str):
163
+ entry = BY_NAME[name]
164
+ sample, reference, source = entry["sample"], entry["reference"], entry["source"]
165
+
166
+ lines = [
167
+ f"### {entry['name']}. {entry['language']} ({entry['gender']}).",
168
+ "",
169
+ f"> {sample['text']}",
170
+ "",
171
+ f"From *{sample['work']}*, seed `{sample['seed']}`.",
172
+ "",
173
+ f"- Reference recording: {reference['duration_s']:.1f} s, {reference['construction']}.",
174
+ f"- Source: [{source['name']}]({source['url']}), {source['license']}.",
175
+ f"- Consent: {source['consent']}.",
176
+ ]
177
+ similarity = entry.get("speaker_similarity")
178
+ if similarity is not None:
179
+ lines.append(f"- Speaker similarity to the reference: {similarity:.3f}.")
180
+ lines.append(f"- Voice profile: `{entry['profile']['hf_path']}`.")
181
+
182
+ return (
183
+ str(HERE / sample["audio"]),
184
+ str(HERE / reference["public_preview"]),
185
+ "\n".join(lines),
186
+ )
187
+
188
+
189
+ ROSTER_TABLE = [
190
+ [
191
+ entry["name"],
192
+ entry["language"],
193
+ entry["gender"],
194
+ entry["source"]["license"],
195
+ f"{entry['speaker_similarity']:.3f}" if entry.get("speaker_similarity") is not None else "",
196
+ ]
197
+ for entry in ORDERED
198
+ ]
199
+
200
+
201
+ # --------------------------------------------------------------------------
202
+ # Speak. GPU.
203
+ # --------------------------------------------------------------------------
204
+
205
+
206
+ def _speak_duration(text, name, language, seed, speed):
207
+ return _estimate(text, overhead=15.0)
208
+
209
+
210
+ @spaces.GPU(duration=_speak_duration)
211
+ def speak(text: str, name: str, language: str, seed: float, speed: float):
212
+ text = _check(text, MAX_CHARS)
213
+ result = engine.synthesize_long(
214
+ text,
215
+ PROFILES[name],
216
+ seed=int(seed),
217
+ language=language or None,
218
+ speed=float(speed),
219
+ )
220
+ label = language or BY_NAME[name]["language_id"]
221
+ return _write(result, voice=name, language=label), _stats(result)
222
+
223
+
224
+ # --------------------------------------------------------------------------
225
+ # Clone. GPU. The microphone is the default path; an upload is gated.
226
+ # --------------------------------------------------------------------------
227
+
228
+
229
+ def _clone_duration(mic, upload, consent, text, language, seed, speed):
230
+ # Enrollment is a fixed cost on top of the render: two encoders and a
231
+ # tokenizer over at most 20 s of audio.
232
+ return _estimate(text, overhead=30.0)
233
+
234
+
235
+ @spaces.GPU(duration=_clone_duration)
236
+ def clone(mic, upload, consent: bool, text: str, language: str, seed: float, speed: float):
237
+ source = mic or upload
238
+ if not source:
239
+ raise gr.Error("Record yourself first, or upload a clip you are allowed to use.")
240
+ if upload and not mic and not consent:
241
+ raise gr.Error("Confirm the uploaded voice is yours, or that you have permission to use it.")
242
+ text = _check(text, MAX_CLONE_CHARS)
243
+
244
+ try:
245
+ import librosa
246
+
247
+ samples, _ = librosa.load(source, sr=24_000, mono=True)
248
+ limit = int(ENROLL_SECONDS * 24_000)
249
+ if samples.size > limit:
250
+ samples = samples[:limit]
251
+
252
+ try:
253
+ profile = enroller.enroll(samples, 24_000, name="your voice")
254
+ except ValueError as exc:
255
+ # The library's own messages name the bound and describe a good
256
+ # input, which is more useful than anything restated here.
257
+ raise gr.Error(str(exc)) from exc
258
+
259
+ # `enroll` writes no language, so every cloned voice would claim English
260
+ # and read its text through the English funnel.
261
+ profile = dataclasses.replace(profile, language=language or "en")
262
+
263
+ result = engine.synthesize_long(
264
+ text, profile, seed=int(seed), language=language or None, speed=float(speed)
265
+ )
266
+ return _write(result, voice="cloned", language=profile.language), _stats(result)
267
+ finally:
268
+ # Nothing the visitor recorded outlives the request that carried it.
269
+ with contextlib.suppress(OSError):
270
+ os.unlink(source)
271
+
272
+
273
+ # --------------------------------------------------------------------------
274
+ # Determinism probe. GPU. Renders the same text twice at the same seed.
275
+ # --------------------------------------------------------------------------
276
+
277
+
278
+ def _probe_duration(text, name, seed):
279
+ return _estimate(text, passes=2, overhead=20.0)
280
+
281
+
282
+ @spaces.GPU(duration=_probe_duration)
283
+ def probe(text: str, name: str, seed: float):
284
+ text = _check(text, MAX_PROBE_CHARS)
285
+ profile = PROFILES[name]
286
+ first = engine.synthesize_long(text, profile, seed=int(seed))
287
+ second = engine.synthesize_long(text, profile, seed=int(seed))
288
+
289
+ left, right = _sha256_audio(first.audio), _sha256_audio(second.audio)
290
+ verdict = "Identical." if left == right else "Different. Please report this."
291
+
292
+ return "\n".join(
293
+ [
294
+ f"**{verdict}**",
295
+ "",
296
+ "```",
297
+ f"render 1 sha256 {left}",
298
+ f"render 2 sha256 {right}",
299
+ f" algo[{first.algorithm_fingerprint}] seed {int(seed)}",
300
+ "```",
301
+ "",
302
+ "Identical within this build and this device. loudkit promises a "
303
+ "bit-identical waveform for the same seed, build, backend and input. "
304
+ "It does not promise that your laptop matches this GPU. "
305
+ f"[Read the identity contract]({IDENTITY_CONTRACT}).",
306
+ ]
307
+ )
308
+
309
+
310
+ # --------------------------------------------------------------------------
311
+ # Interface
312
+ # --------------------------------------------------------------------------
313
+
314
+ # loudreader.io: cream ground, ink text, black pill buttons at 14px.
315
+ CSS = """
316
+ #lk-head h1 { font-size: 2.15rem; margin-bottom: .25rem; letter-spacing: -.02em; }
317
+ #lk-head p { margin-top: 0; }
318
+ .lk-pill {
319
+ display: inline-block; padding: .2rem .75rem; margin: .15rem .35rem .15rem 0;
320
+ border: 1px solid #ded8ce; border-radius: 999px; font-size: .8rem;
321
+ color: #374151; background: #fffdfa;
322
+ }
323
+ .lk-card { background: #fffdfa; border: 1px solid #e7e1d7; border-radius: 14px; padding: .35rem 1rem; }
324
+ footer { display: none !important; }
325
+ """
326
+
327
+ # Gradio follows the visitor's system theme unless told otherwise, and this
328
+ # palette is light-first. Without this the ink-on-cream tokens below land under
329
+ # a dark stylesheet and the text turns near-white on a cream ground.
330
+ FORCE_LIGHT = """
331
+ () => {
332
+ const url = new URL(window.location);
333
+ if (url.searchParams.get('__theme') !== 'light') {
334
+ url.searchParams.set('__theme', 'light');
335
+ window.location.replace(url.href);
336
+ }
337
+ }
338
+ """
339
+
340
+ THEME = gr.themes.Soft(
341
+ primary_hue=gr.themes.colors.gray,
342
+ neutral_hue=gr.themes.colors.stone,
343
+ font=[gr.themes.GoogleFont("Inter"), "system-ui", "sans-serif"],
344
+ ).set(
345
+ body_background_fill="#f7f5f2",
346
+ body_text_color="#111827",
347
+ body_text_color_subdued="#4b5563",
348
+ block_background_fill="#fffdfa",
349
+ block_border_color="#e7e1d7",
350
+ border_color_primary="#e7e1d7",
351
+ input_background_fill="#ffffff",
352
+ button_primary_background_fill="#111827",
353
+ button_primary_background_fill_hover="#374151",
354
+ button_primary_text_color="#ffffff",
355
+ button_large_radius="14px",
356
+ button_small_radius="14px",
357
+ )
358
+
359
+ with gr.Blocks(title="loudkit", theme=THEME, css=CSS, js=FORCE_LIGHT, fill_width=False) as demo:
360
+ gr.Markdown(
361
+ f"""
362
+ # Twenty voices. Ten languages. One engine.
363
+
364
+ On-device text to speech, running here on ZeroGPU.
365
+ [Model](https://huggingface.co/{REPO}) · [Code]({DOCS}) · [Responsible use]({RESPONSIBLE_USE})
366
+
367
+ <span class="lk-pill">Listening costs no GPU</span>
368
+ <span class="lk-pill">Speaking and cloning spend your daily quota</span>
369
+ <span class="lk-pill">algo[{FINGERPRINT}]</span>
370
+ """,
371
+ elem_id="lk-head",
372
+ )
373
+
374
+ with gr.Tabs():
375
+ # ---------------- Listen ----------------
376
+ with gr.Tab("Listen"):
377
+ gr.Markdown(
378
+ "Twenty voices, rendered ahead of time and served as files. "
379
+ "This tab uses no GPU and spends none of your quota. "
380
+ "Each voice is paired with the reference recording it was enrolled from."
381
+ )
382
+ with gr.Row():
383
+ with gr.Column(scale=1):
384
+ pick = gr.Dropdown(
385
+ VOICE_CHOICES, value=ORDERED[0]["name"], label="Voice", filterable=True
386
+ )
387
+ made = gr.Audio(label="loudkit", type="filepath", interactive=False)
388
+ ref = gr.Audio(label="Reference recording", type="filepath", interactive=False)
389
+ with gr.Column(scale=1):
390
+ card = gr.Markdown(elem_classes="lk-card")
391
+
392
+ with gr.Accordion("The whole roster", open=False):
393
+ gr.Dataframe(
394
+ value=ROSTER_TABLE,
395
+ headers=["Voice", "Language", "Gender", "Licence", "Similarity"],
396
+ interactive=False,
397
+ wrap=True,
398
+ )
399
+
400
+ pick.change(listen, pick, [made, ref, card])
401
+ demo.load(listen, pick, [made, ref, card])
402
+
403
+ # ---------------- Speak ----------------
404
+ with gr.Tab("Speak"):
405
+ gr.Markdown(
406
+ f"Your text, in one of the twenty voices. "
407
+ f"Up to {MAX_CHARS:,} characters here. The library itself takes 10,000. "
408
+ "This tab spends your ZeroGPU quota."
409
+ )
410
+ with gr.Row():
411
+ with gr.Column(scale=3):
412
+ say = gr.Textbox(
413
+ label="Text",
414
+ placeholder="Hello from loudkit.",
415
+ lines=4,
416
+ max_length=MAX_CHARS,
417
+ )
418
+ with gr.Column(scale=2):
419
+ say_voice = gr.Dropdown(
420
+ VOICE_CHOICES, value=ORDERED[0]["name"], label="Voice", filterable=True
421
+ )
422
+ say_lang = gr.Dropdown(
423
+ LANGUAGE_CHOICES, value="", label="Read the text as"
424
+ )
425
+ with gr.Row():
426
+ say_seed = gr.Number(value=7, precision=0, label="Seed")
427
+ say_speed = gr.Slider(
428
+ lk.MIN_SPEED, lk.MAX_SPEED, value=1.0, step=0.05, label="Speed"
429
+ )
430
+ say_go = gr.Button("Speak", variant="primary")
431
+ say_out = gr.Audio(label="Speech", type="filepath")
432
+ say_stats = gr.Markdown()
433
+
434
+ say_go.click(
435
+ speak,
436
+ [say, say_voice, say_lang, say_seed, say_speed],
437
+ [say_out, say_stats],
438
+ )
439
+
440
+ with gr.Accordion("Determinism check", open=False):
441
+ gr.Markdown(
442
+ "This renders the same text twice at the same seed and hashes "
443
+ "both waveforms. The digests must match."
444
+ )
445
+ with gr.Row():
446
+ probe_text = gr.Textbox(
447
+ value="The same seed gives the same audio.",
448
+ label="Text",
449
+ max_length=MAX_PROBE_CHARS,
450
+ scale=3,
451
+ )
452
+ probe_voice = gr.Dropdown(
453
+ VOICE_CHOICES, value=ORDERED[0]["name"], label="Voice", scale=2
454
+ )
455
+ probe_seed = gr.Number(value=7, precision=0, label="Seed", scale=1)
456
+ probe_go = gr.Button("Render twice")
457
+ probe_out = gr.Markdown()
458
+ probe_go.click(probe, [probe_text, probe_voice, probe_seed], probe_out)
459
+
460
+ # ---------------- Clone ----------------
461
+ with gr.Tab("Clone"):
462
+ gr.Markdown(
463
+ f"""
464
+ Clone a voice from a short recording, then speak with it.
465
+
466
+ - Record 5 to 10 seconds. Read anything. Speak normally.
467
+ - Clone only your own voice, or a voice you have permission to use.
468
+ - Nothing you record is kept. The recording is deleted when the request ends.
469
+ - See [Responsible use]({RESPONSIBLE_USE}).
470
+ """
471
+ )
472
+ with gr.Row():
473
+ with gr.Column(scale=1):
474
+ mic = gr.Audio(
475
+ sources=["microphone"],
476
+ type="filepath",
477
+ label="Record yourself",
478
+ )
479
+ with gr.Accordion("Upload a file instead", open=False):
480
+ upload = gr.Audio(
481
+ sources=["upload"], type="filepath", label="Audio file"
482
+ )
483
+ consent = gr.Checkbox(
484
+ value=False,
485
+ label=(
486
+ "This is my own voice, or I have permission from the "
487
+ "person who owns it."
488
+ ),
489
+ )
490
+ with gr.Column(scale=1):
491
+ clone_text = gr.Textbox(
492
+ label="Text to speak",
493
+ placeholder="Now in my own voice.",
494
+ lines=3,
495
+ max_length=MAX_CLONE_CHARS,
496
+ )
497
+ clone_lang = gr.Dropdown(
498
+ LANGUAGE_CHOICES[1:], value="en", label="Language of the text"
499
+ )
500
+ with gr.Row():
501
+ clone_seed = gr.Number(value=7, precision=0, label="Seed")
502
+ clone_speed = gr.Slider(
503
+ lk.MIN_SPEED, lk.MAX_SPEED, value=1.0, step=0.05, label="Speed"
504
+ )
505
+ clone_go = gr.Button("Clone and speak", variant="primary")
506
+ clone_out = gr.Audio(label="Speech", type="filepath")
507
+ clone_stats = gr.Markdown()
508
+
509
+ clone_go.click(
510
+ clone,
511
+ [mic, upload, consent, clone_text, clone_lang, clone_seed, clone_speed],
512
+ [clone_out, clone_stats],
513
+ )
514
+
515
+ gr.Markdown(
516
+ f"""
517
+ ---
518
+ Run the same engine locally, where nothing is queued and nothing is metered.
519
+
520
+ ```bash
521
+ pip install "loudkit[torch,audio,enroll,hub]"
522
+ ```
523
+
524
+ ```python
525
+ import loudkit as lk
526
+
527
+ engine = lk.load("{REPO}")
528
+ voice = lk.voice("kathleen", repo="{REPO}")
529
+ engine.synthesize_long("Hello from loudkit.", voice, seed=7).save("hello.wav")
530
+ ```
531
+
532
+ Output files carry C2PA provenance: the fingerprint, the recipe and the seed.
533
+ """
534
+ )
535
+
536
+ # The engine holds one set of weights and renders with an internal producer
537
+ # thread. One render at a time keeps two requests off the same buffers.
538
+ demo.queue(default_concurrency_limit=1, max_size=24)
539
+
540
+ if __name__ == "__main__":
541
+ demo.launch()
hf-loudkit/audio/carmen.opus ADDED
Binary file (40.8 kB). View file
 
hf-loudkit/audio/colette.opus ADDED
Binary file (69.7 kB). View file
 
hf-loudkit/audio/dante.opus ADDED
Binary file (44.7 kB). View file
 
hf-loudkit/audio/darkman.opus ADDED
Binary file (32.9 kB). View file
 
hf-loudkit/audio/dave.opus ADDED
Binary file (36.3 kB). View file