dial fix + final card with phonetics grounding
#5
by akshan-main - opened
README.md
CHANGED
|
@@ -9,7 +9,8 @@ python_version: "3.11"
|
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
| 11 |
license: apache-2.0
|
| 12 |
-
short_description:
|
|
|
|
| 13 |
tags:
|
| 14 |
- gradio
|
| 15 |
- build-small-hackathon
|
|
@@ -27,79 +28,47 @@ tags:
|
|
| 27 |
|
| 28 |
# Glossolalia Dial
|
| 29 |
|
| 30 |
-
Type a sentence.
|
| 31 |
-
|
| 32 |
-
At **0** you hear it spoken cleanly. At **4** you hear it as wordless glossolalia: invented words that obey English sound-rules but mean nothing, in the same voice. **The middle of the dial is the point.** At 2 the sentence is half-dissolved, recognizable but slipping, not a clean cut between speech and noise.
|
| 33 |
-
|
| 34 |
-
The dial is a learned scalar conditioner. A small network maps the dial position to a vector added into F5-TTS's time embedding (the same AdaLN pathway the model uses for the diffusion timestep), co-trained with a LoRA. The naive version (appending a `tongues N` token to the prompt) failed: F5-TTS has no language-model front end, so it read the level word aloud and intelligibility moved the wrong way (Spearman -0.70). Making the conditioning a non-text scalar means the model cannot speak it, and the LoRA only has to learn one thing: the per-level audio transformation.
|
| 35 |
-
|
| 36 |
-
**Live:** turn the dial, hit *play this dial*. Or hit *dissolve* to hear the whole 0 to 4 sweep crossfaded into one take.
|
| 37 |
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
No shipped product, open or closed, gives you a *typed-input, graded, voice-locked* slide into glossolalia. Emotion and prosody sliders (Hume, ElevenLabs) move other axes and optimise *for* intelligibility. The closest research (dysarthric-speech clones, discrete lyric-swap edits) solves a different problem. The originality here is the interaction, not the model: a continuous, learned intelligibility axis on one token.
|
| 41 |
-
|
| 42 |
-
It is also not a DSP trick. Reverb, formant-shift, and vocoders act uniformly on audio that already exists. They cannot read a sentence you just invented and erode specific words into different but plausible ones while holding syllable count, stress, and voice. Only a model trained for it can, which is why this needed a fine-tune.
|
| 43 |
|
| 44 |
## Two modes
|
| 45 |
|
| 46 |
-
- **Tongues**
|
| 47 |
-
- **Ghost**
|
| 48 |
-
|
| 49 |
-
## How it was built
|
| 50 |
-
|
| 51 |
-
There is no dataset of "sentences gradually dissolving into nonsense", so we made one. This is the whole reason a single person can build this: instead of hunting for labeled data that does not exist, we manufacture the training target from plain text.
|
| 52 |
|
| 53 |
-
|
| 54 |
-
2. For each, generate five corrupted phoneme variants at substitution rates 0, 0.25, 0.5, 0.75, 1.0. The corruption keeps the English phoneme inventory, preserves syllable count and stress, and leans toward open CV syllables. This is grounded in the phonetics of real glossolalia (Samarin 1972, Goodman 1972; Link & Tomaschek 2024 measured 95.7% CV structure across 7,486 glossolalic syllables).
|
| 55 |
-
3. Have base F5-TTS read each corrupted variant in two reference voices. That gives 30,000 (audio, original-sentence, level, voice) tuples.
|
| 56 |
-
4. Fine-tune a LoRA so that, given the *original* sentence plus a `tongues N` token, it reproduces the level-N audio. The model never sees the corrupted text. It learns the mapping from the dial alone.
|
| 57 |
|
| 58 |
-
|
| 59 |
|
| 60 |
-
##
|
| 61 |
|
| 62 |
-
|
| 63 |
-
- **Whisper-WER rises across levels.** The output is meant to get less intelligible, so word-error-rate should climb monotonically with the dial. Hallucination-guarded, so invented words at the top do not score a spuriously low WER when Whisper invents coherent text from noise.
|
| 64 |
-
- **Voice preserved.** Resemblyzer speaker-embedding cosine between dial-0 and the higher levels should stay close, so the words dissolve but the speaker does not.
|
| 65 |
-
- **The middle has to exist.** Hand-listen at dial 2 for partial dissolution, not a bimodal jump from clean to gibberish.
|
| 66 |
|
| 67 |
-
The
|
| 68 |
|
| 69 |
-
## Models (all
|
| 70 |
|
| 71 |
| Model | Size | Role |
|
| 72 |
|---|---|---|
|
| 73 |
-
| F5-TTS v1 Base | ~336M | flow-matching TTS, zero-shot
|
| 74 |
-
| Glossolalia LoRA | rank-16
|
| 75 |
-
| DistilGPT-2 | ~82M | Ghost-mode word
|
| 76 |
-
| Whisper base.en | ~74M | clone-reference transcription
|
| 77 |
|
| 78 |
-
|
| 79 |
|
| 80 |
## Badges
|
| 81 |
|
| 82 |
-
- 🎯 **Well-Tuned**: fine-tuned LoRA published at [`akshan-main/glossolalia-dial-lora`](https://huggingface.co/akshan-main/glossolalia-dial-lora).
|
| 83 |
-
- 🔌 **Off the Grid**: no cloud APIs
|
| 84 |
-
- 🎨 **Off-Brand**: the dial is a hand-built circular knob
|
| 85 |
-
- 📓 **Field Notes**:
|
| 86 |
-
|
| 87 |
-
## Use it
|
| 88 |
-
|
| 89 |
-
- **Voice:** pick one of nine presets (warm and calm, high and arch, deep and slow, plus theatrical, haunted, and storyteller character voices from public-domain LibriVox), or open *clone your own voice* to upload or record a 6-12s clip.
|
| 90 |
-
- **Background music:** drop in an instrumental and it tempo-locks, tunes the vocal toward the track's key, then mixes it over.
|
| 91 |
-
- **Hand-tune words:** click any word to change its pronunciation or stretch it.
|
| 92 |
-
- **Space (reverb):** dry to cathedral, applied live without re-running the model.
|
| 93 |
|
| 94 |
## Links
|
| 95 |
|
|
|
|
|
|
|
| 96 |
- **Model (LoRA):** https://huggingface.co/akshan-main/glossolalia-dial-lora
|
| 97 |
-
- **Dataset
|
| 98 |
- **Code:** https://github.com/akshan-main/glossolalia
|
| 99 |
-
- **Field Notes:** https://github.com/akshan-main/glossolalia/blob/main/BLOG.md
|
| 100 |
-
- **Demo video:** https://youtu.be/dDOaBNfihyo
|
| 101 |
-
- **Social post:** https://x.com/frutigeraerosol/status/2066667649338417367
|
| 102 |
-
|
| 103 |
-
## Team
|
| 104 |
-
|
| 105 |
-
- [`akshan-main`](https://huggingface.co/akshan-main)
|
|
|
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
| 11 |
license: apache-2.0
|
| 12 |
+
short_description: Convert clear speech to glossolalia or mondegreen
|
| 13 |
+
|
| 14 |
tags:
|
| 15 |
- gradio
|
| 16 |
- build-small-hackathon
|
|
|
|
| 28 |
|
| 29 |
# Glossolalia Dial
|
| 30 |
|
| 31 |
+
Glossolalia Dial is a text-to-speech toy with one knob. Type a sentence and it speaks it. Turn the knob up and the words come apart into wordless babble that still sounds like a language, in the same voice the whole way. The trick is the middle, where the sentence is half-dissolved instead of cutting straight from speech to noise.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
+
**Demo video:** https://youtu.be/dDOaBNfihyo
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
## Two modes
|
| 36 |
|
| 37 |
+
- **Tongues** is the words slurring into made-up pseudo-words. A LoRA and a learned dial trained into F5-TTS. This is the glossolalia.
|
| 38 |
+
- **Ghost** swaps every word for a real one that sounds close (seashells becomes seagulls), the misheard-lyric thing. Pareidolia, not glossolalia, and labeled so. No model, runs live.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
+
## Use it
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
+
Type a sentence, pick a voice (nine presets or clone your own from a short clip), turn the dial 0 to 4, and hit play. Hit dissolve to hear the whole sweep in one take. You can also mix in a backing track, hand-tune any word, or wrap it in reverb.
|
| 43 |
|
| 44 |
+
## How it works
|
| 45 |
|
| 46 |
+
There's no dataset of sentences falling apart into nonsense, so it builds one: corrupt each sentence's phonemes at five rising rates, have base F5-TTS read each, then train the dial to reproduce that slide from the clean sentence alone. The model never sees the corrupted text, it learns the slide from the dial. Ghost mode trains nothing, it searches CMUdict for close-sounding words live and reranks them with DistilGPT-2.
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
+
The corruption is built to sit inside real glossolalia's structure, not just noise: it only swaps within the English phoneme inventory (speakers reuse their native phonotactics, Samarin 1972, Goodman 1972) and keeps syllable count and stress while leaning toward open CV syllables (measured at 95.7% in glossolalic speech, Link & Tomaschek 2024). The exact sound palette is a taste choice, not a claim to match the distribution.
|
| 49 |
|
| 50 |
+
## Models (all local, all under 32B)
|
| 51 |
|
| 52 |
| Model | Size | Role |
|
| 53 |
|---|---|---|
|
| 54 |
+
| F5-TTS v1 Base | ~336M | the voice (flow-matching TTS, zero-shot clone) |
|
| 55 |
+
| Glossolalia LoRA | rank-16 | the dial |
|
| 56 |
+
| DistilGPT-2 | ~82M | Ghost-mode word reranking |
|
| 57 |
+
| Whisper base.en | ~74M | clone-reference transcription |
|
| 58 |
|
| 59 |
+
No cloud APIs. Everything runs on the Space.
|
| 60 |
|
| 61 |
## Badges
|
| 62 |
|
| 63 |
+
- 🎯 **Well-Tuned**: the fine-tuned LoRA, published at [`akshan-main/glossolalia-dial-lora`](https://huggingface.co/akshan-main/glossolalia-dial-lora).
|
| 64 |
+
- 🔌 **Off the Grid**: no cloud APIs, zero cloud SDKs in `requirements.txt`.
|
| 65 |
+
- 🎨 **Off-Brand**: the dial is a hand-built circular knob, not a default slider.
|
| 66 |
+
- 📓 **Field Notes**: the writeup is linked below.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
## Links
|
| 69 |
|
| 70 |
+
- **Demo video:** https://youtu.be/dDOaBNfihyo
|
| 71 |
+
- **Writeup:** https://x.com/frutigeraerosol/status/2066667649338417367
|
| 72 |
- **Model (LoRA):** https://huggingface.co/akshan-main/glossolalia-dial-lora
|
| 73 |
+
- **Dataset:** https://huggingface.co/datasets/akshan-main/glossolalia-inputs
|
| 74 |
- **Code:** https://github.com/akshan-main/glossolalia
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
app.py
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
"""Glossolalia Dial
|
| 2 |
distinct phonotactic paths:
|
| 3 |
|
| 4 |
Ghost mode: lyric is rewritten as a sequence of real English words (mondegreen substitution).
|
|
@@ -6,11 +6,12 @@ distinct phonotactic paths:
|
|
| 6 |
distance; reranked by DistilGPT-2 for semantic coherence. F5-TTS base reads it.
|
| 7 |
Tongues mode: clean lyric goes into F5-TTS + a fine-tuned LoRA + a learned scalar conditioner
|
| 8 |
(LevelEmbed at AdaLN side). The LoRA produces graded glossolalic audio in the
|
| 9 |
-
user's chosen voice
|
| 10 |
|
| 11 |
Both modes ride F5-TTS for voice cloning + audio synthesis. Off-the-Grid: no cloud APIs.
|
| 12 |
|
| 13 |
-
|
|
|
|
| 14 |
"""
|
| 15 |
|
| 16 |
from __future__ import annotations
|
|
@@ -228,7 +229,7 @@ class TTSEngine:
|
|
| 228 |
inference call. For Ghost mode we set_dial(0) (LevelEmbed contributes ~zero) and feed
|
| 229 |
mondegreen-substituted text. For Tongues mode we set_dial(level) and feed the clean
|
| 230 |
lyric. The base LoRA attention adaptation is always on, but at dial=0 it produces audio
|
| 231 |
-
indistinguishable from F5-TTS base (verified empirically by v5 sweep
|
| 232 |
identical to base output)."""
|
| 233 |
|
| 234 |
def __init__(self):
|
|
@@ -267,13 +268,11 @@ class TTSEngine:
|
|
| 267 |
if self._tts is not None:
|
| 268 |
return
|
| 269 |
try:
|
| 270 |
-
import patches # noqa: F401
|
| 271 |
from f5_tts.api import F5TTS
|
| 272 |
# Lazy-loaded inside generate(), which on ZeroGPU runs under @spaces.GPU, so
|
| 273 |
# F5-TTS's auto device detection picks the allocated GPU. (On the T4 Space it
|
| 274 |
# loads to cuda directly; locally it falls back to CPU.)
|
| 275 |
-
# Auto device detection: cuda on the T4 Space, the allocated GPU inside a ZeroGPU
|
| 276 |
-
# worker (generate runs under @spaces.GPU), cpu locally.
|
| 277 |
self._tts = F5TTS(model="F5TTS_v1_Base")
|
| 278 |
self.live = True
|
| 279 |
print(f"[engine] F5-TTS base loaded (device={self._tts.device})")
|
|
@@ -283,7 +282,7 @@ class TTSEngine:
|
|
| 283 |
self._lora_loaded = True
|
| 284 |
print(f"[engine] LoRA loaded from {LORA_PATH}")
|
| 285 |
except Exception as e:
|
| 286 |
-
print(f"[engine] LoRA load FAILED ({e}); falling back to base model
|
| 287 |
else:
|
| 288 |
print("[engine] no LoRA path configured; running base model only")
|
| 289 |
except ImportError:
|
|
@@ -566,7 +565,7 @@ def reapply_fx(dry_path, postfx_preset, music_path, music_gain_db):
|
|
| 566 |
# ----- CSS (dreamy pastel theme: half-remembered photograph of dusk) -----
|
| 567 |
|
| 568 |
CUSTOM_CSS = """
|
| 569 |
-
/* HEAVEN OR LAS VEGAS
|
| 570 |
violet background, hot red-orange sun in the lower-right hemisphere, gold light trails
|
| 571 |
swooping diagonally. Title in hand-drawn flowing italic script (Pinyon Script ≈ the
|
| 572 |
Vaughan Oliver "Heaven or Las Vegas" lettering). Photographic, luminous, analog. */
|
|
@@ -756,7 +755,7 @@ label, .gr-form > label, span[data-testid="block-info"], .label-wrap, label > sp
|
|
| 756 |
align-items: end !important;
|
| 757 |
}
|
| 758 |
|
| 759 |
-
/* lyric textarea
|
| 760 |
textarea {
|
| 761 |
background: transparent !important;
|
| 762 |
border: none !important;
|
|
@@ -776,9 +775,9 @@ textarea {
|
|
| 776 |
textarea::placeholder { color: var(--cream-mute) !important; opacity: 0.45 !important; font-style: italic !important; }
|
| 777 |
textarea:focus { border-bottom-color: var(--gold-bright) !important; outline: none !important; box-shadow: none !important; }
|
| 778 |
|
| 779 |
-
/* dropdowns + number inputs
|
| 780 |
Need to nuke svelte's wrapper containers AND the inner input.
|
| 781 |
-
DO NOT collapse min-height on the slider's .wrap
|
| 782 |
input, select, .gr-input, .gr-dropdown, [role="listbox"],
|
| 783 |
.secondary-wrap, .container,
|
| 784 |
[data-testid="dropdown"], [data-testid="number"] > div {
|
|
@@ -857,7 +856,7 @@ ul[role="listbox"] [role="option"]:hover,
|
|
| 857 |
color: var(--gold-bright) !important;
|
| 858 |
}
|
| 859 |
|
| 860 |
-
/* mode radio
|
| 861 |
The radio group's outer wrap on its own variant of the svelte class */
|
| 862 |
.wrap.svelte-1mwvhlq, [role="radiogroup"] {
|
| 863 |
display: inline-flex !important; gap: 0 !important;
|
|
@@ -900,7 +899,7 @@ ul[role="listbox"] [role="option"]:hover,
|
|
| 900 |
.gradio-container input[role="listbox"]::placeholder { color: var(--cream-mute) !important; opacity: 1 !important; }
|
| 901 |
.gradio-container .dropdown-arrow.svelte-loyhyk { fill: var(--gold-bright) !important; }
|
| 902 |
|
| 903 |
-
/* Mode radio (Ghost / Tongues)
|
| 904 |
.gradio-container label.svelte-19qdtil {
|
| 905 |
background: rgba(14, 8, 32, 0.7) !important;
|
| 906 |
border: 1px solid var(--gold) !important;
|
|
@@ -925,7 +924,7 @@ ul[role="listbox"] [role="option"]:hover,
|
|
| 925 |
.gradio-container label.svelte-19qdtil:has(input:checked) span.svelte-19qdtil {
|
| 926 |
color: var(--night-deep) !important;
|
| 927 |
}
|
| 928 |
-
/* The native radio dot
|
| 929 |
.gradio-container input[type="radio"].svelte-19qdtil { display: none !important; }
|
| 930 |
[role="radio"][aria-checked="true"] label,
|
| 931 |
input[type="radio"]:checked + label {
|
|
@@ -934,7 +933,7 @@ input[type="radio"]:checked + label {
|
|
| 934 |
box-shadow: 0 0 22px var(--sun-halo), inset 0 1px 0 rgba(255, 220, 180, 0.5) !important;
|
| 935 |
}
|
| 936 |
|
| 937 |
-
/* buttons
|
| 938 |
button.primary, button[variant="primary"], .primary > button, button.lg, .gr-button {
|
| 939 |
background: rgba(14, 8, 32, 0.92) !important; /* opaque enough to read over the sun */
|
| 940 |
color: var(--gold-bright) !important;
|
|
@@ -956,7 +955,7 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 956 |
text-shadow: none;
|
| 957 |
box-shadow: 0 0 36px var(--gold-glow), inset 0 0 18px rgba(255, 219, 138, 0.4) !important;
|
| 958 |
}
|
| 959 |
-
/* second action
|
| 960 |
.action-row button:nth-of-type(2) {
|
| 961 |
border-color: var(--sun-mid) !important;
|
| 962 |
color: var(--sun-mid) !important;
|
|
@@ -1011,7 +1010,7 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 1011 |
background: rgba(20, 12, 40, 0.9) !important;
|
| 1012 |
}
|
| 1013 |
|
| 1014 |
-
/* LIVE PREVIEW TEXTBOX (#ghost-lyric): big, gold-on-dark, italic
|
| 1015 |
"see what the voice will say" surface and must be prominent. */
|
| 1016 |
#ghost-lyric, #ghost-lyric > div, #ghost-lyric .wrap, #ghost-lyric > label > div {
|
| 1017 |
background: rgba(14, 8, 32, 0.92) !important;
|
|
@@ -1031,7 +1030,7 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 1031 |
min-height: 80px !important;
|
| 1032 |
}
|
| 1033 |
|
| 1034 |
-
/* THE DIAL
|
| 1035 |
Tick numbers sit in a semicircle above the knob: 0 on the left, 4 on the right. */
|
| 1036 |
#dial-stack {
|
| 1037 |
display: flex; flex-direction: column; align-items: center;
|
|
@@ -1106,7 +1105,7 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 1106 |
touch-action: none; user-select: none;
|
| 1107 |
transition: filter 0.22s;
|
| 1108 |
}
|
| 1109 |
-
/* Knurled rim
|
| 1110 |
.knob::before {
|
| 1111 |
content: ''; position: absolute; inset: 0; border-radius: 50%; pointer-events: none;
|
| 1112 |
background: repeating-conic-gradient(
|
|
@@ -1145,7 +1144,7 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 1145 |
filter: drop-shadow(0 0 8px var(--gold-glow));
|
| 1146 |
opacity: 0.9;
|
| 1147 |
}
|
| 1148 |
-
/* THE INDICATOR LINE
|
| 1149 |
to the edge of the knurled rim. Rotates with the knob value. */
|
| 1150 |
.knob-pointer {
|
| 1151 |
position: absolute; z-index: 4;
|
|
@@ -1165,11 +1164,11 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 1165 |
}
|
| 1166 |
.knob-pin { display: none; }
|
| 1167 |
|
| 1168 |
-
/* Hide the actual gradio slider
|
| 1169 |
#dial-slider { display: none !important; }
|
| 1170 |
#dial-slider .head, #dial-slider .slider_input_container { display: none !important; }
|
| 1171 |
|
| 1172 |
-
/* readout
|
| 1173 |
.readout { display: grid; grid-template-columns: repeat(4, 1fr); gap: 0; margin-top: 24px;
|
| 1174 |
border-top: 1px solid var(--hairline); border-bottom: 1px solid var(--hairline); padding: 16px 0; }
|
| 1175 |
.readout-cell { background: transparent; padding: 4px 14px; text-align: left; border-right: 1px solid var(--hairline); }
|
|
@@ -1177,7 +1176,7 @@ button.primary:hover, button[variant="primary"]:hover, .gr-button:hover {
|
|
| 1177 |
.readout-label { font-family: 'IBM Plex Mono', monospace; font-size: 9px; letter-spacing: 0.32em; color: var(--gold); margin-bottom: 6px; text-transform: uppercase; opacity: 0.7; }
|
| 1178 |
.readout-val { font-family: 'Cormorant Garamond', serif; font-style: italic; font-size: 22px; font-weight: 400; color: var(--cream); letter-spacing: -0.01em; }
|
| 1179 |
|
| 1180 |
-
/* ghost lyric
|
| 1181 |
.ghost-lyric textarea, [data-testid="textbox"]:not(:first-of-type) textarea {
|
| 1182 |
font-family: 'Cormorant Garamond', serif !important;
|
| 1183 |
font-style: italic !important;
|
|
@@ -1343,7 +1342,7 @@ KNOB_HTML = """
|
|
| 1343 |
mask: radial-gradient(circle, transparent 116px, #000 117px, #000 124px, transparent 125px);
|
| 1344 |
filter: drop-shadow(0 0 8px var(--vermillion-glow));
|
| 1345 |
}
|
| 1346 |
-
/* indicator
|
| 1347 |
.knob-indicator {
|
| 1348 |
position: absolute; z-index: 4; left: 50%; top: 22px;
|
| 1349 |
width: 4px; height: 28px; margin-left: -2px;
|
|
@@ -1354,7 +1353,7 @@ KNOB_HTML = """
|
|
| 1354 |
pointer-events: none;
|
| 1355 |
transition: transform 0.16s cubic-bezier(.34,1.36,.4,1);
|
| 1356 |
}
|
| 1357 |
-
/* the center numeral
|
| 1358 |
.knob-label {
|
| 1359 |
position: absolute; z-index: 5; inset: 0;
|
| 1360 |
display: flex; align-items: center; justify-content: center;
|
|
@@ -1363,7 +1362,7 @@ KNOB_HTML = """
|
|
| 1363 |
letter-spacing: -0.04em; pointer-events: none;
|
| 1364 |
text-shadow: 0 1px 0 rgba(255, 240, 220, 0.5), 0 -1px 0 rgba(42, 31, 45, 0.12);
|
| 1365 |
}
|
| 1366 |
-
/* ticks
|
| 1367 |
.knob-ticks {
|
| 1368 |
display: flex; justify-content: space-between; width: 320px; margin: 28px auto 0;
|
| 1369 |
font-family: 'Fraunces', serif; font-style: italic; font-weight: 300;
|
|
@@ -1471,7 +1470,7 @@ STYLE_PRESETS = {
|
|
| 1471 |
}
|
| 1472 |
|
| 1473 |
with gr.Blocks(title="Glossolalia Dial") as demo:
|
| 1474 |
-
# Inject fonts via <link>
|
| 1475 |
# rule has to land in the document head as a tag, not in the CSS string.
|
| 1476 |
gr.HTML(
|
| 1477 |
"""
|
|
@@ -1486,7 +1485,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1486 |
"""
|
| 1487 |
)
|
| 1488 |
|
| 1489 |
-
# lyric
|
| 1490 |
sentence = gr.Textbox(label="the lyric", value=DEFAULT_TEXT, lines=2,
|
| 1491 |
placeholder="anything; the dial will dissolve it",
|
| 1492 |
elem_id="lyric-input")
|
|
@@ -1498,7 +1497,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1498 |
mode = gr.Radio(choices=list(MODES), value=MODE_TONGUES,
|
| 1499 |
label="the path", scale=1)
|
| 1500 |
|
| 1501 |
-
# custom voice
|
| 1502 |
with gr.Accordion("🎤 Clone your own voice (record or upload 6-12 sec)",
|
| 1503 |
open=False, elem_id="custom-voice-accord"):
|
| 1504 |
with gr.Row():
|
|
@@ -1515,7 +1514,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1515 |
scale=1,
|
| 1516 |
)
|
| 1517 |
|
| 1518 |
-
# background music
|
| 1519 |
with gr.Accordion("🎵 Add background music (mix a backing track under the voice)",
|
| 1520 |
open=False, elem_id="music-accord"):
|
| 1521 |
with gr.Row():
|
|
@@ -1531,7 +1530,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1531 |
scale=1,
|
| 1532 |
)
|
| 1533 |
|
| 1534 |
-
# per-word overrides
|
| 1535 |
# Token preview HTML is Python-rendered on every lyric / state change so the markup
|
| 1536 |
# is always in sync. JS only handles the click-to-edit popover and writes the new
|
| 1537 |
# state back into the textbox below; Python then re-renders on that change.
|
|
@@ -1549,7 +1548,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1549 |
# Click-to-edit on .token chips. Gradio v6 strips inline <script> tags inside
|
| 1550 |
# gr.HTML, so we register the delegated listener via demo.load(js=...) below.
|
| 1551 |
|
| 1552 |
-
# the dial
|
| 1553 |
# Tick numbers sit in a semicircle above the knob (0 on the left, 4 on the right).
|
| 1554 |
# The knob's indicator is a pointer needle that lines up with the active tick.
|
| 1555 |
gr.HTML(
|
|
@@ -1578,6 +1577,20 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1578 |
)
|
| 1579 |
level = gr.Slider(0, 4, value=0, step=1, label="", elem_id="dial-slider",
|
| 1580 |
show_label=False, visible=True)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1581 |
|
| 1582 |
# Live preview of what the audio will say, updates as the dial / mode / lyric change.
|
| 1583 |
ghost_lyric = gr.Textbox(label="what the voice will say at this dial position",
|
|
@@ -1605,7 +1618,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1605 |
# Holds the last DRY (pre-effect) voice so changing post-fx / music re-renders
|
| 1606 |
# instantly from it instead of re-running F5-TTS.
|
| 1607 |
dry_cache = gr.State(None)
|
| 1608 |
-
# Hidden
|
| 1609 |
metrics = gr.HTML(readout(), visible=False)
|
| 1610 |
|
| 1611 |
# Bind the brass knob to the hidden slider. Gradio strips inline <script> tags,
|
|
@@ -1736,7 +1749,7 @@ with gr.Blocks(title="Glossolalia Dial") as demo:
|
|
| 1736 |
}""",
|
| 1737 |
)
|
| 1738 |
|
| 1739 |
-
# Click-to-edit on .token chips. Same demo.load(js=...) pattern as the knob
|
| 1740 |
# inline <script> in gr.HTML is stripped by Gradio v6.
|
| 1741 |
demo.load(
|
| 1742 |
fn=None,
|
|
|
|
| 1 |
+
"""Glossolalia Dial, a single dial that grades a typed lyric into dreamy territory in two
|
| 2 |
distinct phonotactic paths:
|
| 3 |
|
| 4 |
Ghost mode: lyric is rewritten as a sequence of real English words (mondegreen substitution).
|
|
|
|
| 6 |
distance; reranked by DistilGPT-2 for semantic coherence. F5-TTS base reads it.
|
| 7 |
Tongues mode: clean lyric goes into F5-TTS + a fine-tuned LoRA + a learned scalar conditioner
|
| 8 |
(LevelEmbed at AdaLN side). The LoRA produces graded glossolalic audio in the
|
| 9 |
+
user's chosen voice, invented pseudowords, sonorant-leaning palette.
|
| 10 |
|
| 11 |
Both modes ride F5-TTS for voice cloning + audio synthesis. Off-the-Grid: no cloud APIs.
|
| 12 |
|
| 13 |
+
Gradio app (gr.Blocks) with a hand-built circular knob injected as custom HTML/CSS/JS
|
| 14 |
+
(the Off-Brand dial), driving F5-TTS + the trained LoRA.
|
| 15 |
"""
|
| 16 |
|
| 17 |
from __future__ import annotations
|
|
|
|
| 229 |
inference call. For Ghost mode we set_dial(0) (LevelEmbed contributes ~zero) and feed
|
| 230 |
mondegreen-substituted text. For Tongues mode we set_dial(level) and feed the clean
|
| 231 |
lyric. The base LoRA attention adaptation is always on, but at dial=0 it produces audio
|
| 232 |
+
indistinguishable from F5-TTS base (verified empirically by v5 sweep, lv0 sounded
|
| 233 |
identical to base output)."""
|
| 234 |
|
| 235 |
def __init__(self):
|
|
|
|
| 268 |
if self._tts is not None:
|
| 269 |
return
|
| 270 |
try:
|
| 271 |
+
import patches # noqa: F401, installs F5TTS.load_lora before instantiation
|
| 272 |
from f5_tts.api import F5TTS
|
| 273 |
# Lazy-loaded inside generate(), which on ZeroGPU runs under @spaces.GPU, so
|
| 274 |
# F5-TTS's auto device detection picks the allocated GPU. (On the T4 Space it
|
| 275 |
# loads to cuda directly; locally it falls back to CPU.)
|
|
|
|
|
|
|
| 276 |
self._tts = F5TTS(model="F5TTS_v1_Base")
|
| 277 |
self.live = True
|
| 278 |
print(f"[engine] F5-TTS base loaded (device={self._tts.device})")
|
|
|
|
| 282 |
self._lora_loaded = True
|
| 283 |
print(f"[engine] LoRA loaded from {LORA_PATH}")
|
| 284 |
except Exception as e:
|
| 285 |
+
print(f"[engine] LoRA load FAILED ({e}); falling back to base model, Well-Tuned badge forfeit")
|
| 286 |
else:
|
| 287 |
print("[engine] no LoRA path configured; running base model only")
|
| 288 |
except ImportError:
|
|
|
|
| 565 |
# ----- CSS (dreamy pastel theme: half-remembered photograph of dusk) -----
|
| 566 |
|
| 567 |
CUSTOM_CSS = """
|
| 568 |
+
/* HEAVEN OR LAS VEGAS, direct reference. Long-exposure Christmas lights, deep midnight
|
| 569 |
violet background, hot red-orange sun in the lower-right hemisphere, gold light trails
|
| 570 |
swooping diagonally. Title in hand-drawn flowing italic script (Pinyon Script ≈ the
|
| 571 |
Vaughan Oliver "Heaven or Las Vegas" lettering). Photographic, luminous, analog. */
|
|
|
|
| 755 |
align-items: end !important;
|
| 756 |
}
|
| 757 |
|
| 758 |
+
/* lyric textarea, handwriting on light, no card, just a hot gold underline */
|
| 759 |
textarea {
|
| 760 |
background: transparent !important;
|
| 761 |
border: none !important;
|
|
|
|
| 775 |
textarea::placeholder { color: var(--cream-mute) !important; opacity: 0.45 !important; font-style: italic !important; }
|
| 776 |
textarea:focus { border-bottom-color: var(--gold-bright) !important; outline: none !important; box-shadow: none !important; }
|
| 777 |
|
| 778 |
+
/* dropdowns + number inputs, minimal, gold underline only.
|
| 779 |
Need to nuke svelte's wrapper containers AND the inner input.
|
| 780 |
+
DO NOT collapse min-height on the slider's .wrap, it would hide the track. */
|
| 781 |
input, select, .gr-input, .gr-dropdown, [role="listbox"],
|
| 782 |
.secondary-wrap, .container,
|
| 783 |
[data-testid="dropdown"], [data-testid="number"] > div {
|
|
|
|
| 856 |
color: var(--gold-bright) !important;
|
| 857 |
}
|
| 858 |
|
| 859 |
+
/* mode radio, luminous gold tabs.
|
| 860 |
The radio group's outer wrap on its own variant of the svelte class */
|
| 861 |
.wrap.svelte-1mwvhlq, [role="radiogroup"] {
|
| 862 |
display: inline-flex !important; gap: 0 !important;
|
|
|
|
| 899 |
.gradio-container input[role="listbox"]::placeholder { color: var(--cream-mute) !important; opacity: 1 !important; }
|
| 900 |
.gradio-container .dropdown-arrow.svelte-loyhyk { fill: var(--gold-bright) !important; }
|
| 901 |
|
| 902 |
+
/* Mode radio (Ghost / Tongues), Gradio v6 svelte-19qdtil */
|
| 903 |
.gradio-container label.svelte-19qdtil {
|
| 904 |
background: rgba(14, 8, 32, 0.7) !important;
|
| 905 |
border: 1px solid var(--gold) !important;
|
|
|
|
| 924 |
.gradio-container label.svelte-19qdtil:has(input:checked) span.svelte-19qdtil {
|
| 925 |
color: var(--night-deep) !important;
|
| 926 |
}
|
| 927 |
+
/* The native radio dot, hide it; we use the chip-style instead */
|
| 928 |
.gradio-container input[type="radio"].svelte-19qdtil { display: none !important; }
|
| 929 |
[role="radio"][aria-checked="true"] label,
|
| 930 |
input[type="radio"]:checked + label {
|
|
|
|
| 933 |
box-shadow: 0 0 22px var(--sun-halo), inset 0 1px 0 rgba(255, 220, 180, 0.5) !important;
|
| 934 |
}
|
| 935 |
|
| 936 |
+
/* buttons, glowing pill on dark, gold borders */
|
| 937 |
button.primary, button[variant="primary"], .primary > button, button.lg, .gr-button {
|
| 938 |
background: rgba(14, 8, 32, 0.92) !important; /* opaque enough to read over the sun */
|
| 939 |
color: var(--gold-bright) !important;
|
|
|
|
| 955 |
text-shadow: none;
|
| 956 |
box-shadow: 0 0 36px var(--gold-glow), inset 0 0 18px rgba(255, 219, 138, 0.4) !important;
|
| 957 |
}
|
| 958 |
+
/* second action, Morph, in the hot vermillion / red-orange sun palette */
|
| 959 |
.action-row button:nth-of-type(2) {
|
| 960 |
border-color: var(--sun-mid) !important;
|
| 961 |
color: var(--sun-mid) !important;
|
|
|
|
| 1010 |
background: rgba(20, 12, 40, 0.9) !important;
|
| 1011 |
}
|
| 1012 |
|
| 1013 |
+
/* LIVE PREVIEW TEXTBOX (#ghost-lyric): big, gold-on-dark, italic, it's the
|
| 1014 |
"see what the voice will say" surface and must be prominent. */
|
| 1015 |
#ghost-lyric, #ghost-lyric > div, #ghost-lyric .wrap, #ghost-lyric > label > div {
|
| 1016 |
background: rgba(14, 8, 32, 0.92) !important;
|
|
|
|
| 1030 |
min-height: 80px !important;
|
| 1031 |
}
|
| 1032 |
|
| 1033 |
+
/* THE DIAL, brass knob with vermillion arc and a pointer needle.
|
| 1034 |
Tick numbers sit in a semicircle above the knob: 0 on the left, 4 on the right. */
|
| 1035 |
#dial-stack {
|
| 1036 |
display: flex; flex-direction: column; align-items: center;
|
|
|
|
| 1105 |
touch-action: none; user-select: none;
|
| 1106 |
transition: filter 0.22s;
|
| 1107 |
}
|
| 1108 |
+
/* Knurled rim, fine repeating ridges in brass */
|
| 1109 |
.knob::before {
|
| 1110 |
content: ''; position: absolute; inset: 0; border-radius: 50%; pointer-events: none;
|
| 1111 |
background: repeating-conic-gradient(
|
|
|
|
| 1144 |
filter: drop-shadow(0 0 8px var(--gold-glow));
|
| 1145 |
opacity: 0.9;
|
| 1146 |
}
|
| 1147 |
+
/* THE INDICATOR LINE, a thin cream line painted on the knob face, from center
|
| 1148 |
to the edge of the knurled rim. Rotates with the knob value. */
|
| 1149 |
.knob-pointer {
|
| 1150 |
position: absolute; z-index: 4;
|
|
|
|
| 1164 |
}
|
| 1165 |
.knob-pin { display: none; }
|
| 1166 |
|
| 1167 |
+
/* Hide the actual gradio slider, the knob drives it via JS */
|
| 1168 |
#dial-slider { display: none !important; }
|
| 1169 |
#dial-slider .head, #dial-slider .slider_input_container { display: none !important; }
|
| 1170 |
|
| 1171 |
+
/* readout, newspaper-strip-like, gold rule */
|
| 1172 |
.readout { display: grid; grid-template-columns: repeat(4, 1fr); gap: 0; margin-top: 24px;
|
| 1173 |
border-top: 1px solid var(--hairline); border-bottom: 1px solid var(--hairline); padding: 16px 0; }
|
| 1174 |
.readout-cell { background: transparent; padding: 4px 14px; text-align: left; border-right: 1px solid var(--hairline); }
|
|
|
|
| 1176 |
.readout-label { font-family: 'IBM Plex Mono', monospace; font-size: 9px; letter-spacing: 0.32em; color: var(--gold); margin-bottom: 6px; text-transform: uppercase; opacity: 0.7; }
|
| 1177 |
.readout-val { font-family: 'Cormorant Garamond', serif; font-style: italic; font-size: 22px; font-weight: 400; color: var(--cream); letter-spacing: -0.01em; }
|
| 1178 |
|
| 1179 |
+
/* ghost lyric, luminous pulled-quote in gold */
|
| 1180 |
.ghost-lyric textarea, [data-testid="textbox"]:not(:first-of-type) textarea {
|
| 1181 |
font-family: 'Cormorant Garamond', serif !important;
|
| 1182 |
font-style: italic !important;
|
|
|
|
| 1342 |
mask: radial-gradient(circle, transparent 116px, #000 117px, #000 124px, transparent 125px);
|
| 1343 |
filter: drop-shadow(0 0 8px var(--vermillion-glow));
|
| 1344 |
}
|
| 1345 |
+
/* indicator, a tiny vermillion bar with subtle glow */
|
| 1346 |
.knob-indicator {
|
| 1347 |
position: absolute; z-index: 4; left: 50%; top: 22px;
|
| 1348 |
width: 4px; height: 28px; margin-left: -2px;
|
|
|
|
| 1353 |
pointer-events: none;
|
| 1354 |
transition: transform 0.16s cubic-bezier(.34,1.36,.4,1);
|
| 1355 |
}
|
| 1356 |
+
/* the center numeral, italic Fraunces in ink */
|
| 1357 |
.knob-label {
|
| 1358 |
position: absolute; z-index: 5; inset: 0;
|
| 1359 |
display: flex; align-items: center; justify-content: center;
|
|
|
|
| 1362 |
letter-spacing: -0.04em; pointer-events: none;
|
| 1363 |
text-shadow: 0 1px 0 rgba(255, 240, 220, 0.5), 0 -1px 0 rgba(42, 31, 45, 0.12);
|
| 1364 |
}
|
| 1365 |
+
/* ticks, italic editorial numerals around the knob */
|
| 1366 |
.knob-ticks {
|
| 1367 |
display: flex; justify-content: space-between; width: 320px; margin: 28px auto 0;
|
| 1368 |
font-family: 'Fraunces', serif; font-style: italic; font-weight: 300;
|
|
|
|
| 1470 |
}
|
| 1471 |
|
| 1472 |
with gr.Blocks(title="Glossolalia Dial") as demo:
|
| 1473 |
+
# Inject fonts via <link>, Gradio's CSS-in-JS strips @import, so the stylesheet
|
| 1474 |
# rule has to land in the document head as a tag, not in the CSS string.
|
| 1475 |
gr.HTML(
|
| 1476 |
"""
|
|
|
|
| 1485 |
"""
|
| 1486 |
)
|
| 1487 |
|
| 1488 |
+
# lyric, magazine pull-quote spanning full width
|
| 1489 |
sentence = gr.Textbox(label="the lyric", value=DEFAULT_TEXT, lines=2,
|
| 1490 |
placeholder="anything; the dial will dissolve it",
|
| 1491 |
elem_id="lyric-input")
|
|
|
|
| 1497 |
mode = gr.Radio(choices=list(MODES), value=MODE_TONGUES,
|
| 1498 |
label="the path", scale=1)
|
| 1499 |
|
| 1500 |
+
# custom voice, record or upload a 10-sec clip; F5-TTS clones it
|
| 1501 |
with gr.Accordion("🎤 Clone your own voice (record or upload 6-12 sec)",
|
| 1502 |
open=False, elem_id="custom-voice-accord"):
|
| 1503 |
with gr.Row():
|
|
|
|
| 1514 |
scale=1,
|
| 1515 |
)
|
| 1516 |
|
| 1517 |
+
# background music, upload an instrumental; we tempo-lock + mix the vocal over it
|
| 1518 |
with gr.Accordion("🎵 Add background music (mix a backing track under the voice)",
|
| 1519 |
open=False, elem_id="music-accord"):
|
| 1520 |
with gr.Row():
|
|
|
|
| 1530 |
scale=1,
|
| 1531 |
)
|
| 1532 |
|
| 1533 |
+
# per-word overrides, stretch + pronunciation per individual word.
|
| 1534 |
# Token preview HTML is Python-rendered on every lyric / state change so the markup
|
| 1535 |
# is always in sync. JS only handles the click-to-edit popover and writes the new
|
| 1536 |
# state back into the textbox below; Python then re-renders on that change.
|
|
|
|
| 1548 |
# Click-to-edit on .token chips. Gradio v6 strips inline <script> tags inside
|
| 1549 |
# gr.HTML, so we register the delegated listener via demo.load(js=...) below.
|
| 1550 |
|
| 1551 |
+
# the dial, brass knob with vermillion arc and a Fraunces numeral center.
|
| 1552 |
# Tick numbers sit in a semicircle above the knob (0 on the left, 4 on the right).
|
| 1553 |
# The knob's indicator is a pointer needle that lines up with the active tick.
|
| 1554 |
gr.HTML(
|
|
|
|
| 1577 |
)
|
| 1578 |
level = gr.Slider(0, 4, value=0, step=1, label="", elem_id="dial-slider",
|
| 1579 |
show_label=False, visible=True)
|
| 1580 |
+
# The custom knob writes the dial value into this slider's <input type="range"> via JS,
|
| 1581 |
+
# where a range input's value is always a STRING. Gradio's slider bounds-check does
|
| 1582 |
+
# `value < minimum`, which raises "TypeError: '<' not supported between str and int" on a
|
| 1583 |
+
# string and kills the request (it surfaced as the dial erroring when turned). Coerce to a
|
| 1584 |
+
# number before the original preprocess so turning the knob can never crash. Instance-level
|
| 1585 |
+
# override (not a subclass) so the component stays a plain Slider and the knob wiring holds.
|
| 1586 |
+
_level_orig_preprocess = level.preprocess
|
| 1587 |
+
def _level_safe_preprocess(payload, *args, **kwargs):
|
| 1588 |
+
try:
|
| 1589 |
+
payload = float(payload)
|
| 1590 |
+
except (TypeError, ValueError):
|
| 1591 |
+
payload = 0.0
|
| 1592 |
+
return _level_orig_preprocess(payload, *args, **kwargs)
|
| 1593 |
+
level.preprocess = _level_safe_preprocess
|
| 1594 |
|
| 1595 |
# Live preview of what the audio will say, updates as the dial / mode / lyric change.
|
| 1596 |
ghost_lyric = gr.Textbox(label="what the voice will say at this dial position",
|
|
|
|
| 1618 |
# Holds the last DRY (pre-effect) voice so changing post-fx / music re-renders
|
| 1619 |
# instantly from it instead of re-running F5-TTS.
|
| 1620 |
dry_cache = gr.State(None)
|
| 1621 |
+
# Hidden, stub metrics block, only shown after we wire real Whisper-WER + Resemblyzer.
|
| 1622 |
metrics = gr.HTML(readout(), visible=False)
|
| 1623 |
|
| 1624 |
# Bind the brass knob to the hidden slider. Gradio strips inline <script> tags,
|
|
|
|
| 1749 |
}""",
|
| 1750 |
)
|
| 1751 |
|
| 1752 |
+
# Click-to-edit on .token chips. Same demo.load(js=...) pattern as the knob -
|
| 1753 |
# inline <script> in gr.HTML is stripped by Gradio v6.
|
| 1754 |
demo.load(
|
| 1755 |
fn=None,
|