Spaces:
Paused
Paused
fix README demo img
Browse files
README.md
CHANGED
|
@@ -43,25 +43,21 @@ Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gra
|
|
| 43 |
|
| 44 |
## How It Works
|
| 45 |
|
| 46 |
-
|
| 47 |
-
|
|
| 48 |
-
|
|
| 49 |
-
| **
|
| 50 |
-
| **
|
| 51 |
-
| **
|
| 52 |
-
| **
|
| 53 |
-
| **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
|
| 54 |
-
|
| 55 |
|
| 56 |
## Models
|
| 57 |
|
| 58 |
-
|
| 59 |
-
|
|
| 60 |
-
| -------
|
| 61 |
-
| [Qwen3-TTS-
|
| 62 |
-
| [
|
| 63 |
-
| [LiquidAI/LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) | End-to-end story Q&A with audio input and spoken audio output | 1.5B params |
|
| 64 |
-
|
| 65 |
|
| 66 |
The stock narration model and LFM Q&A model are preloaded at startup for fast response times. The 1.7B cloning model is loaded when cloning or cloned narration is needed.
|
| 67 |
|
|
@@ -76,32 +72,27 @@ The stock narration model and LFM Q&A model are preloaded at startup for fast re
|
|
| 76 |
|
| 77 |
## System Requirements
|
| 78 |
|
| 79 |
-
|
| 80 |
-
|
|
| 81 |
-
|
|
| 82 |
-
| Nvidia
|
| 83 |
-
|
|
| 84 |
-
| CPU only | β | β Not viable | LFM2.5-Audio and Qwen3-TTS inference is too slow for real-time use |
|
| 85 |
-
|
| 86 |
|
| 87 |
**Hugging Face Spaces:**
|
| 88 |
-
|
| 89 |
- **A10G Small** β confirmed working, recommended for the hackathon demo
|
| 90 |
- **T4 Small** β runs with limitations; recommend disabling concurrent model loading
|
| 91 |
|
| 92 |
-
HuggingFace Space Hardware Options
|
| 93 |
|
| 94 |
**Memory breakdown (approximate):**
|
| 95 |
|
| 96 |
-
|
| 97 |
-
|
|
| 98 |
-
| --
|
| 99 |
-
|
|
| 100 |
-
| Qwen3-TTS-
|
| 101 |
-
|
|
| 102 |
-
|
|
| 103 |
-
| **Total (all models loaded)** | **~8β9 GB** |
|
| 104 |
-
|
| 105 |
|
| 106 |
The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5β6 GB with only the narration and Q&A models preloaded.
|
| 107 |
|
|
@@ -152,20 +143,18 @@ python app.py
|
|
| 152 |
|
| 153 |
## Stories Included
|
| 154 |
|
| 155 |
-
|
| 156 |
-
|
|
| 157 |
-
|
|
| 158 |
-
| The Tale of
|
| 159 |
-
| The Tale of
|
| 160 |
-
| The Tale of
|
| 161 |
-
| The
|
| 162 |
-
| The
|
| 163 |
-
| The
|
| 164 |
-
| The Little
|
| 165 |
-
| The
|
| 166 |
-
| The
|
| 167 |
-
| The Adventures of Puss in Boots | 503 | Traditional (verse) |
|
| 168 |
-
|
| 169 |
|
| 170 |
All stories are public domain from [Project Gutenberg](https://www.gutenberg.org/).
|
| 171 |
|
|
@@ -188,6 +177,7 @@ All inference runs on the Hugging Face Space GPU. Voice samples, story text, and
|
|
| 188 |
Hackathon project. Stories are public domain.
|
| 189 |
test push verification
|
| 190 |
|
|
|
|
| 191 |
## Application Demo
|
| 192 |
|
| 193 |
# MomsVoice β Demo Walkthrough
|
|
@@ -198,7 +188,7 @@ Step-by-step guide through the app with screenshots.
|
|
| 198 |
|
| 199 |
## Step 1: Explore the Story Shelf
|
| 200 |
|
| 201 |
-
Explore Space
|
| 202 |
|
| 203 |
Open the **Explore Space** tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click **"Tap to Play"** on any card to open it in My Library, or click **"More Info"** for the full story description.
|
| 204 |
|
|
@@ -208,7 +198,7 @@ The featured story highlights the current pick. All 10 stories are public domain
|
|
| 208 |
|
| 209 |
## Step 2: Open My Library and Start Narration
|
| 210 |
|
| 211 |
-
My Library β Now Playing
|
| 212 |
|
| 213 |
Switch to the **My Library** tab. Click any story card to load it into the **Now Playing** panel on the right. The panel shows:
|
| 214 |
|
|
@@ -223,7 +213,7 @@ Narration starts immediately, playing one paragraph at a time using the TTS mode
|
|
| 223 |
|
| 224 |
## Step 3: Ask a Question About the Story
|
| 225 |
|
| 226 |
-
Voice Q&A Panel
|
| 227 |
|
| 228 |
Scroll down in the Library panel to find the **Ask About the Story** section. Click **β Ask a Question** β narration pauses automatically and the microphone starts recording.
|
| 229 |
|
|
@@ -233,12 +223,11 @@ Speak your question naturally (e.g. *"What did Puss in Boots do to help his mast
|
|
| 233 |
|
| 234 |
## Step 4: Hear the Spoken Answer (LFM2.5-Audio)
|
| 235 |
|
| 236 |
-
LFM Answer Audio
|
| 237 |
|
| 238 |
The recorded question waveform appears in the audio widget. Below it, the **LFM2.5-Audio** model generates a grounded answer in 1β2 sentences, based only on the current story context.
|
| 239 |
|
| 240 |
The answer is delivered in two ways simultaneously:
|
| 241 |
-
|
| 242 |
- **Answer text** shown inline (e.g. *"Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath."*)
|
| 243 |
- **Answer Audio** waveform auto-plays the spoken response
|
| 244 |
|
|
@@ -248,10 +237,9 @@ After the answer finishes, narration resumes from where it paused.
|
|
| 248 |
|
| 249 |
## Step 5: Open Clone Voice Studio and Record Your Voice
|
| 250 |
|
| 251 |
-
Clone Studio β Recording
|
| 252 |
|
| 253 |
Go to the **Clone Voice Studio** tab. Fill in:
|
| 254 |
-
|
| 255 |
- **Voice Nickname** β a name for this profile (e.g. *"Mom"*, *"Grandpa Joe"*)
|
| 256 |
- **Voice Gender Avatar** β Female or Male
|
| 257 |
|
|
@@ -261,7 +249,7 @@ A story script is displayed to read aloud. Click the **Record** button and read
|
|
| 261 |
|
| 262 |
## Step 6: Analyze & Synthesize the Vocal Profile
|
| 263 |
|
| 264 |
-
Clone Studio β Processing
|
| 265 |
|
| 266 |
Click **Analyze & Synthesize Customized Vocal Profile**. The **Status Pipeline Visualizer** on the right tracks progress through extraction, de-noising, and pitch tuning stages.
|
| 267 |
|
|
@@ -271,10 +259,9 @@ A progress bar shows synthesis percentage (e.g. *"Generating voice preview... 70
|
|
| 271 |
|
| 272 |
## Step 7: Voice Profile Ready
|
| 273 |
|
| 274 |
-
Clone Studio β Complete
|
| 275 |
|
| 276 |
When synthesis completes, the Status Pipeline Visualizer shows:
|
| 277 |
-
|
| 278 |
- Your **avatar** and nickname with a **"Cloned successfully"** badge
|
| 279 |
- *"Synthesized today using Qwen3-TTS voice cloning"*
|
| 280 |
- A **preview audio player** β tap Play to hear a short sample in your cloned voice
|
|
@@ -286,8 +273,8 @@ The profile is saved to disk and will persist across Space restarts.
|
|
| 286 |
|
| 287 |
## Step 8: Story Narration in Your Cloned Voice
|
| 288 |
|
| 289 |
-
Narration with Cloned Voice
|
| 290 |
|
| 291 |
Return to **My Library** and select a story. The Now Playing panel shows **Active Narrator: [your nickname]** β confirming the cloned voice is active.
|
| 292 |
|
| 293 |
-
Narration plays in your recorded voice, paragraph by paragraph (*"Paragraph 1 / 32"*). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU β no audio leaves the server.
|
|
|
|
| 43 |
|
| 44 |
## How It Works
|
| 45 |
|
| 46 |
+
| Step | What Happens | Model |
|
| 47 |
+
|---|---|---|
|
| 48 |
+
| **Clone** | Parent records or uploads 15β30s of audio. Speaker embedding is extracted and cached. | Qwen3-TTS-1.7B Base |
|
| 49 |
+
| **Listen** | Story plays in the cloned voice as interruptible sentence chunks. Falls back to CustomVoice 0.6B stock voice (speaker "vivian") if no clone. | Qwen3-TTS-1.7B / CustomVoice 0.6B |
|
| 50 |
+
| **Ask** | Child taps Ask β narration pauses, microphone auto-starts. When recording stops, the question is sent directly to the Q&A audio model. | LFM2.5-Audio-1.5B |
|
| 51 |
+
| **Answer** | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
|
| 52 |
+
| **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
|
|
|
|
|
|
|
| 53 |
|
| 54 |
## Models
|
| 55 |
|
| 56 |
+
| Model | Role | Size |
|
| 57 |
+
|---|---|---|
|
| 58 |
+
| [Qwen3-TTS-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | Zero-shot voice cloning + TTS | 1.7B params |
|
| 59 |
+
| [Qwen3-TTS-0.6B-CustomVoice](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice) | Fast stock-voice TTS (predefined speakers) | 0.6B params |
|
| 60 |
+
| [LiquidAI/LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) | End-to-end story Q&A with audio input and spoken audio output | 1.5B params |
|
|
|
|
|
|
|
| 61 |
|
| 62 |
The stock narration model and LFM Q&A model are preloaded at startup for fast response times. The 1.7B cloning model is loaded when cloning or cloned narration is needed.
|
| 63 |
|
|
|
|
| 72 |
|
| 73 |
## System Requirements
|
| 74 |
|
| 75 |
+
| GPU | VRAM | Status | Notes |
|
| 76 |
+
|---|---|---|---|
|
| 77 |
+
| Nvidia A10G (24 GB) | 24 GB | β
Recommended | All three models load comfortably; narration + Q&A run without memory pressure |
|
| 78 |
+
| Nvidia T4 (16 GB) | 16 GB | β οΈ Functional | Works but is tight β voice cloning model may OOM if loaded concurrently with LFM; restart may be needed between heavy operations |
|
| 79 |
+
| CPU only | β | β Not viable | LFM2.5-Audio and Qwen3-TTS inference is too slow for real-time use |
|
|
|
|
|
|
|
| 80 |
|
| 81 |
**Hugging Face Spaces:**
|
|
|
|
| 82 |
- **A10G Small** β confirmed working, recommended for the hackathon demo
|
| 83 |
- **T4 Small** β runs with limitations; recommend disabling concurrent model loading
|
| 84 |
|
| 85 |
+

|
| 86 |
|
| 87 |
**Memory breakdown (approximate):**
|
| 88 |
|
| 89 |
+
| Model | VRAM (bfloat16) |
|
| 90 |
+
|---|---|
|
| 91 |
+
| LFM2.5-Audio-1.5B | ~3 GB |
|
| 92 |
+
| Qwen3-TTS-0.6B-CustomVoice | ~1.2 GB |
|
| 93 |
+
| Qwen3-TTS-1.7B-Base (cloning) | ~3.4 GB |
|
| 94 |
+
| KV cache + activations at inference | ~1β2 GB |
|
| 95 |
+
| **Total (all models loaded)** | **~8β9 GB** |
|
|
|
|
|
|
|
| 96 |
|
| 97 |
The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5β6 GB with only the narration and Q&A models preloaded.
|
| 98 |
|
|
|
|
| 143 |
|
| 144 |
## Stories Included
|
| 145 |
|
| 146 |
+
| Story | Words | Source |
|
| 147 |
+
|---|---|---|
|
| 148 |
+
| The Tale of Peter Rabbit | 948 | Beatrix Potter |
|
| 149 |
+
| The Tale of Benjamin Bunny | 1,118 | Beatrix Potter |
|
| 150 |
+
| The Tale of Jemima Puddle-Duck | 1,245 | Beatrix Potter |
|
| 151 |
+
| The Tale of Tom Kitten | 691 | Beatrix Potter |
|
| 152 |
+
| The History of Tom Thumb | 2,912 | Traditional |
|
| 153 |
+
| The Story of the Three Little Pigs | 956 | Traditional |
|
| 154 |
+
| The Little Red Hen | 1,295 | Traditional |
|
| 155 |
+
| The Little Gingerbread Man | 1,823 | Traditional |
|
| 156 |
+
| The Sleeping Beauty | 1,783 | Traditional |
|
| 157 |
+
| The Adventures of Puss in Boots | 503 | Traditional (verse) |
|
|
|
|
|
|
|
| 158 |
|
| 159 |
All stories are public domain from [Project Gutenberg](https://www.gutenberg.org/).
|
| 160 |
|
|
|
|
| 177 |
Hackathon project. Stories are public domain.
|
| 178 |
test push verification
|
| 179 |
|
| 180 |
+
|
| 181 |
## Application Demo
|
| 182 |
|
| 183 |
# MomsVoice β Demo Walkthrough
|
|
|
|
| 188 |
|
| 189 |
## Step 1: Explore the Story Shelf
|
| 190 |
|
| 191 |
+

|
| 192 |
|
| 193 |
Open the **Explore Space** tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click **"Tap to Play"** on any card to open it in My Library, or click **"More Info"** for the full story description.
|
| 194 |
|
|
|
|
| 198 |
|
| 199 |
## Step 2: Open My Library and Start Narration
|
| 200 |
|
| 201 |
+

|
| 202 |
|
| 203 |
Switch to the **My Library** tab. Click any story card to load it into the **Now Playing** panel on the right. The panel shows:
|
| 204 |
|
|
|
|
| 213 |
|
| 214 |
## Step 3: Ask a Question About the Story
|
| 215 |
|
| 216 |
+

|
| 217 |
|
| 218 |
Scroll down in the Library panel to find the **Ask About the Story** section. Click **β Ask a Question** β narration pauses automatically and the microphone starts recording.
|
| 219 |
|
|
|
|
| 223 |
|
| 224 |
## Step 4: Hear the Spoken Answer (LFM2.5-Audio)
|
| 225 |
|
| 226 |
+

|
| 227 |
|
| 228 |
The recorded question waveform appears in the audio widget. Below it, the **LFM2.5-Audio** model generates a grounded answer in 1β2 sentences, based only on the current story context.
|
| 229 |
|
| 230 |
The answer is delivered in two ways simultaneously:
|
|
|
|
| 231 |
- **Answer text** shown inline (e.g. *"Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath."*)
|
| 232 |
- **Answer Audio** waveform auto-plays the spoken response
|
| 233 |
|
|
|
|
| 237 |
|
| 238 |
## Step 5: Open Clone Voice Studio and Record Your Voice
|
| 239 |
|
| 240 |
+

|
| 241 |
|
| 242 |
Go to the **Clone Voice Studio** tab. Fill in:
|
|
|
|
| 243 |
- **Voice Nickname** β a name for this profile (e.g. *"Mom"*, *"Grandpa Joe"*)
|
| 244 |
- **Voice Gender Avatar** β Female or Male
|
| 245 |
|
|
|
|
| 249 |
|
| 250 |
## Step 6: Analyze & Synthesize the Vocal Profile
|
| 251 |
|
| 252 |
+

|
| 253 |
|
| 254 |
Click **Analyze & Synthesize Customized Vocal Profile**. The **Status Pipeline Visualizer** on the right tracks progress through extraction, de-noising, and pitch tuning stages.
|
| 255 |
|
|
|
|
| 259 |
|
| 260 |
## Step 7: Voice Profile Ready
|
| 261 |
|
| 262 |
+

|
| 263 |
|
| 264 |
When synthesis completes, the Status Pipeline Visualizer shows:
|
|
|
|
| 265 |
- Your **avatar** and nickname with a **"Cloned successfully"** badge
|
| 266 |
- *"Synthesized today using Qwen3-TTS voice cloning"*
|
| 267 |
- A **preview audio player** β tap Play to hear a short sample in your cloned voice
|
|
|
|
| 273 |
|
| 274 |
## Step 8: Story Narration in Your Cloned Voice
|
| 275 |
|
| 276 |
+

|
| 277 |
|
| 278 |
Return to **My Library** and select a story. The Now Playing panel shows **Active Narrator: [your nickname]** β confirming the cloned voice is active.
|
| 279 |
|
| 280 |
+
Narration plays in your recorded voice, paragraph by paragraph (*"Paragraph 1 / 32"*). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU β no audio leaves the server.
|