Spaces:
Paused
A newer version of the Gradio SDK is available: 6.20.0
title: MomsVoice
emoji: 📖
colorFrom: yellow
colorTo: green
sdk: gradio
sdk_version: 5.35.0
app_file: app.py
pinned: false
short_description: MomsVoiceAI is voice cloning audio book app
hf_oauth: true
preload_from_hub:
- LiquidAI/LFM2.5-Audio-1.5B
- Qwen/Qwen3-TTS-12Hz-1.7B-Base
tags:
- track:backyard
- achievement:offgrid
- achievement:offbrand
- achievement:fieldnotes
MomsVoice
A parent records their voice. A bedtime story plays in that voice. The child can ask questions and hear low-latency spoken answers from a local audio model.
Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gradio app on Hugging Face Spaces — no external APIs, no data leaves the server.
Team members
- Minha Hwang : @DrSquare / Linkedin / Huggingface
- Jimin Gu: @Zoe09 / Linkedin / Huggingface
- Seunghwa Song: @sshtel / Linkedin / Huggingface
build-small-hackathon Link
Live Demo at personal Spaces
- 🔗https://huggingface.co/spaces/DrSquare/MomsVoice
- 🔗https://huggingface.co/spaces/sshtel/Huggingface_Hack
YouTube link
Demo Flow
🎤 Record 15s of your voice → 📖 Pick a story → ▶️ Story plays in your voice
↓
❓ Child taps Ask → Story pauses, mic auto-starts
↓
Child asks a question → Answer auto-plays from the Q&A audio model
↓
↩️ Resume → Story continues from where it paused
How It Works
| Step | What Happens | Model |
|---|---|---|
| Clone | Parent records or uploads 15–30s of audio. Speaker embedding is extracted and cached. | Qwen3-TTS-1.7B Base |
| Listen | Story plays in the cloned voice as interruptible sentence chunks. Falls back to CustomVoice 0.6B stock voice (speaker "vivian") if no clone. | Qwen3-TTS-1.7B / CustomVoice 0.6B |
| Ask | Child taps Ask → narration pauses, microphone auto-starts. When recording stops, the question is sent directly to the Q&A audio model. | LFM2.5-Audio-1.5B |
| Answer | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
| Resume | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
Models
| Model | Role | Size |
|---|---|---|
| Qwen3-TTS-1.7B-Base | Zero-shot voice cloning + TTS | 1.7B params |
| Qwen3-TTS-0.6B-CustomVoice | Fast stock-voice TTS (predefined speakers) | 0.6B params |
| LiquidAI/LFM2.5-Audio-1.5B | End-to-end story Q&A with audio input and spoken audio output | 1.5B params |
The stock narration model and LFM Q&A model are preloaded at startup for fast response times. The 1.7B cloning model is loaded when cloning or cloned narration is needed.
Tech Stack
- UI: Gradio 5.35.0 with custom CSS/JS (Google Stitch-inspired warm design)
- Runtime: Single
app.pyprocess on HF Spaces (GPU: A10G or larger recommended) - Stories: 10 public domain
.txtfiles (downloaded and cleaned from Project Gutenberg) - Storage: In-memory session cache + disk-persisted voice profiles (
Voice_Profile/) - TTS Backends: Qwen3-TTS Base (cloned voice) + CustomVoice 0.6B (stock fallback)
- Q&A Backend: LFM2.5-Audio for direct audio-in/audio-out answers with lower interaction latency
System Requirements
| GPU | VRAM | Status | Notes |
|---|---|---|---|
| Nvidia A10G (24 GB) | 24 GB | ✅ Recommended | All three models load comfortably; narration + Q&A run without memory pressure |
| Nvidia T4 (16 GB) | 16 GB | ⚠️ Functional | Works but is tight — voice cloning model may OOM if loaded concurrently with LFM; restart may be needed between heavy operations |
| CPU only | — | ❌ Not viable | LFM2.5-Audio and Qwen3-TTS inference is too slow for real-time use |
Hugging Face Spaces:
- A10G Small — confirmed working, recommended for the hackathon demo
- T4 Small — runs with limitations; recommend disabling concurrent model loading
Memory breakdown (approximate):
| Model | VRAM (bfloat16) |
|---|---|
| LFM2.5-Audio-1.5B | ~3 GB |
| Qwen3-TTS-0.6B-CustomVoice | ~1.2 GB |
| Qwen3-TTS-1.7B-Base (cloning) | ~3.4 GB |
| KV cache + activations at inference | ~1–2 GB |
| Total (all models loaded) | ~8–9 GB |
The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5–6 GB with only the narration and Q&A models preloaded.
Features
- 🎤 Voice Cloning — Record 15s, get a narrator that sounds like you
- 📚 Explore & Library — Browse and select stories with clickable book cards
- ▶️ Chunked Streaming — Stories play paragraph-by-paragraph with real-time highlighting
- ❓ Voice Q&A — Ask questions about the story; auto-submit on recording stop, autoplay answer
- ⏸ Smart Pause — Playback pauses on Ask, tab switch (Explore/Clone Studio), or Pause button
- 🎙️ Voice Selection — Click voice cards to switch between cloned narrators
- 💾 Persistent Profiles — Cloned voices survive app restarts
Quick Start (Local Dev)
git clone https://github.com/MomsVoiceAI/HuggingFace_Hack.git
cd HuggingFace_Hack
pip install -r requirements.txt
python app.py
# → http://localhost:7860
Project Structure
├── app.py # Main Gradio app (UI + all event wiring)
├── voice_clone.py # Qwen3-TTS voice cloning + profile persistence
├── tts.py # Unified TTS interface (Base cloned + CustomVoice stock)
├── inference_lfm.py # LFM2.5-Audio Q&A (audio/text input → text/audio output)
├── inference.py # Legacy Whisper + Qwen text Q&A path
├── requirements.txt # Python deps for the active LFM + Qwen app
├── requirements_lfm.txt # Legacy copy of the LFM requirements
├── stories/ # 10 cleaned public domain story texts (TTS-ready)
├── story_downloader/ # Story acquisition & cleaning pipeline
│ ├── gutenberg_downloader.py # Reusable Project Gutenberg downloader/parser
│ ├── download_stories.py # Downloads 10 children's stories
│ └── clean_stories.py # Strips Gutenberg boilerplate for TTS
├── static/style.css # Custom CSS for warm storybook UI
├── assets/covers/ # Story cover images
├── Voice_Profile/ # Persisted cloned voice profiles
├── test_modules/ # Component and integration tests
├── mission.md # Product vision
├── sprint.md # 2-day hackathon sprint plan
├── tech_stack.md # Technical architecture
└── future_mobile_app_considerations.md # Mobile deployment guidance
Stories Included
| Story | Words | Source |
|---|---|---|
| The Tale of Peter Rabbit | 948 | Beatrix Potter |
| The Tale of Benjamin Bunny | 1,118 | Beatrix Potter |
| The Tale of Jemima Puddle-Duck | 1,245 | Beatrix Potter |
| The Tale of Tom Kitten | 691 | Beatrix Potter |
| The History of Tom Thumb | 2,912 | Traditional |
| The Story of the Three Little Pigs | 956 | Traditional |
| The Little Red Hen | 1,295 | Traditional |
| The Little Gingerbread Man | 1,823 | Traditional |
| The Sleeping Beauty | 1,783 | Traditional |
| The Adventures of Puss in Boots | 503 | Traditional (verse) |
All stories are public domain from Project Gutenberg.
Key Design Decisions
- All local inference — narration, voice cloning, and Q&A run on the Space GPU. No external APIs.
- Interruptible chunked streaming — sentences synthesized and played one at a time for fast start and clean pause/resume.
- Voice-first Q&A — microphone auto-starts on Ask; LFM generates text and spoken audio in one low-latency path. No manual button clicks needed.
- Grounded Q&A — LFM receives the selected story context so answers stay tied to the current story.
- Smart pause — playback pauses on Ask, tab switch, or Pause button. Resume continues from exact position.
- Mounted runtime storage — generated audio, narration cache, and voice profiles are written under
MOMSVOICE_STORAGE_DIR/SPACE_STORAGE_DIR, or/data/momsvoicewhen a Hugging Face bucket is mounted at/data. - Privacy-first — no audio leaves the server, no user accounts, no external database.
Privacy
All inference runs on the Hugging Face Space GPU. Voice samples, story text, and generated audio stay on the server runtime. Voice profiles are saved to the Space's persistent storage but no data is sent to external services.
License
Hackathon project. Stories are public domain. test push verification
Application Demo
MomsVoice — Demo Walkthrough
Step-by-step guide through the app with screenshots.
Step 1: Explore the Story Shelf
Open the Explore Space tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click "Tap to Play" on any card to open it in My Library, or click "More Info" for the full story description.
The featured story highlights the current pick. All 10 stories are public domain classics sourced from Project Gutenberg.
Step 2: Open My Library and Start Narration
Switch to the My Library tab. Click any story card to load it into the Now Playing panel on the right. The panel shows:
- Cover art and title
- Active narrator label (stock Vivian voice or your cloned voice)
- Pause / Resume Story buttons
- Story text scrolling in sync with narration
Narration starts immediately, playing one paragraph at a time using the TTS model. The player highlights the current paragraph as audio streams.
Step 3: Ask a Question About the Story
Scroll down in the Library panel to find the Ask About the Story section. Click ❓ Ask a Question — narration pauses automatically and the microphone starts recording.
Speak your question naturally (e.g. "What did Puss in Boots do to help his master?"). When you stop recording, the question is submitted automatically — no button press needed. You can also type a question in the text fallback field and press Get Answer manually.
Step 4: Hear the Spoken Answer (LFM2.5-Audio)
The recorded question waveform appears in the audio widget. Below it, the LFM2.5-Audio model generates a grounded answer in 1–2 sentences, based only on the current story context.
The answer is delivered in two ways simultaneously:
- Answer text shown inline (e.g. "Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath.")
- Answer Audio waveform auto-plays the spoken response
After the answer finishes, narration resumes from where it paused.
Step 5: Open Clone Voice Studio and Record Your Voice
Go to the Clone Voice Studio tab. Fill in:
- Voice Nickname — a name for this profile (e.g. "Mom", "Grandpa Joe")
- Voice Gender Avatar — Female or Male
A story script is displayed to read aloud. Click the Record button and read the script naturally for at least 10 seconds. The waveform displays your voice in real time. Click Stop when done.
Step 6: Analyze & Synthesize the Vocal Profile
Click Analyze & Synthesize Customized Vocal Profile. The Status Pipeline Visualizer on the right tracks progress through extraction, de-noising, and pitch tuning stages.
A progress bar shows synthesis percentage (e.g. "Generating voice preview... 70.0%") with elapsed time. This step runs Qwen3-TTS-1.7B Base to extract the speaker embedding from your recording.
Step 7: Voice Profile Ready
When synthesis completes, the Status Pipeline Visualizer shows:
- Your avatar and nickname with a "Cloned successfully" badge
- "Synthesized today using Qwen3-TTS voice cloning"
- A preview audio player — tap Play to hear a short sample in your cloned voice
- "Synced successfully! Bind Companion Voice to Inventory" confirmation
The profile is saved to disk and will persist across Space restarts.
Step 8: Story Narration in Your Cloned Voice
Return to My Library and select a story. The Now Playing panel shows Active Narrator: [your nickname] — confirming the cloned voice is active.
Narration plays in your recorded voice, paragraph by paragraph ("Paragraph 1 / 32"). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU — no audio leaves the server.