mission.md

Mission

Product Vision

ReadBookMom — a parent records a short voice sample, then a bedtime story is narrated in that parent's cloned voice. Story Q&A uses a local audio model for fast spoken answers. The demo runs on local models inside a Gradio app on Hugging Face Spaces.

Problem

Parents can't always be there at bedtime. Children want to hear their parent's voice reading to them. Existing TTS is generic and impersonal, while voice cloning APIs are expensive and require sending private family audio to third parties.

Demo Loop

Parent records 15s of voice → Pick a story → Story starts in parent's voice
                                    ↓
                    Child taps Ask mid-story → Narration pauses
                                    ↓
                  Child asks a question → Hears a fast spoken answer
                                    ↓
              Resume story from the same position (or ask another question)
                                    ↓
                      Story finishes → Show "Pick another story" prompt

Latency and natural interruption are part of the demo promise: the story should start quickly, pause cleanly when the child wants to ask something, answer with low delay, then resume without losing the story position. The Q&A answer voice is intentionally generated by the local audio Q&A model rather than the cloned narrator voice. The child may ask multiple questions in a row before resuming. When the last chunk finishes playing, the app returns to the story selection screen.

Demo time budget: A 3-minute live walkthrough allows ~2 Q&A rounds (each round ≈ 15s of dead air: pause + ask + wait + answer + resume). Plan the demo script around 1–2 questions placed at natural story moments.

Hackathon Goal (2 days)

Ship a working Hugging Face Space that demonstrates:

1. Voice cloning — parent uploads/records a short audio clip, Qwen3-TTS extracts speaker embedding
2. Interruptible story narration — Qwen3-TTS streams sentence-sized chunks in the cloned voice (Supertonic fallback for stock voice)
3. Voice Q&A during narration — child taps Ask, narration pauses, LFM2.5-Audio answers from the story context, then speaks the answer directly for lower latency

Target Demo User

A parent at a laptop who records their voice, picks a short story, and shows their child the result.

Success Criteria (Hackathon)

Criteria	Target
Voice clone from ≤ 30s audio	2 of 3 listeners identify the voice in a blind A/B test
Story start latency	First streamed audio chunk in ≤ 5s
Narration interruption	Ask tap pauses playback in ≤ 500ms and preserves story position
Cached story replay	Starts immediately after first generation
Q&A answer (spoken, live)	Spoken answer starts in ≤ 8s from question submit (known compromise — a young child may lose attention)
Q&A answer (spoken, pre-generated)	Sub-1s for anticipated questions matched from background Q&A cache
Story resume latency	Resume from paused position in ≤ 1s when next chunk is cached
Works on HF Spaces	Public link, no local setup needed
Demo length	3-minute live walkthrough
Privacy posture	Voice and Q&A stay on the Space runtime

Scope

In scope:

• Upload/record parent voice sample (15–30s)
• Select from 10 pre-loaded short stories (public domain, sourced from Project Gutenberg)
• Play story narrated in cloned voice with interruptible chunked streaming
• Pause narration through an Ask button, preserve the current story chunk, answer, then resume
• Cache generated narration per voice session and story
• Ask a question about the story, receive a grounded 1–2 sentence LFM2.5-Audio answer, and hear it as low-latency generated speech
• Clean, child-friendly UI (Google Stitch-inspired via gr.Server)

Out of scope:

• User accounts, auth, database
• Offline mode, progress tracking
• Multiple languages
• Always-listening voice barge-in, echo cancellation, and open-mic interruption
• COPPA compliance (demo only)

Guardrails (Lite)

Constraint	Implementation
Content grounding	LFM2.5-Audio receives the selected story text plus a strict answer-from-story instruction
Voice privacy	All inference local on HF Space GPU — no audio leaves the server
Child safety	Pre-curated stories only; no user-uploaded content

Latency Strategy

Bottleneck	Strategy
Voice setup	Compute and cache the voice representation immediately after recording.
Story narration	Use interruptible chunked streaming: generate the first paragraph chunk, play it immediately, then continue generating queued chunks.
Interruption	Add an Ask state that pauses playback, cancels or deprioritizes queued narration generation, and stores the current chunk index.
Story replay	Cache full generated narration by voice session and story ID.
Q&A context	Send the current story position plus the most relevant story passages to LFM2.5-Audio instead of the full story when possible.
Q&A length	Cap answers to 1–2 short child-friendly sentences to reduce spoken-answer latency.
Pre-generated Q&A	While each chunk plays, generate 2–3 anticipated Q&A pairs with audio in the background. Match incoming questions against the cache for sub-1s responses; fall back to live generation on miss.
Story resume	Resume from the paused chunk after the spoken answer; use cached next chunks when available.
Voice question input	Use LFM2.5-Audio directly for audio questions; text questions bypass audio input.

Design Review Notes

Topic	Critique	Upgrade
Privacy	An external LLM API would weaken the privacy story even if audio stayed local.	Use local LFM2.5-Audio for story Q&A so the demo has one clear privacy narrative.
Latency	The cloned narrator voice is valuable for story playback, but forcing every Q&A answer through cloned TTS adds delay to the live interaction loop.	Use LFM2.5-Audio for direct audio-in/audio-out answers, cache voice setup, stream chunked narration, make playback cancellable, and cap Q&A output.
Demo clarity	Voice cloning, narration, ASR, and Q&A can feel like too many moving parts.	Present the loop as three simple actions: record, listen, ask.