update report md

docs/REPORT_LFM_MODEL.md

@@ -0,0 +1,187 @@
+# MomsVoiceAI — Fix & Optimization Report
+
+**Date:** 2026-06-14  
+**Branch:** `fix/voice-clone-card`, `fix/lfm-model-param`
+
+---
+
+## 1. LFM2.5-Audio Inference (`inference_lfm.py`)
+
+### 1.1 System Prompt Fix
+**Before:** Story context was embedded in the system prompt as a custom "friendly storyteller" instruction.  
+**After:** System prompt simplified to the official interleaved speech-to-speech directive:
+```
+"Respond with interleaved text and audio."
+```
+Story context moved to the **user turn** as a text prefix, which is the correct placement for the LFM2.5-Audio interleaved model.
+
+### 1.2 Audio Decode — Streaming → Post-generation
+**Before:** Mimi streaming decode ran frame-by-frame during `generate_interleaved`, requiring `mimi.streaming(1)` context and manual state resets (`_reset_mimi_streaming_state`).  
+**After:** Audio tokens are collected during generation, then decoded in one call via `processor.decode(audio_codes)`. This is more stable and avoids Mimi internal buffer corruption between calls.
+
+```python
+# Before (streaming, fragile)
+with mimi.streaming(1):
+    for t in model.generate_interleaved(...):
+        if t.numel() == 8:
+            wav_chunk = mimi.decode(codes)
+
+# After (post-generation, stable)
+for t in model.generate_interleaved(...):
+    if t.numel() == 8:
+        audio_out.append(t)
+audio_codes = torch.stack(audio_out[:-1], dim=1).unsqueeze(0)  # drop EOS frame
+waveform = processor.decode(audio_codes).cpu().float()[0].numpy()
+```
+
+### 1.3 Removed `torch.compile`
+`torch.compile(mode="reduce-overhead", dynamic=True)` was applied to the LFM model on CUDA. This caused silent audio output and Mimi state bugs in practice. Removed in favor of eager mode.
+
+### 1.4 Mimi Warmup in `get_lfm_model()`
+An eager Mimi decode warmup now runs immediately after model load on CUDA to force lazy construction and catch device errors early:
+```python
+with torch.no_grad(), mimi.streaming(1):
+    mimi.decode(torch.randint(0, 2048, (1, 8, 1), device=device))
+```
+
+### 1.5 `max_new_tokens` 100 → 512 → 150
+**Before (original):** Default was `100` tokens — too short for a complete spoken answer.  
+**After (first fix):** Raised to `512` to give the model room to finish. See §5 for why this was later revised.  
+**After (current):** Reduced to `150`. See §5 for full reasoning.
+
+### 1.6 New Functions Added
+| Function | Purpose |
+|---|---|
+| `get_model_status()` | Returns device, dtype, GPU memory usage for diagnostics |
+| `text_to_audio_lfm()` | LFM-based TTS mode (read text aloud via interleaved generation) |
+| `_assemble_waveform()` | Shared waveform assembly + peak normalization helper |
+
+---
+
+## 2. Q&A Recording Flow (`app.py`)
+
+### 2.1 Removed JS Auto-click on Ask Button
+**Before:** Clicking "❓ Ask a Question" ran JavaScript to auto-click the Record button 100ms later. This fired before mic permission was granted, breaking browser microphone access.  
+**After:** JS removed. The user clicks Record manually (required by browser security model — mic access requires a direct user gesture on the Record button itself).
+
+### 2.2 `stop_recording` Wired Directly to Python
+**Before:** `stop_recording` used `fn=None` + a JS hack that clicked the "Get Answer" submit button after a 600ms delay.  
+**After:** `stop_recording` calls `auto_submit_on_stop` directly in Python, which immediately invokes LFM inference.
+
+```
+Recording stops → stop_recording → auto_submit_on_stop → build_qa_response
+  → answer_question_audio (LFM inference) → answer audio plays automatically
+```
+
+### 2.3 Removed Double-trigger Race Condition
+**Before:** Both `question_audio.change` and `stop_recording` (via JS) triggered submission independently, causing a race condition where LFM was called twice.  
+**After:** `question_audio.change` handler removed. `stop_recording` is the single auto-trigger. `claim_audio_submission` dedup guards against any remaining duplicate calls. The "Get Answer" button remains as a manual fallback for text-only questions.
+
+---
+
+## 3. Voice Clone Card Management (`app.py`)
+
+### 3.1 Persistent Voice Cards Across Page Reloads
+**Before:** `voices_state` was always initialized from `mock_voices` (hardcoded placeholders). Cloned voice profiles saved to `Voice_Profile/` never appeared as cards after a page reload.  
+**After:** On startup, `list_saved_profiles()` reads all saved `.pt` profiles from disk and builds `_initial_voices` to populate `voices_state`. Previously cloned voices appear as cards immediately on next launch.
+
+### 3.2 Voice Resets to Vivian on Page Refresh
+**Before:** `voice_profile_state` was initialized to `_default_profile_id` (most recently saved clone), so refreshing the browser silently loaded the clone.  
+**After:** `voice_profile_state = gr.State(None)` — always starts with the stock Vivian voice. The user explicitly selects a clone by clicking its card.
+
+### 3.3 Active Narrator Indicator
+Added `narrator_info` HTML component in the Library player panel displaying the currently active voice. Updates on:
+- Voice card click
+- Book selection
+
+### 3.4 Removed Fake Placeholder Voice Cards
+`Grandpa Joseph` and `Aunt Sarah` were placeholder cards with no real voice profile. Clicking them silently fell through to the stock Vivian voice, which was misleading. Removed. Only real profiles and `Mom's Voice` (Vivian stock) remain.
+
+---
+
+## 4. Pipeline Comparison — Demo vs Production
+
+The full speech-to-speech pipeline matches `app_qa_flow_demo.py` and `inference_lfm_fix.py`:
+
+| Stage | Demo / Fix | Production (current) |
+|---|---|---|
+| Audio input SR | `librosa.load(sr=16000)` | Same ✓ |
+| System prompt | `"Respond with interleaved text and audio."` | Same ✓ |
+| Story context | User turn prefix | Same ✓ |
+| `max_new_tokens` | `512` | `150` (reduced — see §5) |
+| `audio_temperature` | `1.0` | Same ✓ |
+| `audio_top_k` | `4` | Same ✓ |
+| Audio decode | `processor.decode(audio_codes)` | Same ✓ |
+| Normalization | `* (0.9 / peak)` | Same ✓ |
+| Return values | 4 `(text, wav, sr, gen_stats)` | 3 `(text, wav, sr)` — `gen_stats` dropped for `qa_flow.py` compatibility |
+| `stop_recording` trigger | Direct Python call | Same ✓ |
+| Dedup | `claim_audio_submission` | Same ✓ |
+
+The only intentional difference: `inference_lfm_fix.py` returns a 4th `gen_stats` dict. Dropped in production because `qa_flow.py` unpacks exactly 3 values (`answer_text, waveform, sr = answer_fn(...)`). Stats are logged internally instead.
+
+---
+
+## 5. Q&A Answer Quality — Prompt Engineering & Token Budget
+
+### 5.1 Problem: Long, Repetitive Answers
+
+With `max_new_tokens=512` and a minimal system prompt, the model frequently:
+- Repeated the question before answering
+- Produced 10–20 second audio responses for simple factual questions
+- Ran out of generation budget mid-sentence on longer story contexts due to KV cache pressure
+
+### 5.2 How `generate_interleaved` Counts Tokens
+
+`generate_interleaved` yields one logical step at a time:
+- 1 text token → `t.numel() == 1`
+- 1 audio frame (8 Mimi codec tokens) → `t.numel() == 8`
+
+`max_new_tokens` counts **logical steps**, not raw codec tokens. A 1-2 sentence spoken answer requires approximately:
+- ~30 text tokens
+- ~50 audio frames (~4 seconds at 12.5 fps)
+- **Total: ~80 steps**
+
+`max_new_tokens=512` allowed up to ~41 seconds of audio — 6× more than needed. If the model missed the EOS it would generate until the cap, wasting GPU memory and time.
+
+### 5.3 Fix: `max_new_tokens` 512 → 150
+
+150 steps covers a 4–6 second spoken answer with headroom. `text_to_audio_lfm` (TTS) retains `max_new_tokens=1024` since reading longer text aloud legitimately requires more frames.
+
+### 5.4 Fix: Three-layer Prompt Constraint
+
+Brevity is now enforced at three prompt positions so the model cannot ignore any single one:
+
+**Layer 1 — System prompt** (sets global behavior):
+```
+Before: "Respond with interleaved text and audio."
+After:  "Respond with interleaved text and audio. Give a short, direct answer in 1-2 sentences. Do not repeat the question."
+```
+`"Do not repeat the question"` targets the most common failure mode: the model echoing the question before answering, which consumes ~20 tokens and audio frames with no value.
+
+**Layer 2 — User context prefix** (constrains to story content):
+```
+Before: f"Story context:\n{story_context[:3000]}\n\nAnswer the question in 1-2 short sentences based only on the story."
+After:  f"Story context:\n{story_context[:2000]}\n\nBased only on the story above, answer briefly."
+```
+Context truncation also tightened from 3000 → 2000 characters (~750 → 500 tokens), leaving more of the context window budget for generation.
+
+**Layer 3 — User turn closing** (highest recency — last text model reads before generating):
+```python
+chat.add_text("Answer in 1-2 sentences only.")
+chat.end_turn()
+```
+Placed after the question (audio or text) so it is the final input before generation begins. Recency gives this the most direct influence on output length.
+
+### 5.5 Why Three Layers?
+
+Interleaved audio+text models do not respond to a single brevity instruction as reliably as pure text LLMs. The system prompt sets the prior, the context prefix reinforces it mid-conversation, and the closing instruction overrides drift just before generation starts. All three are needed for consistent short answers across different question types.
+
+### 5.6 Summary of Changes in `answer_question_audio`
+
+| Parameter | Before | After |
+|---|---|---|
+| `max_new_tokens` default | `512` | `150` |
+| System prompt | `"Respond with interleaved text and audio."` | + brevity + no-repeat instruction |
+| Story context truncation | `[:3000]` chars | `[:2000]` chars |
+| Context instruction | `"Answer the question in 1-2 short sentences…"` | `"…answer briefly."` |
+| Closing instruction | None | `"Answer in 1-2 sentences only."` after question |