MSG commited on
Commit
1719c2a
·
1 Parent(s): 83e828a

Feat/voice stuff (#7)

Browse files

* voice plan

* voice models config

* echocoach

* coach echo wip

* wip try fix record

* test record wip

* test echocoach

* wip fix recording

Files changed (40) hide show
  1. .cursor/plans/echocoach_voice_tab_45e774f7.plan.md +249 -0
  2. .cursor/plans/teachervoice_realtime_plan_8950875f.plan.md +211 -0
  3. .env.example +11 -0
  4. .gitignore +2 -1
  5. Dockerfile +5 -1
  6. USAGE.md +50 -1
  7. apps/gradio-space/pyproject.toml +2 -0
  8. apps/gradio-space/src/gradio_space/app.py +24 -6
  9. apps/gradio-space/src/gradio_space/tabs/__init__.py +7 -1
  10. apps/gradio-space/src/gradio_space/tabs/echo_coach.py +263 -0
  11. libs/echocoach/README.md +10 -0
  12. libs/echocoach/pyproject.toml +39 -0
  13. libs/echocoach/src/echocoach/__init__.py +6 -0
  14. libs/echocoach/src/echocoach/analysis/__init__.py +0 -0
  15. libs/echocoach/src/echocoach/analysis/charts.py +96 -0
  16. libs/echocoach/src/echocoach/analysis/fillers.py +82 -0
  17. libs/echocoach/src/echocoach/analysis/pace.py +49 -0
  18. libs/echocoach/src/echocoach/asr/__init__.py +0 -0
  19. libs/echocoach/src/echocoach/asr/cohere.py +51 -0
  20. libs/echocoach/src/echocoach/asr/factory.py +40 -0
  21. libs/echocoach/src/echocoach/asr/whisper_cpp.py +33 -0
  22. libs/echocoach/src/echocoach/audio_io.py +38 -0
  23. libs/echocoach/src/echocoach/coach.py +108 -0
  24. libs/echocoach/src/echocoach/config.py +228 -0
  25. libs/echocoach/src/echocoach/models.py +58 -0
  26. libs/echocoach/src/echocoach/pipeline.py +127 -0
  27. libs/echocoach/src/echocoach/recording.py +348 -0
  28. libs/echocoach/src/echocoach/tts/__init__.py +3 -0
  29. libs/echocoach/src/echocoach/tts/piper.py +123 -0
  30. libs/echocoach/src/echocoach/utils.py +54 -0
  31. libs/echocoach/tests/fixtures/silence_2s.wav +0 -0
  32. libs/echocoach/tests/make_fixture.py +19 -0
  33. libs/echocoach/tests/test_coach_parse.py +18 -0
  34. libs/echocoach/tests/test_fillers.py +17 -0
  35. libs/echocoach/tests/test_pace.py +18 -0
  36. libs/echocoach/tests/test_recording.py +132 -0
  37. pyproject.toml +2 -0
  38. scripts/echo_coach_smoke.sh +12 -0
  39. uv.lock +0 -0
  40. voice_models.yaml +75 -0
.cursor/plans/echocoach_voice_tab_45e774f7.plan.md ADDED
@@ -0,0 +1,249 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ name: EchoCoach voice tab
3
+ overview: "Add an EchoCoach tab to the existing Gradio app: record a pitch locally, transcribe with a configurable voice-model stack (Cohere Transcribe 2B default, Whisper.cpp fallback), analyze fillers/pace with matplotlib, coach via the existing MiniCPM5 text LLM, and speak feedback back with local TTS (Piper; MiniCPM-o 4.5 optional when GPU allows)."
4
+ todos:
5
+ - id: scaffold-echocoach-lib
6
+ content: Create libs/echocoach package with config.py, voice_models.yaml loader, EchoCoachResult types, and workspace pyproject wiring
7
+ status: completed
8
+ - id: analysis-module
9
+ content: Implement filler detection, pace scoring, and matplotlib chart generation (fillers bar + pace timeline)
10
+ status: completed
11
+ - id: asr-backends
12
+ content: Add Cohere Transcribe ASR backend (primary) and pywhispercpp tiny/base fallback with factory pattern
13
+ status: completed
14
+ - id: coach-tts
15
+ content: Wire coach prompts to existing inference backend; add Piper TTS VoiceOut with per-language voice map
16
+ status: completed
17
+ - id: gradio-tab
18
+ content: Build echo_coach.py tab (mic record, language, analyze, transcript HTML, charts, audio out, trace) and register in app.py
19
+ status: completed
20
+ - id: docs-docker
21
+ content: Update voice_models.yaml, .env.example, USAGE.md, Dockerfile copy paths; add unit tests and smoke fixture
22
+ status: completed
23
+ isProject: false
24
+ ---
25
+
26
+ # EchoCoach — Real-Time Voice Practice Coach
27
+
28
+ ## Goal
29
+
30
+ Ship a new **EchoCoach** tab in [`apps/gradio-space/src/gradio_space/app.py`](apps/gradio-space/src/gradio_space/app.py) that runs the hackathon demo end-to-end **locally**:
31
+
32
+ > Record up to 30s pitch → transcript with **filler words highlighted** → **pace score** chart → **rewrite suggestion** from a small text LLM → **VoiceOut** audio reply in the selected language.
33
+
34
+ Per your direction: **no SmolLM3 LoRA**. Voice I/O is config-driven; coaching stays on the existing text preset (`minicpm5-1b`).
35
+
36
+ ## Architecture
37
+
38
+ ```mermaid
39
+ flowchart LR
40
+ subgraph ui [echo_coach.py Gradio tab]
41
+ Mic[gr.Audio mic max 30s]
42
+ Lang[Language dropdown 14 langs]
43
+ VoicePreset[ASR and TTS preset]
44
+ AnalyzeBtn[Analyze pitch]
45
+ end
46
+
47
+ subgraph voice [libs/echocoach]
48
+ ASR[ASR backend factory]
49
+ Analysis[filler + pace + matplotlib]
50
+ Coach[coach prompts via inference]
51
+ TTS[TTS VoiceOut backend]
52
+ end
53
+
54
+ Mic --> ASR
55
+ Lang --> ASR
56
+ ASR --> Transcript[transcript text]
57
+ Transcript --> Analysis
58
+ Analysis --> Charts[pace + filler plots]
59
+ Transcript --> Coach
60
+ Coach --> Feedback[rewrite + tips JSON]
61
+ Feedback --> TTS
62
+ TTS --> AudioOut[gr.Audio playback]
63
+ ```
64
+
65
+ ## Voice model strategy (configurable)
66
+
67
+ Add a dedicated registry — [`voice_models.yaml`](voice_models.yaml) at repo root (parallel to [`models.yaml`](models.yaml)) — so ASR/TTS are swappable without touching the text LLM presets.
68
+
69
+ | Preset key | Role | Stack | When to use |
70
+ |------------|------|-------|-------------|
71
+ | `cohere-transcribe` (default ASR) | Speech → text | `CohereLabs/cohere-transcribe-03-2026` via `transformers>=5.4` | 14 languages, edge-friendly 2B ASR, best accuracy |
72
+ | `whisper-cpp-tiny` | Speech → text fallback | `pywhispercpp` (preferred over stale `whisper-cpp-python`) | CPU-only, fast, English-focused demos |
73
+ | `whisper-cpp-base` | Speech → text fallback | same | Better WER, still lightweight |
74
+ | `piper-multilingual` (default TTS) | Text → speech VoiceOut | Piper voices mapped per language code | Local TTS for all 14 Cohere langs |
75
+ | `minicpm-o-4.5` (optional, stretch) | Speech in + speech out | `openbmb/MiniCPM-o-4_5` with `init_audio=True, init_tts=True` | GPU workstation only (~9B); EN/ZH TTS |
76
+
77
+ **MVP ships:** `cohere-transcribe` + `piper-multilingual` + `minicpm5-1b` coach.
78
+
79
+ **Evaluate MiniCPM-o 4.5** as an alternate preset behind `ECHOCOACH_VOICE_PROFILE=omni` — do not block MVP on its heavier deps / GPU requirements.
80
+
81
+ Env vars (add to [`.env.example`](.env.example)):
82
+
83
+ - `ECHOCOACH_ASR_PRESET` — default `cohere-transcribe`
84
+ - `ECHOCOACH_TTS_PRESET` — default `piper-multilingual`
85
+ - `ECHOCOACH_COACH_MODEL` — default `minicpm5-1b` (reuses [`libs/inference`](libs/inference))
86
+ - `ECHOCOACH_MAX_SECONDS` — default `30`
87
+
88
+ ## New package: `libs/echocoach`
89
+
90
+ Mirror the inference factory pattern in a focused library.
91
+
92
+ ### Layout
93
+
94
+ ```
95
+ libs/echocoach/
96
+ pyproject.toml
97
+ src/echocoach/
98
+ config.py # load voice_models.yaml + env overrides
99
+ asr/
100
+ base.py
101
+ cohere.py # CohereAsrForConditionalGeneration wrapper
102
+ whisper_cpp.py # pywhispercpp tiny/base
103
+ factory.py
104
+ tts/
105
+ base.py
106
+ piper.py # language → voice map, WAV output
107
+ factory.py
108
+ analysis/
109
+ fillers.py # detect um/uh/like/you know/… + highlight HTML
110
+ pace.py # WPM, target band 120–160, 0–100 score
111
+ charts.py # matplotlib → PNG paths for Gradio Image
112
+ coach.py # structured JSON coach via get_backend()
113
+ pipeline.py # orchestrate: audio path → EchoCoachResult
114
+ ```
115
+
116
+ ### ASR: Cohere Transcribe (primary)
117
+
118
+ Follow the official HF quick start (`AutoProcessor`, `CohereAsrForConditionalGeneration`, `language="en"` etc.). Notes for implementation:
119
+
120
+ - Model is **gated** on Hugging Face — document `huggingface-cli login` + accept terms in USAGE.
121
+ - Requires `transformers>=5.4.0`, `soundfile`, `librosa`.
122
+ - Pass explicit `language` from the UI dropdown (`en`, `fr`, `de`, `es`, `it`, `pt`, `nl`, `pl`, `el`, `ar`, `ja`, `zh`, `vi`, `ko`).
123
+ - Resample incoming audio to 16 kHz mono (Gradio uploads may vary).
124
+
125
+ ### ASR: Whisper.cpp fallback
126
+
127
+ Use **`pywhispercpp`** (actively maintained; same whisper.cpp backend you specified). Wrap `Model('tiny'|'base').transcribe(wav_path)` for offline file transcription. Skip `pyaudio` in MVP — Gradio `gr.Audio(sources=["microphone"], type="filepath")` captures mic without PortAudio in the server process.
128
+
129
+ ### Analysis (no LLM)
130
+
131
+ Rule-based, fast, deterministic:
132
+
133
+ - **Fillers:** configurable word list + regex; return spans for HTML highlight in transcript panel.
134
+ - **Pace:** `words / (duration_minutes)`; score vs target band; flag too fast/slow.
135
+ - **Charts** ([`matplotlib`](https://matplotlib.org/) Agg backend):
136
+ - Bar chart: filler counts
137
+ - Line chart: words-per-30s-window over recording duration
138
+
139
+ ### Coach (text LLM)
140
+
141
+ Reuse [`get_backend(model_key).chat()`](libs/inference/src/inference/factory.py) with a tight system prompt asking for JSON:
142
+
143
+ ```json
144
+ {
145
+ "summary": "...",
146
+ "filler_feedback": "...",
147
+ "pace_feedback": "...",
148
+ "rewrite": "...",
149
+ "one_tip": "..."
150
+ }
151
+ ```
152
+
153
+ Parse with existing JSON repair patterns from [`libs/agent/src/agent/runner.py`](libs/agent/src/agent/runner.py) (reuse `_parse_json` style, don't duplicate agent runner).
154
+
155
+ ### TTS VoiceOut
156
+
157
+ - **Piper:** map language code → voice model; synthesize coach `summary + one_tip` (or full rewrite on toggle) to WAV under `AGENT_OUTPUTS_DIR`.
158
+ - Return filepath to `gr.Audio` for playback.
159
+ - If Piper voice missing for a language, fall back to English voice + show UI warning.
160
+
161
+ ## Gradio tab: `echo_coach.py`
162
+
163
+ New file [`apps/gradio-space/src/gradio_space/tabs/echo_coach.py`](apps/gradio-space/src/gradio_space/tabs/echo_coach.py).
164
+
165
+ **UI components:**
166
+
167
+ | Component | Purpose |
168
+ |-----------|---------|
169
+ | `gr.Audio` (mic, max 30s) | Record pitch |
170
+ | Language dropdown (14 codes) | Cohere ASR + TTS voice selection |
171
+ | ASR preset dropdown (dev) | `cohere-transcribe` / `whisper-cpp-tiny` / `whisper-cpp-base` |
172
+ | Coach model status | Reuse `model_status()` from [`model_loading.py`](apps/gradio-space/src/gradio_space/model_loading.py) |
173
+ | Analyze button | Run full pipeline |
174
+ | Transcript HTML | Filler highlights |
175
+ | Markdown report | Pace score, filler count, coach JSON fields |
176
+ | `gr.Image` × 2 | Matplotlib charts |
177
+ | `gr.Audio` output | VoiceOut playback |
178
+ | `gr.JSON` | Trace (Sharing is Caring badge) |
179
+
180
+ Wire into [`app.py`](apps/gradio-space/src/gradio_space/app.py) as a fourth tab and export from [`tabs/__init__.py`](apps/gradio-space/src/gradio_space/tabs/__init__.py).
181
+
182
+ ## Dependencies
183
+
184
+ Add to workspace:
185
+
186
+ | Package | Where | Why |
187
+ |---------|-------|-----|
188
+ | `echocoach` | root `pyproject.toml` workspace member | new lib |
189
+ | `matplotlib`, `soundfile`, `librosa` | `libs/echocoach` | charts + audio I/O |
190
+ | `pywhispercpp` | `libs/echocoach` optional extra `[whisper]` | whisper fallback |
191
+ | `piper-tts` or `piper-phonemize` | `libs/echocoach` optional extra `[piper]` | VoiceOut |
192
+ | bump `transformers>=5.4` | `libs/inference` or `echocoach` | Cohere ASR class |
193
+
194
+ Keep **`pyaudio` out of MVP** (Gradio handles capture). Document optional streaming mode (pyaudio + segment callback) as phase 2.
195
+
196
+ ## Docker / HF Space notes
197
+
198
+ Current [`Dockerfile`](Dockerfile) copies only gradio-space + agent + inference. EchoCoach requires:
199
+
200
+ - Copy `libs/echocoach` + `voice_models.yaml`
201
+ - `apt-get install` `ffmpeg`, `libsndfile1` (audio deps)
202
+ - **Mic access:** HF Space visitors can **upload** audio; live mic is **local dev only**
203
+ - **GPU basic** recommended when `cohere-transcribe` is active (2B ASR + 1B coach)
204
+
205
+ ## Testing
206
+
207
+ Lightweight unit tests in `libs/echocoach/tests/`:
208
+
209
+ - `test_fillers.py` — highlight spans on sample transcript
210
+ - `test_pace.py` — WPM + score for known duration/word count
211
+ - `test_coach_parse.py` — JSON extraction from mocked LLM output
212
+ - Skip GPU/integration tests in CI; smoke script `scripts/echo_coach_smoke.sh` with a bundled 5s WAV fixture
213
+
214
+ ## Docs
215
+
216
+ Update [`USAGE.md`](USAGE.md):
217
+
218
+ - EchoCoach tab walkthrough
219
+ - Voice preset config (`voice_models.yaml`, env vars)
220
+ - HF gated model login for Cohere Transcribe
221
+ - Local-only mic vs Space upload
222
+ - Hardware guidance (CPU whisper vs GPU cohere)
223
+
224
+ ## Implementation order
225
+
226
+ 1. **`libs/echocoach` scaffold** — config, types, `EchoCoachResult` dataclass
227
+ 2. **Analysis module** — fillers, pace, matplotlib (no ML deps)
228
+ 3. **ASR backends** — Cohere first, whisper fallback second
229
+ 4. **Coach module** — prompts + inference integration
230
+ 5. **TTS Piper backend** — language voice map + WAV output
231
+ 6. **Gradio tab** — wire UI + pipeline
232
+ 7. **Registry + docs** — `voice_models.yaml`, `.env.example`, USAGE.md, Dockerfile copy paths
233
+
234
+ ## Risks and mitigations
235
+
236
+ | Risk | Mitigation |
237
+ |------|------------|
238
+ | Cohere model gated / large download | Document HF auth; allow `whisper-cpp-tiny` preset for offline CPU demo |
239
+ | Cohere needs transformers 5.4+ | Pin in `echocoach` pyproject; test alongside existing inference |
240
+ | Piper voice coverage gaps | English fallback + visible warning in UI |
241
+ | MiniCPM-o 4.5 instability / VRAM | Optional preset only; not MVP blocker |
242
+ | Real-time duplex | Deferred; MVP is record-then-analyze (matches demo) |
243
+
244
+ ## Hackathon alignment
245
+
246
+ - **Backyard track** — teacher/student voice practice for someone you know
247
+ - **Tiny Titan** — coach on MiniCPM5 1B; ASR on 2B Cohere (still "small model" stack)
248
+ - **All local** — no cloud LLM API; optional HF weight download only
249
+ - **Sharing is Caring** — JSON trace per analysis under `outputs/traces/`
.cursor/plans/teachervoice_realtime_plan_8950875f.plan.md ADDED
@@ -0,0 +1,211 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ name: TeacherVoice Realtime Plan
3
+ overview: EchoCoach today is a one-shot pitch *review* pipeline (record → analyze → report). TeacherVoice-style real-time teaching needs a new turn-based voice conversation layer that reuses existing ASR/LLM/TTS pieces but changes the interaction model, prompts, and integration with lessons/RAG.
4
+ todos:
5
+ - id: teacher-voice-core
6
+ content: Add teacher_voice.py + mode prompts + TeacherVoiceTurnResult; wire ASR → chat(history) → TTS per turn
7
+ status: pending
8
+ - id: teacher-voice-ui
9
+ content: Create teacher_voice.py Gradio tab (modes, push-to-talk, chatbot, audio reply, optional RAG); register in app.py
10
+ status: pending
11
+ - id: rag-lesson-modes
12
+ content: Integrate ResearchMind retrieval into Explain/Lesson modes (reuse run_research_question / rag scope from chat tab)
13
+ status: pending
14
+ - id: docs-tests
15
+ content: Update USAGE.md with EchoCoach vs TeacherVoice; add mock-backend unit tests and trace skill teacher-voice
16
+ status: pending
17
+ - id: phase2-chunked-tts
18
+ content: "Optional: sentence-chunked Piper + shorter turn cap for faster time-to-first-audio"
19
+ status: pending
20
+ - id: phase3-omni
21
+ content: "Optional: MiniCPM-o 4.5 omni preset behind ECHOCOACH_VOICE_PROFILE=omni for speech-in/speech-out"
22
+ status: pending
23
+ isProject: false
24
+ ---
25
+
26
+ # TeacherVoice — from EchoCoach review to real-time voice teacher
27
+
28
+ ## What “coach help” is today
29
+
30
+ In this repo, **coach help = EchoCoach** — a separate Gradio tab, not a global assistant.
31
+
32
+ | What it does | What it does **not** do |
33
+ |--------------|-------------------------|
34
+ | Records up to 30s of **your** monologue (pitch practice) | Talk back-and-forth in a conversation |
35
+ | Transcribes → scores fillers/pace → one LLM **JSON report** | Explain lesson topics on demand |
36
+ | Speaks **one** Piper TTS clip (summary or rewrite) | Create slides or pitch decks by voice |
37
+ | Runs once per click on **Analyze pitch** | Stream partial audio or interrupt mid-sentence |
38
+
39
+ Pipeline (batch, sequential):
40
+
41
+ ```mermaid
42
+ sequenceDiagram
43
+ participant User
44
+ participant Gradio as EchoCoach_tab
45
+ participant Pipe as run_echo_coach
46
+ participant ASR
47
+ participant Coach as coach.py_JSON
48
+ participant TTS
49
+
50
+ User->>Gradio: Record_stop_then_Analyze
51
+ Gradio->>Pipe: audio_filepath
52
+ Pipe->>ASR: full_file_transcribe
53
+ Pipe->>Pipe: fillers_pace_charts
54
+ Pipe->>Coach: single_shot_JSON_prompt
55
+ Pipe->>TTS: one_WAV_summary
56
+ Pipe->>Gradio: report_charts_audio
57
+ ```
58
+
59
+ Key code: [`libs/echocoach/src/echocoach/pipeline.py`](libs/echocoach/src/echocoach/pipeline.py) orchestrates ASR → analysis → [`coach.py`](libs/echocoach/src/echocoach/coach.py) → Piper. UI is [`apps/gradio-space/src/gradio_space/tabs/echo_coach.py`](apps/gradio-space/src/gradio_space/tabs/echo_coach.py).
60
+
61
+ **Lesson slides** ([`education_pptx.py`](apps/gradio-space/src/gradio_space/tabs/education_pptx.py)) and **Chat (debug)** ([`chat.py`](apps/gradio-space/src/gradio_space/tabs/chat.py)) are unrelated tabs: slides are batch `AgentRunner` jobs; chat is **text-only** multi-turn (optionally RAG via [`rag_aware_chat`](apps/gradio-space/src/gradio_space/research_helpers.py)).
62
+
63
+ There is **no `TeacherVoice` name, WebSocket, or streaming voice** anywhere in the codebase. The EchoCoach plan explicitly deferred duplex:
64
+
65
+ > Real-time duplex | Deferred; MVP is record-then-analyze
66
+
67
+ ---
68
+
69
+ ## How TeacherVoice differs (target behavior)
70
+
71
+ **TeacherVoice** (what you’re describing) is a **conversational voice teacher**:
72
+
73
+ - You speak a **question or short turn** → teacher **replies in voice** → repeat
74
+ - Modes share one chat history but different system prompts:
75
+ - **Explain** — tutor a topic in plain language (optionally grounded in ResearchMind RAG)
76
+ - **Lesson** — discuss/outline a lesson verbally (can hand off topic to Lesson slides tab)
77
+ - **Pitch** — lighter coaching per turn (“try opening with X”) instead of one big JSON report
78
+
79
+ ```mermaid
80
+ sequenceDiagram
81
+ participant User
82
+ participant TV as TeacherVoice_tab
83
+ participant ASR
84
+ participant LLM as chat_with_history
85
+ participant TTS
86
+
87
+ loop each_turn
88
+ User->>TV: push_to_talk_stop
89
+ TV->>ASR: transcribe_turn
90
+ TV->>LLM: messages_plus_mode_system_prompt
91
+ LLM-->>TV: reply_text
92
+ TV->>TTS: synthesize_reply
93
+ TV->>User: play_audio_update_chat
94
+ end
95
+ ```
96
+
97
+ This is **turn-based pseudo-real-time** (typical latency: ~2–8s per turn on GPU). True **duplex** TeacherVoice (interrupt while speaking, sub-second feel) would need streaming ASR + chunked TTS or an omni speech model — not present today.
98
+
99
+ ---
100
+
101
+ ## Recommended architecture (phased)
102
+
103
+ ### Phase 1 — TeacherVoice tab (MVP, fits current stack)
104
+
105
+ Add a **TeacherVoice** tab alongside EchoCoach; keep EchoCoach as the deep pitch *analyzer*.
106
+
107
+ **New module** in `libs/echocoach` (minimal new package surface):
108
+
109
+ - `teacher_voice.py` — `run_teacher_voice_turn(audio_path, history, mode, language, backend, rag_context?) -> TeacherVoiceTurnResult`
110
+ - `prompts.py` — mode system prompts (`explain`, `lesson`, `pitch`) as **plain text** chat (not JSON like EchoCoach)
111
+ - Reuse: [`recording.py`](libs/echocoach/src/echocoach/recording.py), ASR factory, Piper TTS, `InferenceBackend.chat()`
112
+
113
+ **New Gradio tab** `teacher_voice.py`:
114
+
115
+ - Mode dropdown: Explain | Lesson coach | Pitch practice
116
+ - Optional: ResearchMind session/docs (copy pattern from [`chat.py`](apps/gradio-space/src/gradio_space/tabs/chat.py))
117
+ - Topic field for Lesson/Explain modes
118
+ - **Push-to-talk**: reuse Start/Stop recording from EchoCoach (already works server-side)
119
+ - `gr.Chatbot` (text history) + `gr.Audio` auto-play for teacher reply
120
+ - “Clear conversation” button
121
+
122
+ **Turn flow per button press:**
123
+
124
+ 1. Stop recording → WAV path
125
+ 2. ASR → user text (show in chat)
126
+ 3. Build messages: `system(mode)` + `history` + optional RAG context block + `user(transcript)`
127
+ 4. `backend.chat()` — same stack as debug chat ([`inference`](libs/inference/src/inference/base.py) has no streaming today; full response is fine for MVP)
128
+ 5. Piper TTS → play reply
129
+ 6. Append `(user, assistant)` to history
130
+
131
+ **Pitch mode vs EchoCoach:** Pitch mode gives **conversational** tips each turn; EchoCoach remains the **quantitative** tool (WPM, filler charts, rewrite JSON). Link from TeacherVoice: “Deep analysis → open EchoCoach tab with last recording.”
132
+
133
+ **Lesson mode:** Does not generate `.pptx` by voice in MVP. It verbally outlines and explains; user can copy topic into Lesson slides tab. Phase 2 can add “Generate slides from this conversation” button calling `AgentRunner.run_education_pptx`.
134
+
135
+ ### Phase 2 — Faster “feels live” (still not full duplex)
136
+
137
+ - **VAD / max 15s turns** — cap turn length for lower latency
138
+ - **Sentence-chunked TTS** — split LLM reply on `.!?`, synthesize first sentence while rest generates (reuse Piper in a small loop)
139
+ - **Optional streaming ASR** — only if a backend supports partial transcripts (Cohere batch is fine for MVP; whisper stays file-based)
140
+ - Latency target: first audio ~3s after stop on GPU
141
+
142
+ ### Phase 3 — True speech-in/speech-out (optional, GPU-heavy)
143
+
144
+ From [`echocoach_voice_tab` plan](.cursor/plans/echocoach_voice_tab_45e774f7.plan.md), not implemented:
145
+
146
+ - Add `minicpm-o-4.5` preset to [`voice_models.yaml`](voice_models.yaml) behind `ECHOCOACH_VOICE_PROFILE=omni`
147
+ - New omni backend in `libs/echocoach` (or `libs/inference`) with `init_audio=True, init_tts=True`
148
+ - TeacherVoice tab switches to omni when profile=omni and GPU available
149
+ - EN/ZH only initially; falls back to Phase 1 pipeline otherwise
150
+
151
+ Skip WebSocket server unless you need browser-native duplex outside Gradio — Gradio turn-based is enough for hackathon demo.
152
+
153
+ ---
154
+
155
+ ## What to reuse vs build new
156
+
157
+ | Component | Reuse | New work |
158
+ |-----------|-------|----------|
159
+ | Mic capture | `recording.py`, `gr.Audio` | Wire to per-turn handler |
160
+ | ASR | `asr/factory.py` | Call per turn, not once per monologue |
161
+ | LLM | `get_backend().chat()` | Mode prompts + multi-turn history (like chat tab) |
162
+ | RAG | `run_research_question` / `rag_aware_chat` logic | Inject retrieved context into TeacherVoice system prompt |
163
+ | TTS | `tts/piper.py` | Per-turn synthesis; optional chunking in Phase 2 |
164
+ | Pitch analytics | `analysis/fillers.py`, `pace.py` | **Not** on every TeacherVoice turn — keep in EchoCoach only |
165
+ | Lesson PPTX | `AgentRunner.run_education_pptx` | Phase 2 button, not MVP voice loop |
166
+
167
+ ---
168
+
169
+ ## Files to touch (Phase 1)
170
+
171
+ | File | Change |
172
+ |------|--------|
173
+ | [`libs/echocoach/src/echocoach/teacher_voice.py`](libs/echocoach/src/echocoach/teacher_voice.py) | **New** — turn orchestration + result type |
174
+ | [`libs/echocoach/src/echocoach/prompts.py`](libs/echocoach/src/echocoach/prompts.py) | **New** — Explain / Lesson / Pitch system prompts |
175
+ | [`apps/gradio-space/src/gradio_space/tabs/teacher_voice.py`](apps/gradio-space/src/gradio_space/tabs/teacher_voice.py) | **New** — UI |
176
+ | [`apps/gradio-space/src/gradio_space/app.py`](apps/gradio-space/src/gradio_space/app.py) | Register TeacherVoice tab |
177
+ | [`apps/gradio-space/src/gradio_space/tabs/__init__.py`](apps/gradio-space/src/gradio_space/tabs/__init__.py) | Export builder |
178
+ | [`USAGE.md`](USAGE.md) | Document modes, latency expectations, EchoCoach vs TeacherVoice |
179
+ | [`libs/echocoach/tests/test_teacher_voice.py`](libs/echocoach/tests/test_teacher_voice.py) | Mock-backend tests for prompt assembly + history |
180
+
181
+ EchoCoach files stay unchanged except optional cross-link in UI markdown.
182
+
183
+ ---
184
+
185
+ ## Hardware and UX expectations
186
+
187
+ - **CPU-only:** Whisper tiny ASR + MiniCPM5 1B + Piper — workable but ~5–15s per turn; set expectations in UI
188
+ - **GPU:** Cohere Transcribe + same coach model — better for demo
189
+ - **HF Space:** Upload/push-to-talk may be limited; document local-only mic (same as EchoCoach)
190
+ - Label honestly: **“Voice conversation (turn-based)”** until Phase 3 omni
191
+
192
+ ---
193
+
194
+ ## Success criteria for MVP
195
+
196
+ - User can hold a **3+ turn** spoken conversation in Explain mode
197
+ - Lesson mode accepts a topic + optional RAG session and answers from ingested docs with citations in chat text
198
+ - Pitch mode gives short spoken coaching without requiring Analyze pitch
199
+ - EchoCoach tab still works independently for full pitch analysis
200
+ - Trace JSON per session under `outputs/traces/` (skill: `teacher-voice`)
201
+
202
+ ---
203
+
204
+ ## Risks
205
+
206
+ | Risk | Mitigation |
207
+ |------|------------|
208
+ | Users expect ChatGPT Voice latency | UI copy: turn-based; show “Transcribing / Thinking / Speaking” states |
209
+ | RAG + voice doubles latency | Retrieve once per turn; keep `max_tokens` modest (~512) |
210
+ | Piper missing voice for language | Existing English fallback in [`voice_models.yaml`](voice_models.yaml) |
211
+ | Confusion between EchoCoach and TeacherVoice | Separate tabs; Pitch mode points to EchoCoach for charts |
.env.example CHANGED
@@ -50,6 +50,17 @@ ALLOW_MODEL_SWITCH=false
50
  # After training, point Gradio at the adapter preset:
51
  # ACTIVE_MODEL=minicpm5-1b-lesson-lora
52
 
 
 
 
 
 
 
 
 
 
 
 
53
  # --- Ensemble research (research/ensemble/) ---
54
  # Base LLM resolution (first match wins): ENSEMBLE_LLM, LLM_PATH, BASE, MODEL_ID, ACTIVE_MODEL
55
  # LLM_PATH=./models/finetuned/minicpm5-1b-lora-merged
 
50
  # After training, point Gradio at the adapter preset:
51
  # ACTIVE_MODEL=minicpm5-1b-lesson-lora
52
 
53
+ # --- EchoCoach (voice practice coach) ---
54
+ # VOICE_PRESETS_PATH=./voice_models.yaml
55
+ # ECHOCOACH_ASR_PRESET=whisper-cpp-tiny
56
+ # ECHOCOACH_TTS_PRESET=piper-multilingual
57
+ # ECHOCOACH_COACH_MODEL=minicpm5-1b
58
+ # ECHOCOACH_MAX_SECONDS=30
59
+ # ECHOCOACH_CAPTURE_DEVICE= # optional ALSA/PipeWire device (e.g. pipewire, alsa_input.pci-...)
60
+ # PIPER_VOICES_DIR=~/.local/share/piper/voices
61
+ # For Cohere Transcribe ASR: huggingface-cli login + accept model terms, then:
62
+ # ECHOCOACH_ASR_PRESET=cohere-transcribe
63
+
64
  # --- Ensemble research (research/ensemble/) ---
65
  # Base LLM resolution (first match wins): ENSEMBLE_LLM, LLM_PATH, BASE, MODEL_ID, ACTIVE_MODEL
66
  # LLM_PATH=./models/finetuned/minicpm5-1b-lora-merged
.gitignore CHANGED
@@ -14,4 +14,5 @@ outputs/traces
14
 
15
  /results
16
 
17
- outputs/researchmind
 
 
14
 
15
  /results
16
 
17
+ outputs/researchmind
18
+ outputs/echocoach
Dockerfile CHANGED
@@ -7,19 +7,23 @@ ENV PYTHONUNBUFFERED=1 \
7
  RUN apt-get update && apt-get install -y --no-install-recommends \
8
  build-essential \
9
  cmake \
 
 
10
  && rm -rf /var/lib/apt/lists/*
11
 
12
  COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
13
 
14
  WORKDIR /app
15
 
16
- COPY pyproject.toml uv.lock .python-version README.md models.yaml ./
17
  COPY apps/gradio-space/pyproject.toml apps/gradio-space/README.md apps/gradio-space/
18
  COPY libs/inference/pyproject.toml libs/inference/README.md libs/inference/
19
  COPY libs/agent/pyproject.toml libs/agent/README.md libs/agent/
 
20
  COPY apps/gradio-space/src apps/gradio-space/src
21
  COPY libs/inference/src libs/inference/src
22
  COPY libs/agent/src libs/agent/src
 
23
  COPY skills skills
24
 
25
  RUN useradd -m -u 1000 user && \
 
7
  RUN apt-get update && apt-get install -y --no-install-recommends \
8
  build-essential \
9
  cmake \
10
+ ffmpeg \
11
+ libsndfile1 \
12
  && rm -rf /var/lib/apt/lists/*
13
 
14
  COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
15
 
16
  WORKDIR /app
17
 
18
+ COPY pyproject.toml uv.lock .python-version README.md models.yaml voice_models.yaml ./
19
  COPY apps/gradio-space/pyproject.toml apps/gradio-space/README.md apps/gradio-space/
20
  COPY libs/inference/pyproject.toml libs/inference/README.md libs/inference/
21
  COPY libs/agent/pyproject.toml libs/agent/README.md libs/agent/
22
+ COPY libs/echocoach/pyproject.toml libs/echocoach/README.md libs/echocoach/
23
  COPY apps/gradio-space/src apps/gradio-space/src
24
  COPY libs/inference/src libs/inference/src
25
  COPY libs/agent/src libs/agent/src
26
+ COPY libs/echocoach/src libs/echocoach/src
27
  COPY skills skills
28
 
29
  RUN useradd -m -u 1000 user && \
USAGE.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  How to run the **Lesson Agent** Gradio app locally, test it in Docker, and deploy to a Hugging Face Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon).
4
 
5
- The primary UI is the **Lesson slides** tab (topic → local model outline → downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model.
6
 
7
  ## Prerequisites
8
 
@@ -72,6 +72,55 @@ When **Web search** is selected, choose a **search workflow**:
72
 
73
  Web discover/auto search requires network access. MemRAG data is stored under `RESEARCHMIND_DATA_DIR` (default `outputs/researchmind`).
74
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
  ### 5. Upload agent trace (Sharing is Caring badge)
76
 
77
  ```bash
 
2
 
3
  How to run the **Lesson Agent** Gradio app locally, test it in Docker, and deploy to a Hugging Face Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon).
4
 
5
+ The primary UI is the **Lesson slides** tab (topic → local model outline → downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, **EchoCoach** for voice practice feedback, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model.
6
 
7
  ## Prerequisites
8
 
 
72
 
73
  Web discover/auto search requires network access. MemRAG data is stored under `RESEARCHMIND_DATA_DIR` (default `outputs/researchmind`).
74
 
75
+ Web discover/auto search requires network access. MemRAG data is stored under `RESEARCHMIND_DATA_DIR` (default `outputs/researchmind`).
76
+
77
+ ### EchoCoach — voice practice
78
+
79
+ The **EchoCoach** tab records up to 30 seconds, then runs a local pipeline:
80
+
81
+ **Getting audio in**
82
+
83
+ - **Record from this computer** — click **Start recording**, speak, then **Stop recording** (uses PipeWire `pw-record` when available). The slider is a max-length safety cap.
84
+ - **Browser Record** — needs mic permission and a secure context; open **http://localhost:7860** (not `0.0.0.0` or a LAN IP).
85
+ - **Upload** — drop a `.wav` or `.mp3` file (works everywhere, including HF Space).
86
+
87
+ If recordings sound silent, check system mic input/mute or set `ECHOCOACH_CAPTURE_DEVICE` in `.env` (see `arecord -L` or `pw-cli ls Node`).
88
+
89
+ Pipeline steps:
90
+
91
+ 1. **ASR** — Cohere Transcribe 2B (14 languages) or Whisper.cpp tiny/base
92
+ 2. **Analysis** — filler highlights, pace score, matplotlib charts
93
+ 3. **Coach** — rewrite + tips from the text LLM (`ACTIVE_MODEL`, default `minicpm5-1b`)
94
+ 4. **VoiceOut** — Piper TTS speaks the summary (or full rewrite if checked)
95
+
96
+ Install optional extras:
97
+
98
+ ```bash
99
+ # Whisper.cpp fallback ASR (CPU)
100
+ uv sync --package echocoach --extra whisper
101
+
102
+ # Piper VoiceOut TTS
103
+ uv sync --package echocoach --extra piper
104
+ python -m piper.download_voices en_US-lessac-medium
105
+ ```
106
+
107
+ Configure presets in [`voice_models.yaml`](voice_models.yaml) or via `.env`:
108
+
109
+ | Variable | Default | Description |
110
+ | -------- | ------- | ----------- |
111
+ | `ECHOCOACH_ASR_PRESET` | `whisper-cpp-tiny` | ASR preset key |
112
+ | `ECHOCOACH_TTS_PRESET` | `piper-multilingual` | TTS preset key |
113
+ | `ECHOCOACH_COACH_MODEL` | `minicpm5-1b` | Text coach preset (from `models.yaml`) |
114
+ | `ECHOCOACH_MAX_SECONDS` | `30` | Max recording length |
115
+
116
+ **Cohere Transcribe** (`cohere-transcribe`) is gated on Hugging Face — run `huggingface-cli login`, accept the model terms, then set `ECHOCOACH_ASR_PRESET=cohere-transcribe`. GPU recommended for ASR + coach together.
117
+
118
+ Smoke tests (analysis only, no GPU):
119
+
120
+ ```bash
121
+ bash scripts/echo_coach_smoke.sh
122
+ ```
123
+
124
  ### 5. Upload agent trace (Sharing is Caring badge)
125
 
126
  ```bash
apps/gradio-space/pyproject.toml CHANGED
@@ -9,12 +9,14 @@ authors = [
9
  requires-python = ">=3.12"
10
  dependencies = [
11
  "agent",
 
12
  "gradio>=5.0.0",
13
  "inference",
14
  ]
15
 
16
  [tool.uv.sources]
17
  agent = { workspace = true }
 
18
  inference = { workspace = true }
19
 
20
  [build-system]
 
9
  requires-python = ">=3.12"
10
  dependencies = [
11
  "agent",
12
+ "echocoach[whisper]",
13
  "gradio>=5.0.0",
14
  "inference",
15
  ]
16
 
17
  [tool.uv.sources]
18
  agent = { workspace = true }
19
+ echocoach = { workspace = true }
20
  inference = { workspace = true }
21
 
22
  [build-system]
apps/gradio-space/src/gradio_space/app.py CHANGED
@@ -3,8 +3,14 @@ import os
3
  import gradio as gr
4
 
5
  from gradio_space.model_loading import preload_active_model
6
- from gradio_space.tabs import build_chat_tab, build_education_pptx_tab, build_research_mind_tab
 
 
 
 
 
7
  from gradio_space.tabs.education_pptx import gradio_allowed_paths
 
8
  from gradio_space.tabs.research_mind import researchmind_allowed_paths
9
  from inference.config import get_app_config
10
 
@@ -22,9 +28,9 @@ def build_demo() -> gr.Blocks:
22
  with gr.Blocks(title="Lesson Agent + ResearchMind — Build Small Hackathon") as demo:
23
  gr.Markdown(
24
  f"""
25
- # Lesson Agent + ResearchMind
26
 
27
- Local skill-based agents — **lesson slides** and **research with MemRAG** (offline Q&A after ingest).
28
 
29
  - **Model:** `{active.key}` — {active.label}
30
  - **Backend:** `{active.backend}`
@@ -39,6 +45,8 @@ Part of the [Build Small Hackathon](https://huggingface.co/build-small-hackathon
39
  build_education_pptx_tab()
40
  with gr.Tab("ResearchMind"):
41
  build_research_mind_tab()
 
 
42
  with gr.Tab("Chat (debug)"):
43
  build_chat_tab()
44
 
@@ -48,10 +56,20 @@ Part of the [Build Small Hackathon](https://huggingface.co/build-small-hackathon
48
  def main() -> None:
49
  preload_active_model()
50
  demo = build_demo()
 
 
 
 
 
 
51
  demo.launch(
52
- server_name="0.0.0.0",
53
- server_port=int(os.environ.get("PORT", "7860")),
54
- allowed_paths=[*gradio_allowed_paths(), *researchmind_allowed_paths()],
 
 
 
 
55
  )
56
 
57
 
 
3
  import gradio as gr
4
 
5
  from gradio_space.model_loading import preload_active_model
6
+ from gradio_space.tabs import (
7
+ build_chat_tab,
8
+ build_education_pptx_tab,
9
+ build_echo_coach_tab,
10
+ build_research_mind_tab,
11
+ )
12
  from gradio_space.tabs.education_pptx import gradio_allowed_paths
13
+ from gradio_space.tabs.echo_coach import echo_coach_allowed_paths
14
  from gradio_space.tabs.research_mind import researchmind_allowed_paths
15
  from inference.config import get_app_config
16
 
 
28
  with gr.Blocks(title="Lesson Agent + ResearchMind — Build Small Hackathon") as demo:
29
  gr.Markdown(
30
  f"""
31
+ # Lesson Agent + ResearchMind + EchoCoach
32
 
33
+ Local skill-based agents — **lesson slides**, **research with MemRAG**, and **voice practice coaching** (offline).
34
 
35
  - **Model:** `{active.key}` — {active.label}
36
  - **Backend:** `{active.backend}`
 
45
  build_education_pptx_tab()
46
  with gr.Tab("ResearchMind"):
47
  build_research_mind_tab()
48
+ with gr.Tab("EchoCoach"):
49
+ build_echo_coach_tab()
50
  with gr.Tab("Chat (debug)"):
51
  build_chat_tab()
52
 
 
56
  def main() -> None:
57
  preload_active_model()
58
  demo = build_demo()
59
+ port = int(os.environ.get("PORT", "7860"))
60
+ server_name = os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0")
61
+ print(
62
+ f"\n Local UI (browser mic works here): http://127.0.0.1:{port}\n"
63
+ f" Bound address: {server_name}:{port}\n"
64
+ )
65
  demo.launch(
66
+ server_name=server_name,
67
+ server_port=port,
68
+ allowed_paths=[
69
+ *gradio_allowed_paths(),
70
+ *researchmind_allowed_paths(),
71
+ *echo_coach_allowed_paths(),
72
+ ],
73
  )
74
 
75
 
apps/gradio-space/src/gradio_space/tabs/__init__.py CHANGED
@@ -1,5 +1,11 @@
1
  from gradio_space.tabs.chat import build_chat_tab
2
  from gradio_space.tabs.education_pptx import build_education_pptx_tab
 
3
  from gradio_space.tabs.research_mind import build_research_mind_tab
4
 
5
- __all__ = ["build_chat_tab", "build_education_pptx_tab", "build_research_mind_tab"]
 
 
 
 
 
 
1
  from gradio_space.tabs.chat import build_chat_tab
2
  from gradio_space.tabs.education_pptx import build_education_pptx_tab
3
+ from gradio_space.tabs.echo_coach import build_echo_coach_tab
4
  from gradio_space.tabs.research_mind import build_research_mind_tab
5
 
6
+ __all__ = [
7
+ "build_chat_tab",
8
+ "build_education_pptx_tab",
9
+ "build_echo_coach_tab",
10
+ "build_research_mind_tab",
11
+ ]
apps/gradio-space/src/gradio_space/tabs/echo_coach.py ADDED
@@ -0,0 +1,263 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ from pathlib import Path
4
+
5
+ import gradio as gr
6
+
7
+ from echocoach.config import get_echo_coach_config
8
+ from echocoach.pipeline import run_echo_coach
9
+ from echocoach.recording import (
10
+ ServerRecordingError,
11
+ recording_backend_status,
12
+ recording_elapsed_seconds,
13
+ recording_level_warning,
14
+ start_server_recording,
15
+ stop_server_recording,
16
+ )
17
+ from gradio_space.model_loading import ensure_model_loaded, get_active_model_key, model_status
18
+ from inference.factory import get_backend
19
+
20
+ _config = get_echo_coach_config()
21
+ _SAMPLE_AUDIO = (
22
+ Path(__file__).resolve().parents[5]
23
+ / "libs"
24
+ / "echocoach"
25
+ / "tests"
26
+ / "fixtures"
27
+ / "silence_2s.wav"
28
+ )
29
+
30
+
31
+ def _error_outputs(message: str) -> tuple:
32
+ return (
33
+ message,
34
+ f'<p style="color:#8a1f1f;">{message}</p>',
35
+ "",
36
+ None,
37
+ None,
38
+ None,
39
+ message,
40
+ {},
41
+ )
42
+
43
+
44
+ def ui_start_recording(max_seconds: int) -> tuple[str, dict, dict]:
45
+ try:
46
+ start_server_recording(int(max_seconds))
47
+ except ServerRecordingError as exc:
48
+ return (
49
+ str(exc),
50
+ gr.update(interactive=True),
51
+ gr.update(interactive=False),
52
+ )
53
+ return (
54
+ (
55
+ f"Recording… speak now, then click **Stop recording** "
56
+ f"(auto-stops after {int(max_seconds)}s)."
57
+ ),
58
+ gr.update(interactive=False),
59
+ gr.update(interactive=True),
60
+ )
61
+
62
+
63
+ def ui_stop_recording() -> tuple[str | None, str, dict, dict]:
64
+ try:
65
+ elapsed = recording_elapsed_seconds()
66
+ path = stop_server_recording()
67
+ warning = recording_level_warning(path)
68
+ except ServerRecordingError as exc:
69
+ return (
70
+ None,
71
+ str(exc),
72
+ gr.update(interactive=True),
73
+ gr.update(interactive=False),
74
+ )
75
+ except Exception as exc: # noqa: BLE001 — surface unexpected recorder errors
76
+ return (
77
+ None,
78
+ f"Recording failed: {exc}",
79
+ gr.update(interactive=True),
80
+ gr.update(interactive=False),
81
+ )
82
+
83
+ status = f"Recording saved ({elapsed:.1f}s) → `{path}`. Click **Analyze pitch**."
84
+ if warning:
85
+ status += f" Warning: {warning}"
86
+ return (
87
+ gr.update(value=str(path)),
88
+ status,
89
+ gr.update(interactive=True),
90
+ gr.update(interactive=False),
91
+ )
92
+
93
+
94
+ def load_sample_pitch() -> tuple[str | None, str]:
95
+ if not _SAMPLE_AUDIO.is_file():
96
+ return (
97
+ None,
98
+ f"Sample clip missing at `{_SAMPLE_AUDIO}`. Run `uv run python libs/echocoach/tests/make_fixture.py`.",
99
+ )
100
+ return gr.update(value=str(_SAMPLE_AUDIO)), "Loaded 2s sample clip. Click **Analyze pitch** to test the pipeline."
101
+
102
+
103
+ def analyze_pitch(
104
+ audio_path: str | None,
105
+ language: str,
106
+ asr_preset: str,
107
+ speak_rewrite: bool,
108
+ ) -> tuple:
109
+ model_key = get_active_model_key()
110
+ load_error = ensure_model_loaded(model_key)
111
+ if load_error:
112
+ return _error_outputs(load_error)
113
+
114
+ if not audio_path:
115
+ return _error_outputs("Record or upload a pitch (up to 30 seconds), then click **Analyze pitch**.")
116
+
117
+ try:
118
+ result = run_echo_coach(
119
+ audio_path,
120
+ language=language,
121
+ asr_preset=asr_preset,
122
+ backend=get_backend(model_key),
123
+ speak_rewrite=speak_rewrite,
124
+ )
125
+ except Exception as exc: # noqa: BLE001 — surface pipeline errors in UI
126
+ return _error_outputs(f"EchoCoach failed: {exc}")
127
+
128
+ status = "Analysis complete."
129
+ if result.voiceout_warning:
130
+ status += f" VoiceOut: {result.voiceout_warning}"
131
+
132
+ return (
133
+ status,
134
+ result.transcript_html,
135
+ result.report_markdown,
136
+ result.filler_chart_path,
137
+ result.pace_chart_path,
138
+ result.voiceout_path,
139
+ f"Trace saved: `{result.trace_path}`",
140
+ result.trace,
141
+ )
142
+
143
+
144
+ def build_echo_coach_tab() -> None:
145
+ lang_choices = _config.language_choices()
146
+ asr_choices = _config.asr_choices()
147
+ default_lang = lang_choices[0][1] if lang_choices else "en"
148
+ default_asr = _config.asr_preset
149
+ mic_status = recording_backend_status()
150
+
151
+ gr.Markdown(
152
+ f"""
153
+ Record up to **{_config.max_seconds} seconds**, then get local feedback: transcript with **filler highlights**,
154
+ **pace score**, coach **rewrite**, and **VoiceOut** audio — all on-device.
155
+
156
+ - **ASR:** configurable (`voice_models.yaml`) — Cohere Transcribe 2B or Whisper.cpp
157
+ - **Coach:** text LLM preset (`ACTIVE_MODEL` / `ECHOCOACH_COACH_MODEL`)
158
+ - **TTS:** Piper VoiceOut (optional; install `echocoach[piper]`)
159
+
160
+ **Browser mic:** open **http://localhost:7860** in Chrome or Firefox (not Cursor's preview) and allow microphone access.
161
+ If the mic icon fails, use **Start / Stop recording** below or **Upload** a `.wav` / `.mp3`.
162
+ """
163
+ )
164
+
165
+ with gr.Row():
166
+ with gr.Column(scale=1):
167
+ record_status_md = gr.Markdown(mic_status)
168
+ with gr.Accordion("Record from this computer (recommended)", open=True):
169
+ gr.Markdown(
170
+ "Click **Start recording**, speak your pitch, then **Stop recording** when done. "
171
+ "The slider sets the maximum length (auto-stop safety cap)."
172
+ )
173
+ record_seconds = gr.Slider(
174
+ label="Max recording length (seconds)",
175
+ minimum=3,
176
+ maximum=_config.max_seconds,
177
+ value=min(30, _config.max_seconds),
178
+ step=1,
179
+ )
180
+ with gr.Row():
181
+ record_start_btn = gr.Button("Start recording", variant="secondary")
182
+ record_stop_btn = gr.Button("Stop recording", variant="stop", interactive=False)
183
+ sample_btn = gr.Button("Load sample clip", variant="secondary")
184
+ audio_in = gr.Audio(
185
+ label="Your pitch (browser mic or upload)",
186
+ sources=["upload", "microphone"],
187
+ type="filepath",
188
+ format="wav",
189
+ )
190
+ language = gr.Dropdown(
191
+ label="Language",
192
+ choices=lang_choices,
193
+ value=default_lang,
194
+ )
195
+ asr_preset = gr.Dropdown(
196
+ label="ASR preset",
197
+ choices=asr_choices,
198
+ value=default_asr,
199
+ )
200
+ speak_rewrite = gr.Checkbox(
201
+ label="VoiceOut speaks full rewrite (otherwise summary + tip)",
202
+ value=False,
203
+ )
204
+ analyze_btn = gr.Button("Analyze pitch", variant="primary")
205
+ status = gr.Textbox(label="Status", interactive=False, lines=3)
206
+ coach_status = gr.Markdown(model_status(get_active_model_key()))
207
+
208
+ with gr.Column(scale=2):
209
+ transcript_html = gr.HTML(label="Transcript")
210
+ report_md = gr.Markdown(label="Coach report")
211
+ with gr.Row():
212
+ filler_chart = gr.Image(label="Filler words", type="filepath")
213
+ pace_chart = gr.Image(label="Pace timeline", type="filepath")
214
+ voiceout = gr.Audio(label="VoiceOut", type="filepath")
215
+ trace_note = gr.Markdown()
216
+ trace_json = gr.JSON(label="Trace")
217
+
218
+ record_start_btn.click(
219
+ ui_start_recording,
220
+ inputs=[record_seconds],
221
+ outputs=[status, record_start_btn, record_stop_btn],
222
+ )
223
+ record_stop_btn.click(
224
+ ui_stop_recording,
225
+ outputs=[audio_in, status, record_start_btn, record_stop_btn],
226
+ ).then(
227
+ lambda: recording_backend_status(),
228
+ outputs=[record_status_md],
229
+ )
230
+
231
+ sample_btn.click(
232
+ load_sample_pitch,
233
+ outputs=[audio_in, status],
234
+ )
235
+
236
+ analyze_btn.click(
237
+ analyze_pitch,
238
+ inputs=[audio_in, language, asr_preset, speak_rewrite],
239
+ outputs=[
240
+ status,
241
+ transcript_html,
242
+ report_md,
243
+ filler_chart,
244
+ pace_chart,
245
+ voiceout,
246
+ trace_note,
247
+ trace_json,
248
+ ],
249
+ )
250
+
251
+
252
+ def echo_coach_allowed_paths() -> list[str]:
253
+ base = get_echo_coach_config()
254
+ paths: list[str] = []
255
+ if base.presets_path:
256
+ paths.append(str(base.presets_path.parent))
257
+ from echocoach.config import outputs_dir
258
+
259
+ paths.append(str(outputs_dir()))
260
+ paths.append(str(outputs_dir() / "recordings"))
261
+ if _SAMPLE_AUDIO.is_file():
262
+ paths.append(str(_SAMPLE_AUDIO.parent))
263
+ return paths
libs/echocoach/README.md ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ # echocoach
2
+
3
+ Local voice practice coach for the Build Small Hackathon.
4
+
5
+ - **ASR:** Cohere Transcribe 2B or Whisper.cpp (tiny/base)
6
+ - **Analysis:** filler detection, pace scoring, matplotlib charts
7
+ - **Coach:** text LLM via `inference` (default `minicpm5-1b`)
8
+ - **VoiceOut:** Piper TTS (optional extra)
9
+
10
+ Configure presets in repo-root `voice_models.yaml`.
libs/echocoach/pyproject.toml ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [project]
2
+ name = "echocoach"
3
+ version = "0.1.0"
4
+ description = "Local voice practice coach — ASR, pace/filler analysis, LLM coaching, TTS VoiceOut"
5
+ readme = "README.md"
6
+ authors = [
7
+ { name = "MSGhais", email = "msghais135@gmail.com" }
8
+ ]
9
+ requires-python = ">=3.12"
10
+ dependencies = [
11
+ "inference",
12
+ "agent",
13
+ "matplotlib>=3.9.0",
14
+ "numpy>=2.0.0",
15
+ "pyyaml>=6.0.2",
16
+ "soundfile>=0.12.0",
17
+ "sounddevice>=0.5.0",
18
+ "librosa>=0.10.0",
19
+ "transformers>=5.4.0",
20
+ "torch>=2.5.0",
21
+ "accelerate>=1.2.0",
22
+ "huggingface-hub>=0.27.0",
23
+ ]
24
+
25
+ [project.optional-dependencies]
26
+ whisper = [
27
+ "pywhispercpp>=1.3.0",
28
+ ]
29
+ piper = [
30
+ "piper-tts>=1.3.0",
31
+ ]
32
+
33
+ [tool.uv.sources]
34
+ inference = { workspace = true }
35
+ agent = { workspace = true }
36
+
37
+ [build-system]
38
+ requires = ["uv_build>=0.8.13,<0.9.0"]
39
+ build-backend = "uv_build"
libs/echocoach/src/echocoach/__init__.py ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ """EchoCoach — local voice practice coach."""
2
+
3
+ from echocoach.models import CoachFeedback, EchoCoachResult
4
+ from echocoach.pipeline import run_echo_coach
5
+
6
+ __all__ = ["CoachFeedback", "EchoCoachResult", "run_echo_coach"]
libs/echocoach/src/echocoach/analysis/__init__.py ADDED
File without changes
libs/echocoach/src/echocoach/analysis/charts.py ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Matplotlib charts for filler and pace visualization."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from pathlib import Path
6
+
7
+ import matplotlib
8
+
9
+ matplotlib.use("Agg")
10
+ import matplotlib.pyplot as plt # noqa: E402
11
+
12
+ from echocoach.models import FillerAnalysis, PaceAnalysis
13
+
14
+
15
+ def render_filler_chart(analysis: FillerAnalysis, out_path: Path) -> Path:
16
+ out_path.parent.mkdir(parents=True, exist_ok=True)
17
+ if not analysis.counts:
18
+ fig, ax = plt.subplots(figsize=(6, 3))
19
+ ax.bar(["(none)"], [0], color="#4a90d9")
20
+ ax.set_title("Filler words")
21
+ ax.set_ylabel("Count")
22
+ else:
23
+ labels = list(analysis.counts.keys())
24
+ values = [analysis.counts[k] for k in labels]
25
+ fig, ax = plt.subplots(figsize=(6, 3))
26
+ ax.bar(labels, values, color="#e67e22")
27
+ ax.set_title("Filler words")
28
+ ax.set_ylabel("Count")
29
+ plt.xticks(rotation=35, ha="right")
30
+ fig.tight_layout()
31
+ fig.savefig(out_path, dpi=120)
32
+ plt.close(fig)
33
+ return out_path
34
+
35
+
36
+ def render_pace_timeline(
37
+ transcript: str,
38
+ duration_seconds: float,
39
+ pace: PaceAnalysis,
40
+ out_path: Path,
41
+ *,
42
+ window_seconds: float = 10.0,
43
+ ) -> Path:
44
+ out_path.parent.mkdir(parents=True, exist_ok=True)
45
+ words = transcript.split()
46
+ if not words or duration_seconds <= 0:
47
+ fig, ax = plt.subplots(figsize=(6, 3))
48
+ ax.plot([0, max(duration_seconds, 1)], [0, 0], color="#4a90d9")
49
+ ax.set_title("Words per minute (rolling)")
50
+ ax.set_xlabel("Seconds")
51
+ ax.set_ylabel("WPM")
52
+ ax.axhspan(pace.target_low, pace.target_high, alpha=0.15, color="green", label="Target band")
53
+ ax.legend(loc="upper right")
54
+ else:
55
+ n_windows = max(1, int(duration_seconds / window_seconds) + (1 if duration_seconds % window_seconds else 0))
56
+ times: list[float] = []
57
+ wpms: list[float] = []
58
+ words_per_sec = len(words) / duration_seconds
59
+ for i in range(n_windows):
60
+ start = i * window_seconds
61
+ end = min((i + 1) * window_seconds, duration_seconds)
62
+ window_dur = end - start
63
+ if window_dur <= 0:
64
+ continue
65
+ approx_words = words_per_sec * window_dur
66
+ wpm = approx_words / (window_dur / 60.0)
67
+ times.append((start + end) / 2)
68
+ wpms.append(wpm)
69
+
70
+ fig, ax = plt.subplots(figsize=(6, 3))
71
+ ax.plot(times, wpms, marker="o", color="#4a90d9", linewidth=2)
72
+ ax.axhspan(pace.target_low, pace.target_high, alpha=0.15, color="green", label="Target band")
73
+ ax.set_title("Words per minute (by segment)")
74
+ ax.set_xlabel("Seconds")
75
+ ax.set_ylabel("WPM")
76
+ ax.legend(loc="upper right")
77
+
78
+ fig.tight_layout()
79
+ fig.savefig(out_path, dpi=120)
80
+ plt.close(fig)
81
+ return out_path
82
+
83
+
84
+ def build_charts(
85
+ transcript: str,
86
+ duration_seconds: float,
87
+ fillers: FillerAnalysis,
88
+ pace: PaceAnalysis,
89
+ output_dir: Path,
90
+ run_id: str,
91
+ ) -> tuple[Path, Path]:
92
+ filler_path = output_dir / f"{run_id}_fillers.png"
93
+ pace_path = output_dir / f"{run_id}_pace.png"
94
+ render_filler_chart(fillers, filler_path)
95
+ render_pace_timeline(transcript, duration_seconds, pace, pace_path)
96
+ return filler_path, pace_path
libs/echocoach/src/echocoach/analysis/fillers.py ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Filler word detection and HTML highlighting."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import re
6
+
7
+ from echocoach.models import FillerAnalysis, FillerSpan
8
+
9
+ DEFAULT_FILLERS = [
10
+ "um",
11
+ "uh",
12
+ "uhm",
13
+ "erm",
14
+ "like",
15
+ "you know",
16
+ "basically",
17
+ "actually",
18
+ "literally",
19
+ "sort of",
20
+ "kind of",
21
+ "i mean",
22
+ "right",
23
+ "okay so",
24
+ "so yeah",
25
+ ]
26
+
27
+ # Longer phrases first so "you know" wins over "you"
28
+ _SORTED_FILLERS = sorted(DEFAULT_FILLERS, key=len, reverse=True)
29
+ _PATTERN = re.compile(
30
+ r"\b(" + "|".join(re.escape(f) for f in _SORTED_FILLERS) + r")\b",
31
+ re.IGNORECASE,
32
+ )
33
+
34
+
35
+ def analyze_fillers(transcript: str, fillers: list[str] | None = None) -> FillerAnalysis:
36
+ if fillers is None:
37
+ pattern = _PATTERN
38
+ else:
39
+ ordered = sorted(fillers, key=len, reverse=True)
40
+ pattern = re.compile(
41
+ r"\b(" + "|".join(re.escape(f) for f in ordered) + r")\b",
42
+ re.IGNORECASE,
43
+ )
44
+
45
+ counts: dict[str, int] = {}
46
+ spans: list[FillerSpan] = []
47
+ for match in pattern.finditer(transcript):
48
+ word = match.group(1).lower()
49
+ counts[word] = counts.get(word, 0) + 1
50
+ spans.append(FillerSpan(start=match.start(), end=match.end(), word=word))
51
+
52
+ return FillerAnalysis(counts=counts, spans=spans, total=sum(counts.values()))
53
+
54
+
55
+ def highlight_fillers_html(transcript: str, analysis: FillerAnalysis) -> str:
56
+ if not analysis.spans:
57
+ safe = _escape_html(transcript)
58
+ return f'<p style="line-height:1.6;">{safe}</p>'
59
+
60
+ parts: list[str] = []
61
+ cursor = 0
62
+ for span in sorted(analysis.spans, key=lambda s: s.start):
63
+ if span.start < cursor:
64
+ continue
65
+ parts.append(_escape_html(transcript[cursor : span.start]))
66
+ parts.append(
67
+ f'<mark style="background:#ffe08a;padding:0 2px;border-radius:3px;">'
68
+ f"{_escape_html(transcript[span.start : span.end])}</mark>"
69
+ )
70
+ cursor = span.end
71
+ parts.append(_escape_html(transcript[cursor:]))
72
+ body = "".join(parts)
73
+ return f'<p style="line-height:1.6;">{body}</p>'
74
+
75
+
76
+ def _escape_html(text: str) -> str:
77
+ return (
78
+ text.replace("&", "&amp;")
79
+ .replace("<", "&lt;")
80
+ .replace(">", "&gt;")
81
+ .replace('"', "&quot;")
82
+ )
libs/echocoach/src/echocoach/analysis/pace.py ADDED
@@ -0,0 +1,49 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Speaking pace scoring."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from echocoach.audio_io import count_words
6
+ from echocoach.models import PaceAnalysis
7
+
8
+ TARGET_LOW = 120
9
+ TARGET_HIGH = 160
10
+
11
+
12
+ def analyze_pace(
13
+ transcript: str,
14
+ duration_seconds: float,
15
+ *,
16
+ target_low: int = TARGET_LOW,
17
+ target_high: int = TARGET_HIGH,
18
+ ) -> PaceAnalysis:
19
+ word_count = count_words(transcript)
20
+ if duration_seconds <= 0:
21
+ wpm = 0.0
22
+ else:
23
+ wpm = word_count / (duration_seconds / 60.0)
24
+
25
+ score, label = _score_wpm(wpm, target_low, target_high)
26
+ return PaceAnalysis(
27
+ word_count=word_count,
28
+ duration_seconds=duration_seconds,
29
+ wpm=round(wpm, 1),
30
+ score=score,
31
+ label=label,
32
+ target_low=target_low,
33
+ target_high=target_high,
34
+ )
35
+
36
+
37
+ def _score_wpm(wpm: float, low: int, high: int) -> tuple[int, str]:
38
+ if wpm <= 0:
39
+ return 0, "No speech detected"
40
+ if low <= wpm <= high:
41
+ return 100, "Ideal pace"
42
+ if wpm < low:
43
+ ratio = wpm / low
44
+ score = max(20, int(100 * ratio))
45
+ return score, "Too slow — pick up the energy"
46
+ # too fast
47
+ overshoot = (wpm - high) / high
48
+ score = max(20, int(100 * (1.0 - min(overshoot, 0.8))))
49
+ return score, "Too fast — pause and breathe"
libs/echocoach/src/echocoach/asr/__init__.py ADDED
File without changes
libs/echocoach/src/echocoach/asr/cohere.py ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Cohere Transcribe 2B ASR backend."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from echocoach.audio_io import TARGET_SAMPLE_RATE, load_audio_mono_16k
6
+ from echocoach.config import AsrPreset
7
+
8
+
9
+ class CohereAsrBackend:
10
+ def __init__(self, preset: AsrPreset) -> None:
11
+ self._preset = preset
12
+ self._processor = None
13
+ self._model = None
14
+
15
+ def _load(self) -> None:
16
+ if self._model is not None:
17
+ return
18
+ from transformers import AutoProcessor, CohereAsrForConditionalGeneration
19
+
20
+ model_id = self._preset.model_id or "CohereLabs/cohere-transcribe-03-2026"
21
+ self._processor = AutoProcessor.from_pretrained(model_id)
22
+ self._model = CohereAsrForConditionalGeneration.from_pretrained(
23
+ model_id,
24
+ device_map="auto",
25
+ )
26
+
27
+ def transcribe(self, audio_path: str, *, language: str) -> str:
28
+ self._load()
29
+ assert self._processor is not None
30
+ assert self._model is not None
31
+
32
+ audio, _ = load_audio_mono_16k(audio_path)
33
+ inputs = self._processor(
34
+ audio,
35
+ sampling_rate=TARGET_SAMPLE_RATE,
36
+ return_tensors="pt",
37
+ language=language,
38
+ )
39
+ audio_chunk_index = inputs.get("audio_chunk_index")
40
+ inputs = inputs.to(self._model.device, dtype=self._model.dtype)
41
+
42
+ outputs = self._model.generate(**inputs, max_new_tokens=512)
43
+ decoded = self._processor.decode(
44
+ outputs,
45
+ skip_special_tokens=True,
46
+ audio_chunk_index=audio_chunk_index,
47
+ language=language,
48
+ )
49
+ if isinstance(decoded, list):
50
+ return decoded[0].strip() if decoded else ""
51
+ return str(decoded).strip()
libs/echocoach/src/echocoach/asr/factory.py ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """ASR backend protocol and factory."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from typing import Protocol
6
+
7
+ from echocoach.config import AsrPreset, get_echo_coach_config
8
+
9
+
10
+ class AsrBackend(Protocol):
11
+ def transcribe(self, audio_path: str, *, language: str) -> str: ...
12
+
13
+
14
+ _asr_cache: dict[tuple, AsrBackend] = {}
15
+
16
+
17
+ def get_asr_backend(preset_key: str | None = None) -> AsrBackend:
18
+ config = get_echo_coach_config()
19
+ preset = config.get_asr(preset_key)
20
+ cache_key = (preset.key, preset.backend, preset.model_id, preset.model_size)
21
+ if cache_key in _asr_cache:
22
+ return _asr_cache[cache_key]
23
+
24
+ if preset.backend == "cohere":
25
+ from echocoach.asr.cohere import CohereAsrBackend
26
+
27
+ backend: AsrBackend = CohereAsrBackend(preset)
28
+ elif preset.backend == "whisper_cpp":
29
+ from echocoach.asr.whisper_cpp import WhisperCppBackend
30
+
31
+ backend = WhisperCppBackend(preset)
32
+ else:
33
+ raise ValueError(f"Unsupported ASR backend: {preset.backend}")
34
+
35
+ _asr_cache[cache_key] = backend
36
+ return backend
37
+
38
+
39
+ def reset_asr_backends() -> None:
40
+ _asr_cache.clear()
libs/echocoach/src/echocoach/asr/whisper_cpp.py ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Whisper.cpp ASR via pywhispercpp."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from echocoach.config import AsrPreset
6
+
7
+
8
+ class WhisperCppBackend:
9
+ def __init__(self, preset: AsrPreset) -> None:
10
+ self._preset = preset
11
+ self._model = None
12
+
13
+ def _load(self) -> None:
14
+ if self._model is not None:
15
+ return
16
+ try:
17
+ from pywhispercpp.model import Model
18
+ except ImportError as exc:
19
+ raise ImportError(
20
+ "Whisper.cpp backend requires pywhispercpp. "
21
+ "Install with: uv sync --package echocoach --extra whisper"
22
+ ) from exc
23
+
24
+ size = self._preset.model_size or "tiny"
25
+ self._model = Model(size, print_realtime=False, print_progress=False)
26
+
27
+ def transcribe(self, audio_path: str, *, language: str) -> str:
28
+ self._load()
29
+ assert self._model is not None
30
+
31
+ segments = self._model.transcribe(audio_path, language=language)
32
+ parts = [seg.text.strip() for seg in segments if getattr(seg, "text", "")]
33
+ return " ".join(parts).strip()
libs/echocoach/src/echocoach/audio_io.py ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import re
4
+ from pathlib import Path
5
+
6
+ import numpy as np
7
+
8
+ TARGET_SAMPLE_RATE = 16_000
9
+
10
+
11
+ def load_audio_mono_16k(path: str | Path) -> tuple[np.ndarray, float]:
12
+ """Load audio as mono float32 at 16 kHz; return (samples, duration_seconds)."""
13
+ import librosa
14
+
15
+ audio, _ = librosa.load(str(path), sr=TARGET_SAMPLE_RATE, mono=True)
16
+ duration = len(audio) / TARGET_SAMPLE_RATE if len(audio) else 0.0
17
+ return audio.astype(np.float32), duration
18
+
19
+
20
+ def clamp_duration(audio: np.ndarray, max_seconds: float) -> np.ndarray:
21
+ max_samples = int(max_seconds * TARGET_SAMPLE_RATE)
22
+ if len(audio) > max_samples:
23
+ return audio[:max_samples]
24
+ return audio
25
+
26
+
27
+ def write_wav_temp(audio: np.ndarray, directory: Path, stem: str = "clip") -> Path:
28
+ import soundfile as sf
29
+
30
+ directory.mkdir(parents=True, exist_ok=True)
31
+ out = directory / f"{stem}.wav"
32
+ sf.write(out, audio, TARGET_SAMPLE_RATE)
33
+ return out
34
+
35
+
36
+ def count_words(text: str) -> int:
37
+ tokens = re.findall(r"\b[\w']+\b", text, flags=re.UNICODE)
38
+ return len(tokens)
libs/echocoach/src/echocoach/coach.py ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """LLM coaching prompts and parsing."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from inference.base import InferenceBackend
6
+ from inference.response_clean import strip_reasoning_output
7
+
8
+ from echocoach.models import CoachFeedback, FillerAnalysis, PaceAnalysis
9
+ from echocoach.utils import extract_json
10
+
11
+ COACH_SYSTEM = """You are EchoCoach, a concise public-speaking coach for students and teachers.
12
+ Respond with a single JSON object only — no markdown fences, no extra text.
13
+
14
+ Required keys:
15
+ - summary: 1-2 sentence overall assessment
16
+ - filler_feedback: specific advice about filler words
17
+ - pace_feedback: specific advice about speaking pace
18
+ - rewrite: improved 2-4 sentence version of the pitch (same language as the transcript)
19
+ - one_tip: one actionable tip for the next attempt
20
+ """
21
+
22
+
23
+ def coach_user_prompt(
24
+ transcript: str,
25
+ fillers: FillerAnalysis,
26
+ pace: PaceAnalysis,
27
+ language: str,
28
+ ) -> str:
29
+ filler_list = ", ".join(f"{k} ({v})" for k, v in fillers.counts.items()) or "none"
30
+ return f"""Language: {language}
31
+ Duration: {pace.duration_seconds:.1f}s
32
+ Word count: {pace.word_count}
33
+ Pace: {pace.wpm} WPM (target {pace.target_low}-{pace.target_high}) — {pace.label} (score {pace.score}/100)
34
+ Filler words: {fillers.total} total — {filler_list}
35
+
36
+ Transcript:
37
+ {transcript}
38
+ """
39
+
40
+
41
+ def run_coach(
42
+ backend: InferenceBackend,
43
+ transcript: str,
44
+ fillers: FillerAnalysis,
45
+ pace: PaceAnalysis,
46
+ language: str,
47
+ ) -> tuple[CoachFeedback, str, str]:
48
+ user_content = coach_user_prompt(transcript, fillers, pace, language)
49
+ messages = [
50
+ {"role": "system", "content": COACH_SYSTEM},
51
+ {"role": "user", "content": user_content},
52
+ ]
53
+ raw = backend.chat(messages, max_tokens=768, temperature=0.4)
54
+ raw = strip_reasoning_output(raw)
55
+ feedback = parse_coach_response(raw)
56
+ return feedback, COACH_SYSTEM, raw
57
+
58
+
59
+ def parse_coach_response(raw: str) -> CoachFeedback:
60
+ data = extract_json(raw)
61
+ return CoachFeedback(
62
+ summary=str(data.get("summary", "")).strip(),
63
+ filler_feedback=str(data.get("filler_feedback", "")).strip(),
64
+ pace_feedback=str(data.get("pace_feedback", "")).strip(),
65
+ rewrite=str(data.get("rewrite", "")).strip(),
66
+ one_tip=str(data.get("one_tip", "")).strip(),
67
+ )
68
+
69
+
70
+ def format_report_markdown(
71
+ coach: CoachFeedback,
72
+ fillers: FillerAnalysis,
73
+ pace: PaceAnalysis,
74
+ ) -> str:
75
+ filler_lines = (
76
+ "\n".join(f"- **{word}**: {count}" for word, count in fillers.counts.items())
77
+ or "- None detected"
78
+ )
79
+ return f"""## Pace
80
+
81
+ - **Score:** {pace.score}/100 — {pace.label}
82
+ - **WPM:** {pace.wpm} (target {pace.target_low}–{pace.target_high})
83
+ - **Duration:** {pace.duration_seconds:.1f}s · **Words:** {pace.word_count}
84
+
85
+ ## Fillers ({fillers.total})
86
+
87
+ {filler_lines}
88
+
89
+ ## Coach summary
90
+
91
+ {coach.summary}
92
+
93
+ ### Filler feedback
94
+
95
+ {coach.filler_feedback}
96
+
97
+ ### Pace feedback
98
+
99
+ {coach.pace_feedback}
100
+
101
+ ### Suggested rewrite
102
+
103
+ {coach.rewrite}
104
+
105
+ ### One tip
106
+
107
+ {coach.one_tip}
108
+ """
libs/echocoach/src/echocoach/config.py ADDED
@@ -0,0 +1,228 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ import os
5
+ import re
6
+ from dataclasses import dataclass, replace
7
+ from pathlib import Path
8
+ from typing import Any, Literal
9
+
10
+ AsrBackendName = Literal["cohere", "whisper_cpp"]
11
+ TtsBackendName = Literal["piper"]
12
+
13
+
14
+ @dataclass(frozen=True)
15
+ class LanguageOption:
16
+ code: str
17
+ label: str
18
+
19
+
20
+ @dataclass(frozen=True)
21
+ class AsrPreset:
22
+ key: str
23
+ label: str
24
+ backend: AsrBackendName
25
+ model_id: str | None = None
26
+ model_size: str | None = None
27
+
28
+
29
+ @dataclass(frozen=True)
30
+ class TtsPreset:
31
+ key: str
32
+ label: str
33
+ backend: TtsBackendName
34
+ voices: dict[str, str]
35
+ fallback_voice: str
36
+
37
+
38
+ @dataclass(frozen=True)
39
+ class EchoCoachConfig:
40
+ asr_preset: str
41
+ tts_preset: str
42
+ coach_model: str
43
+ max_seconds: int
44
+ languages: list[LanguageOption]
45
+ asr_presets: dict[str, AsrPreset]
46
+ tts_presets: dict[str, TtsPreset]
47
+ presets_path: Path | None = None
48
+
49
+ def get_asr(self, key: str | None = None) -> AsrPreset:
50
+ preset_key = key or self.asr_preset
51
+ if preset_key not in self.asr_presets:
52
+ known = ", ".join(sorted(self.asr_presets))
53
+ raise KeyError(f"Unknown ASR preset {preset_key!r}. Known: {known}")
54
+ return self.asr_presets[preset_key]
55
+
56
+ def get_tts(self, key: str | None = None) -> TtsPreset:
57
+ preset_key = key or self.tts_preset
58
+ if preset_key not in self.tts_presets:
59
+ known = ", ".join(sorted(self.tts_presets))
60
+ raise KeyError(f"Unknown TTS preset {preset_key!r}. Known: {known}")
61
+ return self.tts_presets[preset_key]
62
+
63
+ def asr_choices(self) -> list[tuple[str, str]]:
64
+ return [(p.label, p.key) for p in self.asr_presets.values()]
65
+
66
+ def language_choices(self) -> list[tuple[str, str]]:
67
+ return [(lang.label, lang.code) for lang in self.languages]
68
+
69
+
70
+ def _find_voice_presets_path() -> Path | None:
71
+ env_path = os.environ.get("VOICE_PRESETS_PATH")
72
+ if env_path:
73
+ path = Path(env_path)
74
+ if path.is_file():
75
+ return path.resolve()
76
+
77
+ for base in (Path.cwd(), *Path.cwd().parents):
78
+ candidate = base / "voice_models.yaml"
79
+ if candidate.is_file():
80
+ return candidate.resolve()
81
+ return None
82
+
83
+
84
+ def _builtin_config() -> EchoCoachConfig:
85
+ langs = [
86
+ LanguageOption("en", "English"),
87
+ LanguageOption("fr", "French"),
88
+ LanguageOption("de", "German"),
89
+ ]
90
+ asr = {
91
+ "whisper-cpp-tiny": AsrPreset(
92
+ key="whisper-cpp-tiny",
93
+ label="Whisper.cpp tiny",
94
+ backend="whisper_cpp",
95
+ model_size="tiny",
96
+ ),
97
+ }
98
+ tts = {
99
+ "piper-multilingual": TtsPreset(
100
+ key="piper-multilingual",
101
+ label="Piper TTS",
102
+ backend="piper",
103
+ voices={"en": "en_US-lessac-medium"},
104
+ fallback_voice="en_US-lessac-medium",
105
+ ),
106
+ }
107
+ return EchoCoachConfig(
108
+ asr_preset="whisper-cpp-tiny",
109
+ tts_preset="piper-multilingual",
110
+ coach_model="minicpm5-1b",
111
+ max_seconds=30,
112
+ languages=langs,
113
+ asr_presets=asr,
114
+ tts_presets=tts,
115
+ )
116
+
117
+
118
+ def _parse_asr_entry(key: str, raw: dict[str, Any]) -> AsrPreset:
119
+ backend = str(raw.get("backend", "whisper_cpp"))
120
+ if backend not in ("cohere", "whisper_cpp"):
121
+ raise ValueError(f"ASR preset {key!r}: backend must be cohere or whisper_cpp")
122
+ return AsrPreset(
123
+ key=key,
124
+ label=str(raw.get("label", key)),
125
+ backend=backend, # type: ignore[arg-type]
126
+ model_id=raw.get("model_id"),
127
+ model_size=raw.get("model_size"),
128
+ )
129
+
130
+
131
+ def _parse_tts_entry(key: str, raw: dict[str, Any]) -> TtsPreset:
132
+ backend = str(raw.get("backend", "piper"))
133
+ if backend != "piper":
134
+ raise ValueError(f"TTS preset {key!r}: only piper backend is supported in MVP")
135
+ voices = raw.get("voices") or {}
136
+ if not isinstance(voices, dict) or not voices:
137
+ raise ValueError(f"TTS preset {key!r}: voices mapping is required")
138
+ return TtsPreset(
139
+ key=key,
140
+ label=str(raw.get("label", key)),
141
+ backend="piper",
142
+ voices={str(k): str(v) for k, v in voices.items()},
143
+ fallback_voice=str(raw.get("fallback_voice", "en_US-lessac-medium")),
144
+ )
145
+
146
+
147
+ def load_echo_coach_config() -> EchoCoachConfig:
148
+ presets_path = _find_voice_presets_path()
149
+ if presets_path is None:
150
+ config = _builtin_config()
151
+ else:
152
+ try:
153
+ import yaml
154
+ except ImportError as exc:
155
+ raise ImportError(
156
+ "Loading voice_models.yaml requires PyYAML. Install with: uv sync"
157
+ ) from exc
158
+
159
+ data = yaml.safe_load(presets_path.read_text()) or {}
160
+ defaults = data.get("defaults", {})
161
+ raw_langs = data.get("languages", [])
162
+ raw_asr = data.get("asr", {})
163
+ raw_tts = data.get("tts", {})
164
+
165
+ languages = [
166
+ LanguageOption(code=str(item["code"]), label=str(item["label"]))
167
+ for item in raw_langs
168
+ ]
169
+ asr_presets = {
170
+ key: _parse_asr_entry(key, value) for key, value in raw_asr.items()
171
+ }
172
+ tts_presets = {
173
+ key: _parse_tts_entry(key, value) for key, value in raw_tts.items()
174
+ }
175
+
176
+ asr_default = defaults.get("asr_preset", "whisper-cpp-tiny")
177
+ tts_default = defaults.get("tts_preset", "piper-multilingual")
178
+ if asr_default not in asr_presets:
179
+ asr_default = next(iter(asr_presets))
180
+ if tts_default not in tts_presets:
181
+ tts_default = next(iter(tts_presets))
182
+
183
+ config = EchoCoachConfig(
184
+ asr_preset=asr_default,
185
+ tts_preset=tts_default,
186
+ coach_model=str(defaults.get("coach_model", "minicpm5-1b")),
187
+ max_seconds=int(defaults.get("max_seconds", 30)),
188
+ languages=languages,
189
+ asr_presets=asr_presets,
190
+ tts_presets=tts_presets,
191
+ presets_path=presets_path,
192
+ )
193
+
194
+ updates: dict[str, Any] = {}
195
+ if os.environ.get("ECHOCOACH_ASR_PRESET"):
196
+ updates["asr_preset"] = os.environ["ECHOCOACH_ASR_PRESET"]
197
+ if os.environ.get("ECHOCOACH_TTS_PRESET"):
198
+ updates["tts_preset"] = os.environ["ECHOCOACH_TTS_PRESET"]
199
+ if os.environ.get("ECHOCOACH_COACH_MODEL"):
200
+ updates["coach_model"] = os.environ["ECHOCOACH_COACH_MODEL"]
201
+ if os.environ.get("ECHOCOACH_MAX_SECONDS"):
202
+ updates["max_seconds"] = int(os.environ["ECHOCOACH_MAX_SECONDS"])
203
+
204
+ return replace(config, **updates) if updates else config
205
+
206
+
207
+ _config: EchoCoachConfig | None = None
208
+
209
+
210
+ def get_echo_coach_config(reload: bool = False) -> EchoCoachConfig:
211
+ global _config
212
+ if _config is None or reload:
213
+ _config = load_echo_coach_config()
214
+ return _config
215
+
216
+
217
+ def outputs_dir() -> Path:
218
+ env = os.environ.get("AGENT_OUTPUTS_DIR")
219
+ if env:
220
+ return Path(env)
221
+ for base in (Path.cwd(), *Path.cwd().parents):
222
+ if (base / "voice_models.yaml").is_file() or (base / "pyproject.toml").is_file():
223
+ path = base / "outputs" / "echocoach"
224
+ path.mkdir(parents=True, exist_ok=True)
225
+ return path
226
+ path = Path("/tmp/echocoach_outputs")
227
+ path.mkdir(parents=True, exist_ok=True)
228
+ return path
libs/echocoach/src/echocoach/models.py ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ import re
5
+ from dataclasses import dataclass, field
6
+ from typing import Any
7
+
8
+
9
+ @dataclass(frozen=True)
10
+ class FillerSpan:
11
+ start: int
12
+ end: int
13
+ word: str
14
+
15
+
16
+ @dataclass(frozen=True)
17
+ class FillerAnalysis:
18
+ counts: dict[str, int]
19
+ spans: list[FillerSpan]
20
+ total: int
21
+
22
+
23
+ @dataclass(frozen=True)
24
+ class PaceAnalysis:
25
+ word_count: int
26
+ duration_seconds: float
27
+ wpm: float
28
+ score: int
29
+ label: str
30
+ target_low: int = 120
31
+ target_high: int = 160
32
+
33
+
34
+ @dataclass(frozen=True)
35
+ class CoachFeedback:
36
+ summary: str
37
+ filler_feedback: str
38
+ pace_feedback: str
39
+ rewrite: str
40
+ one_tip: str
41
+
42
+
43
+ @dataclass
44
+ class EchoCoachResult:
45
+ transcript: str
46
+ transcript_html: str
47
+ language: str
48
+ duration_seconds: float
49
+ fillers: FillerAnalysis
50
+ pace: PaceAnalysis
51
+ coach: CoachFeedback
52
+ report_markdown: str
53
+ filler_chart_path: str | None
54
+ pace_chart_path: str | None
55
+ voiceout_path: str | None
56
+ voiceout_warning: str | None
57
+ trace_path: str
58
+ trace: dict[str, Any] = field(default_factory=dict)
libs/echocoach/src/echocoach/pipeline.py ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """End-to-end EchoCoach pipeline."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import uuid
6
+ from pathlib import Path
7
+ from typing import Any
8
+
9
+ from agent.trace import TraceRecorder
10
+ from inference.base import InferenceBackend
11
+
12
+ from echocoach.analysis.charts import build_charts
13
+ from echocoach.analysis.fillers import analyze_fillers, highlight_fillers_html
14
+ from echocoach.analysis.pace import analyze_pace
15
+ from echocoach.asr.factory import get_asr_backend
16
+ from echocoach.audio_io import clamp_duration, load_audio_mono_16k, write_wav_temp
17
+ from echocoach.coach import format_report_markdown, run_coach
18
+ from echocoach.config import get_echo_coach_config, outputs_dir
19
+ from echocoach.models import EchoCoachResult
20
+ from echocoach.tts.piper import get_tts_backend
21
+
22
+
23
+ def run_echo_coach(
24
+ audio_path: str,
25
+ *,
26
+ language: str = "en",
27
+ asr_preset: str | None = None,
28
+ tts_preset: str | None = None,
29
+ coach_model: str | None = None,
30
+ backend: InferenceBackend,
31
+ speak_rewrite: bool = False,
32
+ ) -> EchoCoachResult:
33
+ if not audio_path:
34
+ raise ValueError("No audio recording provided.")
35
+
36
+ config = get_echo_coach_config()
37
+ asr_key = asr_preset or config.asr_preset
38
+ tts_key = tts_preset or config.tts_preset
39
+ model_key = coach_model or config.coach_model
40
+ run_id = uuid.uuid4().hex[:12]
41
+ out_base = outputs_dir()
42
+
43
+ trace = TraceRecorder(
44
+ skill="echo-coach",
45
+ model=model_key,
46
+ user_input={
47
+ "language": language,
48
+ "asr_preset": asr_key,
49
+ "tts_preset": tts_key,
50
+ "audio_path": audio_path,
51
+ "speak_rewrite": speak_rewrite,
52
+ },
53
+ run_id=run_id,
54
+ )
55
+
56
+ audio, duration = load_audio_mono_16k(audio_path)
57
+ audio = clamp_duration(audio, config.max_seconds)
58
+ duration = len(audio) / 16_000
59
+ clipped_path = write_wav_temp(audio, out_base / "clips", stem=run_id)
60
+
61
+ trace.log_note("audio_loaded", duration_seconds=duration, path=str(clipped_path))
62
+
63
+ asr = get_asr_backend(asr_key)
64
+ transcript = asr.transcribe(str(clipped_path), language=language)
65
+ trace.log_note("asr_complete", preset=asr_key, chars=len(transcript))
66
+
67
+ fillers = analyze_fillers(transcript)
68
+ pace = analyze_pace(transcript, duration)
69
+ transcript_html = highlight_fillers_html(transcript, fillers)
70
+
71
+ filler_chart, pace_chart = build_charts(
72
+ transcript,
73
+ duration,
74
+ fillers,
75
+ pace,
76
+ out_base / "charts",
77
+ run_id,
78
+ )
79
+
80
+ coach_feedback, system_prompt, coach_raw = run_coach(
81
+ backend,
82
+ transcript,
83
+ fillers,
84
+ pace,
85
+ language,
86
+ )
87
+ trace.log_llm(system_prompt + "\n\n" + coach_user_for_trace(transcript, fillers, pace, language), coach_raw)
88
+
89
+ report = format_report_markdown(coach_feedback, fillers, pace)
90
+
91
+ voice_text = coach_feedback.rewrite if speak_rewrite else (
92
+ f"{coach_feedback.summary} {coach_feedback.one_tip}".strip()
93
+ )
94
+ tts = get_tts_backend(tts_key)
95
+ voiceout_path, voiceout_warning = tts.synthesize(
96
+ voice_text,
97
+ language=language,
98
+ out_dir=out_base / "voiceout",
99
+ )
100
+ if voiceout_path:
101
+ trace.set_artifact(voiceout_path)
102
+
103
+ trace_path = trace.save()
104
+ trace_dict: dict[str, Any] = trace.to_dict()
105
+
106
+ return EchoCoachResult(
107
+ transcript=transcript,
108
+ transcript_html=transcript_html,
109
+ language=language,
110
+ duration_seconds=duration,
111
+ fillers=fillers,
112
+ pace=pace,
113
+ coach=coach_feedback,
114
+ report_markdown=report,
115
+ filler_chart_path=str(filler_chart),
116
+ pace_chart_path=str(pace_chart),
117
+ voiceout_path=voiceout_path,
118
+ voiceout_warning=voiceout_warning,
119
+ trace_path=str(trace_path),
120
+ trace=trace_dict,
121
+ )
122
+
123
+
124
+ def coach_user_for_trace(transcript: str, fillers, pace, language: str) -> str:
125
+ from echocoach.coach import coach_user_prompt
126
+
127
+ return coach_user_prompt(transcript, fillers, pace, language)
libs/echocoach/src/echocoach/recording.py ADDED
@@ -0,0 +1,348 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Record audio from the server's local microphone (bypasses browser getUserMedia)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import os
6
+ import shutil
7
+ import signal
8
+ import subprocess
9
+ import threading
10
+ import time
11
+ import uuid
12
+ from dataclasses import dataclass
13
+ from pathlib import Path
14
+ from typing import Literal
15
+
16
+ from echocoach.audio_io import TARGET_SAMPLE_RATE, load_audio_mono_16k
17
+ from echocoach.config import get_echo_coach_config, outputs_dir
18
+
19
+ CaptureBackend = Literal["pw-record", "sounddevice", "arecord"]
20
+ SILENT_RMS_THRESHOLD = 0.002
21
+
22
+
23
+ class ServerRecordingError(RuntimeError):
24
+ """Raised when server-side capture is unavailable or fails."""
25
+
26
+
27
+ @dataclass
28
+ class _RecordingSession:
29
+ process: subprocess.Popen[bytes]
30
+ out_path: Path
31
+ backend: CaptureBackend
32
+ max_seconds: int
33
+ started_at: float
34
+ watchdog: threading.Timer | None = None
35
+
36
+
37
+ _session: _RecordingSession | None = None
38
+ _last_recording_path: Path | None = None
39
+ _session_lock = threading.Lock()
40
+
41
+
42
+ def _capture_device() -> str | None:
43
+ device = os.environ.get("ECHOCOACH_CAPTURE_DEVICE", "").strip()
44
+ return device or None
45
+
46
+
47
+ def _pw_record_available() -> bool:
48
+ return shutil.which("pw-record") is not None
49
+
50
+
51
+ def _sounddevice_available() -> bool:
52
+ try:
53
+ import sounddevice as sd # noqa: PLC0415
54
+ except (ImportError, OSError):
55
+ return False
56
+ try:
57
+ sd.query_devices()
58
+ except Exception:
59
+ return False
60
+ return True
61
+
62
+
63
+ def _arecord_available() -> bool:
64
+ if shutil.which("arecord") is None:
65
+ return False
66
+ try:
67
+ result = subprocess.run(
68
+ ["arecord", "-l"],
69
+ capture_output=True,
70
+ text=True,
71
+ timeout=3,
72
+ check=False,
73
+ )
74
+ except (OSError, subprocess.TimeoutExpired):
75
+ return False
76
+ output = f"{result.stdout}\n{result.stderr}".lower()
77
+ return "card" in output
78
+
79
+
80
+ def select_capture_backend() -> CaptureBackend | None:
81
+ """Pick the best capture tool for this machine."""
82
+ if _pw_record_available():
83
+ return "pw-record"
84
+ if _sounddevice_available():
85
+ return "sounddevice"
86
+ if _arecord_available():
87
+ return "arecord"
88
+ return None
89
+
90
+
91
+ def recording_backend_status() -> str:
92
+ backend = select_capture_backend()
93
+ device = _capture_device()
94
+ if backend == "pw-record":
95
+ note = "PipeWire pw-record"
96
+ elif backend == "sounddevice":
97
+ note = "sounddevice / PortAudio"
98
+ elif backend == "arecord":
99
+ note = "ALSA arecord"
100
+ else:
101
+ note = None
102
+
103
+ if note:
104
+ extra = f" (device: `{device}`)" if device else ""
105
+ return f"Server microphone: ready ({note}{extra}). Click **Start recording**, speak, then **Stop recording**."
106
+
107
+ hints: list[str] = []
108
+ if not _pw_record_available() and not _arecord_available():
109
+ hints.append("install PipeWire (`pw-record`) or ALSA utils (`arecord`)")
110
+ elif not _arecord_available():
111
+ hints.append("enable a microphone in system sound settings")
112
+ if not _sounddevice_available():
113
+ hints.append(
114
+ "optional: `sudo apt install libportaudio2` for sounddevice fallback"
115
+ )
116
+ hint = "; ".join(hints) if hints else "no capture backend available"
117
+ return (
118
+ f"Server microphone: unavailable — {hint}. "
119
+ "Use **Upload** or open **http://localhost:7860** in Chrome/Firefox for browser mic."
120
+ )
121
+
122
+
123
+ def is_recording_active() -> bool:
124
+ with _session_lock:
125
+ session = _session
126
+ return session is not None and session.process.poll() is None
127
+
128
+
129
+ def recording_elapsed_seconds() -> float:
130
+ with _session_lock:
131
+ session = _session
132
+ if session is None:
133
+ return 0.0
134
+ return max(0.0, time.monotonic() - session.started_at)
135
+
136
+
137
+ def start_server_recording(max_seconds: int | None = None) -> None:
138
+ """Begin an open-ended capture; call stop_server_recording() to finish."""
139
+ global _session, _last_recording_path
140
+
141
+ config = get_echo_coach_config()
142
+ seconds = max_seconds if max_seconds is not None else config.max_seconds
143
+ if seconds <= 0:
144
+ raise ServerRecordingError("Recording duration must be positive.")
145
+
146
+ backend = select_capture_backend()
147
+ if backend is None:
148
+ raise ServerRecordingError(recording_backend_status())
149
+
150
+ out_dir = outputs_dir() / "recordings"
151
+ out_dir.mkdir(parents=True, exist_ok=True)
152
+ out_path = out_dir / f"server_{uuid.uuid4().hex[:8]}.wav"
153
+
154
+ with _session_lock:
155
+ if _session is not None and _session.process.poll() is None:
156
+ raise ServerRecordingError("Already recording. Click **Stop recording** first.")
157
+
158
+ _last_recording_path = None
159
+ process = _spawn_capture_process(backend, out_path)
160
+ watchdog = threading.Timer(seconds, _auto_stop_recording)
161
+ watchdog.daemon = True
162
+ watchdog.start()
163
+ _session = _RecordingSession(
164
+ process=process,
165
+ out_path=out_path,
166
+ backend=backend,
167
+ max_seconds=seconds,
168
+ started_at=time.monotonic(),
169
+ watchdog=watchdog,
170
+ )
171
+
172
+
173
+ def stop_server_recording() -> Path:
174
+ """Stop the active capture and return the WAV path."""
175
+ global _last_recording_path
176
+
177
+ with _session_lock:
178
+ session = _session
179
+ if session is None or session.process.poll() is not None:
180
+ if _last_recording_path is not None and _last_recording_path.is_file():
181
+ path = _last_recording_path
182
+ _last_recording_path = None
183
+ return path
184
+ raise ServerRecordingError("Not recording. Click **Start recording** first.")
185
+ return _finalize_session(session)
186
+
187
+
188
+ def _auto_stop_recording() -> None:
189
+ with _session_lock:
190
+ session = _session
191
+ if session is None or session.process.poll() is not None:
192
+ return
193
+ try:
194
+ _finalize_session(session)
195
+ except ServerRecordingError:
196
+ pass
197
+
198
+
199
+ def _finalize_session(session: _RecordingSession) -> Path:
200
+ global _session, _last_recording_path
201
+
202
+ if session.watchdog is not None:
203
+ session.watchdog.cancel()
204
+
205
+ process = session.process
206
+ if process.poll() is None:
207
+ process.send_signal(signal.SIGINT)
208
+ try:
209
+ process.wait(timeout=5)
210
+ except subprocess.TimeoutExpired:
211
+ process.kill()
212
+ process.wait(timeout=2)
213
+
214
+ out_path = session.out_path
215
+ if out_path.is_file() and out_path.stat().st_size > 44:
216
+ _session = None
217
+ _last_recording_path = out_path
218
+ return out_path
219
+
220
+ ok_codes = {
221
+ 0,
222
+ 1, # pw-record often exits 1 after SIGINT even with a valid WAV
223
+ -signal.SIGINT,
224
+ 128 + signal.SIGINT,
225
+ -signal.SIGTERM,
226
+ 128 + signal.SIGTERM,
227
+ }
228
+ if process.returncode not in ok_codes:
229
+ detail = ""
230
+ if process.stderr:
231
+ detail = process.stderr.read().decode("utf-8", errors="replace").strip()
232
+ msg = f"Capture stopped with exit code {process.returncode}"
233
+ if detail:
234
+ msg = f"{msg}: {detail}"
235
+ _session = None
236
+ _last_recording_path = None
237
+ raise ServerRecordingError(msg)
238
+
239
+ if not out_path.is_file() or out_path.stat().st_size == 0:
240
+ _session = None
241
+ _last_recording_path = None
242
+ raise ServerRecordingError("Recording finished but produced an empty file.")
243
+
244
+ _session = None
245
+ _last_recording_path = out_path
246
+ return out_path
247
+
248
+
249
+ def analyze_recording_levels(path: str | Path) -> tuple[float, float, float]:
250
+ audio, duration = load_audio_mono_16k(path)
251
+ if len(audio) == 0:
252
+ return 0.0, 0.0, duration
253
+ import numpy as np
254
+
255
+ rms = float(np.sqrt(np.mean(np.square(audio))))
256
+ peak = float(np.max(np.abs(audio)))
257
+ return rms, peak, duration
258
+
259
+
260
+ def recording_level_warning(path: str | Path) -> str | None:
261
+ rms, peak, duration = analyze_recording_levels(path)
262
+ if duration < 0.2:
263
+ return "Clip is very short — try recording a bit longer."
264
+ if rms < SILENT_RMS_THRESHOLD and peak < SILENT_RMS_THRESHOLD * 4:
265
+ return (
266
+ "Recording looks silent. Check system mic input/mute, pick the right input device "
267
+ "(set `ECHOCOACH_CAPTURE_DEVICE`), or use **Upload**."
268
+ )
269
+ return None
270
+
271
+
272
+ def record_server_wav(
273
+ max_seconds: int | None = None,
274
+ *,
275
+ sample_rate: int = TARGET_SAMPLE_RATE,
276
+ ) -> Path:
277
+ """Fixed-length capture (used in tests and scripts)."""
278
+ start_server_recording(max_seconds)
279
+ time.sleep(max_seconds if max_seconds is not None else get_echo_coach_config().max_seconds)
280
+ return stop_server_recording()
281
+
282
+
283
+ def _spawn_capture_process(backend: CaptureBackend, out_path: Path) -> subprocess.Popen[bytes]:
284
+ device = _capture_device()
285
+ if backend == "pw-record":
286
+ cmd = [
287
+ "pw-record",
288
+ "--media-category",
289
+ "Capture",
290
+ "--media-role",
291
+ "Speech",
292
+ "--rate",
293
+ str(TARGET_SAMPLE_RATE),
294
+ "--channels",
295
+ "1",
296
+ "--format",
297
+ "s16",
298
+ ]
299
+ if device:
300
+ cmd.extend(["--target", device])
301
+ cmd.append(str(out_path))
302
+ elif backend == "arecord":
303
+ cmd = [
304
+ "arecord",
305
+ "-q",
306
+ "-f",
307
+ "S16_LE",
308
+ "-r",
309
+ str(TARGET_SAMPLE_RATE),
310
+ "-c",
311
+ "1",
312
+ ]
313
+ if device:
314
+ cmd.extend(["-D", device])
315
+ else:
316
+ cmd.extend(["-D", "pipewire"])
317
+ cmd.append(str(out_path))
318
+ else:
319
+ raise ServerRecordingError(
320
+ "sounddevice does not support open-ended capture yet; install `pw-record` or use arecord."
321
+ )
322
+
323
+ try:
324
+ return subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
325
+ except OSError as exc:
326
+ raise ServerRecordingError(f"Failed to start {backend}: {exc}") from exc
327
+
328
+
329
+ def _record_sounddevice(out_path: Path, seconds: int, sample_rate: int) -> None:
330
+ import numpy as np
331
+ import sounddevice as sd
332
+ import soundfile as sf
333
+
334
+ frames = int(seconds * sample_rate)
335
+ device = _capture_device()
336
+ try:
337
+ recording = sd.rec(
338
+ frames,
339
+ samplerate=sample_rate,
340
+ channels=1,
341
+ dtype="float32",
342
+ device=device,
343
+ )
344
+ sd.wait()
345
+ except Exception as exc: # noqa: BLE001 — surface device errors to UI
346
+ raise ServerRecordingError(f"sounddevice capture failed: {exc}") from exc
347
+
348
+ sf.write(out_path, np.squeeze(recording), sample_rate)
libs/echocoach/src/echocoach/tts/__init__.py ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ from echocoach.tts.piper import PiperTtsBackend, get_tts_backend
2
+
3
+ __all__ = ["PiperTtsBackend", "get_tts_backend"]
libs/echocoach/src/echocoach/tts/piper.py ADDED
@@ -0,0 +1,123 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """TTS VoiceOut backends."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import uuid
6
+ from pathlib import Path
7
+ from typing import Protocol
8
+
9
+ from echocoach.config import TtsPreset, get_echo_coach_config, outputs_dir
10
+
11
+
12
+ class TtsBackend(Protocol):
13
+ def synthesize(self, text: str, *, language: str, out_dir: Path | None = None) -> tuple[str | None, str | None]: ...
14
+
15
+
16
+ _tts_cache: dict[str, "PiperTtsBackend"] = {}
17
+
18
+
19
+ class PiperTtsBackend:
20
+ def __init__(self, preset: TtsPreset) -> None:
21
+ self._preset = preset
22
+ self._voices: dict[str, object] = {}
23
+
24
+ def _voice_name(self, language: str) -> tuple[str, str | None]:
25
+ voice = self._preset.voices.get(language)
26
+ warning = None
27
+ if not voice:
28
+ voice = self._preset.fallback_voice
29
+ warning = f"No Piper voice for {language!r}; using {voice}."
30
+ return voice, warning
31
+
32
+ def _load_voice(self, voice_name: str):
33
+ if voice_name in self._voices:
34
+ return self._voices[voice_name]
35
+ try:
36
+ from piper import PiperVoice
37
+ except ImportError as exc:
38
+ raise ImportError(
39
+ "Piper TTS requires piper-tts. "
40
+ "Install with: uv sync --package echocoach --extra piper"
41
+ ) from exc
42
+
43
+ onnx_path = self._resolve_voice_path(voice_name)
44
+ voice = PiperVoice.load(str(onnx_path))
45
+ self._voices[voice_name] = voice
46
+ return voice
47
+
48
+ def _resolve_voice_path(self, voice_name: str) -> Path:
49
+ env_dir = __import__("os").environ.get("PIPER_VOICES_DIR")
50
+ candidates: list[Path] = []
51
+ if env_dir:
52
+ candidates.append(Path(env_dir) / f"{voice_name}.onnx")
53
+ home = Path.home() / ".local" / "share" / "piper" / "voices"
54
+ candidates.append(home / f"{voice_name}.onnx")
55
+ candidates.append(Path.cwd() / "models" / "piper" / f"{voice_name}.onnx")
56
+
57
+ for path in candidates:
58
+ if path.is_file():
59
+ return path
60
+
61
+ try:
62
+ from piper.download_voices import download_voice
63
+ except ImportError:
64
+ download_voice = None
65
+
66
+ if download_voice is not None:
67
+ try:
68
+ downloaded = download_voice(voice_name)
69
+ return Path(downloaded)
70
+ except Exception:
71
+ pass
72
+
73
+ import subprocess
74
+ import sys
75
+
76
+ subprocess.run(
77
+ [sys.executable, "-m", "piper.download_voices", voice_name],
78
+ check=True,
79
+ )
80
+ for path in candidates:
81
+ if path.is_file():
82
+ return path
83
+
84
+ raise FileNotFoundError(
85
+ f"Piper voice {voice_name!r} not found. "
86
+ f"Run: python -m piper.download_voices {voice_name}"
87
+ )
88
+
89
+ def synthesize(
90
+ self,
91
+ text: str,
92
+ *,
93
+ language: str,
94
+ out_dir: Path | None = None,
95
+ ) -> tuple[str | None, str | None]:
96
+ if not text.strip():
97
+ return None, "No text to synthesize."
98
+
99
+ voice_name, warning = self._voice_name(language)
100
+ try:
101
+ import wave
102
+
103
+ from piper import PiperVoice
104
+
105
+ voice = self._load_voice(voice_name)
106
+ base = out_dir or outputs_dir()
107
+ base.mkdir(parents=True, exist_ok=True)
108
+ out_path = base / f"voiceout_{uuid.uuid4().hex[:10]}.wav"
109
+ with wave.open(str(out_path), "wb") as wav_file:
110
+ voice.synthesize_wav(text, wav_file)
111
+ return str(out_path), warning
112
+ except ImportError:
113
+ return None, "piper-tts not installed; VoiceOut skipped."
114
+ except Exception as exc: # noqa: BLE001
115
+ return None, f"VoiceOut failed: {exc}"
116
+
117
+
118
+ def get_tts_backend(preset_key: str | None = None) -> TtsBackend:
119
+ config = get_echo_coach_config()
120
+ preset = config.get_tts(preset_key)
121
+ if preset.key not in _tts_cache:
122
+ _tts_cache[preset.key] = PiperTtsBackend(preset)
123
+ return _tts_cache[preset.key]
libs/echocoach/src/echocoach/utils.py ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ import re
5
+ from typing import Any
6
+
7
+
8
+ def extract_json(text: str) -> dict[str, Any]:
9
+ """Parse JSON from an LLM response (fenced blocks or trailing prose)."""
10
+ cleaned = text.strip()
11
+ fence = re.search(r"```(?:json)?\s*(.*?)\s*```", cleaned, re.DOTALL | re.IGNORECASE)
12
+ if fence:
13
+ cleaned = fence.group(1).strip()
14
+
15
+ start = cleaned.find("{")
16
+ if start < 0:
17
+ return json.loads(cleaned)
18
+
19
+ end = _matching_brace_end(cleaned, start)
20
+ if end is not None:
21
+ return json.loads(cleaned[start : end + 1])
22
+
23
+ fallback_end = cleaned.rfind("}")
24
+ if fallback_end > start:
25
+ return json.loads(cleaned[start : fallback_end + 1])
26
+ return json.loads(cleaned)
27
+
28
+
29
+ def _matching_brace_end(text: str, start: int) -> int | None:
30
+ if start >= len(text) or text[start] != "{":
31
+ return None
32
+ depth = 0
33
+ in_string = False
34
+ escape = False
35
+ for index in range(start, len(text)):
36
+ char = text[index]
37
+ if escape:
38
+ escape = False
39
+ continue
40
+ if char == "\\" and in_string:
41
+ escape = True
42
+ continue
43
+ if char == '"':
44
+ in_string = not in_string
45
+ continue
46
+ if in_string:
47
+ continue
48
+ if char == "{":
49
+ depth += 1
50
+ elif char == "}":
51
+ depth -= 1
52
+ if depth == 0:
53
+ return index
54
+ return None
libs/echocoach/tests/fixtures/silence_2s.wav ADDED
Binary file (64 kB). View file
 
libs/echocoach/tests/make_fixture.py ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Generate a short silent WAV fixture for smoke tests."""
2
+
3
+ from pathlib import Path
4
+
5
+ import numpy as np
6
+ import soundfile as sf
7
+
8
+ OUT = Path(__file__).resolve().parent / "fixtures" / "silence_2s.wav"
9
+
10
+
11
+ def main() -> None:
12
+ OUT.parent.mkdir(parents=True, exist_ok=True)
13
+ audio = np.zeros(32_000, dtype=np.float32) # 2s @ 16kHz
14
+ sf.write(OUT, audio, 16_000)
15
+ print(f"Wrote {OUT}")
16
+
17
+
18
+ if __name__ == "__main__":
19
+ main()
libs/echocoach/tests/test_coach_parse.py ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from echocoach.coach import parse_coach_response
2
+
3
+
4
+ def test_parse_coach_json_from_fenced_block():
5
+ raw = """Here is feedback:
6
+ ```json
7
+ {
8
+ "summary": "Good energy.",
9
+ "filler_feedback": "Reduce um.",
10
+ "pace_feedback": "Slow down.",
11
+ "rewrite": "We should start now.",
12
+ "one_tip": "Pause after each point."
13
+ }
14
+ ```
15
+ """
16
+ feedback = parse_coach_response(raw)
17
+ assert feedback.summary == "Good energy."
18
+ assert feedback.one_tip == "Pause after each point."
libs/echocoach/tests/test_fillers.py ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from echocoach.analysis.fillers import analyze_fillers, highlight_fillers_html
2
+
3
+
4
+ def test_detects_common_fillers():
5
+ text = "So um I think like you know we should basically start."
6
+ analysis = analyze_fillers(text)
7
+ assert analysis.total >= 4
8
+ assert "um" in analysis.counts
9
+ assert "like" in analysis.counts
10
+
11
+
12
+ def test_highlight_wraps_fillers():
13
+ text = "Um hello there."
14
+ analysis = analyze_fillers(text)
15
+ html = highlight_fillers_html(text, analysis)
16
+ assert "<mark" in html
17
+ assert "Um" in html or "um" in html.lower()
libs/echocoach/tests/test_pace.py ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from echocoach.analysis.pace import analyze_pace
2
+
3
+
4
+ def test_ideal_pace_scores_high():
5
+ # 150 words in 60s => 150 WPM
6
+ text = " ".join(["word"] * 150)
7
+ pace = analyze_pace(text, 60.0)
8
+ assert pace.wpm == 150.0
9
+ assert pace.score == 100
10
+ assert pace.label == "Ideal pace"
11
+
12
+
13
+ def test_slow_pace_penalized():
14
+ text = " ".join(["word"] * 50)
15
+ pace = analyze_pace(text, 60.0)
16
+ assert pace.wpm == 50.0
17
+ assert pace.score < 100
18
+ assert "slow" in pace.label.lower()
libs/echocoach/tests/test_recording.py ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ from pathlib import Path
4
+
5
+ import pytest
6
+
7
+ from echocoach.recording import (
8
+ ServerRecordingError,
9
+ recording_level_warning,
10
+ select_capture_backend,
11
+ start_server_recording,
12
+ stop_server_recording,
13
+ )
14
+
15
+
16
+ class _FakeProcess:
17
+ def __init__(self, out_path: Path) -> None:
18
+ self._running = True
19
+ self.returncode: int | None = None
20
+ self.stderr = None
21
+ self._out_path = out_path
22
+
23
+ def poll(self) -> int | None:
24
+ return None if self._running else self.returncode
25
+
26
+ def send_signal(self, sig: int) -> None:
27
+ self._running = False
28
+ self.returncode = 1 # pw-record exits 1 on SIGINT
29
+ self._out_path.write_bytes(b"RIFF" + b"x" * 100)
30
+
31
+ def wait(self, timeout: float | None = None) -> int:
32
+ return 0
33
+
34
+
35
+ def test_select_capture_backend_prefers_pw_record(monkeypatch):
36
+ monkeypatch.setattr("echocoach.recording._pw_record_available", lambda: True)
37
+ monkeypatch.setattr("echocoach.recording._sounddevice_available", lambda: True)
38
+ monkeypatch.setattr("echocoach.recording._arecord_available", lambda: True)
39
+ assert select_capture_backend() == "pw-record"
40
+
41
+
42
+ class _FakeTimer:
43
+ def __init__(self, *_args, **_kwargs) -> None:
44
+ self.daemon = False
45
+
46
+ def start(self) -> None:
47
+ return None
48
+
49
+ def cancel(self) -> None:
50
+ return None
51
+
52
+
53
+ def test_start_stop_session(monkeypatch, tmp_path):
54
+ import echocoach.recording as rec
55
+
56
+ rec._session = None
57
+
58
+ def fake_spawn(backend, path: Path):
59
+ return _FakeProcess(path)
60
+
61
+ monkeypatch.setattr("echocoach.recording.select_capture_backend", lambda: "pw-record")
62
+ monkeypatch.setattr("echocoach.recording.outputs_dir", lambda: tmp_path)
63
+ monkeypatch.setattr("echocoach.recording._spawn_capture_process", fake_spawn)
64
+ monkeypatch.setattr("echocoach.recording.threading.Timer", _FakeTimer)
65
+
66
+ start_server_recording(10)
67
+ path = stop_server_recording()
68
+ assert path.is_file()
69
+ rec._session = None
70
+
71
+
72
+ def test_start_while_recording_raises(monkeypatch, tmp_path):
73
+ import echocoach.recording as rec
74
+
75
+ rec._session = None
76
+
77
+ def fake_spawn(backend, path: Path):
78
+ return _FakeProcess(path)
79
+
80
+ monkeypatch.setattr("echocoach.recording.select_capture_backend", lambda: "pw-record")
81
+ monkeypatch.setattr("echocoach.recording.outputs_dir", lambda: tmp_path)
82
+ monkeypatch.setattr("echocoach.recording._spawn_capture_process", fake_spawn)
83
+ monkeypatch.setattr("echocoach.recording.threading.Timer", _FakeTimer)
84
+
85
+ start_server_recording(10)
86
+ with pytest.raises(ServerRecordingError, match="Already recording"):
87
+ start_server_recording(10)
88
+ stop_server_recording()
89
+ rec._session = None
90
+
91
+
92
+ def test_stop_without_start_raises():
93
+ import echocoach.recording as rec
94
+
95
+ rec._session = None
96
+ rec._last_recording_path = None
97
+ with pytest.raises(ServerRecordingError, match="Not recording"):
98
+ stop_server_recording()
99
+
100
+
101
+ def test_recording_level_warning_detects_silence(tmp_path):
102
+ import numpy as np
103
+ import soundfile as sf
104
+
105
+ silent = tmp_path / "silent.wav"
106
+ sf.write(silent, np.zeros(16_000, dtype=np.float32), 16_000)
107
+ assert "silent" in (recording_level_warning(silent) or "").lower()
108
+
109
+
110
+ def test_finalize_accepts_pw_record_exit_code_one(tmp_path, monkeypatch):
111
+ import echocoach.recording as rec
112
+
113
+ rec._session = None
114
+ rec._last_recording_path = None
115
+
116
+ out_path = tmp_path / "recordings" / "server_pw.wav"
117
+ out_path.parent.mkdir(parents=True)
118
+ fake_proc = _FakeProcess(out_path)
119
+
120
+ session = rec._RecordingSession(
121
+ process=fake_proc,
122
+ out_path=out_path,
123
+ backend="pw-record",
124
+ max_seconds=10,
125
+ started_at=0.0,
126
+ watchdog=None,
127
+ )
128
+ rec._session = session
129
+ path = rec.stop_server_recording()
130
+ assert path == out_path
131
+ assert path.stat().st_size > 44
132
+ rec._session = None
pyproject.toml CHANGED
@@ -6,6 +6,7 @@ readme = "README.md"
6
  requires-python = ">=3.12"
7
  dependencies = [
8
  "agent",
 
9
  "ensemble",
10
  "gradio-space",
11
  "inference",
@@ -44,6 +45,7 @@ members = [
44
 
45
  [tool.uv.sources]
46
  agent = { workspace = true }
 
47
  ensemble = { workspace = true }
48
  gradio-space = { workspace = true }
49
  inference = { workspace = true }
 
6
  requires-python = ">=3.12"
7
  dependencies = [
8
  "agent",
9
+ "echocoach",
10
  "ensemble",
11
  "gradio-space",
12
  "inference",
 
45
 
46
  [tool.uv.sources]
47
  agent = { workspace = true }
48
+ echocoach = { workspace = true }
49
  ensemble = { workspace = true }
50
  gradio-space = { workspace = true }
51
  inference = { workspace = true }
scripts/echo_coach_smoke.sh ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ # Smoke test for EchoCoach analysis (no GPU / no ASR models).
3
+ set -euo pipefail
4
+ cd "$(dirname "$0")/.."
5
+
6
+ FIXTURE="libs/echocoach/tests/fixtures/silence_2s.wav"
7
+ if [[ ! -f "$FIXTURE" ]]; then
8
+ uv run python libs/echocoach/tests/make_fixture.py
9
+ fi
10
+
11
+ uv run pytest libs/echocoach/tests/test_fillers.py libs/echocoach/tests/test_pace.py libs/echocoach/tests/test_coach_parse.py -q
12
+ echo "EchoCoach smoke tests passed."
uv.lock CHANGED
The diff for this file is too large to render. See raw diff
 
voice_models.yaml ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Voice model presets for EchoCoach (ASR + TTS).
2
+ # Override defaults via ECHOCOACH_ASR_PRESET / ECHOCOACH_TTS_PRESET in .env
3
+
4
+ defaults:
5
+ asr_preset: whisper-cpp-tiny
6
+ tts_preset: piper-multilingual
7
+ coach_model: minicpm5-1b
8
+ max_seconds: 30
9
+
10
+ languages:
11
+ - code: en
12
+ label: English
13
+ - code: fr
14
+ label: French
15
+ - code: de
16
+ label: German
17
+ - code: es
18
+ label: Spanish
19
+ - code: it
20
+ label: Italian
21
+ - code: pt
22
+ label: Portuguese
23
+ - code: nl
24
+ label: Dutch
25
+ - code: pl
26
+ label: Polish
27
+ - code: el
28
+ label: Greek
29
+ - code: ar
30
+ label: Arabic
31
+ - code: ja
32
+ label: Japanese
33
+ - code: zh
34
+ label: Chinese (Mandarin)
35
+ - code: vi
36
+ label: Vietnamese
37
+ - code: ko
38
+ label: Korean
39
+
40
+ asr:
41
+ cohere-transcribe:
42
+ label: Cohere Transcribe 2B (14 languages)
43
+ backend: cohere
44
+ model_id: CohereLabs/cohere-transcribe-03-2026
45
+
46
+ whisper-cpp-tiny:
47
+ label: Whisper.cpp tiny (CPU, fast)
48
+ backend: whisper_cpp
49
+ model_size: tiny
50
+
51
+ whisper-cpp-base:
52
+ label: Whisper.cpp base (CPU, better WER)
53
+ backend: whisper_cpp
54
+ model_size: base
55
+
56
+ tts:
57
+ piper-multilingual:
58
+ label: Piper TTS (local VoiceOut)
59
+ backend: piper
60
+ voices:
61
+ en: en_US-lessac-medium
62
+ fr: fr_FR-siwis-medium
63
+ de: de_DE-thorsten-medium
64
+ es: es_ES-sharvard-medium
65
+ it: it_IT-riccardo-medium
66
+ pt: pt_BR-faber-medium
67
+ nl: nl_NL-mls-medium
68
+ pl: pl_PL-darkman-medium
69
+ el: en_US-lessac-medium
70
+ ar: ar_JO-kareem-medium
71
+ ja: ja_JP-natsuki-medium
72
+ zh: zh_CN-huayan-medium
73
+ vi: vi_VN-25hours-single
74
+ ko: ko_KR-kss-medium
75
+ fallback_voice: en_US-lessac-medium