MSG commited on
Commit ·
1719c2a
1
Parent(s): 83e828a
Feat/voice stuff (#7)
Browse files* voice plan
* voice models config
* echocoach
* coach echo wip
* wip try fix record
* test record wip
* test echocoach
* wip fix recording
- .cursor/plans/echocoach_voice_tab_45e774f7.plan.md +249 -0
- .cursor/plans/teachervoice_realtime_plan_8950875f.plan.md +211 -0
- .env.example +11 -0
- .gitignore +2 -1
- Dockerfile +5 -1
- USAGE.md +50 -1
- apps/gradio-space/pyproject.toml +2 -0
- apps/gradio-space/src/gradio_space/app.py +24 -6
- apps/gradio-space/src/gradio_space/tabs/__init__.py +7 -1
- apps/gradio-space/src/gradio_space/tabs/echo_coach.py +263 -0
- libs/echocoach/README.md +10 -0
- libs/echocoach/pyproject.toml +39 -0
- libs/echocoach/src/echocoach/__init__.py +6 -0
- libs/echocoach/src/echocoach/analysis/__init__.py +0 -0
- libs/echocoach/src/echocoach/analysis/charts.py +96 -0
- libs/echocoach/src/echocoach/analysis/fillers.py +82 -0
- libs/echocoach/src/echocoach/analysis/pace.py +49 -0
- libs/echocoach/src/echocoach/asr/__init__.py +0 -0
- libs/echocoach/src/echocoach/asr/cohere.py +51 -0
- libs/echocoach/src/echocoach/asr/factory.py +40 -0
- libs/echocoach/src/echocoach/asr/whisper_cpp.py +33 -0
- libs/echocoach/src/echocoach/audio_io.py +38 -0
- libs/echocoach/src/echocoach/coach.py +108 -0
- libs/echocoach/src/echocoach/config.py +228 -0
- libs/echocoach/src/echocoach/models.py +58 -0
- libs/echocoach/src/echocoach/pipeline.py +127 -0
- libs/echocoach/src/echocoach/recording.py +348 -0
- libs/echocoach/src/echocoach/tts/__init__.py +3 -0
- libs/echocoach/src/echocoach/tts/piper.py +123 -0
- libs/echocoach/src/echocoach/utils.py +54 -0
- libs/echocoach/tests/fixtures/silence_2s.wav +0 -0
- libs/echocoach/tests/make_fixture.py +19 -0
- libs/echocoach/tests/test_coach_parse.py +18 -0
- libs/echocoach/tests/test_fillers.py +17 -0
- libs/echocoach/tests/test_pace.py +18 -0
- libs/echocoach/tests/test_recording.py +132 -0
- pyproject.toml +2 -0
- scripts/echo_coach_smoke.sh +12 -0
- uv.lock +0 -0
- voice_models.yaml +75 -0
.cursor/plans/echocoach_voice_tab_45e774f7.plan.md
ADDED
|
@@ -0,0 +1,249 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: EchoCoach voice tab
|
| 3 |
+
overview: "Add an EchoCoach tab to the existing Gradio app: record a pitch locally, transcribe with a configurable voice-model stack (Cohere Transcribe 2B default, Whisper.cpp fallback), analyze fillers/pace with matplotlib, coach via the existing MiniCPM5 text LLM, and speak feedback back with local TTS (Piper; MiniCPM-o 4.5 optional when GPU allows)."
|
| 4 |
+
todos:
|
| 5 |
+
- id: scaffold-echocoach-lib
|
| 6 |
+
content: Create libs/echocoach package with config.py, voice_models.yaml loader, EchoCoachResult types, and workspace pyproject wiring
|
| 7 |
+
status: completed
|
| 8 |
+
- id: analysis-module
|
| 9 |
+
content: Implement filler detection, pace scoring, and matplotlib chart generation (fillers bar + pace timeline)
|
| 10 |
+
status: completed
|
| 11 |
+
- id: asr-backends
|
| 12 |
+
content: Add Cohere Transcribe ASR backend (primary) and pywhispercpp tiny/base fallback with factory pattern
|
| 13 |
+
status: completed
|
| 14 |
+
- id: coach-tts
|
| 15 |
+
content: Wire coach prompts to existing inference backend; add Piper TTS VoiceOut with per-language voice map
|
| 16 |
+
status: completed
|
| 17 |
+
- id: gradio-tab
|
| 18 |
+
content: Build echo_coach.py tab (mic record, language, analyze, transcript HTML, charts, audio out, trace) and register in app.py
|
| 19 |
+
status: completed
|
| 20 |
+
- id: docs-docker
|
| 21 |
+
content: Update voice_models.yaml, .env.example, USAGE.md, Dockerfile copy paths; add unit tests and smoke fixture
|
| 22 |
+
status: completed
|
| 23 |
+
isProject: false
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
# EchoCoach — Real-Time Voice Practice Coach
|
| 27 |
+
|
| 28 |
+
## Goal
|
| 29 |
+
|
| 30 |
+
Ship a new **EchoCoach** tab in [`apps/gradio-space/src/gradio_space/app.py`](apps/gradio-space/src/gradio_space/app.py) that runs the hackathon demo end-to-end **locally**:
|
| 31 |
+
|
| 32 |
+
> Record up to 30s pitch → transcript with **filler words highlighted** → **pace score** chart → **rewrite suggestion** from a small text LLM → **VoiceOut** audio reply in the selected language.
|
| 33 |
+
|
| 34 |
+
Per your direction: **no SmolLM3 LoRA**. Voice I/O is config-driven; coaching stays on the existing text preset (`minicpm5-1b`).
|
| 35 |
+
|
| 36 |
+
## Architecture
|
| 37 |
+
|
| 38 |
+
```mermaid
|
| 39 |
+
flowchart LR
|
| 40 |
+
subgraph ui [echo_coach.py Gradio tab]
|
| 41 |
+
Mic[gr.Audio mic max 30s]
|
| 42 |
+
Lang[Language dropdown 14 langs]
|
| 43 |
+
VoicePreset[ASR and TTS preset]
|
| 44 |
+
AnalyzeBtn[Analyze pitch]
|
| 45 |
+
end
|
| 46 |
+
|
| 47 |
+
subgraph voice [libs/echocoach]
|
| 48 |
+
ASR[ASR backend factory]
|
| 49 |
+
Analysis[filler + pace + matplotlib]
|
| 50 |
+
Coach[coach prompts via inference]
|
| 51 |
+
TTS[TTS VoiceOut backend]
|
| 52 |
+
end
|
| 53 |
+
|
| 54 |
+
Mic --> ASR
|
| 55 |
+
Lang --> ASR
|
| 56 |
+
ASR --> Transcript[transcript text]
|
| 57 |
+
Transcript --> Analysis
|
| 58 |
+
Analysis --> Charts[pace + filler plots]
|
| 59 |
+
Transcript --> Coach
|
| 60 |
+
Coach --> Feedback[rewrite + tips JSON]
|
| 61 |
+
Feedback --> TTS
|
| 62 |
+
TTS --> AudioOut[gr.Audio playback]
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
## Voice model strategy (configurable)
|
| 66 |
+
|
| 67 |
+
Add a dedicated registry — [`voice_models.yaml`](voice_models.yaml) at repo root (parallel to [`models.yaml`](models.yaml)) — so ASR/TTS are swappable without touching the text LLM presets.
|
| 68 |
+
|
| 69 |
+
| Preset key | Role | Stack | When to use |
|
| 70 |
+
|------------|------|-------|-------------|
|
| 71 |
+
| `cohere-transcribe` (default ASR) | Speech → text | `CohereLabs/cohere-transcribe-03-2026` via `transformers>=5.4` | 14 languages, edge-friendly 2B ASR, best accuracy |
|
| 72 |
+
| `whisper-cpp-tiny` | Speech → text fallback | `pywhispercpp` (preferred over stale `whisper-cpp-python`) | CPU-only, fast, English-focused demos |
|
| 73 |
+
| `whisper-cpp-base` | Speech → text fallback | same | Better WER, still lightweight |
|
| 74 |
+
| `piper-multilingual` (default TTS) | Text → speech VoiceOut | Piper voices mapped per language code | Local TTS for all 14 Cohere langs |
|
| 75 |
+
| `minicpm-o-4.5` (optional, stretch) | Speech in + speech out | `openbmb/MiniCPM-o-4_5` with `init_audio=True, init_tts=True` | GPU workstation only (~9B); EN/ZH TTS |
|
| 76 |
+
|
| 77 |
+
**MVP ships:** `cohere-transcribe` + `piper-multilingual` + `minicpm5-1b` coach.
|
| 78 |
+
|
| 79 |
+
**Evaluate MiniCPM-o 4.5** as an alternate preset behind `ECHOCOACH_VOICE_PROFILE=omni` — do not block MVP on its heavier deps / GPU requirements.
|
| 80 |
+
|
| 81 |
+
Env vars (add to [`.env.example`](.env.example)):
|
| 82 |
+
|
| 83 |
+
- `ECHOCOACH_ASR_PRESET` — default `cohere-transcribe`
|
| 84 |
+
- `ECHOCOACH_TTS_PRESET` — default `piper-multilingual`
|
| 85 |
+
- `ECHOCOACH_COACH_MODEL` — default `minicpm5-1b` (reuses [`libs/inference`](libs/inference))
|
| 86 |
+
- `ECHOCOACH_MAX_SECONDS` — default `30`
|
| 87 |
+
|
| 88 |
+
## New package: `libs/echocoach`
|
| 89 |
+
|
| 90 |
+
Mirror the inference factory pattern in a focused library.
|
| 91 |
+
|
| 92 |
+
### Layout
|
| 93 |
+
|
| 94 |
+
```
|
| 95 |
+
libs/echocoach/
|
| 96 |
+
pyproject.toml
|
| 97 |
+
src/echocoach/
|
| 98 |
+
config.py # load voice_models.yaml + env overrides
|
| 99 |
+
asr/
|
| 100 |
+
base.py
|
| 101 |
+
cohere.py # CohereAsrForConditionalGeneration wrapper
|
| 102 |
+
whisper_cpp.py # pywhispercpp tiny/base
|
| 103 |
+
factory.py
|
| 104 |
+
tts/
|
| 105 |
+
base.py
|
| 106 |
+
piper.py # language → voice map, WAV output
|
| 107 |
+
factory.py
|
| 108 |
+
analysis/
|
| 109 |
+
fillers.py # detect um/uh/like/you know/… + highlight HTML
|
| 110 |
+
pace.py # WPM, target band 120–160, 0–100 score
|
| 111 |
+
charts.py # matplotlib → PNG paths for Gradio Image
|
| 112 |
+
coach.py # structured JSON coach via get_backend()
|
| 113 |
+
pipeline.py # orchestrate: audio path → EchoCoachResult
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
### ASR: Cohere Transcribe (primary)
|
| 117 |
+
|
| 118 |
+
Follow the official HF quick start (`AutoProcessor`, `CohereAsrForConditionalGeneration`, `language="en"` etc.). Notes for implementation:
|
| 119 |
+
|
| 120 |
+
- Model is **gated** on Hugging Face — document `huggingface-cli login` + accept terms in USAGE.
|
| 121 |
+
- Requires `transformers>=5.4.0`, `soundfile`, `librosa`.
|
| 122 |
+
- Pass explicit `language` from the UI dropdown (`en`, `fr`, `de`, `es`, `it`, `pt`, `nl`, `pl`, `el`, `ar`, `ja`, `zh`, `vi`, `ko`).
|
| 123 |
+
- Resample incoming audio to 16 kHz mono (Gradio uploads may vary).
|
| 124 |
+
|
| 125 |
+
### ASR: Whisper.cpp fallback
|
| 126 |
+
|
| 127 |
+
Use **`pywhispercpp`** (actively maintained; same whisper.cpp backend you specified). Wrap `Model('tiny'|'base').transcribe(wav_path)` for offline file transcription. Skip `pyaudio` in MVP — Gradio `gr.Audio(sources=["microphone"], type="filepath")` captures mic without PortAudio in the server process.
|
| 128 |
+
|
| 129 |
+
### Analysis (no LLM)
|
| 130 |
+
|
| 131 |
+
Rule-based, fast, deterministic:
|
| 132 |
+
|
| 133 |
+
- **Fillers:** configurable word list + regex; return spans for HTML highlight in transcript panel.
|
| 134 |
+
- **Pace:** `words / (duration_minutes)`; score vs target band; flag too fast/slow.
|
| 135 |
+
- **Charts** ([`matplotlib`](https://matplotlib.org/) Agg backend):
|
| 136 |
+
- Bar chart: filler counts
|
| 137 |
+
- Line chart: words-per-30s-window over recording duration
|
| 138 |
+
|
| 139 |
+
### Coach (text LLM)
|
| 140 |
+
|
| 141 |
+
Reuse [`get_backend(model_key).chat()`](libs/inference/src/inference/factory.py) with a tight system prompt asking for JSON:
|
| 142 |
+
|
| 143 |
+
```json
|
| 144 |
+
{
|
| 145 |
+
"summary": "...",
|
| 146 |
+
"filler_feedback": "...",
|
| 147 |
+
"pace_feedback": "...",
|
| 148 |
+
"rewrite": "...",
|
| 149 |
+
"one_tip": "..."
|
| 150 |
+
}
|
| 151 |
+
```
|
| 152 |
+
|
| 153 |
+
Parse with existing JSON repair patterns from [`libs/agent/src/agent/runner.py`](libs/agent/src/agent/runner.py) (reuse `_parse_json` style, don't duplicate agent runner).
|
| 154 |
+
|
| 155 |
+
### TTS VoiceOut
|
| 156 |
+
|
| 157 |
+
- **Piper:** map language code → voice model; synthesize coach `summary + one_tip` (or full rewrite on toggle) to WAV under `AGENT_OUTPUTS_DIR`.
|
| 158 |
+
- Return filepath to `gr.Audio` for playback.
|
| 159 |
+
- If Piper voice missing for a language, fall back to English voice + show UI warning.
|
| 160 |
+
|
| 161 |
+
## Gradio tab: `echo_coach.py`
|
| 162 |
+
|
| 163 |
+
New file [`apps/gradio-space/src/gradio_space/tabs/echo_coach.py`](apps/gradio-space/src/gradio_space/tabs/echo_coach.py).
|
| 164 |
+
|
| 165 |
+
**UI components:**
|
| 166 |
+
|
| 167 |
+
| Component | Purpose |
|
| 168 |
+
|-----------|---------|
|
| 169 |
+
| `gr.Audio` (mic, max 30s) | Record pitch |
|
| 170 |
+
| Language dropdown (14 codes) | Cohere ASR + TTS voice selection |
|
| 171 |
+
| ASR preset dropdown (dev) | `cohere-transcribe` / `whisper-cpp-tiny` / `whisper-cpp-base` |
|
| 172 |
+
| Coach model status | Reuse `model_status()` from [`model_loading.py`](apps/gradio-space/src/gradio_space/model_loading.py) |
|
| 173 |
+
| Analyze button | Run full pipeline |
|
| 174 |
+
| Transcript HTML | Filler highlights |
|
| 175 |
+
| Markdown report | Pace score, filler count, coach JSON fields |
|
| 176 |
+
| `gr.Image` × 2 | Matplotlib charts |
|
| 177 |
+
| `gr.Audio` output | VoiceOut playback |
|
| 178 |
+
| `gr.JSON` | Trace (Sharing is Caring badge) |
|
| 179 |
+
|
| 180 |
+
Wire into [`app.py`](apps/gradio-space/src/gradio_space/app.py) as a fourth tab and export from [`tabs/__init__.py`](apps/gradio-space/src/gradio_space/tabs/__init__.py).
|
| 181 |
+
|
| 182 |
+
## Dependencies
|
| 183 |
+
|
| 184 |
+
Add to workspace:
|
| 185 |
+
|
| 186 |
+
| Package | Where | Why |
|
| 187 |
+
|---------|-------|-----|
|
| 188 |
+
| `echocoach` | root `pyproject.toml` workspace member | new lib |
|
| 189 |
+
| `matplotlib`, `soundfile`, `librosa` | `libs/echocoach` | charts + audio I/O |
|
| 190 |
+
| `pywhispercpp` | `libs/echocoach` optional extra `[whisper]` | whisper fallback |
|
| 191 |
+
| `piper-tts` or `piper-phonemize` | `libs/echocoach` optional extra `[piper]` | VoiceOut |
|
| 192 |
+
| bump `transformers>=5.4` | `libs/inference` or `echocoach` | Cohere ASR class |
|
| 193 |
+
|
| 194 |
+
Keep **`pyaudio` out of MVP** (Gradio handles capture). Document optional streaming mode (pyaudio + segment callback) as phase 2.
|
| 195 |
+
|
| 196 |
+
## Docker / HF Space notes
|
| 197 |
+
|
| 198 |
+
Current [`Dockerfile`](Dockerfile) copies only gradio-space + agent + inference. EchoCoach requires:
|
| 199 |
+
|
| 200 |
+
- Copy `libs/echocoach` + `voice_models.yaml`
|
| 201 |
+
- `apt-get install` `ffmpeg`, `libsndfile1` (audio deps)
|
| 202 |
+
- **Mic access:** HF Space visitors can **upload** audio; live mic is **local dev only**
|
| 203 |
+
- **GPU basic** recommended when `cohere-transcribe` is active (2B ASR + 1B coach)
|
| 204 |
+
|
| 205 |
+
## Testing
|
| 206 |
+
|
| 207 |
+
Lightweight unit tests in `libs/echocoach/tests/`:
|
| 208 |
+
|
| 209 |
+
- `test_fillers.py` — highlight spans on sample transcript
|
| 210 |
+
- `test_pace.py` — WPM + score for known duration/word count
|
| 211 |
+
- `test_coach_parse.py` — JSON extraction from mocked LLM output
|
| 212 |
+
- Skip GPU/integration tests in CI; smoke script `scripts/echo_coach_smoke.sh` with a bundled 5s WAV fixture
|
| 213 |
+
|
| 214 |
+
## Docs
|
| 215 |
+
|
| 216 |
+
Update [`USAGE.md`](USAGE.md):
|
| 217 |
+
|
| 218 |
+
- EchoCoach tab walkthrough
|
| 219 |
+
- Voice preset config (`voice_models.yaml`, env vars)
|
| 220 |
+
- HF gated model login for Cohere Transcribe
|
| 221 |
+
- Local-only mic vs Space upload
|
| 222 |
+
- Hardware guidance (CPU whisper vs GPU cohere)
|
| 223 |
+
|
| 224 |
+
## Implementation order
|
| 225 |
+
|
| 226 |
+
1. **`libs/echocoach` scaffold** — config, types, `EchoCoachResult` dataclass
|
| 227 |
+
2. **Analysis module** — fillers, pace, matplotlib (no ML deps)
|
| 228 |
+
3. **ASR backends** — Cohere first, whisper fallback second
|
| 229 |
+
4. **Coach module** — prompts + inference integration
|
| 230 |
+
5. **TTS Piper backend** — language voice map + WAV output
|
| 231 |
+
6. **Gradio tab** — wire UI + pipeline
|
| 232 |
+
7. **Registry + docs** — `voice_models.yaml`, `.env.example`, USAGE.md, Dockerfile copy paths
|
| 233 |
+
|
| 234 |
+
## Risks and mitigations
|
| 235 |
+
|
| 236 |
+
| Risk | Mitigation |
|
| 237 |
+
|------|------------|
|
| 238 |
+
| Cohere model gated / large download | Document HF auth; allow `whisper-cpp-tiny` preset for offline CPU demo |
|
| 239 |
+
| Cohere needs transformers 5.4+ | Pin in `echocoach` pyproject; test alongside existing inference |
|
| 240 |
+
| Piper voice coverage gaps | English fallback + visible warning in UI |
|
| 241 |
+
| MiniCPM-o 4.5 instability / VRAM | Optional preset only; not MVP blocker |
|
| 242 |
+
| Real-time duplex | Deferred; MVP is record-then-analyze (matches demo) |
|
| 243 |
+
|
| 244 |
+
## Hackathon alignment
|
| 245 |
+
|
| 246 |
+
- **Backyard track** — teacher/student voice practice for someone you know
|
| 247 |
+
- **Tiny Titan** — coach on MiniCPM5 1B; ASR on 2B Cohere (still "small model" stack)
|
| 248 |
+
- **All local** — no cloud LLM API; optional HF weight download only
|
| 249 |
+
- **Sharing is Caring** — JSON trace per analysis under `outputs/traces/`
|
.cursor/plans/teachervoice_realtime_plan_8950875f.plan.md
ADDED
|
@@ -0,0 +1,211 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
name: TeacherVoice Realtime Plan
|
| 3 |
+
overview: EchoCoach today is a one-shot pitch *review* pipeline (record → analyze → report). TeacherVoice-style real-time teaching needs a new turn-based voice conversation layer that reuses existing ASR/LLM/TTS pieces but changes the interaction model, prompts, and integration with lessons/RAG.
|
| 4 |
+
todos:
|
| 5 |
+
- id: teacher-voice-core
|
| 6 |
+
content: Add teacher_voice.py + mode prompts + TeacherVoiceTurnResult; wire ASR → chat(history) → TTS per turn
|
| 7 |
+
status: pending
|
| 8 |
+
- id: teacher-voice-ui
|
| 9 |
+
content: Create teacher_voice.py Gradio tab (modes, push-to-talk, chatbot, audio reply, optional RAG); register in app.py
|
| 10 |
+
status: pending
|
| 11 |
+
- id: rag-lesson-modes
|
| 12 |
+
content: Integrate ResearchMind retrieval into Explain/Lesson modes (reuse run_research_question / rag scope from chat tab)
|
| 13 |
+
status: pending
|
| 14 |
+
- id: docs-tests
|
| 15 |
+
content: Update USAGE.md with EchoCoach vs TeacherVoice; add mock-backend unit tests and trace skill teacher-voice
|
| 16 |
+
status: pending
|
| 17 |
+
- id: phase2-chunked-tts
|
| 18 |
+
content: "Optional: sentence-chunked Piper + shorter turn cap for faster time-to-first-audio"
|
| 19 |
+
status: pending
|
| 20 |
+
- id: phase3-omni
|
| 21 |
+
content: "Optional: MiniCPM-o 4.5 omni preset behind ECHOCOACH_VOICE_PROFILE=omni for speech-in/speech-out"
|
| 22 |
+
status: pending
|
| 23 |
+
isProject: false
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
# TeacherVoice — from EchoCoach review to real-time voice teacher
|
| 27 |
+
|
| 28 |
+
## What “coach help” is today
|
| 29 |
+
|
| 30 |
+
In this repo, **coach help = EchoCoach** — a separate Gradio tab, not a global assistant.
|
| 31 |
+
|
| 32 |
+
| What it does | What it does **not** do |
|
| 33 |
+
|--------------|-------------------------|
|
| 34 |
+
| Records up to 30s of **your** monologue (pitch practice) | Talk back-and-forth in a conversation |
|
| 35 |
+
| Transcribes → scores fillers/pace → one LLM **JSON report** | Explain lesson topics on demand |
|
| 36 |
+
| Speaks **one** Piper TTS clip (summary or rewrite) | Create slides or pitch decks by voice |
|
| 37 |
+
| Runs once per click on **Analyze pitch** | Stream partial audio or interrupt mid-sentence |
|
| 38 |
+
|
| 39 |
+
Pipeline (batch, sequential):
|
| 40 |
+
|
| 41 |
+
```mermaid
|
| 42 |
+
sequenceDiagram
|
| 43 |
+
participant User
|
| 44 |
+
participant Gradio as EchoCoach_tab
|
| 45 |
+
participant Pipe as run_echo_coach
|
| 46 |
+
participant ASR
|
| 47 |
+
participant Coach as coach.py_JSON
|
| 48 |
+
participant TTS
|
| 49 |
+
|
| 50 |
+
User->>Gradio: Record_stop_then_Analyze
|
| 51 |
+
Gradio->>Pipe: audio_filepath
|
| 52 |
+
Pipe->>ASR: full_file_transcribe
|
| 53 |
+
Pipe->>Pipe: fillers_pace_charts
|
| 54 |
+
Pipe->>Coach: single_shot_JSON_prompt
|
| 55 |
+
Pipe->>TTS: one_WAV_summary
|
| 56 |
+
Pipe->>Gradio: report_charts_audio
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
Key code: [`libs/echocoach/src/echocoach/pipeline.py`](libs/echocoach/src/echocoach/pipeline.py) orchestrates ASR → analysis → [`coach.py`](libs/echocoach/src/echocoach/coach.py) → Piper. UI is [`apps/gradio-space/src/gradio_space/tabs/echo_coach.py`](apps/gradio-space/src/gradio_space/tabs/echo_coach.py).
|
| 60 |
+
|
| 61 |
+
**Lesson slides** ([`education_pptx.py`](apps/gradio-space/src/gradio_space/tabs/education_pptx.py)) and **Chat (debug)** ([`chat.py`](apps/gradio-space/src/gradio_space/tabs/chat.py)) are unrelated tabs: slides are batch `AgentRunner` jobs; chat is **text-only** multi-turn (optionally RAG via [`rag_aware_chat`](apps/gradio-space/src/gradio_space/research_helpers.py)).
|
| 62 |
+
|
| 63 |
+
There is **no `TeacherVoice` name, WebSocket, or streaming voice** anywhere in the codebase. The EchoCoach plan explicitly deferred duplex:
|
| 64 |
+
|
| 65 |
+
> Real-time duplex | Deferred; MVP is record-then-analyze
|
| 66 |
+
|
| 67 |
+
---
|
| 68 |
+
|
| 69 |
+
## How TeacherVoice differs (target behavior)
|
| 70 |
+
|
| 71 |
+
**TeacherVoice** (what you’re describing) is a **conversational voice teacher**:
|
| 72 |
+
|
| 73 |
+
- You speak a **question or short turn** → teacher **replies in voice** → repeat
|
| 74 |
+
- Modes share one chat history but different system prompts:
|
| 75 |
+
- **Explain** — tutor a topic in plain language (optionally grounded in ResearchMind RAG)
|
| 76 |
+
- **Lesson** — discuss/outline a lesson verbally (can hand off topic to Lesson slides tab)
|
| 77 |
+
- **Pitch** — lighter coaching per turn (“try opening with X”) instead of one big JSON report
|
| 78 |
+
|
| 79 |
+
```mermaid
|
| 80 |
+
sequenceDiagram
|
| 81 |
+
participant User
|
| 82 |
+
participant TV as TeacherVoice_tab
|
| 83 |
+
participant ASR
|
| 84 |
+
participant LLM as chat_with_history
|
| 85 |
+
participant TTS
|
| 86 |
+
|
| 87 |
+
loop each_turn
|
| 88 |
+
User->>TV: push_to_talk_stop
|
| 89 |
+
TV->>ASR: transcribe_turn
|
| 90 |
+
TV->>LLM: messages_plus_mode_system_prompt
|
| 91 |
+
LLM-->>TV: reply_text
|
| 92 |
+
TV->>TTS: synthesize_reply
|
| 93 |
+
TV->>User: play_audio_update_chat
|
| 94 |
+
end
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
This is **turn-based pseudo-real-time** (typical latency: ~2–8s per turn on GPU). True **duplex** TeacherVoice (interrupt while speaking, sub-second feel) would need streaming ASR + chunked TTS or an omni speech model — not present today.
|
| 98 |
+
|
| 99 |
+
---
|
| 100 |
+
|
| 101 |
+
## Recommended architecture (phased)
|
| 102 |
+
|
| 103 |
+
### Phase 1 — TeacherVoice tab (MVP, fits current stack)
|
| 104 |
+
|
| 105 |
+
Add a **TeacherVoice** tab alongside EchoCoach; keep EchoCoach as the deep pitch *analyzer*.
|
| 106 |
+
|
| 107 |
+
**New module** in `libs/echocoach` (minimal new package surface):
|
| 108 |
+
|
| 109 |
+
- `teacher_voice.py` — `run_teacher_voice_turn(audio_path, history, mode, language, backend, rag_context?) -> TeacherVoiceTurnResult`
|
| 110 |
+
- `prompts.py` — mode system prompts (`explain`, `lesson`, `pitch`) as **plain text** chat (not JSON like EchoCoach)
|
| 111 |
+
- Reuse: [`recording.py`](libs/echocoach/src/echocoach/recording.py), ASR factory, Piper TTS, `InferenceBackend.chat()`
|
| 112 |
+
|
| 113 |
+
**New Gradio tab** `teacher_voice.py`:
|
| 114 |
+
|
| 115 |
+
- Mode dropdown: Explain | Lesson coach | Pitch practice
|
| 116 |
+
- Optional: ResearchMind session/docs (copy pattern from [`chat.py`](apps/gradio-space/src/gradio_space/tabs/chat.py))
|
| 117 |
+
- Topic field for Lesson/Explain modes
|
| 118 |
+
- **Push-to-talk**: reuse Start/Stop recording from EchoCoach (already works server-side)
|
| 119 |
+
- `gr.Chatbot` (text history) + `gr.Audio` auto-play for teacher reply
|
| 120 |
+
- “Clear conversation” button
|
| 121 |
+
|
| 122 |
+
**Turn flow per button press:**
|
| 123 |
+
|
| 124 |
+
1. Stop recording → WAV path
|
| 125 |
+
2. ASR → user text (show in chat)
|
| 126 |
+
3. Build messages: `system(mode)` + `history` + optional RAG context block + `user(transcript)`
|
| 127 |
+
4. `backend.chat()` — same stack as debug chat ([`inference`](libs/inference/src/inference/base.py) has no streaming today; full response is fine for MVP)
|
| 128 |
+
5. Piper TTS → play reply
|
| 129 |
+
6. Append `(user, assistant)` to history
|
| 130 |
+
|
| 131 |
+
**Pitch mode vs EchoCoach:** Pitch mode gives **conversational** tips each turn; EchoCoach remains the **quantitative** tool (WPM, filler charts, rewrite JSON). Link from TeacherVoice: “Deep analysis → open EchoCoach tab with last recording.”
|
| 132 |
+
|
| 133 |
+
**Lesson mode:** Does not generate `.pptx` by voice in MVP. It verbally outlines and explains; user can copy topic into Lesson slides tab. Phase 2 can add “Generate slides from this conversation” button calling `AgentRunner.run_education_pptx`.
|
| 134 |
+
|
| 135 |
+
### Phase 2 — Faster “feels live” (still not full duplex)
|
| 136 |
+
|
| 137 |
+
- **VAD / max 15s turns** — cap turn length for lower latency
|
| 138 |
+
- **Sentence-chunked TTS** — split LLM reply on `.!?`, synthesize first sentence while rest generates (reuse Piper in a small loop)
|
| 139 |
+
- **Optional streaming ASR** — only if a backend supports partial transcripts (Cohere batch is fine for MVP; whisper stays file-based)
|
| 140 |
+
- Latency target: first audio ~3s after stop on GPU
|
| 141 |
+
|
| 142 |
+
### Phase 3 — True speech-in/speech-out (optional, GPU-heavy)
|
| 143 |
+
|
| 144 |
+
From [`echocoach_voice_tab` plan](.cursor/plans/echocoach_voice_tab_45e774f7.plan.md), not implemented:
|
| 145 |
+
|
| 146 |
+
- Add `minicpm-o-4.5` preset to [`voice_models.yaml`](voice_models.yaml) behind `ECHOCOACH_VOICE_PROFILE=omni`
|
| 147 |
+
- New omni backend in `libs/echocoach` (or `libs/inference`) with `init_audio=True, init_tts=True`
|
| 148 |
+
- TeacherVoice tab switches to omni when profile=omni and GPU available
|
| 149 |
+
- EN/ZH only initially; falls back to Phase 1 pipeline otherwise
|
| 150 |
+
|
| 151 |
+
Skip WebSocket server unless you need browser-native duplex outside Gradio — Gradio turn-based is enough for hackathon demo.
|
| 152 |
+
|
| 153 |
+
---
|
| 154 |
+
|
| 155 |
+
## What to reuse vs build new
|
| 156 |
+
|
| 157 |
+
| Component | Reuse | New work |
|
| 158 |
+
|-----------|-------|----------|
|
| 159 |
+
| Mic capture | `recording.py`, `gr.Audio` | Wire to per-turn handler |
|
| 160 |
+
| ASR | `asr/factory.py` | Call per turn, not once per monologue |
|
| 161 |
+
| LLM | `get_backend().chat()` | Mode prompts + multi-turn history (like chat tab) |
|
| 162 |
+
| RAG | `run_research_question` / `rag_aware_chat` logic | Inject retrieved context into TeacherVoice system prompt |
|
| 163 |
+
| TTS | `tts/piper.py` | Per-turn synthesis; optional chunking in Phase 2 |
|
| 164 |
+
| Pitch analytics | `analysis/fillers.py`, `pace.py` | **Not** on every TeacherVoice turn — keep in EchoCoach only |
|
| 165 |
+
| Lesson PPTX | `AgentRunner.run_education_pptx` | Phase 2 button, not MVP voice loop |
|
| 166 |
+
|
| 167 |
+
---
|
| 168 |
+
|
| 169 |
+
## Files to touch (Phase 1)
|
| 170 |
+
|
| 171 |
+
| File | Change |
|
| 172 |
+
|------|--------|
|
| 173 |
+
| [`libs/echocoach/src/echocoach/teacher_voice.py`](libs/echocoach/src/echocoach/teacher_voice.py) | **New** — turn orchestration + result type |
|
| 174 |
+
| [`libs/echocoach/src/echocoach/prompts.py`](libs/echocoach/src/echocoach/prompts.py) | **New** — Explain / Lesson / Pitch system prompts |
|
| 175 |
+
| [`apps/gradio-space/src/gradio_space/tabs/teacher_voice.py`](apps/gradio-space/src/gradio_space/tabs/teacher_voice.py) | **New** — UI |
|
| 176 |
+
| [`apps/gradio-space/src/gradio_space/app.py`](apps/gradio-space/src/gradio_space/app.py) | Register TeacherVoice tab |
|
| 177 |
+
| [`apps/gradio-space/src/gradio_space/tabs/__init__.py`](apps/gradio-space/src/gradio_space/tabs/__init__.py) | Export builder |
|
| 178 |
+
| [`USAGE.md`](USAGE.md) | Document modes, latency expectations, EchoCoach vs TeacherVoice |
|
| 179 |
+
| [`libs/echocoach/tests/test_teacher_voice.py`](libs/echocoach/tests/test_teacher_voice.py) | Mock-backend tests for prompt assembly + history |
|
| 180 |
+
|
| 181 |
+
EchoCoach files stay unchanged except optional cross-link in UI markdown.
|
| 182 |
+
|
| 183 |
+
---
|
| 184 |
+
|
| 185 |
+
## Hardware and UX expectations
|
| 186 |
+
|
| 187 |
+
- **CPU-only:** Whisper tiny ASR + MiniCPM5 1B + Piper — workable but ~5–15s per turn; set expectations in UI
|
| 188 |
+
- **GPU:** Cohere Transcribe + same coach model — better for demo
|
| 189 |
+
- **HF Space:** Upload/push-to-talk may be limited; document local-only mic (same as EchoCoach)
|
| 190 |
+
- Label honestly: **“Voice conversation (turn-based)”** until Phase 3 omni
|
| 191 |
+
|
| 192 |
+
---
|
| 193 |
+
|
| 194 |
+
## Success criteria for MVP
|
| 195 |
+
|
| 196 |
+
- User can hold a **3+ turn** spoken conversation in Explain mode
|
| 197 |
+
- Lesson mode accepts a topic + optional RAG session and answers from ingested docs with citations in chat text
|
| 198 |
+
- Pitch mode gives short spoken coaching without requiring Analyze pitch
|
| 199 |
+
- EchoCoach tab still works independently for full pitch analysis
|
| 200 |
+
- Trace JSON per session under `outputs/traces/` (skill: `teacher-voice`)
|
| 201 |
+
|
| 202 |
+
---
|
| 203 |
+
|
| 204 |
+
## Risks
|
| 205 |
+
|
| 206 |
+
| Risk | Mitigation |
|
| 207 |
+
|------|------------|
|
| 208 |
+
| Users expect ChatGPT Voice latency | UI copy: turn-based; show “Transcribing / Thinking / Speaking” states |
|
| 209 |
+
| RAG + voice doubles latency | Retrieve once per turn; keep `max_tokens` modest (~512) |
|
| 210 |
+
| Piper missing voice for language | Existing English fallback in [`voice_models.yaml`](voice_models.yaml) |
|
| 211 |
+
| Confusion between EchoCoach and TeacherVoice | Separate tabs; Pitch mode points to EchoCoach for charts |
|
.env.example
CHANGED
|
@@ -50,6 +50,17 @@ ALLOW_MODEL_SWITCH=false
|
|
| 50 |
# After training, point Gradio at the adapter preset:
|
| 51 |
# ACTIVE_MODEL=minicpm5-1b-lesson-lora
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
# --- Ensemble research (research/ensemble/) ---
|
| 54 |
# Base LLM resolution (first match wins): ENSEMBLE_LLM, LLM_PATH, BASE, MODEL_ID, ACTIVE_MODEL
|
| 55 |
# LLM_PATH=./models/finetuned/minicpm5-1b-lora-merged
|
|
|
|
| 50 |
# After training, point Gradio at the adapter preset:
|
| 51 |
# ACTIVE_MODEL=minicpm5-1b-lesson-lora
|
| 52 |
|
| 53 |
+
# --- EchoCoach (voice practice coach) ---
|
| 54 |
+
# VOICE_PRESETS_PATH=./voice_models.yaml
|
| 55 |
+
# ECHOCOACH_ASR_PRESET=whisper-cpp-tiny
|
| 56 |
+
# ECHOCOACH_TTS_PRESET=piper-multilingual
|
| 57 |
+
# ECHOCOACH_COACH_MODEL=minicpm5-1b
|
| 58 |
+
# ECHOCOACH_MAX_SECONDS=30
|
| 59 |
+
# ECHOCOACH_CAPTURE_DEVICE= # optional ALSA/PipeWire device (e.g. pipewire, alsa_input.pci-...)
|
| 60 |
+
# PIPER_VOICES_DIR=~/.local/share/piper/voices
|
| 61 |
+
# For Cohere Transcribe ASR: huggingface-cli login + accept model terms, then:
|
| 62 |
+
# ECHOCOACH_ASR_PRESET=cohere-transcribe
|
| 63 |
+
|
| 64 |
# --- Ensemble research (research/ensemble/) ---
|
| 65 |
# Base LLM resolution (first match wins): ENSEMBLE_LLM, LLM_PATH, BASE, MODEL_ID, ACTIVE_MODEL
|
| 66 |
# LLM_PATH=./models/finetuned/minicpm5-1b-lora-merged
|
.gitignore
CHANGED
|
@@ -14,4 +14,5 @@ outputs/traces
|
|
| 14 |
|
| 15 |
/results
|
| 16 |
|
| 17 |
-
outputs/researchmind
|
|
|
|
|
|
| 14 |
|
| 15 |
/results
|
| 16 |
|
| 17 |
+
outputs/researchmind
|
| 18 |
+
outputs/echocoach
|
Dockerfile
CHANGED
|
@@ -7,19 +7,23 @@ ENV PYTHONUNBUFFERED=1 \
|
|
| 7 |
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 8 |
build-essential \
|
| 9 |
cmake \
|
|
|
|
|
|
|
| 10 |
&& rm -rf /var/lib/apt/lists/*
|
| 11 |
|
| 12 |
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
|
| 13 |
|
| 14 |
WORKDIR /app
|
| 15 |
|
| 16 |
-
COPY pyproject.toml uv.lock .python-version README.md models.yaml ./
|
| 17 |
COPY apps/gradio-space/pyproject.toml apps/gradio-space/README.md apps/gradio-space/
|
| 18 |
COPY libs/inference/pyproject.toml libs/inference/README.md libs/inference/
|
| 19 |
COPY libs/agent/pyproject.toml libs/agent/README.md libs/agent/
|
|
|
|
| 20 |
COPY apps/gradio-space/src apps/gradio-space/src
|
| 21 |
COPY libs/inference/src libs/inference/src
|
| 22 |
COPY libs/agent/src libs/agent/src
|
|
|
|
| 23 |
COPY skills skills
|
| 24 |
|
| 25 |
RUN useradd -m -u 1000 user && \
|
|
|
|
| 7 |
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 8 |
build-essential \
|
| 9 |
cmake \
|
| 10 |
+
ffmpeg \
|
| 11 |
+
libsndfile1 \
|
| 12 |
&& rm -rf /var/lib/apt/lists/*
|
| 13 |
|
| 14 |
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
|
| 15 |
|
| 16 |
WORKDIR /app
|
| 17 |
|
| 18 |
+
COPY pyproject.toml uv.lock .python-version README.md models.yaml voice_models.yaml ./
|
| 19 |
COPY apps/gradio-space/pyproject.toml apps/gradio-space/README.md apps/gradio-space/
|
| 20 |
COPY libs/inference/pyproject.toml libs/inference/README.md libs/inference/
|
| 21 |
COPY libs/agent/pyproject.toml libs/agent/README.md libs/agent/
|
| 22 |
+
COPY libs/echocoach/pyproject.toml libs/echocoach/README.md libs/echocoach/
|
| 23 |
COPY apps/gradio-space/src apps/gradio-space/src
|
| 24 |
COPY libs/inference/src libs/inference/src
|
| 25 |
COPY libs/agent/src libs/agent/src
|
| 26 |
+
COPY libs/echocoach/src libs/echocoach/src
|
| 27 |
COPY skills skills
|
| 28 |
|
| 29 |
RUN useradd -m -u 1000 user && \
|
USAGE.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
How to run the **Lesson Agent** Gradio app locally, test it in Docker, and deploy to a Hugging Face Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon).
|
| 4 |
|
| 5 |
-
The primary UI is the **Lesson slides** tab (topic → local model outline → downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model.
|
| 6 |
|
| 7 |
## Prerequisites
|
| 8 |
|
|
@@ -72,6 +72,55 @@ When **Web search** is selected, choose a **search workflow**:
|
|
| 72 |
|
| 73 |
Web discover/auto search requires network access. MemRAG data is stored under `RESEARCHMIND_DATA_DIR` (default `outputs/researchmind`).
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
### 5. Upload agent trace (Sharing is Caring badge)
|
| 76 |
|
| 77 |
```bash
|
|
|
|
| 2 |
|
| 3 |
How to run the **Lesson Agent** Gradio app locally, test it in Docker, and deploy to a Hugging Face Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon).
|
| 4 |
|
| 5 |
+
The primary UI is the **Lesson slides** tab (topic → local model outline → downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, **EchoCoach** for voice practice feedback, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model.
|
| 6 |
|
| 7 |
## Prerequisites
|
| 8 |
|
|
|
|
| 72 |
|
| 73 |
Web discover/auto search requires network access. MemRAG data is stored under `RESEARCHMIND_DATA_DIR` (default `outputs/researchmind`).
|
| 74 |
|
| 75 |
+
Web discover/auto search requires network access. MemRAG data is stored under `RESEARCHMIND_DATA_DIR` (default `outputs/researchmind`).
|
| 76 |
+
|
| 77 |
+
### EchoCoach — voice practice
|
| 78 |
+
|
| 79 |
+
The **EchoCoach** tab records up to 30 seconds, then runs a local pipeline:
|
| 80 |
+
|
| 81 |
+
**Getting audio in**
|
| 82 |
+
|
| 83 |
+
- **Record from this computer** — click **Start recording**, speak, then **Stop recording** (uses PipeWire `pw-record` when available). The slider is a max-length safety cap.
|
| 84 |
+
- **Browser Record** — needs mic permission and a secure context; open **http://localhost:7860** (not `0.0.0.0` or a LAN IP).
|
| 85 |
+
- **Upload** — drop a `.wav` or `.mp3` file (works everywhere, including HF Space).
|
| 86 |
+
|
| 87 |
+
If recordings sound silent, check system mic input/mute or set `ECHOCOACH_CAPTURE_DEVICE` in `.env` (see `arecord -L` or `pw-cli ls Node`).
|
| 88 |
+
|
| 89 |
+
Pipeline steps:
|
| 90 |
+
|
| 91 |
+
1. **ASR** — Cohere Transcribe 2B (14 languages) or Whisper.cpp tiny/base
|
| 92 |
+
2. **Analysis** — filler highlights, pace score, matplotlib charts
|
| 93 |
+
3. **Coach** — rewrite + tips from the text LLM (`ACTIVE_MODEL`, default `minicpm5-1b`)
|
| 94 |
+
4. **VoiceOut** — Piper TTS speaks the summary (or full rewrite if checked)
|
| 95 |
+
|
| 96 |
+
Install optional extras:
|
| 97 |
+
|
| 98 |
+
```bash
|
| 99 |
+
# Whisper.cpp fallback ASR (CPU)
|
| 100 |
+
uv sync --package echocoach --extra whisper
|
| 101 |
+
|
| 102 |
+
# Piper VoiceOut TTS
|
| 103 |
+
uv sync --package echocoach --extra piper
|
| 104 |
+
python -m piper.download_voices en_US-lessac-medium
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
Configure presets in [`voice_models.yaml`](voice_models.yaml) or via `.env`:
|
| 108 |
+
|
| 109 |
+
| Variable | Default | Description |
|
| 110 |
+
| -------- | ------- | ----------- |
|
| 111 |
+
| `ECHOCOACH_ASR_PRESET` | `whisper-cpp-tiny` | ASR preset key |
|
| 112 |
+
| `ECHOCOACH_TTS_PRESET` | `piper-multilingual` | TTS preset key |
|
| 113 |
+
| `ECHOCOACH_COACH_MODEL` | `minicpm5-1b` | Text coach preset (from `models.yaml`) |
|
| 114 |
+
| `ECHOCOACH_MAX_SECONDS` | `30` | Max recording length |
|
| 115 |
+
|
| 116 |
+
**Cohere Transcribe** (`cohere-transcribe`) is gated on Hugging Face — run `huggingface-cli login`, accept the model terms, then set `ECHOCOACH_ASR_PRESET=cohere-transcribe`. GPU recommended for ASR + coach together.
|
| 117 |
+
|
| 118 |
+
Smoke tests (analysis only, no GPU):
|
| 119 |
+
|
| 120 |
+
```bash
|
| 121 |
+
bash scripts/echo_coach_smoke.sh
|
| 122 |
+
```
|
| 123 |
+
|
| 124 |
### 5. Upload agent trace (Sharing is Caring badge)
|
| 125 |
|
| 126 |
```bash
|
apps/gradio-space/pyproject.toml
CHANGED
|
@@ -9,12 +9,14 @@ authors = [
|
|
| 9 |
requires-python = ">=3.12"
|
| 10 |
dependencies = [
|
| 11 |
"agent",
|
|
|
|
| 12 |
"gradio>=5.0.0",
|
| 13 |
"inference",
|
| 14 |
]
|
| 15 |
|
| 16 |
[tool.uv.sources]
|
| 17 |
agent = { workspace = true }
|
|
|
|
| 18 |
inference = { workspace = true }
|
| 19 |
|
| 20 |
[build-system]
|
|
|
|
| 9 |
requires-python = ">=3.12"
|
| 10 |
dependencies = [
|
| 11 |
"agent",
|
| 12 |
+
"echocoach[whisper]",
|
| 13 |
"gradio>=5.0.0",
|
| 14 |
"inference",
|
| 15 |
]
|
| 16 |
|
| 17 |
[tool.uv.sources]
|
| 18 |
agent = { workspace = true }
|
| 19 |
+
echocoach = { workspace = true }
|
| 20 |
inference = { workspace = true }
|
| 21 |
|
| 22 |
[build-system]
|
apps/gradio-space/src/gradio_space/app.py
CHANGED
|
@@ -3,8 +3,14 @@ import os
|
|
| 3 |
import gradio as gr
|
| 4 |
|
| 5 |
from gradio_space.model_loading import preload_active_model
|
| 6 |
-
from gradio_space.tabs import
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
from gradio_space.tabs.education_pptx import gradio_allowed_paths
|
|
|
|
| 8 |
from gradio_space.tabs.research_mind import researchmind_allowed_paths
|
| 9 |
from inference.config import get_app_config
|
| 10 |
|
|
@@ -22,9 +28,9 @@ def build_demo() -> gr.Blocks:
|
|
| 22 |
with gr.Blocks(title="Lesson Agent + ResearchMind — Build Small Hackathon") as demo:
|
| 23 |
gr.Markdown(
|
| 24 |
f"""
|
| 25 |
-
# Lesson Agent + ResearchMind
|
| 26 |
|
| 27 |
-
Local skill-based agents — **lesson slides**
|
| 28 |
|
| 29 |
- **Model:** `{active.key}` — {active.label}
|
| 30 |
- **Backend:** `{active.backend}`
|
|
@@ -39,6 +45,8 @@ Part of the [Build Small Hackathon](https://huggingface.co/build-small-hackathon
|
|
| 39 |
build_education_pptx_tab()
|
| 40 |
with gr.Tab("ResearchMind"):
|
| 41 |
build_research_mind_tab()
|
|
|
|
|
|
|
| 42 |
with gr.Tab("Chat (debug)"):
|
| 43 |
build_chat_tab()
|
| 44 |
|
|
@@ -48,10 +56,20 @@ Part of the [Build Small Hackathon](https://huggingface.co/build-small-hackathon
|
|
| 48 |
def main() -> None:
|
| 49 |
preload_active_model()
|
| 50 |
demo = build_demo()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
demo.launch(
|
| 52 |
-
server_name=
|
| 53 |
-
server_port=
|
| 54 |
-
allowed_paths=[
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
)
|
| 56 |
|
| 57 |
|
|
|
|
| 3 |
import gradio as gr
|
| 4 |
|
| 5 |
from gradio_space.model_loading import preload_active_model
|
| 6 |
+
from gradio_space.tabs import (
|
| 7 |
+
build_chat_tab,
|
| 8 |
+
build_education_pptx_tab,
|
| 9 |
+
build_echo_coach_tab,
|
| 10 |
+
build_research_mind_tab,
|
| 11 |
+
)
|
| 12 |
from gradio_space.tabs.education_pptx import gradio_allowed_paths
|
| 13 |
+
from gradio_space.tabs.echo_coach import echo_coach_allowed_paths
|
| 14 |
from gradio_space.tabs.research_mind import researchmind_allowed_paths
|
| 15 |
from inference.config import get_app_config
|
| 16 |
|
|
|
|
| 28 |
with gr.Blocks(title="Lesson Agent + ResearchMind — Build Small Hackathon") as demo:
|
| 29 |
gr.Markdown(
|
| 30 |
f"""
|
| 31 |
+
# Lesson Agent + ResearchMind + EchoCoach
|
| 32 |
|
| 33 |
+
Local skill-based agents — **lesson slides**, **research with MemRAG**, and **voice practice coaching** (offline).
|
| 34 |
|
| 35 |
- **Model:** `{active.key}` — {active.label}
|
| 36 |
- **Backend:** `{active.backend}`
|
|
|
|
| 45 |
build_education_pptx_tab()
|
| 46 |
with gr.Tab("ResearchMind"):
|
| 47 |
build_research_mind_tab()
|
| 48 |
+
with gr.Tab("EchoCoach"):
|
| 49 |
+
build_echo_coach_tab()
|
| 50 |
with gr.Tab("Chat (debug)"):
|
| 51 |
build_chat_tab()
|
| 52 |
|
|
|
|
| 56 |
def main() -> None:
|
| 57 |
preload_active_model()
|
| 58 |
demo = build_demo()
|
| 59 |
+
port = int(os.environ.get("PORT", "7860"))
|
| 60 |
+
server_name = os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0")
|
| 61 |
+
print(
|
| 62 |
+
f"\n Local UI (browser mic works here): http://127.0.0.1:{port}\n"
|
| 63 |
+
f" Bound address: {server_name}:{port}\n"
|
| 64 |
+
)
|
| 65 |
demo.launch(
|
| 66 |
+
server_name=server_name,
|
| 67 |
+
server_port=port,
|
| 68 |
+
allowed_paths=[
|
| 69 |
+
*gradio_allowed_paths(),
|
| 70 |
+
*researchmind_allowed_paths(),
|
| 71 |
+
*echo_coach_allowed_paths(),
|
| 72 |
+
],
|
| 73 |
)
|
| 74 |
|
| 75 |
|
apps/gradio-space/src/gradio_space/tabs/__init__.py
CHANGED
|
@@ -1,5 +1,11 @@
|
|
| 1 |
from gradio_space.tabs.chat import build_chat_tab
|
| 2 |
from gradio_space.tabs.education_pptx import build_education_pptx_tab
|
|
|
|
| 3 |
from gradio_space.tabs.research_mind import build_research_mind_tab
|
| 4 |
|
| 5 |
-
__all__ = [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
from gradio_space.tabs.chat import build_chat_tab
|
| 2 |
from gradio_space.tabs.education_pptx import build_education_pptx_tab
|
| 3 |
+
from gradio_space.tabs.echo_coach import build_echo_coach_tab
|
| 4 |
from gradio_space.tabs.research_mind import build_research_mind_tab
|
| 5 |
|
| 6 |
+
__all__ = [
|
| 7 |
+
"build_chat_tab",
|
| 8 |
+
"build_education_pptx_tab",
|
| 9 |
+
"build_echo_coach_tab",
|
| 10 |
+
"build_research_mind_tab",
|
| 11 |
+
]
|
apps/gradio-space/src/gradio_space/tabs/echo_coach.py
ADDED
|
@@ -0,0 +1,263 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
|
| 5 |
+
import gradio as gr
|
| 6 |
+
|
| 7 |
+
from echocoach.config import get_echo_coach_config
|
| 8 |
+
from echocoach.pipeline import run_echo_coach
|
| 9 |
+
from echocoach.recording import (
|
| 10 |
+
ServerRecordingError,
|
| 11 |
+
recording_backend_status,
|
| 12 |
+
recording_elapsed_seconds,
|
| 13 |
+
recording_level_warning,
|
| 14 |
+
start_server_recording,
|
| 15 |
+
stop_server_recording,
|
| 16 |
+
)
|
| 17 |
+
from gradio_space.model_loading import ensure_model_loaded, get_active_model_key, model_status
|
| 18 |
+
from inference.factory import get_backend
|
| 19 |
+
|
| 20 |
+
_config = get_echo_coach_config()
|
| 21 |
+
_SAMPLE_AUDIO = (
|
| 22 |
+
Path(__file__).resolve().parents[5]
|
| 23 |
+
/ "libs"
|
| 24 |
+
/ "echocoach"
|
| 25 |
+
/ "tests"
|
| 26 |
+
/ "fixtures"
|
| 27 |
+
/ "silence_2s.wav"
|
| 28 |
+
)
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def _error_outputs(message: str) -> tuple:
|
| 32 |
+
return (
|
| 33 |
+
message,
|
| 34 |
+
f'<p style="color:#8a1f1f;">{message}</p>',
|
| 35 |
+
"",
|
| 36 |
+
None,
|
| 37 |
+
None,
|
| 38 |
+
None,
|
| 39 |
+
message,
|
| 40 |
+
{},
|
| 41 |
+
)
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def ui_start_recording(max_seconds: int) -> tuple[str, dict, dict]:
|
| 45 |
+
try:
|
| 46 |
+
start_server_recording(int(max_seconds))
|
| 47 |
+
except ServerRecordingError as exc:
|
| 48 |
+
return (
|
| 49 |
+
str(exc),
|
| 50 |
+
gr.update(interactive=True),
|
| 51 |
+
gr.update(interactive=False),
|
| 52 |
+
)
|
| 53 |
+
return (
|
| 54 |
+
(
|
| 55 |
+
f"Recording… speak now, then click **Stop recording** "
|
| 56 |
+
f"(auto-stops after {int(max_seconds)}s)."
|
| 57 |
+
),
|
| 58 |
+
gr.update(interactive=False),
|
| 59 |
+
gr.update(interactive=True),
|
| 60 |
+
)
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def ui_stop_recording() -> tuple[str | None, str, dict, dict]:
|
| 64 |
+
try:
|
| 65 |
+
elapsed = recording_elapsed_seconds()
|
| 66 |
+
path = stop_server_recording()
|
| 67 |
+
warning = recording_level_warning(path)
|
| 68 |
+
except ServerRecordingError as exc:
|
| 69 |
+
return (
|
| 70 |
+
None,
|
| 71 |
+
str(exc),
|
| 72 |
+
gr.update(interactive=True),
|
| 73 |
+
gr.update(interactive=False),
|
| 74 |
+
)
|
| 75 |
+
except Exception as exc: # noqa: BLE001 — surface unexpected recorder errors
|
| 76 |
+
return (
|
| 77 |
+
None,
|
| 78 |
+
f"Recording failed: {exc}",
|
| 79 |
+
gr.update(interactive=True),
|
| 80 |
+
gr.update(interactive=False),
|
| 81 |
+
)
|
| 82 |
+
|
| 83 |
+
status = f"Recording saved ({elapsed:.1f}s) → `{path}`. Click **Analyze pitch**."
|
| 84 |
+
if warning:
|
| 85 |
+
status += f" Warning: {warning}"
|
| 86 |
+
return (
|
| 87 |
+
gr.update(value=str(path)),
|
| 88 |
+
status,
|
| 89 |
+
gr.update(interactive=True),
|
| 90 |
+
gr.update(interactive=False),
|
| 91 |
+
)
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
def load_sample_pitch() -> tuple[str | None, str]:
|
| 95 |
+
if not _SAMPLE_AUDIO.is_file():
|
| 96 |
+
return (
|
| 97 |
+
None,
|
| 98 |
+
f"Sample clip missing at `{_SAMPLE_AUDIO}`. Run `uv run python libs/echocoach/tests/make_fixture.py`.",
|
| 99 |
+
)
|
| 100 |
+
return gr.update(value=str(_SAMPLE_AUDIO)), "Loaded 2s sample clip. Click **Analyze pitch** to test the pipeline."
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def analyze_pitch(
|
| 104 |
+
audio_path: str | None,
|
| 105 |
+
language: str,
|
| 106 |
+
asr_preset: str,
|
| 107 |
+
speak_rewrite: bool,
|
| 108 |
+
) -> tuple:
|
| 109 |
+
model_key = get_active_model_key()
|
| 110 |
+
load_error = ensure_model_loaded(model_key)
|
| 111 |
+
if load_error:
|
| 112 |
+
return _error_outputs(load_error)
|
| 113 |
+
|
| 114 |
+
if not audio_path:
|
| 115 |
+
return _error_outputs("Record or upload a pitch (up to 30 seconds), then click **Analyze pitch**.")
|
| 116 |
+
|
| 117 |
+
try:
|
| 118 |
+
result = run_echo_coach(
|
| 119 |
+
audio_path,
|
| 120 |
+
language=language,
|
| 121 |
+
asr_preset=asr_preset,
|
| 122 |
+
backend=get_backend(model_key),
|
| 123 |
+
speak_rewrite=speak_rewrite,
|
| 124 |
+
)
|
| 125 |
+
except Exception as exc: # noqa: BLE001 — surface pipeline errors in UI
|
| 126 |
+
return _error_outputs(f"EchoCoach failed: {exc}")
|
| 127 |
+
|
| 128 |
+
status = "Analysis complete."
|
| 129 |
+
if result.voiceout_warning:
|
| 130 |
+
status += f" VoiceOut: {result.voiceout_warning}"
|
| 131 |
+
|
| 132 |
+
return (
|
| 133 |
+
status,
|
| 134 |
+
result.transcript_html,
|
| 135 |
+
result.report_markdown,
|
| 136 |
+
result.filler_chart_path,
|
| 137 |
+
result.pace_chart_path,
|
| 138 |
+
result.voiceout_path,
|
| 139 |
+
f"Trace saved: `{result.trace_path}`",
|
| 140 |
+
result.trace,
|
| 141 |
+
)
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def build_echo_coach_tab() -> None:
|
| 145 |
+
lang_choices = _config.language_choices()
|
| 146 |
+
asr_choices = _config.asr_choices()
|
| 147 |
+
default_lang = lang_choices[0][1] if lang_choices else "en"
|
| 148 |
+
default_asr = _config.asr_preset
|
| 149 |
+
mic_status = recording_backend_status()
|
| 150 |
+
|
| 151 |
+
gr.Markdown(
|
| 152 |
+
f"""
|
| 153 |
+
Record up to **{_config.max_seconds} seconds**, then get local feedback: transcript with **filler highlights**,
|
| 154 |
+
**pace score**, coach **rewrite**, and **VoiceOut** audio — all on-device.
|
| 155 |
+
|
| 156 |
+
- **ASR:** configurable (`voice_models.yaml`) — Cohere Transcribe 2B or Whisper.cpp
|
| 157 |
+
- **Coach:** text LLM preset (`ACTIVE_MODEL` / `ECHOCOACH_COACH_MODEL`)
|
| 158 |
+
- **TTS:** Piper VoiceOut (optional; install `echocoach[piper]`)
|
| 159 |
+
|
| 160 |
+
**Browser mic:** open **http://localhost:7860** in Chrome or Firefox (not Cursor's preview) and allow microphone access.
|
| 161 |
+
If the mic icon fails, use **Start / Stop recording** below or **Upload** a `.wav` / `.mp3`.
|
| 162 |
+
"""
|
| 163 |
+
)
|
| 164 |
+
|
| 165 |
+
with gr.Row():
|
| 166 |
+
with gr.Column(scale=1):
|
| 167 |
+
record_status_md = gr.Markdown(mic_status)
|
| 168 |
+
with gr.Accordion("Record from this computer (recommended)", open=True):
|
| 169 |
+
gr.Markdown(
|
| 170 |
+
"Click **Start recording**, speak your pitch, then **Stop recording** when done. "
|
| 171 |
+
"The slider sets the maximum length (auto-stop safety cap)."
|
| 172 |
+
)
|
| 173 |
+
record_seconds = gr.Slider(
|
| 174 |
+
label="Max recording length (seconds)",
|
| 175 |
+
minimum=3,
|
| 176 |
+
maximum=_config.max_seconds,
|
| 177 |
+
value=min(30, _config.max_seconds),
|
| 178 |
+
step=1,
|
| 179 |
+
)
|
| 180 |
+
with gr.Row():
|
| 181 |
+
record_start_btn = gr.Button("Start recording", variant="secondary")
|
| 182 |
+
record_stop_btn = gr.Button("Stop recording", variant="stop", interactive=False)
|
| 183 |
+
sample_btn = gr.Button("Load sample clip", variant="secondary")
|
| 184 |
+
audio_in = gr.Audio(
|
| 185 |
+
label="Your pitch (browser mic or upload)",
|
| 186 |
+
sources=["upload", "microphone"],
|
| 187 |
+
type="filepath",
|
| 188 |
+
format="wav",
|
| 189 |
+
)
|
| 190 |
+
language = gr.Dropdown(
|
| 191 |
+
label="Language",
|
| 192 |
+
choices=lang_choices,
|
| 193 |
+
value=default_lang,
|
| 194 |
+
)
|
| 195 |
+
asr_preset = gr.Dropdown(
|
| 196 |
+
label="ASR preset",
|
| 197 |
+
choices=asr_choices,
|
| 198 |
+
value=default_asr,
|
| 199 |
+
)
|
| 200 |
+
speak_rewrite = gr.Checkbox(
|
| 201 |
+
label="VoiceOut speaks full rewrite (otherwise summary + tip)",
|
| 202 |
+
value=False,
|
| 203 |
+
)
|
| 204 |
+
analyze_btn = gr.Button("Analyze pitch", variant="primary")
|
| 205 |
+
status = gr.Textbox(label="Status", interactive=False, lines=3)
|
| 206 |
+
coach_status = gr.Markdown(model_status(get_active_model_key()))
|
| 207 |
+
|
| 208 |
+
with gr.Column(scale=2):
|
| 209 |
+
transcript_html = gr.HTML(label="Transcript")
|
| 210 |
+
report_md = gr.Markdown(label="Coach report")
|
| 211 |
+
with gr.Row():
|
| 212 |
+
filler_chart = gr.Image(label="Filler words", type="filepath")
|
| 213 |
+
pace_chart = gr.Image(label="Pace timeline", type="filepath")
|
| 214 |
+
voiceout = gr.Audio(label="VoiceOut", type="filepath")
|
| 215 |
+
trace_note = gr.Markdown()
|
| 216 |
+
trace_json = gr.JSON(label="Trace")
|
| 217 |
+
|
| 218 |
+
record_start_btn.click(
|
| 219 |
+
ui_start_recording,
|
| 220 |
+
inputs=[record_seconds],
|
| 221 |
+
outputs=[status, record_start_btn, record_stop_btn],
|
| 222 |
+
)
|
| 223 |
+
record_stop_btn.click(
|
| 224 |
+
ui_stop_recording,
|
| 225 |
+
outputs=[audio_in, status, record_start_btn, record_stop_btn],
|
| 226 |
+
).then(
|
| 227 |
+
lambda: recording_backend_status(),
|
| 228 |
+
outputs=[record_status_md],
|
| 229 |
+
)
|
| 230 |
+
|
| 231 |
+
sample_btn.click(
|
| 232 |
+
load_sample_pitch,
|
| 233 |
+
outputs=[audio_in, status],
|
| 234 |
+
)
|
| 235 |
+
|
| 236 |
+
analyze_btn.click(
|
| 237 |
+
analyze_pitch,
|
| 238 |
+
inputs=[audio_in, language, asr_preset, speak_rewrite],
|
| 239 |
+
outputs=[
|
| 240 |
+
status,
|
| 241 |
+
transcript_html,
|
| 242 |
+
report_md,
|
| 243 |
+
filler_chart,
|
| 244 |
+
pace_chart,
|
| 245 |
+
voiceout,
|
| 246 |
+
trace_note,
|
| 247 |
+
trace_json,
|
| 248 |
+
],
|
| 249 |
+
)
|
| 250 |
+
|
| 251 |
+
|
| 252 |
+
def echo_coach_allowed_paths() -> list[str]:
|
| 253 |
+
base = get_echo_coach_config()
|
| 254 |
+
paths: list[str] = []
|
| 255 |
+
if base.presets_path:
|
| 256 |
+
paths.append(str(base.presets_path.parent))
|
| 257 |
+
from echocoach.config import outputs_dir
|
| 258 |
+
|
| 259 |
+
paths.append(str(outputs_dir()))
|
| 260 |
+
paths.append(str(outputs_dir() / "recordings"))
|
| 261 |
+
if _SAMPLE_AUDIO.is_file():
|
| 262 |
+
paths.append(str(_SAMPLE_AUDIO.parent))
|
| 263 |
+
return paths
|
libs/echocoach/README.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# echocoach
|
| 2 |
+
|
| 3 |
+
Local voice practice coach for the Build Small Hackathon.
|
| 4 |
+
|
| 5 |
+
- **ASR:** Cohere Transcribe 2B or Whisper.cpp (tiny/base)
|
| 6 |
+
- **Analysis:** filler detection, pace scoring, matplotlib charts
|
| 7 |
+
- **Coach:** text LLM via `inference` (default `minicpm5-1b`)
|
| 8 |
+
- **VoiceOut:** Piper TTS (optional extra)
|
| 9 |
+
|
| 10 |
+
Configure presets in repo-root `voice_models.yaml`.
|
libs/echocoach/pyproject.toml
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[project]
|
| 2 |
+
name = "echocoach"
|
| 3 |
+
version = "0.1.0"
|
| 4 |
+
description = "Local voice practice coach — ASR, pace/filler analysis, LLM coaching, TTS VoiceOut"
|
| 5 |
+
readme = "README.md"
|
| 6 |
+
authors = [
|
| 7 |
+
{ name = "MSGhais", email = "msghais135@gmail.com" }
|
| 8 |
+
]
|
| 9 |
+
requires-python = ">=3.12"
|
| 10 |
+
dependencies = [
|
| 11 |
+
"inference",
|
| 12 |
+
"agent",
|
| 13 |
+
"matplotlib>=3.9.0",
|
| 14 |
+
"numpy>=2.0.0",
|
| 15 |
+
"pyyaml>=6.0.2",
|
| 16 |
+
"soundfile>=0.12.0",
|
| 17 |
+
"sounddevice>=0.5.0",
|
| 18 |
+
"librosa>=0.10.0",
|
| 19 |
+
"transformers>=5.4.0",
|
| 20 |
+
"torch>=2.5.0",
|
| 21 |
+
"accelerate>=1.2.0",
|
| 22 |
+
"huggingface-hub>=0.27.0",
|
| 23 |
+
]
|
| 24 |
+
|
| 25 |
+
[project.optional-dependencies]
|
| 26 |
+
whisper = [
|
| 27 |
+
"pywhispercpp>=1.3.0",
|
| 28 |
+
]
|
| 29 |
+
piper = [
|
| 30 |
+
"piper-tts>=1.3.0",
|
| 31 |
+
]
|
| 32 |
+
|
| 33 |
+
[tool.uv.sources]
|
| 34 |
+
inference = { workspace = true }
|
| 35 |
+
agent = { workspace = true }
|
| 36 |
+
|
| 37 |
+
[build-system]
|
| 38 |
+
requires = ["uv_build>=0.8.13,<0.9.0"]
|
| 39 |
+
build-backend = "uv_build"
|
libs/echocoach/src/echocoach/__init__.py
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""EchoCoach — local voice practice coach."""
|
| 2 |
+
|
| 3 |
+
from echocoach.models import CoachFeedback, EchoCoachResult
|
| 4 |
+
from echocoach.pipeline import run_echo_coach
|
| 5 |
+
|
| 6 |
+
__all__ = ["CoachFeedback", "EchoCoachResult", "run_echo_coach"]
|
libs/echocoach/src/echocoach/analysis/__init__.py
ADDED
|
File without changes
|
libs/echocoach/src/echocoach/analysis/charts.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Matplotlib charts for filler and pace visualization."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
import matplotlib
|
| 8 |
+
|
| 9 |
+
matplotlib.use("Agg")
|
| 10 |
+
import matplotlib.pyplot as plt # noqa: E402
|
| 11 |
+
|
| 12 |
+
from echocoach.models import FillerAnalysis, PaceAnalysis
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def render_filler_chart(analysis: FillerAnalysis, out_path: Path) -> Path:
|
| 16 |
+
out_path.parent.mkdir(parents=True, exist_ok=True)
|
| 17 |
+
if not analysis.counts:
|
| 18 |
+
fig, ax = plt.subplots(figsize=(6, 3))
|
| 19 |
+
ax.bar(["(none)"], [0], color="#4a90d9")
|
| 20 |
+
ax.set_title("Filler words")
|
| 21 |
+
ax.set_ylabel("Count")
|
| 22 |
+
else:
|
| 23 |
+
labels = list(analysis.counts.keys())
|
| 24 |
+
values = [analysis.counts[k] for k in labels]
|
| 25 |
+
fig, ax = plt.subplots(figsize=(6, 3))
|
| 26 |
+
ax.bar(labels, values, color="#e67e22")
|
| 27 |
+
ax.set_title("Filler words")
|
| 28 |
+
ax.set_ylabel("Count")
|
| 29 |
+
plt.xticks(rotation=35, ha="right")
|
| 30 |
+
fig.tight_layout()
|
| 31 |
+
fig.savefig(out_path, dpi=120)
|
| 32 |
+
plt.close(fig)
|
| 33 |
+
return out_path
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def render_pace_timeline(
|
| 37 |
+
transcript: str,
|
| 38 |
+
duration_seconds: float,
|
| 39 |
+
pace: PaceAnalysis,
|
| 40 |
+
out_path: Path,
|
| 41 |
+
*,
|
| 42 |
+
window_seconds: float = 10.0,
|
| 43 |
+
) -> Path:
|
| 44 |
+
out_path.parent.mkdir(parents=True, exist_ok=True)
|
| 45 |
+
words = transcript.split()
|
| 46 |
+
if not words or duration_seconds <= 0:
|
| 47 |
+
fig, ax = plt.subplots(figsize=(6, 3))
|
| 48 |
+
ax.plot([0, max(duration_seconds, 1)], [0, 0], color="#4a90d9")
|
| 49 |
+
ax.set_title("Words per minute (rolling)")
|
| 50 |
+
ax.set_xlabel("Seconds")
|
| 51 |
+
ax.set_ylabel("WPM")
|
| 52 |
+
ax.axhspan(pace.target_low, pace.target_high, alpha=0.15, color="green", label="Target band")
|
| 53 |
+
ax.legend(loc="upper right")
|
| 54 |
+
else:
|
| 55 |
+
n_windows = max(1, int(duration_seconds / window_seconds) + (1 if duration_seconds % window_seconds else 0))
|
| 56 |
+
times: list[float] = []
|
| 57 |
+
wpms: list[float] = []
|
| 58 |
+
words_per_sec = len(words) / duration_seconds
|
| 59 |
+
for i in range(n_windows):
|
| 60 |
+
start = i * window_seconds
|
| 61 |
+
end = min((i + 1) * window_seconds, duration_seconds)
|
| 62 |
+
window_dur = end - start
|
| 63 |
+
if window_dur <= 0:
|
| 64 |
+
continue
|
| 65 |
+
approx_words = words_per_sec * window_dur
|
| 66 |
+
wpm = approx_words / (window_dur / 60.0)
|
| 67 |
+
times.append((start + end) / 2)
|
| 68 |
+
wpms.append(wpm)
|
| 69 |
+
|
| 70 |
+
fig, ax = plt.subplots(figsize=(6, 3))
|
| 71 |
+
ax.plot(times, wpms, marker="o", color="#4a90d9", linewidth=2)
|
| 72 |
+
ax.axhspan(pace.target_low, pace.target_high, alpha=0.15, color="green", label="Target band")
|
| 73 |
+
ax.set_title("Words per minute (by segment)")
|
| 74 |
+
ax.set_xlabel("Seconds")
|
| 75 |
+
ax.set_ylabel("WPM")
|
| 76 |
+
ax.legend(loc="upper right")
|
| 77 |
+
|
| 78 |
+
fig.tight_layout()
|
| 79 |
+
fig.savefig(out_path, dpi=120)
|
| 80 |
+
plt.close(fig)
|
| 81 |
+
return out_path
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
def build_charts(
|
| 85 |
+
transcript: str,
|
| 86 |
+
duration_seconds: float,
|
| 87 |
+
fillers: FillerAnalysis,
|
| 88 |
+
pace: PaceAnalysis,
|
| 89 |
+
output_dir: Path,
|
| 90 |
+
run_id: str,
|
| 91 |
+
) -> tuple[Path, Path]:
|
| 92 |
+
filler_path = output_dir / f"{run_id}_fillers.png"
|
| 93 |
+
pace_path = output_dir / f"{run_id}_pace.png"
|
| 94 |
+
render_filler_chart(fillers, filler_path)
|
| 95 |
+
render_pace_timeline(transcript, duration_seconds, pace, pace_path)
|
| 96 |
+
return filler_path, pace_path
|
libs/echocoach/src/echocoach/analysis/fillers.py
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Filler word detection and HTML highlighting."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import re
|
| 6 |
+
|
| 7 |
+
from echocoach.models import FillerAnalysis, FillerSpan
|
| 8 |
+
|
| 9 |
+
DEFAULT_FILLERS = [
|
| 10 |
+
"um",
|
| 11 |
+
"uh",
|
| 12 |
+
"uhm",
|
| 13 |
+
"erm",
|
| 14 |
+
"like",
|
| 15 |
+
"you know",
|
| 16 |
+
"basically",
|
| 17 |
+
"actually",
|
| 18 |
+
"literally",
|
| 19 |
+
"sort of",
|
| 20 |
+
"kind of",
|
| 21 |
+
"i mean",
|
| 22 |
+
"right",
|
| 23 |
+
"okay so",
|
| 24 |
+
"so yeah",
|
| 25 |
+
]
|
| 26 |
+
|
| 27 |
+
# Longer phrases first so "you know" wins over "you"
|
| 28 |
+
_SORTED_FILLERS = sorted(DEFAULT_FILLERS, key=len, reverse=True)
|
| 29 |
+
_PATTERN = re.compile(
|
| 30 |
+
r"\b(" + "|".join(re.escape(f) for f in _SORTED_FILLERS) + r")\b",
|
| 31 |
+
re.IGNORECASE,
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def analyze_fillers(transcript: str, fillers: list[str] | None = None) -> FillerAnalysis:
|
| 36 |
+
if fillers is None:
|
| 37 |
+
pattern = _PATTERN
|
| 38 |
+
else:
|
| 39 |
+
ordered = sorted(fillers, key=len, reverse=True)
|
| 40 |
+
pattern = re.compile(
|
| 41 |
+
r"\b(" + "|".join(re.escape(f) for f in ordered) + r")\b",
|
| 42 |
+
re.IGNORECASE,
|
| 43 |
+
)
|
| 44 |
+
|
| 45 |
+
counts: dict[str, int] = {}
|
| 46 |
+
spans: list[FillerSpan] = []
|
| 47 |
+
for match in pattern.finditer(transcript):
|
| 48 |
+
word = match.group(1).lower()
|
| 49 |
+
counts[word] = counts.get(word, 0) + 1
|
| 50 |
+
spans.append(FillerSpan(start=match.start(), end=match.end(), word=word))
|
| 51 |
+
|
| 52 |
+
return FillerAnalysis(counts=counts, spans=spans, total=sum(counts.values()))
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def highlight_fillers_html(transcript: str, analysis: FillerAnalysis) -> str:
|
| 56 |
+
if not analysis.spans:
|
| 57 |
+
safe = _escape_html(transcript)
|
| 58 |
+
return f'<p style="line-height:1.6;">{safe}</p>'
|
| 59 |
+
|
| 60 |
+
parts: list[str] = []
|
| 61 |
+
cursor = 0
|
| 62 |
+
for span in sorted(analysis.spans, key=lambda s: s.start):
|
| 63 |
+
if span.start < cursor:
|
| 64 |
+
continue
|
| 65 |
+
parts.append(_escape_html(transcript[cursor : span.start]))
|
| 66 |
+
parts.append(
|
| 67 |
+
f'<mark style="background:#ffe08a;padding:0 2px;border-radius:3px;">'
|
| 68 |
+
f"{_escape_html(transcript[span.start : span.end])}</mark>"
|
| 69 |
+
)
|
| 70 |
+
cursor = span.end
|
| 71 |
+
parts.append(_escape_html(transcript[cursor:]))
|
| 72 |
+
body = "".join(parts)
|
| 73 |
+
return f'<p style="line-height:1.6;">{body}</p>'
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def _escape_html(text: str) -> str:
|
| 77 |
+
return (
|
| 78 |
+
text.replace("&", "&")
|
| 79 |
+
.replace("<", "<")
|
| 80 |
+
.replace(">", ">")
|
| 81 |
+
.replace('"', """)
|
| 82 |
+
)
|
libs/echocoach/src/echocoach/analysis/pace.py
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Speaking pace scoring."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from echocoach.audio_io import count_words
|
| 6 |
+
from echocoach.models import PaceAnalysis
|
| 7 |
+
|
| 8 |
+
TARGET_LOW = 120
|
| 9 |
+
TARGET_HIGH = 160
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def analyze_pace(
|
| 13 |
+
transcript: str,
|
| 14 |
+
duration_seconds: float,
|
| 15 |
+
*,
|
| 16 |
+
target_low: int = TARGET_LOW,
|
| 17 |
+
target_high: int = TARGET_HIGH,
|
| 18 |
+
) -> PaceAnalysis:
|
| 19 |
+
word_count = count_words(transcript)
|
| 20 |
+
if duration_seconds <= 0:
|
| 21 |
+
wpm = 0.0
|
| 22 |
+
else:
|
| 23 |
+
wpm = word_count / (duration_seconds / 60.0)
|
| 24 |
+
|
| 25 |
+
score, label = _score_wpm(wpm, target_low, target_high)
|
| 26 |
+
return PaceAnalysis(
|
| 27 |
+
word_count=word_count,
|
| 28 |
+
duration_seconds=duration_seconds,
|
| 29 |
+
wpm=round(wpm, 1),
|
| 30 |
+
score=score,
|
| 31 |
+
label=label,
|
| 32 |
+
target_low=target_low,
|
| 33 |
+
target_high=target_high,
|
| 34 |
+
)
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def _score_wpm(wpm: float, low: int, high: int) -> tuple[int, str]:
|
| 38 |
+
if wpm <= 0:
|
| 39 |
+
return 0, "No speech detected"
|
| 40 |
+
if low <= wpm <= high:
|
| 41 |
+
return 100, "Ideal pace"
|
| 42 |
+
if wpm < low:
|
| 43 |
+
ratio = wpm / low
|
| 44 |
+
score = max(20, int(100 * ratio))
|
| 45 |
+
return score, "Too slow — pick up the energy"
|
| 46 |
+
# too fast
|
| 47 |
+
overshoot = (wpm - high) / high
|
| 48 |
+
score = max(20, int(100 * (1.0 - min(overshoot, 0.8))))
|
| 49 |
+
return score, "Too fast — pause and breathe"
|
libs/echocoach/src/echocoach/asr/__init__.py
ADDED
|
File without changes
|
libs/echocoach/src/echocoach/asr/cohere.py
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Cohere Transcribe 2B ASR backend."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from echocoach.audio_io import TARGET_SAMPLE_RATE, load_audio_mono_16k
|
| 6 |
+
from echocoach.config import AsrPreset
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
class CohereAsrBackend:
|
| 10 |
+
def __init__(self, preset: AsrPreset) -> None:
|
| 11 |
+
self._preset = preset
|
| 12 |
+
self._processor = None
|
| 13 |
+
self._model = None
|
| 14 |
+
|
| 15 |
+
def _load(self) -> None:
|
| 16 |
+
if self._model is not None:
|
| 17 |
+
return
|
| 18 |
+
from transformers import AutoProcessor, CohereAsrForConditionalGeneration
|
| 19 |
+
|
| 20 |
+
model_id = self._preset.model_id or "CohereLabs/cohere-transcribe-03-2026"
|
| 21 |
+
self._processor = AutoProcessor.from_pretrained(model_id)
|
| 22 |
+
self._model = CohereAsrForConditionalGeneration.from_pretrained(
|
| 23 |
+
model_id,
|
| 24 |
+
device_map="auto",
|
| 25 |
+
)
|
| 26 |
+
|
| 27 |
+
def transcribe(self, audio_path: str, *, language: str) -> str:
|
| 28 |
+
self._load()
|
| 29 |
+
assert self._processor is not None
|
| 30 |
+
assert self._model is not None
|
| 31 |
+
|
| 32 |
+
audio, _ = load_audio_mono_16k(audio_path)
|
| 33 |
+
inputs = self._processor(
|
| 34 |
+
audio,
|
| 35 |
+
sampling_rate=TARGET_SAMPLE_RATE,
|
| 36 |
+
return_tensors="pt",
|
| 37 |
+
language=language,
|
| 38 |
+
)
|
| 39 |
+
audio_chunk_index = inputs.get("audio_chunk_index")
|
| 40 |
+
inputs = inputs.to(self._model.device, dtype=self._model.dtype)
|
| 41 |
+
|
| 42 |
+
outputs = self._model.generate(**inputs, max_new_tokens=512)
|
| 43 |
+
decoded = self._processor.decode(
|
| 44 |
+
outputs,
|
| 45 |
+
skip_special_tokens=True,
|
| 46 |
+
audio_chunk_index=audio_chunk_index,
|
| 47 |
+
language=language,
|
| 48 |
+
)
|
| 49 |
+
if isinstance(decoded, list):
|
| 50 |
+
return decoded[0].strip() if decoded else ""
|
| 51 |
+
return str(decoded).strip()
|
libs/echocoach/src/echocoach/asr/factory.py
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""ASR backend protocol and factory."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from typing import Protocol
|
| 6 |
+
|
| 7 |
+
from echocoach.config import AsrPreset, get_echo_coach_config
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
class AsrBackend(Protocol):
|
| 11 |
+
def transcribe(self, audio_path: str, *, language: str) -> str: ...
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
_asr_cache: dict[tuple, AsrBackend] = {}
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def get_asr_backend(preset_key: str | None = None) -> AsrBackend:
|
| 18 |
+
config = get_echo_coach_config()
|
| 19 |
+
preset = config.get_asr(preset_key)
|
| 20 |
+
cache_key = (preset.key, preset.backend, preset.model_id, preset.model_size)
|
| 21 |
+
if cache_key in _asr_cache:
|
| 22 |
+
return _asr_cache[cache_key]
|
| 23 |
+
|
| 24 |
+
if preset.backend == "cohere":
|
| 25 |
+
from echocoach.asr.cohere import CohereAsrBackend
|
| 26 |
+
|
| 27 |
+
backend: AsrBackend = CohereAsrBackend(preset)
|
| 28 |
+
elif preset.backend == "whisper_cpp":
|
| 29 |
+
from echocoach.asr.whisper_cpp import WhisperCppBackend
|
| 30 |
+
|
| 31 |
+
backend = WhisperCppBackend(preset)
|
| 32 |
+
else:
|
| 33 |
+
raise ValueError(f"Unsupported ASR backend: {preset.backend}")
|
| 34 |
+
|
| 35 |
+
_asr_cache[cache_key] = backend
|
| 36 |
+
return backend
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def reset_asr_backends() -> None:
|
| 40 |
+
_asr_cache.clear()
|
libs/echocoach/src/echocoach/asr/whisper_cpp.py
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Whisper.cpp ASR via pywhispercpp."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from echocoach.config import AsrPreset
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
class WhisperCppBackend:
|
| 9 |
+
def __init__(self, preset: AsrPreset) -> None:
|
| 10 |
+
self._preset = preset
|
| 11 |
+
self._model = None
|
| 12 |
+
|
| 13 |
+
def _load(self) -> None:
|
| 14 |
+
if self._model is not None:
|
| 15 |
+
return
|
| 16 |
+
try:
|
| 17 |
+
from pywhispercpp.model import Model
|
| 18 |
+
except ImportError as exc:
|
| 19 |
+
raise ImportError(
|
| 20 |
+
"Whisper.cpp backend requires pywhispercpp. "
|
| 21 |
+
"Install with: uv sync --package echocoach --extra whisper"
|
| 22 |
+
) from exc
|
| 23 |
+
|
| 24 |
+
size = self._preset.model_size or "tiny"
|
| 25 |
+
self._model = Model(size, print_realtime=False, print_progress=False)
|
| 26 |
+
|
| 27 |
+
def transcribe(self, audio_path: str, *, language: str) -> str:
|
| 28 |
+
self._load()
|
| 29 |
+
assert self._model is not None
|
| 30 |
+
|
| 31 |
+
segments = self._model.transcribe(audio_path, language=language)
|
| 32 |
+
parts = [seg.text.strip() for seg in segments if getattr(seg, "text", "")]
|
| 33 |
+
return " ".join(parts).strip()
|
libs/echocoach/src/echocoach/audio_io.py
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import re
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
import numpy as np
|
| 7 |
+
|
| 8 |
+
TARGET_SAMPLE_RATE = 16_000
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def load_audio_mono_16k(path: str | Path) -> tuple[np.ndarray, float]:
|
| 12 |
+
"""Load audio as mono float32 at 16 kHz; return (samples, duration_seconds)."""
|
| 13 |
+
import librosa
|
| 14 |
+
|
| 15 |
+
audio, _ = librosa.load(str(path), sr=TARGET_SAMPLE_RATE, mono=True)
|
| 16 |
+
duration = len(audio) / TARGET_SAMPLE_RATE if len(audio) else 0.0
|
| 17 |
+
return audio.astype(np.float32), duration
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def clamp_duration(audio: np.ndarray, max_seconds: float) -> np.ndarray:
|
| 21 |
+
max_samples = int(max_seconds * TARGET_SAMPLE_RATE)
|
| 22 |
+
if len(audio) > max_samples:
|
| 23 |
+
return audio[:max_samples]
|
| 24 |
+
return audio
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def write_wav_temp(audio: np.ndarray, directory: Path, stem: str = "clip") -> Path:
|
| 28 |
+
import soundfile as sf
|
| 29 |
+
|
| 30 |
+
directory.mkdir(parents=True, exist_ok=True)
|
| 31 |
+
out = directory / f"{stem}.wav"
|
| 32 |
+
sf.write(out, audio, TARGET_SAMPLE_RATE)
|
| 33 |
+
return out
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def count_words(text: str) -> int:
|
| 37 |
+
tokens = re.findall(r"\b[\w']+\b", text, flags=re.UNICODE)
|
| 38 |
+
return len(tokens)
|
libs/echocoach/src/echocoach/coach.py
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""LLM coaching prompts and parsing."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
from inference.base import InferenceBackend
|
| 6 |
+
from inference.response_clean import strip_reasoning_output
|
| 7 |
+
|
| 8 |
+
from echocoach.models import CoachFeedback, FillerAnalysis, PaceAnalysis
|
| 9 |
+
from echocoach.utils import extract_json
|
| 10 |
+
|
| 11 |
+
COACH_SYSTEM = """You are EchoCoach, a concise public-speaking coach for students and teachers.
|
| 12 |
+
Respond with a single JSON object only — no markdown fences, no extra text.
|
| 13 |
+
|
| 14 |
+
Required keys:
|
| 15 |
+
- summary: 1-2 sentence overall assessment
|
| 16 |
+
- filler_feedback: specific advice about filler words
|
| 17 |
+
- pace_feedback: specific advice about speaking pace
|
| 18 |
+
- rewrite: improved 2-4 sentence version of the pitch (same language as the transcript)
|
| 19 |
+
- one_tip: one actionable tip for the next attempt
|
| 20 |
+
"""
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def coach_user_prompt(
|
| 24 |
+
transcript: str,
|
| 25 |
+
fillers: FillerAnalysis,
|
| 26 |
+
pace: PaceAnalysis,
|
| 27 |
+
language: str,
|
| 28 |
+
) -> str:
|
| 29 |
+
filler_list = ", ".join(f"{k} ({v})" for k, v in fillers.counts.items()) or "none"
|
| 30 |
+
return f"""Language: {language}
|
| 31 |
+
Duration: {pace.duration_seconds:.1f}s
|
| 32 |
+
Word count: {pace.word_count}
|
| 33 |
+
Pace: {pace.wpm} WPM (target {pace.target_low}-{pace.target_high}) — {pace.label} (score {pace.score}/100)
|
| 34 |
+
Filler words: {fillers.total} total — {filler_list}
|
| 35 |
+
|
| 36 |
+
Transcript:
|
| 37 |
+
{transcript}
|
| 38 |
+
"""
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def run_coach(
|
| 42 |
+
backend: InferenceBackend,
|
| 43 |
+
transcript: str,
|
| 44 |
+
fillers: FillerAnalysis,
|
| 45 |
+
pace: PaceAnalysis,
|
| 46 |
+
language: str,
|
| 47 |
+
) -> tuple[CoachFeedback, str, str]:
|
| 48 |
+
user_content = coach_user_prompt(transcript, fillers, pace, language)
|
| 49 |
+
messages = [
|
| 50 |
+
{"role": "system", "content": COACH_SYSTEM},
|
| 51 |
+
{"role": "user", "content": user_content},
|
| 52 |
+
]
|
| 53 |
+
raw = backend.chat(messages, max_tokens=768, temperature=0.4)
|
| 54 |
+
raw = strip_reasoning_output(raw)
|
| 55 |
+
feedback = parse_coach_response(raw)
|
| 56 |
+
return feedback, COACH_SYSTEM, raw
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def parse_coach_response(raw: str) -> CoachFeedback:
|
| 60 |
+
data = extract_json(raw)
|
| 61 |
+
return CoachFeedback(
|
| 62 |
+
summary=str(data.get("summary", "")).strip(),
|
| 63 |
+
filler_feedback=str(data.get("filler_feedback", "")).strip(),
|
| 64 |
+
pace_feedback=str(data.get("pace_feedback", "")).strip(),
|
| 65 |
+
rewrite=str(data.get("rewrite", "")).strip(),
|
| 66 |
+
one_tip=str(data.get("one_tip", "")).strip(),
|
| 67 |
+
)
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def format_report_markdown(
|
| 71 |
+
coach: CoachFeedback,
|
| 72 |
+
fillers: FillerAnalysis,
|
| 73 |
+
pace: PaceAnalysis,
|
| 74 |
+
) -> str:
|
| 75 |
+
filler_lines = (
|
| 76 |
+
"\n".join(f"- **{word}**: {count}" for word, count in fillers.counts.items())
|
| 77 |
+
or "- None detected"
|
| 78 |
+
)
|
| 79 |
+
return f"""## Pace
|
| 80 |
+
|
| 81 |
+
- **Score:** {pace.score}/100 — {pace.label}
|
| 82 |
+
- **WPM:** {pace.wpm} (target {pace.target_low}–{pace.target_high})
|
| 83 |
+
- **Duration:** {pace.duration_seconds:.1f}s · **Words:** {pace.word_count}
|
| 84 |
+
|
| 85 |
+
## Fillers ({fillers.total})
|
| 86 |
+
|
| 87 |
+
{filler_lines}
|
| 88 |
+
|
| 89 |
+
## Coach summary
|
| 90 |
+
|
| 91 |
+
{coach.summary}
|
| 92 |
+
|
| 93 |
+
### Filler feedback
|
| 94 |
+
|
| 95 |
+
{coach.filler_feedback}
|
| 96 |
+
|
| 97 |
+
### Pace feedback
|
| 98 |
+
|
| 99 |
+
{coach.pace_feedback}
|
| 100 |
+
|
| 101 |
+
### Suggested rewrite
|
| 102 |
+
|
| 103 |
+
{coach.rewrite}
|
| 104 |
+
|
| 105 |
+
### One tip
|
| 106 |
+
|
| 107 |
+
{coach.one_tip}
|
| 108 |
+
"""
|
libs/echocoach/src/echocoach/config.py
ADDED
|
@@ -0,0 +1,228 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import os
|
| 5 |
+
import re
|
| 6 |
+
from dataclasses import dataclass, replace
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
from typing import Any, Literal
|
| 9 |
+
|
| 10 |
+
AsrBackendName = Literal["cohere", "whisper_cpp"]
|
| 11 |
+
TtsBackendName = Literal["piper"]
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
@dataclass(frozen=True)
|
| 15 |
+
class LanguageOption:
|
| 16 |
+
code: str
|
| 17 |
+
label: str
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
@dataclass(frozen=True)
|
| 21 |
+
class AsrPreset:
|
| 22 |
+
key: str
|
| 23 |
+
label: str
|
| 24 |
+
backend: AsrBackendName
|
| 25 |
+
model_id: str | None = None
|
| 26 |
+
model_size: str | None = None
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
@dataclass(frozen=True)
|
| 30 |
+
class TtsPreset:
|
| 31 |
+
key: str
|
| 32 |
+
label: str
|
| 33 |
+
backend: TtsBackendName
|
| 34 |
+
voices: dict[str, str]
|
| 35 |
+
fallback_voice: str
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
@dataclass(frozen=True)
|
| 39 |
+
class EchoCoachConfig:
|
| 40 |
+
asr_preset: str
|
| 41 |
+
tts_preset: str
|
| 42 |
+
coach_model: str
|
| 43 |
+
max_seconds: int
|
| 44 |
+
languages: list[LanguageOption]
|
| 45 |
+
asr_presets: dict[str, AsrPreset]
|
| 46 |
+
tts_presets: dict[str, TtsPreset]
|
| 47 |
+
presets_path: Path | None = None
|
| 48 |
+
|
| 49 |
+
def get_asr(self, key: str | None = None) -> AsrPreset:
|
| 50 |
+
preset_key = key or self.asr_preset
|
| 51 |
+
if preset_key not in self.asr_presets:
|
| 52 |
+
known = ", ".join(sorted(self.asr_presets))
|
| 53 |
+
raise KeyError(f"Unknown ASR preset {preset_key!r}. Known: {known}")
|
| 54 |
+
return self.asr_presets[preset_key]
|
| 55 |
+
|
| 56 |
+
def get_tts(self, key: str | None = None) -> TtsPreset:
|
| 57 |
+
preset_key = key or self.tts_preset
|
| 58 |
+
if preset_key not in self.tts_presets:
|
| 59 |
+
known = ", ".join(sorted(self.tts_presets))
|
| 60 |
+
raise KeyError(f"Unknown TTS preset {preset_key!r}. Known: {known}")
|
| 61 |
+
return self.tts_presets[preset_key]
|
| 62 |
+
|
| 63 |
+
def asr_choices(self) -> list[tuple[str, str]]:
|
| 64 |
+
return [(p.label, p.key) for p in self.asr_presets.values()]
|
| 65 |
+
|
| 66 |
+
def language_choices(self) -> list[tuple[str, str]]:
|
| 67 |
+
return [(lang.label, lang.code) for lang in self.languages]
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def _find_voice_presets_path() -> Path | None:
|
| 71 |
+
env_path = os.environ.get("VOICE_PRESETS_PATH")
|
| 72 |
+
if env_path:
|
| 73 |
+
path = Path(env_path)
|
| 74 |
+
if path.is_file():
|
| 75 |
+
return path.resolve()
|
| 76 |
+
|
| 77 |
+
for base in (Path.cwd(), *Path.cwd().parents):
|
| 78 |
+
candidate = base / "voice_models.yaml"
|
| 79 |
+
if candidate.is_file():
|
| 80 |
+
return candidate.resolve()
|
| 81 |
+
return None
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
def _builtin_config() -> EchoCoachConfig:
|
| 85 |
+
langs = [
|
| 86 |
+
LanguageOption("en", "English"),
|
| 87 |
+
LanguageOption("fr", "French"),
|
| 88 |
+
LanguageOption("de", "German"),
|
| 89 |
+
]
|
| 90 |
+
asr = {
|
| 91 |
+
"whisper-cpp-tiny": AsrPreset(
|
| 92 |
+
key="whisper-cpp-tiny",
|
| 93 |
+
label="Whisper.cpp tiny",
|
| 94 |
+
backend="whisper_cpp",
|
| 95 |
+
model_size="tiny",
|
| 96 |
+
),
|
| 97 |
+
}
|
| 98 |
+
tts = {
|
| 99 |
+
"piper-multilingual": TtsPreset(
|
| 100 |
+
key="piper-multilingual",
|
| 101 |
+
label="Piper TTS",
|
| 102 |
+
backend="piper",
|
| 103 |
+
voices={"en": "en_US-lessac-medium"},
|
| 104 |
+
fallback_voice="en_US-lessac-medium",
|
| 105 |
+
),
|
| 106 |
+
}
|
| 107 |
+
return EchoCoachConfig(
|
| 108 |
+
asr_preset="whisper-cpp-tiny",
|
| 109 |
+
tts_preset="piper-multilingual",
|
| 110 |
+
coach_model="minicpm5-1b",
|
| 111 |
+
max_seconds=30,
|
| 112 |
+
languages=langs,
|
| 113 |
+
asr_presets=asr,
|
| 114 |
+
tts_presets=tts,
|
| 115 |
+
)
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _parse_asr_entry(key: str, raw: dict[str, Any]) -> AsrPreset:
|
| 119 |
+
backend = str(raw.get("backend", "whisper_cpp"))
|
| 120 |
+
if backend not in ("cohere", "whisper_cpp"):
|
| 121 |
+
raise ValueError(f"ASR preset {key!r}: backend must be cohere or whisper_cpp")
|
| 122 |
+
return AsrPreset(
|
| 123 |
+
key=key,
|
| 124 |
+
label=str(raw.get("label", key)),
|
| 125 |
+
backend=backend, # type: ignore[arg-type]
|
| 126 |
+
model_id=raw.get("model_id"),
|
| 127 |
+
model_size=raw.get("model_size"),
|
| 128 |
+
)
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def _parse_tts_entry(key: str, raw: dict[str, Any]) -> TtsPreset:
|
| 132 |
+
backend = str(raw.get("backend", "piper"))
|
| 133 |
+
if backend != "piper":
|
| 134 |
+
raise ValueError(f"TTS preset {key!r}: only piper backend is supported in MVP")
|
| 135 |
+
voices = raw.get("voices") or {}
|
| 136 |
+
if not isinstance(voices, dict) or not voices:
|
| 137 |
+
raise ValueError(f"TTS preset {key!r}: voices mapping is required")
|
| 138 |
+
return TtsPreset(
|
| 139 |
+
key=key,
|
| 140 |
+
label=str(raw.get("label", key)),
|
| 141 |
+
backend="piper",
|
| 142 |
+
voices={str(k): str(v) for k, v in voices.items()},
|
| 143 |
+
fallback_voice=str(raw.get("fallback_voice", "en_US-lessac-medium")),
|
| 144 |
+
)
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
def load_echo_coach_config() -> EchoCoachConfig:
|
| 148 |
+
presets_path = _find_voice_presets_path()
|
| 149 |
+
if presets_path is None:
|
| 150 |
+
config = _builtin_config()
|
| 151 |
+
else:
|
| 152 |
+
try:
|
| 153 |
+
import yaml
|
| 154 |
+
except ImportError as exc:
|
| 155 |
+
raise ImportError(
|
| 156 |
+
"Loading voice_models.yaml requires PyYAML. Install with: uv sync"
|
| 157 |
+
) from exc
|
| 158 |
+
|
| 159 |
+
data = yaml.safe_load(presets_path.read_text()) or {}
|
| 160 |
+
defaults = data.get("defaults", {})
|
| 161 |
+
raw_langs = data.get("languages", [])
|
| 162 |
+
raw_asr = data.get("asr", {})
|
| 163 |
+
raw_tts = data.get("tts", {})
|
| 164 |
+
|
| 165 |
+
languages = [
|
| 166 |
+
LanguageOption(code=str(item["code"]), label=str(item["label"]))
|
| 167 |
+
for item in raw_langs
|
| 168 |
+
]
|
| 169 |
+
asr_presets = {
|
| 170 |
+
key: _parse_asr_entry(key, value) for key, value in raw_asr.items()
|
| 171 |
+
}
|
| 172 |
+
tts_presets = {
|
| 173 |
+
key: _parse_tts_entry(key, value) for key, value in raw_tts.items()
|
| 174 |
+
}
|
| 175 |
+
|
| 176 |
+
asr_default = defaults.get("asr_preset", "whisper-cpp-tiny")
|
| 177 |
+
tts_default = defaults.get("tts_preset", "piper-multilingual")
|
| 178 |
+
if asr_default not in asr_presets:
|
| 179 |
+
asr_default = next(iter(asr_presets))
|
| 180 |
+
if tts_default not in tts_presets:
|
| 181 |
+
tts_default = next(iter(tts_presets))
|
| 182 |
+
|
| 183 |
+
config = EchoCoachConfig(
|
| 184 |
+
asr_preset=asr_default,
|
| 185 |
+
tts_preset=tts_default,
|
| 186 |
+
coach_model=str(defaults.get("coach_model", "minicpm5-1b")),
|
| 187 |
+
max_seconds=int(defaults.get("max_seconds", 30)),
|
| 188 |
+
languages=languages,
|
| 189 |
+
asr_presets=asr_presets,
|
| 190 |
+
tts_presets=tts_presets,
|
| 191 |
+
presets_path=presets_path,
|
| 192 |
+
)
|
| 193 |
+
|
| 194 |
+
updates: dict[str, Any] = {}
|
| 195 |
+
if os.environ.get("ECHOCOACH_ASR_PRESET"):
|
| 196 |
+
updates["asr_preset"] = os.environ["ECHOCOACH_ASR_PRESET"]
|
| 197 |
+
if os.environ.get("ECHOCOACH_TTS_PRESET"):
|
| 198 |
+
updates["tts_preset"] = os.environ["ECHOCOACH_TTS_PRESET"]
|
| 199 |
+
if os.environ.get("ECHOCOACH_COACH_MODEL"):
|
| 200 |
+
updates["coach_model"] = os.environ["ECHOCOACH_COACH_MODEL"]
|
| 201 |
+
if os.environ.get("ECHOCOACH_MAX_SECONDS"):
|
| 202 |
+
updates["max_seconds"] = int(os.environ["ECHOCOACH_MAX_SECONDS"])
|
| 203 |
+
|
| 204 |
+
return replace(config, **updates) if updates else config
|
| 205 |
+
|
| 206 |
+
|
| 207 |
+
_config: EchoCoachConfig | None = None
|
| 208 |
+
|
| 209 |
+
|
| 210 |
+
def get_echo_coach_config(reload: bool = False) -> EchoCoachConfig:
|
| 211 |
+
global _config
|
| 212 |
+
if _config is None or reload:
|
| 213 |
+
_config = load_echo_coach_config()
|
| 214 |
+
return _config
|
| 215 |
+
|
| 216 |
+
|
| 217 |
+
def outputs_dir() -> Path:
|
| 218 |
+
env = os.environ.get("AGENT_OUTPUTS_DIR")
|
| 219 |
+
if env:
|
| 220 |
+
return Path(env)
|
| 221 |
+
for base in (Path.cwd(), *Path.cwd().parents):
|
| 222 |
+
if (base / "voice_models.yaml").is_file() or (base / "pyproject.toml").is_file():
|
| 223 |
+
path = base / "outputs" / "echocoach"
|
| 224 |
+
path.mkdir(parents=True, exist_ok=True)
|
| 225 |
+
return path
|
| 226 |
+
path = Path("/tmp/echocoach_outputs")
|
| 227 |
+
path.mkdir(parents=True, exist_ok=True)
|
| 228 |
+
return path
|
libs/echocoach/src/echocoach/models.py
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import re
|
| 5 |
+
from dataclasses import dataclass, field
|
| 6 |
+
from typing import Any
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
@dataclass(frozen=True)
|
| 10 |
+
class FillerSpan:
|
| 11 |
+
start: int
|
| 12 |
+
end: int
|
| 13 |
+
word: str
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
@dataclass(frozen=True)
|
| 17 |
+
class FillerAnalysis:
|
| 18 |
+
counts: dict[str, int]
|
| 19 |
+
spans: list[FillerSpan]
|
| 20 |
+
total: int
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
@dataclass(frozen=True)
|
| 24 |
+
class PaceAnalysis:
|
| 25 |
+
word_count: int
|
| 26 |
+
duration_seconds: float
|
| 27 |
+
wpm: float
|
| 28 |
+
score: int
|
| 29 |
+
label: str
|
| 30 |
+
target_low: int = 120
|
| 31 |
+
target_high: int = 160
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
@dataclass(frozen=True)
|
| 35 |
+
class CoachFeedback:
|
| 36 |
+
summary: str
|
| 37 |
+
filler_feedback: str
|
| 38 |
+
pace_feedback: str
|
| 39 |
+
rewrite: str
|
| 40 |
+
one_tip: str
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
@dataclass
|
| 44 |
+
class EchoCoachResult:
|
| 45 |
+
transcript: str
|
| 46 |
+
transcript_html: str
|
| 47 |
+
language: str
|
| 48 |
+
duration_seconds: float
|
| 49 |
+
fillers: FillerAnalysis
|
| 50 |
+
pace: PaceAnalysis
|
| 51 |
+
coach: CoachFeedback
|
| 52 |
+
report_markdown: str
|
| 53 |
+
filler_chart_path: str | None
|
| 54 |
+
pace_chart_path: str | None
|
| 55 |
+
voiceout_path: str | None
|
| 56 |
+
voiceout_warning: str | None
|
| 57 |
+
trace_path: str
|
| 58 |
+
trace: dict[str, Any] = field(default_factory=dict)
|
libs/echocoach/src/echocoach/pipeline.py
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""End-to-end EchoCoach pipeline."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import uuid
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
from typing import Any
|
| 8 |
+
|
| 9 |
+
from agent.trace import TraceRecorder
|
| 10 |
+
from inference.base import InferenceBackend
|
| 11 |
+
|
| 12 |
+
from echocoach.analysis.charts import build_charts
|
| 13 |
+
from echocoach.analysis.fillers import analyze_fillers, highlight_fillers_html
|
| 14 |
+
from echocoach.analysis.pace import analyze_pace
|
| 15 |
+
from echocoach.asr.factory import get_asr_backend
|
| 16 |
+
from echocoach.audio_io import clamp_duration, load_audio_mono_16k, write_wav_temp
|
| 17 |
+
from echocoach.coach import format_report_markdown, run_coach
|
| 18 |
+
from echocoach.config import get_echo_coach_config, outputs_dir
|
| 19 |
+
from echocoach.models import EchoCoachResult
|
| 20 |
+
from echocoach.tts.piper import get_tts_backend
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def run_echo_coach(
|
| 24 |
+
audio_path: str,
|
| 25 |
+
*,
|
| 26 |
+
language: str = "en",
|
| 27 |
+
asr_preset: str | None = None,
|
| 28 |
+
tts_preset: str | None = None,
|
| 29 |
+
coach_model: str | None = None,
|
| 30 |
+
backend: InferenceBackend,
|
| 31 |
+
speak_rewrite: bool = False,
|
| 32 |
+
) -> EchoCoachResult:
|
| 33 |
+
if not audio_path:
|
| 34 |
+
raise ValueError("No audio recording provided.")
|
| 35 |
+
|
| 36 |
+
config = get_echo_coach_config()
|
| 37 |
+
asr_key = asr_preset or config.asr_preset
|
| 38 |
+
tts_key = tts_preset or config.tts_preset
|
| 39 |
+
model_key = coach_model or config.coach_model
|
| 40 |
+
run_id = uuid.uuid4().hex[:12]
|
| 41 |
+
out_base = outputs_dir()
|
| 42 |
+
|
| 43 |
+
trace = TraceRecorder(
|
| 44 |
+
skill="echo-coach",
|
| 45 |
+
model=model_key,
|
| 46 |
+
user_input={
|
| 47 |
+
"language": language,
|
| 48 |
+
"asr_preset": asr_key,
|
| 49 |
+
"tts_preset": tts_key,
|
| 50 |
+
"audio_path": audio_path,
|
| 51 |
+
"speak_rewrite": speak_rewrite,
|
| 52 |
+
},
|
| 53 |
+
run_id=run_id,
|
| 54 |
+
)
|
| 55 |
+
|
| 56 |
+
audio, duration = load_audio_mono_16k(audio_path)
|
| 57 |
+
audio = clamp_duration(audio, config.max_seconds)
|
| 58 |
+
duration = len(audio) / 16_000
|
| 59 |
+
clipped_path = write_wav_temp(audio, out_base / "clips", stem=run_id)
|
| 60 |
+
|
| 61 |
+
trace.log_note("audio_loaded", duration_seconds=duration, path=str(clipped_path))
|
| 62 |
+
|
| 63 |
+
asr = get_asr_backend(asr_key)
|
| 64 |
+
transcript = asr.transcribe(str(clipped_path), language=language)
|
| 65 |
+
trace.log_note("asr_complete", preset=asr_key, chars=len(transcript))
|
| 66 |
+
|
| 67 |
+
fillers = analyze_fillers(transcript)
|
| 68 |
+
pace = analyze_pace(transcript, duration)
|
| 69 |
+
transcript_html = highlight_fillers_html(transcript, fillers)
|
| 70 |
+
|
| 71 |
+
filler_chart, pace_chart = build_charts(
|
| 72 |
+
transcript,
|
| 73 |
+
duration,
|
| 74 |
+
fillers,
|
| 75 |
+
pace,
|
| 76 |
+
out_base / "charts",
|
| 77 |
+
run_id,
|
| 78 |
+
)
|
| 79 |
+
|
| 80 |
+
coach_feedback, system_prompt, coach_raw = run_coach(
|
| 81 |
+
backend,
|
| 82 |
+
transcript,
|
| 83 |
+
fillers,
|
| 84 |
+
pace,
|
| 85 |
+
language,
|
| 86 |
+
)
|
| 87 |
+
trace.log_llm(system_prompt + "\n\n" + coach_user_for_trace(transcript, fillers, pace, language), coach_raw)
|
| 88 |
+
|
| 89 |
+
report = format_report_markdown(coach_feedback, fillers, pace)
|
| 90 |
+
|
| 91 |
+
voice_text = coach_feedback.rewrite if speak_rewrite else (
|
| 92 |
+
f"{coach_feedback.summary} {coach_feedback.one_tip}".strip()
|
| 93 |
+
)
|
| 94 |
+
tts = get_tts_backend(tts_key)
|
| 95 |
+
voiceout_path, voiceout_warning = tts.synthesize(
|
| 96 |
+
voice_text,
|
| 97 |
+
language=language,
|
| 98 |
+
out_dir=out_base / "voiceout",
|
| 99 |
+
)
|
| 100 |
+
if voiceout_path:
|
| 101 |
+
trace.set_artifact(voiceout_path)
|
| 102 |
+
|
| 103 |
+
trace_path = trace.save()
|
| 104 |
+
trace_dict: dict[str, Any] = trace.to_dict()
|
| 105 |
+
|
| 106 |
+
return EchoCoachResult(
|
| 107 |
+
transcript=transcript,
|
| 108 |
+
transcript_html=transcript_html,
|
| 109 |
+
language=language,
|
| 110 |
+
duration_seconds=duration,
|
| 111 |
+
fillers=fillers,
|
| 112 |
+
pace=pace,
|
| 113 |
+
coach=coach_feedback,
|
| 114 |
+
report_markdown=report,
|
| 115 |
+
filler_chart_path=str(filler_chart),
|
| 116 |
+
pace_chart_path=str(pace_chart),
|
| 117 |
+
voiceout_path=voiceout_path,
|
| 118 |
+
voiceout_warning=voiceout_warning,
|
| 119 |
+
trace_path=str(trace_path),
|
| 120 |
+
trace=trace_dict,
|
| 121 |
+
)
|
| 122 |
+
|
| 123 |
+
|
| 124 |
+
def coach_user_for_trace(transcript: str, fillers, pace, language: str) -> str:
|
| 125 |
+
from echocoach.coach import coach_user_prompt
|
| 126 |
+
|
| 127 |
+
return coach_user_prompt(transcript, fillers, pace, language)
|
libs/echocoach/src/echocoach/recording.py
ADDED
|
@@ -0,0 +1,348 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Record audio from the server's local microphone (bypasses browser getUserMedia)."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import os
|
| 6 |
+
import shutil
|
| 7 |
+
import signal
|
| 8 |
+
import subprocess
|
| 9 |
+
import threading
|
| 10 |
+
import time
|
| 11 |
+
import uuid
|
| 12 |
+
from dataclasses import dataclass
|
| 13 |
+
from pathlib import Path
|
| 14 |
+
from typing import Literal
|
| 15 |
+
|
| 16 |
+
from echocoach.audio_io import TARGET_SAMPLE_RATE, load_audio_mono_16k
|
| 17 |
+
from echocoach.config import get_echo_coach_config, outputs_dir
|
| 18 |
+
|
| 19 |
+
CaptureBackend = Literal["pw-record", "sounddevice", "arecord"]
|
| 20 |
+
SILENT_RMS_THRESHOLD = 0.002
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
class ServerRecordingError(RuntimeError):
|
| 24 |
+
"""Raised when server-side capture is unavailable or fails."""
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
@dataclass
|
| 28 |
+
class _RecordingSession:
|
| 29 |
+
process: subprocess.Popen[bytes]
|
| 30 |
+
out_path: Path
|
| 31 |
+
backend: CaptureBackend
|
| 32 |
+
max_seconds: int
|
| 33 |
+
started_at: float
|
| 34 |
+
watchdog: threading.Timer | None = None
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
_session: _RecordingSession | None = None
|
| 38 |
+
_last_recording_path: Path | None = None
|
| 39 |
+
_session_lock = threading.Lock()
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def _capture_device() -> str | None:
|
| 43 |
+
device = os.environ.get("ECHOCOACH_CAPTURE_DEVICE", "").strip()
|
| 44 |
+
return device or None
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def _pw_record_available() -> bool:
|
| 48 |
+
return shutil.which("pw-record") is not None
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _sounddevice_available() -> bool:
|
| 52 |
+
try:
|
| 53 |
+
import sounddevice as sd # noqa: PLC0415
|
| 54 |
+
except (ImportError, OSError):
|
| 55 |
+
return False
|
| 56 |
+
try:
|
| 57 |
+
sd.query_devices()
|
| 58 |
+
except Exception:
|
| 59 |
+
return False
|
| 60 |
+
return True
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def _arecord_available() -> bool:
|
| 64 |
+
if shutil.which("arecord") is None:
|
| 65 |
+
return False
|
| 66 |
+
try:
|
| 67 |
+
result = subprocess.run(
|
| 68 |
+
["arecord", "-l"],
|
| 69 |
+
capture_output=True,
|
| 70 |
+
text=True,
|
| 71 |
+
timeout=3,
|
| 72 |
+
check=False,
|
| 73 |
+
)
|
| 74 |
+
except (OSError, subprocess.TimeoutExpired):
|
| 75 |
+
return False
|
| 76 |
+
output = f"{result.stdout}\n{result.stderr}".lower()
|
| 77 |
+
return "card" in output
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def select_capture_backend() -> CaptureBackend | None:
|
| 81 |
+
"""Pick the best capture tool for this machine."""
|
| 82 |
+
if _pw_record_available():
|
| 83 |
+
return "pw-record"
|
| 84 |
+
if _sounddevice_available():
|
| 85 |
+
return "sounddevice"
|
| 86 |
+
if _arecord_available():
|
| 87 |
+
return "arecord"
|
| 88 |
+
return None
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def recording_backend_status() -> str:
|
| 92 |
+
backend = select_capture_backend()
|
| 93 |
+
device = _capture_device()
|
| 94 |
+
if backend == "pw-record":
|
| 95 |
+
note = "PipeWire pw-record"
|
| 96 |
+
elif backend == "sounddevice":
|
| 97 |
+
note = "sounddevice / PortAudio"
|
| 98 |
+
elif backend == "arecord":
|
| 99 |
+
note = "ALSA arecord"
|
| 100 |
+
else:
|
| 101 |
+
note = None
|
| 102 |
+
|
| 103 |
+
if note:
|
| 104 |
+
extra = f" (device: `{device}`)" if device else ""
|
| 105 |
+
return f"Server microphone: ready ({note}{extra}). Click **Start recording**, speak, then **Stop recording**."
|
| 106 |
+
|
| 107 |
+
hints: list[str] = []
|
| 108 |
+
if not _pw_record_available() and not _arecord_available():
|
| 109 |
+
hints.append("install PipeWire (`pw-record`) or ALSA utils (`arecord`)")
|
| 110 |
+
elif not _arecord_available():
|
| 111 |
+
hints.append("enable a microphone in system sound settings")
|
| 112 |
+
if not _sounddevice_available():
|
| 113 |
+
hints.append(
|
| 114 |
+
"optional: `sudo apt install libportaudio2` for sounddevice fallback"
|
| 115 |
+
)
|
| 116 |
+
hint = "; ".join(hints) if hints else "no capture backend available"
|
| 117 |
+
return (
|
| 118 |
+
f"Server microphone: unavailable — {hint}. "
|
| 119 |
+
"Use **Upload** or open **http://localhost:7860** in Chrome/Firefox for browser mic."
|
| 120 |
+
)
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
def is_recording_active() -> bool:
|
| 124 |
+
with _session_lock:
|
| 125 |
+
session = _session
|
| 126 |
+
return session is not None and session.process.poll() is None
|
| 127 |
+
|
| 128 |
+
|
| 129 |
+
def recording_elapsed_seconds() -> float:
|
| 130 |
+
with _session_lock:
|
| 131 |
+
session = _session
|
| 132 |
+
if session is None:
|
| 133 |
+
return 0.0
|
| 134 |
+
return max(0.0, time.monotonic() - session.started_at)
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def start_server_recording(max_seconds: int | None = None) -> None:
|
| 138 |
+
"""Begin an open-ended capture; call stop_server_recording() to finish."""
|
| 139 |
+
global _session, _last_recording_path
|
| 140 |
+
|
| 141 |
+
config = get_echo_coach_config()
|
| 142 |
+
seconds = max_seconds if max_seconds is not None else config.max_seconds
|
| 143 |
+
if seconds <= 0:
|
| 144 |
+
raise ServerRecordingError("Recording duration must be positive.")
|
| 145 |
+
|
| 146 |
+
backend = select_capture_backend()
|
| 147 |
+
if backend is None:
|
| 148 |
+
raise ServerRecordingError(recording_backend_status())
|
| 149 |
+
|
| 150 |
+
out_dir = outputs_dir() / "recordings"
|
| 151 |
+
out_dir.mkdir(parents=True, exist_ok=True)
|
| 152 |
+
out_path = out_dir / f"server_{uuid.uuid4().hex[:8]}.wav"
|
| 153 |
+
|
| 154 |
+
with _session_lock:
|
| 155 |
+
if _session is not None and _session.process.poll() is None:
|
| 156 |
+
raise ServerRecordingError("Already recording. Click **Stop recording** first.")
|
| 157 |
+
|
| 158 |
+
_last_recording_path = None
|
| 159 |
+
process = _spawn_capture_process(backend, out_path)
|
| 160 |
+
watchdog = threading.Timer(seconds, _auto_stop_recording)
|
| 161 |
+
watchdog.daemon = True
|
| 162 |
+
watchdog.start()
|
| 163 |
+
_session = _RecordingSession(
|
| 164 |
+
process=process,
|
| 165 |
+
out_path=out_path,
|
| 166 |
+
backend=backend,
|
| 167 |
+
max_seconds=seconds,
|
| 168 |
+
started_at=time.monotonic(),
|
| 169 |
+
watchdog=watchdog,
|
| 170 |
+
)
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
def stop_server_recording() -> Path:
|
| 174 |
+
"""Stop the active capture and return the WAV path."""
|
| 175 |
+
global _last_recording_path
|
| 176 |
+
|
| 177 |
+
with _session_lock:
|
| 178 |
+
session = _session
|
| 179 |
+
if session is None or session.process.poll() is not None:
|
| 180 |
+
if _last_recording_path is not None and _last_recording_path.is_file():
|
| 181 |
+
path = _last_recording_path
|
| 182 |
+
_last_recording_path = None
|
| 183 |
+
return path
|
| 184 |
+
raise ServerRecordingError("Not recording. Click **Start recording** first.")
|
| 185 |
+
return _finalize_session(session)
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
def _auto_stop_recording() -> None:
|
| 189 |
+
with _session_lock:
|
| 190 |
+
session = _session
|
| 191 |
+
if session is None or session.process.poll() is not None:
|
| 192 |
+
return
|
| 193 |
+
try:
|
| 194 |
+
_finalize_session(session)
|
| 195 |
+
except ServerRecordingError:
|
| 196 |
+
pass
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
def _finalize_session(session: _RecordingSession) -> Path:
|
| 200 |
+
global _session, _last_recording_path
|
| 201 |
+
|
| 202 |
+
if session.watchdog is not None:
|
| 203 |
+
session.watchdog.cancel()
|
| 204 |
+
|
| 205 |
+
process = session.process
|
| 206 |
+
if process.poll() is None:
|
| 207 |
+
process.send_signal(signal.SIGINT)
|
| 208 |
+
try:
|
| 209 |
+
process.wait(timeout=5)
|
| 210 |
+
except subprocess.TimeoutExpired:
|
| 211 |
+
process.kill()
|
| 212 |
+
process.wait(timeout=2)
|
| 213 |
+
|
| 214 |
+
out_path = session.out_path
|
| 215 |
+
if out_path.is_file() and out_path.stat().st_size > 44:
|
| 216 |
+
_session = None
|
| 217 |
+
_last_recording_path = out_path
|
| 218 |
+
return out_path
|
| 219 |
+
|
| 220 |
+
ok_codes = {
|
| 221 |
+
0,
|
| 222 |
+
1, # pw-record often exits 1 after SIGINT even with a valid WAV
|
| 223 |
+
-signal.SIGINT,
|
| 224 |
+
128 + signal.SIGINT,
|
| 225 |
+
-signal.SIGTERM,
|
| 226 |
+
128 + signal.SIGTERM,
|
| 227 |
+
}
|
| 228 |
+
if process.returncode not in ok_codes:
|
| 229 |
+
detail = ""
|
| 230 |
+
if process.stderr:
|
| 231 |
+
detail = process.stderr.read().decode("utf-8", errors="replace").strip()
|
| 232 |
+
msg = f"Capture stopped with exit code {process.returncode}"
|
| 233 |
+
if detail:
|
| 234 |
+
msg = f"{msg}: {detail}"
|
| 235 |
+
_session = None
|
| 236 |
+
_last_recording_path = None
|
| 237 |
+
raise ServerRecordingError(msg)
|
| 238 |
+
|
| 239 |
+
if not out_path.is_file() or out_path.stat().st_size == 0:
|
| 240 |
+
_session = None
|
| 241 |
+
_last_recording_path = None
|
| 242 |
+
raise ServerRecordingError("Recording finished but produced an empty file.")
|
| 243 |
+
|
| 244 |
+
_session = None
|
| 245 |
+
_last_recording_path = out_path
|
| 246 |
+
return out_path
|
| 247 |
+
|
| 248 |
+
|
| 249 |
+
def analyze_recording_levels(path: str | Path) -> tuple[float, float, float]:
|
| 250 |
+
audio, duration = load_audio_mono_16k(path)
|
| 251 |
+
if len(audio) == 0:
|
| 252 |
+
return 0.0, 0.0, duration
|
| 253 |
+
import numpy as np
|
| 254 |
+
|
| 255 |
+
rms = float(np.sqrt(np.mean(np.square(audio))))
|
| 256 |
+
peak = float(np.max(np.abs(audio)))
|
| 257 |
+
return rms, peak, duration
|
| 258 |
+
|
| 259 |
+
|
| 260 |
+
def recording_level_warning(path: str | Path) -> str | None:
|
| 261 |
+
rms, peak, duration = analyze_recording_levels(path)
|
| 262 |
+
if duration < 0.2:
|
| 263 |
+
return "Clip is very short — try recording a bit longer."
|
| 264 |
+
if rms < SILENT_RMS_THRESHOLD and peak < SILENT_RMS_THRESHOLD * 4:
|
| 265 |
+
return (
|
| 266 |
+
"Recording looks silent. Check system mic input/mute, pick the right input device "
|
| 267 |
+
"(set `ECHOCOACH_CAPTURE_DEVICE`), or use **Upload**."
|
| 268 |
+
)
|
| 269 |
+
return None
|
| 270 |
+
|
| 271 |
+
|
| 272 |
+
def record_server_wav(
|
| 273 |
+
max_seconds: int | None = None,
|
| 274 |
+
*,
|
| 275 |
+
sample_rate: int = TARGET_SAMPLE_RATE,
|
| 276 |
+
) -> Path:
|
| 277 |
+
"""Fixed-length capture (used in tests and scripts)."""
|
| 278 |
+
start_server_recording(max_seconds)
|
| 279 |
+
time.sleep(max_seconds if max_seconds is not None else get_echo_coach_config().max_seconds)
|
| 280 |
+
return stop_server_recording()
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
def _spawn_capture_process(backend: CaptureBackend, out_path: Path) -> subprocess.Popen[bytes]:
|
| 284 |
+
device = _capture_device()
|
| 285 |
+
if backend == "pw-record":
|
| 286 |
+
cmd = [
|
| 287 |
+
"pw-record",
|
| 288 |
+
"--media-category",
|
| 289 |
+
"Capture",
|
| 290 |
+
"--media-role",
|
| 291 |
+
"Speech",
|
| 292 |
+
"--rate",
|
| 293 |
+
str(TARGET_SAMPLE_RATE),
|
| 294 |
+
"--channels",
|
| 295 |
+
"1",
|
| 296 |
+
"--format",
|
| 297 |
+
"s16",
|
| 298 |
+
]
|
| 299 |
+
if device:
|
| 300 |
+
cmd.extend(["--target", device])
|
| 301 |
+
cmd.append(str(out_path))
|
| 302 |
+
elif backend == "arecord":
|
| 303 |
+
cmd = [
|
| 304 |
+
"arecord",
|
| 305 |
+
"-q",
|
| 306 |
+
"-f",
|
| 307 |
+
"S16_LE",
|
| 308 |
+
"-r",
|
| 309 |
+
str(TARGET_SAMPLE_RATE),
|
| 310 |
+
"-c",
|
| 311 |
+
"1",
|
| 312 |
+
]
|
| 313 |
+
if device:
|
| 314 |
+
cmd.extend(["-D", device])
|
| 315 |
+
else:
|
| 316 |
+
cmd.extend(["-D", "pipewire"])
|
| 317 |
+
cmd.append(str(out_path))
|
| 318 |
+
else:
|
| 319 |
+
raise ServerRecordingError(
|
| 320 |
+
"sounddevice does not support open-ended capture yet; install `pw-record` or use arecord."
|
| 321 |
+
)
|
| 322 |
+
|
| 323 |
+
try:
|
| 324 |
+
return subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
|
| 325 |
+
except OSError as exc:
|
| 326 |
+
raise ServerRecordingError(f"Failed to start {backend}: {exc}") from exc
|
| 327 |
+
|
| 328 |
+
|
| 329 |
+
def _record_sounddevice(out_path: Path, seconds: int, sample_rate: int) -> None:
|
| 330 |
+
import numpy as np
|
| 331 |
+
import sounddevice as sd
|
| 332 |
+
import soundfile as sf
|
| 333 |
+
|
| 334 |
+
frames = int(seconds * sample_rate)
|
| 335 |
+
device = _capture_device()
|
| 336 |
+
try:
|
| 337 |
+
recording = sd.rec(
|
| 338 |
+
frames,
|
| 339 |
+
samplerate=sample_rate,
|
| 340 |
+
channels=1,
|
| 341 |
+
dtype="float32",
|
| 342 |
+
device=device,
|
| 343 |
+
)
|
| 344 |
+
sd.wait()
|
| 345 |
+
except Exception as exc: # noqa: BLE001 — surface device errors to UI
|
| 346 |
+
raise ServerRecordingError(f"sounddevice capture failed: {exc}") from exc
|
| 347 |
+
|
| 348 |
+
sf.write(out_path, np.squeeze(recording), sample_rate)
|
libs/echocoach/src/echocoach/tts/__init__.py
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from echocoach.tts.piper import PiperTtsBackend, get_tts_backend
|
| 2 |
+
|
| 3 |
+
__all__ = ["PiperTtsBackend", "get_tts_backend"]
|
libs/echocoach/src/echocoach/tts/piper.py
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""TTS VoiceOut backends."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import uuid
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
from typing import Protocol
|
| 8 |
+
|
| 9 |
+
from echocoach.config import TtsPreset, get_echo_coach_config, outputs_dir
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
class TtsBackend(Protocol):
|
| 13 |
+
def synthesize(self, text: str, *, language: str, out_dir: Path | None = None) -> tuple[str | None, str | None]: ...
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
_tts_cache: dict[str, "PiperTtsBackend"] = {}
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
class PiperTtsBackend:
|
| 20 |
+
def __init__(self, preset: TtsPreset) -> None:
|
| 21 |
+
self._preset = preset
|
| 22 |
+
self._voices: dict[str, object] = {}
|
| 23 |
+
|
| 24 |
+
def _voice_name(self, language: str) -> tuple[str, str | None]:
|
| 25 |
+
voice = self._preset.voices.get(language)
|
| 26 |
+
warning = None
|
| 27 |
+
if not voice:
|
| 28 |
+
voice = self._preset.fallback_voice
|
| 29 |
+
warning = f"No Piper voice for {language!r}; using {voice}."
|
| 30 |
+
return voice, warning
|
| 31 |
+
|
| 32 |
+
def _load_voice(self, voice_name: str):
|
| 33 |
+
if voice_name in self._voices:
|
| 34 |
+
return self._voices[voice_name]
|
| 35 |
+
try:
|
| 36 |
+
from piper import PiperVoice
|
| 37 |
+
except ImportError as exc:
|
| 38 |
+
raise ImportError(
|
| 39 |
+
"Piper TTS requires piper-tts. "
|
| 40 |
+
"Install with: uv sync --package echocoach --extra piper"
|
| 41 |
+
) from exc
|
| 42 |
+
|
| 43 |
+
onnx_path = self._resolve_voice_path(voice_name)
|
| 44 |
+
voice = PiperVoice.load(str(onnx_path))
|
| 45 |
+
self._voices[voice_name] = voice
|
| 46 |
+
return voice
|
| 47 |
+
|
| 48 |
+
def _resolve_voice_path(self, voice_name: str) -> Path:
|
| 49 |
+
env_dir = __import__("os").environ.get("PIPER_VOICES_DIR")
|
| 50 |
+
candidates: list[Path] = []
|
| 51 |
+
if env_dir:
|
| 52 |
+
candidates.append(Path(env_dir) / f"{voice_name}.onnx")
|
| 53 |
+
home = Path.home() / ".local" / "share" / "piper" / "voices"
|
| 54 |
+
candidates.append(home / f"{voice_name}.onnx")
|
| 55 |
+
candidates.append(Path.cwd() / "models" / "piper" / f"{voice_name}.onnx")
|
| 56 |
+
|
| 57 |
+
for path in candidates:
|
| 58 |
+
if path.is_file():
|
| 59 |
+
return path
|
| 60 |
+
|
| 61 |
+
try:
|
| 62 |
+
from piper.download_voices import download_voice
|
| 63 |
+
except ImportError:
|
| 64 |
+
download_voice = None
|
| 65 |
+
|
| 66 |
+
if download_voice is not None:
|
| 67 |
+
try:
|
| 68 |
+
downloaded = download_voice(voice_name)
|
| 69 |
+
return Path(downloaded)
|
| 70 |
+
except Exception:
|
| 71 |
+
pass
|
| 72 |
+
|
| 73 |
+
import subprocess
|
| 74 |
+
import sys
|
| 75 |
+
|
| 76 |
+
subprocess.run(
|
| 77 |
+
[sys.executable, "-m", "piper.download_voices", voice_name],
|
| 78 |
+
check=True,
|
| 79 |
+
)
|
| 80 |
+
for path in candidates:
|
| 81 |
+
if path.is_file():
|
| 82 |
+
return path
|
| 83 |
+
|
| 84 |
+
raise FileNotFoundError(
|
| 85 |
+
f"Piper voice {voice_name!r} not found. "
|
| 86 |
+
f"Run: python -m piper.download_voices {voice_name}"
|
| 87 |
+
)
|
| 88 |
+
|
| 89 |
+
def synthesize(
|
| 90 |
+
self,
|
| 91 |
+
text: str,
|
| 92 |
+
*,
|
| 93 |
+
language: str,
|
| 94 |
+
out_dir: Path | None = None,
|
| 95 |
+
) -> tuple[str | None, str | None]:
|
| 96 |
+
if not text.strip():
|
| 97 |
+
return None, "No text to synthesize."
|
| 98 |
+
|
| 99 |
+
voice_name, warning = self._voice_name(language)
|
| 100 |
+
try:
|
| 101 |
+
import wave
|
| 102 |
+
|
| 103 |
+
from piper import PiperVoice
|
| 104 |
+
|
| 105 |
+
voice = self._load_voice(voice_name)
|
| 106 |
+
base = out_dir or outputs_dir()
|
| 107 |
+
base.mkdir(parents=True, exist_ok=True)
|
| 108 |
+
out_path = base / f"voiceout_{uuid.uuid4().hex[:10]}.wav"
|
| 109 |
+
with wave.open(str(out_path), "wb") as wav_file:
|
| 110 |
+
voice.synthesize_wav(text, wav_file)
|
| 111 |
+
return str(out_path), warning
|
| 112 |
+
except ImportError:
|
| 113 |
+
return None, "piper-tts not installed; VoiceOut skipped."
|
| 114 |
+
except Exception as exc: # noqa: BLE001
|
| 115 |
+
return None, f"VoiceOut failed: {exc}"
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def get_tts_backend(preset_key: str | None = None) -> TtsBackend:
|
| 119 |
+
config = get_echo_coach_config()
|
| 120 |
+
preset = config.get_tts(preset_key)
|
| 121 |
+
if preset.key not in _tts_cache:
|
| 122 |
+
_tts_cache[preset.key] = PiperTtsBackend(preset)
|
| 123 |
+
return _tts_cache[preset.key]
|
libs/echocoach/src/echocoach/utils.py
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import re
|
| 5 |
+
from typing import Any
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
def extract_json(text: str) -> dict[str, Any]:
|
| 9 |
+
"""Parse JSON from an LLM response (fenced blocks or trailing prose)."""
|
| 10 |
+
cleaned = text.strip()
|
| 11 |
+
fence = re.search(r"```(?:json)?\s*(.*?)\s*```", cleaned, re.DOTALL | re.IGNORECASE)
|
| 12 |
+
if fence:
|
| 13 |
+
cleaned = fence.group(1).strip()
|
| 14 |
+
|
| 15 |
+
start = cleaned.find("{")
|
| 16 |
+
if start < 0:
|
| 17 |
+
return json.loads(cleaned)
|
| 18 |
+
|
| 19 |
+
end = _matching_brace_end(cleaned, start)
|
| 20 |
+
if end is not None:
|
| 21 |
+
return json.loads(cleaned[start : end + 1])
|
| 22 |
+
|
| 23 |
+
fallback_end = cleaned.rfind("}")
|
| 24 |
+
if fallback_end > start:
|
| 25 |
+
return json.loads(cleaned[start : fallback_end + 1])
|
| 26 |
+
return json.loads(cleaned)
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
def _matching_brace_end(text: str, start: int) -> int | None:
|
| 30 |
+
if start >= len(text) or text[start] != "{":
|
| 31 |
+
return None
|
| 32 |
+
depth = 0
|
| 33 |
+
in_string = False
|
| 34 |
+
escape = False
|
| 35 |
+
for index in range(start, len(text)):
|
| 36 |
+
char = text[index]
|
| 37 |
+
if escape:
|
| 38 |
+
escape = False
|
| 39 |
+
continue
|
| 40 |
+
if char == "\\" and in_string:
|
| 41 |
+
escape = True
|
| 42 |
+
continue
|
| 43 |
+
if char == '"':
|
| 44 |
+
in_string = not in_string
|
| 45 |
+
continue
|
| 46 |
+
if in_string:
|
| 47 |
+
continue
|
| 48 |
+
if char == "{":
|
| 49 |
+
depth += 1
|
| 50 |
+
elif char == "}":
|
| 51 |
+
depth -= 1
|
| 52 |
+
if depth == 0:
|
| 53 |
+
return index
|
| 54 |
+
return None
|
libs/echocoach/tests/fixtures/silence_2s.wav
ADDED
|
Binary file (64 kB). View file
|
|
|
libs/echocoach/tests/make_fixture.py
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generate a short silent WAV fixture for smoke tests."""
|
| 2 |
+
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
|
| 5 |
+
import numpy as np
|
| 6 |
+
import soundfile as sf
|
| 7 |
+
|
| 8 |
+
OUT = Path(__file__).resolve().parent / "fixtures" / "silence_2s.wav"
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def main() -> None:
|
| 12 |
+
OUT.parent.mkdir(parents=True, exist_ok=True)
|
| 13 |
+
audio = np.zeros(32_000, dtype=np.float32) # 2s @ 16kHz
|
| 14 |
+
sf.write(OUT, audio, 16_000)
|
| 15 |
+
print(f"Wrote {OUT}")
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
if __name__ == "__main__":
|
| 19 |
+
main()
|
libs/echocoach/tests/test_coach_parse.py
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from echocoach.coach import parse_coach_response
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
def test_parse_coach_json_from_fenced_block():
|
| 5 |
+
raw = """Here is feedback:
|
| 6 |
+
```json
|
| 7 |
+
{
|
| 8 |
+
"summary": "Good energy.",
|
| 9 |
+
"filler_feedback": "Reduce um.",
|
| 10 |
+
"pace_feedback": "Slow down.",
|
| 11 |
+
"rewrite": "We should start now.",
|
| 12 |
+
"one_tip": "Pause after each point."
|
| 13 |
+
}
|
| 14 |
+
```
|
| 15 |
+
"""
|
| 16 |
+
feedback = parse_coach_response(raw)
|
| 17 |
+
assert feedback.summary == "Good energy."
|
| 18 |
+
assert feedback.one_tip == "Pause after each point."
|
libs/echocoach/tests/test_fillers.py
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from echocoach.analysis.fillers import analyze_fillers, highlight_fillers_html
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
def test_detects_common_fillers():
|
| 5 |
+
text = "So um I think like you know we should basically start."
|
| 6 |
+
analysis = analyze_fillers(text)
|
| 7 |
+
assert analysis.total >= 4
|
| 8 |
+
assert "um" in analysis.counts
|
| 9 |
+
assert "like" in analysis.counts
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def test_highlight_wraps_fillers():
|
| 13 |
+
text = "Um hello there."
|
| 14 |
+
analysis = analyze_fillers(text)
|
| 15 |
+
html = highlight_fillers_html(text, analysis)
|
| 16 |
+
assert "<mark" in html
|
| 17 |
+
assert "Um" in html or "um" in html.lower()
|
libs/echocoach/tests/test_pace.py
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from echocoach.analysis.pace import analyze_pace
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
def test_ideal_pace_scores_high():
|
| 5 |
+
# 150 words in 60s => 150 WPM
|
| 6 |
+
text = " ".join(["word"] * 150)
|
| 7 |
+
pace = analyze_pace(text, 60.0)
|
| 8 |
+
assert pace.wpm == 150.0
|
| 9 |
+
assert pace.score == 100
|
| 10 |
+
assert pace.label == "Ideal pace"
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
def test_slow_pace_penalized():
|
| 14 |
+
text = " ".join(["word"] * 50)
|
| 15 |
+
pace = analyze_pace(text, 60.0)
|
| 16 |
+
assert pace.wpm == 50.0
|
| 17 |
+
assert pace.score < 100
|
| 18 |
+
assert "slow" in pace.label.lower()
|
libs/echocoach/tests/test_recording.py
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
|
| 5 |
+
import pytest
|
| 6 |
+
|
| 7 |
+
from echocoach.recording import (
|
| 8 |
+
ServerRecordingError,
|
| 9 |
+
recording_level_warning,
|
| 10 |
+
select_capture_backend,
|
| 11 |
+
start_server_recording,
|
| 12 |
+
stop_server_recording,
|
| 13 |
+
)
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
class _FakeProcess:
|
| 17 |
+
def __init__(self, out_path: Path) -> None:
|
| 18 |
+
self._running = True
|
| 19 |
+
self.returncode: int | None = None
|
| 20 |
+
self.stderr = None
|
| 21 |
+
self._out_path = out_path
|
| 22 |
+
|
| 23 |
+
def poll(self) -> int | None:
|
| 24 |
+
return None if self._running else self.returncode
|
| 25 |
+
|
| 26 |
+
def send_signal(self, sig: int) -> None:
|
| 27 |
+
self._running = False
|
| 28 |
+
self.returncode = 1 # pw-record exits 1 on SIGINT
|
| 29 |
+
self._out_path.write_bytes(b"RIFF" + b"x" * 100)
|
| 30 |
+
|
| 31 |
+
def wait(self, timeout: float | None = None) -> int:
|
| 32 |
+
return 0
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def test_select_capture_backend_prefers_pw_record(monkeypatch):
|
| 36 |
+
monkeypatch.setattr("echocoach.recording._pw_record_available", lambda: True)
|
| 37 |
+
monkeypatch.setattr("echocoach.recording._sounddevice_available", lambda: True)
|
| 38 |
+
monkeypatch.setattr("echocoach.recording._arecord_available", lambda: True)
|
| 39 |
+
assert select_capture_backend() == "pw-record"
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
class _FakeTimer:
|
| 43 |
+
def __init__(self, *_args, **_kwargs) -> None:
|
| 44 |
+
self.daemon = False
|
| 45 |
+
|
| 46 |
+
def start(self) -> None:
|
| 47 |
+
return None
|
| 48 |
+
|
| 49 |
+
def cancel(self) -> None:
|
| 50 |
+
return None
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def test_start_stop_session(monkeypatch, tmp_path):
|
| 54 |
+
import echocoach.recording as rec
|
| 55 |
+
|
| 56 |
+
rec._session = None
|
| 57 |
+
|
| 58 |
+
def fake_spawn(backend, path: Path):
|
| 59 |
+
return _FakeProcess(path)
|
| 60 |
+
|
| 61 |
+
monkeypatch.setattr("echocoach.recording.select_capture_backend", lambda: "pw-record")
|
| 62 |
+
monkeypatch.setattr("echocoach.recording.outputs_dir", lambda: tmp_path)
|
| 63 |
+
monkeypatch.setattr("echocoach.recording._spawn_capture_process", fake_spawn)
|
| 64 |
+
monkeypatch.setattr("echocoach.recording.threading.Timer", _FakeTimer)
|
| 65 |
+
|
| 66 |
+
start_server_recording(10)
|
| 67 |
+
path = stop_server_recording()
|
| 68 |
+
assert path.is_file()
|
| 69 |
+
rec._session = None
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def test_start_while_recording_raises(monkeypatch, tmp_path):
|
| 73 |
+
import echocoach.recording as rec
|
| 74 |
+
|
| 75 |
+
rec._session = None
|
| 76 |
+
|
| 77 |
+
def fake_spawn(backend, path: Path):
|
| 78 |
+
return _FakeProcess(path)
|
| 79 |
+
|
| 80 |
+
monkeypatch.setattr("echocoach.recording.select_capture_backend", lambda: "pw-record")
|
| 81 |
+
monkeypatch.setattr("echocoach.recording.outputs_dir", lambda: tmp_path)
|
| 82 |
+
monkeypatch.setattr("echocoach.recording._spawn_capture_process", fake_spawn)
|
| 83 |
+
monkeypatch.setattr("echocoach.recording.threading.Timer", _FakeTimer)
|
| 84 |
+
|
| 85 |
+
start_server_recording(10)
|
| 86 |
+
with pytest.raises(ServerRecordingError, match="Already recording"):
|
| 87 |
+
start_server_recording(10)
|
| 88 |
+
stop_server_recording()
|
| 89 |
+
rec._session = None
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
def test_stop_without_start_raises():
|
| 93 |
+
import echocoach.recording as rec
|
| 94 |
+
|
| 95 |
+
rec._session = None
|
| 96 |
+
rec._last_recording_path = None
|
| 97 |
+
with pytest.raises(ServerRecordingError, match="Not recording"):
|
| 98 |
+
stop_server_recording()
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def test_recording_level_warning_detects_silence(tmp_path):
|
| 102 |
+
import numpy as np
|
| 103 |
+
import soundfile as sf
|
| 104 |
+
|
| 105 |
+
silent = tmp_path / "silent.wav"
|
| 106 |
+
sf.write(silent, np.zeros(16_000, dtype=np.float32), 16_000)
|
| 107 |
+
assert "silent" in (recording_level_warning(silent) or "").lower()
|
| 108 |
+
|
| 109 |
+
|
| 110 |
+
def test_finalize_accepts_pw_record_exit_code_one(tmp_path, monkeypatch):
|
| 111 |
+
import echocoach.recording as rec
|
| 112 |
+
|
| 113 |
+
rec._session = None
|
| 114 |
+
rec._last_recording_path = None
|
| 115 |
+
|
| 116 |
+
out_path = tmp_path / "recordings" / "server_pw.wav"
|
| 117 |
+
out_path.parent.mkdir(parents=True)
|
| 118 |
+
fake_proc = _FakeProcess(out_path)
|
| 119 |
+
|
| 120 |
+
session = rec._RecordingSession(
|
| 121 |
+
process=fake_proc,
|
| 122 |
+
out_path=out_path,
|
| 123 |
+
backend="pw-record",
|
| 124 |
+
max_seconds=10,
|
| 125 |
+
started_at=0.0,
|
| 126 |
+
watchdog=None,
|
| 127 |
+
)
|
| 128 |
+
rec._session = session
|
| 129 |
+
path = rec.stop_server_recording()
|
| 130 |
+
assert path == out_path
|
| 131 |
+
assert path.stat().st_size > 44
|
| 132 |
+
rec._session = None
|
pyproject.toml
CHANGED
|
@@ -6,6 +6,7 @@ readme = "README.md"
|
|
| 6 |
requires-python = ">=3.12"
|
| 7 |
dependencies = [
|
| 8 |
"agent",
|
|
|
|
| 9 |
"ensemble",
|
| 10 |
"gradio-space",
|
| 11 |
"inference",
|
|
@@ -44,6 +45,7 @@ members = [
|
|
| 44 |
|
| 45 |
[tool.uv.sources]
|
| 46 |
agent = { workspace = true }
|
|
|
|
| 47 |
ensemble = { workspace = true }
|
| 48 |
gradio-space = { workspace = true }
|
| 49 |
inference = { workspace = true }
|
|
|
|
| 6 |
requires-python = ">=3.12"
|
| 7 |
dependencies = [
|
| 8 |
"agent",
|
| 9 |
+
"echocoach",
|
| 10 |
"ensemble",
|
| 11 |
"gradio-space",
|
| 12 |
"inference",
|
|
|
|
| 45 |
|
| 46 |
[tool.uv.sources]
|
| 47 |
agent = { workspace = true }
|
| 48 |
+
echocoach = { workspace = true }
|
| 49 |
ensemble = { workspace = true }
|
| 50 |
gradio-space = { workspace = true }
|
| 51 |
inference = { workspace = true }
|
scripts/echo_coach_smoke.sh
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# Smoke test for EchoCoach analysis (no GPU / no ASR models).
|
| 3 |
+
set -euo pipefail
|
| 4 |
+
cd "$(dirname "$0")/.."
|
| 5 |
+
|
| 6 |
+
FIXTURE="libs/echocoach/tests/fixtures/silence_2s.wav"
|
| 7 |
+
if [[ ! -f "$FIXTURE" ]]; then
|
| 8 |
+
uv run python libs/echocoach/tests/make_fixture.py
|
| 9 |
+
fi
|
| 10 |
+
|
| 11 |
+
uv run pytest libs/echocoach/tests/test_fillers.py libs/echocoach/tests/test_pace.py libs/echocoach/tests/test_coach_parse.py -q
|
| 12 |
+
echo "EchoCoach smoke tests passed."
|
uv.lock
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
voice_models.yaml
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Voice model presets for EchoCoach (ASR + TTS).
|
| 2 |
+
# Override defaults via ECHOCOACH_ASR_PRESET / ECHOCOACH_TTS_PRESET in .env
|
| 3 |
+
|
| 4 |
+
defaults:
|
| 5 |
+
asr_preset: whisper-cpp-tiny
|
| 6 |
+
tts_preset: piper-multilingual
|
| 7 |
+
coach_model: minicpm5-1b
|
| 8 |
+
max_seconds: 30
|
| 9 |
+
|
| 10 |
+
languages:
|
| 11 |
+
- code: en
|
| 12 |
+
label: English
|
| 13 |
+
- code: fr
|
| 14 |
+
label: French
|
| 15 |
+
- code: de
|
| 16 |
+
label: German
|
| 17 |
+
- code: es
|
| 18 |
+
label: Spanish
|
| 19 |
+
- code: it
|
| 20 |
+
label: Italian
|
| 21 |
+
- code: pt
|
| 22 |
+
label: Portuguese
|
| 23 |
+
- code: nl
|
| 24 |
+
label: Dutch
|
| 25 |
+
- code: pl
|
| 26 |
+
label: Polish
|
| 27 |
+
- code: el
|
| 28 |
+
label: Greek
|
| 29 |
+
- code: ar
|
| 30 |
+
label: Arabic
|
| 31 |
+
- code: ja
|
| 32 |
+
label: Japanese
|
| 33 |
+
- code: zh
|
| 34 |
+
label: Chinese (Mandarin)
|
| 35 |
+
- code: vi
|
| 36 |
+
label: Vietnamese
|
| 37 |
+
- code: ko
|
| 38 |
+
label: Korean
|
| 39 |
+
|
| 40 |
+
asr:
|
| 41 |
+
cohere-transcribe:
|
| 42 |
+
label: Cohere Transcribe 2B (14 languages)
|
| 43 |
+
backend: cohere
|
| 44 |
+
model_id: CohereLabs/cohere-transcribe-03-2026
|
| 45 |
+
|
| 46 |
+
whisper-cpp-tiny:
|
| 47 |
+
label: Whisper.cpp tiny (CPU, fast)
|
| 48 |
+
backend: whisper_cpp
|
| 49 |
+
model_size: tiny
|
| 50 |
+
|
| 51 |
+
whisper-cpp-base:
|
| 52 |
+
label: Whisper.cpp base (CPU, better WER)
|
| 53 |
+
backend: whisper_cpp
|
| 54 |
+
model_size: base
|
| 55 |
+
|
| 56 |
+
tts:
|
| 57 |
+
piper-multilingual:
|
| 58 |
+
label: Piper TTS (local VoiceOut)
|
| 59 |
+
backend: piper
|
| 60 |
+
voices:
|
| 61 |
+
en: en_US-lessac-medium
|
| 62 |
+
fr: fr_FR-siwis-medium
|
| 63 |
+
de: de_DE-thorsten-medium
|
| 64 |
+
es: es_ES-sharvard-medium
|
| 65 |
+
it: it_IT-riccardo-medium
|
| 66 |
+
pt: pt_BR-faber-medium
|
| 67 |
+
nl: nl_NL-mls-medium
|
| 68 |
+
pl: pl_PL-darkman-medium
|
| 69 |
+
el: en_US-lessac-medium
|
| 70 |
+
ar: ar_JO-kareem-medium
|
| 71 |
+
ja: ja_JP-natsuki-medium
|
| 72 |
+
zh: zh_CN-huayan-medium
|
| 73 |
+
vi: vi_VN-25hours-single
|
| 74 |
+
ko: ko_KR-kss-medium
|
| 75 |
+
fallback_voice: en_US-lessac-medium
|