v1.3 — fast natural voice + audio-reactive motion

- Switch primary TTS to edge-tts (Microsoft neural voices: Ava, Andrew,
Sonia, Ryan, …). Sub-second per paragraph, much more natural than
pyttsx3/gTTS.
- Stream speech paragraph-by-paragraph: synthesise + push chunk N
while chunk N-1 is already playing. First words now reach the
speaker ~1-2s after the LLM finishes instead of 25-30s.
- Voice picker in the in-app UI (US/UK/IE/AU, male/female), backed by a
curated catalogue exposed via /api/state and /api/voices.
- New /api/speech/voice endpoint to change voice live.
- Audio-reactive motion: speech.compute_envelope() emits a 50 ms RMS
loudness signal, _AppState rolls it forward in real time, and
motion._speaking_frame adds head dips, antenna lifts, body sway in
proportion to the actual audio amplitude. Robot moves with the voice
rather than on a fixed loop.
- Bigger amplitudes across all motion states (idle, thinking,
responding, speaking, acknowledge_toggle) for a more embodied feel.
- pyproject: add edge-tts>=6.1.10.
- README + landing page updated with new voice/motion behaviour and
env vars (MARGINALIA_VOICE, MARGINALIA_EDGE_RATE, MARGINALIA_EDGE_PITCH).

README.md

@@ -37,12 +37,17 @@ Below the answer, an "In the margin" panel shows the Wikipedia hits the
 system used to ground its reading — title, intro extract, and a thumbnail
 portrait or cover, when one is available.
 
-**Reachy reads each answer aloud.** Once the LLM finishes, the answer
-prose is synthesised and pushed to the robot's speaker while gentle
-head, antenna, and body movements pace the sentences. Prefer to read in
-silence? Twist *either* antenna and Reachy switches to **read mode** —
+**Reachy reads each answer aloud, paragraph by paragraph.** As soon as
+the first paragraph is synthesised it starts playing through the robot's
+speaker — Reachy doesn't wait for the whole answer before opening its
+mouth. The remaining paragraphs synthesise in the background and queue
+up seamlessly. While speaking, the head, antennas, and body move *with
+the audio*: the head dips on emphasised words, the antennas perk up at
+loud syllables, the torso sways through long sentences. Prefer to read
+in silence? Twist *either* antenna and Reachy switches to **read mode** —
 text only, no voice. Twist again to bring the voice back. The same
-toggle is mirrored as a small pill in the in-app UI.
+toggle is mirrored as a small pill in the in-app UI, alongside a voice
+picker (US/UK/IE/AU, male/female).
 
 This is a Reachy Mini **Python app**. It runs on the robot via the
 [Reachy Mini Control App](https://huggingface.co/docs/reachy_mini), serves
@@ -81,8 +86,11 @@ to the robot (or simulator) currently selected in the Control App.
 | `MARGINALIA_LLM_MODEL`        | `meta-llama/Llama-3.3-70B-Instruct`    | Any chat-capable model on the HF inference API.          |
 | `MARGINALIA_LLM_MAX_TOKENS`   | `1100`                                 | Cap per response.                                        |
 | `MARGINALIA_LLM_TEMPERATURE`  | `0.45`                                 | Lower = more careful, higher = more associative.         |
-| `MARGINALIA_TTS_MODE`         | `auto`                                 | `auto` / `hf` / `offline` / `gtts`. `auto` tries HF → pyttsx3 → gTTS. |
-| `MARGINALIA_HF_TTS_MODEL`     | `facebook/mms-tts-eng`                 | HF Inference text-to-speech model. The default works text-only; SpeechT5 needs speaker embeddings the simple API does not supply. |
+| `MARGINALIA_TTS_MODE`         | `auto`                                 | `auto` / `edge` / `hf` / `offline` / `gtts`. `auto` tries edge-tts → HF → pyttsx3 → gTTS. |
+| `MARGINALIA_VOICE`            | `en-US-AvaNeural`                      | Default edge-tts voice. The in-app UI exposes a curated picker; this is the fallback when no voice is selected yet. |
+| `MARGINALIA_EDGE_RATE`        | `+0%`                                  | Edge-tts speaking rate, e.g. `+10%`, `-5%`.              |
+| `MARGINALIA_EDGE_PITCH`       | `+0Hz`                                 | Edge-tts pitch shift, e.g. `+30Hz`, `-20Hz`.             |
+| `MARGINALIA_HF_TTS_MODEL`     | `facebook/mms-tts-eng`                 | HF Inference text-to-speech model used as a fallback if edge-tts is unreachable. |
 | `MARGINALIA_TTS_RATE`         | `175`                                  | pyttsx3 speech rate (words per minute).                  |
 | `MARGINALIA_TTS_VOICE`        | unset                                  | Optional pyttsx3 voice id (system-dependent).            |
 | `MARGINALIA_TTS_GTTS_LANG`    | `en`                                   | gTTS language code.                                      |
@@ -109,17 +117,24 @@ to the robot (or simulator) currently selected in the Control App.
                           │   answer / how to read /  │
                           │   further reading         │
                           ├───────────────────────────┤
-                          │ speech.synthesize() —     │
-                          │   HF Inference TTS,       │
-                          │   pyttsx3 fallback,       │
+                          │ speech.split_for_speech() │
+                          │   → paragraph chunks      │
+                          │ speech.synthesize() per   │
+                          │   chunk — edge-tts → HF → │
+                          │   pyttsx3 → gTTS,         │
                           │   16 kHz mono float32     │
+                          │ compute_envelope() per    │
+                          │   chunk → audio loudness  │
                           └──────┬────────────────────┘
                                  ▼
                           ┌──────────────────────────┐
-                          │ motion state machine     │
-                          │ → Reachy head, antennas, │
-                          │   body_yaw  +            │
-                          │   media.push_audio_sample│
+                          │ motion state machine,    │
+                          ��� audio-reactive overlay   │
+                          │ → head, antennas, body   │
+                          │   move *with* the audio  │
+                          │ + media.push_audio_sample│
+                          │   (chunk N+1 synthesises │
+                          │   while N is playing)    │
                           └──────────────────────────┘
                                  ▲
                                  │ antenna twist polled
@@ -139,19 +154,28 @@ to the robot (or simulator) currently selected in the Control App.
   sections in markdown: answer, *How to read this*, *Further reading*.
 * **`marginalia/speech.py`** — `clean_for_speech()` strips citations,
   URLs, and the trailing sections, leaving just speakable prose;
-  `synthesize()` tries HF Inference TTS first then falls back to
-  `pyttsx3`, always returning 16 kHz mono float32 PCM ready for the
-  robot speaker.
+  `split_for_speech()` cuts the prose into paragraph-sized chunks for
+  low-latency streaming TTS; `synthesize()` tries edge-tts (Microsoft
+  neural voices) first, then HF Inference TTS, then `pyttsx3`, then
+  gTTS — always returning 16 kHz mono float32 PCM ready for the robot
+  speaker. `compute_envelope()` produces a per-frame loudness signal
+  that drives audio-reactive head and antenna motion. `VOICES` is the
+  curated voice catalogue exposed to the UI dropdown.
 * **`marginalia/main.py`** — `Marginalia(ReachyMiniApp)` registers
-  `/api/health`, `/api/state`, `/api/speech/toggle`, `/api/events`
-  (ambient SSE), and `/api/ask` (per-question SSE) on the SDK's bundled
-  FastAPI app, serves the static UI from `static/`, and polls antenna
-  positions in the run loop to toggle speech on a user twist.
+  `/api/health`, `/api/state`, `/api/speech/toggle`, `/api/speech/voice`,
+  `/api/voices`, `/api/events` (ambient SSE), and `/api/ask`
+  (per-question SSE) on the SDK's bundled FastAPI app, serves the static
+  UI from `static/`, and polls antenna positions in the run loop to
+  toggle speech on a user twist. The run loop reads the live audio
+  loudness from `_AppState` and feeds it to `motion.pose_for_state` so
+  the robot moves with the voice.
 * **`marginalia/motion.py`** — a 50 Hz state machine returning
   `MotionFrame(head, antennas, body_yaw)`:
   *idle → thinking → nodding → responding → speaking → idle*, plus a
   short *acknowledge_toggle* burst when the user mutes/unmutes by
-  twisting an antenna.
+  twisting an antenna. The `speaking` state takes an extra `audio_env`
+  argument: a 0–1 instantaneous loudness that adds head dips, antenna
+  lifts, and body sway in time with the actual audio.
 
 ## Tweaking
 
@@ -165,9 +189,12 @@ to the robot (or simulator) currently selected in the Control App.
   `marginalia/motion.py` with named constants for amplitude, period,
   and tilt for each state. Try a bigger `THINKING_BODY_AMP_DEG` or a
   slower `SPEAKING_NOD_PERIOD_S`.
-* **Different voice.** Pick any text-to-speech model on the HF inference
-  API via `MARGINALIA_HF_TTS_MODEL`, force the offline voice with
-  `MARGINALIA_TTS_MODE=offline`, or tweak `MARGINALIA_TTS_RATE`.
+* **Different voice.** Pick any voice from the in-app dropdown — Ava,
+  Andrew, Sonia, Ryan, Emily, Natasha, etc. — or pin a default via
+  `MARGINALIA_VOICE=en-GB-RyanNeural`. Speed it up with
+  `MARGINALIA_EDGE_RATE=+15%` or pitch it down with
+  `MARGINALIA_EDGE_PITCH=-25Hz`. Force a specific backend with
+  `MARGINALIA_TTS_MODE=edge|hf|offline|gtts`.
 * **Different mute gesture sensitivity.** Adjust `TWIST_THRESHOLD_RAD`
   (default `0.4` ≈ 23°) and `TWIST_COOLDOWN_S` in `marginalia/main.py`
   if the robot mishears your twists.

index.html

@@ -50,7 +50,8 @@
                 <div class="preview-meta">
                     <span class="preview-pill">§ How to read this</span>
                     <span class="preview-pill alt">¶ Further reading</span>
-                    <span class="preview-pill voice">♪ Read aloud</span>
+                    <span class="preview-pill voice">♪ Ava — neural voice</span>
+                    <span class="preview-pill voice">♪ Andrew — companion</span>
                 </div>
 
                 <p class="preview-note">
@@ -80,13 +81,17 @@
                     <code>http://0.0.0.0:8042</code> automatically.
                 </li>
                 <li>
-                    <strong>Ask a question.</strong> Reachy gently sways its
-                    head while it thinks, nods once when the answer begins,
-                    and then reads it aloud through the speaker while small
-                    antenna and body movements punctuate the prose. Each
-                    answer comes with a <em>How&nbsp;to&nbsp;read&nbsp;this</em>
-                    panel of tips tailored to the work, plus
-                    <em>Further&nbsp;reading</em> for amplification.
+                    <strong>Ask a question.</strong> Reachy sways its head
+                    while it thinks, nods the moment the first token lands,
+                    then starts speaking the answer paragraph by paragraph
+                    &mdash; the first words come through the speaker about a
+                    second after the LLM finishes, while the rest of the
+                    answer is still being synthesised in the background.
+                    The head dips, antennas perk up, and the body sways
+                    <em>with</em> the audio. Each answer comes with a
+                    <em>How&nbsp;to&nbsp;read&nbsp;this</em> panel of tips
+                    tailored to the work, plus <em>Further&nbsp;reading</em>
+                    for amplification.
                 </li>
                 <li>
                     <strong>Prefer to read in silence?</strong> Twist either
@@ -126,10 +131,14 @@
                     glance at the source material before clicking through.
                 </li>
                 <li>
-                    <strong>An embodied reading.</strong> The robot speaks the
-                    answer prose aloud while gentle head, antenna, and body
-                    movements pace the sentences. Twist an antenna any time
-                    to switch to silent <em>read mode</em>.
+                    <strong>An embodied reading.</strong> The robot speaks
+                    the answer paragraph by paragraph in a natural neural
+                    voice (Microsoft Edge TTS &mdash; pick from US, UK, IE,
+                    AU, male or female from a small dropdown), and the
+                    head, antennas, and body move <em>in time with the
+                    audio waveform</em>: dips on emphasised words, antenna
+                    lifts on loud syllables. Twist an antenna any time to
+                    switch to silent <em>read mode</em>.
                 </li>
             </ul>
         </section>
@@ -162,16 +171,21 @@
                     machine drives the head, antennas, and body together:
                     idle sway → thinking sway with curling antennas → a
                     single nod the moment the first token arrives → an
-                    attentive bob while the rest streams in → theatrical
-                    head/antenna gestures while the answer is read aloud.
+                    attentive bob while the rest streams in → audio-reactive
+                    head and antenna motion while the answer is read aloud,
+                    so the robot moves <em>with</em> the voice rather than
+                    on a fixed loop.
                 </li>
                 <li>
-                    <strong>Voice.</strong> After the LLM finishes, the prose
-                    is synthesised via Hugging&nbsp;Face Inference TTS (with
-                    an offline <code>pyttsx3</code> fallback), resampled to
-                    16&nbsp;kHz, and pushed to the robot speaker through
-                    <code>reachy_mini.media</code>. Twisting either antenna
-                    above ~23&deg; toggles spoken vs silent mode and is
+                    <strong>Voice.</strong> The cleaned answer prose is split
+                    into paragraph chunks. Each chunk is synthesised through
+                    <strong>edge-tts</strong> (Microsoft's free neural
+                    voices &mdash; Ava, Andrew, Sonia, Ryan, &hellip;) and
+                    pushed to the robot speaker as soon as it's ready, so
+                    the next chunk renders <em>while the previous one is
+                    already playing</em>. Fallbacks: HF Inference TTS,
+                    <code>pyttsx3</code>, gTTS. Twisting either antenna above
+                    ~23&deg; toggles spoken vs silent mode and is
                     acknowledged with a small head nod.
                 </li>
             </ol>

marginalia/main.py

@@ -10,8 +10,13 @@ antennas for user-induced twists (which mute or un-mute the spoken
 answer), and shares state with the FastAPI handlers via `_AppState`.
 
 A `POST /api/ask` request streams the LLM answer to the browser via
-Server-Sent Events. When speech is enabled, the full answer is then
-synthesised with `marginalia.speech` and pushed to the robot's speaker.
+Server-Sent Events. When speech is enabled, the answer is split into
+paragraph-sized chunks and synthesised + pushed to the robot speaker
+one chunk at a time, so the first words start playing within ~1-2 s
+instead of waiting for the whole answer to render. While the robot
+speaks, the run loop reads the per-frame audio loudness from
+`_AppState` and feeds it to `motion.pose_for_state` for audio-reactive
+head and antenna movement.
 """
 
 from __future__ import annotations
@@ -43,10 +48,11 @@ TWIST_COOLDOWN_S = 1.5
 class _AppState:
     """Thread-safe shared state.
 
-    Tracks the motion state (drives the run loop) and a ``speech_enabled``
-    flag (drives whether spoken answers play through the robot speaker).
-    State changes notify any subscribed asyncio queue so the FastAPI
-    handlers can broadcast `state_changed` SSE frames.
+    Tracks the motion state (drives the run loop), a ``speech_enabled``
+    flag, the active TTS voice, and a rolling audio-loudness envelope
+    used to drive audio-reactive head and antenna motion while the robot
+    speaks. State changes notify any subscribed asyncio queue so the
+    FastAPI handlers can broadcast `state_changed` SSE frames.
     """
 
     def __init__(self) -> None:
@@ -54,9 +60,15 @@ class _AppState:
         self._state: motion.State = "idle"
         self._t_change: float = time.time()
         self._speech_enabled: bool = True
+        self._voice: str = speech.DEFAULT_VOICE
         self._listeners: list[asyncio.Queue] = []
         self._listeners_lock = threading.Lock()
 
+        self._envelope: np.ndarray = np.zeros(0, dtype=np.float32)
+        self._envelope_dt: float = speech.ENVELOPE_FRAME_MS / 1000.0
+        self._envelope_start_mono: float = 0.0
+        self._envelope_active: bool = False
+
     def get(self) -> tuple[motion.State, float]:
         with self._lock:
             return self._state, time.time() - self._t_change
@@ -97,9 +109,30 @@ class _AppState:
         self._broadcast({"speech_enabled": value})
         return value
 
+    @property
+    def voice(self) -> str:
+        with self._lock:
+            return self._voice
+
+    def set_voice(self, voice_id: str) -> bool:
+        if voice_id not in speech.VOICE_IDS:
+            return False
+        with self._lock:
+            changed = self._voice != voice_id
+            self._voice = voice_id
+        if changed:
+            logger.info("voice -> %s", voice_id)
+            self._broadcast({"voice": voice_id})
+        return changed
+
     def snapshot(self) -> dict[str, Any]:
         with self._lock:
-            return {"state": self._state, "speech_enabled": self._speech_enabled}
+            return {
+                "state": self._state,
+                "speech_enabled": self._speech_enabled,
+                "voice": self._voice,
+                "voices": speech.VOICES,
+            }
 
     def subscribe(self) -> asyncio.Queue:
         q: asyncio.Queue = asyncio.Queue(maxsize=32)
@@ -121,11 +154,46 @@ class _AppState:
             except asyncio.QueueFull:
                 pass
 
+    def begin_envelope(self, start_mono: float) -> None:
+        with self._lock:
+            self._envelope = np.zeros(0, dtype=np.float32)
+            self._envelope_dt = speech.ENVELOPE_FRAME_MS / 1000.0
+            self._envelope_start_mono = start_mono
+            self._envelope_active = True
+
+    def append_envelope(self, env: np.ndarray, dt_s: float) -> None:
+        if env is None or len(env) == 0:
+            return
+        with self._lock:
+            if not self._envelope_active:
+                return
+            self._envelope_dt = float(dt_s)
+            self._envelope = np.concatenate([self._envelope, env.astype(np.float32, copy=False)])
+
+    def end_envelope(self) -> None:
+        with self._lock:
+            self._envelope_active = False
+            self._envelope = np.zeros(0, dtype=np.float32)
+
+    def current_audio_level(self) -> float:
+        with self._lock:
+            if not self._envelope_active or len(self._envelope) == 0:
+                return 0.0
+            elapsed = time.monotonic() - self._envelope_start_mono
+            idx = int(elapsed / self._envelope_dt)
+            if idx < 0 or idx >= len(self._envelope):
+                return 0.0
+            return float(self._envelope[idx])
+
 
 class _AskRequest(BaseModel):
     question: str
 
 
+class _VoiceRequest(BaseModel):
+    voice: str
+
+
 def _sse(event: str, data: Any) -> str:
     return f"event: {event}\ndata: {json.dumps(data, ensure_ascii=False)}\n\n"
 
@@ -155,11 +223,24 @@ class Marginalia(ReachyMiniApp):
         def get_state() -> JSONResponse:
             return JSONResponse(self._app_state.snapshot())
 
+        @self.settings_app.get("/api/voices")
+        def list_voices() -> JSONResponse:
+            return JSONResponse(
+                {"voice": self._app_state.voice, "voices": speech.VOICES}
+            )
+
         @self.settings_app.post("/api/speech/toggle")
         def toggle_speech() -> JSONResponse:
             value = self._app_state.toggle_speech()
             return JSONResponse({"speech_enabled": value})
 
+        @self.settings_app.post("/api/speech/voice")
+        def set_voice(req: _VoiceRequest) -> JSONResponse:
+            ok = self._app_state.set_voice(req.voice)
+            return JSONResponse(
+                {"ok": ok, "voice": self._app_state.voice}, status_code=200 if ok else 400
+            )
+
         @self.settings_app.get("/api/events")
         async def events(request: Request) -> StreamingResponse:
             return StreamingResponse(
@@ -248,26 +329,42 @@ class Marginalia(ReachyMiniApp):
                     yield frame
         finally:
             self._app_state.set("idle")
+            self._app_state.end_envelope()
 
     async def _speak(self, full_text: str, request: Request):
-        """Synthesise the answer prose and play it through the robot speaker."""
+        """Synthesise + play the answer prose paragraph-by-paragraph.
+
+        For low latency we don't synthesise the whole answer up front.
+        We split the cleaned text into paragraph chunks, synthesise the
+        first one, push it to the speaker, and then synthesise + push
+        the rest while the first is already playing. The audio engine
+        queues samples, so multiple chunks play seamlessly.
+        """
         loop = asyncio.get_event_loop()
 
         spoken_text = speech.clean_for_speech(full_text)
         if not spoken_text:
             return
+        chunks = speech.split_for_speech(spoken_text)
+        if not chunks:
+            return
+
+        voice = self._app_state.voice
+
+        def synth(text: str) -> np.ndarray:
+            return speech.synthesize(text, voice=voice)
+
+        total_chunks = len(chunks)
 
         try:
-            pcm = await loop.run_in_executor(None, speech.synthesize, spoken_text)
+            first_pcm = await loop.run_in_executor(None, synth, chunks[0])
         except Exception as e:
-            logger.warning("synthesize failed: %s", e)
+            logger.warning("synthesize failed: %r", e)
             yield _sse("speech_error", {"message": str(e)})
             return
 
-        if pcm is None or len(pcm) == 0:
-            logger.warning(
-                "synthesize returned no audio for %d chars of prose", len(spoken_text)
-            )
+        if first_pcm is None or len(first_pcm) == 0:
+            logger.warning("first chunk produced no audio (%d chars)", len(chunks[0]))
             yield _sse(
                 "speech_error",
                 {"message": "Reachy couldn't speak this one — see app logs for the TTS backend error."},
@@ -276,37 +373,88 @@ class Marginalia(ReachyMiniApp):
         if not self._app_state.speech_enabled:
             return
 
-        duration_s = float(len(pcm)) / float(speech.ROBOT_SAMPLE_RATE)
-        self._app_state.set("speaking")
-        yield _sse("speaking", {"duration_s": duration_s})
+        if self._reachy is None:
+            yield _sse("speech_error", {"message": "Robot media not available."})
+            return
+
+        sample_rate = float(speech.ROBOT_SAMPLE_RATE)
+        playback_deadline = time.monotonic()
+
+        try:
+            with self._media_lock:
+                media = self._reachy.media
+                media.start_playing()
+
+            self._app_state.begin_envelope(time.monotonic())
+            self._app_state.set("speaking")
+            yield _sse(
+                "speaking",
+                {"chunks": total_chunks, "voice": voice},
+            )
 
-        played = False
-        if self._reachy is not None:
             try:
                 with self._media_lock:
-                    media = self._reachy.media
-                    media.start_playing()
-                    media.push_audio_sample(np.asarray(pcm, dtype=np.float32))
-                played = True
+                    self._reachy.media.push_audio_sample(
+                        np.asarray(first_pcm, dtype=np.float32)
+                    )
+                env, dt = speech.compute_envelope(first_pcm)
+                self._app_state.append_envelope(env, dt)
+                playback_deadline = max(playback_deadline, time.monotonic()) + (
+                    len(first_pcm) / sample_rate
+                )
+                yield _sse("chunk_played", {"index": 0, "of": total_chunks})
             except Exception as e:
-                logger.warning("audio push failed: %s", e)
+                logger.warning("audio push failed: %r", e)
+                yield _sse("speech_error", {"message": str(e)})
+                return
 
-        if played:
-            sleep_s = duration_s
-            while sleep_s > 0 and self._app_state.speech_enabled:
-                tick = min(0.25, sleep_s)
+            for i, chunk in enumerate(chunks[1:], start=1):
+                if not self._app_state.speech_enabled:
+                    break
+                if await request.is_disconnected():
+                    break
+
+                try:
+                    pcm = await loop.run_in_executor(None, synth, chunk)
+                except Exception as e:
+                    logger.warning("synthesize failed on chunk %d: %r", i, e)
+                    continue
+
+                if pcm is None or len(pcm) == 0:
+                    continue
+
+                try:
+                    with self._media_lock:
+                        if self._reachy is None:
+                            break
+                        self._reachy.media.push_audio_sample(
+                            np.asarray(pcm, dtype=np.float32)
+                        )
+                    env, dt = speech.compute_envelope(pcm)
+                    self._app_state.append_envelope(env, dt)
+                    playback_deadline += len(pcm) / sample_rate
+                    yield _sse("chunk_played", {"index": i, "of": total_chunks})
+                except Exception as e:
+                    logger.warning("audio push failed on chunk %d: %r", i, e)
+                    break
+
+            while self._app_state.speech_enabled:
+                remaining = playback_deadline - time.monotonic()
+                if remaining <= 0.0:
+                    break
+                tick = min(0.25, remaining)
                 await asyncio.sleep(tick)
-                sleep_s -= tick
                 if await request.is_disconnected():
                     break
+
+        finally:
             try:
                 with self._media_lock:
                     if self._reachy is not None:
                         self._reachy.media.stop_playing()
             except Exception as e:
-                logger.warning("audio stop failed: %s", e)
-        else:
-            await asyncio.sleep(0.05)
+                logger.warning("audio stop failed: %r", e)
+            self._app_state.end_envelope()
 
         yield _sse("spoken_done", {})
 
@@ -329,7 +477,12 @@ class Marginalia(ReachyMiniApp):
                     self._app_state.set("idle")
                     state, t_state = self._app_state.get()
 
-                frame = motion.pose_for_state(state, t_state)
+                audio_env = (
+                    self._app_state.current_audio_level()
+                    if state == "speaking"
+                    else 0.0
+                )
+                frame = motion.pose_for_state(state, t_state, audio_env=audio_env)
 
                 try:
                     reachy_mini.set_target(

marginalia/motion.py

@@ -55,46 +55,61 @@ class MotionFrame:
     body_yaw: float
 
 
-IDLE_YAW_AMP_DEG = 4.0
-IDLE_PITCH_AMP_DEG = 1.5
-IDLE_HEAD_PERIOD_S = 6.0
-IDLE_BODY_AMP_DEG = 5.0
-IDLE_BODY_PERIOD_S = 20.0
-IDLE_ANTENNA_REST_DEG = -10.0
-IDLE_ANTENNA_BREATH_DEG = 3.0
-
-THINKING_YAW_AMP_DEG = 14.0
-THINKING_PITCH_AMP_DEG = 6.0
-THINKING_HEAD_PERIOD_S = 3.2
-THINKING_BODY_AMP_DEG = 12.0
-THINKING_BODY_PERIOD_S = 5.0
-THINKING_ANTENNA_FORWARD_DEG = 25.0
-THINKING_ANTENNA_PERIOD_S = 2.4
-
-NODDING_DURATION_S = 0.7
-NODDING_PITCH_DOWN_DEG = 14.0
-NODDING_PITCH_UP_DEG = -6.0
-NODDING_ANTENNA_DEG = 18.0
-
-RESPONDING_BOB_AMP_DEG = 2.5
-RESPONDING_PITCH_BIAS_DEG = 6.0
-RESPONDING_PERIOD_S = 4.0
-RESPONDING_ANTENNA_AMP_DEG = 10.0
-RESPONDING_ANTENNA_PERIOD_S = 3.5
-
-SPEAKING_NOD_PERIOD_S = 2.0
-SPEAKING_NOD_AMP_DEG = 5.5
-SPEAKING_PITCH_BIAS_DEG = 4.0
-SPEAKING_YAW_AMP_DEG = 6.0
-SPEAKING_YAW_PERIOD_S = 7.0
-SPEAKING_BODY_AMP_DEG = 8.0
-SPEAKING_BODY_PERIOD_S = 11.0
-SPEAKING_ANTENNA_AMP_DEG = 14.0
-SPEAKING_ANTENNA_PERIOD_S = 2.6
-
-ACK_DURATION_S = 0.5
-ACK_ANTENNA_DEG = 30.0
-ACK_PITCH_DOWN_DEG = 10.0
+IDLE_YAW_AMP_DEG = 6.0
+IDLE_PITCH_AMP_DEG = 2.5
+IDLE_ROLL_AMP_DEG = 2.0
+IDLE_HEAD_PERIOD_S = 5.5
+IDLE_BODY_AMP_DEG = 9.0
+IDLE_BODY_PERIOD_S = 14.0
+IDLE_ANTENNA_REST_DEG = -8.0
+IDLE_ANTENNA_BREATH_DEG = 5.0
+
+THINKING_YAW_AMP_DEG = 22.0
+THINKING_PITCH_AMP_DEG = 10.0
+THINKING_ROLL_AMP_DEG = 7.0
+THINKING_HEAD_PERIOD_S = 2.6
+THINKING_BODY_AMP_DEG = 22.0
+THINKING_BODY_PERIOD_S = 4.0
+THINKING_ANTENNA_FORWARD_DEG = 32.0
+THINKING_ANTENNA_PERIOD_S = 1.9
+
+NODDING_DURATION_S = 0.8
+NODDING_PITCH_DOWN_DEG = 18.0
+NODDING_PITCH_UP_DEG = -8.0
+NODDING_ANTENNA_DEG = 22.0
+
+RESPONDING_BOB_AMP_DEG = 4.5
+RESPONDING_PITCH_BIAS_DEG = 8.0
+RESPONDING_YAW_AMP_DEG = 6.0
+RESPONDING_PERIOD_S = 3.4
+RESPONDING_BODY_AMP_DEG = 6.0
+RESPONDING_BODY_PERIOD_S = 9.0
+RESPONDING_ANTENNA_AMP_DEG = 14.0
+RESPONDING_ANTENNA_PERIOD_S = 3.0
+
+SPEAKING_NOD_PERIOD_S = 1.6
+SPEAKING_NOD_AMP_DEG = 9.0
+SPEAKING_PITCH_BIAS_DEG = 8.0
+SPEAKING_YAW_AMP_DEG = 10.0
+SPEAKING_YAW_PERIOD_S = 5.5
+SPEAKING_ROLL_AMP_DEG = 4.0
+SPEAKING_ROLL_PERIOD_S = 8.0
+SPEAKING_BODY_AMP_DEG = 14.0
+SPEAKING_BODY_PERIOD_S = 9.0
+SPEAKING_ANTENNA_AMP_DEG = 22.0
+SPEAKING_ANTENNA_PERIOD_S = 2.3
+
+# Audio-reactive overlay applied during the ``speaking`` state. The
+# ``audio_env`` arg is an instantaneous loudness in [0, 1].
+AUDIO_REACTIVE_PITCH_DEG = 8.0
+AUDIO_REACTIVE_YAW_DEG = 4.0
+AUDIO_REACTIVE_ANTENNA_DEG = 18.0
+AUDIO_REACTIVE_BODY_DEG = 6.0
+
+ACK_DURATION_S = 0.55
+ACK_ANTENNA_DEG = 38.0
+ACK_PITCH_DOWN_DEG = 14.0
+ACK_BODY_DEG = 12.0
 
 
 def _deg(angle_deg: float) -> float:
@@ -105,12 +120,13 @@ def _idle_frame(t: float) -> MotionFrame:
     head_phase = 2.0 * math.pi * t / IDLE_HEAD_PERIOD_S
     yaw = IDLE_YAW_AMP_DEG * math.sin(head_phase)
     pitch = IDLE_PITCH_AMP_DEG * math.sin(head_phase * 0.5)
+    roll = IDLE_ROLL_AMP_DEG * math.sin(head_phase * 0.7 + 0.4)
     body_yaw = IDLE_BODY_AMP_DEG * math.sin(2.0 * math.pi * t / IDLE_BODY_PERIOD_S)
-    breath = IDLE_ANTENNA_BREATH_DEG * math.sin(2.0 * math.pi * t / 8.0)
+    breath = IDLE_ANTENNA_BREATH_DEG * math.sin(2.0 * math.pi * t / 7.0)
     ant_deg = IDLE_ANTENNA_REST_DEG + breath
     return MotionFrame(
-        head=create_head_pose(yaw=yaw, pitch=pitch, degrees=True),
-        antennas=[_deg(ant_deg), _deg(ant_deg)],
+        head=create_head_pose(yaw=yaw, pitch=pitch, roll=roll, degrees=True),
+        antennas=[_deg(ant_deg), _deg(ant_deg + breath * 0.4)],
         body_yaw=_deg(body_yaw),
     )
 
@@ -118,8 +134,8 @@ def _idle_frame(t: float) -> MotionFrame:
 def _thinking_frame(t: float) -> MotionFrame:
     head_phase = 2.0 * math.pi * t / THINKING_HEAD_PERIOD_S
     yaw = THINKING_YAW_AMP_DEG * math.sin(head_phase)
-    pitch = -THINKING_PITCH_AMP_DEG * (0.5 - 0.5 * math.cos(head_phase * 1.5))
-    roll = 0.6 * THINKING_PITCH_AMP_DEG * math.sin(head_phase * 0.5)
+    pitch = -THINKING_PITCH_AMP_DEG * (0.5 - 0.5 * math.cos(head_phase * 1.7))
+    roll = THINKING_ROLL_AMP_DEG * math.sin(head_phase * 0.5)
     body_yaw = THINKING_BODY_AMP_DEG * math.sin(2.0 * math.pi * t / THINKING_BODY_PERIOD_S)
 
     ant_phase = 2.0 * math.pi * t / THINKING_ANTENNA_PERIOD_S
@@ -149,9 +165,13 @@ def _nodding_frame(t: float) -> MotionFrame:
 
 def _responding_frame(t: float) -> MotionFrame:
     phase = 2.0 * math.pi * t / RESPONDING_PERIOD_S
-    yaw = 0.6 * RESPONDING_BOB_AMP_DEG * math.sin(phase * 0.7)
+    yaw = RESPONDING_YAW_AMP_DEG * math.sin(phase * 0.7)
     pitch = RESPONDING_PITCH_BIAS_DEG + RESPONDING_BOB_AMP_DEG * math.sin(phase)
 
+    body_yaw = RESPONDING_BODY_AMP_DEG * math.sin(
+        2.0 * math.pi * t / RESPONDING_BODY_PERIOD_S
+    )
+
     ant_phase = 2.0 * math.pi * t / RESPONDING_ANTENNA_PERIOD_S
     right = RESPONDING_ANTENNA_AMP_DEG * math.sin(ant_phase)
     left = RESPONDING_ANTENNA_AMP_DEG * math.sin(ant_phase + math.pi)
@@ -159,28 +179,43 @@ def _responding_frame(t: float) -> MotionFrame:
     return MotionFrame(
         head=create_head_pose(yaw=yaw, pitch=pitch, degrees=True),
         antennas=[_deg(right), _deg(left)],
-        body_yaw=0.0,
+        body_yaw=_deg(body_yaw),
     )
 
 
-def _speaking_frame(t: float) -> MotionFrame:
+def _speaking_frame(t: float, audio_env: float) -> MotionFrame:
+    """Speaking state with audio-reactive overlay.
+
+    ``audio_env`` ∈ [0, ~1.5] is the current loudness; we add a head dip,
+    a small yaw flick, and an antenna lift proportional to it on top of
+    the steady speaking sway.
+    """
+    env = max(0.0, float(audio_env))
+
     nod_phase = 2.0 * math.pi * t / SPEAKING_NOD_PERIOD_S
-    pitch = SPEAKING_PITCH_BIAS_DEG + SPEAKING_NOD_AMP_DEG * (
+    base_pitch = SPEAKING_PITCH_BIAS_DEG + SPEAKING_NOD_AMP_DEG * (
         0.5 - 0.5 * math.cos(nod_phase)
     )
+    pitch = base_pitch + AUDIO_REACTIVE_PITCH_DEG * env
+
     yaw_phase = 2.0 * math.pi * t / SPEAKING_YAW_PERIOD_S
-    yaw = SPEAKING_YAW_AMP_DEG * math.sin(yaw_phase)
+    yaw = SPEAKING_YAW_AMP_DEG * math.sin(yaw_phase) + AUDIO_REACTIVE_YAW_DEG * (
+        env * math.sin(2.0 * math.pi * t / 0.5)
+    )
+    roll = SPEAKING_ROLL_AMP_DEG * math.sin(2.0 * math.pi * t / SPEAKING_ROLL_PERIOD_S)
 
     body_yaw = SPEAKING_BODY_AMP_DEG * math.sin(
         2.0 * math.pi * t / SPEAKING_BODY_PERIOD_S
-    )
+    ) + AUDIO_REACTIVE_BODY_DEG * env * 0.5
 
     ant_phase = 2.0 * math.pi * t / SPEAKING_ANTENNA_PERIOD_S
-    right = SPEAKING_ANTENNA_AMP_DEG * math.sin(ant_phase)
-    left = SPEAKING_ANTENNA_AMP_DEG * math.sin(ant_phase + math.pi * 0.7)
+    base_right = SPEAKING_ANTENNA_AMP_DEG * math.sin(ant_phase)
+    base_left = SPEAKING_ANTENNA_AMP_DEG * math.sin(ant_phase + math.pi * 0.7)
+    right = base_right + AUDIO_REACTIVE_ANTENNA_DEG * env
+    left = base_left + AUDIO_REACTIVE_ANTENNA_DEG * env
 
     return MotionFrame(
-        head=create_head_pose(yaw=yaw, pitch=pitch, degrees=True),
+        head=create_head_pose(yaw=yaw, pitch=pitch, roll=roll, degrees=True),
         antennas=[_deg(right), _deg(left)],
         body_yaw=_deg(body_yaw),
     )
@@ -189,20 +224,24 @@ def _speaking_frame(t: float) -> MotionFrame:
 def _acknowledge_frame(t: float) -> MotionFrame:
     p = max(0.0, min(1.0, t / ACK_DURATION_S))
     burst = math.sin(math.pi * p)
-    pitch = ACK_PITCH_DOWN_DEG * burst * 0.5
+    pitch = ACK_PITCH_DOWN_DEG * burst * 0.6
     ant_deg = ACK_ANTENNA_DEG * burst
+    body_yaw = ACK_BODY_DEG * burst * 0.5
     return MotionFrame(
         head=create_head_pose(pitch=pitch, degrees=True),
         antennas=[_deg(ant_deg), _deg(ant_deg)],
-        body_yaw=0.0,
+        body_yaw=_deg(body_yaw),
     )
 
 
-def pose_for_state(state: State, t_state: float) -> MotionFrame:
+def pose_for_state(
+    state: State, t_state: float, audio_env: float = 0.0
+) -> MotionFrame:
     """Return the motion frame for `state` at local time `t_state` (seconds).
 
     `t_state` resets to 0 every time the state changes, so each pattern
-    starts from a clean phase.
+    starts from a clean phase. ``audio_env`` is only consulted by the
+    ``speaking`` state — for everything else it is ignored.
     """
     if state == "thinking":
         return _thinking_frame(t_state)
@@ -211,7 +250,7 @@ def pose_for_state(state: State, t_state: float) -> MotionFrame:
     if state == "responding":
         return _responding_frame(t_state)
     if state == "speaking":
-        return _speaking_frame(t_state)
+        return _speaking_frame(t_state, audio_env)
     if state == "acknowledge_toggle":
         return _acknowledge_frame(t_state)
     return _idle_frame(t_state)

marginalia/speech.py

@@ -1,20 +1,20 @@
 """Text-to-speech for Marginalia.
 
-Three synthesis backends, tried in order until one returns real audio:
-
-* **Hugging Face Inference TTS** — ``InferenceClient.text_to_speech()``.
-  Default model is ``facebook/mms-tts-eng`` because it works directly with
-  a plain text input; ``microsoft/speecht5_tts`` and many other TTS models
-  require speaker-embedding tensors that the simple API doesn't supply.
-* **pyttsx3** — offline. Reliable on macOS (NSSpeech), but on Linux/espeak
-  it sometimes writes a header-only WAV; we validate the output and treat
-  empty WAVs as a failure so we move on instead of silently playing
-  nothing.
-* **gTTS** — Google TTS over HTTP, MP3 output. Cloud, but no token, and
-  almost always works as a last resort.
+Backends, in order of preference:
+
+* **edge-tts** — Microsoft Edge's neural voices over HTTP. Fast (sub-second
+  per paragraph), natural-sounding, free, no API key. The primary backend
+  and the only one with a curated voice picker.
+* **Hugging Face Inference TTS** — falls back to ``InferenceClient.text_to_speech``
+  (default ``facebook/mms-tts-eng``) if edge-tts can't reach Microsoft.
+* **pyttsx3** — offline. Often produces an empty WAV on Linux/espeak; we
+  validate the file and treat empty output as a failure so the next
+  backend takes over.
+* **gTTS** — Google TTS over HTTP. Last-resort cloud fallback.
 
 ``MARGINALIA_TTS_MODE``:
-    ``auto`` (default) — HF → pyttsx3 → gTTS.
+    ``auto`` (default) — edge → hf → pyttsx3 → gtts.
+    ``edge``           — edge-tts only.
     ``hf``             — HF only.
     ``offline``        — pyttsx3 only.
     ``gtts``           — gTTS only.
@@ -22,10 +22,18 @@ Three synthesis backends, tried in order until one returns real audio:
 ``synthesize()`` always returns mono ``float32`` PCM at **16 kHz**, ready
 for ``reachy_mini.media.push_audio_sample()``. Empty input or total
 failure returns an empty array (caller treats it as silence).
+
+This module also exposes:
+* ``split_for_speech(text)`` — break a clean answer into paragraph chunks
+  for low-latency streaming TTS.
+* ``compute_envelope(pcm)`` — RMS amplitude per ~50 ms frame, used to
+  drive audio-reactive head and antenna motion while the robot speaks.
+* ``VOICES`` and ``DEFAULT_VOICE`` — curated edge-tts voice catalogue.
 """
 
 from __future__ import annotations
 
+import asyncio
 import io
 import logging
 import os
@@ -43,6 +51,7 @@ logger = logging.getLogger("marginalia.speech")
 ROBOT_SAMPLE_RATE: Final[int] = 16000
 MIN_PCM_SAMPLES: Final[int] = 800
 MIN_WAV_BYTES: Final[int] = 1024
+ENVELOPE_FRAME_MS: Final[int] = 50
 
 TTS_MODE = os.environ.get("MARGINALIA_TTS_MODE", "auto").lower().strip()
 HF_TTS_MODEL = os.environ.get(
@@ -53,6 +62,29 @@ PYTTSX3_VOICE = os.environ.get("MARGINALIA_TTS_VOICE", "")
 GTTS_LANG = os.environ.get("MARGINALIA_TTS_GTTS_LANG", "en")
 TTS_DEBUG = os.environ.get("MARGINALIA_TTS_DEBUG", "").lower() in {"1", "true", "yes"}
 
+EDGE_RATE = os.environ.get("MARGINALIA_EDGE_RATE", "+0%")
+EDGE_PITCH = os.environ.get("MARGINALIA_EDGE_PITCH", "+0Hz")
+
+# Curated voice catalogue. `id` is the edge-tts voice short-name, `label`
+# is what the UI shows. The first entry is the default if no voice is
+# pinned via env var or per-call.
+VOICES: list[dict[str, str]] = [
+    {"id": "en-US-AvaNeural", "label": "Ava — calm reader (US, female)"},
+    {"id": "en-US-AndrewNeural", "label": "Andrew — companion (US, male)"},
+    {"id": "en-US-EmmaNeural", "label": "Emma — bright reader (US, female)"},
+    {"id": "en-US-GuyNeural", "label": "Guy — warm narrator (US, male)"},
+    {"id": "en-US-DavisNeural", "label": "Davis — soothsayer (US, male, deep)"},
+    {"id": "en-GB-SoniaNeural", "label": "Sonia — librarian (UK, female)"},
+    {"id": "en-GB-RyanNeural", "label": "Ryan — bookish (UK, male)"},
+    {"id": "en-IE-EmilyNeural", "label": "Emily — Joycean (IE, female)"},
+    {"id": "en-AU-NatashaNeural", "label": "Natasha — antipodean (AU, female)"},
+]
+VOICE_IDS = {v["id"] for v in VOICES}
+DEFAULT_VOICE = os.environ.get("MARGINALIA_VOICE", VOICES[0]["id"]).strip()
+if DEFAULT_VOICE not in VOICE_IDS:
+    DEFAULT_VOICE = VOICES[0]["id"]
+
+
 _CITATION_RE = re.compile(r"\[\d+\]")
 _URL_RE = re.compile(r"https?://\S+")
 _MD_LINK_RE = re.compile(r"\[([^\]]+)\]\([^)]+\)")
@@ -65,14 +97,14 @@ _HR_RE = re.compile(r"^---+$", flags=re.MULTILINE)
 
 
 def clean_for_speech(markdown: str) -> str:
-    """Reduce a Marginalia markdown answer to a speakable string.
-
-    Drops the "How to read this" + "Further reading" sections (those stay
-    on the page only), then strips citations, URLs, code spans, and
-    markdown emphasis.
-    """
+    """Reduce a Marginalia markdown answer to a speakable string."""
     text = markdown or ""
-    for cut in ("### How to read this", "### Further reading", "## How to read", "## Further reading"):
+    for cut in (
+        "### How to read this",
+        "### Further reading",
+        "## How to read",
+        "## Further reading",
+    ):
         idx = text.find(cut)
         if idx != -1:
             text = text[:idx]
@@ -92,8 +124,63 @@ def clean_for_speech(markdown: str) -> str:
     return text.strip()
 
 
+def split_for_speech(text: str, target_chars: int = 280) -> list[str]:
+    """Break clean prose into roughly paragraph-sized speakable chunks.
+
+    Splits on blank lines first; very small pieces are merged into the
+    next chunk so we don't waste a TTS round-trip on a sentence fragment.
+    """
+    text = (text or "").strip()
+    if not text:
+        return []
+
+    paragraphs = [p.strip() for p in re.split(r"\n\s*\n", text) if p.strip()]
+    if not paragraphs:
+        return []
+
+    chunks: list[str] = []
+    buf = ""
+    for p in paragraphs:
+        if not buf:
+            buf = p
+        elif len(buf) < target_chars and len(p) < target_chars:
+            buf = buf + "\n\n" + p
+            if len(buf) >= target_chars:
+                chunks.append(buf)
+                buf = ""
+        else:
+            chunks.append(buf)
+            buf = p
+    if buf:
+        chunks.append(buf)
+    return chunks
+
+
+def compute_envelope(
+    pcm: np.ndarray, frame_ms: int = ENVELOPE_FRAME_MS
+) -> tuple[np.ndarray, float]:
+    """Return (envelope, frame_dt_seconds).
+
+    Envelope is RMS amplitude per ``frame_ms`` window, normalised so the
+    99th percentile maps to roughly 1.0. Used by the run loop to drive
+    audio-reactive head/antenna motion while the robot speaks.
+    """
+    if pcm is None or len(pcm) == 0:
+        return np.zeros(0, dtype=np.float32), frame_ms / 1000.0
+    samples_per_frame = max(1, int(ROBOT_SAMPLE_RATE * frame_ms / 1000))
+    n_frames = len(pcm) // samples_per_frame
+    if n_frames <= 0:
+        return np.zeros(0, dtype=np.float32), frame_ms / 1000.0
+    trimmed = pcm[: n_frames * samples_per_frame]
+    framed = trimmed.reshape(n_frames, samples_per_frame).astype(np.float32, copy=False)
+    rms = np.sqrt(np.mean(framed * framed, axis=1) + 1e-9)
+    p99 = float(np.quantile(rms, 0.99))
+    if p99 > 0:
+        rms = np.clip(rms / p99, 0.0, 1.5).astype(np.float32)
+    return rms, frame_ms / 1000.0
+
+
 def _ensure_mono_float32(samples: np.ndarray) -> np.ndarray:
-    """Convert any sample buffer into mono float32 in [-1, 1]."""
     a = np.asarray(samples)
     if a.ndim == 2:
         a = a.mean(axis=1)
@@ -126,8 +213,30 @@ def _decode_audio_bytes(audio_bytes: bytes) -> np.ndarray:
     return _resample(samples, src_sr=int(sr), dst_sr=ROBOT_SAMPLE_RATE)
 
 
+def _synthesize_edge(text: str, voice: str) -> np.ndarray:
+    """Microsoft Edge neural TTS over HTTP. Fast, natural, no key."""
+    import edge_tts
+
+    async def _gather() -> bytes:
+        communicate = edge_tts.Communicate(
+            text, voice, rate=EDGE_RATE, pitch=EDGE_PITCH
+        )
+        chunks = bytearray()
+        async for chunk in communicate.stream():
+            if chunk.get("type") == "audio":
+                chunks.extend(chunk["data"])
+        return bytes(chunks)
+
+    audio_bytes = asyncio.run(_gather())
+    if not audio_bytes:
+        raise RuntimeError(f"edge-tts ({voice}) returned empty audio")
+    samples = _decode_audio_bytes(audio_bytes)
+    if len(samples) < MIN_PCM_SAMPLES:
+        raise RuntimeError(f"edge-tts ({voice}) produced only {len(samples)} samples")
+    return samples
+
+
 def _synthesize_hf(text: str) -> np.ndarray:
-    """HF Inference TTS. Raises on any failure so caller can fall back."""
     from huggingface_hub import InferenceClient
 
     client = InferenceClient(model=HF_TTS_MODEL)
@@ -143,12 +252,6 @@ def _synthesize_hf(text: str) -> np.ndarray:
 
 
 def _synthesize_pyttsx3(text: str) -> np.ndarray:
-    """Offline TTS via pyttsx3 → WAV → resample to 16 kHz.
-
-    On Linux + espeak, ``save_to_file`` sometimes writes a header-only WAV
-    with no audio data. We validate the file size after `runAndWait()` and
-    raise so the next backend can take over.
-    """
     import pyttsx3
 
     engine = pyttsx3.init()
@@ -187,7 +290,6 @@ def _synthesize_pyttsx3(text: str) -> np.ndarray:
 
 
 def _synthesize_gtts(text: str) -> np.ndarray:
-    """Google TTS over HTTP. MP3 output decoded by libsndfile."""
     from gtts import gTTS
 
     buf = io.BytesIO()
@@ -203,49 +305,45 @@ def _synthesize_gtts(text: str) -> np.ndarray:
     return samples
 
 
-_BACKENDS = {
-    "hf": _synthesize_hf,
-    "offline": _synthesize_pyttsx3,
-    "gtts": _synthesize_gtts,
-}
-
-
-def synthesize(text: str) -> np.ndarray:
+def synthesize(text: str, voice: str | None = None) -> np.ndarray:
     """Return mono float32 PCM at 16 kHz speaking ``text``.
 
-    Honours ``MARGINALIA_TTS_MODE``. Empty input returns an empty array.
-    Errors return an empty array (with a warning) so the caller never
-    blocks the answer pipeline on a broken speaker.
+    Honours ``MARGINALIA_TTS_MODE``. ``voice`` only affects the edge-tts
+    backend; the others ignore it. Empty input or total failure returns
+    an empty array so the caller never blocks the answer pipeline on a
+    broken speaker.
     """
     text = (text or "").strip()
     if not text:
         return np.zeros(0, dtype=np.float32)
+    voice = (voice or DEFAULT_VOICE).strip()
+    if voice not in VOICE_IDS:
+        voice = DEFAULT_VOICE
 
-    mode = TTS_MODE if TTS_MODE in {"auto", "hf", "offline", "gtts"} else "auto"
-    tries: list[str]
+    mode = TTS_MODE if TTS_MODE in {"auto", "edge", "hf", "offline", "gtts"} else "auto"
     if mode == "auto":
-        tries = ["hf", "offline", "gtts"]
+        tries = ["edge", "hf", "offline", "gtts"]
     else:
         tries = [mode]
 
     last_err: Exception | None = None
     for backend in tries:
-        fn = _BACKENDS.get(backend)
-        if fn is None:
-            continue
         try:
-            logger.info(
-                "synthesize via %s%s",
-                backend,
-                f" ({HF_TTS_MODEL})" if backend == "hf" else "",
-            )
-            return fn(text)
+            if backend == "edge":
+                logger.info("synthesize via edge-tts (%s)", voice)
+                return _synthesize_edge(text, voice)
+            if backend == "hf":
+                logger.info("synthesize via hf (%s)", HF_TTS_MODEL)
+                return _synthesize_hf(text)
+            if backend == "offline":
+                logger.info("synthesize via pyttsx3")
+                return _synthesize_pyttsx3(text)
+            if backend == "gtts":
+                logger.info("synthesize via gtts")
+                return _synthesize_gtts(text)
         except Exception as e:
             last_err = e
-            logger.warning(
-                "TTS backend %s failed: %r",
-                backend, e,
-            )
+            logger.warning("TTS backend %s failed: %r", backend, e)
             if TTS_DEBUG:
                 logger.warning("traceback:\n%s", traceback.format_exc())
 

marginalia/static/index.html

@@ -42,6 +42,10 @@
                         <span class="pill-icon" aria-hidden="true"></span>
                         <span class="pill-label">Read aloud</span>
                     </button>
+                    <label class="voice-picker" for="voice-select" title="Choose Reachy's reading voice">
+                        <span class="voice-picker-label">Voice</span>
+                        <select id="voice-select" class="voice-select" aria-label="Reading voice"></select>
+                    </label>
                 </div>
                 <p class="hint">Tip — twist either antenna to mute or unmute the spoken answer.</p>
             </form>

marginalia/static/main.js

@@ -14,9 +14,12 @@
     const sourcesList = document.getElementById("sources-list");
     const speechPill = document.getElementById("speech-pill");
     const pillLabel = speechPill ? speechPill.querySelector(".pill-label") : null;
+    const voiceSelect = document.getElementById("voice-select");
 
     let speechEnabled = true;
     let speakingSpoken = false;
+    let currentVoice = null;
+    let voicesPopulated = false;
 
     if (window.marked && marked.setOptions) {
         marked.setOptions({ gfm: true, breaks: false });
@@ -47,12 +50,34 @@
         else speechPill.classList.remove("speaking");
     }
 
+    function populateVoices(voices, selected) {
+        if (!voiceSelect || voicesPopulated) return;
+        voiceSelect.innerHTML = "";
+        for (const v of voices || []) {
+            const opt = document.createElement("option");
+            opt.value = v.id;
+            opt.textContent = v.label || v.id;
+            if (v.id === selected) opt.selected = true;
+            voiceSelect.appendChild(opt);
+        }
+        voicesPopulated = true;
+    }
+
+    function applyVoice(voiceId) {
+        currentVoice = voiceId;
+        if (voiceSelect && voiceSelect.value !== voiceId) {
+            voiceSelect.value = voiceId;
+        }
+    }
+
     async function loadInitialState() {
         try {
             const resp = await fetch("/api/state", { headers: { Accept: "application/json" } });
             if (resp.ok) {
                 const data = await resp.json();
                 applySpeechEnabled(data.speech_enabled !== false);
+                if (Array.isArray(data.voices)) populateVoices(data.voices, data.voice);
+                if (data.voice) applyVoice(data.voice);
                 return;
             }
         } catch (_) { /* offline-first: keep default true */ }
@@ -71,6 +96,20 @@
         }
     }
 
+    async function changeVoice(voiceId) {
+        try {
+            const resp = await fetch("/api/speech/voice", {
+                method: "POST",
+                headers: { "Content-Type": "application/json" },
+                body: JSON.stringify({ voice: voiceId }),
+            });
+            if (resp.ok) {
+                const data = await resp.json();
+                if (data.voice) applyVoice(data.voice);
+            }
+        } catch (_) { /* keep last value */ }
+    }
+
     function renderAnswer(text) {
         let html;
         if (window.marked && typeof marked.parse === "function") {
@@ -168,12 +207,15 @@
                 try {
                     const d = JSON.parse(ev.data);
                     if (typeof d.speech_enabled === "boolean") applySpeechEnabled(d.speech_enabled);
+                    if (Array.isArray(d.voices)) populateVoices(d.voices, d.voice);
+                    if (d.voice) applyVoice(d.voice);
                 } catch (_) {}
             });
             es.addEventListener("state_changed", (ev) => {
                 try {
                     const d = JSON.parse(ev.data);
                     if (typeof d.speech_enabled === "boolean") applySpeechEnabled(d.speech_enabled);
+                    if (typeof d.voice === "string") applyVoice(d.voice);
                 } catch (_) {}
             });
             es.onerror = () => { /* let the browser reconnect */ };
@@ -218,6 +260,8 @@
                     setSpeakingVisual(true);
                     setStatus("Reachy is reading aloud", "speaking");
                     speakingSpoken = true;
+                } else if (ev.event === "chunk_played") {
+                    /* Each paragraph push lands here; keeps the UI lively. */
                 } else if (ev.event === "spoken_done") {
                     setSpeakingVisual(false);
                     setStatus("", "");
@@ -242,6 +286,13 @@
 
     if (speechPill) speechPill.addEventListener("click", toggleSpeech);
 
+    if (voiceSelect) {
+        voiceSelect.addEventListener("change", (e) => {
+            const v = e.target.value;
+            if (v) changeVoice(v);
+        });
+    }
+
     form.addEventListener("submit", (e) => {
         e.preventDefault();
         const q = questionEl.value.trim();

marginalia/static/style.css

@@ -278,6 +278,60 @@ button.seal:disabled {
     50%      { transform: scale(1.25); }
 }
 
+/* voice picker */
+.voice-picker {
+    display: inline-flex;
+    align-items: center;
+    gap: 0.45rem;
+    padding: 0.3rem 0.55rem 0.3rem 0.7rem;
+    background: rgba(27, 33, 56, 0.05);
+    border: 1px solid var(--rule);
+    border-radius: 999px;
+    transition: border-color 0.15s ease, background 0.15s ease;
+}
+
+.voice-picker:hover {
+    border-color: var(--emerald-mid);
+    background: rgba(14, 53, 35, 0.06);
+}
+
+.voice-picker-label {
+    font-family: "IM Fell English SC", "Cormorant Garamond", serif;
+    font-size: 0.72rem;
+    letter-spacing: 0.16em;
+    text-transform: uppercase;
+    color: var(--ink-faint);
+}
+
+.voice-select {
+    appearance: none;
+    -webkit-appearance: none;
+    border: none;
+    background: transparent;
+    color: var(--ink);
+    font-family: "EB Garamond", "Garamond", serif;
+    font-size: 0.92rem;
+    font-style: italic;
+    cursor: pointer;
+    padding: 0.05rem 1.4rem 0.05rem 0.1rem;
+    background-image: url("data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' width='10' height='6' viewBox='0 0 10 6'><path fill='%2358402f' d='M0 0l5 6 5-6z'/></svg>");
+    background-repeat: no-repeat;
+    background-position: right 0.25rem center;
+    background-size: 8px 5px;
+    max-width: 16em;
+}
+
+.voice-select:focus {
+    outline: none;
+    color: var(--emerald);
+}
+
+.voice-select option {
+    color: var(--ink);
+    background: var(--parchment);
+    font-style: normal;
+}
+
 .hint {
     margin: 0.2rem 0 0;
     font-size: 0.82rem;
@@ -575,6 +629,16 @@ button.seal:disabled {
         margin-left: 0;
     }
 
+    .voice-picker {
+        margin-left: 0;
+        flex: 1 1 auto;
+    }
+
+    .voice-select {
+        max-width: 100%;
+        width: 100%;
+    }
+
     .sources-list li {
         grid-template-columns: 1fr;
     }

pyproject.toml

@@ -15,6 +15,7 @@ dependencies = [
     "reachy-mini>=1.7.0,<2.0.0",
     "huggingface-hub>=1.0,<2.0",
     "requests>=2.28",
+    "edge-tts>=6.1.10",
     "pyttsx3>=2.90",
     "soundfile>=0.12",
     "scipy>=1.10",