fix(stt): transcode browser webm/opus → wav before Sarvam (D-024)

Root cause: Browser MediaRecorder emits audio/webm;codecs=opus by default.
Sarvam STT only accepts WAV/MP3/FLAC/OGG/M4A. The /api/transcribe endpoint
silently coerced webm→wav in the Content-Type but uploaded the raw webm
bytes, so Sarvam returned HTTP 400 Bad Request:

STT failed: HTTPStatusError: Client error '400 Bad Request' for url
'https://api.sarvam.ai/speech-to-text'

Fix in three layers:

1) sarvam_stt.py — _transcode_to_wav() helper using pydub (ffmpeg-backed)
converts any non-native container to 16 kHz mono WAV before upload.
Also short-audio guard: <1 KB bytes return empty STTResult instead of
500ing on Sarvam's silence rejection.

2) main.py — extend the format whitelist to include webm/opus/mp4 so the
real extension is passed through to the provider (provider then decides
to transcode).

3) Dockerfile — add ffmpeg apt package (pydub's runtime dependency).
requirements.txt — add pydub==0.25.1.

This is the first of three live-product issues surfaced today:
Issue 3 (this commit): STT 400 errors
Issue 2 + 4 (next): fact-find recording STT garbage as profile fields
AND character-splitting health_conditions list
Issue 1 (last): full-duplex barge-in / interrupt-while-speaking

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Dockerfile

@@ -25,12 +25,15 @@ RUN npm run build
 FROM python:3.11-slim
 WORKDIR /app
 
-# System deps for pdfplumber + torch CPU + sentence-transformers
+# System deps:
+#   pdfplumber + torch CPU + sentence-transformers → build-essential, libpoppler
+#   pydub (webm→wav transcode for Sarvam STT) → ffmpeg
 RUN apt-get update && apt-get install -y --no-install-recommends \
     build-essential \
     libpoppler-cpp-dev \
     pkg-config \
     poppler-utils \
+    ffmpeg \
     && rm -rf /var/lib/apt/lists/*
 
 # Install Python deps

backend/main.py

@@ -233,10 +233,12 @@ async def transcribe(
     t0 = time.time()
     audio_bytes = await file.read()
     ext = (file.filename or "audio.wav").rsplit(".", 1)[-1].lower()
+    # Pass the real extension through; sarvam_stt.py transcodes non-native
+    # containers (webm/opus from browser MediaRecorder) to WAV before upload.
     try:
         result = await get_stt().transcribe(
             audio_bytes=audio_bytes,
-            audio_format=ext if ext in ("wav", "mp3", "flac", "ogg", "m4a") else "wav",
+            audio_format=ext if ext in ("wav", "mp3", "flac", "ogg", "m4a", "webm", "opus", "mp4") else "wav",
             language_code=language_code,
         )
     except Exception as e:

backend/providers/sarvam_stt.py

@@ -32,15 +32,55 @@ class SarvamSTT(STTProvider):
         if not self.api_key:
             raise RuntimeError("SARVAM_API_KEY not set in .env")
 
+    # Sarvam STT accepts these container formats per their API. webm is NOT
+    # in this list — browser MediaRecorder defaults to webm/opus, so we
+    # transcode webm → wav (16kHz mono) on the fly via pydub before calling
+    # Sarvam. This is the fix for the live "400 Bad Request" we hit when
+    # the browser's webm bytes were uploaded as if they were wav.
+    _SARVAM_NATIVE_FORMATS = {"wav", "mp3", "flac", "ogg", "m4a"}
+
+    @staticmethod
+    def _transcode_to_wav(audio_bytes: bytes, src_format: str) -> bytes:
+        """Convert any pydub-readable container to 16 kHz mono WAV.
+
+        Sarvam's recommended sampling rate is 16 kHz mono — what Saarika
+        was trained on. Down-mixing + resampling at the gateway also
+        prevents Sarvam from doing it server-side, which keeps latency tight.
+        """
+        from pydub import AudioSegment
+        audio = AudioSegment.from_file(io.BytesIO(audio_bytes), format=src_format)
+        audio = audio.set_frame_rate(16000).set_channels(1).set_sample_width(2)
+        buf = io.BytesIO()
+        audio.export(buf, format="wav")
+        return buf.getvalue()
+
     async def transcribe(
         self,
         audio_bytes: bytes,
         audio_format: str = "wav",
         language_code: Optional[str] = None,
     ) -> STTResult:
+        if not audio_bytes or len(audio_bytes) < 1024:
+            # < 1 KB audio is almost certainly silence or a record-and-immediately-stop;
+            # Sarvam 400s on these. Surface a clean empty result instead of a 500.
+            return STTResult(text="", language_code=language_code, confidence=0.0, raw={"reason": "audio_too_short"})
+
+        fmt = (audio_format or "wav").lower().lstrip(".")
+        # Browser MediaRecorder uses webm/opus by default; Sarvam rejects it.
+        # Transcode in-process when we get a non-native format.
+        if fmt not in self._SARVAM_NATIVE_FORMATS:
+            try:
+                audio_bytes = self._transcode_to_wav(audio_bytes, fmt)
+                fmt = "wav"
+            except Exception as e:
+                # If pydub/ffmpeg fails (e.g., truly corrupt audio), let Sarvam
+                # see the original bytes and return its own error rather than
+                # silently swallowing.
+                pass
+
         url = f"{settings.SARVAM_BASE_URL}{settings.SARVAM_STT_PATH}"
         files = {
-            "file": (f"audio.{audio_format}", io.BytesIO(audio_bytes), f"audio/{audio_format}"),
+            "file": (f"audio.{fmt}", io.BytesIO(audio_bytes), f"audio/{fmt}"),
         }
         data = {"model": self.model}
         if language_code:

requirements.txt

@@ -26,6 +26,11 @@ duckdb==1.1.3
 sentence-transformers==5.1.2
 torch==2.11.0  # CPU-only fine for our usage
 
+# Audio transcoding — browser MediaRecorder emits webm/opus by default;
+# Sarvam STT accepts WAV/MP3/FLAC/OGG/M4A but NOT webm. pydub (ffmpeg-backed)
+# converts in-process. The Dockerfile installs the ffmpeg apt package.
+pydub==0.25.1
+
 # Utility
 numpy==1.26.4
 tqdm==4.66.6