{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Voice Layer — Usage Guide\n", "\n", "A quick tour of the `voice/` package (`src/voice/`) for **Fatema** (RAG) and **Shahd** (UI).\n", "This is a teaching notebook, not a test suite — see `tests/` for the real test suite and\n", "`src/voice/CONTRACT.md` for the full frozen-contract reference.\n", "\n", "**What it does:** Arabic (MSA) Speech-to-Text and Text-to-Speech, behind two functions:\n", "\n", "```python\n", "transcribe_audio(audio) -> TranscriptionResult # speech -> MSA text\n", "synthesize_speech(text) -> SynthesisResult # MSA text -> spoken .wav\n", "```\n", "\n", "**Mock vs real:**\n", "- **Mock mode (default — this whole notebook)** — zero setup: no models, no GPU, no API key.\n", " `transcribe_audio` returns a fixed sample transcript; `synthesize_speech` writes a short\n", " placeholder tone. Perfect for building the RAG/UI plumbing before real models are needed.\n", "- **Real mode** — Whisper Large-v3 for STT, Azure Neural or offline Piper for TTS. Switched on\n", " with one env var (`VOICE_BACKEND=real`). Covered at the end of this notebook.\n", "\n", "Every failure is an **exception** under `VoiceError` — you never get an error string back,\n", "you always `try`/`except`. That's the one thing to remember." ] }, { "cell_type": "code", "execution_count": 4, "id": "70e45b64", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "voice package imported - running in mock mode by default, no setup needed.\n" ] } ], "source": [ "import os, sys\n", "\n", "# Find src/ whether this notebook is launched with the repo root as the\n", "# working directory (root/voice_usage_guide.ipynb, the old location) or with\n", "# docs/ as the working directory (docs/voice_usage_guide.ipynb, here now).\n", "for _candidate in (\"src\", \"../src\"):\n", " if os.path.isdir(_candidate):\n", " sys.path.insert(0, _candidate)\n", " break\n", "else:\n", " raise RuntimeError(\"could not find src/ - run this notebook from the repo root or docs/\")\n", "\n", "# Force mock mode explicitly, before importing voice. voice/config.py loads a\n", "# local .env if python-dotenv is installed; if a .env left over from real-mode\n", "# testing sets VOICE_BACKEND=real, this notebook would silently try to run\n", "# Whisper/Azure instead of mock. Setting it here first wins (dotenv defaults\n", "# to never overriding an already-set env var), so this notebook is always\n", "# guaranteed to run in mock mode regardless of your local .env/shell state.\n", "os.environ[\"VOICE_BACKEND\"] = \"real\"\n", "\n", "from voice import (\n", " transcribe_audio, transcribe_audio_async,\n", " synthesize_speech, synthesize_speech_async,\n", " VoiceError, AudioFormatError, TextValidationError,\n", " AZURE_VOICES,\n", ")\n", "\n", "print(\"voice package imported - running in mock mode by default, no setup needed.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. Speech → Text\n", "\n", "`transcribe_audio` takes a file path, raw bytes, or a numpy waveform. In mock mode it doesn't\n", "actually listen to the audio — it just validates the input and returns a fixed sample\n", "transcript, so you can build against real-shaped data immediately." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "text: ما هي المهارات التي سأكتسبها عند دراسة تخصص الذكاء الاصطناعي وعلم البيانات؟\n", "backend: mock-stt\n", "language: ar\n" ] } ], "source": [ "# In real usage this would be a real recording (bytes from the browser, or a file path).\n", "# In mock mode, any non-empty bytes works - the content is never inspected.\n", "fake_audio = b\"...pretend this is a recording from the UI...\"\n", "\n", "transcript = transcribe_audio(fake_audio)\n", "\n", "print(\"text: \", transcript.text)\n", "print(\"backend: \", transcript.backend) # \"mock-stt\" here; \"whisper-large-v3\" in real mode\n", "print(\"language:\", transcript.language)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. Text → Speech\n", "\n", "`synthesize_speech` takes MSA Arabic text and writes a `.wav` file. In mock mode it writes a\n", "short placeholder tone (a real, valid, playable wav) instead of calling a TTS engine." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "audio_path: C:\\Users\\sajaa\\AppData\\Local\\Temp\\tmpfni6f30z.wav\n", "backend: azure:ar-SA-HamedNeural\n", "duration: 8.41 sec\n" ] } ], "source": [ "answer_text = \"يتطلب تخصص الذكاء الاصطناعي وعلم البيانات إتمام مئة وستة وعشرين ساعة معتمدة للتخرج.\"\n", "\n", "speech = synthesize_speech(answer_text)\n", "\n", "print(\"audio_path:\", speech.audio_path)\n", "print(\"backend: \", speech.backend) # \"mock-tts\" here; \"piper:ar_JO-kareem\" / \"azure:...\" for real\n", "print(\"duration: \", speech.audio_duration_sec, \"sec\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. Error handling — the one contract to remember\n", "\n", "The voice layer **raises**, it never returns an error string. Every exception it can raise\n", "inherits from `VoiceError`, so `except VoiceError` always catches a voice-layer failure if\n", "you don't care about the specific reason. Catch a more specific subclass first if you want a\n", "tailored message (e.g. \"I didn't catch that\" for silence vs. \"couldn't read that file\" for a\n", "bad upload)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "STT: bad input caught as AudioFormatError - ok\n", "TTS: empty text caught as TextValidationError - ok\n" ] } ], "source": [ "# A bad input type -> AudioFormatError (a subclass of VoiceError)\n", "try:\n", " transcribe_audio(12345) # not a path, bytes, or numpy array\n", "except AudioFormatError:\n", " print(\"STT: bad input caught as AudioFormatError -\", \"ok\")\n", "except VoiceError:\n", " print(\"STT: caught as a generic VoiceError\")\n", "\n", "# Empty text -> TextValidationError (also a VoiceError)\n", "try:\n", " synthesize_speech(\"\")\n", "except TextValidationError:\n", " print(\"TTS: empty text caught as TextValidationError -\", \"ok\")\n", "except VoiceError:\n", " print(\"TTS: caught as a generic VoiceError\")\n", "\n", "# This is the pattern to use in the real RAG/UI loop - catch the base class\n", "# if you just want to know \"did the voice layer fail?\":\n", "try:\n", " text = transcribe_audio(fake_audio).text\n", "except VoiceError:\n", " text = None # show a fallback message to the student instead of crashing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. Async variants (for FastAPI)\n", "\n", "Both functions have `_async` twins that offload the (potentially slow, real-mode) work to a\n", "thread so an async event loop stays responsive. Same inputs, same return types, same\n", "exceptions." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "async STT backend: mock-stt\n", "async TTS backend: mock-tts\n" ] } ], "source": [ "async def demo_async():\n", " transcript = await transcribe_audio_async(fake_audio)\n", " speech = await synthesize_speech_async(transcript.text)\n", " return transcript, speech\n", "\n", "# In Jupyter, await the coroutine directly — the notebook already has a running loop\n", "async_transcript, async_speech = await demo_async()\n", "print(\"async STT backend:\", async_transcript.backend)\n", "print(\"async TTS backend:\", async_speech.backend)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. The end-to-end loop\n", "\n", "This is the shape of the real pipeline: audio in → transcribe → **Fatema's RAG answers the\n", "question** → synthesize → audio out. The only thing that changes between mock and real mode\n", "is which engine actually runs — this code never changes." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "answer text: يتطلب التخصص إتمام مئة وستة وعشرين ساعة معتمدة للتخرج.\n", "answer audio: C:\\Users\\sajaa\\AppData\\Local\\Temp\\tmparus70nv.wav\n" ] } ], "source": [ "def fake_rag_answer(question: str) -> str:\n", " \"\"\"PLACEHOLDER - this is where Fatema's real RAG pipeline plugs in.\n", " Takes the transcribed question, returns an MSA answer string.\"\"\"\n", " return \"يتطلب التخصص إتمام مئة وستة وعشرين ساعة معتمدة للتخرج.\"\n", "\n", "\n", "def voice_loop(audio):\n", " try:\n", " question = transcribe_audio(audio).text\n", " except VoiceError:\n", " return None, \"لم أفهم ما قلته، من فضلك حاول مرة أخرى.\" # \"I didn't catch that\"\n", "\n", " # <<< SWAP THIS LINE for Fatema's real RAG call >>>\n", " answer_text = fake_rag_answer(question)\n", "\n", " try:\n", " speech = synthesize_speech(answer_text)\n", " return speech.audio_path, answer_text\n", " except VoiceError:\n", " return None, answer_text # still show the text even if TTS failed\n", "\n", "\n", "audio_path, answer_text = voice_loop(fake_audio)\n", "print(\"answer text: \", answer_text)\n", "print(\"answer audio:\", audio_path)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. Switching to real mode + picking a voice\n", "\n", "Real mode is one env var away — nothing about the calls above changes.\n", "\n", "| Env var | Values | Purpose |\n", "|---|---|---|\n", "| `VOICE_BACKEND` | `mock` (default) / `real` | turns real Whisper + real TTS on |\n", "| `VOICE_TTS` | `azure` / `piper` / `auto` (default) | which TTS engine, real mode only |\n", "| `AZURE_SPEECH_KEY`, `AZURE_SPEECH_REGION` | your Azure key + region | needed for `azure`/`auto` |\n", "| `PIPER_MODEL_PATH` (optional), `PIPER_LENGTH_SCALE` | local `.onnx` override / speaking rate | `piper`/`auto` |\n", "\n", "`auto` (the real-mode default) tries Azure first and falls back to the fully-offline Piper\n", "engine on any Azure failure, so real mode still works with no API key at all — just slower,\n", "and needs `ffmpeg` and `piper` on PATH. Not sure your machine has what real mode needs?\n", "`voice.health_check()` reports what's importable/configured, with no model loading or\n", "network calls:\n", "\n", "```python\n", "from voice import health_check\n", "health_check()\n", "```\n", "\n", "Turning real mode on (not run in this notebook — needs real credentials/models installed):\n", "\n", "```python\n", "import os\n", "os.environ[\"VOICE_BACKEND\"] = \"real\"\n", "os.environ[\"VOICE_TTS\"] = \"auto\" # or \"azure\" / \"piper\"\n", "os.environ[\"AZURE_SPEECH_KEY\"] = \"...\"\n", "os.environ[\"AZURE_SPEECH_REGION\"] = \"eastus\"\n", "\n", "transcript = transcribe_audio(real_audio_bytes) # now runs Whisper Large-v3\n", "speech = synthesize_speech(transcript.text) # now runs Azure or Piper\n", "```\n", "\n", "**Picking a voice** works the same call in mock or real mode — `voice=\"male\"`/`\"female\"` is\n", "resolved via `AZURE_VOICES` and only affects the Azure engine (mock and Piper ignore it, but\n", "happily accept the argument so your code doesn't need an `if` for it):" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "available named voices: {'female': 'ar-SA-ZariyahNeural', 'male': 'ar-SA-HamedNeural'}\n", "male voice backend: mock-tts\n", "female voice backend: mock-tts\n" ] } ], "source": [ "print(\"available named voices:\", AZURE_VOICES) # {\"female\": \"...\", \"male\": \"...\"}\n", "\n", "male_voice = synthesize_speech(\"مرحباً\", voice=\"male\")\n", "female_voice = synthesize_speech(\"مرحباً\", voice=\"female\")\n", "print(\"male voice backend: \", male_voice.backend)\n", "print(\"female voice backend:\", female_voice.backend)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## That's it\n", "\n", "Two functions, one exception hierarchy, mock mode always available. For the full reference\n", "(exact signatures, every exception type, the settled design decisions) see\n", "`src/voice/CONTRACT.md`. For real bugs, open a GitHub issue against `voice/` with the failing\n", "input attached — not Slack." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 5 }