| # Realtime Voice Loop | |
| The realtime path runs the Gradio app as an ASGI app: | |
| - `/app` serves the UI. | |
| - `/` redirects to `/app`. | |
| - `/realtime/voice` is the browser microphone WebSocket. | |
| The browser sends 16 kHz mono PCM16 chunks, uses local VAD to commit a speech | |
| turn after silence, and plays returned audio chunks through an ordered queue. | |
| ## Local Fixture Smoke | |
| This does not require Modal or external model credentials. It starts the actual | |
| ASGI app on a temporary localhost port, opens a real WebSocket, sends synthetic | |
| PCM, and verifies transcript, text, and audio events. | |
| ```powershell | |
| .\.runtime-venv\Scripts\python.exe scripts\realtime_ws_smoke.py | |
| ``` | |
| ## Start The App | |
| ```powershell | |
| $env:TIME_MACHINE_ADAPTER_PROFILE="modal" | |
| $env:GRADIO_SERVER_NAME="127.0.0.1" | |
| $env:GRADIO_SERVER_PORT="7860" | |
| .\scripts\run_realtime_app.ps1 | |
| ``` | |
| Open `http://127.0.0.1:7860/app`, launch an encounter, then use the `Live voice` | |
| control in the cockpit. | |
| ## Start Modal Audio | |
| The original command still works if `.venv` is healthy: | |
| ```powershell | |
| $env:MODAL_PROFILE="manikandanj" | |
| $env:PYTHONUTF8="1" | |
| $env:PYTHONIOENCODING="utf-8" | |
| $env:TTY_COMPATIBLE="0" | |
| .\.venv\Scripts\modal.exe serve scripts\modal_audio.py | |
| ``` | |
| In this workspace `.venv\Scripts\modal.exe` points at a missing Python install. | |
| Use this wrapper to prefer the working `.runtime-venv` Modal CLI: | |
| ```powershell | |
| .\scripts\run_modal_audio_runtime.ps1 | |
| ``` | |
| When `modal serve` prints endpoint URLs, set: | |
| ```powershell | |
| $env:TIME_MACHINE_MODAL_STT_URL="https://..." | |
| $env:TIME_MACHINE_MODAL_TTS_URL="https://..." | |
| $env:TIME_MACHINE_LLM_API_KEY="..." | |
| # or $env:TOGETHER_API_KEY="..." | |
| ``` | |
| Then start the app with `TIME_MACHINE_ADAPTER_PROFILE=modal`. | |
| ## Current Cascade | |
| ```text | |
| browser mic/VAD | |
| -> /realtime/voice WebSocket | |
| -> STT turn commit | |
| -> streaming LLM text | |
| -> sentence-level TTS work queue | |
| -> ordered browser audio playback | |
| ``` | |
| The current Modal STT endpoint still transcribes per committed speech turn. The | |
| browser no longer requires clicking record for each turn, and LLM/TTS output is | |
| streamed through the app as soon as usable segments are available. | |