Upload full project to Space

IMPLEMENTATION_NOTES.md

@@ -0,0 +1,135 @@
+# Implementation Notes
+
+## Architecture Overview
+
+The STT system is built in 5 progressive steps, each adding functionality on top of the previous:
+
+1. **Step 1**: Basic offline transcription (Whisper/Vosk)
+2. **Step 2**: HTTP API for file uploads
+3. **Step 3**: WebSocket streaming for real-time audio
+4. **Step 4**: Telephony audio format support (Twilio/Exotel)
+5. **Step 5**: Production-ready with stability features
+
+## Key Components
+
+### Audio Processing
+
+- **TelephonyAudioConverter**: Handles format conversion
+  - Twilio: 8kHz μ-law → 16kHz PCM
+  - Exotel: 8kHz PCM → 16kHz PCM
+  - Uses scipy.signal.resample for sample rate conversion
+
+### Voice Activity Detection (VAD)
+
+- Simple energy-based VAD in Step 5
+- Threshold: 0.01 (configurable)
+- Frame-based analysis (25ms frames)
+- Detects speech vs silence
+
+### Audio Buffering
+
+- **AudioBuffer**: Accumulates audio chunks
+- Configurable chunk duration (default: 1.0s)
+- Minimum interval between transcriptions (0.5s)
+- Handles silence timeouts (3.0s)
+
+### Duplicate Prevention
+
+- Compares new transcriptions with previous
+- Prevents sending identical text multiple times
+- Simple substring matching (can be enhanced)
+
+## Things to Consider
+
+### Performance
+
+1. **Model Loading**: Whisper models are loaded per connection (lazy loading)
+   - Consider model caching/pooling for production
+   - Larger models (medium/large) are more accurate but slower
+
+2. **Chunk Size**: Balance between latency and accuracy
+   - Smaller chunks = lower latency but less context
+   - Larger chunks = better accuracy but higher latency
+
+3. **Concurrent Connections**: Each connection loads its own model
+   - Consider shared model instances for multiple connections
+   - Monitor memory usage with many concurrent calls
+
+### Audio Quality
+
+1. **Sample Rate**: Whisper works best with 16kHz
+   - Telephony audio (8kHz) must be upsampled
+   - Quality may be reduced compared to native 16kHz
+
+2. **Noise**: Telephony audio often has background noise
+   - Consider noise reduction preprocessing
+   - VAD helps filter silence but not noise
+
+3. **Format Conversion**: μ-law to PCM conversion may introduce artifacts
+   - Test with real telephony audio
+   - Consider alternative conversion methods if quality is poor
+
+### Stability & Reliability
+
+1. **Disconnections**: Handled gracefully in Step 5
+   - Final transcription on remaining buffer
+   - Session cleanup on disconnect
+
+2. **Error Handling**: Comprehensive error catching
+   - Logs errors per call
+   - Continues processing on individual failures
+
+3. **Logging**: Per-call logging in Step 5
+   - Logs stored in `logs/stt.log`
+   - Includes call_id for tracking
+
+### Scaling Considerations
+
+1. **Model Memory**: Whisper models are large (base ~150MB, large ~3GB)
+   - Consider GPU acceleration for faster inference
+   - Model quantization for reduced memory
+
+2. **API Rate Limiting**: No rate limiting implemented
+   - Add rate limiting for production
+   - Consider request queuing
+
+3. **Database**: No persistent storage
+   - Add database for call transcripts
+   - Store session metadata
+
+4. **Load Balancing**: Single server implementation
+   - Consider multiple workers/instances
+   - Use message queue for audio processing
+
+### Security
+
+1. **Authentication**: No authentication implemented
+   - Add API keys/tokens
+   - WebSocket authentication
+
+2. **Input Validation**: Basic validation
+   - Validate audio format/size
+   - Rate limit per client
+
+3. **Data Privacy**: Transcripts logged
+   - Consider encryption for sensitive data
+   - Implement data retention policies
+
+## Testing Recommendations
+
+1. **Unit Tests**: Test audio conversion functions
+2. **Integration Tests**: Test WebSocket streaming with real audio
+3. **Load Tests**: Test with multiple concurrent connections
+4. **Telephony Tests**: Test with actual Twilio/Exotel audio streams
+
+## Future Enhancements
+
+1. **Better VAD**: Use more sophisticated VAD (e.g., WebRTC VAD)
+2. **Streaming Model**: Use streaming-capable models for lower latency
+3. **Language Detection**: Auto-detect language
+4. **Speaker Diarization**: Identify different speakers
+5. **Punctuation**: Better punctuation in transcripts
+6. **Timestamping**: Word-level timestamps
+7. **Confidence Scores**: Return confidence scores per word
+
+

README.md

@@ -1,13 +1,277 @@
----
-title: NeuralvoiceGPU
-emoji: 🔥
-colorFrom: gray
-colorTo: green
-sdk: gradio
-sdk_version: 6.3.0
-python_version: '3.12'
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# NeuralVoice AI
+
+A real-time voice AI assistant system that combines Speech-to-Text (STT), Large Language Model (LLM), and Text-to-Speech (TTS) capabilities for natural phone conversations. Built with FastAPI, Vosk, OpenAI, and Piper TTS, integrated with Twilio for telephony.
+
+## 🎯 Overview
+
+NeuralVoice AI enables real-time bidirectional voice conversations over phone calls. The system:
+- **Listens** to caller speech using Vosk STT
+- **Understands** and responds using OpenAI's GPT models
+- **Speaks** back using Piper TTS with phone-optimized audio processing
+- **Handles** natural conversation flow with barge-in support and voice activity detection
+
+## ✨ Features
+
+- **Real-time Speech Recognition**: Vosk-based STT with voice activity detection (VAD)
+- **Intelligent Responses**: OpenAI GPT integration for contextual conversations
+- **Natural Voice Synthesis**: Piper TTS with phone-optimized audio filters
+- **Barge-in Support**: Callers can interrupt the AI mid-sentence
+- **WebSocket Streaming**: Low-latency bidirectional audio streaming via Twilio Media Streams
+- **Web Dashboard**: React-based frontend for monitoring live call transcripts
+- **Production Ready**: Includes error handling, keepalive, and session management
+
+## 🏗️ Architecture
+
+```
+┌─────────────┐
+│   Twilio    │ ← Phone calls
+└──────┬──────┘
+       │ WebSocket (8kHz μ-law)
+       ▼
+┌─────────────────────────────────┐
+│   FastAPI Backend (Python)     │
+│  ┌──────────────────────────┐  │
+│  │  STT (Vosk)              │  │
+│  │  ↓                       │  │
+│  │  LLM (OpenAI GPT)        │  │
+│  │  ↓                       │  │
+│  │  TTS (Piper + ffmpeg)    │  │
+│  └──────────────────────────┘  │
+└──────┬──────────────────────────┘
+       │
+       ├─→ WebSocket → React Frontend (Live Transcripts)
+       └─→ WebSocket → Twilio (Audio Playback)
+```
+
+## 📁 Project Structure
+
+```
+nv2/
+├── stt_llm_ttsopenai.py      # Main production server (STT+LLM+TTS pipeline)
+├── pipe_method3.py            # Alternative implementation with improved VAD
+├── download_models.py         # Script to download Vosk and Piper models
+├── requirements.txt           # Python dependencies
+├── Dockerfile                 # Docker configuration
+├── start.sh                   # Startup script
+├── IMPLEMENTATION_NOTES.md    # Technical implementation details
+├── README.md                  # This file
+└── web_demo/                  # React frontend
+    ├── src/
+    │   ├── App.jsx            # Main React app
+    │   ├── components/
+    │   │   ├── MicrophoneTest.jsx    # STT testing component
+    │   │   ├── TextToSpeech.jsx      # TTS testing component
+    │   │   └── SttLlmTts.jsx         # Full pipeline testing
+    │   └── ...
+    ├── package.json
+    └── vite.config.js
+```
+
+## 🚀 Quick Start
+
+### Prerequisites
+
+- Python 3.8+
+- Node.js 16+ (for frontend)
+- ffmpeg (for audio processing)
+- Piper TTS binary (or install via package manager)
+- OpenAI API key
+- Twilio account (for phone integration)
+
+### Installation
+
+1. **Clone the repository**
+   ```bash
+   git clone https://github.com/NuralVoice-AI-Model/NeuralVoiceAI.git
+   cd NeuralVoiceAI
+   ```
+
+2. **Install Python dependencies**
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. **Download AI models**
+   ```bash
+   python download_models.py
+   ```
+   This will download:
+   - Vosk STT model (English, ~1.8GB)
+   - Piper TTS model (English, ~50MB)
+
+4. **Install frontend dependencies**
+   ```bash
+   cd web_demo
+   npm install
+   cd ..
+   ```
+
+5. **Set environment variables**
+   ```bash
+   export OPENAI_API_KEY="your-openai-api-key"
+   export OPENAI_MODEL="gpt-4o-mini"  # or gpt-4, gpt-3.5-turbo
+   export PIPER_BIN="piper"  # or full path to piper binary
+   export PIPER_MODEL_PATH="models/piper/en_US-lessac-medium.onnx"
+   export VOSK_MODEL_PATH="models/vosk-model-en-us-0.22-lgraph"
+   export TWILIO_STREAM_URL="wss://your-domain.com/stream"  # For Twilio integration
+   export PORT=8080
+   ```
+
+### Running the Application
+
+1. **Start the backend server**
+   ```bash
+   python stt_llm_ttsopenai.py
+   # or
+   python pipe_method3.py
+   ```
+   Server will start on `http://0.0.0.0:8080`
+
+2. **Start the frontend (optional)**
+   ```bash
+   cd web_demo
+   npm run dev
+   ```
+   Frontend will be available at `http://localhost:5173`
+
+3. **Configure Twilio (for phone calls)**
+   - Set your Twilio voice webhook to: `https://your-domain.com/voice`
+   - Ensure `TWILIO_STREAM_URL` points to your WebSocket endpoint
+   - Use ngrok or similar for local development:
+     ```bash
+     ngrok http 8080
+     # Set TWILIO_STREAM_URL to wss://your-ngrok-url.ngrok.io/stream
+     ```
+
+## 🔧 Configuration
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `OPENAI_API_KEY` | OpenAI API key (required) | - |
+| `OPENAI_MODEL` | OpenAI model to use | `gpt-4o-mini` |
+| `VOSK_MODEL_PATH` | Path to Vosk STT model | `models/vosk-model-en-us-0.22-lgraph` |
+| `PIPER_BIN` | Path to Piper TTS binary | `piper` |
+| `PIPER_MODEL_PATH` | Path to Piper TTS model | - |
+| `TWILIO_STREAM_URL` | WebSocket URL for Twilio streams | - |
+| `HOST` | Server host | `0.0.0.0` |
+| `PORT` | Server port | `8080` |
+
+### Tuning Parameters
+
+In `stt_llm_ttsopenai.py` or `pipe_method3.py`, you can adjust:
+
+- **STT Latency**: `SILENCE_MS`, `STABLE_PARTIAL_MS`
+- **VAD Sensitivity**: `RMS_SPEECH_THRESHOLD`, `SPEECH_START_FRAMES`
+- **LLM Response**: `SYSTEM_PROMPT`, `max_tokens`, `temperature`
+- **TTS Chunking**: `CHUNK_MAX_CHARS`, `CHUNK_END_RE`
+
+## 📡 API Endpoints
+
+### HTTP Endpoints
+
+- `GET /health` - Health check
+- `POST /voice` - Twilio webhook (returns TwiML)
+- `GET /voice` - Twilio webhook (GET method)
+
+### WebSocket Endpoints
+
+- `WS /stream` - Main audio streaming endpoint for Twilio
+- `WS /client-ws` - Frontend client WebSocket for live transcripts
+
+## 🎤 Usage
+
+### Making a Phone Call
+
+1. Configure Twilio to call your `/voice` endpoint
+2. The system will:
+   - Answer the call
+   - Stream audio bidirectionally
+   - Transcribe speech in real-time
+   - Generate AI responses
+   - Speak responses back to caller
+
+### Testing Components
+
+The web dashboard (`web_demo`) provides three testing interfaces:
+
+1. **Microphone Test**: Test STT with your microphone
+2. **Text-to-Speech**: Test TTS with custom text
+3. **STT-LLM-TTS**: Test the full pipeline
+
+### Example Conversation Flow
+
+```
+Caller: "Hello, I need help with my account"
+  ↓ [STT: Vosk transcribes]
+  ↓ [LLM: OpenAI generates response]
+  ↓ [TTS: Piper synthesizes audio]
+AI: "I'd be happy to help. What's your account number?"
+  ↓ [Caller can interrupt/barge-in at any time]
+Caller: "It's 12345"
+  ↓ [Process repeats...]
+```
+
+## 🐳 Docker Deployment
+
+```bash
+docker build -t neuralvoice-ai .
+docker run -p 8080:8080 \
+  -e OPENAI_API_KEY="your-key" \
+  -e PIPER_MODEL_PATH="/app/models/piper/en_US-lessac-medium.onnx" \
+  -v $(pwd)/models:/app/models \
+  neuralvoice-ai
+```
+
+## 🔍 Monitoring
+
+- **Logs**: Check console output for real-time STT, LLM, and TTS logs
+- **Web Dashboard**: View live call transcripts in the React frontend
+- **Health Endpoint**: `GET /health` for service status
+
+## 🛠️ Development
+
+### Key Components
+
+1. **STT Engine** (`vosk`): Offline speech recognition
+2. **LLM Integration** (`openai`): GPT-based conversation
+3. **TTS Engine** (`piper`): Neural text-to-speech
+4. **Audio Processing** (`audioop`, `ffmpeg`): Format conversion and filtering
+5. **WebSocket Handler**: Real-time bidirectional streaming
+
+### Code Flow
+
+1. Twilio sends 8kHz μ-law audio chunks via WebSocket
+2. Audio is converted to 16kHz PCM for Vosk
+3. Vosk performs real-time transcription
+4. VAD detects speech endpoints
+5. User utterances trigger OpenAI API calls
+6. LLM responses are chunked and sent to Piper TTS
+7. TTS audio is processed with phone-optimized filters
+8. Audio is converted back to 8kHz μ-law and streamed to Twilio
+
+## 📝 Notes
+
+- **Latency**: Typical end-to-end latency is 1-3 seconds
+- **Barge-in**: Callers can interrupt the AI by speaking (detected via VAD)
+- **Audio Quality**: Phone-optimized filters (highpass/lowpass/compand) improve clarity
+- **Model Size**: Vosk model is ~1.8GB, ensure sufficient disk space
+- **Memory**: Each call loads the Vosk model (cached after first load)
+
+## 🤝 Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+## 📄 License
+
+Copyright © 2026 Blink Digital India Pvt Ltd. All rights reserved.
+
+All code in this repository is the property of Blink Digital India Pvt Ltd. Unauthorized copying, modification, distribution, or use of this software, via any medium, is strictly prohibited without express written permission from Blink Digital India Pvt Ltd.
+
+## 🙏 Acknowledgments
+
+- [Vosk](https://alphacephei.com/vosk/) - Speech recognition
+- [OpenAI](https://openai.com/) - Language models
+- [Piper TTS](https://github.com/rhasspy/piper) - Text-to-speech
+- [Twilio](https://www.twilio.com/) - Telephony platform

download_models.py

@@ -0,0 +1,40 @@
+import os
+import urllib.request
+import zipfile
+import tarfile
+
+def download_file(url, dest):
+    if os.path.exists(dest):
+        print(f"File already exists: {dest}")
+        return
+    print(f"Downloading {url} to {dest}...")
+    urllib.request.urlretrieve(url, dest)
+    print("Download complete.")
+
+def setup_models():
+    # Vosk Model
+    vosk_dir = "models/vosk-model-en-us-0.22-lgraph"
+    if not os.path.exists(vosk_dir):
+        os.makedirs("models", exist_ok=True)
+        zip_path = "models/vosk-model.zip"
+        download_file("https://alphacephei.com/vosk/models/vosk-model-en-us-0.22-lgraph.zip", zip_path)
+        print("Extracting Vosk model...")
+        with zipfile.ZipFile(zip_path, 'r') as zip_ref:
+            zip_ref.extractall("models")
+        os.remove(zip_path)
+        print("Vosk model setup complete.")
+    else:
+        print("Vosk model already exists.")
+
+    # Piper Model
+    piper_model_dir = "models/piper"
+    os.makedirs(piper_model_dir, exist_ok=True)
+    
+    piper_onnx = os.path.join(piper_model_dir, "en_US-lessac-medium.onnx")
+    piper_json = os.path.join(piper_model_dir, "en_US-lessac-medium.onnx.json")
+    
+    download_file("https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx", piper_onnx)
+    download_file("https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json", piper_json)
+
+if __name__ == "__main__":
+    setup_models()

pipe_method3.py

@@ -0,0 +1,572 @@
+"""
+Twilio Media Streams (bidirectional) + Vosk + OpenAI Answer + Piper -> Twilio playback
+
+What this version does:
+- NO intent / NO clarify JSON
+- Logs only:
+  STT_FINAL> ...
+  LLM_ANS> ...
+  TTS> ...
+- Generation-id safe TTS (no self-cancel on Railway)
+- Better phone clarity using ffmpeg filters (highpass/lowpass/compand)
+- Proper 20ms pacing + keepalive marks to prevent WS idle timeouts
+"""
+
+import asyncio
+import base64
+import json
+import logging
+import os
+import re
+import tempfile
+import time
+import audioop
+import subprocess
+import threading
+from dataclasses import dataclass, field
+from typing import Optional, List, Dict
+
+from fastapi import FastAPI, WebSocket, WebSocketDisconnect, Request
+from fastapi.responses import PlainTextResponse, Response
+from fastapi.middleware.cors import CORSMiddleware
+from vosk import Model, KaldiRecognizer
+from openai import OpenAI
+
+# ----------------------------
+# Logging
+# ----------------------------
+logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
+log = logging.getLogger("app")
+
+def P(tag: str, msg: str):
+    print(f"{tag} {msg}", flush=True)
+
+# ----------------------------
+# Env
+# ----------------------------
+VOSK_MODEL_PATH = os.getenv("VOSK_MODEL_PATH", "/app/models/vosk-model-en-us-0.22-lgraph").strip()
+TWILIO_STREAM_URL = os.getenv("TWILIO_STREAM_URL", "").strip()
+
+OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "").strip()
+OPENAI_MODEL = os.getenv("OPENAI_MODEL", "gpt-4o-mini").strip()
+
+PIPER_BIN = os.getenv("PIPER_BIN", "piper").strip()
+PIPER_MODEL_PATH = os.getenv("PIPER_MODEL_PATH", "").strip()
+
+HOST = "0.0.0.0"
+PORT = int(os.getenv("PORT", "8080"))
+
+# ----------------------------
+# FastAPI
+# ----------------------------
+app = FastAPI()
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
+
+# ----------------------------
+# Audio / Twilio
+# ----------------------------
+FRAME_MS = 20
+INPUT_RATE = 8000
+STT_RATE = 16000
+BYTES_PER_20MS_MULAW = int(INPUT_RATE * (FRAME_MS / 1000.0))  # 160 bytes @ 8kHz, 20ms
+
+# ----------------------------
+# VAD settings
+# ----------------------------
+RMS_SPEECH_THRESHOLD = 450
+SPEECH_START_FRAMES = 3
+SPEECH_END_SILENCE_FRAMES = 40  # 800ms
+MAX_UTTERANCE_MS = 12000
+PARTIAL_EMIT_EVERY_MS = 250
+
+# ----------------------------
+# LLM prompt
+# ----------------------------
+SYSTEM_PROMPT = (
+    "You are a phone-call assistant. "
+    "Reply in 1 short sentence (max 15 words). "
+    "No filler. No greetings unless user greets first."
+)
+
+# ----------------------------
+# Cached Vosk model
+# ----------------------------
+_VOSK_MODEL = None
+
+def now_ms() -> int:
+    return int(time.time() * 1000)
+
+def build_twiml(stream_url: str) -> str:
+    return f"""<?xml version="1.0" encoding="UTF-8"?>
+<Response>
+  <Connect>
+    <Stream url="{stream_url}" />
+  </Connect>
+  <Pause length="600"/>
+</Response>
+"""
+
+def split_mulaw_frames(mulaw_bytes: bytes) -> List[bytes]:
+    frames = []
+    for i in range(0, len(mulaw_bytes), BYTES_PER_20MS_MULAW):
+        chunk = mulaw_bytes[i:i + BYTES_PER_20MS_MULAW]
+        if len(chunk) < BYTES_PER_20MS_MULAW:
+            chunk += b"\xFF" * (BYTES_PER_20MS_MULAW - len(chunk))
+        frames.append(chunk)
+    return frames
+
+async def drain_queue(q: asyncio.Queue):
+    try:
+        while True:
+            q.get_nowait()
+            q.task_done()
+    except asyncio.QueueEmpty:
+        return
+
+# ----------------------------
+# OpenAI
+# ----------------------------
+def openai_client() -> OpenAI:
+    if not OPENAI_API_KEY:
+        raise RuntimeError("OPENAI_API_KEY not set")
+    return OpenAI(api_key=OPENAI_API_KEY)
+
+def openai_answer_blocking(history: List[Dict], user_text: str) -> str:
+    client = openai_client()
+    msgs = [{"role": "system", "content": SYSTEM_PROMPT}]
+    # short tail context
+    tail = history[-6:] if len(history) > 1 else []
+    msgs.extend(tail)
+    msgs.append({"role": "user", "content": user_text})
+
+    resp = client.chat.completions.create(
+        model=OPENAI_MODEL,
+        messages=msgs,
+        temperature=0.3,
+        max_tokens=80,
+    )
+    ans = (resp.choices[0].message.content or "").strip()
+    return ans
+
+# ----------------------------
+# Piper TTS -> 8k mulaw (clarity improved)
+# ----------------------------
+def piper_tts_to_mulaw(text: str) -> bytes:
+    if not PIPER_MODEL_PATH:
+        raise RuntimeError("PIPER_MODEL_PATH not set")
+
+    text = (text or "").strip()
+    if not text:
+        return b""
+
+    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as wavf:
+        wav_path = wavf.name
+    with tempfile.NamedTemporaryFile(suffix=".mulaw", delete=False) as mlf:
+        mulaw_path = mlf.name
+
+    try:
+        r1 = subprocess.run(
+            [PIPER_BIN, "--model", PIPER_MODEL_PATH, "--output_file", wav_path],
+            input=text.encode("utf-8"),
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+        )
+        if r1.returncode != 0:
+            raise RuntimeError(f"piper rc={r1.returncode} stderr={r1.stderr.decode('utf-8','ignore')[:500]}")
+
+        # Phone-clarity filter chain:
+        # - highpass removes rumble
+        # - lowpass removes harshness
+        # - compand evens volume (helps “clarity” on phone)
+        # - dynaudnorm is avoided (can pump / distort at 8k)
+        af = "highpass=f=200,lowpass=f=3400,compand=attacks=0:decays=0.3:points=-80/-80|-20/-10|0/-3"
+
+        r2 = subprocess.run(
+            ["ffmpeg", "-y", "-i", wav_path,
+             "-ac", "1", "-ar", "8000",
+             "-af", af,
+             "-f", "mulaw", mulaw_path],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+        )
+        if r2.returncode != 0:
+            raise RuntimeError(f"ffmpeg rc={r2.returncode} stderr={r2.stderr.decode('utf-8','ignore')[:500]}")
+
+        with open(mulaw_path, "rb") as f:
+            data = f.read()
+
+        P("TTS>", f"audio_bytes={len(data)}")
+        return data
+    finally:
+        for p in (wav_path, mulaw_path):
+            try:
+                os.unlink(p)
+            except Exception:
+                pass
+
+# ----------------------------
+# Call state
+# ----------------------------
+@dataclass
+class CancelFlag:
+    is_set: bool = False
+    def set(self):
+        self.is_set = True
+
+@dataclass
+class CallState:
+    call_id: str
+    stream_sid: str = ""
+
+    # vad
+    in_speech: bool = False
+    speech_start_count: int = 0
+    silence_count: int = 0
+    utter_start_ms: int = 0
+
+    rec: Optional[KaldiRecognizer] = None
+
+    # partials
+    last_partial: str = ""
+    last_partial_emit_ms: int = 0
+
+    # outbound
+    outbound_q: asyncio.Queue = field(default_factory=lambda: asyncio.Queue(maxsize=50000))
+    outbound_task: Optional[asyncio.Task] = None
+    keepalive_task: Optional[asyncio.Task] = None
+    mark_i: int = 0
+
+    # speaking / generation
+    bot_speaking: bool = False
+    cancel_llm: CancelFlag = field(default_factory=CancelFlag)
+    tts_generation_id: int = 0
+
+    # conversation history
+    history: List[Dict] = field(default_factory=list)
+    bot_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
+
+    def bump_tts_generation(self) -> int:
+        self.tts_generation_id += 1
+        return self.tts_generation_id
+
+# ----------------------------
+# Keepalive marks (prevents WS ping timeout)
+# ----------------------------
+async def twilio_keepalive(ws: WebSocket, st: CallState):
+    try:
+        while True:
+            await asyncio.sleep(10)
+            if st.stream_sid:
+                st.mark_i += 1
+                name = f"ka_{st.mark_i}"
+                await ws.send_text(json.dumps({
+                    "event": "mark",
+                    "streamSid": st.stream_sid,
+                    "mark": {"name": name},
+                }))
+                P("TWILIO>", f"keepalive_mark={name}")
+    except asyncio.CancelledError:
+        return
+    except Exception as e:
+        P("SYS>", f"keepalive_error={e}")
+
+# ----------------------------
+# HTTP
+# ----------------------------
+@app.get("/health")
+async def health():
+    return {"ok": True}
+
+@app.post("/voice")
+async def voice(request: Request):
+    stream_url = TWILIO_STREAM_URL
+    if not stream_url:
+        host = request.headers.get("host")
+        if host:
+            stream_url = f"wss://{host}/stream"
+            P("SYS>", f"auto_stream_url={stream_url}")
+    if not stream_url:
+        return PlainTextResponse("TWILIO_STREAM_URL not set and host not found", status_code=500)
+    return Response(content=build_twiml(stream_url), media_type="application/xml")
+
+@app.get("/voice")
+async def voice_get(request: Request):
+    return await voice(request)
+
+# ----------------------------
+# WebSocket /stream
+# ----------------------------
+@app.websocket("/stream")
+async def stream(ws: WebSocket):
+    await ws.accept()
+    st = CallState(call_id=str(id(ws)))
+    st.history = [{"role": "system", "content": SYSTEM_PROMPT}]
+    P("SYS>", f"ws_open call_id={st.call_id}")
+
+    global _VOSK_MODEL
+    if _VOSK_MODEL is None:
+        P("SYS>", f"loading_vosk={VOSK_MODEL_PATH}")
+        _VOSK_MODEL = Model(VOSK_MODEL_PATH)
+        P("SYS>", "vosk_loaded")
+
+    st.rec = KaldiRecognizer(_VOSK_MODEL, STT_RATE)
+    st.rec.SetWords(False)
+
+    st.outbound_task = asyncio.create_task(outbound_sender(ws, st))
+
+    try:
+        while True:
+            raw = await ws.receive_text()
+            msg = json.loads(raw)
+            event = msg.get("event")
+
+            if event == "start":
+                st.stream_sid = msg["start"]["streamSid"]
+                P("TWILIO>", f"start streamSid={st.stream_sid}")
+
+                if st.keepalive_task is None:
+                    st.keepalive_task = asyncio.create_task(twilio_keepalive(ws, st))
+
+                # optional short greeting
+                asyncio.create_task(speak_text(ws, st, "Hi! How can I help?"))
+
+            elif event == "media":
+                mulaw = base64.b64decode(msg["media"]["payload"])
+                pcm16_8k = audioop.ulaw2lin(mulaw, 2)
+                pcm16_16k, _ = audioop.ratecv(pcm16_8k, 2, 1, INPUT_RATE, STT_RATE, None)
+
+                rms = audioop.rms(pcm16_16k, 2)
+                is_speech = rms >= RMS_SPEECH_THRESHOLD
+
+                # barge-in: cancel current bot audio if caller speaks
+                if st.bot_speaking and is_speech:
+                    await barge_in(ws, st)
+
+                await vad_and_stt(ws, st, pcm16_16k, is_speech)
+
+            elif event == "mark":
+                name = (msg.get("mark") or {}).get("name")
+                P("TWILIO>", f"mark_received={name}")
+
+            elif event == "stop":
+                P("TWILIO>", "stop")
+                break
+
+    except WebSocketDisconnect:
+        P("SYS>", "ws_disconnect")
+    except Exception as e:
+        P("SYS>", f"ws_error={e}")
+        log.exception("ws_error")
+    finally:
+        if st.keepalive_task:
+            st.keepalive_task.cancel()
+        if st.outbound_task:
+            st.outbound_task.cancel()
+        P("SYS>", "ws_closed")
+
+# ----------------------------
+# VAD + STT
+# ----------------------------
+async def vad_and_stt(ws: WebSocket, st: CallState, pcm16_16k: bytes, is_speech: bool):
+    t = now_ms()
+
+    if not st.in_speech:
+        if is_speech:
+            st.speech_start_count += 1
+            if st.speech_start_count >= SPEECH_START_FRAMES:
+                st.in_speech = True
+                st.silence_count = 0
+                st.utter_start_ms = t
+                st.speech_start_count = 0
+                st.last_partial = ""
+                st.last_partial_emit_ms = 0
+
+                st.rec = KaldiRecognizer(_VOSK_MODEL, STT_RATE)
+                st.rec.SetWords(False)
+        else:
+            st.speech_start_count = 0
+        return
+
+    st.rec.AcceptWaveform(pcm16_16k)
+
+    # partial logging only (you said UI later)
+    if t - st.last_partial_emit_ms >= PARTIAL_EMIT_EVERY_MS:
+        st.last_partial_emit_ms = t
+        try:
+            pj = json.loads(st.rec.PartialResult() or "{}")
+            partial = (pj.get("partial") or "").strip()
+        except Exception:
+            partial = ""
+        if partial and partial != st.last_partial:
+            st.last_partial = partial
+            P("STT_PART>", partial)
+
+    if (t - st.utter_start_ms) > MAX_UTTERANCE_MS:
+        await finalize_utterance(ws, st, "max_utterance")
+        return
+
+    if is_speech:
+        st.silence_count = 0
+        return
+
+    st.silence_count += 1
+    if st.silence_count >= SPEECH_END_SILENCE_FRAMES:
+        await finalize_utterance(ws, st, f"vad_silence_{SPEECH_END_SILENCE_FRAMES*FRAME_MS}ms")
+
+async def finalize_utterance(ws: WebSocket, st: CallState, reason: str):
+    if not st.in_speech:
+        return
+
+    st.in_speech = False
+    st.silence_count = 0
+    st.speech_start_count = 0
+    st.last_partial = ""
+
+    try:
+        j = json.loads(st.rec.FinalResult() or "{}")
+    except Exception:
+        j = {}
+
+    user_text = (j.get("text") or "").strip()
+    if not user_text:
+        return
+
+    P("STT_FINAL>", f"{user_text}  ({reason})")
+
+    async def bot_job():
+        async with st.bot_lock:
+            await answer_and_speak(ws, st, user_text)
+
+    asyncio.create_task(bot_job())
+
+# ----------------------------
+# LLM Answer -> Speak
+# ----------------------------
+async def answer_and_speak(ws: WebSocket, st: CallState, user_text: str):
+    st.cancel_llm = CancelFlag(False)
+
+    # store user
+    st.history.append({"role": "user", "content": user_text})
+    st.history = st.history[:1] + st.history[-8:]
+
+    loop = asyncio.get_running_loop()
+
+    def worker():
+        return openai_answer_blocking(st.history, user_text)
+
+    ans = await loop.run_in_executor(None, worker)
+    ans = (ans or "").strip()
+    if not ans:
+        ans = "Sorry, I didn’t catch that."
+
+    P("LLM_ANS>", ans)
+
+    # store assistant
+    st.history.append({"role": "assistant", "content": ans})
+    st.history = st.history[:1] + st.history[-8:]
+
+    await speak_text(ws, st, ans)
+
+# ----------------------------
+# Barge-in (clear + drain)
+# ----------------------------
+async def barge_in(ws: WebSocket, st: CallState):
+    st.cancel_llm.set()
+    st.bump_tts_generation()  # invalidate older audio
+
+    if st.stream_sid:
+        try:
+            await ws.send_text(json.dumps({"event": "clear", "streamSid": st.stream_sid}))
+            P("TWILIO>", "sent_clear")
+        except Exception:
+            pass
+
+    await drain_queue(st.outbound_q)
+    st.bot_speaking = False
+
+# ----------------------------
+# Speak / TTS with generation-id
+# ----------------------------
+async def speak_text(ws: WebSocket, st: CallState, text: str):
+    gen = st.bump_tts_generation()
+
+    # clear previous audio
+    if st.stream_sid:
+        try:
+            await ws.send_text(json.dumps({"event": "clear", "streamSid": st.stream_sid}))
+            P("TWILIO>", "sent_clear")
+        except Exception:
+            pass
+    await drain_queue(st.outbound_q)
+
+    await tts_enqueue(st, text, gen)
+
+async def tts_enqueue(st: CallState, text: str, gen: int):
+    my_gen = gen
+    st.bot_speaking = True
+    P("TTS>", f"text={text} gen={my_gen}")
+
+    loop = asyncio.get_running_loop()
+    try:
+        mulaw_bytes = await loop.run_in_executor(None, piper_tts_to_mulaw, text)
+    except Exception as e:
+        P("TTS_ERR>", str(e))
+        st.bot_speaking = False
+        return
+
+    if my_gen != st.tts_generation_id:
+        P("TTS>", f"discard_gen my_gen={my_gen} current_gen={st.tts_generation_id}")
+        return
+
+    for fr in split_mulaw_frames(mulaw_bytes):
+        if my_gen != st.tts_generation_id:
+            P("TTS>", f"discard_midstream my_gen={my_gen} current_gen={st.tts_generation_id}")
+            return
+        await st.outbound_q.put(base64.b64encode(fr).decode("ascii"))
+
+    await st.outbound_q.put("__END_CHUNK__")
+
+async def outbound_sender(ws: WebSocket, st: CallState):
+    try:
+        while True:
+            item = await st.outbound_q.get()
+
+            if item == "__END_CHUNK__":
+                await asyncio.sleep(0.02)
+                if st.outbound_q.empty():
+                    st.bot_speaking = False
+                st.outbound_q.task_done()
+                continue
+
+            if not st.stream_sid:
+                st.outbound_q.task_done()
+                continue
+
+            await ws.send_text(json.dumps({
+                "event": "media",
+                "streamSid": st.stream_sid,
+                "media": {"payload": item},
+            }))
+
+            st.outbound_q.task_done()
+            await asyncio.sleep(FRAME_MS / 1000.0)
+
+    except asyncio.CancelledError:
+        return
+    except Exception as e:
+        P("SYS>", f"outbound_sender_error={e}")
+        log.exception("outbound_sender_error")
+
+# ----------------------------
+# main
+# ----------------------------
+if __name__ == "__main__":
+    import uvicorn
+    P("SYS>", f"starting {HOST}:{PORT}")
+    uvicorn.run(app, host=HOST, port=PORT)

start.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+
+# Colors
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+echo -e "${GREEN}Starting setup for pipe_method3.py...${NC}"
+
+# 1. Activate Virtual Environment
+if [ -d ".venv" ]; then
+    source .venv/bin/activate
+else
+    echo -e "${YELLOW}Virtual environment not found. Creating one...${NC}"
+    python3 -m venv .venv
+    source .venv/bin/activate
+    pip install fastapi vosk openai uvicorn websockets
+fi
+
+# 2. Set Piper Paths
+export PIPER_BIN="$(pwd)/.venv/bin/piper"
+export PIPER_MODEL_PATH="$(pwd)/models/piper/en_US-lessac-medium.onnx"
+
+# 3. Check OpenAI API Key
+if [ -z "$OPENAI_API_KEY" ]; then
+    echo -e "${YELLOW}OPENAI_API_KEY is not set.${NC}"
+    read -p "Please enter your OpenAI API Key: " OPENAI_API_KEY
+    export OPENAI_API_KEY
+fi
+
+# 4. Setup Ngrok
+echo -e "${GREEN}Checking ngrok...${NC}"
+NGROK_URL=""
+
+# Check if ngrok is already running
+if pgrep -x "ngrok" > /dev/null; then
+    echo "ngrok is already running."
+else
+    echo "Starting ngrok..."
+    ngrok http 8002 > /dev/null &
+    sleep 3
+fi
+
+# Fetch ngrok URL
+NGROK_API_URL="http://127.0.0.1:4040/api/tunnels"
+if command -v curl > /dev/null; then
+    NGROK_URL=$(curl -s $NGROK_API_URL | grep -o '"public_url":"[^"]*' | grep -o '[^"]*$' | head -n 1)
+fi
+
+if [ -z "$NGROK_URL" ]; then
+    echo -e "${YELLOW}Could not automatically fetch ngrok URL.${NC}"
+    echo "Please ensure ngrok is running (ngrok http 8002) and set TWILIO_STREAM_URL manually if needed."
+else
+    # Convert http/https to wss
+    WSS_URL="${NGROK_URL/https/wss}"
+    WSS_URL="${WSS_URL/http/wss}"
+    WSS_URL="$WSS_URL/stream"
+    
+    export TWILIO_STREAM_URL="$WSS_URL"
+    echo -e "${GREEN}Twilio Stream URL set to: ${WSS_URL}${NC}"
+    echo -e "${YELLOW}IMPORTANT: Copy the URL above (or the https version for the webhook) to your Twilio Phone Number configuration.${NC}"
+    echo -e "Webhook URL: ${NGROK_URL}/voice"
+fi
+
+# 5. Run Script
+echo -e "${GREEN}Starting pipe_method3.py...${NC}"
+python3 pipe_method3.py

stt_llm_ttsopenai.py

@@ -0,0 +1,636 @@
+import asyncio
+import base64
+import json
+import logging
+import os
+import re
+import tempfile
+import time
+import audioop
+import subprocess
+from dataclasses import dataclass, field
+from typing import Optional, List, Dict
+
+from fastapi import FastAPI, WebSocket, WebSocketDisconnect, Request
+from fastapi.responses import PlainTextResponse, Response
+from vosk import Model, KaldiRecognizer
+
+from openai import OpenAI
+
+# ----------------------------
+# Logging
+# ----------------------------
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s %(levelname)s %(message)s",
+)
+log = logging.getLogger("stt-llm-tts")
+
+# ----------------------------
+# Env
+# ----------------------------
+VOSK_MODEL_PATH = os.getenv("VOSK_MODEL_PATH", "models/vosk-model-en-us-0.22-lgraph")
+TWILIO_STREAM_URL = os.getenv("TWILIO_STREAM_URL")  # must be wss://.../stream
+
+OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")
+OPENAI_MODEL = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
+
+PIPER_BIN = os.getenv("PIPER_BIN", "piper")
+PIPER_MODEL_PATH = os.getenv("PIPER_MODEL_PATH", "")
+
+HOST = os.getenv("HOST", "0.0.0.0")
+PORT = int(os.getenv("PORT", "8002"))
+
+# ----------------------------
+# Tunables (latency vs accuracy)
+# ----------------------------
+# Endpointing:
+SILENCE_MS = 700            # if no audio frames for this long -> commit utterance
+STABLE_PARTIAL_MS = 650     # if partial text hasn't changed for this long -> commit
+MIN_UTTER_WORDS = 2         # ignore single-word junk
+MAX_UTTER_CHARS = 220       # safety cap
+
+# LLM->TTS chunking:
+CHUNK_MAX_CHARS = 90        # emit TTS chunk if buffer grows
+CHUNK_END_RE = re.compile(r"[.!?\n]")
+
+# Twilio pacing:
+FRAME_MS = 20
+MULAW_RATE = 8000
+BYTES_PER_20MS_MULAW = int(MULAW_RATE * (FRAME_MS / 1000.0))  # 160 bytes at 8kHz mulaw
+
+# Filter common garbage utterances:
+SINGLE_WORD_IGNORE = {
+    "the", "a", "an", "yeah", "yes", "no", "okay", "ok", "hmm", "um", "uh"
+}
+
+SYSTEM_PROMPT = (
+    "You are a fast phone-call assistant. "
+    "Reply in 1-2 short sentences. "
+    "Ask only one question at a time. "
+    "Be concise."
+)
+
+app = FastAPI()
+
+# ----------------------------
+# Frontend Clients
+# ----------------------------
+connected_clients: List[WebSocket] = []
+
+
+async def broadcast_transcript(role: str, text: str):
+    """Broadcasts a transcript message to all connected frontend clients."""
+    if not connected_clients:
+        return
+    
+    message = {
+        "type": "transcript",
+        "role": role,
+        "text": text,
+        "timestamp": now_ms()
+    }
+    
+    disconnected = []
+    for client in connected_clients:
+        try:
+            await client.send_json(message)
+        except Exception:
+            disconnected.append(client)
+            
+    for client in disconnected:
+        if client in connected_clients:
+            connected_clients.remove(client)
+
+
+# ----------------------------
+# Helpers
+# ----------------------------
+def now_ms() -> int:
+    return int(time.time() * 1000)
+
+
+def safe_strip_key(key: str) -> str:
+    return (key or "").strip().replace("\r", "").replace("\n", "")
+
+
+def split_mulaw_frames(mulaw_bytes: bytes) -> List[bytes]:
+    frames = []
+    for i in range(0, len(mulaw_bytes), BYTES_PER_20MS_MULAW):
+        chunk = mulaw_bytes[i:i + BYTES_PER_20MS_MULAW]
+        if len(chunk) < BYTES_PER_20MS_MULAW:
+            # pad with silence (mu-law silence is 0xFF)
+            chunk += b"\xFF" * (BYTES_PER_20MS_MULAW - len(chunk))
+        frames.append(chunk)
+    return frames
+
+
+def is_junk_utterance(text: str) -> bool:
+    t = (text or "").strip().lower()
+    if not t:
+        return True
+    if len(t) > MAX_UTTER_CHARS:
+        return False
+    words = [w for w in t.split() if w]
+    if len(words) < MIN_UTTER_WORDS and (t in SINGLE_WORD_IGNORE):
+        return True
+    if len(words) < MIN_UTTER_WORDS and len(t) < 4:
+        return True
+    return False
+
+
+def build_twiml(stream_url: str) -> str:
+    return f"""<?xml version="1.0" encoding="UTF-8"?>
+<Response>
+  <Connect>
+    <Stream url="{stream_url}" />
+  </Connect>
+  <Pause length="600"/>
+</Response>
+"""
+
+
+async def drain_queue(q: asyncio.Queue):
+    try:
+        while True:
+            q.get_nowait()
+            q.task_done()
+    except asyncio.QueueEmpty:
+        return
+
+
+# ----------------------------
+# Piper TTS -> mulaw 8k
+# ----------------------------
+def piper_tts_to_mulaw(text: str) -> bytes:
+    """
+    Generates 8k mulaw raw bytes suitable for Twilio Media Streams.
+    Pipeline: piper -> wav (22k/16k depends) -> ffmpeg -> mulaw 8k raw
+    """
+    if not PIPER_MODEL_PATH:
+        raise RuntimeError("Set PIPER_MODEL_PATH to a valid .onnx voice model")
+    if not shutil_which(PIPER_BIN):
+        raise RuntimeError(f"piper binary not found: {PIPER_BIN} (set PIPER_BIN to full path)")
+
+    text = (text or "").strip()
+    if not text:
+        return b""
+
+    with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as wavf:
+        wav_path = wavf.name
+    with tempfile.NamedTemporaryFile(suffix=".mulaw", delete=False) as mlf:
+        mulaw_path = mlf.name
+
+    try:
+        # piper writes wav
+        # piper usage: piper --model <onnx> --output_file out.wav
+        # it reads text from stdin
+        p = subprocess.run(
+            [PIPER_BIN, "--model", PIPER_MODEL_PATH, "--output_file", wav_path],
+            input=text.encode("utf-8"),
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            check=True
+        )
+
+        # Convert wav -> raw mulaw 8k
+        subprocess.run(
+            [
+                "ffmpeg", "-y",
+                "-i", wav_path,
+                "-ac", "1",
+                "-ar", "8000",
+                "-f", "mulaw",
+                mulaw_path
+            ],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            check=True
+        )
+
+        with open(mulaw_path, "rb") as f:
+            return f.read()
+
+    finally:
+        try:
+            os.unlink(wav_path)
+        except Exception:
+            pass
+        try:
+            os.unlink(mulaw_path)
+        except Exception:
+            pass
+
+
+def shutil_which(cmd: str) -> Optional[str]:
+    # tiny "which" helper without importing shutil (keeps deps minimal)
+    if os.path.isabs(cmd) and os.path.exists(cmd):
+        return cmd
+    for p in os.getenv("PATH", "").split(os.pathsep):
+        full = os.path.join(p, cmd)
+        if os.path.exists(full) and os.access(full, os.X_OK):
+            return full
+    return None
+
+
+# ----------------------------
+# OpenAI streaming
+# ----------------------------
+def openai_stream_tokens_blocking(messages: List[Dict], model: str, cancel_flag: "CancelFlag"):
+    """
+    Blocking generator for streaming tokens.
+    We stop yielding if cancel_flag is set.
+    """
+    key = safe_strip_key(OPENAI_API_KEY)
+    if not key:
+        raise RuntimeError("OPENAI_API_KEY is empty. Set it in your environment.")
+    client = OpenAI(api_key=key)
+
+    stream = client.chat.completions.create(
+        model=model,
+        messages=messages,
+        temperature=0.4,
+        stream=True,
+        max_tokens=180,
+    )
+
+    for event in stream:
+        if cancel_flag.is_set:
+            break
+        delta = event.choices[0].delta
+        if delta and delta.content:
+            yield delta.content
+
+
+class CancelFlag:
+    def __init__(self):
+        self.is_set = False
+
+    def set(self):
+        self.is_set = True
+
+
+# ----------------------------
+# Call state
+# ----------------------------
+@dataclass
+class CallState:
+    call_id: str
+    stream_sid: str = ""
+    # audio
+    last_audio_ms: int = field(default_factory=now_ms)
+    # partial tracking
+    last_partial: str = ""
+    last_partial_change_ms: int = field(default_factory=now_ms)
+    # recognizer
+    rec: Optional[KaldiRecognizer] = None
+    # outbound audio
+    outbound_q: asyncio.Queue = field(default_factory=lambda: asyncio.Queue(maxsize=4000))
+    # barge-in / cancellation
+    bot_speaking: bool = False
+    cancel_llm: CancelFlag = field(default_factory=CancelFlag)
+    cancel_tts: CancelFlag = field(default_factory=CancelFlag)
+    # conversation
+    history: List[Dict] = field(default_factory=list)
+    # tasks
+    outbound_task: Optional[asyncio.Task] = None
+    # lock so only one bot response at a time
+    bot_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
+
+    def reset_cancels(self):
+        self.cancel_llm = CancelFlag()
+        self.cancel_tts = CancelFlag()
+
+
+# ----------------------------
+# FastAPI endpoints
+# ----------------------------
+@app.get("/health")
+async def health():
+    return {"ok": True}
+
+
+@app.post("/voice")
+async def voice(request: Request):
+    if not TWILIO_STREAM_URL:
+        return PlainTextResponse("TWILIO_STREAM_URL is not set", status_code=500)
+    xml = build_twiml(TWILIO_STREAM_URL)
+    log.info("Returning TwiML:\n%s", xml)
+    return Response(content=xml, media_type="application/xml")
+
+@app.websocket("/client-ws")
+async def client_websocket(ws: WebSocket):
+    await ws.accept()
+    connected_clients.append(ws)
+    log.info("Frontend client connected. Total clients: %d", len(connected_clients))
+    try:
+        while True:
+            # Keep connection alive
+            await ws.receive_text()
+    except WebSocketDisconnect:
+        if ws in connected_clients:
+            connected_clients.remove(ws)
+        log.info("Frontend client disconnected. Total clients: %d", len(connected_clients))
+    except Exception as e:
+        log.error("Frontend client error: %s", e)
+        if ws in connected_clients:
+            connected_clients.remove(ws)
+
+@app.websocket("/stream")
+async def stream(ws: WebSocket):
+    await ws.accept()
+
+    call_id = str(id(ws))
+    st = CallState(call_id=call_id)
+    st.history = [{"role": "system", "content": SYSTEM_PROMPT}]
+
+    log.info("[%s] connection open", call_id)
+
+    # Load Vosk model once per process? (simple: load per call is slow)
+    # We'll cache it globally to reduce call start latency.
+    global _VOSK_MODEL
+    if _VOSK_MODEL is None:
+        log.info("Loading Vosk model: %s", VOSK_MODEL_PATH)
+        _VOSK_MODEL = Model(VOSK_MODEL_PATH)
+        log.info("Vosk model loaded.")
+    st.rec = KaldiRecognizer(_VOSK_MODEL, 16000)
+    st.rec.SetWords(False)
+
+    # Outbound sender task (pacing)
+    st.outbound_task = asyncio.create_task(outbound_sender(ws, st))
+
+    try:
+        while True:
+            raw = await ws.receive_text()
+            msg = json.loads(raw)
+
+            event = msg.get("event")
+            if event == "start":
+                st.stream_sid = msg["start"]["streamSid"]
+                enc = msg["start"].get("mediaFormat", {}).get("encoding") or msg["start"].get("mediaFormat", {}).get("codec")
+                sr = msg["start"].get("mediaFormat", {}).get("sampleRate")
+                log.info("[%s] start streamSid=%s encoding=%s sr=%s", call_id, st.stream_sid, enc, sr)
+
+                # greet once
+                await speak_text(ws, st, "Hi! How can I help you today?")
+
+            elif event == "media":
+                st.last_audio_ms = now_ms()
+                payload_b64 = msg["media"]["payload"]
+                mulaw = base64.b64decode(payload_b64)
+
+                # decode mulaw 8k -> PCM 16k 16-bit
+                pcm16 = audioop.ulaw2lin(mulaw, 2)  # 16-bit
+                pcm16_16k, _ = audioop.ratecv(pcm16, 2, 1, 8000, 16000, None)
+
+                # feed recognizer
+                if st.rec.AcceptWaveform(pcm16_16k):
+                    j = json.loads(st.rec.Result() or "{}")
+                    text = (j.get("text") or "").strip()
+                    if text:
+                        await on_utterance(ws, st, text, reason="vosk_final")
+                else:
+                    j = json.loads(st.rec.PartialResult() or "{}")
+                    partial = (j.get("partial") or "").strip()
+                    if partial:
+                        await on_partial(ws, st, partial)
+
+                # endpointing checks (silence/stability)
+                await maybe_endpoint(ws, st)
+
+            elif event == "stop":
+                log.info("[%s] stop", call_id)
+                break
+
+    except WebSocketDisconnect:
+        log.info("[%s] websocket disconnected", call_id)
+    except Exception as e:
+        log.exception("[%s] websocket error: %s", call_id, e)
+    finally:
+        # cancel outbound task
+        if st.outbound_task:
+            st.outbound_task.cancel()
+        log.info("[%s] connection closed", call_id)
+
+
+# cache model
+_VOSK_MODEL = None
+
+
+# ----------------------------
+# Endpointing + Barge-in
+# ----------------------------
+async def on_partial(ws: WebSocket, st: CallState, partial: str):
+    # barge-in trigger: user starts speaking while bot speaking
+    # use partial length threshold
+    words = partial.split()
+    if st.bot_speaking and len(words) >= 2:
+        log.info("[%s] BARGE-IN detected (partial=%r)", st.call_id, partial)
+        await barge_in(ws, st)
+
+    if partial != st.last_partial:
+        st.last_partial = partial
+        st.last_partial_change_ms = now_ms()
+        log.info("[%s] partial: %s", st.call_id, partial)
+
+
+async def maybe_endpoint(ws: WebSocket, st: CallState):
+    # stable partial endpoint
+    if st.last_partial:
+        stable_ms = now_ms() - st.last_partial_change_ms
+        if stable_ms >= STABLE_PARTIAL_MS:
+            # commit partial as utterance
+            text = st.last_partial.strip()
+            st.last_partial = ""
+            if text and not is_junk_utterance(text):
+                await on_utterance(ws, st, text, reason=f"stable_partial_{stable_ms}ms")
+
+    # silence endpoint
+    silence_ms = now_ms() - st.last_audio_ms
+    if silence_ms >= SILENCE_MS and st.last_partial:
+        text = st.last_partial.strip()
+        st.last_partial = ""
+        if text and not is_junk_utterance(text):
+            await on_utterance(ws, st, text, reason=f"silence_{silence_ms}ms")
+
+
+async def on_utterance(ws: WebSocket, st: CallState, text: str, reason: str):
+    text = (text or "").strip()
+    if not text:
+        return
+    if is_junk_utterance(text):
+        log.info("[%s] ignore utterance=%r reason=%s", st.call_id, text, reason)
+        return
+
+    # print to terminal clearly
+    print("\n" + "=" * 70)
+    print(f"STT ({reason}): {text}")
+    print("LLM: ", end="", flush=True)
+
+    # broadcast to frontend
+    await broadcast_transcript("user", text)
+
+    # ensure only one bot response runs at a time
+    async with st.bot_lock:
+        st.reset_cancels()
+        await run_llm_stream_and_tts(ws, st, text)
+
+
+async def barge_in(ws: WebSocket, st: CallState):
+    # cancel ongoing LLM/TTS
+    st.cancel_llm.set()
+    st.cancel_tts.set()
+
+    # stop playback on Twilio side (important)
+    try:
+        await ws.send_text(json.dumps({"event": "clear", "streamSid": st.stream_sid}))
+    except Exception:
+        pass
+
+    # clear our outbound queue
+    await drain_queue(st.outbound_q)
+
+    st.bot_speaking = False
+
+
+# ----------------------------
+# LLM streaming -> chunk -> TTS -> queue -> paced playback
+# ----------------------------
+async def run_llm_stream_and_tts(ws: WebSocket, st: CallState, user_text: str):
+    # build short rolling history
+    st.history.append({"role": "user", "content": user_text})
+    st.history = st.history[:1] + st.history[-8:]  # keep system + last 8 msgs
+
+    loop = asyncio.get_running_loop()
+    token_q: asyncio.Queue = asyncio.Queue()
+
+    def worker():
+        try:
+            for tok in openai_stream_tokens_blocking(st.history, OPENAI_MODEL, st.cancel_llm):
+                if st.cancel_llm.is_set:
+                    break
+                asyncio.run_coroutine_threadsafe(token_q.put(tok), loop)
+        finally:
+            asyncio.run_coroutine_threadsafe(token_q.put(None), loop)
+
+    # start blocking OpenAI stream in a thread
+    await loop.run_in_executor(None, worker)
+
+    # read tokens and chunk them
+    buf = ""
+    full = ""
+
+    while True:
+        tok = await token_q.get()
+        if tok is None:
+            break
+        if st.cancel_llm.is_set:
+            break
+
+        full += tok
+        buf += tok
+
+        # print as it streams
+        print(tok, end="", flush=True)
+
+        # chunk rule: punctuation OR length
+        if CHUNK_END_RE.search(buf) or len(buf) >= CHUNK_MAX_CHARS:
+            chunk = buf.strip()
+            buf = ""
+            if chunk:
+                await tts_enqueue(ws, st, chunk)
+
+    # flush remaining
+    rem = buf.strip()
+    if rem and not st.cancel_llm.is_set:
+        await tts_enqueue(ws, st, rem)
+
+    # store assistant message for context (only if not cancelled)
+    if full.strip() and not st.cancel_llm.is_set:
+        st.history.append({"role": "assistant", "content": full.strip()})
+        # broadcast to frontend
+        await broadcast_transcript("assistant", full.strip())
+
+
+async def tts_enqueue(ws: WebSocket, st: CallState, text: str):
+    if st.cancel_tts.is_set:
+        return
+
+    st.bot_speaking = True
+    log.info("[%s] TTS start (chars=%d)", st.call_id, len(text))
+
+    # run piper+ffmpeg in executor (blocking)
+    loop = asyncio.get_running_loop()
+    mulaw_bytes = await loop.run_in_executor(None, piper_tts_to_mulaw, text)
+
+    if st.cancel_tts.is_set:
+        return
+
+    frames = split_mulaw_frames(mulaw_bytes)
+    log.info("[%s] TTS ready (frames=%d)", st.call_id, len(frames))
+
+    # enqueue frames
+    for fr in frames:
+        if st.cancel_tts.is_set:
+            break
+        b64 = base64.b64encode(fr).decode("ascii")
+        await st.outbound_q.put(b64)
+
+    # marker: end of this chunk
+    await st.outbound_q.put("__END_CHUNK__")
+
+
+async def speak_text(ws: WebSocket, st: CallState, text: str):
+    # used for initial greeting
+    await barge_in(ws, st)  # clear any previous
+    await tts_enqueue(ws, st, text)
+
+
+async def outbound_sender(ws: WebSocket, st: CallState):
+    """
+    Sends queued audio to Twilio at real-time pace (20ms per frame).
+    Also turns off bot_speaking after chunk ends and queue drains.
+    """
+    sent_last_sec = 0
+    sec_tick = time.time()
+
+    try:
+        while True:
+            item = await st.outbound_q.get()
+
+            if item == "__END_CHUNK__":
+                # if queue empty after a short moment -> bot not speaking
+                await asyncio.sleep(0.02)
+                if st.outbound_q.empty():
+                    st.bot_speaking = False
+                st.outbound_q.task_done()
+                continue
+
+            # Twilio media message
+            msg = {
+                "event": "media",
+                "streamSid": st.stream_sid,
+                "media": {"payload": item},
+            }
+            await ws.send_text(json.dumps(msg))
+            st.outbound_q.task_done()
+
+            # pacing
+            await asyncio.sleep(FRAME_MS / 1000.0)
+
+            # stats
+            sent_last_sec += 1
+            if time.time() - sec_tick >= 1.0:
+                log.info("[%s] outbound media messages sent last 1s: %d", st.call_id, sent_last_sec)
+                sent_last_sec = 0
+                sec_tick = time.time()
+
+    except asyncio.CancelledError:
+        return
+    except Exception as e:
+        log.exception("[%s] outbound sender error: %s", st.call_id, e)
+
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(app, host=HOST, port=PORT)

web_demo/.gitignore

@@ -0,0 +1,24 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
+dist
+dist-ssr
+*.local
+.env
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+.DS_Store
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?

web_demo/README.md

@@ -0,0 +1,16 @@
+# React + Vite
+
+This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
+
+Currently, two official plugins are available:
+
+- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh
+- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
+
+## React Compiler
+
+The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
+
+## Expanding the ESLint configuration
+
+If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and [`typescript-eslint`](https://typescript-eslint.io) in your project.

web_demo/envdatavars.txt

@@ -0,0 +1 @@
+ VITE_OPENAI_API_KEY=

web_demo/eslint.config.js

@@ -0,0 +1,29 @@
+import js from '@eslint/js'
+import globals from 'globals'
+import reactHooks from 'eslint-plugin-react-hooks'
+import reactRefresh from 'eslint-plugin-react-refresh'
+import { defineConfig, globalIgnores } from 'eslint/config'
+
+export default defineConfig([
+  globalIgnores(['dist']),
+  {
+    files: ['**/*.{js,jsx}'],
+    extends: [
+      js.configs.recommended,
+      reactHooks.configs.flat.recommended,
+      reactRefresh.configs.vite,
+    ],
+    languageOptions: {
+      ecmaVersion: 2020,
+      globals: globals.browser,
+      parserOptions: {
+        ecmaVersion: 'latest',
+        ecmaFeatures: { jsx: true },
+        sourceType: 'module',
+      },
+    },
+    rules: {
+      'no-unused-vars': ['error', { varsIgnorePattern: '^[A-Z_]' }],
+    },
+  },
+])

web_demo/index.html

@@ -0,0 +1,13 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="UTF-8" />
+    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>web_demo</title>
+  </head>
+  <body>
+    <div id="root"></div>
+    <script type="module" src="/src/main.jsx"></script>
+  </body>
+</html>

web_demo/package.json

@@ -0,0 +1,28 @@
+{
+  "name": "web_demo",
+  "private": true,
+  "version": "0.0.0",
+  "type": "module",
+  "scripts": {
+    "dev": "vite",
+    "build": "vite build",
+    "lint": "eslint .",
+    "preview": "vite preview"
+  },
+  "dependencies": {
+    "lucide-react": "^0.562.0",
+    "react": "^19.2.0",
+    "react-dom": "^19.2.0"
+  },
+  "devDependencies": {
+    "@eslint/js": "^9.39.1",
+    "@types/react": "^19.2.5",
+    "@types/react-dom": "^19.2.3",
+    "@vitejs/plugin-react": "^5.1.1",
+    "eslint": "^9.39.1",
+    "eslint-plugin-react-hooks": "^7.0.1",
+    "eslint-plugin-react-refresh": "^0.4.24",
+    "globals": "^16.5.0",
+    "vite": "^7.2.4"
+  }
+}

web_demo/src/App.css

@@ -0,0 +1,211 @@
+:root {
+  --bg-color: #0f172a;
+  --card-bg: #1e293b;
+  --text-primary: #f8fafc;
+  --text-secondary: #94a3b8;
+  --accent-color: #3b82f6;
+  --user-bubble: #334155;
+  --assistant-bubble: #2563eb;
+  --error-color: #ef4444;
+  --success-color: #22c55e;
+}
+
+* {
+  box-sizing: border-box;
+  margin: 0;
+  padding: 0;
+}
+
+body {
+  font-family: 'Inter', system-ui, -apple-system, sans-serif;
+  background-color: var(--bg-color);
+  color: var(--text-primary);
+  height: 100vh;
+  overflow: hidden;
+}
+
+.app-container {
+  display: flex;
+  flex-direction: column;
+  height: 100vh;
+  width: 100%;
+  max-width: 100vw;
+  margin: 0;
+  padding: 1.5rem;
+}
+
+.header {
+  display: flex;
+  justify-content: space-between;
+  align-items: center;
+  padding: 1rem;
+  background-color: var(--card-bg);
+  border-radius: 12px;
+  margin-bottom: 1rem;
+  box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1);
+}
+
+/* Tabs */
+.tabs {
+  display: flex;
+  gap: 0.5rem;
+  margin-bottom: 1rem;
+  background-color: var(--card-bg);
+  padding: 0.5rem;
+  border-radius: 12px;
+  box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1);
+}
+
+.tab {
+  flex: 1;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  gap: 0.5rem;
+  padding: 0.75rem 1rem;
+  background: transparent;
+  border: none;
+  border-radius: 8px;
+  color: var(--text-secondary);
+  font-size: 0.875rem;
+  font-weight: 500;
+  cursor: pointer;
+  transition: all 0.2s ease;
+  font-family: inherit;
+}
+
+.tab:hover {
+  background-color: rgba(255, 255, 255, 0.05);
+  color: var(--text-primary);
+}
+
+.tab.active {
+  background-color: var(--accent-color);
+  color: white;
+  box-shadow: 0 2px 8px rgba(59, 130, 246, 0.3);
+}
+
+.tab svg {
+  flex-shrink: 0;
+}
+
+.logo {
+  display: flex;
+  align-items: center;
+  gap: 0.75rem;
+}
+
+.icon-logo {
+  color: var(--accent-color);
+}
+
+h1 {
+  font-size: 1.25rem;
+  font-weight: 600;
+}
+
+.status-badge {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  padding: 0.5rem 1rem;
+  border-radius: 9999px;
+  font-size: 0.875rem;
+  background-color: rgba(255, 255, 255, 0.05);
+}
+
+.status-badge.connected {
+  color: var(--success-color);
+  background-color: rgba(34, 197, 94, 0.1);
+}
+
+.status-badge.disconnected {
+  color: var(--error-color);
+  background-color: rgba(239, 68, 68, 0.1);
+}
+
+.main-content {
+  flex: 1;
+  background-color: var(--card-bg);
+  border-radius: 12px;
+  padding: 1rem;
+  overflow-y: auto;
+  position: relative;
+  box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1);
+}
+
+.transcript-container {
+  display: flex;
+  flex-direction: column;
+  gap: 1rem;
+}
+
+.empty-state {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  justify-content: center;
+  height: 100%;
+  color: var(--text-secondary);
+  gap: 1rem;
+  margin-top: 4rem;
+}
+
+.transcript-item {
+  display: flex;
+  flex-direction: column;
+  gap: 0.25rem;
+  max-width: 80%;
+}
+
+.transcript-item.user {
+  align-self: flex-end;
+  align-items: flex-end;
+}
+
+.transcript-item.assistant {
+  align-self: flex-start;
+  align-items: flex-start;
+}
+
+.message-header {
+  display: flex;
+  gap: 0.5rem;
+  font-size: 0.75rem;
+  color: var(--text-secondary);
+}
+
+.message-bubble {
+  padding: 0.75rem 1rem;
+  border-radius: 12px;
+  line-height: 1.5;
+  box-shadow: 0 1px 2px rgba(0, 0, 0, 0.1);
+}
+
+.transcript-item.user .message-bubble {
+  background-color: var(--user-bubble);
+  border-bottom-right-radius: 2px;
+}
+
+.transcript-item.assistant .message-bubble {
+  background-color: var(--assistant-bubble);
+  border-bottom-left-radius: 2px;
+}
+
+/* Scrollbar styling */
+::-webkit-scrollbar {
+  width: 8px;
+}
+
+::-webkit-scrollbar-track {
+  background: transparent;
+}
+
+::-webkit-scrollbar-thumb {
+  background: #475569;
+  border-radius: 4px;
+}
+
+::-webkit-scrollbar-thumb:hover {
+  background: #64748b;
+}

web_demo/src/App.jsx

@@ -0,0 +1,162 @@
+import React, { useState, useEffect, useRef, useCallback } from 'react';
+import { Activity, Wifi, WifiOff, Terminal, MessageSquare, Mic, Volume2, Settings } from 'lucide-react';
+import './App.css';
+import MicrophoneTest from './components/MicrophoneTest.jsx';
+import TextToSpeech from './components/TextToSpeech.jsx';
+import SttLlmTts from './components/SttLlmTts.jsx';
+
+function App() {
+  const [activeTab, setActiveTab] = useState('transcript');
+  const [isConnected, setIsConnected] = useState(false);
+  const [transcripts, setTranscripts] = useState([]);
+  const [status, setStatus] = useState('Disconnected');
+  const wsRef = useRef(null);
+  const transcriptsEndRef = useRef(null);
+
+  // Scroll to bottom of transcripts
+  useEffect(() => {
+    transcriptsEndRef.current?.scrollIntoView({ behavior: 'smooth' });
+  }, [transcripts]);
+
+  // Connect to WebSocket
+  const connectWebSocket = useCallback(() => {
+    setStatus('Connecting...');
+    // Connect to the backend's frontend-client endpoint
+    const wsUrl = `ws://localhost:8080/client-ws`;
+    const ws = new WebSocket(wsUrl);
+
+    ws.onopen = () => {
+      setIsConnected(true);
+      setStatus('Connected (Waiting for call data)');
+    };
+
+    ws.onclose = () => {
+      setIsConnected(false);
+      setStatus('Disconnected');
+      // Try reconnecting after 3 seconds
+      setTimeout(connectWebSocket, 3000);
+    };
+
+    ws.onerror = (error) => {
+      console.error('WebSocket error:', error);
+      setStatus('Error connecting');
+    };
+
+    ws.onmessage = (event) => {
+      try {
+        const data = JSON.parse(event.data);
+        handleServerMessage(data);
+      } catch (e) {
+        console.error('Error parsing message:', e);
+      }
+    };
+
+    wsRef.current = ws;
+  }, []);
+
+  useEffect(() => {
+    connectWebSocket();
+    return () => {
+      if (wsRef.current) {
+        wsRef.current.close();
+      }
+    };
+  }, [connectWebSocket]);
+
+  const handleServerMessage = (data) => {
+    if (data.type === 'transcript') {
+      setTranscripts(prev => [...prev, {
+        id: Date.now(),
+        role: data.role,
+        text: data.text,
+        timestamp: new Date(data.timestamp).toLocaleTimeString()
+      }]);
+    }
+  };
+
+  return (
+    <div className="app-container">
+      <header className="header">
+        <div className="logo">
+          <Activity className="icon-logo" />
+          <h1>NeuralVoice AI</h1>
+        </div>
+        <div className={`status-badge ${isConnected ? 'connected' : 'disconnected'}`}>
+          {isConnected ? <Wifi size={16} /> : <WifiOff size={16} />}
+          <span>{status}</span>
+        </div>
+      </header>
+
+      <div className="tabs">
+        <button
+          className={`tab ${activeTab === 'transcript' ? 'active' : ''}`}
+          onClick={() => setActiveTab('transcript')}
+        >
+          <MessageSquare size={18} />
+          <span>Live Call Transcript</span>
+        </button>
+        <button
+          className={`tab ${activeTab === 'microphone' ? 'active' : ''}`}
+          onClick={() => setActiveTab('microphone')}
+        >
+          <Mic size={18} />
+          <span>Microphone Test (STT)</span>
+        </button>
+        <button
+          className={`tab ${activeTab === 'tts' ? 'active' : ''}`}
+          onClick={() => setActiveTab('tts')}
+        >
+          <Volume2 size={18} />
+          <span>Text-to-Speech (TTS)</span>
+        </button>
+        <button
+          className={`tab ${activeTab === 'stt-llm-tts' ? 'active' : ''}`}
+          onClick={() => setActiveTab('stt-llm-tts')}
+        >
+          <Settings size={18} />
+          <span>STT-LLM-TTS</span>
+        </button>
+      </div>
+
+      <main className="main-content">
+        {activeTab === 'transcript' && (
+          <div className="transcript-container">
+            {transcripts.length === 0 && (
+              <div className="empty-state">
+                <Terminal size={48} />
+                <p>Waiting for call activity...</p>
+              </div>
+            )}
+
+            {transcripts.map((t) => (
+              <div key={t.id} className={`transcript-item ${t.role}`}>
+                <div className="message-header">
+                  <span className="role">{t.role === 'user' ? 'Caller' : 'AI Assistant'}</span>
+                  <span className="timestamp">{t.timestamp}</span>
+                </div>
+                <div className="message-bubble">
+                  <p className="text">{t.text}</p>
+                </div>
+              </div>
+            ))}
+            <div ref={transcriptsEndRef} />
+          </div>
+        )}
+
+        {activeTab === 'microphone' && (
+          <MicrophoneTest />
+        )}
+
+        {activeTab === 'tts' && (
+          <TextToSpeech />
+        )}
+
+        {activeTab === 'stt-llm-tts' && (
+          <SttLlmTts />
+        )}
+      </main>
+    </div>
+  );
+}
+
+export default App;

web_demo/src/components/MicrophoneTest.css

@@ -0,0 +1,315 @@
+.microphone-test {
+  display: flex;
+  flex-direction: column;
+  gap: 1.5rem;
+  padding: 0;
+  height: 100%;
+  width: 100%;
+}
+
+.mic-header {
+  text-align: center;
+}
+
+.mic-header h2 {
+  font-size: 1.5rem;
+  font-weight: 600;
+  margin-bottom: 0.5rem;
+  color: var(--text-primary);
+}
+
+.language-selector-stt {
+  display: flex;
+  flex-direction: column;
+  align-items: center;
+  gap: 0.5rem;
+  background: rgba(255, 255, 255, 0.03);
+  padding: 1rem;
+  border-radius: 8px;
+  margin-bottom: 1rem;
+}
+
+.language-selector-stt label {
+  font-size: 0.875rem;
+  color: var(--text-secondary);
+  font-weight: 500;
+}
+
+.tiny-info {
+  font-size: 0.75rem;
+  color: var(--error-color);
+  margin-top: 0.25rem;
+}
+
+.subtitle {
+  color: var(--text-secondary);
+  font-size: 0.875rem;
+}
+
+.error-message {
+  background-color: rgba(239, 68, 68, 0.1);
+  border: 1px solid var(--error-color);
+  color: var(--error-color);
+  padding: 0.75rem 1rem;
+  border-radius: 8px;
+  font-size: 0.875rem;
+}
+
+/* Audio Visualizer */
+.audio-visualizer {
+  background: rgba(255, 255, 255, 0.03);
+  border-radius: 12px;
+  padding: 1.5rem;
+  display: flex;
+  flex-direction: column;
+  gap: 1rem;
+}
+
+.visualizer-container {
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  gap: 4px;
+  height: 80px;
+  background: rgba(0, 0, 0, 0.2);
+  border-radius: 8px;
+  padding: 1rem;
+}
+
+.visualizer-bar {
+  width: 6px;
+  min-height: 5px;
+  background: linear-gradient(to top, var(--accent-color), #60a5fa);
+  border-radius: 3px;
+  transition: height 0.1s ease;
+}
+
+.audio-level-indicator {
+  display: flex;
+  align-items: center;
+  gap: 0.75rem;
+  color: var(--text-secondary);
+}
+
+.level-bar {
+  flex: 1;
+  height: 8px;
+  background: rgba(255, 255, 255, 0.1);
+  border-radius: 4px;
+  overflow: hidden;
+}
+
+.level-fill {
+  height: 100%;
+  background: linear-gradient(to right, var(--success-color), #4ade80);
+  transition: width 0.1s ease;
+  border-radius: 4px;
+}
+
+/* Controls */
+.controls {
+  display: flex;
+  gap: 0.75rem;
+  justify-content: center;
+  flex-wrap: wrap;
+}
+
+.btn {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  padding: 0.75rem 1.5rem;
+  border: none;
+  border-radius: 8px;
+  font-size: 0.875rem;
+  font-weight: 500;
+  cursor: pointer;
+  transition: all 0.2s ease;
+  font-family: inherit;
+}
+
+.btn:hover {
+  transform: translateY(-2px);
+  box-shadow: 0 4px 12px rgba(0, 0, 0, 0.2);
+}
+
+.btn:active {
+  transform: translateY(0);
+}
+
+.btn-primary {
+  background: var(--accent-color);
+  color: white;
+}
+
+.btn-primary:hover {
+  background: #2563eb;
+}
+
+.btn-success {
+  background: var(--success-color);
+  color: white;
+}
+
+.btn-success:hover {
+  background: #16a34a;
+}
+
+.btn-warning {
+  background: #f59e0b;
+  color: white;
+}
+
+.btn-warning:hover {
+  background: #d97706;
+}
+
+.btn-danger {
+  background: var(--error-color);
+  color: white;
+}
+
+.btn-danger:hover {
+  background: #dc2626;
+}
+
+.btn-secondary {
+  background: rgba(255, 255, 255, 0.1);
+  color: var(--text-primary);
+}
+
+.btn-secondary:hover {
+  background: rgba(255, 255, 255, 0.15);
+}
+
+/* Transcript Box */
+.transcript-box {
+  flex: 1;
+  background: rgba(255, 255, 255, 0.03);
+  border-radius: 12px;
+  padding: 1.5rem;
+  display: flex;
+  flex-direction: column;
+  min-height: 200px;
+}
+
+.transcript-header {
+  display: flex;
+  justify-content: space-between;
+  align-items: center;
+  margin-bottom: 1rem;
+  padding-bottom: 0.75rem;
+  border-bottom: 1px solid rgba(255, 255, 255, 0.1);
+}
+
+.transcript-header h3 {
+  font-size: 1.125rem;
+  font-weight: 600;
+  color: var(--text-primary);
+}
+
+.recording-indicator {
+  display: flex;
+  align-items: center;
+  gap: 0.5rem;
+  font-size: 0.875rem;
+  color: var(--error-color);
+  font-weight: 500;
+}
+
+.pulse-dot {
+  width: 8px;
+  height: 8px;
+  background: var(--error-color);
+  border-radius: 50%;
+  animation: pulse 1.5s ease-in-out infinite;
+}
+
+@keyframes pulse {
+
+  0%,
+  100% {
+    opacity: 1;
+    transform: scale(1);
+  }
+
+  50% {
+    opacity: 0.5;
+    transform: scale(1.2);
+  }
+}
+
+.transcript-content {
+  flex: 1;
+  overflow-y: auto;
+  line-height: 1.6;
+}
+
+.placeholder {
+  color: var(--text-secondary);
+  font-style: italic;
+  text-align: center;
+  margin-top: 2rem;
+}
+
+.final-transcript {
+  color: var(--text-primary);
+  margin-bottom: 0.5rem;
+}
+
+.interim-transcript {
+  color: var(--text-secondary);
+  font-style: italic;
+}
+
+/* Info Box */
+.info-box {
+  background: rgba(59, 130, 246, 0.1);
+  border: 1px solid rgba(59, 130, 246, 0.3);
+  border-radius: 8px;
+  padding: 1rem;
+}
+
+.info-box h4 {
+  color: var(--accent-color);
+  font-size: 0.875rem;
+  font-weight: 600;
+  margin-bottom: 0.5rem;
+}
+
+.info-box ul {
+  list-style: none;
+  padding: 0;
+  margin: 0;
+}
+
+.info-box li {
+  color: var(--text-secondary);
+  font-size: 0.8125rem;
+  padding: 0.25rem 0;
+  padding-left: 1.25rem;
+  position: relative;
+}
+
+.info-box li::before {
+  content: "•";
+  position: absolute;
+  left: 0.5rem;
+  color: var(--accent-color);
+}
+
+/* Responsive */
+@media (max-width: 640px) {
+  .controls {
+    flex-direction: column;
+  }
+
+  .btn {
+    width: 100%;
+    justify-content: center;
+  }
+
+  .visualizer-container {
+    height: 60px;
+  }
+}

web_demo/src/components/MicrophoneTest.jsx

@@ -0,0 +1,307 @@
+import React, { useState, useRef, useEffect } from 'react';
+import { Mic, MicOff, Play, Pause, Trash2, Volume2 } from 'lucide-react';
+import './MicrophoneTest.css';
+
+function MicrophoneTest() {
+    const [isRecording, setIsRecording] = useState(false);
+    const [isPaused, setIsPaused] = useState(false);
+    const [transcript, setTranscript] = useState('');
+    const [interimTranscript, setInterimTranscript] = useState('');
+    const [audioLevel, setAudioLevel] = useState(0);
+    const [error, setError] = useState('');
+    const [selectedLang, setSelectedLang] = useState('en-IN'); // Default to Indian English
+
+    const recognitionRef = useRef(null);
+    const audioContextRef = useRef(null);
+    const analyserRef = useRef(null);
+    const microphoneRef = useRef(null);
+    const animationFrameRef = useRef(null);
+
+    const languages = [
+        { code: 'en-IN', name: 'English (India)' },
+        { code: 'en-US', name: 'English (US)' },
+        { code: 'en-GB', name: 'English (UK)' },
+        { code: 'hi-IN', name: 'Hindi (India)' },
+    ];
+
+    // Initialize Web Speech API
+    useEffect(() => {
+        if ('webkitSpeechRecognition' in window || 'SpeechRecognition' in window) {
+            const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
+            recognitionRef.current = new SpeechRecognition();
+            recognitionRef.current.continuous = true;
+            recognitionRef.current.interimResults = true;
+            recognitionRef.current.lang = selectedLang;
+
+            recognitionRef.current.onresult = (event) => {
+                let interim = '';
+                let final = '';
+
+                for (let i = event.resultIndex; i < event.results.length; i++) {
+                    const transcriptPiece = event.results[i][0].transcript;
+                    if (event.results[i].isFinal) {
+                        final += transcriptPiece + ' ';
+                    } else {
+                        interim += transcriptPiece;
+                    }
+                }
+
+                if (final) {
+                    setTranscript(prev => prev + final);
+                    setInterimTranscript('');
+                } else {
+                    setInterimTranscript(interim);
+                }
+            };
+
+            recognitionRef.current.onerror = (event) => {
+                console.error('Speech recognition error:', event.error);
+                if (event.error === 'no-speech') {
+                    // Ignore no-speech errors to prevent UI flicker
+                    return;
+                }
+                setError(`Error: ${event.error}`);
+            };
+
+            recognitionRef.current.onend = () => {
+                if (isRecording && !isPaused) {
+                    try {
+                        recognitionRef.current.start();
+                    } catch (e) {
+                        console.error("Failed to restart recognition:", e);
+                    }
+                }
+            };
+        } else {
+            setError('Speech recognition is not supported in this browser. Please use Chrome or Edge.');
+        }
+
+        return () => {
+            if (recognitionRef.current) {
+                recognitionRef.current.stop();
+            }
+            stopAudioVisualization();
+        };
+    }, [selectedLang]); // Re-initialize when language changes
+
+    // Audio visualization
+    const startAudioVisualization = async () => {
+        try {
+            const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+            audioContextRef.current = new (window.AudioContext || window.webkitAudioContext)();
+            analyserRef.current = audioContextRef.current.createAnalyser();
+            microphoneRef.current = audioContextRef.current.createMediaStreamSource(stream);
+
+            analyserRef.current.fftSize = 256;
+            microphoneRef.current.connect(analyserRef.current);
+
+            const dataArray = new Uint8Array(analyserRef.current.frequencyBinCount);
+
+            const updateLevel = () => {
+                if (!analyserRef.current) return;
+                analyserRef.current.getByteFrequencyData(dataArray);
+                const average = dataArray.reduce((a, b) => a + b) / dataArray.length;
+                setAudioLevel(average);
+                animationFrameRef.current = requestAnimationFrame(updateLevel);
+            };
+
+            updateLevel();
+        } catch (err) {
+            console.error('Error accessing microphone:', err);
+            setError('Could not access microphone. Please check permissions.');
+        }
+    };
+
+    const stopAudioVisualization = () => {
+        if (animationFrameRef.current) {
+            cancelAnimationFrame(animationFrameRef.current);
+        }
+        if (microphoneRef.current && microphoneRef.current.mediaStream) {
+            microphoneRef.current.mediaStream.getTracks().forEach(track => track.stop());
+        }
+        if (audioContextRef.current) {
+            audioContextRef.current.close();
+            audioContextRef.current = null;
+            analyserRef.current = null;
+        }
+    };
+
+    const startRecording = () => {
+        setError('');
+        setIsRecording(true);
+        setIsPaused(false);
+
+        if (recognitionRef.current) {
+            try {
+                recognitionRef.current.start();
+            } catch (e) {
+                console.error("Recognition already started:", e);
+            }
+        }
+        startAudioVisualization();
+    };
+
+    const pauseRecording = () => {
+        setIsPaused(true);
+        if (recognitionRef.current) {
+            recognitionRef.current.stop();
+        }
+    };
+
+    const resumeRecording = () => {
+        setIsPaused(false);
+        if (recognitionRef.current) {
+            try {
+                recognitionRef.current.start();
+            } catch (e) {
+                console.error("Failed to resume recognition:", e);
+            }
+        }
+    };
+
+    const stopRecording = () => {
+        setIsRecording(false);
+        setIsPaused(false);
+
+        if (recognitionRef.current) {
+            recognitionRef.current.stop();
+        }
+        stopAudioVisualization();
+        setAudioLevel(0);
+    };
+
+    const clearTranscript = () => {
+        setTranscript('');
+        setInterimTranscript('');
+        setError('');
+    };
+
+    return (
+        <div className="microphone-test">
+            <div className="mic-header">
+                <h2>Speech-to-Text Test</h2>
+                <p className="subtitle">Test your microphone with Indian English support</p>
+            </div>
+
+            <div className="language-selector-stt">
+                <label htmlFor="stt-lang">Recognition Language: </label>
+                <select
+                    id="stt-lang"
+                    value={selectedLang}
+                    onChange={(e) => {
+                        const wasRecording = isRecording;
+                        if (wasRecording) stopRecording();
+                        setSelectedLang(e.target.value);
+                    }}
+                    disabled={isRecording}
+                    className="voice-select"
+                >
+                    {languages.map(lang => (
+                        <option key={lang.code} value={lang.code}>{lang.name}</option>
+                    ))}
+                </select>
+                {isRecording && <p className="tiny-info">Stop recording to change language</p>}
+            </div>
+
+            {error && (
+                <div className="error-message">
+                    <span>⚠️ {error}</span>
+                </div>
+            )}
+
+            <div className="audio-visualizer">
+                <div className="visualizer-container">
+                    {[...Array(20)].map((_, i) => (
+                        <div
+                            key={i}
+                            className="visualizer-bar"
+                            style={{
+                                height: `${isRecording && !isPaused ? Math.random() * audioLevel * 2 : 5}px`,
+                                animationDelay: `${i * 0.05}s`
+                            }}
+                        />
+                    ))}
+                </div>
+                <div className="audio-level-indicator">
+                    <Volume2 size={20} />
+                    <div className="level-bar">
+                        <div
+                            className="level-fill"
+                            style={{ width: `${Math.min(audioLevel, 100)}%` }}
+                        />
+                    </div>
+                    <span>{Math.round(audioLevel)}%</span>
+                </div>
+            </div>
+
+            <div className="controls">
+                {!isRecording ? (
+                    <button className="btn btn-primary" onClick={startRecording}>
+                        <Mic size={20} />
+                        <span>Start Recording</span>
+                    </button>
+                ) : (
+                    <>
+                        {!isPaused ? (
+                            <button className="btn btn-warning" onClick={pauseRecording}>
+                                <Pause size={20} />
+                                <span>Pause</span>
+                            </button>
+                        ) : (
+                            <button className="btn btn-success" onClick={resumeRecording}>
+                                <Play size={20} />
+                                <span>Resume</span>
+                            </button>
+                        )}
+                        <button className="btn btn-danger" onClick={stopRecording}>
+                            <MicOff size={20} />
+                            <span>Stop</span>
+                        </button>
+                    </>
+                )}
+                {transcript && (
+                    <button className="btn btn-secondary" onClick={clearTranscript}>
+                        <Trash2 size={20} />
+                        <span>Clear</span>
+                    </button>
+                )}
+            </div>
+
+            <div className="transcript-box">
+                <div className="transcript-header">
+                    <h3>Transcript ({languages.find(l => l.code === selectedLang)?.name})</h3>
+                    {isRecording && (
+                        <span className="recording-indicator">
+                            <span className="pulse-dot"></span>
+                            {isPaused ? 'Paused' : 'Recording...'}
+                        </span>
+                    )}
+                </div>
+                <div className="transcript-content">
+                    {!transcript && !interimTranscript ? (
+                        <p className="placeholder">Your transcription will appear here...</p>
+                    ) : (
+                        <>
+                            <p className="final-transcript">{transcript}</p>
+                            {interimTranscript && (
+                                <p className="interim-transcript">{interimTranscript}</p>
+                            )}
+                        </>
+                    )}
+                </div>
+            </div>
+
+            <div className="info-box">
+                <h4>💡 Tips:</h4>
+                <ul>
+                    <li>Selecting <b>English (India)</b> will significantly improve recognition for Indian accents.</li>
+                    <li>You can even try <b>Hindi</b> if you want to test multilingual support!</li>
+                    <li>Make sure your browser has microphone permissions enabled</li>
+                    <li>Works best in Chrome, Edge, or Safari</li>
+                </ul>
+            </div>
+        </div>
+    );
+}
+
+export default MicrophoneTest;

web_demo/src/components/SttLlmTts.css

@@ -0,0 +1,653 @@
+.stt-llm-tts-test {
+    display: flex;
+    flex-direction: column;
+    gap: 2rem;
+    padding: 1rem 0;
+    height: calc(100vh - 180px);
+    /* Fill the vertical space */
+    width: 100%;
+    margin: 0;
+}
+
+.test-header {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    background: var(--card-bg);
+    padding: 1.25rem;
+    border-radius: 12px;
+    box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1);
+}
+
+.title-group h2 {
+    font-size: 1.25rem;
+    font-weight: 600;
+    margin-bottom: 0.25rem;
+}
+
+.title-group .subtitle {
+    font-size: 0.875rem;
+    color: var(--text-secondary);
+}
+
+.action-buttons {
+    display: flex;
+    gap: 0.75rem;
+    align-items: center;
+}
+
+.settings-btn {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    width: 42px;
+    height: 42px;
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    background: rgba(255, 255, 255, 0.05);
+    color: var(--text-secondary);
+    border-radius: 8px;
+    cursor: pointer;
+    transition: all 0.2s ease;
+}
+
+.settings-btn:hover {
+    background: rgba(59, 130, 246, 0.1);
+    color: var(--accent-color);
+    border-color: var(--accent-color);
+}
+
+.record-toggle {
+    display: flex;
+    align-items: center;
+    gap: 0.75rem;
+    padding: 0.75rem 1.5rem;
+    border: none;
+    border-radius: 8px;
+    background: var(--accent-color);
+    color: white;
+    font-weight: 600;
+    cursor: pointer;
+    transition: all 0.2s ease;
+}
+
+.record-toggle.recording {
+    background: var(--error-color);
+    animation: pulse-red 2s infinite;
+}
+
+.reset-session {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    width: 42px;
+    height: 42px;
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    background: rgba(255, 255, 255, 0.05);
+    color: var(--text-secondary);
+    border-radius: 8px;
+    cursor: pointer;
+    transition: all 0.2s ease;
+}
+
+.reset-session:hover {
+    background: rgba(255, 255, 255, 0.1);
+    color: var(--text-primary);
+}
+
+/* Pipeline Columns */
+.pipeline-columns {
+    display: grid;
+    grid-template-columns: repeat(3, 1fr);
+    gap: 1.5rem;
+    flex: 2;
+    /* Take more relative space */
+    min-height: 450px;
+    /* Force substantial height */
+}
+
+.pipeline-col {
+    background: var(--card-bg);
+    border-radius: 12px;
+    display: flex;
+    flex-direction: column;
+    overflow: hidden;
+    border: 1px solid rgba(255, 255, 255, 0.05);
+    box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1);
+}
+
+.col-header {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    gap: 0.75rem;
+    padding: 1rem 1.25rem;
+    background: rgba(255, 255, 255, 0.03);
+    border-bottom: 1px solid rgba(255, 255, 255, 0.05);
+}
+
+.title-with-model {
+    display: flex;
+    align-items: center;
+    gap: 0.75rem;
+}
+
+.model-tag {
+    font-size: 0.625rem;
+    font-weight: 700;
+    text-transform: uppercase;
+    padding: 0.25rem 0.625rem;
+    background: rgba(255, 255, 255, 0.06);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 999px;
+    color: var(--text-secondary);
+    letter-spacing: 0.05em;
+}
+
+.col-header.stt .model-tag {
+    color: #60a5fa;
+    border-color: rgba(96, 165, 250, 0.3);
+    background: rgba(96, 165, 250, 0.1);
+}
+
+.col-header.llm .model-tag {
+    color: #a78bfa;
+    border-color: rgba(167, 139, 250, 0.3);
+    background: rgba(167, 139, 250, 0.1);
+}
+
+.col-header.tts .model-tag {
+    color: #4ade80;
+    border-color: rgba(74, 222, 128, 0.3);
+    background: rgba(74, 222, 128, 0.1);
+}
+
+.col-header h3 {
+    font-size: 1rem;
+    font-weight: 700;
+    letter-spacing: 0.025em;
+    text-transform: uppercase;
+}
+
+.col-header.stt {
+    color: #60a5fa;
+}
+
+.col-header.llm {
+    color: #a78bfa;
+}
+
+.col-header.tts {
+    color: #4ade80;
+}
+
+.col-content {
+    flex: 1;
+    padding: 1.25rem;
+    display: flex;
+    flex-direction: column;
+    overflow-y: auto;
+    position: relative;
+}
+
+.text-display {
+    flex: 1;
+    padding: 1.5rem;
+    background: rgba(15, 23, 42, 0.4);
+    border: 1px solid rgba(255, 255, 255, 0.05);
+    border-radius: 12px;
+    font-size: 1rem;
+    line-height: 1.7;
+    backdrop-filter: blur(4px);
+    transition: all 0.3s ease;
+}
+
+.text-display:hover {
+    background: rgba(15, 23, 42, 0.6);
+    border-color: rgba(255, 255, 255, 0.1);
+}
+
+.empty-msg {
+    color: var(--text-secondary);
+    font-style: italic;
+    font-size: 0.875rem;
+    text-align: center;
+    margin-top: 2rem;
+}
+
+/* STT Column */
+.final-text {
+    color: var(--text-primary);
+}
+
+.interim-text {
+    color: var(--text-secondary);
+    font-style: italic;
+}
+
+.recording-pulse {
+    margin-top: 1rem;
+    font-size: 0.75rem;
+    color: var(--error-color);
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+}
+
+.recording-pulse::before {
+    content: '';
+    width: 8px;
+    height: 8px;
+    background: var(--error-color);
+    border-radius: 50%;
+    animation: pulse-red 1s infinite;
+}
+
+.mic-muted-status {
+    margin-top: 1rem;
+    font-size: 0.75rem;
+    color: var(--accent-color);
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+    padding: 0.5rem;
+    background: rgba(59, 130, 246, 0.1);
+    border-radius: 6px;
+    animation: fade-in 0.3s ease-out;
+}
+
+/* LLM Column */
+.loading-state {
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    justify-content: center;
+    height: 100%;
+    gap: 1rem;
+    color: #a78bfa;
+}
+
+.spinner {
+    animation: rotate 2s linear infinite;
+}
+
+.response-box {
+    animation: fade-in 0.3s ease-out;
+}
+
+.response-text {
+    color: #e9d5ff;
+}
+
+/* TTS Column */
+.tts-status {
+    flex: 1;
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    justify-content: center;
+    gap: 1.5rem;
+}
+
+.status-indicator {
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    gap: 1rem;
+    color: var(--text-secondary);
+    transition: all 0.3s ease;
+}
+
+.status-indicator.playing {
+    color: #4ade80;
+}
+
+.bouncing {
+    animation: bounce 1s infinite ease-in-out;
+}
+
+.replay-btn {
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+    padding: 0.5rem 1rem;
+    background: rgba(255, 255, 255, 0.1);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 6px;
+    color: var(--text-primary);
+    font-size: 0.8125rem;
+    cursor: pointer;
+}
+
+.voice-selection-compact {
+    display: flex;
+    flex-direction: column;
+    gap: 0.5rem;
+    margin-bottom: 1.5rem;
+    padding: 1rem;
+    background: rgba(255, 255, 255, 0.03);
+    border: 1px solid rgba(255, 255, 255, 0.05);
+    border-radius: 10px;
+}
+
+.voice-selection-compact label {
+    font-size: 0.75rem;
+    text-transform: uppercase;
+    font-weight: 700;
+    color: var(--text-secondary);
+    letter-spacing: 0.05em;
+}
+
+.voice-selection-compact select {
+    background: rgba(15, 23, 42, 0.6);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    color: white;
+    padding: 0.625rem;
+    border-radius: 6px;
+    font-size: 0.875rem;
+    outline: none;
+    cursor: pointer;
+    transition: all 0.2s ease;
+}
+
+.voice-selection-compact select:hover {
+    background: rgba(15, 23, 42, 0.8);
+    border-color: var(--accent-color);
+}
+
+.auto-toggle {
+    display: flex;
+    align-items: center;
+    gap: 0.75rem;
+    padding-top: 1.25rem;
+    border-top: 1px solid rgba(255, 255, 255, 0.05);
+    font-size: 0.8125rem;
+    color: var(--text-secondary);
+}
+
+/* History Tray */
+.history-tray {
+    flex: 1;
+    /* Limit growth compared to pipeline */
+    min-height: 150px;
+    background: rgba(30, 41, 59, 0.5);
+    backdrop-filter: blur(8px);
+    border-radius: 16px;
+    padding: 1.5rem;
+    border: 1px solid rgba(255, 255, 255, 0.08);
+    display: flex;
+    flex-direction: column;
+}
+
+.history-tray h4 {
+    font-size: 0.875rem;
+    font-weight: 600;
+    margin-bottom: 1rem;
+    color: var(--text-secondary);
+}
+
+.history-list {
+    display: flex;
+    flex-direction: column;
+    gap: 0.75rem;
+    max-height: 300px;
+    overflow-y: auto;
+}
+
+.history-item {
+    font-size: 0.875rem;
+    display: flex;
+    gap: 0.5rem;
+}
+
+.h-role {
+    font-weight: 600;
+    min-width: 40px;
+}
+
+.user .h-role {
+    color: #60a5fa;
+}
+
+.assistant .h-role {
+    color: #a78bfa;
+}
+
+.no-history {
+    font-size: 0.8125rem;
+    color: var(--text-secondary);
+    font-style: italic;
+}
+
+/* Switch Toggle */
+.switch {
+    position: relative;
+    display: inline-block;
+    width: 34px;
+    height: 20px;
+}
+
+.switch input {
+    opacity: 0;
+    width: 0;
+    height: 0;
+}
+
+.slider {
+    position: absolute;
+    cursor: pointer;
+    top: 0;
+    left: 0;
+    right: 0;
+    bottom: 0;
+    background-color: #334155;
+    transition: .4s;
+}
+
+.slider:before {
+    position: absolute;
+    content: "";
+    height: 14px;
+    width: 14px;
+    left: 3px;
+    bottom: 3px;
+    background-color: white;
+    transition: .4s;
+}
+
+input:checked+.slider {
+    background-color: var(--success-color);
+}
+
+input:checked+.slider:before {
+    transform: translateX(14px);
+}
+
+.slider.round {
+    border-radius: 34px;
+}
+
+.slider.round:before {
+    border-radius: 50%;
+}
+
+/* Animations */
+@keyframes pulse-red {
+    0% {
+        box-shadow: 0 0 0 0 rgba(239, 68, 68, 0.4);
+    }
+
+    70% {
+        box-shadow: 0 0 0 10px rgba(239, 68, 68, 0);
+    }
+
+    100% {
+        box-shadow: 0 0 0 0 rgba(239, 68, 68, 0);
+    }
+}
+
+@keyframes rotate {
+    from {
+        transform: rotate(0deg);
+    }
+
+    to {
+        transform: rotate(360deg);
+    }
+}
+
+@keyframes bounce {
+
+    0%,
+    100% {
+        transform: translateY(0);
+    }
+
+    50% {
+        transform: translateY(-10px);
+    }
+}
+
+@keyframes fade-in {
+    from {
+        opacity: 0;
+        transform: translateY(5px);
+    }
+
+    to {
+        opacity: 1;
+        transform: translateY(0);
+    }
+}
+
+/* Settings Overlay */
+.pipeline-settings-overlay {
+    position: fixed;
+    top: 0;
+    left: 0;
+    right: 0;
+    bottom: 0;
+    background: rgba(15, 23, 42, 0.8);
+    backdrop-filter: blur(8px);
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    z-index: 1000;
+    animation: fade-in 0.2s ease-out;
+}
+
+.settings-card {
+    background: var(--card-bg);
+    padding: 2.5rem;
+    border-radius: 20px;
+    width: 100%;
+    max-width: 650px;
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.5);
+}
+
+.settings-card h3 {
+    margin-bottom: 1.5rem;
+    font-size: 1.25rem;
+}
+
+.setting-item {
+    display: flex;
+    flex-direction: column;
+    gap: 0.75rem;
+    margin-bottom: 2rem;
+}
+
+.setting-item label {
+    font-size: 0.875rem;
+    color: var(--text-secondary);
+    font-weight: 500;
+}
+
+.status-badge {
+    padding: 0.75rem 1rem;
+    background: rgba(0, 0, 0, 0.2);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 8px;
+    font-size: 0.875rem;
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+}
+
+.setting-item textarea {
+    width: 100%;
+    padding: 0.75rem 1rem;
+    background: rgba(0, 0, 0, 0.3);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 8px;
+    color: white;
+    font-size: 0.9rem;
+    resize: none;
+    line-height: 1.5;
+    outline: none;
+    transition: border-color 0.2s;
+}
+
+.setting-item textarea:focus {
+    border-color: var(--accent-color);
+}
+
+.prompt-presets {
+    margin-top: 1.5rem;
+    margin-bottom: 1.5rem;
+}
+
+.prompt-presets label {
+    display: block;
+    font-size: 0.75rem;
+    text-transform: uppercase;
+    font-weight: 700;
+    color: var(--text-secondary);
+    margin-bottom: 0.75rem;
+}
+
+.preset-btns {
+    display: flex;
+    flex-wrap: wrap;
+    gap: 0.5rem;
+}
+
+.preset-btn {
+    background: rgba(255, 255, 255, 0.05);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    color: var(--text-secondary);
+    padding: 0.5rem 0.75rem;
+    border-radius: 6px;
+    font-size: 0.8rem;
+    cursor: pointer;
+    transition: all 0.2s;
+}
+
+.preset-btn:hover {
+    background: rgba(255, 255, 255, 0.1);
+    color: white;
+    border-color: var(--accent-color);
+}
+
+.hint {
+    font-size: 0.75rem;
+    color: var(--text-secondary);
+    font-style: italic;
+}
+
+.close-settings {
+    width: 100%;
+    padding: 1rem;
+    background: var(--accent-color);
+    border: none;
+    border-radius: 10px;
+    color: white;
+    font-weight: 600;
+    cursor: pointer;
+    transition: all 0.2s ease;
+}
+
+.close-settings:hover {
+    background: var(--accent-hover);
+    transform: translateY(-2px);
+}

web_demo/src/components/SttLlmTts.jsx

@@ -0,0 +1,505 @@
+import React, { useState, useEffect, useRef } from 'react';
+import { Mic, MicOff, MessageSquare, Volume2, Loader2, Send, RotateCcw, Settings } from 'lucide-react';
+import './SttLlmTts.css';
+
+function SttLlmTts() {
+    const [isRecording, setIsRecording] = useState(false);
+    const [sttText, setSttText] = useState('');
+    const [interimStt, setInterimStt] = useState('');
+    const [llmResponse, setLlmResponse] = useState('');
+    const [isLlmLoading, setIsLlmLoading] = useState(false);
+    const [ttsStatus, setTtsStatus] = useState('Idle');
+    const [history, setHistory] = useState(() => {
+        const saved = localStorage.getItem('nv_history');
+        return saved ? JSON.parse(saved) : [];
+    });
+    const [error, setError] = useState('');
+    const [autoMode, setAutoMode] = useState(true);
+    const [apiKey, setApiKey] = useState(import.meta.env.VITE_OPENAI_API_KEY || ''); // Load from .env if available
+    const [showSettings, setShowSettings] = useState(false);
+    const [voices, setVoices] = useState([]);
+    const [selectedVoiceURI, setSelectedVoiceURI] = useState('');
+    const [systemPrompt, setSystemPrompt] = useState(() => {
+        return localStorage.getItem('nv_system_prompt') || 'You are a professional Health Insurance Seller. Start by greeting the user and asking if they want a plan for themselves or their family. Keep answers brief.';
+    });
+
+    const recognitionRef = useRef(null);
+    const synthRef = useRef(window.speechSynthesis);
+    const scrollRef = useRef(null);
+    const isBusyRef = useRef(false);
+    const autoModeRef = useRef(true);
+    const isRecordingRef = useRef(false); // New: Track recording state for handlers
+    const isMicActiveRef = useRef(false); // New: Track hardware status to prevent lock-up
+    const silenceTimerRef = useRef(null); // Ref for auto-processing on silence
+    const [isMicActuallyListening, setIsMicActuallyListening] = useState(false);
+
+    // Persistent Storage
+    useEffect(() => {
+        localStorage.setItem('nv_history', JSON.stringify(history));
+    }, [history]);
+
+    useEffect(() => {
+        localStorage.setItem('nv_system_prompt', systemPrompt);
+    }, [systemPrompt]);
+
+    // Auto-scroll to bottom
+    useEffect(() => {
+        if (scrollRef.current) {
+            scrollRef.current.scrollTop = scrollRef.current.scrollHeight;
+        }
+    }, [history, sttText, interimStt, llmResponse]);
+
+    // Load voices
+    useEffect(() => {
+        const loadVoices = () => {
+            const availableVoices = synthRef.current.getVoices();
+            setVoices(availableVoices);
+
+            // Default to Indian English if not already set
+            if (!selectedVoiceURI && availableVoices.length > 0) {
+                const indianVoice = availableVoices.find(v => v.lang === 'en-IN' || v.name.includes('India'));
+                const defaultVoice = indianVoice || availableVoices[0];
+                setSelectedVoiceURI(defaultVoice.voiceURI || defaultVoice.name);
+            }
+        };
+
+        loadVoices();
+        if (synthRef.current.onvoiceschanged !== undefined) {
+            synthRef.current.onvoiceschanged = loadVoices;
+        }
+    }, [selectedVoiceURI]);
+
+    // Initialize STT
+    useEffect(() => {
+        if ('webkitSpeechRecognition' in window || 'SpeechRecognition' in window) {
+            const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;
+            recognitionRef.current = new SpeechRecognition();
+            recognitionRef.current.continuous = false; // Stop when the user pauses
+            recognitionRef.current.interimResults = true;
+            recognitionRef.current.lang = 'en-IN'; // Indian English as default
+
+            recognitionRef.current.onresult = (event) => {
+                let interim = '';
+                let final = '';
+
+                for (let i = event.resultIndex; i < event.results.length; i++) {
+                    const piece = event.results[i][0].transcript;
+                    if (event.results[i].isFinal) {
+                        final += piece;
+                    } else {
+                        interim += piece;
+                    }
+                }
+
+                if (final) {
+                    clearTimeout(silenceTimerRef.current);
+                    handleFinalStt(final);
+                } else {
+                    setInterimStt(interim);
+
+                    // AUTO-STOP: If we have interim text but no final for 1.5s, process it anyway
+                    if (interim.trim()) {
+                        clearTimeout(silenceTimerRef.current);
+                        silenceTimerRef.current = setTimeout(() => {
+                            handleFinalStt(interim);
+                            if (recognitionRef.current) recognitionRef.current.stop();
+                        }, 1500);
+                    }
+                }
+            };
+
+            recognitionRef.current.onerror = (event) => {
+                if (event.error !== 'no-speech') {
+                    setError(`STT Error: ${event.error}`);
+                }
+            };
+
+            recognitionRef.current.onstart = () => {
+                isMicActiveRef.current = true;
+                setIsMicActuallyListening(true);
+            };
+
+            recognitionRef.current.onend = () => {
+                isMicActiveRef.current = false;
+                setIsMicActuallyListening(false);
+
+                // Hardware cooldown: Wait 300ms before attempting to restart to avoid hardware lock
+                setTimeout(() => {
+                    if (isRecordingRef.current && !isBusyRef.current && !isMicActiveRef.current) {
+                        try {
+                            recognitionRef.current.start();
+                        } catch (e) {
+                            console.log("Mic restart safe-check:", e.message);
+                        }
+                    }
+                }, 300);
+            };
+        } else {
+            setError('Speech recognition not supported in this browser.');
+        }
+
+        return () => {
+            if (recognitionRef.current) recognitionRef.current.stop();
+            synthRef.current.cancel();
+        };
+    }, [isRecording]);
+
+    const handleFinalStt = (text) => {
+        if (!text.trim() || isBusyRef.current) return;
+
+        clearTimeout(silenceTimerRef.current);
+        setSttText(text); // Clear previous and show new turn
+        setInterimStt('');
+
+        // STEP 1: Lock the turn
+        isBusyRef.current = true;
+
+        if (recognitionRef.current) {
+            try { recognitionRef.current.stop(); } catch (e) { }
+        }
+        processLlm(text);
+    };
+
+    const processLlm = async (text) => {
+        setIsLlmLoading(true);
+        setLlmResponse('Thinking...');
+
+        try {
+            let responseText = '';
+
+            if (apiKey) {
+                // REAL AI CALL
+                const response = await fetch('https://api.openai.com/v1/chat/completions', {
+                    method: 'POST',
+                    headers: {
+                        'Content-Type': 'application/json',
+                        'Authorization': `Bearer ${apiKey}`
+                    },
+                    body: JSON.stringify({
+                        model: 'gpt-4o-mini',
+                        messages: [
+                            { role: 'system', content: systemPrompt },
+                            ...history.slice(-12),
+                            { role: 'user', content: text }
+                        ],
+                        temperature: 0.7,
+                        max_tokens: 100
+                    })
+                });
+
+                const data = await response.json();
+                if (data.error) throw new Error(data.error.message);
+                responseText = data.choices[0].message.content;
+            } else {
+                // FALLBACK SMART MOCK (for when no key is present)
+                await new Promise(r => setTimeout(r, 1000));
+                responseText = generateFallbackResponse(text);
+            }
+
+            setLlmResponse(responseText);
+            setHistory(prev => [...prev, { role: 'user', content: text }, { role: 'assistant', content: responseText }]);
+
+            if (autoModeRef.current) {
+                speakText(responseText);
+            } else {
+                isBusyRef.current = false;
+                // Re-awaken mic if auto-play is off
+                setTimeout(() => {
+                    if (isRecordingRef.current && !isMicActiveRef.current) {
+                        try { recognitionRef.current.start(); } catch (e) { }
+                    }
+                }, 300);
+            }
+        } catch (err) {
+            setError(`LLM Error: ${err.message}`);
+            setIsLlmLoading(false);
+            isBusyRef.current = false;
+            // Re-awaken mic on error
+            setTimeout(() => {
+                if (isRecordingRef.current && !isMicActiveRef.current) {
+                    try { recognitionRef.current.start(); } catch (e) { }
+                }
+            }, 300);
+        } finally {
+            setIsLlmLoading(false);
+        }
+    };
+
+    const generateFallbackResponse = (input) => {
+        const text = input.toLowerCase();
+        if (text.includes('prime minister') && text.includes('india')) return "As of January 2026, Narendra Modi is the Prime Minister of India.";
+        if (text.includes('hello') || text.includes('hi')) return "Hello! How can I help you today?";
+        if (text.includes('time')) return `The current time is ${new Date().toLocaleTimeString()}.`;
+        return `I processed your request: "${input}". For real answers, please add your OpenAI API Key in settings.`;
+    };
+
+    const speakText = (text) => {
+        synthRef.current.cancel();
+        const utterance = new SpeechSynthesisUtterance(text);
+
+        // ALWAYS find the fresh voice object from the system right before speaking
+        const currentVoices = synthRef.current.getVoices();
+        const voice = currentVoices.find(v => (v.voiceURI || v.name) === selectedVoiceURI);
+
+        if (voice) {
+            utterance.voice = voice;
+        }
+
+        utterance.onstart = () => {
+            setTtsStatus('Playing...');
+            // Mic is already stopped by handleFinalStt, but we ensure busy state remains
+            isBusyRef.current = true;
+        };
+
+        utterance.onend = () => {
+            setTtsStatus('Finished');
+            isBusyRef.current = false;
+            // STEP 3: Clear busy state and resume mic with cooldown
+            setTimeout(() => {
+                if (isRecordingRef.current && !isMicActiveRef.current) {
+                    try { recognitionRef.current.start(); } catch (e) { }
+                }
+            }, 300);
+        };
+
+        utterance.onerror = () => {
+            setTtsStatus('Error');
+            isBusyRef.current = false;
+            setTimeout(() => {
+                if (isRecordingRef.current && !isMicActiveRef.current) {
+                    try { recognitionRef.current.start(); } catch (e) { }
+                }
+            }, 300);
+        };
+
+        synthRef.current.speak(utterance);
+    };
+
+    const toggleRecording = () => {
+        if (isRecording) {
+            isRecordingRef.current = false;
+            isBusyRef.current = false;
+            try { recognitionRef.current.stop(); } catch (e) { }
+            setIsRecording(false);
+        } else {
+            setSttText('');
+            setInterimStt('');
+            setError('');
+            isRecordingRef.current = true;
+            isBusyRef.current = false;
+            try { recognitionRef.current.start(); } catch (e) { }
+            setIsRecording(true);
+        }
+    };
+
+    const resetAll = () => {
+        setHistory([]);
+        setSttText('');
+        setInterimStt('');
+        setLlmResponse('');
+        setTtsStatus('Idle');
+        synthRef.current.cancel();
+    };
+
+    return (
+        <div className="stt-llm-tts-test">
+            <div className="test-header">
+                <div className="title-group">
+                    <h2>STT → LLM → TTS Pipeline</h2>
+                    <p className="subtitle">Full Loop: Voice In, AI Processing, Voice Out</p>
+                </div>
+                <div className="action-buttons">
+                    <button className="settings-btn" onClick={() => setShowSettings(!showSettings)} title="AI Configuration">
+                        <Settings size={18} />
+                    </button>
+                    <button className={`record-toggle ${isRecording ? 'recording' : ''}`} onClick={toggleRecording}>
+                        {isRecording ? <MicOff size={20} /> : <Mic size={20} />}
+                        <span>{isRecording ? 'Stop Session' : 'Start Session'}</span>
+                    </button>
+                    <button className="reset-session" onClick={resetAll}>
+                        <RotateCcw size={18} />
+                    </button>
+                </div>
+            </div>
+
+            {showSettings && (
+                <div className="pipeline-settings-overlay">
+                    <div className="settings-card">
+                        <h3>AI Agent Configuration</h3>
+                        <div className="setting-item">
+                            <label>Agent Personality (System Prompt)</label>
+                            <textarea
+                                value={systemPrompt}
+                                onChange={(e) => setSystemPrompt(e.target.value)}
+                                rows={4}
+                                placeholder="Example: You are a professional doctor assistant..."
+                            />
+                            <p className="hint">This tells the AI how to behave and what to ask.</p>
+                        </div>
+
+                        <div className="prompt-presets">
+                            <label>Quick Presets</label>
+                            <div className="preset-btns">
+                                <button className="preset-btn" onClick={() => setSystemPrompt('You are a concise voice assistant. Give short answers (max 20 words).')}>
+                                    General Assistant
+                                </button>
+                                <button className="preset-btn" onClick={() => setSystemPrompt(`You are a highly professional Health Insurance Sales Agent. 
+Follow this EXACT conversation flow:
+1. Greet the user and ask if they are looking for a plan for themselves or their family.
+2. Once they answer, ask for the ages of the people to be insured.
+3. Next, ask if anyone has any pre-existing medical conditions (Yes/No).
+4. Finally, ask for their preferred annual budget.
+
+Rules:
+- Ask only ONE question at a time.
+- Keep your responses under 20 words.
+- Be polite, empathetic, and professional.
+- If they say something unrelated, steer them back to the last question.`)}>
+                                    Health Insurance Agent (Structured)
+                                </button>
+                                <button className="preset-btn" onClick={() => setSystemPrompt('You are a helpful travel agent. Ask the user about their favorite destination. Keep answers short.')}>
+                                    Travel Agent
+                                </button>
+                            </div>
+                        </div>
+
+                        <button className="close-settings" onClick={() => setShowSettings(false)}>Save & Close</button>
+                    </div>
+                </div>
+            )}
+
+            <div className="pipeline-columns">
+                {/* Column 1: STT */}
+                <div className="pipeline-col">
+                    <div className="col-header stt">
+                        <div className="title-with-model">
+                            <Mic size={18} />
+                            <h3>1. Speech-to-Text</h3>
+                        </div>
+                        <span className="model-tag">Web Speech API / Vosk</span>
+                    </div>
+                    <div className="col-content" ref={scrollRef}>
+                        <div className="text-display stt-display">
+                            {sttText && <p className="final-text">{sttText}</p>}
+                            {interimStt && <p className="interim-text">{interimStt}</p>}
+                            {!sttText && !interimStt && (
+                                <p className="empty-msg">Speak into your mic to start...</p>
+                            )}
+                        </div>
+                        {isMicActuallyListening && (
+                            <div className="mic-muted-status listening">
+                                <div className="pulse-dot"></div> Listening...
+                            </div>
+                        )}
+                        {isRecording && isBusyRef.current && (
+                            <div className="mic-muted-status processing">
+                                <Volume2 size={14} /> AI is processing/speaking...
+                            </div>
+                        )}
+                    </div>
+                </div>
+
+                {/* Column 2: LLM */}
+                <div className="pipeline-col">
+                    <div className="col-header llm">
+                        <div className="title-with-model">
+                            <MessageSquare size={18} />
+                            <h3>2. LLM Processing</h3>
+                        </div>
+                        <span className="model-tag">GPT-4o-mini</span>
+                    </div>
+                    <div className="col-content">
+                        <div className="text-display llm-display">
+                            {isLlmLoading ? (
+                                <div className="loading-state">
+                                    <Loader2 className="spinner" size={32} />
+                                    <p>Processing...</p>
+                                </div>
+                            ) : llmResponse ? (
+                                <div className="response-box">
+                                    <p className="response-text">{llmResponse}</p>
+                                </div>
+                            ) : (
+                                <p className="empty-msg">Waiting for STT input...</p>
+                            )}
+                        </div>
+                    </div>
+                </div>
+
+                {/* Column 3: TTS */}
+                <div className="pipeline-col">
+                    <div className="col-header tts">
+                        <div className="title-with-model">
+                            <Volume2 size={18} />
+                            <h3>3. Text-to-Speech</h3>
+                        </div>
+                        <span className="model-tag">Web Speech API / Piper</span>
+                    </div>
+                    <div className="col-content">
+                        <div className="tts-status">
+                            <div className={`status-indicator ${ttsStatus.toLowerCase().replace('...', '')}`}>
+                                <Volume2 size={48} className={ttsStatus === 'Playing...' ? 'bouncing' : ''} />
+                                <p>{ttsStatus}</p>
+                            </div>
+                            {llmResponse && !isLlmLoading && (
+                                <button className="replay-btn" onClick={() => speakText(llmResponse)}>
+                                    <Volume2 size={16} /> Replay
+                                </button>
+                            )}
+                        </div>
+
+                        <div className="voice-selection-compact">
+                            <label>Voice Output</label>
+                            <select
+                                value={selectedVoiceURI}
+                                onChange={(e) => setSelectedVoiceURI(e.target.value)}
+                            >
+                                {voices.map(v => (
+                                    <option key={v.voiceURI || v.name} value={v.voiceURI || v.name}>
+                                        {v.name} ({v.lang})
+                                    </option>
+                                ))}
+                            </select>
+                        </div>
+
+                        <div className="auto-toggle">
+                            <label className="switch">
+                                <input
+                                    type="checkbox"
+                                    checked={autoMode}
+                                    onChange={(e) => {
+                                        const val = e.target.checked;
+                                        setAutoMode(val);
+                                        autoModeRef.current = val;
+                                    }}
+                                />
+                                <span className="slider round"></span>
+                            </label>
+                            <span>Auto-play TTS</span>
+                        </div>
+                    </div>
+                </div>
+            </div>
+
+            {error && <div className="pipeline-error">{error}</div>}
+
+            <div className="history-tray">
+                <h4>Recent Interactions</h4>
+                <div className="history-list">
+                    {history.length === 0 ? (
+                        <p className="no-history">No history yet</p>
+                    ) : (
+                        history.map((h, i) => (
+                            <div key={i} className={`history-item ${h.role}`}>
+                                <span className="h-role">{h.role === 'user' ? 'You' : 'AI'}:</span>
+                                <span className="h-text">{h.content}</span>
+                            </div>
+                        ))
+                    )}
+                </div>
+            </div>
+        </div>
+    );
+}
+
+export default SttLlmTts;

web_demo/src/components/TextToSpeech.css

@@ -0,0 +1,321 @@
+.text-to-speech {
+    display: flex;
+    flex-direction: column;
+    gap: 1.5rem;
+    padding: 0;
+    height: 100%;
+    width: 100%;
+}
+
+.tts-header {
+    text-align: center;
+}
+
+.tts-header h2 {
+    font-size: 1.5rem;
+    font-weight: 600;
+    margin-bottom: 0.5rem;
+    color: var(--text-primary);
+}
+
+/* Text Input Section */
+.text-input-section {
+    display: flex;
+    flex-direction: column;
+    gap: 0.5rem;
+}
+
+.input-header {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+}
+
+.input-header label {
+    font-weight: 500;
+    color: var(--text-primary);
+    font-size: 0.875rem;
+}
+
+.char-count {
+    font-size: 0.75rem;
+    color: var(--text-secondary);
+}
+
+.text-input {
+    width: 100%;
+    padding: 1rem;
+    background: rgba(255, 255, 255, 0.05);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 8px;
+    color: var(--text-primary);
+    font-size: 0.875rem;
+    font-family: inherit;
+    resize: vertical;
+    transition: all 0.2s ease;
+}
+
+.text-input:focus {
+    outline: none;
+    border-color: var(--accent-color);
+    background: rgba(255, 255, 255, 0.08);
+    box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.1);
+}
+
+.text-input::placeholder {
+    color: var(--text-secondary);
+}
+
+/* Sample Texts */
+.sample-texts {
+    background: rgba(255, 255, 255, 0.03);
+    border-radius: 8px;
+    padding: 1rem;
+}
+
+.sample-label {
+    font-size: 0.75rem;
+    color: var(--text-secondary);
+    margin-bottom: 0.5rem;
+    font-weight: 500;
+}
+
+.sample-buttons {
+    display: flex;
+    gap: 0.5rem;
+    flex-wrap: wrap;
+}
+
+.sample-btn {
+    padding: 0.5rem 0.75rem;
+    background: rgba(255, 255, 255, 0.1);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 6px;
+    color: var(--text-secondary);
+    font-size: 0.75rem;
+    cursor: pointer;
+    transition: all 0.2s ease;
+    font-family: inherit;
+}
+
+.sample-btn:hover {
+    background: rgba(255, 255, 255, 0.15);
+    color: var(--text-primary);
+    border-color: var(--accent-color);
+}
+
+/* Voice Selector */
+.voice-selector {
+    display: flex;
+    flex-direction: column;
+    gap: 0.5rem;
+}
+
+.voice-selector label {
+    display: flex;
+    align-items: center;
+    gap: 0.5rem;
+    font-weight: 500;
+    color: var(--text-primary);
+    font-size: 0.875rem;
+}
+
+.voice-select {
+    width: 100%;
+    padding: 0.75rem;
+    background: rgba(255, 255, 255, 0.05);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 8px;
+    color: var(--text-primary);
+    font-size: 0.875rem;
+    cursor: pointer;
+    font-family: inherit;
+    transition: all 0.2s ease;
+}
+
+.voice-select:focus {
+    outline: none;
+    border-color: var(--accent-color);
+    box-shadow: 0 0 0 3px rgba(59, 130, 246, 0.1);
+}
+
+.voice-select option {
+    background: var(--card-bg);
+    color: var(--text-primary);
+}
+
+/* Settings Section */
+.settings-section {
+    background: rgba(255, 255, 255, 0.03);
+    border-radius: 8px;
+    overflow: hidden;
+}
+
+.settings-toggle {
+    width: 100%;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    gap: 0.5rem;
+    padding: 0.75rem;
+    background: transparent;
+    border: none;
+    color: var(--text-secondary);
+    font-size: 0.875rem;
+    cursor: pointer;
+    transition: all 0.2s ease;
+    font-family: inherit;
+}
+
+.settings-toggle:hover {
+    background: rgba(255, 255, 255, 0.05);
+    color: var(--text-primary);
+}
+
+.settings-panel {
+    padding: 1rem;
+    display: flex;
+    flex-direction: column;
+    gap: 1.5rem;
+    border-top: 1px solid rgba(255, 255, 255, 0.1);
+}
+
+.setting-control {
+    display: flex;
+    flex-direction: column;
+    gap: 0.5rem;
+}
+
+.setting-control label {
+    font-size: 0.875rem;
+    font-weight: 500;
+    color: var(--text-primary);
+}
+
+.slider {
+    width: 100%;
+    height: 6px;
+    border-radius: 3px;
+    background: rgba(255, 255, 255, 0.1);
+    outline: none;
+    -webkit-appearance: none;
+    appearance: none;
+}
+
+.slider::-webkit-slider-thumb {
+    -webkit-appearance: none;
+    appearance: none;
+    width: 18px;
+    height: 18px;
+    border-radius: 50%;
+    background: var(--accent-color);
+    cursor: pointer;
+    transition: all 0.2s ease;
+}
+
+.slider::-webkit-slider-thumb:hover {
+    background: #2563eb;
+    transform: scale(1.1);
+}
+
+.slider::-moz-range-thumb {
+    width: 18px;
+    height: 18px;
+    border-radius: 50%;
+    background: var(--accent-color);
+    cursor: pointer;
+    border: none;
+    transition: all 0.2s ease;
+}
+
+.slider::-moz-range-thumb:hover {
+    background: #2563eb;
+    transform: scale(1.1);
+}
+
+.slider-labels {
+    display: flex;
+    justify-content: space-between;
+    font-size: 0.7rem;
+    color: var(--text-secondary);
+}
+
+.reset-btn {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    gap: 0.5rem;
+    padding: 0.5rem 1rem;
+    background: rgba(255, 255, 255, 0.1);
+    border: 1px solid rgba(255, 255, 255, 0.1);
+    border-radius: 6px;
+    color: var(--text-secondary);
+    font-size: 0.8125rem;
+    cursor: pointer;
+    transition: all 0.2s ease;
+    font-family: inherit;
+    align-self: flex-start;
+}
+
+.reset-btn:hover {
+    background: rgba(255, 255, 255, 0.15);
+    color: var(--text-primary);
+    border-color: var(--accent-color);
+}
+
+/* Speaking Indicator */
+.speaking-indicator {
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    gap: 1rem;
+    padding: 1.5rem;
+    background: rgba(59, 130, 246, 0.1);
+    border: 1px solid rgba(59, 130, 246, 0.3);
+    border-radius: 8px;
+}
+
+.sound-wave {
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    gap: 4px;
+    height: 40px;
+}
+
+.wave-bar {
+    width: 4px;
+    height: 10px;
+    background: var(--accent-color);
+    border-radius: 2px;
+    animation: wave 1s ease-in-out infinite;
+}
+
+@keyframes wave {
+
+    0%,
+    100% {
+        height: 10px;
+    }
+
+    50% {
+        height: 30px;
+    }
+}
+
+.speaking-indicator span {
+    color: var(--accent-color);
+    font-weight: 500;
+    font-size: 0.875rem;
+}
+
+/* Responsive */
+@media (max-width: 640px) {
+    .sample-buttons {
+        flex-direction: column;
+    }
+
+    .sample-btn {
+        width: 100%;
+    }
+}

web_demo/src/components/TextToSpeech.jsx

@@ -0,0 +1,327 @@
+import React, { useState, useEffect, useRef } from 'react';
+import { Volume2, VolumeX, Play, Pause, RotateCcw, Settings } from 'lucide-react';
+import './TextToSpeech.css';
+
+function TextToSpeech() {
+    const [text, setText] = useState('');
+    const [isSpeaking, setIsSpeaking] = useState(false);
+    const [isPaused, setIsPaused] = useState(false);
+    const [voices, setVoices] = useState([]);
+    const [selectedVoice, setSelectedVoice] = useState(null);
+    const [rate, setRate] = useState(1);
+    const [pitch, setPitch] = useState(1);
+    const [volume, setVolume] = useState(1);
+    const [error, setError] = useState('');
+    const [showSettings, setShowSettings] = useState(false);
+
+    const synthRef = useRef(window.speechSynthesis);
+
+    // Load available voices
+    useEffect(() => {
+        const loadVoices = () => {
+            const availableVoices = synthRef.current.getVoices();
+            setVoices(availableVoices);
+
+            // Prioritize Indian English voices (en-IN)
+            const indianVoice = availableVoices.find(voice =>
+                voice.lang === 'en-IN' ||
+                voice.lang === 'en_IN' ||
+                voice.name.toLowerCase().includes('india')
+            );
+
+            const defaultVoice = indianVoice ||
+                availableVoices.find(voice => voice.lang.startsWith('en')) ||
+                availableVoices[0];
+
+            setSelectedVoice(defaultVoice);
+        };
+
+        loadVoices();
+
+        // Chrome loads voices asynchronously
+        if (synthRef.current.onvoiceschanged !== undefined) {
+            synthRef.current.onvoiceschanged = loadVoices;
+        }
+
+        return () => {
+            synthRef.current.cancel();
+        };
+    }, []);
+
+    const speak = () => {
+        if (!text.trim()) {
+            setError('Please enter some text to speak');
+            return;
+        }
+
+        setError('');
+        synthRef.current.cancel(); // Cancel any ongoing speech
+
+        const utterance = new SpeechSynthesisUtterance(text);
+
+        if (selectedVoice) {
+            utterance.voice = selectedVoice;
+        }
+
+        utterance.rate = rate;
+        utterance.pitch = pitch;
+        utterance.volume = volume;
+
+        utterance.onstart = () => {
+            setIsSpeaking(true);
+            setIsPaused(false);
+        };
+
+        utterance.onend = () => {
+            setIsSpeaking(false);
+            setIsPaused(false);
+        };
+
+        utterance.onerror = (event) => {
+            console.error('Speech synthesis error:', event);
+            setError(`Error: ${event.error}`);
+            setIsSpeaking(false);
+            setIsPaused(false);
+        };
+
+        synthRef.current.speak(utterance);
+    };
+
+    const pause = () => {
+        if (synthRef.current.speaking && !synthRef.current.paused) {
+            synthRef.current.pause();
+            setIsPaused(true);
+        }
+    };
+
+    const resume = () => {
+        if (synthRef.current.paused) {
+            synthRef.current.resume();
+            setIsPaused(false);
+        }
+    };
+
+    const stop = () => {
+        synthRef.current.cancel();
+        setIsSpeaking(false);
+        setIsPaused(false);
+    };
+
+    const reset = () => {
+        setRate(1);
+        setPitch(1);
+        setVolume(1);
+    };
+
+    const sampleTexts = [
+        "Namaste! This is an Indian English voice test. How can I help you today?",
+        "Hello! This is a test of the text-to-speech system.",
+        "The quick brown fox jumps over the lazy dog.",
+        "Welcome to NeuralVoice AI. We're testing the speech synthesis capabilities of your browser.",
+    ];
+
+    const loadSampleText = (sample) => {
+        setText(sample);
+        setError('');
+    };
+
+    return (
+        <div className="text-to-speech">
+            <div className="tts-header">
+                <h2>Text-to-Speech Test</h2>
+                <p className="subtitle">Enter text and hear it spoken aloud</p>
+            </div>
+
+            {error && (
+                <div className="error-message">
+                    <span>⚠️ {error}</span>
+                </div>
+            )}
+
+            <div className="text-input-section">
+                <div className="input-header">
+                    <label htmlFor="text-input">Enter Text</label>
+                    <span className="char-count">{text.length} characters</span>
+                </div>
+                <textarea
+                    id="text-input"
+                    className="text-input"
+                    value={text}
+                    onChange={(e) => setText(e.target.value)}
+                    placeholder="Type or paste text here to convert to speech..."
+                    rows={6}
+                />
+            </div>
+
+            <div className="sample-texts">
+                <p className="sample-label">Quick samples:</p>
+                <div className="sample-buttons">
+                    {sampleTexts.map((sample, index) => (
+                        <button
+                            key={index}
+                            className="sample-btn"
+                            onClick={() => loadSampleText(sample)}
+                        >
+                            Sample {index + 1}
+                        </button>
+                    ))}
+                </div>
+            </div>
+
+            <div className="voice-selector">
+                <label htmlFor="voice-select">
+                    <Settings size={16} />
+                    Voice
+                </label>
+                <select
+                    id="voice-select"
+                    value={selectedVoice?.name || ''}
+                    onChange={(e) => {
+                        const voice = voices.find(v => v.name === e.target.value);
+                        setSelectedVoice(voice);
+                    }}
+                    className="voice-select"
+                >
+                    {voices.map((voice) => (
+                        <option key={voice.name} value={voice.name}>
+                            {voice.name} ({voice.lang})
+                        </option>
+                    ))}
+                </select>
+            </div>
+
+            <div className="settings-section">
+                <button
+                    className="settings-toggle"
+                    onClick={() => setShowSettings(!showSettings)}
+                >
+                    <Settings size={18} />
+                    <span>{showSettings ? 'Hide' : 'Show'} Advanced Settings</span>
+                </button>
+
+                {showSettings && (
+                    <div className="settings-panel">
+                        <div className="setting-control">
+                            <label>
+                                Speed: {rate.toFixed(1)}x
+                            </label>
+                            <input
+                                type="range"
+                                min="0.5"
+                                max="2"
+                                step="0.1"
+                                value={rate}
+                                onChange={(e) => setRate(parseFloat(e.target.value))}
+                                className="slider"
+                            />
+                            <div className="slider-labels">
+                                <span>Slow</span>
+                                <span>Normal</span>
+                                <span>Fast</span>
+                            </div>
+                        </div>
+
+                        <div className="setting-control">
+                            <label>
+                                Pitch: {pitch.toFixed(1)}
+                            </label>
+                            <input
+                                type="range"
+                                min="0.5"
+                                max="2"
+                                step="0.1"
+                                value={pitch}
+                                onChange={(e) => setPitch(parseFloat(e.target.value))}
+                                className="slider"
+                            />
+                            <div className="slider-labels">
+                                <span>Low</span>
+                                <span>Normal</span>
+                                <span>High</span>
+                            </div>
+                        </div>
+
+                        <div className="setting-control">
+                            <label>
+                                Volume: {Math.round(volume * 100)}%
+                            </label>
+                            <input
+                                type="range"
+                                min="0"
+                                max="1"
+                                step="0.1"
+                                value={volume}
+                                onChange={(e) => setVolume(parseFloat(e.target.value))}
+                                className="slider"
+                            />
+                            <div className="slider-labels">
+                                <span>Quiet</span>
+                                <span>Normal</span>
+                                <span>Loud</span>
+                            </div>
+                        </div>
+
+                        <button className="reset-btn" onClick={reset}>
+                            <RotateCcw size={16} />
+                            Reset to Defaults
+                        </button>
+                    </div>
+                )}
+            </div>
+
+            <div className="controls">
+                {!isSpeaking ? (
+                    <button className="btn btn-primary" onClick={speak}>
+                        <Play size={20} />
+                        <span>Speak</span>
+                    </button>
+                ) : (
+                    <>
+                        {!isPaused ? (
+                            <button className="btn btn-warning" onClick={pause}>
+                                <Pause size={20} />
+                                <span>Pause</span>
+                            </button>
+                        ) : (
+                            <button className="btn btn-success" onClick={resume}>
+                                <Play size={20} />
+                                <span>Resume</span>
+                            </button>
+                        )}
+                        <button className="btn btn-danger" onClick={stop}>
+                            <VolumeX size={20} />
+                            <span>Stop</span>
+                        </button>
+                    </>
+                )}
+            </div>
+
+            {isSpeaking && (
+                <div className="speaking-indicator">
+                    <div className="sound-wave">
+                        {[...Array(5)].map((_, i) => (
+                            <div
+                                key={i}
+                                className="wave-bar"
+                                style={{ animationDelay: `${i * 0.1}s` }}
+                            />
+                        ))}
+                    </div>
+                    <span>{isPaused ? 'Paused' : 'Speaking...'}</span>
+                </div>
+            )}
+
+            <div className="info-box">
+                <h4>💡 Tips:</h4>
+                <ul>
+                    <li>Choose different voices to hear various accents and styles</li>
+                    <li>Adjust speed, pitch, and volume for customized speech</li>
+                    <li>Works in all modern browsers (Chrome, Firefox, Safari, Edge)</li>
+                    <li>Try longer texts to test natural speech flow</li>
+                </ul>
+            </div>
+        </div>
+    );
+}
+
+export default TextToSpeech;

web_demo/src/index.css

@@ -0,0 +1,47 @@
+:root {
+  --bg-primary: #0f172a;
+  --bg-secondary: #1e293b;
+  --text-primary: #f8fafc;
+  --text-secondary: #94a3b8;
+  --accent-primary: #3b82f6;
+  --accent-hover: #2563eb;
+  --accent-glow: rgba(59, 130, 246, 0.5);
+  --success: #10b981;
+  --error: #ef4444;
+  --font-sans: 'Inter', system-ui, -apple-system, sans-serif;
+}
+
+body {
+  margin: 0;
+  padding: 0;
+  background-color: var(--bg-primary);
+  color: var(--text-primary);
+  font-family: var(--font-sans);
+  -webkit-font-smoothing: antialiased;
+  min-height: 100vh;
+}
+
+#root {
+  width: 100%;
+  min-height: 100vh;
+  display: flex;
+  flex-direction: column;
+}
+
+/* Scrollbar */
+::-webkit-scrollbar {
+  width: 8px;
+}
+
+::-webkit-scrollbar-track {
+  background: var(--bg-primary);
+}
+
+::-webkit-scrollbar-thumb {
+  background: var(--bg-secondary);
+  border-radius: 4px;
+}
+
+::-webkit-scrollbar-thumb:hover {
+  background: #334155;
+}

web_demo/src/main.jsx

@@ -0,0 +1,10 @@
+import { StrictMode } from 'react'
+import { createRoot } from 'react-dom/client'
+import './index.css'
+import App from './App.jsx'
+
+createRoot(document.getElementById('root')).render(
+  <StrictMode>
+    <App />
+  </StrictMode>,
+)

web_demo/vite.config.js

@@ -0,0 +1,7 @@
+import { defineConfig } from 'vite'
+import react from '@vitejs/plugin-react'
+
+// https://vite.dev/config/
+export default defineConfig({
+  plugins: [react()],
+})