---
title: Voice AI Demo
emoji: 🎙️
colorFrom: indigo
colorTo: blue
sdk: docker
app_port: 8765
pinned: false
---

## Voice AI Demo — Real-Time Speech Assistant

A real-time voice AI assistant built on Alibaba Cloud Model Studio (Bailian). It supports the full pipeline of speech input, LLM reasoning, and speech playback. All three services (ASR, LLM, TTS) are freely configurable via the web UI or config file, making it easy to adapt to different customer deployments.

---

### Features

- **Real-time voice interaction** — Tap the mic to record. The system automatically runs speech recognition, LLM inference, and text-to-speech in sequence.
- **Text input** — Type a message directly and chat with the LLM.
- **Multi-language switching** — Ships with English and Yoruba presets. Add more languages in `config.json`.
- **Flexible configuration** — ASR, LLM, and TTS endpoints, API keys, and models can all be changed from the Settings panel or config file.
- **DashScope auto-detection** — When a Bailian MaaS URL is detected, the app automatically switches to the DashScope native API format.
- **OpenAI-compatible** — Also works with any OpenAI-compatible ASR / LLM / TTS endpoint.

### Architecture

```
User Speech → ASR (qwen3-asr-flash) → Text → LLM (qwen-plus) → Reply → TTS (qwen3-tts-flash) → Audio Playback
```

| Service | Bailian Model | Notes |
|---------|--------------|-------|
| ASR | qwen3-asr-flash | Speech recognition with auto language detection |
| LLM | qwen-plus | Large language model with streaming output |
| TTS | qwen3-tts-flash | Text-to-speech with multiple voice options |

---

### Quick Start

**Prerequisites**

- Python 3.10+
- A modern browser (Chrome, Edge, Safari, or Firefox)

**Step 1: Install dependencies**

```bash
cd voice-ai-demo
python3 -m venv venv
source venv/bin/activate        # On Windows: venv\Scripts\activate
pip install -r requirements.txt
```

**Step 2: Configure your model services**

Edit `config.json` and fill in your Bailian MaaS instance details:

```json
{
  "asr": {
    "mode": "api",
    "base_url": "https://<your-instance>.maas.aliyuncs.com/compatible-mode/v1",
    "api_key": "<your-api-key>",
    "model": "qwen3-asr-flash"
  },
  "llm": {
    "base_url": "https://<your-instance>.maas.aliyuncs.com/compatible-mode/v1",
    "api_key": "<your-api-key>",
    "model": "qwen-plus"
  },
  "tts": {
    "base_url": "https://<your-instance>.maas.aliyuncs.com/compatible-mode/v1",
    "api_key": "<your-api-key>",
    "model": "qwen3-tts-flash",
    "voice": "Cherry"
  }
}
```

> **Tip:** ASR, LLM, and TTS can each point to a different MaaS instance and API key. Just fill in the corresponding fields independently.

**Step 3: Run**

```bash
python3 app.py
```

The server starts on `http://localhost:8765` by default. Open this URL in your browser to begin.

---

### Deploying to Hugging Face Spaces (Docker)

This app is a FastAPI server with its own frontend, so it runs on a Space using the **Docker SDK** (not the Gradio SDK). The included `Dockerfile` and the YAML front matter at the top of this README are all that's needed.

**Step 1: Create the Space**

Create a new Space, choose **Docker → Blank**, then push these files (or upload via the web UI):

```
app.py  config.json  requirements.txt  Dockerfile  README.md  .gitignore
```

The `sdk: docker` and `app_port: 8765` lines in this README's front matter tell the Space how to build and route to the app.

**Step 2: Add your keys as Secrets — do not commit them**

In **Settings → Variables and secrets**, add your credentials as *Secrets*. The app reads these at startup and they always override `config.json`, so real keys never touch the repo.

**Simplest setup — one Bailian/DashScope key for everything (two secrets total):**

| Secret | Value |
|--------|-------|
| `DASHSCOPE_API_KEY` | Your Model Studio key (`sk-...`) — applied to ASR + LLM + TTS |
| `DASHSCOPE_BASE_URL` | Your region's endpoint (see below) — applied to all three services |

Pick the base URL for your region:

- International / outside mainland China: `https://dashscope-intl.aliyuncs.com/compatible-mode/v1`
- Mainland China (Beijing): `https://dashscope.aliyuncs.com/compatible-mode/v1`

The app auto-detects these DashScope endpoints and uses the native multimodal format for ASR (`qwen3-asr-flash`) and TTS (`qwen3-tts-flash`), while the LLM (`qwen-plus`) uses the OpenAI-compatible chat endpoint.

**Per-service override (only if services differ):** set any of `ASR_BASE_URL` / `LLM_BASE_URL` / `TTS_BASE_URL` and `ASR_API_KEY` / `LLM_API_KEY` / `TTS_API_KEY` to point a single service somewhere else. A per-service value wins over the shared `DASHSCOPE_*` value. Optional model/voice overrides: `ASR_MODEL`, `LLM_MODEL`, `TTS_MODEL`, `TTS_VOICE`.

**Per-language ASR (speech-to-text):** a language can route its voice input to a different ASR engine — useful because Qwen's `qwen3-asr-flash` doesn't cover Yoruba, while Whisper does. Set `asr_base_url` / `asr_api_key` / `asr_model` on the language (or via env `ASR_<LANGID>_BASE_URL` etc.), and it uses that OpenAI-compatible `/audio/transcriptions` endpoint with the language's `asr_lang` hint. Yoruba ships pointed at a Whisper model (`whisper-large-v3`, hint `yo`); set `ASR_YORUBA_BASE_URL` to any OpenAI-compatible Whisper endpoint (e.g. `https://api.groq.com/openai/v1` or `https://api.openai.com/v1`) plus `ASR_YORUBA_API_KEY`. A language with its own `asr_base_url` never inherits the global Qwen ASR endpoint.

**Per-language TTS backend:** each language can use its own TTS engine, so e.g. English speaks via Qwen while Yoruba speaks via a self-hosted model. A language uses its own endpoint when it sets `tts_format` / `tts_base_url` / `tts_api_key` (in `config.json` or via env). Supported `tts_format` values: `dashscope`, `openai`, or `custom`. The `custom` format POSTs `{text, speed}` to the URL **verbatim** (so an API Gateway invoke URL like `.../prod/tts` is used exactly as given), sends both `Authorization: Bearer` and `x-api-key` when a key is set, and accepts either raw audio (`audio/*`) or JSON carrying base64 audio (key `audio` / `audio_base64` / `data` / `wav` / `audio_content`). Per-language env vars follow the pattern `TTS_<LANGID>_BASE_URL`, `TTS_<LANGID>_API_KEY`, `TTS_<LANGID>_FORMAT`, `TTS_<LANGID>_MODEL`, `TTS_<LANGID>_VOICE` — e.g. `TTS_YORUBA_BASE_URL`. A `custom`-format language never inherits the global Qwen URL/key, so a missing value fails safely instead of sending text to the wrong engine.

Leave the placeholder values in `config.json` as they are — they're scrubbed automatically at load, and the Secrets fill in the real values.

**Step 3: Open the Space**

Once the build finishes, the app is live. Microphone capture works out of the box because Spaces are served over HTTPS (browsers require HTTPS or localhost for mic access).

> **Note on the in-app Settings panel:** A Space's filesystem is ephemeral, so any keys you type into the Settings panel are session-only and reset on restart/rebuild. Secrets set via env vars are the durable source of truth and are re-applied on every reload. For persistence across restarts beyond env vars, attach Persistent Storage to the Space.

---

### Usage

**Voice chat** — Click the microphone button at the bottom to start recording. Click again to stop. The system handles speech recognition, LLM inference, and speech synthesis automatically.

**Text chat** — Type a message in the input box at the bottom, then press Enter or click the send button.

**Switch language** — Click the language buttons in the top bar. Chat history is cleared when switching languages.

**Change settings** — Click the gear icon in the top-right corner to open the Settings panel. You can modify the ASR / LLM / TTS endpoint, API key, model, and voice directly from the UI. Changes take effect immediately after clicking Save.

---

### Configuration Reference

Full field reference for `config.json`:

| Field | Description | Example |
|-------|-------------|---------|
| `asr.mode` | ASR mode: `api` (remote) or `local` (on-device Whisper) | `"api"` |
| `asr.base_url` | ASR service base URL | Bailian MaaS URL |
| `asr.api_key` | ASR API key | `sk-xxx` |
| `asr.model` | ASR model name | `"qwen3-asr-flash"` |
| `llm.base_url` | LLM service base URL | Bailian MaaS URL |
| `llm.api_key` | LLM API key | `sk-xxx` |
| `llm.model` | LLM model name | `"qwen-plus"` |
| `llm.max_tokens` | Max output tokens for LLM | `512` |
| `llm.temperature` | LLM sampling temperature | `0.7` |
| `tts.base_url` | TTS service base URL | Bailian MaaS URL |
| `tts.api_key` | TTS API key | `sk-xxx` |
| `tts.model` | TTS model name | `"qwen3-tts-flash"` |
| `tts.voice` | TTS voice name | `"Cherry"` |
| `tts.speed` | TTS playback speed multiplier | `1.0` |
| `languages` | Language presets (add or remove freely) | See file |
| `app.host` | Server listen address | `"0.0.0.0"` |
| `app.port` | Server listen port | `8765` |

**Adding languages** — Add a new entry to the `languages` array in `config.json`. Each language can have its own `asr_lang` (ASR language hint), `system_prompt` (LLM system prompt), and `tts_model` (TTS model override).

---

### DashScope Auto-Detection

When the base URL contains `maas.aliyuncs.com`, the app automatically switches to the DashScope native API format:

- **ASR** calls `/api/v1/services/aigc/multimodal-generation/generation` using the `qwen3-asr-flash` model, with audio transmitted as a base64 data URI.
- **TTS** calls the same endpoint using the `qwen3-tts-flash` model and automatically downloads the audio from the returned temporary URL.

For non-Bailian URLs, the app uses standard OpenAI-compatible endpoints (`/audio/transcriptions`, `/chat/completions`, `/audio/speech`).

---

### FAQ

**Q: The microphone button does not respond?**

Make sure your browser has granted microphone permission. In Chrome, check the lock icon next to the address bar. Microphone access also requires HTTPS or localhost.

**Q: Speech recognition returns empty results?**

This may be caused by excessive background noise or a recording that is too short. Ensure recordings are at least 1 second long and use the app in a relatively quiet environment.

**Q: No audio from TTS?**

Check that the `voice` field in `config.json` matches a voice supported by your TTS model. The Bailian `qwen3-tts-flash` model supports voices such as Cherry, Serena, and Ethan.

**Q: How do I switch models?**

Change the Model field directly in the Settings panel from the web UI, or edit the corresponding field in `config.json` and restart the server.

---

### Project Structure

```
voice-ai-demo/
├── app.py              # Main application (FastAPI backend + embedded HTML frontend)
├── config.json         # Configuration (endpoints, API keys, languages, etc.)
├── requirements.txt    # Python dependencies
├── Dockerfile          # Container build for Hugging Face Spaces (Docker SDK)
├── .gitignore          # Excludes venv, caches, local .env
└── README.md           # This document (front matter configures the Space)
```