README.md

---
title: Qwen ASR VLM
emoji: 😻
colorFrom: yellow
colorTo: blue
sdk: docker
pinned: false
---

# Qwen3-ASR vLLM Streaming Server

A FastAPI server that wraps **Qwen3-ASR-1.7B** (via `qwen-asr[vllm]`) and exposes a
DashScope- / OpenAI-realtime-compatible WebSocket API with server-side VAD, plus a
Gradio UI for testing it. This repo ships the **Chughtai Lab variant**
(`app_chughtai.py`), which biases the model toward medical-lab speech and leaves the
language unforced for natural Urdu/Hindi/English code-switching.

## Requirements

- A CUDA GPU (the model runs on vLLM; ~3.5 GB weights download on first run)
- Python **3.11+** (pinned in `.python-version`)
- [**uv**](https://docs.astral.sh/uv/) for dependency and environment management

## 1. Set up the environment with `uv`

Install `uv` if you don't have it:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```

Create the virtual environment and install all dependencies from `pyproject.toml` /
`uv.lock` (uv reads `.python-version` and provisions Python 3.11 automatically):

```bash
uv sync
```

This creates a `.venv/` in the project. You don't need to activate it — `uv run`
uses it automatically. (To activate manually anyway: `source .venv/bin/activate`.)

## 2. Configure your `.env`

Copy the template and edit the values for your GPU and use case:

```bash
cp .env.example .env
```

Key settings (see `.env.example` for the full annotated list):

| Variable | Purpose | Default |
|----------|---------|---------|
| `MODEL_ID` | HuggingFace model to load | `Qwen/Qwen3-ASR-1.7B` |
| `GPU_MEMORY_UTILIZATION` | Fraction of VRAM vLLM may reserve (lower if sharing the GPU) | `0.80` |
| `MAX_MODEL_LEN` | Context-window cap (biggest VRAM saver; `0` = auto) | `4096` |
| `PORT` | Server port | `7860` |
| `LANGUAGE` | Default language when the client doesn't specify | `English` |
| `VAD_THRESHOLD` | Speech-detection strictness | `0.7` |
| `HF_HOME` / `TORCH_HOME` | Persistent model-cache dirs (avoid re-downloading weights) | — |

## 3. Run the server

The ASR server is **`app_chughtai.py`**. It serves on `0.0.0.0:7860`:

```bash
uv run python app_chughtai.py
```

Check it's alive:

```bash
curl http://localhost:7860/health
```

Endpoints:
- `POST /v1/audio/transcriptions` — batch file upload
- `GET/WS /v1/realtime` — streaming WebSocket (DashScope/OpenAI-compatible)

To run two servers side-by-side on one GPU, give the second a different port and a
smaller VRAM share:

```bash
PORT=7861 GPU_MEMORY_UTILIZATION=0.40 uv run python app_chughtai.py
```

## 4. Run the UI

The Gradio UI is **`ui_chughtai.py`**. It serves on `0.0.0.0:7870` and talks to the
server over the `/v1/realtime` WebSocket:

```bash
uv run python ui_chughtai.py
```

Then open <http://localhost:7870>. By default the UI connects to the server at
`http://localhost:7860`. Point it elsewhere with `CHUGHTAI_SERVER_URL`:

```bash
CHUGHTAI_SERVER_URL=http://192.168.99.117:7860 uv run python ui_chughtai.py
```

Change the UI port with `UI_PORT` if `7870` is taken.

## Typical workflow

```bash
# terminal 1 — start the ASR server
uv run python app_chughtai.py

# terminal 2 — start the UI (once the server logs "model ready")
uv run python ui_chughtai.py
```

Open <http://localhost:7870>, record or upload audio, and watch transcripts stream in.

---

This project is also deployed as a HuggingFace Space using the Docker SDK — see the
frontmatter above and the [Dockerfile](Dockerfile). Configuration reference:
<https://huggingface.co/docs/hub/spaces-config-reference>