README.md

metadata

title: Qwen ASR VLM
emoji: 😻
colorFrom: yellow
colorTo: blue
sdk: docker
pinned: false

Qwen3-ASR vLLM Streaming Server

A FastAPI server that wraps Qwen3-ASR-1.7B (via qwen-asr[vllm]) and exposes a DashScope- / OpenAI-realtime-compatible WebSocket API with server-side VAD, plus a Gradio UI for testing it. This repo ships the Chughtai Lab variant (app_chughtai.py), which biases the model toward medical-lab speech and leaves the language unforced for natural Urdu/Hindi/English code-switching.

Requirements

• A CUDA GPU (the model runs on vLLM; ~3.5 GB weights download on first run)
• Python 3.11+ (pinned in .python-version)
• uv for dependency and environment management

1. Set up the environment with uv

Install uv if you don't have it:

curl -LsSf https://astral.sh/uv/install.sh | sh

Create the virtual environment and install all dependencies from pyproject.toml / uv.lock (uv reads .python-version and provisions Python 3.11 automatically):

uv sync

This creates a .venv/ in the project. You don't need to activate it — uv run uses it automatically. (To activate manually anyway: source .venv/bin/activate.)

2. Configure your .env

Copy the template and edit the values for your GPU and use case:

cp .env.example .env

Key settings (see .env.example for the full annotated list):

Variable	Purpose	Default
MODEL_ID	HuggingFace model to load	Qwen/Qwen3-ASR-1.7B
GPU_MEMORY_UTILIZATION	Fraction of VRAM vLLM may reserve (lower if sharing the GPU)	0.80
MAX_MODEL_LEN	Context-window cap (biggest VRAM saver; 0 = auto)	4096
PORT	Server port	7860
LANGUAGE	Default language when the client doesn't specify	English
VAD_THRESHOLD	Speech-detection strictness	0.7
HF_HOME / TORCH_HOME	Persistent model-cache dirs (avoid re-downloading weights)	—

3. Run the server

The ASR server is app_chughtai.py. It serves on 0.0.0.0:7860:

uv run python app_chughtai.py

Check it's alive:

curl http://localhost:7860/health

Endpoints:

• POST /v1/audio/transcriptions — batch file upload
• GET/WS /v1/realtime — streaming WebSocket (DashScope/OpenAI-compatible)

To run two servers side-by-side on one GPU, give the second a different port and a smaller VRAM share:

PORT=7861 GPU_MEMORY_UTILIZATION=0.40 uv run python app_chughtai.py

4. Run the UI

The Gradio UI is ui_chughtai.py. It serves on 0.0.0.0:7870 and talks to the server over the /v1/realtime WebSocket:

uv run python ui_chughtai.py

Then open http://localhost:7870. By default the UI connects to the server at http://localhost:7860. Point it elsewhere with CHUGHTAI_SERVER_URL:

CHUGHTAI_SERVER_URL=http://192.168.99.117:7860 uv run python ui_chughtai.py

Change the UI port with UI_PORT if 7870 is taken.

Typical workflow

# terminal 1 — start the ASR server
uv run python app_chughtai.py

# terminal 2 — start the UI (once the server logs "model ready")
uv run python ui_chughtai.py

Open http://localhost:7870, record or upload audio, and watch transcripts stream in.

---

This project is also deployed as a HuggingFace Space using the Docker SDK — see the frontmatter above and the Dockerfile. Configuration reference: https://huggingface.co/docs/hub/spaces-config-reference