docs/deploy_mvp.md

Urdu S2S MVP Deployment

This is the deployable MVP path for the current Urdu S2S service:

Audio input -> Faster Whisper ASR -> OpenAI Urdu reply -> Devanagari speech text -> Chatterbox Praxy voice output.

What Is Frozen

• ASR: faster-whisper
• Reply model: OPENAI_MODEL, default gpt-4o-mini
• Speech text: OpenAI-compatible chat client with Urdu-to-Devanagari prompt
• Local policy: Urdu/Hindi wording fixes, abbreviation expansion, gender guardrails
• TTS: ResembleAI/chatterbox:v3 with Praxy anchor bench_025.wav
• Known non-blocking note: bench_038 can sound like ya instead of ye

Required Runtime

Use a CUDA GPU supported by Chatterbox's pinned Torch stack. Good choices:

• RTX 3090 / 3090 Ti
• A10 / A10G
• A100
• L40 / L40S
• H100 / H200

Avoid RTX 50-series and Blackwell RTX PRO machines for this container unless the Torch stack is upgraded, because the current Chatterbox dependency set may not support sm_120.

Environment

Required:

OPENAI_API_KEY=...

Recommended production defaults:

OPENAI_MODEL=gpt-4o-mini
S2S_DEFAULT_MODE=live_tts
S2S_WHISPER_MODEL=small
S2S_WHISPER_DEVICE=cuda
S2S_WHISPER_COMPUTE_TYPE=float16
S2S_CHATTERBOX_DEVICE=cuda
S2S_PORT=8017

CPU/text-only local defaults:

S2S_DEFAULT_MODE=text_only
S2S_WHISPER_DEVICE=cpu
S2S_WHISPER_COMPUTE_TYPE=int8

Local API

python3.11 -m venv .venv-deploy
source .venv-deploy/bin/activate
python -m pip install --upgrade pip wheel
python -m pip install -r requirements-api.txt -r requirements-live-s2s.txt

export OPENAI_API_KEY="..."
export OPENAI_MODEL="gpt-4o-mini"
export S2S_DEFAULT_MODE=live_tts
export S2S_WHISPER_MODEL=small
export S2S_WHISPER_DEVICE=cuda
export S2S_WHISPER_COMPUTE_TYPE=float16
export S2S_CHATTERBOX_DEVICE=cuda

PYTHONPATH=src:scripts python scripts/serve_s2s_api.py --host 0.0.0.0 --port 8017

Open:

http://127.0.0.1:8017/

Health:

curl http://127.0.0.1:8017/health

Live request:

curl -X POST "http://127.0.0.1:8017/s2s?mode=live_tts&request_id=demo_001" \
  -F "audio=@data/processed/benchmarks/gemini_urdu_s2s_v1/audio_canonical/bench_001.wav"

The response includes tts_audio_url; fetch it from the same host.

Docker

Build:

docker build -t urdu-s2s-mvp:latest .

Run on a GPU host:

docker run --gpus all --rm -p 8017:7860 \
  -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  -e OPENAI_MODEL="gpt-4o-mini" \
  urdu-s2s-mvp:latest

Then open:

http://127.0.0.1:8017/

Hugging Face Space

Use a Docker Space with GPU hardware.

Billing prerequisite:

• Personal namespace: Hugging Face Pro is required for Docker Spaces.
• Organization namespace: Team/Enterprise is required for Docker Spaces.
• GPU hardware also requires prepaid credits or billing enabled.

1. Create a new Space.
2. Select SDK: Docker.
3. Push this repo or the deployment bundle.
4. Add secret OPENAI_API_KEY.
5. Set GPU hardware.
6. The container listens on port 7860, which matches Hugging Face's Docker Space default.

Vast / RunPod

Use a non-Blackwell GPU, preferably RTX 3090/A10/A100/L40S/H100.

Fastest path:

tar -xzf artifacts/live_s2s_remote_bundle.tar.gz
cd live_s2s_remote_bundle
python -m pip install --upgrade pip wheel
python -m pip install -r requirements-api.txt -r requirements-live-s2s.txt
python -m pip install torchvision==0.21.0

export OPENAI_API_KEY="..."
export S2S_DEFAULT_MODE=live_tts
export S2S_WHISPER_MODEL=small
export S2S_WHISPER_DEVICE=cuda
export S2S_WHISPER_COMPUTE_TYPE=float16
export S2S_CHATTERBOX_DEVICE=cuda

PYTHONPATH=src:scripts python scripts/serve_s2s_api.py --host 0.0.0.0 --port 8017

Next Product Work

• Add auth/rate limits before public launch.
• Add request logging without storing secrets.
• Add a simple queue if concurrent TTS requests pile up.
• Later: run bigger regression batches and fix pronunciation notes.