CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

A self-hosted FastAPI wrapper around the `k2-fsa/OmniVoice` TTS model, packaged as a HuggingFace Space (Docker SDK, T4 GPU). The model is loaded **in-process at startup** — there is no proxying to the public Gradio Space. The entire service is one file: [app.py](app.py).

Primary use case is Urdu / Punjabi synthesis (plus 600+ languages OmniVoice supports), consumed by a downstream voice-agent client that retries 502/503/504 with backoff to absorb cold starts.

## Commands

```bash
# Run locally (downloads ~2 GB model to HF_HOME on first request)
pip install -r requirements.txt          # torch is NOT here — see below
uvicorn app:app --host 0.0.0.0 --port 7860

# torch must be installed separately to match upstream's CUDA build:
pip install torch==2.8.0 torchaudio==2.8.0 --extra-index-url https://download.pytorch.org/whl/cu128

# Build the Space image
docker build -t omnivoice-tts .

# Smoke-test endpoints
curl localhost:7860/health
curl -X POST localhost:7860/tts -H 'Content-Type: application/json' \
     -d '{"text":"آپ کا شکریہ","language":"Urdu","gender":"Female"}' --output out.wav
```

There is no test suite, linter config, or build step beyond the Docker image.

## Architecture

Everything lives in [app.py](app.py). Request flow:

1. **Model load (module top-level)** — `OmniVoice.from_pretrained()` runs once at import time. `SAMPLE_RATE` is read from the loaded model. This means *any* import of `app.py` triggers the full model download/load; there is no lazy init.
2. **`TTSRequest` (pydantic)** — the shared request schema for `/tts`, `/tts/stream`, and `/ws/tts`. `/tts/clone` uses multipart form fields instead and constructs a `TTSRequest` internally.
3. **Voice-design attribute layer** — `gender`/`age`/`pitch`/`style`/`accent`/`dialect` are normalized then composed into a single OmniVoice `instruct` string:
   - `_normalize_label()` maps friendly aliases (`"female"`, `"young"`, `"low"`) → canonical bilingual labels (`"Female / 女"`).
   - `_attr_part()` then picks the **English half** of each label — *except dialects, where it picks the Chinese half* (`is_dialect=True`). This mirrors the upstream `omnivoice/cli/demo.py` convention; preserve it.
   - A caller-supplied `instruct` string bypasses this whole layer (raw override).
4. **`_synthesize()`** — the single inference path for all endpoints. Returns int16 PCM (1-D ndarray). Voice cloning is the same path with `voice_clone_prompt` added: either a prebuilt prompt passed in (registry reuse) or one built inline via `model.create_voice_clone_prompt()` from a `ref_audio_path` (ad-hoc `/tts/clone`).
   - **Voice registry (clone once, reuse many)**: `POST /voices` runs `create_voice_clone_prompt()` a single time and caches it in `_VOICE_CACHE` (a thread-safe `OrderedDict` LRU, lock = `_VOICE_LOCK`, cap = `OMNIVOICE_MAX_VOICES`, default 50) under a client-chosen name. Any `TTSRequest` with `voice_id` set then reuses the cached prompt across `/tts`, `/tts/stream`, and `/ws/tts` — skipping all reference re-encoding. Unknown `voice_id` → 404. The cache is in-process/in-memory only (lost on restart). `_resolve_voice()` does the lookup-or-404 for HTTP endpoints; the WebSocket handler checks inline and replies with an error frame instead.
5. **WAV framing** — `_wav_header()` builds a 44-byte int16-mono header; `/tts` wraps the full buffer, `/tts/stream` and `/ws/tts` emit the header then chunk raw PCM (~0.2 s HTTP frames, ~0.5 s WS frames).

Inference is always wrapped in `asyncio.to_thread(_synthesize, ...)` so the event loop stays responsive — keep new synthesis calls off the main thread the same way.

## Conventions and gotchas

- **The attribute vocabulary sets** (`GENDERS`, `AGES`, `PITCHES`, `STYLES`, `ACCENTS`, `DIALECTS`) must stay in sync with the upstream Gradio demo's `"English / 中文"` labels. The `/` (GET root) descriptor advertises these to clients, so changes here are part of the public API.
- **Dependency pins are load-bearing.** `transformers==5.3` is required — older versions break the `higgs_audio_v2_tokenizer` that `omnivoice` imports. `torch==2.8.0` matches the upstream Space. The Dockerfile installs torch *before* `requirements.txt` so other deps don't drag in a different CUDA variant. Don't unpin these casually.
- **Config is env-var driven** (read at module load): `OMNIVOICE_MODEL`, `OMNIVOICE_DEVICE` (auto CPU fallback), `OMNIVOICE_LOAD_ASR` (`0` skips ~1 GB whisper download — but then cloning/registration **must** supply `ref_text`), `OMNIVOICE_MAX_VOICES` (registry LRU cap, default 50), `LOG_LEVEL`, `PORT` (7860, the HF Spaces standard).
- **HF Spaces persistence**: caches go to `/data/.cache/...` (set via `HF_HOME`/`TORCH_HOME` in the Dockerfile); `/data` is the writeable persistent mount.
- **Generation params** carry validated bounds: `nfe_steps` 4–64, `guidance` 0–4. Higher `nfe_steps` = better quality, slower. Lowering `nfe_steps` (e.g. 16) is the main lever for faster responses / better RTF.
- **`/tts/stream` and `/ws/tts` are pseudo-streaming.** The upstream `omnivoice` library has no incremental decoding — `generate()` returns the *complete* audio. Both endpoints call `_synthesize` fully (blocking until the whole utterance is generated) and *then* chunk the finished PCM buffer. So they reduce transmission framing latency but **do not** lower time-to-first-audio or improve RTF. Don't market them as real-time streaming.

## Deploying

This repo *is* the Space. Push to the HF Space remote (`huggingface.co/spaces/ebitlogix/omnivoice-tts`) and set hardware to `t4-small`+ in Settings → Hardware. README front-matter (`sdk: docker`, `suggested_hardware: t4-small`) configures the Space.