README.md

# PhoWhisper Runtime

Production runtime for PhoWhisper ASR API. This package is designed to be
portable: copy the prepared `phowhisper/` directory to another Linux host,
install Python dependencies, create `.env`, validate `models/`, then run
`server_api.py`. No training repository is required on the target host.

## Repository Contents

Runtime package contents:

- API/runtime code
- Post-processing rules
- Runtime configuration templates
- Model tokenizer/config metadata and prepared checkpoint directories under `models/`
- Optional ChunkFormer reference ASR package under `vendor/chunkformer/`
- Optional ChunkFormer launcher and checkpoint under `scripts/chunkformer.py` and `models/chunkformer-model/`
- Lightweight validation scripts

Not included in a clean git checkout unless restored through Git LFS or an
artifact bundle:

- Primary PhoWhisper and punctuation model weight files
- Optional base-model weight files

Not part of the portable runtime package:

- Environment secrets
- Uploaded audio
- Runtime outputs and logs

If you copy the whole prepared directory from a working machine, model weights
under `models/` are carried with it. If you clone from git, run `git lfs pull`
or restore the model artifact bundle before starting the API.

## 1. System Requirements

Minimum supported environment:

| Component | Requirement |
| --- | --- |
| OS | Ubuntu 22.04/24.04 LTS or compatible Linux x86_64 |
| Python | 3.12 |
| Audio tooling | `ffmpeg` |
| RAM | 16GB minimum, 32GB recommended |
| Disk | 15GB minimum for model artifacts and runtime cache |
| GPU | NVIDIA GPU recommended for production throughput |

CPU execution is supported for validation and low-volume workloads. Production traffic should use a CUDA-capable GPU with a PyTorch build matching the host driver.

Install host packages:

```bash
sudo apt-get update
sudo apt-get install -y git ffmpeg python3.12 python3.12-venv python3-pip
```

## 2. Deployment Layout

Recommended installation path:

```text
/opt/phowhisper
```

Create the deployment directory:

```bash
sudo mkdir -p /opt/phowhisper
sudo chown "$USER":"$USER" /opt/phowhisper
```

Clone the repository:

```bash
git clone <repo-url> /opt/phowhisper
cd /opt/phowhisper
```

Or copy a prepared runtime folder from another machine:

```bash
cd /path/to/source
tar \
  --exclude='phowhisper/.env' \
  --exclude='phowhisper/.venv' \
  --exclude='phowhisper/tmp/*' \
  --exclude='phowhisper/outputs/*' \
  --exclude='*/__pycache__' \
  -czf phowhisper-runtime.tar.gz phowhisper

tar -xzf phowhisper-runtime.tar.gz -C /opt
cd /opt/phowhisper
```

## 3. Python Environment

Use Conda when the target machine standardizes Python runtimes through Conda:

```bash
conda create -y -n phowhisper_runtime python=3.12
conda activate phowhisper_runtime
export PYTHONNOUSERSITE=1
python -m pip install -U pip setuptools wheel
python -m pip install -r requirements.txt
```

Use `venv` when Conda is not available:

```bash
python3.12 -m venv .venv
. .venv/bin/activate
export PYTHONNOUSERSITE=1
python -m pip install -U pip setuptools wheel
python -m pip install -r requirements.txt
```

Validate PyTorch and CUDA visibility:

```bash
PYTHONNOUSERSITE=1 python - <<'PY'
import torch

print("torch_version:", torch.__version__)
print("cuda_available:", torch.cuda.is_available())
print("cuda_device:", torch.cuda.get_device_name(0) if torch.cuda.is_available() else "cpu")
PY
```

If `cuda_available` is `False` on a GPU host, install the PyTorch wheel that matches the installed NVIDIA driver and CUDA runtime.

`PYTHONNOUSERSITE=1` is intentional. It prevents the runtime from importing packages from `~/.local` and makes dependency validation reflect the active environment only.

## 4. Model Artifacts

The runtime expects all model paths to live under `models/` unless overridden in
`.env`.

Required for normal API startup:

```text
models/phowhisper-merged/model.safetensors
models/punctuation-bartpho/model.safetensors
models/phowhisper-base/
```

`models/phowhisper-base/` must exist because the server uses its tokenizer and
configuration metadata. Its large `pytorch_model.bin` weight is only required
when the ASR checkpoint is used as a LoRA adapter or when a base-model fallback
is explicitly needed.

Required only when `PHOWHISPER_CRITICAL_COMPARE_PROVIDER=chunkformer`:

```text
models/chunkformer-model/pytorch_model.pt
models/chunkformer-model/config.yaml
models/chunkformer-model/global_cmvn
models/chunkformer-model/vocab.txt
models/chunkformer-model/tokenizer/
```

The runtime already includes `scripts/chunkformer.py` and `vendor/chunkformer/`,
so the target host does not need the original ChunkFormer training repository.

Pyannote diarization weights are resolved by `pyannote.audio` through Hugging Face. For online deployments, provide `HF_TOKEN`. For offline deployments, pre-populate the Hugging Face cache on the target machine or disable pyannote diarization in `.env`.

Package artifacts from the source machine:

```bash
cd /path/to/source/phowhisper
tar -czf phowhisper-model-artifacts.tar.gz \
  models/phowhisper-merged/model.safetensors \
  models/punctuation-bartpho/model.safetensors \
  models/phowhisper-base \
  models/chunkformer-model
```

Restore artifacts on the target machine:

```bash
cd /opt/phowhisper
tar -xzf /path/to/phowhisper-model-artifacts.tar.gz
```

Validate artifact placement:

```bash
. .venv/bin/activate
export PYTHONNOUSERSITE=1
python scripts/check_models.py
```

With Conda:

```bash
conda activate phowhisper_runtime
export PYTHONNOUSERSITE=1
python scripts/check_models.py
```

Expected output:

```text
model artifacts look ready
```

If you copied the entire prepared `phowhisper/` folder, this check is still the
source of truth. It confirms the copied model layout is usable before the server
loads large checkpoints.

## 5. Runtime Configuration

Create a local environment file:

```bash
cp .env.example .env
chmod 600 .env
```

Baseline configuration:

```env
PORT=8000
USE_NGROK=false

PHOWHISPER_ASR_MODEL_PATH=models/phowhisper-merged
PHOWHISPER_BASE_MODEL_PATH=models/phowhisper-base
PHOWHISPER_PUNCT_MODEL_PATH=models/punctuation-bartpho
PHOWHISPER_DOMAIN_CONFIG=configs/domain_correction.yaml

PHOWHISPER_CRITICAL_COMPARE_PROVIDER=none
PHOWHISPER_CHUNKFORMER_SCRIPT=scripts/chunkformer.py
PHOWHISPER_CHUNKFORMER_MODEL_PATH=models/chunkformer-model
PHOWHISPER_CHUNKFORMER_DEVICE=auto
PHOWHISPER_ENABLE_DURATION_CONFUSION_RULE=true

PHOWHISPER_USE_VAD=false
PHOWHISPER_TURN_MODE=off
PHOWHISPER_DIARIZATION_SIDECAR=true
PHOWHISPER_NUM_SPEAKERS=2
```

Keep `PHOWHISPER_CRITICAL_COMPARE_PROVIDER=none` for the fastest default API
path. Set it to `chunkformer` only when the secondary ASR comparison is needed.
`PHOWHISPER_ENABLE_DURATION_CONFUSION_RULE=true` can run independently of
ChunkFormer and fixes duration phrases such as `hai mươi lăm` in supported
duration contexts.

Pyannote configuration for mono speaker diarization:

```env
HF_TOKEN=<huggingface-token>
PHOWHISPER_ENABLE_PYANNOTE=true
PHOWHISPER_PYANNOTE_MODEL_ID=pyannote/speaker-diarization-community-1
PHOWHISPER_PYANNOTE_FALLBACK_MODEL_IDS=pyannote/speaker-diarization-3.1
```

Ngrok configuration, only when a public tunnel is required:

```env
USE_NGROK=true
NGROK_AUTHTOKEN=<ngrok-token>
NGROK_REGION=ap
NGROK_DOMAIN=
```

Do not commit `.env`. It may contain service tokens and deployment-specific paths.

## 6. Preflight Checks

Run syntax and artifact checks before starting the service:

```bash
. .venv/bin/activate
export PYTHONNOUSERSITE=1
python -m py_compile \
  server_api.py \
  scripts/chunkformer.py \
  scripts/infer_audio.py \
  scripts/check_models.py \
  src/domain_correction/*.py \
  src/punctuation/*.py \
  src/turns/*.py

python scripts/check_models.py
```

## 7. Start The API

Foreground start:

```bash
. .venv/bin/activate
export PYTHONNOUSERSITE=1
python server_api.py
```

The service exposes:

```text
GET  /health
POST /transcribe
POST /api/transcribe
POST /upload
```

Health check:

```bash
curl -fsS http://127.0.0.1:8000/health
```

Minimum healthy response fields:

```json
{
  "status": "ok",
  "asr_ready": true,
  "punctuation_ready": true,
  "domain_ready": true
}
```

## 8. API Usage

Multipart upload:

```bash
curl -X POST \
  -F "file=@/path/to/audio.wav" \
  http://127.0.0.1:8000/transcribe
```

Multipart upload with speaker count hint:

```bash
curl -X POST \
  -F "file=@/path/to/audio.wav" \
  -F "num_speakers=2" \
  http://127.0.0.1:8000/transcribe
```

Supported request formats:

| Format | Field |
| --- | --- |
| multipart/form-data | `file` |
| multipart/form-data | `audio` |
| application/json | `audio_base64` |

Primary response fields:

| Field | Description |
| --- | --- |
| `text`, `full_transcription` | Final transcript after post-processing |
| `text_raw` | Raw ASR text |
| `segments` | Speaker turns returned to clients |
| `conversation_text` | Timestamped conversation transcript |
| `diarization_segments` | Raw diarization timeline |
| `elapsed_seconds` | End-to-end processing time |

## 9. CLI Inference

Single-file inference:

```bash
. .venv/bin/activate
export PYTHONNOUSERSITE=1
python scripts/infer_audio.py /path/to/audio.wav \
  --turn-mode off \
  --output-json outputs/result.json \
  --output-text outputs/result.txt
```

VAD-based chunking:

```bash
export PYTHONNOUSERSITE=1
python scripts/infer_audio.py /path/to/audio.wav \
  --vad \
  --vad-max-segment-s 18 \
  --output-json outputs/result.json
```

## 10. systemd Service

Create `/etc/systemd/system/phowhisper.service`:

```bash
sudo tee /etc/systemd/system/phowhisper.service >/dev/null <<'EOF'
[Unit]
Description=PhoWhisper Runtime API
After=network.target

[Service]
Type=simple
WorkingDirectory=/opt/phowhisper
Environment=PYTHONUNBUFFERED=1
Environment=PYTHONNOUSERSITE=1
ExecStart=/opt/phowhisper/.venv/bin/python /opt/phowhisper/server_api.py
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF
```

Enable and start:

```bash
sudo systemctl daemon-reload
sudo systemctl enable phowhisper
sudo systemctl start phowhisper
sudo systemctl status phowhisper --no-pager
```

Logs:

```bash
journalctl -u phowhisper -f
```

Restart after code/config changes:

```bash
sudo systemctl restart phowhisper
```

## 11. Diarization Behavior

Default runtime flow:

1. Run ASR on the full audio to preserve transcript continuity.
2. Run pyannote as a sidecar diarization pass.
3. Align ASR timestamps to the diarization timeline.
4. Smooth short speaker blips.
5. Apply punctuation, capitalization, domain correction, money normalization and phone normalization.

Mono speaker diarization is probabilistic. Accuracy can degrade when speakers overlap, responses are very short, voices are similar, or the recording is heavily compressed. Stereo or dual-channel call recordings are preferred when available.

## 12. Artifact And Secret Policy

Do not commit secrets or runtime output:

```text
.env
tmp/
outputs/
```

Large model files should be copied as deployment artifacts or committed only
through Git LFS. This package intentionally allows
`models/chunkformer-model/pytorch_model.pt` through Git LFS when ChunkFormer is
part of the portable release.

Recommended artifact storage:

- Hugging Face Hub
- S3-compatible object storage
- Internal artifact registry
- Git LFS, only if the repository is explicitly configured for large files

Git LFS setup, if required:

```bash
git lfs install
git lfs track "*.safetensors" "*.bin" "*.pt" "*.pth" "*.ckpt"
git add .gitattributes
```

## 13. Troubleshooting

Port already in use:

```bash
lsof -tiTCP:8000 -sTCP:LISTEN -n -P
kill <pid>
```

Missing model artifacts:

```bash
python scripts/check_models.py
```

Pyannote fails to load:

- Verify `HF_TOKEN`.
- Confirm model access has been accepted on Hugging Face.
- Keep `PHOWHISPER_PYANNOTE_FALLBACK_MODEL_IDS=pyannote/speaker-diarization-3.1` enabled for compatibility fallback.

CUDA out of memory:

```env
PHOWHISPER_ASR_BATCH_SIZE=1
PHOWHISPER_PUNCT_BATCH_SIZE=1
```

CPU-only mode:

```env
PHOWHISPER_ASR_DEVICE=cpu
PHOWHISPER_PUNCT_DEVICE=cpu
```

CPU-only mode is intended for functional validation or low-throughput jobs.

## 14. Release Checklist

Before handing over a deployment:

```bash
. .venv/bin/activate
export PYTHONNOUSERSITE=1
python -m py_compile \
  server_api.py \
  scripts/chunkformer.py \
  scripts/infer_audio.py \
  scripts/check_models.py \
  src/domain_correction/*.py \
  src/punctuation/*.py \
  src/turns/*.py
python scripts/check_models.py
curl -fsS http://127.0.0.1:8000/health
git status --short
git check-ignore -v .env
```