Spaces:
Running
Running
| # OmniVoice Studio — Speaker Diarization | |
| Diarization splits a single audio stream into per-speaker tracks: who said | |
| what, and when. OmniVoice uses **pyannote** + **WhisperX** under the hood — | |
| the same stack the original WhisperX paper used. | |
| ## What diarization buys you | |
| - Multi-speaker dubbing: each detected speaker gets its own voice clone in | |
| the target language. | |
| - Subtitle styling: speaker labels (`SPEAKER_00:`, `SPEAKER_01:`, …) on the | |
| exported SRT/VTT files. | |
| - Audio editing: per-speaker tracks in the timeline view. | |
| ## License acceptance flow | |
| The diarization model — `pyannote/speaker-diarization-3.1` — is **gated** on | |
| HuggingFace. A valid HF token alone is not enough: you also need to accept | |
| the model's license once. | |
| 1. Get a HF token if you don't have one — see | |
| [docs/setup/huggingface-token.md](../setup/huggingface-token.md). | |
| 2. Set the token via **Settings → API Keys** (or any of the other supported | |
| paths). | |
| 3. While signed in to HuggingFace with the same account, visit: | |
| - https://huggingface.co/pyannote/speaker-diarization-3.1 → **"Agree and | |
| access repository"**. | |
| - https://huggingface.co/pyannote/segmentation-3.0 → same. | |
| 4. Restart the dub job. The first run downloads ~600 MB of model weights. | |
| If you skip the license acceptance, the HF API returns `401 Unauthorized` for | |
| the download — the same error class the in-app **"Open docs for this error"** | |
| button deeplinks to. | |
| ## Fallback behaviour | |
| When diarization is unavailable (no HF token, license not accepted, model | |
| download failed mid-run), OmniVoice's dub pipeline falls back to a | |
| **silence-gap heuristic** that splits speakers on long quiet stretches. | |
| You'll see a warning toast and the `dub_core.py` reason string surfaces in | |
| the job log: | |
| - `"diarization_skipped:no_token"` — no token resolved from the cascade. | |
| - `"diarization_skipped:401"` — token present but unauthorised on the gated | |
| model (license not accepted). | |
| - `"diarization_skipped:network"` — model download interrupted. | |
| The heuristic is not as accurate as pyannote — speakers with similar pitch | |
| or rapid turn-taking conversation get merged — but it lets the dub finish | |
| end-to-end instead of erroring. | |
| ## HF token requirement | |
| Diarization is the one OmniVoice feature where a HF token is **required**, not | |
| just recommended. See | |
| [docs/setup/huggingface-token.md](../setup/huggingface-token.md) for the | |
| three-source cascade and how the in-app **Settings → API Keys** panel works. | |
| ## Troubleshooting | |
| - HF 401 → see [troubleshooting.md#2-hf-401--pyannote-license-not-accepted](../install/troubleshooting.md). | |
| - Model download stuck → check `~/.cache/huggingface/hub/models--pyannote--*` | |
| size grows during the dub; if it stalls at 0 bytes, your token isn't being | |
| read — confirm in **Settings → API Keys** that the active source has a | |
| green checkmark. | |