File size: 4,883 Bytes
c1a2228 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | {
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": "# OmniVoice Quick Start\n\n[](https://colab.research.google.com/github/k2-fsa/OmniVoice/blob/master/docs/OmniVoice.ipynb)\n\nThis notebook demonstrates the basic usage of [OmniVoice](https://github.com/k2-fsa/OmniVoice), a massively multilingual zero-shot TTS model supporting 600+ languages.\n\n**Contents:**\n1. Installation\n2. Option A — Gradio Demo (interactive web UI, no code needed)\n3. Option B — Python API\n - 3.1 Load Model\n - 3.2 Voice Cloning\n - 3.3 Voice Design\n - 3.4 Auto Voice"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Installation\n",
"\n",
"Colab already provides a compatible PyTorch + CUDA environment, so we only need to install OmniVoice."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install omnivoice"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## 2. Option A — Gradio Demo\n\nLaunch an interactive web UI with a public Gradio link. The `--share` flag creates a temporary public URL so you can access the demo from any browser.\n\n> **If you prefer to use the Python API directly, skip to Option B below.**"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!omnivoice-demo --share"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## 3. Option B — Python API\n\n### 3.1 Load Model"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from omnivoice import OmniVoice\nimport soundfile as sf\nimport torch\nfrom IPython.display import Audio, display\n\nmodel = OmniVoice.from_pretrained(\n \"k2-fsa/OmniVoice\",\n device_map=\"cuda:0\",\n dtype=torch.float16,\n load_asr=True,\n)"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "### 3.2 Voice Cloning\n\nClone a voice from a short (3-10s) reference audio clip. Upload your own `ref.wav` or use any audio file.\n\n`ref_text` is optional — if omitted, the model uses Whisper ASR to auto-transcribe it."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from google.colab import files\n",
"\n",
"print(\"Upload a reference audio file (wav/mp3/flac):\")\n",
"uploaded = files.upload()\n",
"ref_audio_path = list(uploaded.keys())[0]\n",
"print(f\"Uploaded: {ref_audio_path}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"audio = model.generate(\n",
" text=\"Hello, this is a test of zero-shot voice cloning.\",\n",
" ref_audio=ref_audio_path,\n",
" # ref_text=\"Transcription of the reference audio.\", # optional\n",
")\n",
"\n",
"sf.write(\"clone_out.wav\", audio[0], 24000)\n",
"display(Audio(audio[0], rate=24000))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": "### 3.3 Voice Design\n\nDescribe the desired voice with speaker attributes — no reference audio needed.\n\nSupported attributes: gender, age, pitch, style (whisper), English accent, Chinese dialect. See [docs/voice-design.md](https://github.com/k2-fsa/OmniVoice/blob/master/docs/voice-design.md) for the full list."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"audio = model.generate(\n",
" text=\"Hello, this is a test of zero-shot voice design.\",\n",
" instruct=\"female, low pitch, british accent\",\n",
")\n",
"\n",
"sf.write(\"design_out.wav\", audio[0], 24000)\n",
"display(Audio(audio[0], rate=24000))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": "### 3.4 Auto Voice\n\nLet the model choose a voice automatically — no reference audio or instruct needed."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"audio = model.generate(\n",
" text=\"This is a sentence generated with automatic voice selection.\",\n",
")\n",
"\n",
"sf.write(\"auto_out.wav\", audio[0], 24000)\n",
"display(Audio(audio[0], rate=24000))"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 0
} |