{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ๐๏ธ VoxCPM2 - Interactive TTS Web App (Gradio UI)\n",
"**Launch your own voice cloning & synthesis web app in Google Colab!**\n",
"\n",
"[Model: openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) | [Paper](https://arxiv.org/abs/2509.24650) | [Docs](https://voxcpm.readthedocs.io/)\n",
"\n",
"### ๐ MAIN FEATURE: Interactive Gradio Web UI\n",
"Run a beautiful web interface with tabs for:\n",
"- ๐ **Basic TTS** โ Type text, get 48kHz studio-quality speech\n",
"- ๐จ **Voice Design** โ Describe any voice with natural language\n",
"- ๐ค **Voice Clone** โ Upload a 3-10s voice sample, clone it!\n",
"- ๐ **Multilingual** โ Type in any language โ auto-detects & speaks it\n",
"- ๐๐ค **Multilingual + Clone** โ Clone a voice & speak ANY language\n",
"- ๐๐จ **Multilingual + Design** โ Design a voice & speak ANY language\n",
"\n",
"**You'll get a public URL** to share or use from your phone!\n",
"\n",
"---\n",
"\n",
"### โ๏ธ Quick Setup (2 steps then launch UI)\n",
"1. Run **Step 1** โ Install packages\n",
"2. Run **Step 2** โ Load model\n",
"3. Run **Step 3 (MAIN)** โ Launch Gradio UI ๐ฏ\n",
"\n",
"*Optional: Scroll down for individual mode demos if you want to experiment with code.*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## ๐ฆ Step 1: Install Dependencies (Run First)\n",
"*Note: First run downloads ~4.6GB model weights. Takes ~3-5 minutes.*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install -q voxcpm soundfile gradio\n",
"\n",
"import torch\n",
"print(f\"PyTorch: {torch.__version__}\")\n",
"print(f\"CUDA available: {torch.cuda.is_available()}\")\n",
"if torch.cuda.is_available():\n",
" print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n",
" print(f\"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1024**3:.1f} GB\")\n",
"else:\n",
" print(\"\\nโ ๏ธ WARNING: No GPU detected! VoxCPM2 needs a GPU runtime.\")\n",
" print(\"โก๏ธ Run: Runtime -> Change runtime type -> T4 GPU -> Save -> Restart session\")\n",
" print(\"Then re-run all cells from the top.\")\n",
" raise SystemExit(\"Please switch to GPU runtime and restart.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## ๐ Step 2: Load Model (Run Second)\n",
"**โ ๏ธ Must use GPU runtime.** (Runtime โ Change runtime type โ T4 GPU)\n",
"\n",
"*Torch compilation (`optimize=True`) is disabled on T4 because bfloat16 is not natively supported.*\n",
"*This saves ~50-100 seconds of failed compilation during warmup.*\n",
"\n",
"*If CPU is detected, model auto-converts to float32 (slower but works).*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from voxcpm import VoxCPM\n",
"import time\n",
"\n",
"print(\"\\n\" + \"=\"*60)\n",
"print(\"LOADING VoxCPM2...\")\n",
"print(\"=\"*60 + \"\\n\")\n",
"\n",
"import torch\n",
"device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
"print(f\"Detected device: {device}\")\n",
"\n",
"start = time.time()\n",
"\n",
"# โก SPEED TIP: T4 GPU does NOT support bfloat16 torch.compile natively.\n",
"# Setting optimize=False saves ~50-100 seconds of failed compilation during warmup.\n",
"model = VoxCPM.from_pretrained(\n",
" \"openbmb/VoxCPM2\",\n",
" load_denoiser=False, # Disable to save VRAM (~500MB)\n",
" optimize=False, # torch.compile fails on T4 bfloat16, skip it\n",
")\n",
"\n",
"elapsed = time.time() - start\n",
"print(f\"\\nโ
Model loaded in {elapsed:.1f}s\")\n",
"print(f\"๐ข Output sample rate: {model.tts_model.sample_rate} Hz\")\n",
"print(f\"๐ฅ๏ธ Device: {device}\")\n",
"\n",
"# CPU fallback: bfloat16 crashes on CPU, convert to float32\n",
"if not torch.cuda.is_available():\n",
" print(\"\\nโ ๏ธ CPU detected! Converting model from bfloat16 to float32...\")\n",
" model.tts_model = model.tts_model.to(torch.float32)\n",
" print(\"โ
Model converted to float32 (slower but works on CPU)\")\n",
"else:\n",
" print(f\"๐ฎ GPU VRAM used: {torch.cuda.memory_allocated() / 1024**3:.1f} GB\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## ๐ฏ Step 3 (MAIN): Launch Interactive Gradio Web UI\n",
"**This is the primary way to use VoxCPM2.** Run this cell and you'll get a public URL!\n",
"\n",
"### What's inside the UI:\n",
"| Tab | What it does |\n",
"|-----|-------------|\n",
"| ๐ **Basic TTS** | Type any text โ get 48kHz speech |\n",
"| ๐จ **Voice Design** | Describe a voice (e.g. \"warm female narrator\") โ get custom voice |\n",
"| ๐ค **Voice Clone** | Upload 3-10s audio โ clone that voice |\n",
"| ๐ **Multilingual** | Type in any language โ auto-detects & speaks it |\n",
"| ๐๐ค **Multilingual + Clone** | Clone a voice AND speak in ANY language |\n",
"| ๐๐จ **Multilingual + Design** | Design a voice AND speak in ANY language |\n",
"| โน๏ธ **About** | Tips, capabilities, speed/quality guide |\n",
"\n",
"**Controls in every tab:**\n",
"- **Timesteps slider** (4-30): Lower = faster, Higher = better quality\n",
"- **CFG Scale slider** (1.0-3.0): Lower = more natural, Higher = tighter control\n",
"\n",
"*The UI will generate a public `gradio.live` URL you can open anywhere โ even your phone!*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import gradio as gr\n",
"import soundfile as sf\n",
"import numpy as np\n",
"\n",
"print(\"\\n\" + \"=\"*60)\n",
"print(\"๐ LAUNCHING VoxCPM2 INTERACTIVE WEB UI\")\n",
"print(\"=\"*60)\n",
"print(\"\\nTabs: ๐ Basic | ๐จ Design | ๐ค Clone | ๐ Multilingual | ๐๐ค Multi+Clone | ๐๐จ Multi+Design\")\n",
"print(\"Controls: Timesteps (4-30) | CFG Scale (1.0-3.0)\")\n",
"print(\"\\nโณ Starting Gradio server...\\n\")\n",
"\n",
"def tts_basic(text, timesteps, cfg):\n",
" \"\"\"Basic text-to-speech generation.\"\"\"\n",
" if not text or not text.strip():\n",
" return None, \"Please enter some text.\"\n",
" wav = model.generate(\n",
" text=text.strip(),\n",
" cfg_value=float(cfg),\n",
" inference_timesteps=int(timesteps),\n",
" )\n",
" output = \"/content/gradio_basic.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
" return output, f\"โ
Generated! Duration: {len(wav)/model.tts_model.sample_rate:.1f}s\"\n",
"\n",
"def tts_design(text, description, timesteps, cfg):\n",
" \"\"\"Voice design via natural language description.\"\"\"\n",
" if not text or not text.strip():\n",
" return None, \"Please enter some text.\"\n",
" full_text = f\"({description.strip()}) {text.strip()}\"\n",
" wav = model.generate(\n",
" text=full_text,\n",
" cfg_value=float(cfg),\n",
" inference_timesteps=int(timesteps),\n",
" )\n",
" output = \"/content/gradio_design.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
" return output, f\"โ
Generated with voice design! Duration: {len(wav)/model.tts_model.sample_rate:.1f}s\"\n",
"\n",
"def tts_clone(text, audio_file, timesteps, cfg):\n",
" \"\"\"Zero-shot voice cloning from reference audio.\"\"\"\n",
" if not text or not text.strip():\n",
" return None, \"Please enter some text.\"\n",
" if audio_file is None:\n",
" return None, \"โ ๏ธ Please upload a reference audio file (3-10s WAV or MP3).\"\n",
" wav = model.generate(\n",
" text=text.strip(),\n",
" reference_wav_path=audio_file,\n",
" cfg_value=float(cfg),\n",
" inference_timesteps=int(timesteps),\n",
" )\n",
" output = \"/content/gradio_clone.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
" return output, f\"โ
Voice cloned! Duration: {len(wav)/model.tts_model.sample_rate:.1f}s\"\n",
"\n",
"def tts_multilingual(text, timesteps, cfg):\n",
" \"\"\"Multilingual TTS โ just type in any supported language.\"\"\"\n",
" if not text or not text.strip():\n",
" return None, \"Please enter some text.\"\n",
" wav = model.generate(\n",
" text=text.strip(),\n",
" cfg_value=float(cfg),\n",
" inference_timesteps=int(timesteps),\n",
" )\n",
" output = \"/content/gradio_multilingual.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
" return output, f\"โ
Multilingual audio generated! Duration: {len(wav)/model.tts_model.sample_rate:.1f}s\"\n",
"\n",
"def tts_multilingual_clone(text, audio_file, timesteps, cfg):\n",
" \"\"\"Clone a voice and speak in ANY language!\"\"\"\n",
" if not text or not text.strip():\n",
" return None, \"Please enter some text.\"\n",
" if audio_file is None:\n",
" return None, \"โ ๏ธ Please upload a reference audio file (3-10s). The cloned voice will speak your text in any language you type!\"\n",
" wav = model.generate(\n",
" text=text.strip(),\n",
" reference_wav_path=audio_file,\n",
" cfg_value=float(cfg),\n",
" inference_timesteps=int(timesteps),\n",
" )\n",
" output = \"/content/gradio_multilingual_clone.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
" lang_note = \"๐ The cloned voice now speaks your text in whatever language you typed!\"\n",
" return output, f\"โ
Multilingual voice cloned! Duration: {len(wav)/model.tts_model.sample_rate:.1f}s | {lang_note}\"\n",
"\n",
"def tts_multilingual_design(text, description, timesteps, cfg):\n",
" \"\"\"Design a voice and speak in ANY language!\"\"\"\n",
" if not text or not text.strip():\n",
" return None, \"Please enter some text.\"\n",
" full_text = f\"({description.strip()}) {text.strip()}\"\n",
" wav = model.generate(\n",
" text=full_text,\n",
" cfg_value=float(cfg),\n",
" inference_timesteps=int(timesteps),\n",
" )\n",
" output = \"/content/gradio_multilingual_design.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
" lang_note = \"๐ The designed voice speaks your text in whatever language you typed!\"\n",
" return output, f\"โ
Multilingual voice designed! Duration: {len(wav)/model.tts_model.sample_rate:.1f}s | {lang_note}\"\n",
"\n",
"# โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
"# BUILD THE GRADIO INTERFACE\n",
"# โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
"with gr.Blocks() as demo:\n",
"\n",
" gr.Markdown(\"\"\"\n",
" \n",
"
๐๏ธ VoxCPM2 โ Voice Synthesis & Cloning
\n",
"
Tokenizer-Free TTS โข Voice Design โข Zero-Shot Voice Cloning โข 30+ Languages โข Multilingual Cloning\n",
"
๐ค Model |\n",
"
๐ Paper |\n",
"
๐ Docs\n",
"
\n",
" \"\"\")\n",
"\n",
" # โโ Shared controls component โโ\n",
" with gr.Row():\n",
" with gr.Column(scale=1):\n",
" shared_steps = gr.Slider(\n",
" minimum=4, maximum=30, value=10, step=1,\n",
" label=\"โฑ๏ธ Timesteps (quality vs speed)\",\n",
" info=\"Lower = faster draft | Higher = better quality\"\n",
" )\n",
" with gr.Column(scale=1):\n",
" shared_cfg = gr.Slider(\n",
" minimum=1.0, maximum=3.0, value=2.0, step=0.1,\n",
" label=\"๐๏ธ CFG Scale (control vs naturalness)\",\n",
" info=\"Lower = more natural | Higher = tighter control\"\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 1: Basic TTS\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"๐ Basic TTS\"):\n",
" gr.Markdown(\"Type any text and generate studio-quality 48kHz speech instantly.\")\n",
" with gr.Row():\n",
" basic_text = gr.Textbox(\n",
" label=\"Text to Speak\",\n",
" value=\"Hello! This is VoxCPM2, a powerful tokenizer-free text-to-speech model.\",\n",
" lines=4,\n",
" placeholder=\"Enter text in any of 30+ languages...\"\n",
" )\n",
" with gr.Row():\n",
" basic_btn = gr.Button(\"๐ Generate Speech\", variant=\"primary\", size=\"lg\")\n",
" with gr.Row():\n",
" basic_audio = gr.Audio(label=\"Generated Audio\", type=\"filepath\", autoplay=False)\n",
" basic_status = gr.Textbox(label=\"Status\", interactive=False)\n",
" basic_btn.click(\n",
" tts_basic,\n",
" [basic_text, shared_steps, shared_cfg],\n",
" [basic_audio, basic_status]\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 2: Voice Design\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"๐จ Voice Design\"):\n",
" gr.Markdown(\"Describe any voice with natural language โ the model will create it!\")\n",
" with gr.Row():\n",
" with gr.Column(scale=2):\n",
" design_text = gr.Textbox(\n",
" label=\"Text to Speak\",\n",
" value=\"Welcome to the presentation. Today we explore the frontiers of artificial intelligence.\",\n",
" lines=3,\n",
" )\n",
" with gr.Column(scale=1):\n",
" design_desc = gr.Textbox(\n",
" label=\"Voice Description\",\n",
" value=\"A warm, professional female narrator with a calm and reassuring tone\",\n",
" lines=3,\n",
" placeholder=\"e.g., A deep male voice, an enthusiastic child, a robot...\"\n",
" )\n",
" with gr.Row():\n",
" gr.Examples(\n",
" examples=[\n",
" [\"Hello! How can I help you today?\", \"A young woman, gentle and soothing voice\"],\n",
" [\"Welcome to the nightly news.\", \"A deep male narrator, professional and authoritative\"],\n",
" [\"Hey there! Ready to build something amazing?\", \"An energetic young man, enthusiastic and upbeat\"],\n",
" [\"System initialized. All parameters nominal.\", \"A robot, monotone synthetic voice\"],\n",
" [\"In a world where technology knows no bounds...\", \"A dramatic movie trailer voice, deep and intense\"],\n",
" ],\n",
" inputs=[design_text, design_desc],\n",
" label=\"๐จ Quick Voice Presets\"\n",
" )\n",
" with gr.Row():\n",
" design_btn = gr.Button(\"๐จ Design Voice\", variant=\"primary\", size=\"lg\")\n",
" with gr.Row():\n",
" design_audio = gr.Audio(label=\"Designed Voice\", type=\"filepath\")\n",
" design_status = gr.Textbox(label=\"Status\", interactive=False)\n",
" design_btn.click(\n",
" tts_design,\n",
" [design_text, design_desc, shared_steps, shared_cfg],\n",
" [design_audio, design_status]\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 3: Voice Clone\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"๐ค Voice Clone\"):\n",
" gr.Markdown(\"Upload a 3-10 second audio clip of any voice. The model will clone it!\")\n",
" with gr.Row():\n",
" with gr.Column(scale=2):\n",
" clone_text = gr.Textbox(\n",
" label=\"Text to Speak in Cloned Voice\",\n",
" value=\"Hello! This is my voice cloned by artificial intelligence. The quality is remarkably accurate.\",\n",
" lines=3,\n",
" )\n",
" with gr.Column(scale=1):\n",
" clone_audio_in = gr.Audio(\n",
" label=\"๐ค Upload Reference Voice (3-10s)\",\n",
" type=\"filepath\"\n",
" )\n",
" with gr.Row():\n",
" clone_btn = gr.Button(\"๐ค Clone Voice\", variant=\"primary\", size=\"lg\")\n",
" with gr.Row():\n",
" clone_audio_out = gr.Audio(label=\"๐ Cloned Output\", type=\"filepath\")\n",
" clone_status = gr.Textbox(label=\"Status\", interactive=False)\n",
" clone_btn.click(\n",
" tts_clone,\n",
" [clone_text, clone_audio_in, shared_steps, shared_cfg],\n",
" [clone_audio_out, clone_status]\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 4: Multilingual\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"๐ Multilingual\"):\n",
" gr.Markdown(\"Type in ANY language โ no language tags needed! Supports 30+ languages.\")\n",
" with gr.Row():\n",
" multi_text = gr.Textbox(\n",
" label=\"Text in Any Language\",\n",
" value=\"ไฝ ๅฅฝ๏ผ่ฟๆฏVoxCPM2็ๅค่ฏญ่จ่ฏญ้ณๅๆๆผ็คบใHello! This is multilingual too. ยกHola! Esto es espaรฑol.\",\n",
" lines=3,\n",
" placeholder=\"English, ไธญๆ, Espaรฑol, Franรงais, Deutsch, ๆฅๆฌ่ช, ํ๊ตญ์ด, เคนเคฟเคจเฅเคฆเฅ, and more...\"\n",
" )\n",
" with gr.Row():\n",
" gr.Examples(\n",
" examples=[\n",
" [\"The quick brown fox jumps over the lazy dog.\"],\n",
" [\"ไฝ ๅฅฝ๏ผ่ฟๆฏไธญๆ่ฏญ้ณๅๆๆผ็คบใ\"],\n",
" [\"ยกHola! VoxCPM2 es un modelo de sรญntesis de voz multilingรผe.\"],\n",
" [\"Bonjour! VoxCPM2 est un modรจle de synthรจse vocale multilingue.\"],\n",
" [\"ใใใซใกใฏใVoxCPM2ใฏๅค่จ่ชๅฏพๅฟใฎ้ซๅ่ณชใช้ณๅฃฐๅๆใขใใซใงใใ\"],\n",
" [\"์๋
ํ์ธ์, VoxCPM2๋ ๊ณ ํ์ง ๋ค๊ตญ์ด ์์ฑ ํฉ์ฑ ๋ชจ๋ธ์
๋๋ค.\"],\n",
" ],\n",
" inputs=[multi_text],\n",
" label=\"๐ Language Examples\"\n",
" )\n",
" with gr.Row():\n",
" multi_btn = gr.Button(\"๐ Generate Multilingual Speech\", variant=\"primary\", size=\"lg\")\n",
" with gr.Row():\n",
" multi_audio = gr.Audio(label=\"Multilingual Output\", type=\"filepath\")\n",
" multi_status = gr.Textbox(label=\"Status\", interactive=False)\n",
" multi_btn.click(\n",
" tts_multilingual,\n",
" [multi_text, shared_steps, shared_cfg],\n",
" [multi_audio, multi_status]\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 5: Multilingual + Voice Clone โญ NEW\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"๐๐ค Multilingual + Clone\"):\n",
" gr.Markdown(\"\"\"\n",
" **Clone a voice and make it speak ANY language!** ๐\n",
" \n",
" Upload a voice sample (e.g., an English speaker), then type text in Chinese, Spanish, French, Japanese, Hindi โ anything!\n",
" The cloned voice will naturally speak your text in that language.\n",
" \"\"\")\n",
" with gr.Row():\n",
" with gr.Column(scale=2):\n",
" mc_text = gr.Textbox(\n",
" label=\"Text to Speak (Any Language!)\",\n",
" value=\"ไฝ ๅฅฝ๏ผ่ฟๆฏๆๅ
้็ๅฃฐ้ณๅจ่ฏดไธญๆใยกHola! Esta es mi voz clonada hablando espaรฑol.\",\n",
" lines=3,\n",
" placeholder=\"Type in ANY language: English, ไธญๆ, Espaรฑol, Franรงais, ๆฅๆฌ่ช, ํ๊ตญ์ด, เคนเคฟเคจเฅเคฆเฅ...\"\n",
" )\n",
" with gr.Column(scale=1):\n",
" mc_audio_in = gr.Audio(\n",
" label=\"๐ค Upload Voice to Clone (3-10s)\",\n",
" type=\"filepath\"\n",
" )\n",
" with gr.Row():\n",
" gr.Examples(\n",
" examples=[\n",
" [\"ไฝ ๅฅฝ๏ผ่ฟๆฏๅ
้็ๅฃฐ้ณๅจ่ฏดไธญๆใ\", None],\n",
" [\"ยกHola! Esta es mi voz clonada hablando espaรฑol.\", None],\n",
" [\"Bonjour! C'est ma voix clonรฉe parlant franรงais.\", None],\n",
" [\"ใใใซใกใฏใใใใฏๅ
้ใใใๅฃฐใๆฅๆฌ่ชใ่ฉฑใใฆใใพใใ\", None],\n",
" [\"เคจเคฎเคธเฅเคคเฅ! เคฏเคน เคฎเฅเคฐเฅ เคเฅเคฒเฅเคจ เคเฅ เคเคตเคพเคเคผ เคนเคฟเคเคฆเฅ เคฎเฅเค เคฌเฅเคฒ เคฐเคนเฅ เคนเฅเฅค\", None],\n",
" ],\n",
" inputs=[mc_text, mc_audio_in],\n",
" label=\"๐ Multilingual Clone Examples (upload your own voice!)\"\n",
" )\n",
" with gr.Row():\n",
" mc_btn = gr.Button(\"๐๐ค Clone Voice + Speak Any Language\", variant=\"primary\", size=\"lg\")\n",
" with gr.Row():\n",
" mc_audio_out = gr.Audio(label=\"๐ Multilingual Cloned Output\", type=\"filepath\")\n",
" mc_status = gr.Textbox(label=\"Status\", interactive=False)\n",
" mc_btn.click(\n",
" tts_multilingual_clone,\n",
" [mc_text, mc_audio_in, shared_steps, shared_cfg],\n",
" [mc_audio_out, mc_status]\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 6: Multilingual + Voice Design โญ NEW\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"๐๐จ Multilingual + Design\"):\n",
" gr.Markdown(\"\"\"\n",
" **Design a voice with words and make it speak ANY language!** ๐จ๐\n",
" \n",
" Describe the voice you want (e.g., \"warm elderly British woman\"), then type text in any language.\n",
" The designed voice will speak your text in that language.\n",
" \"\"\")\n",
" with gr.Row():\n",
" with gr.Column(scale=2):\n",
" md_text = gr.Textbox(\n",
" label=\"Text to Speak (Any Language!)\",\n",
" value=\"ไฝ ๅฅฝ๏ผ่ฟๆฏ็จ่ฎพ่ฎก็ๆธฉๆๅฅณๅฃฐ่ฏด็ไธญๆใBonjour! C'est une voix franรงaise chaleureuse et รฉlรฉgante.\",\n",
" lines=3,\n",
" placeholder=\"Type in ANY language...\"\n",
" )\n",
" with gr.Column(scale=1):\n",
" md_desc = gr.Textbox(\n",
" label=\"Voice Description\",\n",
" value=\"A warm, gentle female voice, clear and articulate\",\n",
" lines=3,\n",
" placeholder=\"e.g., A deep British male, a cheerful child, a wise elderly woman...\"\n",
" )\n",
" with gr.Row():\n",
" gr.Examples(\n",
" examples=[\n",
" [\"ไฝ ๅฅฝ๏ผ่ฟๆฏไธไธชๆธฉๆ็ไธญๆๅฅณๅฃฐใ\", \"A warm young Chinese woman, gentle and friendly\"],\n",
" [\"ยกHola! Esta es una voz espaรฑola profesional y clara.\", \"A confident Spanish female narrator, professional\"],\n",
" [\"Bonjour! C'est une voix franรงaise รฉlรฉgante et douce.\", \"A sophisticated French woman, elegant and soft-spoken\"],\n",
" [\"ใใใซใกใฏใใใใฏๆฅๆฌ่ชใฎไธๅฏงใชๅฅณๆงใฎๅฃฐใงใใ\", \"A polite Japanese woman, soft and respectful\"],\n",
" [\"เคจเคฎเคธเฅเคคเฅ! เคฏเคน เคเค เคเคฐเฅเคฎเคเฅเคถเฅ เคญเคพเคฐเคคเฅเคฏ เคฎเคนเคฟเคฒเคพ เคเฅ เคเคตเคพเคเคผ เคนเฅเฅค\", \"A warm Indian woman, friendly and expressive\"],\n",
" ],\n",
" inputs=[md_text, md_desc],\n",
" label=\"๐จ๐ Multilingual Design Examples\"\n",
" )\n",
" with gr.Row():\n",
" md_btn = gr.Button(\"๐๐จ Design Voice + Speak Any Language\", variant=\"primary\", size=\"lg\")\n",
" with gr.Row():\n",
" md_audio_out = gr.Audio(label=\"๐ Multilingual Designed Output\", type=\"filepath\")\n",
" md_status = gr.Textbox(label=\"Status\", interactive=False)\n",
" md_btn.click(\n",
" tts_multilingual_design,\n",
" [md_text, md_desc, shared_steps, shared_cfg],\n",
" [md_audio_out, md_status]\n",
" )\n",
"\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" # TAB 7: About / Tips\n",
" # โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ\n",
" with gr.Tab(\"โน๏ธ About & Tips\"):\n",
" gr.Markdown(\"\"\"\n",
" ## โก Speed vs Quality Guide\n",
"\n",
" | Timesteps | Quality | Speed | Best For |\n",
" |-----------|---------|-------|----------|\n",
" | **4-5** | Draft | โก Fast | Quick testing |\n",
" | **8-10** | Good | ๐ Normal | Default, balanced |\n",
" | **15-20** | High | ๐ข Slow | Voice cloning |\n",
" | **25-30** | Best | ๐ Very Slow | Audiobooks, production |\n",
"\n",
" ## ๐๏ธ CFG Scale Guide\n",
" | CFG Value | Effect |\n",
" |-----------|--------|\n",
" | **1.0-1.5** | More natural, relaxed |\n",
" | **2.0** | Balanced (default) |\n",
" | **2.5-3.0** | Tighter control, more precise |\n",
"\n",
" ## ๐ Model Info\n",
" - **2B parameters** โ tokenizer-free diffusion TTS\n",
" - **48kHz** studio-quality output\n",
" - **30+ languages** โ no language tags needed\n",
" - **Zero-shot voice cloning** โ 3-10s samples\n",
" - **Voice design** โ describe any voice in words\n",
" - **Multilingual voice cloning** โ clone voice & speak any language โญ NEW\n",
" - **Multilingual voice design** โ design voice & speak any language โญ NEW\n",
" - **Apache-2.0 license** โ free for commercial use\n",
"\n",
" ## ๐ก Pro Tips\n",
" - **Multilingual + Clone**: Upload an English voice, type in Chinese โ the English voice speaks Chinese!\n",
" - **Multilingual + Design**: Describe \"a warm French woman\" and type in Japanese โ a French-accented voice speaks Japanese!\n",
" - For voice cloning, upload **clean 3-10s WAV/MP3** of a single speaker\n",
" - For voice design, be descriptive: *\"A young British woman, soft and caring\"*\n",
" - Lower timesteps for quick drafts, higher for final quality\n",
" - The model auto-detects language โ just type in any language!\n",
" - **No language tags needed** โ just type naturally in any language\n",
" \"\"\")\n",
"\n",
"# โโ LAUNCH โโ\n",
"demo.launch(share=True, debug=True, theme=gr.themes.Soft())\n",
"\n",
"print(\"\\nโ
Gradio UI is running!\")\n",
"print(\"\\n๐ก Use the public URL above to access from any device.\")\n",
"print(\"๐ก The URL is active as long as this Colab session runs.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## ๐ฅ Download All Generated Audio\n",
"After using the UI (or any optional mode below), download all your audio files as a ZIP."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import glob\n",
"import os\n",
"from google.colab import files\n",
"import zipfile\n",
"\n",
"wav_files = glob.glob(\"/content/*.wav\")\n",
"print(f\"Found {len(wav_files)} WAV files:\")\n",
"for f in wav_files:\n",
" size = os.path.getsize(f) / 1024**2\n",
" print(f\" - {os.path.basename(f)} ({size:.1f} MB)\")\n",
"\n",
"if wav_files:\n",
" zip_path = \"/content/voxcpm2_outputs.zip\"\n",
" with zipfile.ZipFile(zip_path, 'w') as zf:\n",
" for f in wav_files:\n",
" zf.write(f, os.path.basename(f))\n",
" print(f\"\\n๐ฆ ZIP created: {zip_path}\")\n",
" files.download(zip_path)\n",
"else:\n",
" print(\"No WAV files found. Use the Gradio UI above to generate audio first.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## ๐งน Cleanup (Free GPU Memory)\n",
"Run this when done to release GPU memory."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import gc\n",
"\n",
"if 'model' in globals():\n",
" del model\n",
"if 'demo' in globals():\n",
" del demo\n",
"\n",
"gc.collect()\n",
"\n",
"if torch.cuda.is_available():\n",
" torch.cuda.empty_cache()\n",
" torch.cuda.ipc_collect()\n",
" print(f\"๐งน GPU memory cleared.\")\n",
"\n",
"print(\"โ
Cleanup complete. Reload the model if you want to use it again.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"---\n",
"## ๐ฌ OPTIONAL: Individual Mode Experiments\n",
"*The cells below let you experiment with each VoxCPM2 capability directly in code.*\n",
"*These are completely optional โ the Gradio UI above covers all of this already.*\n",
"\n",
"*Skip to the bottom if you only need the UI.*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ๐ [OPTIONAL] Mode 1: Basic Text-to-Speech (Code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Basic TTS via code\n",
"import soundfile as sf\n",
"import IPython.display as ipd\n",
"\n",
"text = \"Hello! VoxCPM2 produces studio-quality 48 kilohertz audio.\"\n",
"print(f\"Text: {text}\")\n",
"\n",
"start = time.time()\n",
"wav = model.generate(text=text, cfg_value=2.0, inference_timesteps=10)\n",
"elapsed = time.time() - start\n",
"\n",
"output_path = \"/content/optional_basic_tts.wav\"\n",
"sf.write(output_path, wav, model.tts_model.sample_rate)\n",
"\n",
"duration = len(wav) / model.tts_model.sample_rate\n",
"print(f\"โ
Saved: {output_path}\")\n",
"print(f\"โฑ๏ธ Time: {elapsed:.1f}s | Duration: {duration:.1f}s\")\n",
"ipd.Audio(output_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ๐จ [OPTIONAL] Mode 2: Voice Design (Code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Voice design via code\n",
"description = \"A deep male narrator, professional and authoritative\"\n",
"text = \"Welcome to today's presentation. We will cover the fundamentals of machine learning.\"\n",
"full_text = f\"({description}) {text}\"\n",
"\n",
"print(f\"๐จ Voice Design: {description}\")\n",
"\n",
"start = time.time()\n",
"wav = model.generate(text=full_text, cfg_value=2.0, inference_timesteps=12)\n",
"elapsed = time.time() - start\n",
"\n",
"output_path = \"/content/optional_voice_design.wav\"\n",
"sf.write(output_path, wav, model.tts_model.sample_rate)\n",
"\n",
"duration = len(wav) / model.tts_model.sample_rate\n",
"print(f\"โ
Saved: {output_path}\")\n",
"print(f\"โฑ๏ธ Time: {elapsed:.1f}s | Duration: {duration:.1f}s\")\n",
"ipd.Audio(output_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ๐ค [OPTIONAL] Mode 3: Voice Cloning (Code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Voice cloning via code โ upload your own audio file first\n",
"from google.colab import files\n",
"\n",
"print(\"๐ค Upload a short voice sample (3-10s WAV or MP3):\")\n",
"uploaded = files.upload()\n",
"\n",
"if uploaded:\n",
" ref_filename = list(uploaded.keys())[0]\n",
" ref_path = f\"/content/{ref_filename}\"\n",
" print(f\"\\nโ
Reference ready: {ref_path}\")\n",
"\n",
" clone_text = \"Hello! This is my cloned voice speaking through artificial intelligence.\"\n",
" print(f\"๐๏ธ Cloning: {clone_text}\")\n",
"\n",
" start = time.time()\n",
" wav = model.generate(\n",
" text=clone_text,\n",
" reference_wav_path=ref_path,\n",
" cfg_value=2.0,\n",
" inference_timesteps=15,\n",
" )\n",
" elapsed = time.time() - start\n",
"\n",
" output_path = \"/content/optional_voice_clone.wav\"\n",
" sf.write(output_path, wav, model.tts_model.sample_rate)\n",
"\n",
" duration = len(wav) / model.tts_model.sample_rate\n",
" print(f\"\\nโ
Cloned! Saved: {output_path}\")\n",
" print(f\"โฑ๏ธ Time: {elapsed:.1f}s | Duration: {duration:.1f}s\")\n",
" ipd.Audio(output_path)\n",
"else:\n",
" print(\"โ No file uploaded.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ๐ [OPTIONAL] Mode 4: Multilingual Voice Clone (Code) โญ NEW"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Clone a voice and speak in ANY language!\n",
"from google.colab import files\n",
"\n",
"print(\"๐ค Upload a voice sample to clone (3-10s):\")\n",
"uploaded = files.upload()\n",
"\n",
"if uploaded:\n",
" ref_filename = list(uploaded.keys())[0]\n",
" ref_path = f\"/content/{ref_filename}\"\n",
" print(f\"\\nโ
Reference ready: {ref_path}\")\n",
"\n",
" # Type in ANY language โ the cloned voice will speak it!\n",
" multilingual_text = \"ไฝ ๅฅฝ๏ผ่ฟๆฏๆๅ
้็ๅฃฐ้ณๅจ่ฏดไธญๆใ้ๅธธ่ช็ถๅๆต็
๏ผ\"\n",
" print(f\"๐ Multilingual Clone: {multilingual_text}\")\n",
"\n",
" start = time.time()\n",
" wav = model.generate(\n",
" text=multilingual_text,\n",
" reference_wav_path=ref_path,\n",
" cfg_value=2.0,\n",
" inference_timesteps=15,\n",
" )\n",
" elapsed = time.time() - start\n",
"\n",
" output_path = \"/content/optional_multilingual_clone.wav\"\n",
" sf.write(output_path, wav, model.tts_model.sample_rate)\n",
"\n",
" duration = len(wav) / model.tts_model.sample_rate\n",
" print(f\"\\nโ
Multilingual clone saved: {output_path}\")\n",
" print(f\"โฑ๏ธ Time: {elapsed:.1f}s | Duration: {duration:.1f}s\")\n",
" print(\"๐ The cloned voice now speaks Chinese! Try Spanish, French, Japanese, Hindi, etc.\")\n",
" ipd.Audio(output_path)\n",
"else:\n",
" print(\"โ No file uploaded.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ๐ต [OPTIONAL] Mode 5: Hi-Fi Ultimate Cloning (Code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Hi-Fi cloning โ needs prompt audio + prompt text + reference audio\n",
"print(\"๐ค Upload PROMPT audio (with known text) first:\")\n",
"up1 = files.upload()\n",
"if up1:\n",
" prompt_file = list(up1.keys())[0]\n",
" !cp /content/{prompt_file} /content/prompt.wav\n",
" print(\"\\n๐ค Upload REFERENCE audio (clean voice) now:\")\n",
" up2 = files.upload()\n",
" if up2:\n",
" ref_file = list(up2.keys())[0]\n",
" !cp /content/{ref_file} /content/reference.wav\n",
" prompt_text = input(\"\\nEnter EXACT transcript of prompt audio: \")\n",
" hifi_text = \"This is the ultimate high fidelity cloned voice.\"\n",
"\n",
" print(\"\\n๐ต Hi-Fi Cloning...\")\n",
" start = time.time()\n",
" wav = model.generate(\n",
" text=hifi_text,\n",
" prompt_wav_path=\"/content/prompt.wav\",\n",
" prompt_text=prompt_text,\n",
" reference_wav_path=\"/content/reference.wav\",\n",
" cfg_value=2.5,\n",
" inference_timesteps=20,\n",
" )\n",
" elapsed = time.time() - start\n",
"\n",
" output_path = \"/content/optional_hifi_clone.wav\"\n",
" sf.write(output_path, wav, model.tts_model.sample_rate)\n",
" print(f\"\\nโ
Hi-Fi clone saved: {output_path}\")\n",
" print(f\"โฑ๏ธ Time: {elapsed:.1f}s\")\n",
" ipd.Audio(output_path)\n",
" else:\n",
" print(\"โ No reference file.\")\n",
"else:\n",
" print(\"โ No prompt file.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ๐ก [OPTIONAL] Mode 6: Streaming Generation (Code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Streaming generation for long texts\n",
"import numpy as np\n",
"\n",
"long_text = \"This is a demonstration of streaming text-to-speech generation. \" * 5\n",
"long_text += \"The model processes text in chunks, useful for long-form content.\"\n",
"\n",
"print(f\"๐ Text length: {len(long_text)} chars\")\n",
"print(\"๐ก Streaming...\\n\")\n",
"\n",
"all_chunks = []\n",
"chunk_num = 0\n",
"start = time.time()\n",
"\n",
"for chunk in model.generate_streaming(text=long_text, cfg_value=2.0, inference_timesteps=10):\n",
" chunk_num += 1\n",
" all_chunks.append(chunk)\n",
" if chunk_num <= 3:\n",
" print(f\"๐ฆ Chunk #{chunk_num}: {len(chunk)} samples\")\n",
" elif chunk_num == 4:\n",
" print(\"... (more chunks) ...\")\n",
"\n",
"elapsed = time.time() - start\n",
"full_audio = np.concatenate(all_chunks)\n",
"output_path = \"/content/optional_streaming.wav\"\n",
"sf.write(output_path, full_audio, model.tts_model.sample_rate)\n",
"\n",
"duration = len(full_audio) / model.tts_model.sample_rate\n",
"print(f\"\\nโ
Streaming complete! {chunk_num} chunks | {duration:.1f}s audio\")\n",
"print(f\"โฑ๏ธ Total time: {elapsed:.1f}s\")\n",
"ipd.Audio(output_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### โก [OPTIONAL] Mode 7: Speed vs Quality Comparison (Code)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# OPTIONAL: Compare different timesteps (quality vs speed tradeoff)\n",
"test_text = \"Quality versus speed tradeoff demonstration.\"\n",
"timesteps_to_test = [5, 10, 20]\n",
"\n",
"for steps in timesteps_to_test:\n",
" print(f\"\\nโ๏ธ Testing timesteps={steps}\")\n",
" start = time.time()\n",
" wav = model.generate(text=test_text, cfg_value=2.0, inference_timesteps=steps)\n",
" elapsed = time.time() - start\n",
"\n",
" output = f\"/content/optional_quality_{steps}steps.wav\"\n",
" sf.write(output, wav, model.tts_model.sample_rate)\n",
"\n",
" print(f\"โ
Saved: {output}\")\n",
" print(f\"โฑ๏ธ Time: {elapsed:.1f}s\")\n",
" display(ipd.Audio(output))\n",
" print(\"-\"*50)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}