moonbox-dev/VoxCPM2-MLX

VoxCPM2 MLX

MLX builds of VoxCPM2, made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's files together. Only the parts listed below are converted: the rest of the pipeline comes from the original repository unless the section below says otherwise.

Files

The last column is how far each build is from the original PyTorch model on the same inputs (see the section below for what was measured).

Precision	File	Size	sha256	Checked against the original
bf16	bf16/config.json	6.6 kB	0b5bee352207	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.967, DNSMOS 3.12 / 2.33 (original 0.0 % / 0.0 %, 0.972 / 0.964, 2.93 / 2.28)
bf16	bf16/model.safetensors	4.77 GB	8ec57a6a9761	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.967, DNSMOS 3.12 / 2.33 (original 0.0 % / 0.0 %, 0.972 / 0.964, 2.93 / 2.28)
bf16	bf16/model.safetensors.index.json	65.7 kB	1a4680fb582f	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.967, DNSMOS 3.12 / 2.33 (original 0.0 % / 0.0 %, 0.972 / 0.964, 2.93 / 2.28)
bf16	bf16/special_tokens_map.json	1.6 kB	068594063e37	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.967, DNSMOS 3.12 / 2.33 (original 0.0 % / 0.0 %, 0.972 / 0.964, 2.93 / 2.28)
bf16	bf16/tokenizer.json	3.7 MB	f8984687e4a9	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.967, DNSMOS 3.12 / 2.33 (original 0.0 % / 0.0 %, 0.972 / 0.964, 2.93 / 2.28)
bf16	bf16/tokenizer_config.json	5.0 kB	418be464e38f	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.967, DNSMOS 3.12 / 2.33 (original 0.0 % / 0.0 %, 0.972 / 0.964, 2.93 / 2.28)
fp16	fp16/config.json	6.6 kB	0b5bee352207	WER 0.0 % / CER 0.0 %, speaker sim 0.970 / 0.967, DNSMOS 3.06 / 2.11
fp16	fp16/model.safetensors	4.77 GB	88b5a05243e9	WER 0.0 % / CER 0.0 %, speaker sim 0.970 / 0.967, DNSMOS 3.06 / 2.11
fp16	fp16/model.safetensors.index.json	65.7 kB	1a4680fb582f	WER 0.0 % / CER 0.0 %, speaker sim 0.970 / 0.967, DNSMOS 3.06 / 2.11
fp16	fp16/special_tokens_map.json	1.6 kB	068594063e37	WER 0.0 % / CER 0.0 %, speaker sim 0.970 / 0.967, DNSMOS 3.06 / 2.11
fp16	fp16/tokenizer.json	3.7 MB	f8984687e4a9	WER 0.0 % / CER 0.0 %, speaker sim 0.970 / 0.967, DNSMOS 3.06 / 2.11
fp16	fp16/tokenizer_config.json	5.0 kB	418be464e38f	WER 0.0 % / CER 0.0 %, speaker sim 0.970 / 0.967, DNSMOS 3.06 / 2.11
8bit	8bit/config.json	6.8 kB	775a1f6b74b1	WER 0.0 % / CER 0.0 %, speaker sim 0.975 / 0.966, DNSMOS 3.12 / 2.31
8bit	8bit/model.safetensors	2.62 GB	818a3bb5564d	WER 0.0 % / CER 0.0 %, speaker sim 0.975 / 0.966, DNSMOS 3.12 / 2.31
8bit	8bit/model.safetensors.index.json	0.1 MB	1f3b958dfdf9	WER 0.0 % / CER 0.0 %, speaker sim 0.975 / 0.966, DNSMOS 3.12 / 2.31
8bit	8bit/special_tokens_map.json	1.6 kB	068594063e37	WER 0.0 % / CER 0.0 %, speaker sim 0.975 / 0.966, DNSMOS 3.12 / 2.31
8bit	8bit/tokenizer.json	3.7 MB	f8984687e4a9	WER 0.0 % / CER 0.0 %, speaker sim 0.975 / 0.966, DNSMOS 3.12 / 2.31
8bit	8bit/tokenizer_config.json	5.0 kB	418be464e38f	WER 0.0 % / CER 0.0 %, speaker sim 0.975 / 0.966, DNSMOS 3.12 / 2.31
6bit	6bit/config.json	6.8 kB	13c7d8db7354	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.966, DNSMOS 3.07 / 2.55
6bit	6bit/model.safetensors	2.05 GB	90c38a61655b	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.966, DNSMOS 3.07 / 2.55
6bit	6bit/model.safetensors.index.json	0.1 MB	312445674ab9	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.966, DNSMOS 3.07 / 2.55
6bit	6bit/special_tokens_map.json	1.6 kB	068594063e37	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.966, DNSMOS 3.07 / 2.55
6bit	6bit/tokenizer.json	3.7 MB	f8984687e4a9	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.966, DNSMOS 3.07 / 2.55
6bit	6bit/tokenizer_config.json	5.0 kB	418be464e38f	WER 0.0 % / CER 0.0 %, speaker sim 0.973 / 0.966, DNSMOS 3.07 / 2.55
4bit	4bit/config.json	6.8 kB	096014d84678	WER 0.0 % / CER 0.0 %, speaker sim 0.972 / 0.964, DNSMOS 3.12 / 2.22
4bit	4bit/model.safetensors	1.48 GB	54ad41f92a33	WER 0.0 % / CER 0.0 %, speaker sim 0.972 / 0.964, DNSMOS 3.12 / 2.22
4bit	4bit/model.safetensors.index.json	0.1 MB	90e318b263fd	WER 0.0 % / CER 0.0 %, speaker sim 0.972 / 0.964, DNSMOS 3.12 / 2.22
4bit	4bit/special_tokens_map.json	1.6 kB	068594063e37	WER 0.0 % / CER 0.0 %, speaker sim 0.972 / 0.964, DNSMOS 3.12 / 2.22
4bit	4bit/tokenizer.json	3.7 MB	f8984687e4a9	WER 0.0 % / CER 0.0 %, speaker sim 0.972 / 0.964, DNSMOS 3.12 / 2.22
4bit	4bit/tokenizer_config.json	5.0 kB	418be464e38f	WER 0.0 % / CER 0.0 %, speaker sim 0.972 / 0.964, DNSMOS 3.12 / 2.22

How to run

VoxCPM2 (2B, tokenizer-free multilingual TTS with 48 kHz output: zero-shot speech, voice design and voice cloning) for mlx-audio. Each folder is a complete mlx-audio model: model.safetensors (the MiniCPM backbone, residual LM, local encoder, flow-matching DiT and the AudioVAE), config.json and the tokenizer. Tested with mlx-audio==0.6.5:

from mlx_audio.tts.utils import load_model
import numpy as np, soundfile as sf

model = load_model("8bit")              # a downloaded folder
res = list(model.generate(text="Hello from Moonbox.",
                          ref_audio="voice.wav",                                  # voice cloning
                          prompt_audio="voice.wav", prompt_text="Exact transcript of voice.wav."))
sf.write("out.wav", np.concatenate([np.array(r.audio) for r in res]), res[0].sample_rate)   # 48 kHz

Voice design: model.generate(text=..., instruct="A young woman, warm and gentle voice"). Defaults are the original's: 10 flow-matching steps, CFG 2.0.

The tokenizer is the plain Llama tokenizer: mlx-audio splits multi-character Chinese tokens itself, which the original repository does with remote code (tokenization_voxcpm2.py, left out here). 8bit/, 6bit/ and 4bit/ quantize the transformer layers (MLX affine, group size 64); the AudioVAE's convolutions stay bf16.

How they were checked. Voice cloning of 6 English and 4 Chinese sentences, two seeds each (the prompt both as reference audio and as continuation prefix), with the original PyTorch model (float32, voxcpm 2.0.3, its bad-case retry off) and every MLX build. Judged like the original: Whisper large-v3-turbo WER (English) / CER (Chinese), WavLM-base-plus-sv speaker similarity to the prompt, DNSMOS P.835. Pass: error rate at most 2 points above the original's, similarity at most 0.03 below, DNSMOS at most 0.2 below. The Chinese prompt is a noisy AISHELL clip, which the model reproduces (hence the low Chinese DNSMOS for every system).

Precision	Size	WER en / CER zh	Speaker similarity en / zh	DNSMOS en / zh	Real-time factor, M5 24 GB
original (PyTorch fp32, MPS)	-	0.0 % / 0.0 %	0.972 / 0.964	2.93 / 2.28	1.89
bf16/	4.77 GB	0.0 % / 0.0 %	0.973 / 0.967	3.12 / 2.33	1.08
fp16/	4.77 GB	0.0 % / 0.0 %	0.970 / 0.967	3.06 / 2.11	1.12
8bit/	2.62 GB	0.0 % / 0.0 %	0.975 / 0.966	3.12 / 2.31	1.29
6bit/	2.05 GB	0.0 % / 0.0 %	0.973 / 0.966	3.07 / 2.55	1.30
4bit/	1.48 GB	0.0 % / 0.0 %	0.972 / 0.964	3.12 / 2.22	1.14

Real-time factor = generation time / audio length (lower is faster), one sentence at a time.

How these were made

From VoxCPM2 at revision 7936080b41: the original weights loaded in MLX and stored as bf16 or fp16, or quantized with MLX's affine quantization (8bit, 6bit, 4bit, group size 64; layers MLX can't quantize, such as convolutions and norms, stay bf16). Every build was checked against the original before upload; the sha256 column above is what the files must hash to.

License

Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.

---

Original model card

VoxCPM2

VoxCPM2 is a tokenizer-free, diffusion autoregressive Text-to-Speech model — 2B parameters, 30 languages, 48kHz audio output, trained on over 2 million hours of multilingual speech data.

Highlights

• 🌍 30-Language Multilingual — No language tag needed; input text in any supported language directly
• 🎨 Voice Design — Generate a novel voice from a natural-language description alone (gender, age, tone, emotion, pace…); no reference audio required
• 🎛️ Controllable Cloning — Clone any voice from a short clip, with optional style guidance to steer emotion, pace, and expression while preserving timbre
• 🎙️ Ultimate Cloning — Provide reference audio + its transcript for audio-continuation cloning; every vocal nuance faithfully reproduced
• 🔊 48kHz Studio-Quality Output — Accepts 16kHz reference; outputs 48kHz via AudioVAE V2's built-in super-resolution, no external upsampler needed
• 🧠 Context-Aware Synthesis — Automatically infers appropriate prosody and expressiveness from text content
• ⚡ Real-Time Streaming — RTF as low as ~0.3 on NVIDIA RTX 4090, and ~0.13 accelerated by Nano-VLLM
• 📜 Fully Open-Source & Commercial-Ready — Apache-2.0 license, free for commercial use

Supported Languages (30)

Arabic, Burmese, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, Vietnamese

Chinese Dialects: 四川话, 粤语, 吴语, 东北话, 河南话, 陕西话, 山东话, 天津话, 闽南话

Quick Start

Installation

pip install voxcpm

Requirements: Python ≥ 3.10, PyTorch ≥ 2.5.0, CUDA ≥ 12.0 · Full Quick Start →

Text-to-Speech

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained("openbmb/VoxCPM2", load_denoiser=False)

wav = model.generate(
    text="VoxCPM2 brings multilingual support, creative voice design, and controllable voice cloning.",
    cfg_value=2.0,
    inference_timesteps=10,
)
sf.write("output.wav", wav, model.tts_model.sample_rate)

Voice Design

Put the voice description in parentheses at the start of text, followed by the content to synthesize:

wav = model.generate(
    text="(A young woman, gentle and sweet voice)Hello, welcome to VoxCPM2!",
    cfg_value=2.0,
    inference_timesteps=10,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)

Controllable Voice Cloning

# Basic cloning
wav = model.generate(
    text="This is a cloned voice generated by VoxCPM2.",
    reference_wav_path="speaker.wav",
)
sf.write("clone.wav", wav, model.tts_model.sample_rate)

# Cloning with style control
wav = model.generate(
    text="(slightly faster, cheerful tone)This is a cloned voice with style control.",
    reference_wav_path="speaker.wav",
    cfg_value=2.0,
    inference_timesteps=10,
)
sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)

Ultimate Cloning

Provide both the reference audio and its exact transcript for maximum fidelity. Pass the same clip to both reference_wav_path and prompt_wav_path for highest similarity:

wav = model.generate(
    text="This is an ultimate cloning demonstration using VoxCPM2.",
    prompt_wav_path="speaker_reference.wav",
    prompt_text="The transcript of the reference audio.",
    reference_wav_path="speaker_reference.wav",
)
sf.write("hifi_clone.wav", wav, model.tts_model.sample_rate)

Streaming

import numpy as np

chunks = []
for chunk in model.generate_streaming(text="Streaming is easy with VoxCPM!"):
    chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("streaming.wav", wav, model.tts_model.sample_rate)

Model Details

Property	Value
Architecture	Tokenizer-free Diffusion Autoregressive (LocEnc → TSLM → RALM → LocDiT)
Backbone	Based on MiniCPM-4, totally 2B parameters
Audio VAE	AudioVAE V2 (asymmetric encode/decode, 16kHz in → 48kHz out)
Training Data	2M+ hours multilingual speech
LM Token Rate	6.25 Hz
Max Sequence Length	8192 tokens
dtype	bfloat16
VRAM	~8 GB
RTF (RTX 4090)	~0.30 (standard) / ~0.13 (Nano-vLLM)

Performance

VoxCPM2 achieves state-of-the-art or competitive results on major zero-shot and controllable TTS benchmarks.

See the GitHub repo for full benchmark tables (Seed-TTS-eval, CV3-eval, InstructTTSEval, MiniMax Multilingual Test).

Fine-tuning

VoxCPM2 supports both full SFT and LoRA fine-tuning with as little as 5–10 minutes of audio:

# LoRA fine-tuning (recommended)
python scripts/train_voxcpm_finetune.py \
    --config_path conf/voxcpm_v2/voxcpm_finetune_lora.yaml

# Full fine-tuning
python scripts/train_voxcpm_finetune.py \
    --config_path conf/voxcpm_v2/voxcpm_finetune_all.yaml

See the Fine-tuning Guide for full instructions.

Limitations

• Voice Design and Style Control results may vary between runs; generating 1–3 times is recommended to obtain the desired output.
• Performance varies across languages depending on training data availability.
• Occasional instability may occur with very long or highly expressive inputs.
• Strictly forbidden to use for impersonation, fraud, or disinformation. AI-generated content should be clearly labeled.

Citation

@article{voxcpm2_2026,
  title   = {VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning},
  author  = {VoxCPM Team},
  journal = {GitHub},
  year    = {2026},
}

@article{voxcpm2025,
  title   = {VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning},
  author  = {Zhou, Yixuan and Zeng, Guoyang and Liu, Xin and Li, Xiang and
             Yu, Renjie and Wang, Ziyang and Ye, Runchuan and Sun, Weiyue and
             Gui, Jiancheng and Li, Kehan and Wu, Zhiyong and Liu, Zhiyuan},
  journal = {arXiv preprint arXiv:2509.24650},
  year    = {2025},
}

License

Released under the Apache-2.0 license, free for commercial use. For production deployments, we recommend thorough testing and safety evaluation tailored to your use case.