| --- |
| license: apache-2.0 |
| base_model: openbmb/VoxCPM2 |
| pipeline_tag: text-to-speech |
| library_name: ncnn |
| tags: |
| - voxcpm2 |
| - ncnn |
| - text-to-speech |
| - speech-synthesis |
| - voice-cloning |
| - edge-ai |
| language: |
| - ar |
| - my |
| - zh |
| - da |
| - nl |
| - en |
| - fi |
| - fr |
| - de |
| - el |
| - he |
| - hi |
| - id |
| - it |
| - ja |
| - km |
| - ko |
| - lo |
| - ms |
| - "no" |
| - pl |
| - pt |
| - ru |
| - es |
| - sw |
| - sv |
| - tl |
| - th |
| - tr |
| - vi |
| --- |
| |
| # VoxCPM2 NCNN |
|
|
| This repository contains an NCNN export of [openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) for use with the `voxcpm2-ncnn` C++ runtime. |
|
|
| It is a converted runtime asset package, not a newly trained or fine-tuned model. The model weights keep the same Apache-2.0 license as the upstream VoxCPM2 release. |
|
|
| Model repository: <https://huggingface.co/lyrin/voxpm2-ncnn> |
|
|
| Runtime source: <https://github.com/LiYulin-s/voxcpm2-ncnn.git> |
|
|
| ## Model Details |
|
|
| - Base model: `openbmb/VoxCPM2` |
| - Format: fp16 NCNN `.param` / `.bin` component graphs |
| - Task: multilingual text-to-speech |
| - Output audio: 48 kHz mono PCM, written by the runtime through FFmpeg |
| - Runtime target: `voxcpm2-ncnn` |
| - License: Apache-2.0 for the model assets |
|
|
| VoxCPM2 is a multilingual controllable speech generation model. The upstream release describes support for 30 languages, 9 Chinese dialects, voice design, style-controllable voice cloning, and high-fidelity continuation cloning. This NCNN package targets the modes currently exposed by the `voxcpm2-ncnn` runtime. |
|
|
| ## Files |
|
|
| The exported model directory contains the runtime assets and a local `LICENSE` copy. The repository root keeps an additional `LICENSE` copy for hosting tools that expect the license at the top level. |
|
|
| - `model.json`: NCNN component manifest and runtime settings |
| - `*.ncnn.param`, `*.ncnn.bin`: exported NCNN component graphs and weights |
| - `tokenizer.json`: tokenizer asset used by the runtime |
| - `LICENSE`: Apache-2.0 license text for the model assets |
|
|
| Export-time intermediate files such as TorchScript, PNNX graphs, and generated Python wrappers are intentionally not included. |
|
|
| ## Usage |
|
|
| Download the model assets into the runtime repository: |
|
|
| ```bash |
| huggingface-cli download lyrin/voxpm2-ncnn --local-dir assets/voxcpm2 |
| ``` |
|
|
| Smoke-test the NCNN components: |
|
|
| ```bash |
| xmake run voxcpm2 -m assets/voxcpm2 --smoke-components |
| xmake run voxcpm2 -m assets/voxcpm2 --smoke-components --vulkan |
| ``` |
|
|
| Generate speech from text: |
|
|
| ```bash |
| xmake run voxcpm2 -m assets/voxcpm2 \ |
| -t "你好，欢迎使用 VoxCPM2 NCNN。" \ |
| -o out.wav |
| ``` |
|
|
| Use prompt continuation with prompt audio: |
|
|
| ```bash |
| xmake run voxcpm2 -m assets/voxcpm2 \ |
| -t "这是续写测试。" \ |
| --prompt "你好。" \ |
| --prompt-audio prompt.wav \ |
| -o out.wav |
| ``` |
|
|
| Use reference audio: |
|
|
| ```bash |
| xmake run voxcpm2 -m assets/voxcpm2 \ |
| -t "这是参考音频测试。" \ |
| --reference-audio reference.wav \ |
| -o out.flac |
| ``` |
|
|
| The output format is inferred from the `-o` extension. |
|
|
| ## Conversion Notes |
|
|
| This package splits VoxCPM2 into NCNN component graphs: |
|
|
| - text embedding |
| - base and residual decoder steps |
| - FSQ and projection layers |
| - DiT estimator |
| - stop-token head |
| - AudioVAE encoder and decoder |
|
|
| The runtime uses a page-style KV cache internally while adapting to the current exported decoder-step NCNN graphs. |
|
|
| ## Limitations |
|
|
| - This is a conversion package; numerical behavior and performance can differ from the upstream PyTorch runtime. |
| - Not all upstream inference modes are necessarily exposed by the C++ runtime. |
| - Quality, speaker similarity, latency, and memory use depend on the NCNN build, device, Vulkan driver, and input audio quality. |
| - Generated speech and voice cloning should be used only with appropriate rights, consent, and safety review. |
|
|
| ## Attribution |
|
|
| The original VoxCPM2 model is by OpenBMB / ModelBest. Please refer to the upstream [VoxCPM2 model card](https://huggingface.co/openbmb/VoxCPM2), [project repository](https://github.com/OpenBMB/VoxCPM), and [technical report](https://arxiv.org/abs/2606.06928) for model architecture, training, evaluation, and intended-use details. |
|
|
| ## License |
|
|
| The model assets in this repository are released under Apache-2.0, matching the upstream VoxCPM2 release. The license text is included in `LICENSE`. |
|
|
