| --- |
| license: cc-by-nc-sa-4.0 |
| language: |
| - zh |
| - en |
| pipeline_tag: audio-to-audio |
| tags: |
| - speech |
| - voice-conversion |
| - whisper-to-normal |
| - whispervc |
| base_model: |
| - openai/whisper-large-v3 |
| --- |
| |
| # WhisperVC |
|
|
| Official Chinese and English inference checkpoints for **WhisperVC: Decoupled Cross-Domain Alignment and Speech Generation for Low-Resource Whisper-to-Normal Conversion**. |
|
|
| Paper: https://arxiv.org/abs/2511.01056 |
|
|
| ## Model files |
|
|
| - `whispervc_cn_v1.safetensors`: Chinese inference bundle. |
| - `whispervc_en_v1.safetensors`: English inference bundle. |
|
|
| Each file contains the Stage2 model, HiFi-GAN generator, Gate, and Whisper Flamingo encoder/adapter weights. The WeSpeaker checkpoint is not included and must be obtained separately from its official release. |
|
|
| ## Intended use |
|
|
| The checkpoints are provided for non-commercial research and educational use, including whisper-to-normal speech conversion and voice-conversion research. Commercial use is not permitted by the model license. |
|
|
| ## Training data |
|
|
| - AISHELL6-Whisper |
| - wTIMIT |
| - LibriTTS |
|
|
| No training, evaluation, or baseline data is included in this repository. |
|
|
| ## Access and download |
|
|
| This repository uses automatic gated access. Request access on this page, authenticate locally, and download the required bundle: |
|
|
| ```bash |
| hf auth login |
| hf download RaydonLiu/WhisperVC whispervc_cn_v1.safetensors |
| hf download RaydonLiu/WhisperVC whispervc_en_v1.safetensors |
| ``` |
|
|
| ## License |
|
|
| The model weights are licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). The accompanying WhisperVC inference source code is licensed separately under Apache-2.0. Third-party models and dependencies retain their upstream licenses. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{liu2025whispervc, |
| title={WhisperVC: Decoupled Cross-Domain Alignment and Speech Generation for Low-Resource Whisper-to-Normal Conversion}, |
| author={Liu, Dong and Liu, Juan and Ju, Wei and Tian, Yao and Li, Ming}, |
| journal={arXiv preprint arXiv:2511.01056}, |
| year={2025} |
| } |
| ``` |
|
|
