license: cc-by-nc-sa-4.0
language:
- zh
- en
pipeline_tag: audio-to-audio
tags:
- speech
- voice-conversion
- whisper-to-normal
- whispervc
base_model:
- openai/whisper-large-v3
WhisperVC
Official Chinese and English inference checkpoints for WhisperVC: Decoupled Cross-Domain Alignment and Speech Generation for Low-Resource Whisper-to-Normal Conversion.
Paper: https://arxiv.org/abs/2511.01056
Model files
whispervc_cn_v1.safetensors: Chinese inference bundle.whispervc_en_v1.safetensors: English inference bundle.
Each file contains the Stage2 model, HiFi-GAN generator, Gate, and Whisper Flamingo encoder/adapter weights. The WeSpeaker checkpoint is not included and must be obtained separately from its official release.
Intended use
The checkpoints are provided for non-commercial research and educational use, including whisper-to-normal speech conversion and voice-conversion research. Commercial use is not permitted by the model license.
Training data
- AISHELL6-Whisper
- wTIMIT
- LibriTTS
No training, evaluation, or baseline data is included in this repository.
Access and download
This repository uses automatic gated access. Request access on this page, authenticate locally, and download the required bundle:
hf auth login
hf download RaydonLiu/WhisperVC whispervc_cn_v1.safetensors
hf download RaydonLiu/WhisperVC whispervc_en_v1.safetensors
License
The model weights are licensed under CC BY-NC-SA 4.0. The accompanying WhisperVC inference source code is licensed separately under Apache-2.0. Third-party models and dependencies retain their upstream licenses.
Citation
@article{liu2025whispervc,
title={WhisperVC: Decoupled Cross-Domain Alignment and Speech Generation for Low-Resource Whisper-to-Normal Conversion},
author={Liu, Dong and Liu, Juan and Ju, Wei and Tian, Yao and Li, Ming},
journal={arXiv preprint arXiv:2511.01056},
year={2025}
}