README.md

metadata

title: YingMusic-Singer
emoji: 🎤
colorFrom: pink
colorTo: blue
sdk: gradio
python_version: '3.10'
app_file: app.py
tags:
  - singing-voice-synthesis
  - lyric-editing
  - diffusion-model
  - reinforcement-learning
short_description: Edit lyrics, keep the melody
fullWidth: true

YingMusic-Singer

YingMusic-Singer: Controllable Singing Voice Synthesis with Flexible Lyric Manipulation and Annotation-free Melody Guidance

Environment Setup

1. Install from Scratch

conda create -n YingMusic-Singer python=3.10
conda activate YingMusic-Singer

# uv is much quicker
pip install uv
uv pip install -r requirements.txt

2. Pre-built Conda Environment for One-Click Deployment (Nvidia / AMD CPU Only)

Coming soon

推理

使用huggingface Space（线上体验）

访问https://huggingface.co/spaces/ASLP-lab/YingMusic-Singer之后，就可以快速体验

使用Docker运行

docker build -t yingmusic-singer .

使用python运行

git clone cd python initialization.py --task infer

for Gradio

python app.py

多进程 Inference

1. 你需要确保所有输入模型的均为分离之后的纯人声，如果没有分离，可以参考/src/third_party/MusicSourceSeparationTraining/inference_api.py 进行分离

2. jsonl 文件的格式为，每行一个json，{}

python batch_infer.py
--input_type jsonl
--input_path /path/to/input.jsonl
--output_dir /path/to/output
--ckpt_path /path/to/ckpts
--num_gpus 4

多进程 Inference(LyricEditBench melody control)

python inference_mp.py
--input_type lyric_edit_bench_melody_control
--output_dir path/to/
LyricEditBench_melody_control
--ckpt_path ASLP-lab/YingMusic-Singer
--num_gpus 8

多进程 Inference(LyricEditBench sing edit)

python inference_mp.py
--input_type lyric_edit_bench_sing_edit
--output_dir path/to/
LyricEditBench_melody_control
--ckpt_path ASLP-lab/YingMusic-Singer
--num_gpus 8

License

The code and model weights in this project are licensed under CC BY 4.0, except for the following components:

The VAE model weights and inference code (in src/YingMusic-Singer/utils/stable-audio-tools) are derived from Stable Audio Open by Stability AI, and are licensed under the Stability AI Community License.