Upload 8 files

.gitattributes

@@ -35,3 +35,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer_config.json filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/logo.png filter=lfs diff=lfs merge=lfs -text
+assets/VoiceSculptor.png filter=lfs diff=lfs merge=lfs -text
+assets/wechat.png filter=lfs diff=lfs merge=lfs -text
+demo/demo_video.mov filter=lfs diff=lfs merge=lfs -text
+demo/demo_video.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,7 +1,178 @@
----
-license: apache-2.0
-base_model:
-- HKUSTAudio/Llasa-3B
-pipeline_tag: text-to-speech
----
-Github:https://github.com/ASLP-lab/VoiceSculptor
+<div align="center">
+    <h1>
+    VoiceSculptor
+    </h1>
+    <p>Official inference code for <br>
+    <b><em>VoiceSculptor: Your Voice, Designed By You</em></b>
+    </p>
+    <p>
+    <img src="assets/logo.png" style="width: 400px; height: 400px;">
+    </p>
+    <a href="https://hujingbin1.github.io/VoiceSculptor-Demo"><img src="https://img.shields.io/badge/Demo-Page-lightgrey" alt="version"></a>
+    <a href="https://huggingface.co/ASLP-lab/VoiceSculptor-VD"><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue' alt="HF-model"></a>
+    <a href="https://github.com/ASLP-lab/VoiceSculptor"><img src='https://img.shields.io/badge/Report-Github?label=Technical&color=red' alt="technical report"></a>
+    <a href="https://huggingface.co/ASLP-lab/VoiceSculptor-VD"><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-blue' alt="HF-demo"></a>
+    <a href="https://github.com/ASLP-lab/VoiceSculptor"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="Apache-2.0"></a>
+</div>
+
+## 📊 Instruct TTS Eval
+
+#### Instruct TTS Eval (ZH)
+
+| Model | APS (%) | DSD (%) | RP (%) | AVG (%) |
+|------|---------|---------|--------|---------|
+| Gemini 2.5-Flash* | 88.2 | 90.9 | 77.3 | 85.4 |
+| Gemini 2.5-Pro* | 89.0 | 90.1 | 75.5 | 84.8 |
+| GPT-4o-Mini-TTS* | 54.9 | 52.3 | 46.0 | 51.1 |
+| ElevenLabs* | 42.8 | 50.9 | 59.1 | 50.9 |
+| VoxInstruct | 47.5 | 52.3 | 42.6 | 47.5 |
+| MiMo-Audio-7B-Instruct | 70.1 | 66.1 | 57.1 | 64.5 |
+| **VoiceSculptor** | **75.7** | **64.7** | **61.5** | **67.6** |
+
+> **Note**
+>
+> - Models marked with `*` are commercial models.  
+> - **InstructTTSEval** — Huang, K., Tu, Q., Fan, L., Yang, C., Zhang, D., Li, S., Fei, Z., Cheng, Q., & Qiu, X. (2025).  
+>   *InstructTTSEval: Benchmarking Complex Natural-Language Instruction Following in Text-to-Speech Systems.*  
+>   arXiv preprint arXiv:2506.16381.  
+>   [arXiv](https://arxiv.org/abs/2506.16381)
+
+
+## ✨ Demo Video
+
+<div align="center">
+
+<https://github.com/user-attachments/assets/88c7230f-3ad8-4f37-b7fb-33a52f2b469d>
+
+</div>
+
+## 🔥 News
+
+
+- **[2026-1-2]** We opened the repository and uploaded the voice design models! [VoiceSculptor](https://huggingface.co/ASLP-lab/VoiceSculptor-VD)
+
+## 🚀 Getting Started
+
+### 1. Environment Setup
+
+Follow the steps below to clone the repository and install the required environment.
+
+```bash
+# Clone the repository and enter the directory
+git clone https://github.com/ASLP-lab/VoiceSculptor.git
+cd VoiceSculptor
+
+# Create and activate a Conda environment
+conda create -n VoiceSculptor python=3.10 -y
+conda activate VoiceSculptor
+
+# Install dependencies
+pip install -r requirements.txt
+```
+
+### 2. Download Pre-trained Models
+
+```bash
+git lfs install
+git clone https://huggingface.co/ASLP-lab/VoiceSculptor-VD
+```
+
+### 3. Infer
+
+For detailed instructions on how to design high-quality voice prompts,  
+please refer to [Voice Design Guide](docs/voice_design.md) or [Voice Design Guide EN](docs/voice_design_en.md).
+
+```bash
+python infer.py
+```
+
+<!-- ### 4. WebUI
+
+```bash
+python gradio.py
+```
+
+
+### 5. RAG
+
+```bash
+python build_rag.py
+``` -->
+
+
+## 📋 TODO
+- [x] 🌐 **Demo website**
+- [x] 🔓 **Release inference code**
+- [x] 🤗 **Release HuggingFace model**
+- [ ] 🤗 **HuggingFace Space**
+- [ ] 📝 **Release Technical Report**
+- [ ] 🔓 **Release gradio code**
+- [ ] 🔓 **Release RAG code**
+- [ ] 🔓 **Support vLLM**
+- [ ] 🔓 **Release training code**
+
+## Citation
+
+```bibtex
+@misc{VoiceSculptor,
+      title={VoiceSculptor: Your Voice, Designed By You},
+      author={Jingbin Hu and Huakang Chen and Linhan Ma and Dake Guo and Qirui Zhan and Wenhao Li and Haoyu Zhang and Kangxiang Xia and Ziyu Zhang and Wenjie Tian and Chengyou Wang and Jinrui Liang and Shuhan Guo and Zihang Yang and Bengu Wu and Binbin Zhang and Pengcheng Zhu and Pengyuan Xie and Chuan Xie and Qiang Zhang and Jie Liu and Lei Xie},
+      year={2026},
+      url={https://github.com/ASLP-lab/VoiceSculptor},
+}
+@misc{ye2025llasascalingtraintimeinferencetime,
+      title={Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis},
+      author={Zhen Ye and Xinfa Zhu and Chi-Min Chan and Xinsheng Wang and Xu Tan and Jiahe Lei and Yi Peng and Haohe Liu and Yizhu Jin and Zheqi Dai and Hongzhan Lin and Jianyi Chen and Xingjian Du and Liumeng Xue and Yunlin Chen and Zhifei Li and Lei Xie and Qiuqiang Kong and Yike Guo and Wei Xue},
+      year={2025},
+      eprint={2502.04128},
+      archivePrefix={arXiv},
+      primaryClass={eess.AS},
+      url={https://arxiv.org/abs/2502.04128},
+}
+```
+
+## License
+
+We use the Apache 2.0 license. Researchers and developers are free to use the codes and model weights of our VoiceSculptor. Check the license at [LICENSE](LICENSE.txt) for more details.
+
+## Acknowledgement
+- This repo benefits from [LLaSA](https://github.com/zhenye234/LLaSA_training)
+- This repo benefits from [CosyVoice](https://github.com/FunAudioLLM/CosyVoice)
+
+
+##  Usage Disclaimer
+Additional Notice on Generated Voices
+
+This project provides a speech synthesis model for voice design, intended for academic research, educational purposes, and legitimate applications, such as personalized speech synthesis, assistive technologies, and linguistic research.
+
+Please note:
+
+Do not use this model for unauthorized voice cloning, impersonation, fraud, scams, deepfakes, or any illegal or malicious activities.
+
+Ensure compliance with local laws and regulations when using this model and uphold ethical standards.
+
+The developers assume no liability for any misuse of this model.
+
+Important clarification regarding generated voices:
+
+As a generative model, the voices produced by this system are synthetic outputs inferred by the model, not recordings of real human voices.
+
+The generated voice characteristics do not represent or reproduce any specific real individual, and are not derived from or intended to imitate identifiable persons.
+
+We advocate for the responsible development and use of AI and encourage the community to uphold safety and ethical principles in AI research and applications. 
+
+## Contact Us
+If you are interested in leaving a message to our work, feel free to email jingbin.hu@mail.nwpu.edu.cn or lxie@nwpu.edu.cn
+
+You’re welcome to join our WeChat group for technical discussions, updates.
+<p align="center">
+  <!-- <em>Due to group limits, if you can't scan the QR code, please add my WeChat for group access  -->
+      <!-- : <strong>Tiamo James</strong></em> -->
+  <br>
+  <span style="display: inline-block; margin-right: 10px;">
+    <img src="assets/wechat.png" width="300" alt="WeChat Group QR Code"/>
+  </span>
+  <!-- <span style="display: inline-block;">
+    <img src="assets/wechat_tiamo.jpg" width="300" alt="WeChat QR Code"/>
+  </span> -->
+</p>

demo/demo_video.mov

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c45f92667db5b386e30aa76a7cab2b8dcd8c34e3e5f77fb34486987020c06e2
+size 6236134

demo/demo_video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ac81bc9989127a811f089bce9457ef55d60fd0eec3b1c1ea9783e79cc06a756
+size 5176511

docs/voice_design.md

@@ -0,0 +1,101 @@
+# 音色设计指南（Voice Design README）
+
+## 概述
+本指南用于创建高质量的声音描述（voice_prompt），帮助生成符合特定需求的理想音色。声音描述是声音设计的蓝图，直接决定模型输出的音色质量。
+
+---
+
+## 技术约束
+| 项目 | 说明 |
+|------|------|
+| **长度限制** | 单个 voice_prompt ≤ 200 个字符 |
+| **支持语言** | 中文，当前版本暂不支持英文，后续会更新|
+
+---
+
+## 五大核心原则
+
+### 1️⃣ 具体而非模糊
+**✅ 正确做法**：使用可感知的声音特质词
+- 音调：低沉、高亢、清脆、浑厚
+- 语速：偏快、缓慢、急促、平缓
+- 质感：磁性、沙哑、圆润、明亮
+
+**❌ 错误示例**："好听"、"普通"、"不错"（过于主观，缺乏信息量）
+
+### 2️⃣ 多维而非单一
+**✅ 正确做法**：至少组合3-4个维度构建立体声像
+- 人设(适用场景) + 性别 + 年龄 + 音调 + 语速 + 音量 + 音质 + 情感
+
+**❌ 错误示例**：仅"女声"或仅"低沉"（过于宽泛，特色不鲜明）
+
+### 3️⃣ 客观而非主观
+**✅ 正确做法**：描述声音物理特征
+- "音调偏高，带活力"
+- "语速缓慢，吐字清晰"
+
+**❌ 错误示例**："我最喜欢的声音"、"这个声音很棒"
+
+### 4️⃣ 原创而非模仿
+**⚠️ 版权风险**：禁止使用"模仿XX明星"、"像XX演员"等描述
+**✅ 正确做法**：描述声音特质本身，而非特定人物
+
+### 5️⃣ 简洁而非冗余
+**✅ 正确做法**：每个词都承载信息
+**❌ 错误示例**："非常非常棒的声音"、"极其极其温柔"
+
+---
+
+## 描述维度参考
+基于优秀案例，建议从以下维度组合描述：
+
+| 维度 | 可选项示例 |
+|------|----------|
+| **人设（适用场景）** | 新闻播报、广告配音、有声书、动画角色、纪录片解说 |
+| **性别** | 男性、女性|
+| **年龄** | 儿童(约8岁)、青年(20-30岁)、中年(40-50岁)、老年 |
+| **性格特质** | 活泼、沉稳、温柔、知性、可爱、严肃 |
+| **语速节奏** | 较快、缓慢、适中、急促、平缓 |
+| **语调特点** | 上扬、平和、激昂、慵懒 |
+| **音色质感** | 低沉磁性、清脆明亮、沙哑圆润、稚气 |
+
+---
+
+## 优秀案例
+
+### ✅ 推荐模板
+
+**案例1：诗歌朗诵**
+> "一位男性现代诗朗诵者，用深沉磁性的低音，以顿挫有力的节奏演绎艾青诗歌，音量洪亮，情感激昂澎湃。"
+
+**案例2：新闻风格**
+> "这是一位女性新闻主播，用标准普通话以清晰明亮的中高音，以平稳专业的语速播报时事新闻，音量洪亮，情感客观中立。"
+
+**案例3：广告配音**
+> "这是一位男性白酒品牌广告配音，用沧桑浑厚的嗓音，以缓慢而豪迈的语速，音量洪亮，传递历史底蕴和男人情怀。"
+
+---
+
+## 常见错误与改进
+
+| 类型 | ❌ 不推荐描述 | ✅ 改进建议 |
+|------|------------|----------|
+| **过于笼统** | "女声，好听" | "年轻女性，音调清脆，语速适中" |
+| **主观评价** | "很棒的声音" | "音色明亮，富有感染力" |
+| **单一维度** | "低沉的男声" | "中年男性，音色低沉，语速缓慢，适合纪录片" |
+| **冗余重复** | "非常非常温柔的声音" | "温柔知性的女性声音" |
+| **模仿请求** | "像XX明星的声音" | "**禁止**，请用客观特质词描述 |
+
+---
+
+## 快速检查清单
+在提交 voice_prompt 前，请确认：
+- [ ] 长度 ≤ 200 字符
+- [ ] 包含至少3个不同维度
+- [ ] 无主观评价词（好听/很棒/喜欢）
+- [ ] 无模仿人物描述
+- [ ] 无重复强调词
+- [ ] 明确了适用场景
+- [ ] 所有词汇均可感知、具象化
+
+---

docs/voice_design_en.md

@@ -0,0 +1,129 @@
+# Voice Design Guide (Voice Design README)
+
+## Overview
+
+This guide provides instructions for creating high-quality **voice descriptions (voice_prompt)** to generate voices that meet specific requirements. The voice description serves as the blueprint for voice design and directly determines the quality of the generated voice.
+
+---
+
+## Technical Constraints
+
+| Item                    | Description                                                                                       |
+| ----------------------- | ------------------------------------------------------------------------------------------------- |
+| **Length Limit**        | Each voice_prompt ≤ 200 characters                                                                |
+| **Supported Languages** | Chinese only. English is not supported in the current version and will be added in future updates |
+
+---
+
+## Five Core Principles
+
+### 1️⃣ Be Specific, Not Vague
+
+**✅ Recommended**: Use perceptible and concrete voice attributes
+
+* Pitch: low, high, bright, rich
+* Speaking rate: fast, slow, rapid, steady
+* Timbre: magnetic, husky, smooth, clear
+
+**❌ Avoid**:
+“Nice”, “normal”, “good” (too subjective and uninformative)
+
+---
+
+### 2️⃣ Multi-Dimensional, Not Single-Attribute
+
+**✅ Recommended**: Combine at least 3–4 dimensions to create a vivid voice profile
+
+* Persona (usage scenario) + gender + age + pitch + speaking rate + volume + timbre + emotion
+
+**❌ Avoid**:
+Only “female voice” or only “low-pitched” (too generic, lacks distinctiveness)
+
+---
+
+### 3️⃣ Objective, Not Subjective
+
+**✅ Recommended**: Describe physical and acoustic characteristics
+
+* “Slightly high-pitched with energetic delivery”
+* “Slow speaking rate with clear articulation”
+
+**❌ Avoid**:
+“My favorite voice”, “This voice sounds great”
+
+---
+
+### 4️⃣ Original, Not Imitative
+
+**⚠️ Copyright Notice**: Descriptions such as “sounds like XX celebrity” or “imitates XX actor” are prohibited
+**✅ Recommended**: Describe voice characteristics directly rather than referencing specific individuals
+
+---
+
+### 5️⃣ Concise, Not Redundant
+
+**✅ Recommended**: Ensure every word conveys meaningful information
+**❌ Avoid**:
+“Very, very good voice”, “Extremely, extremely gentle”
+
+---
+
+## Reference Dimensions for Voice Description
+
+Based on high-quality examples, we recommend composing voice prompts using the following dimensions:
+
+| Dimension                    | Example Options                                                                                   |
+| ---------------------------- | ------------------------------------------------------------------------------------------------- |
+| **Persona (Usage Scenario)** | News broadcasting, advertising voice-over, audiobooks, animated characters, documentary narration |
+| **Gender**                   | Male, Female                                                                                      |
+| **Age**                      | Child (~8 years), Young adult (20–30), Middle-aged (40–50), Elderly                               |
+| **Personality Traits**       | Lively, calm, gentle, intellectual, cute, serious                                                 |
+| **Speaking Rate & Rhythm**   | Fast, slow, moderate, urgent, steady                                                              |
+| **Intonation Style**         | Rising, neutral, passionate, relaxed                                                              |
+| **Timbre**                   | Deep and magnetic, crisp and bright, husky and warm, youthful                                     |
+
+---
+
+## High-Quality Examples
+
+### ✅ Recommended Templates
+
+**Example 1: Poetry Recitation**
+
+> “A male modern poetry reciter with a deep, magnetic low voice, delivering poetry with strong rhythmic pauses, powerful volume, and intense emotional expression.”
+
+**Example 2: News Style**
+
+> “A female news anchor speaking standard Mandarin with a clear and bright mid-to-high pitch, steady professional pacing, strong volume, and a neutral, objective tone.”
+
+**Example 3: Advertising Voice-Over**
+
+> “A male voice for liquor brand advertising, featuring a rich and weathered timbre, slow and bold speaking rate, strong volume, conveying a sense of history and masculinity.”
+
+---
+
+## Common Mistakes and Improvements
+
+| Type                      | ❌ Not Recommended          | ✅ Improved Version                                                           |
+| ------------------------- | -------------------------- | ---------------------------------------------------------------------------- |
+| **Too Generic**           | “Female voice, nice”       | “Young female voice with a clear pitch and moderate speaking rate”           |
+| **Subjective Evaluation** | “A great-sounding voice”   | “Bright timbre with strong expressiveness”                                   |
+| **Single Dimension**      | “Low-pitched male voice”   | “Middle-aged male with a low pitch, slow pacing, suitable for documentaries” |
+| **Redundant Wording**     | “Very, very gentle voice”  | “Gentle and intellectual female voice”                                       |
+| **Imitation Request**     | “Sounds like XX celebrity” | **Prohibited** — describe objective voice traits instead                     |
+
+---
+
+## Quick Checklist
+
+Before submitting a voice_prompt, make sure that:
+
+* [ ] Length ≤ 200 characters
+* [ ] At least 3 different descriptive dimensions are included
+* [ ] No subjective evaluation words (e.g., “nice”, “great”, “favorite”)
+* [ ] No references to real individuals or imitation requests
+* [ ] No repetitive or exaggerated wording
+* [ ] Usage scenario is clearly defined
+* [ ] All descriptors are perceptible and concrete
+
+---