Metacebertrunk commited on
Commit
3ec18ce
·
verified ·
1 Parent(s): dd528a4

Update README.md

Browse files

Update project info

Files changed (1) hide show
  1. README.md +52 -24
README.md CHANGED
@@ -1,35 +1,63 @@
1
- ---
2
- license: apache-2.0
3
- title: QuarkAudio
4
- sdk: gradio
5
- emoji: 🚀
6
- colorFrom: blue
7
- colorTo: yellow
8
- pinned: true
9
- short_description: Audio processing and generation
10
- sdk_version: 6.0.2
11
- ---
12
  # QuarkAudio: An Open-Source Project to Unify Audio Processing and Generation
13
 
14
- This project contains a series of works developed for audio (including speech, music, and general audio events) processing and generation, which helps reproducible research in the field of audio. The target of **Unified-Audio** is to explore a unified framework to handle **different audio processing and generation tasks**, including:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
- - **SR**: Speech Restoration (⛳ supported)
17
- - **TSE**: Target Speaker Extraction (⛳ supported)
18
- - **SS**: Speech Separation (⛳ supported)
19
- - **VC**: Voice Conversion (⛳ supported)
20
- - **LASS**: Language-Queried Audio Source Separation (⛳ supported)
21
- - **CODEC**: Audio Tokenization (⛳ supported)
22
- - **AE**: Audio Editing (⛳ developing)
23
- - **TTA**: Text to Audio (⛳ developing)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  - more...
25
 
26
  In addition to the frameworks for specific audio tasks, **QuarkAudio** also provides works involving **neural audio codec (NAC)**, which is the fundamental module to combine audio modality with language models.
27
 
28
 
29
  ## 🚀 News
30
- - **2025/09/22**: We release [***UniSE***](https://github.com/hyyan2k/UniSE), a foundation model for unified speech generation. The system supports target speaker extraction, universal speech enhancement.[***demo***](https://hyyan2k.github.io/UniSE/), [![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/abs/2510.20441),Code will comming soon.
31
- - **2025/10/26**: We release [***UniTok-Audio***](https://github.com/alibaba/unified-audio), The system supports target speaker extraction, universal speech enhancement, Speech Restoration, Voice Conversion, Language-Queried Audio Source Separation, Audio Tokenization,[***demo***](https://alibaba.github.io/unified-audio/), [![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/abs/2510.26372)
 
32
  ## key Works
33
  ### UniSE
34
- [UniSE](https://github.com/alibaba/unified-audio/tree/main/UniSE): A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement[![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/abs/2510.20441)
35
- supported tasks: **SR**, **TSE**, **SS**
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # QuarkAudio: An Open-Source Project to Unify Audio Processing and Generation
2
 
3
+ <p align="center">
4
+ <a href="https://arxiv.org/pdf/2512.20151">
5
+ <img src="https://img.shields.io/badge/Paper-ArXiv-red.svg" alt="Paper">
6
+ </a>
7
+ <a href="https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE">
8
+ <img src="https://img.shields.io/badge/GitHub-Code-green.svg" alt="GitHub">
9
+ </a>
10
+ <a href="https://alibaba.github.io/unified-audio//">
11
+ <img src="https://img.shields.io/badge/Demo-Page-blue.svg" alt="Demo">
12
+ </a>
13
+ <a href="https://huggingface.co/QuarkAudio/">
14
+ <img src="https://img.shields.io/badge/Model-Hugging%20Face-yellow.svg" alt="Hugging Face">
15
+ </a>
16
+ <a href="https://www.modelscope.cn/organization/QuarkAudio/">
17
+ <img src="https://img.shields.io/badge/Model-%20%E9%AD%94%E6%90%AD-orange.svg" alt="ModelScope">
18
+ </a>
19
+
20
+ </p>
21
 
22
+ <p align="center">
23
+ <a href="https://arxiv.org/pdf/2512.20151"><img src="QuarkAudio.jpg" width="70%" /></a>
24
+ </p>
25
+
26
+ ## Introduction
27
+ This project contains a series of works developed for audio (including speech, music, and general audio events) processing and generation, which helps reproducible research in the field of audio. The target of **QuarkAudio** is to explore a unified framework to handle **different audio processing and generation tasks**, including:
28
+
29
+ 🚀 **Key Highlights**:
30
+ - ✅ **Unified & Prompt-Free**: Handles multiple tasks without explicit instruction.
31
+ - ⚙️ **Decoder-only AR-LM Backbone**: Leverages LLM-style autoregressive generation for speech token prediction.
32
+ - 🔄 **End-to-End Compatible**: Integrates WavLM/Hubert (feature extractor), H-Codec (discrete codec), and LM into one pipeline.
33
+ - 🌍 **Multitask Support**: SE, SR, TSE, SS, EDIT, VC, LASS, TTA, and more — all in a single model.
34
+
35
+ 📄 **Paper**: [arXiv:2510.20441](https://arxiv.org/pdf/2512.20151) | 🎤 **Listen**: [Demo Page](https://alibaba.github.io/unified-audio/) | 🤗 **Model**: [Hugging Face Spaces](https://huggingface.co/QuarkAudio//)
36
+
37
+ ## 📋 Supported Tasks
38
+
39
+ | Task | Full Name | Status | Description |
40
+ |------|-----------|--------|-------------|
41
+ | **SR** | Speech Restoration | ⛳ supported | Recover clean speech from corrupted inputs (e.g., noise, reverb, packet loss) |
42
+ | **TSE** | Target Speaker Extraction | ⛳ supported | Extract target speaker using reference enrollment audio |
43
+ | **SS** | Speech Separation | ⛳ supported | Separate mixed speakers or sound sources |
44
+ | **VC** | Voice Conversion | ⛳ supported | Convert the speaker identity of input speech while preserving linguistic content |
45
+ | **LASS** | Language-Queried Audio Source Separatio | ⛳ supported | Separate sound sources based on natural language queries (e.g., "remove the man's voice") |
46
+ | **CODEC** | Audio Tokenization | ⛳ supported | Encode speech into compact discrete tokens and reconstruct high-fidelity audio via decoding |
47
+ | **AE** | Audio Editing | ⛳ supported | Edit spoken content by inserting, deleting, or substituting words/phrases in the audio domain |
48
+ | **TTA** | Text to Audio |⏳ Developing | Generate speech or environmental sounds directly from text prompts (upcoming in next release) |
49
+ | **AEC** | Acoustic Echo Cancellation | ⏳ Developing | Remove echo artifacts in teleconferencing scenarios (upcoming in next release) |
50
  - more...
51
 
52
  In addition to the frameworks for specific audio tasks, **QuarkAudio** also provides works involving **neural audio codec (NAC)**, which is the fundamental module to combine audio modality with language models.
53
 
54
 
55
  ## 🚀 News
56
+ - **2025/12/24**: We release [***QuarkAudio***](https://github.com/alibaba/unified-audio), an Open-Source Project to Unify Audio Processing and Generation.[![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/pdf/2512.20151). The code is publicly available at: [QuarkAudio-HCodec](https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-HCodec), along with pretrained models and inference examples.
57
+ - **2025/10/26**: We release [***UniTok-Audio***](https://github.com/alibaba/unified-audio), The system supports target speaker extraction, universal speech enhancement, Speech Restoration, Voice Conversion, Language-Queried Audio Source Separation, Audio Tokenization,[***demo***](https://alibaba.github.io/unified-audio/), [![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/abs/2510.26372). Code will comming soon.
58
+ - **2025/09/22**: We release [***UniSE***](https://github.com/hyyan2k/UniSE), a foundation model for unified speech generation. The system supports target speaker extraction, universal speech enhancement.[***demo***](https://hyyan2k.github.io/UniSE/), [![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/abs/2510.20441). The code is publicly available at: [QuarkAudio-UniSE](https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE), along with pretrained models and inference examples.
59
  ## key Works
60
  ### UniSE
61
+ [UniSE](https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE): A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement[![arXiv](https://img.shields.io/badge/arXiv-Paper-COLOR.svg)](https://arxiv.org/abs/2510.20441)
62
+ supported tasks: **SR**, **TSE**, **SS**
63
+