Spaces:
Configuration error
Configuration error
Update README.md
Browse filesUpdate project info
README.md
CHANGED
|
@@ -1,35 +1,63 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
title: QuarkAudio
|
| 4 |
-
sdk: gradio
|
| 5 |
-
emoji: 🚀
|
| 6 |
-
colorFrom: blue
|
| 7 |
-
colorTo: yellow
|
| 8 |
-
pinned: true
|
| 9 |
-
short_description: Audio processing and generation
|
| 10 |
-
sdk_version: 6.0.2
|
| 11 |
-
---
|
| 12 |
# QuarkAudio: An Open-Source Project to Unify Audio Processing and Generation
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
- more...
|
| 25 |
|
| 26 |
In addition to the frameworks for specific audio tasks, **QuarkAudio** also provides works involving **neural audio codec (NAC)**, which is the fundamental module to combine audio modality with language models.
|
| 27 |
|
| 28 |
|
| 29 |
## 🚀 News
|
| 30 |
-
- **2025/
|
| 31 |
-
- **2025/10/26**: We release [***UniTok-Audio***](https://github.com/alibaba/unified-audio), The system supports target speaker extraction, universal speech enhancement, Speech Restoration, Voice Conversion, Language-Queried Audio Source Separation, Audio Tokenization,[***demo***](https://alibaba.github.io/unified-audio/), [](https://arxiv.org/abs/2510.26372)
|
|
|
|
| 32 |
## key Works
|
| 33 |
### UniSE
|
| 34 |
-
[UniSE](https://github.com/alibaba/unified-audio/tree/main/UniSE): A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement[](https://arxiv.org/abs/2510.20441)
|
| 35 |
-
supported tasks: **SR**, **TSE**, **SS**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# QuarkAudio: An Open-Source Project to Unify Audio Processing and Generation
|
| 2 |
|
| 3 |
+
<p align="center">
|
| 4 |
+
<a href="https://arxiv.org/pdf/2512.20151">
|
| 5 |
+
<img src="https://img.shields.io/badge/Paper-ArXiv-red.svg" alt="Paper">
|
| 6 |
+
</a>
|
| 7 |
+
<a href="https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE">
|
| 8 |
+
<img src="https://img.shields.io/badge/GitHub-Code-green.svg" alt="GitHub">
|
| 9 |
+
</a>
|
| 10 |
+
<a href="https://alibaba.github.io/unified-audio//">
|
| 11 |
+
<img src="https://img.shields.io/badge/Demo-Page-blue.svg" alt="Demo">
|
| 12 |
+
</a>
|
| 13 |
+
<a href="https://huggingface.co/QuarkAudio/">
|
| 14 |
+
<img src="https://img.shields.io/badge/Model-Hugging%20Face-yellow.svg" alt="Hugging Face">
|
| 15 |
+
</a>
|
| 16 |
+
<a href="https://www.modelscope.cn/organization/QuarkAudio/">
|
| 17 |
+
<img src="https://img.shields.io/badge/Model-%20%E9%AD%94%E6%90%AD-orange.svg" alt="ModelScope">
|
| 18 |
+
</a>
|
| 19 |
+
|
| 20 |
+
</p>
|
| 21 |
|
| 22 |
+
<p align="center">
|
| 23 |
+
<a href="https://arxiv.org/pdf/2512.20151"><img src="QuarkAudio.jpg" width="70%" /></a>
|
| 24 |
+
</p>
|
| 25 |
+
|
| 26 |
+
## Introduction
|
| 27 |
+
This project contains a series of works developed for audio (including speech, music, and general audio events) processing and generation, which helps reproducible research in the field of audio. The target of **QuarkAudio** is to explore a unified framework to handle **different audio processing and generation tasks**, including:
|
| 28 |
+
|
| 29 |
+
🚀 **Key Highlights**:
|
| 30 |
+
- ✅ **Unified & Prompt-Free**: Handles multiple tasks without explicit instruction.
|
| 31 |
+
- ⚙️ **Decoder-only AR-LM Backbone**: Leverages LLM-style autoregressive generation for speech token prediction.
|
| 32 |
+
- 🔄 **End-to-End Compatible**: Integrates WavLM/Hubert (feature extractor), H-Codec (discrete codec), and LM into one pipeline.
|
| 33 |
+
- 🌍 **Multitask Support**: SE, SR, TSE, SS, EDIT, VC, LASS, TTA, and more — all in a single model.
|
| 34 |
+
|
| 35 |
+
📄 **Paper**: [arXiv:2510.20441](https://arxiv.org/pdf/2512.20151) | 🎤 **Listen**: [Demo Page](https://alibaba.github.io/unified-audio/) | 🤗 **Model**: [Hugging Face Spaces](https://huggingface.co/QuarkAudio//)
|
| 36 |
+
|
| 37 |
+
## 📋 Supported Tasks
|
| 38 |
+
|
| 39 |
+
| Task | Full Name | Status | Description |
|
| 40 |
+
|------|-----------|--------|-------------|
|
| 41 |
+
| **SR** | Speech Restoration | ⛳ supported | Recover clean speech from corrupted inputs (e.g., noise, reverb, packet loss) |
|
| 42 |
+
| **TSE** | Target Speaker Extraction | ⛳ supported | Extract target speaker using reference enrollment audio |
|
| 43 |
+
| **SS** | Speech Separation | ⛳ supported | Separate mixed speakers or sound sources |
|
| 44 |
+
| **VC** | Voice Conversion | ⛳ supported | Convert the speaker identity of input speech while preserving linguistic content |
|
| 45 |
+
| **LASS** | Language-Queried Audio Source Separatio | ⛳ supported | Separate sound sources based on natural language queries (e.g., "remove the man's voice") |
|
| 46 |
+
| **CODEC** | Audio Tokenization | ⛳ supported | Encode speech into compact discrete tokens and reconstruct high-fidelity audio via decoding |
|
| 47 |
+
| **AE** | Audio Editing | ⛳ supported | Edit spoken content by inserting, deleting, or substituting words/phrases in the audio domain |
|
| 48 |
+
| **TTA** | Text to Audio |⏳ Developing | Generate speech or environmental sounds directly from text prompts (upcoming in next release) |
|
| 49 |
+
| **AEC** | Acoustic Echo Cancellation | ⏳ Developing | Remove echo artifacts in teleconferencing scenarios (upcoming in next release) |
|
| 50 |
- more...
|
| 51 |
|
| 52 |
In addition to the frameworks for specific audio tasks, **QuarkAudio** also provides works involving **neural audio codec (NAC)**, which is the fundamental module to combine audio modality with language models.
|
| 53 |
|
| 54 |
|
| 55 |
## 🚀 News
|
| 56 |
+
- **2025/12/24**: We release [***QuarkAudio***](https://github.com/alibaba/unified-audio), an Open-Source Project to Unify Audio Processing and Generation.[](https://arxiv.org/pdf/2512.20151). The code is publicly available at: [QuarkAudio-HCodec](https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-HCodec), along with pretrained models and inference examples.
|
| 57 |
+
- **2025/10/26**: We release [***UniTok-Audio***](https://github.com/alibaba/unified-audio), The system supports target speaker extraction, universal speech enhancement, Speech Restoration, Voice Conversion, Language-Queried Audio Source Separation, Audio Tokenization,[***demo***](https://alibaba.github.io/unified-audio/), [](https://arxiv.org/abs/2510.26372). Code will comming soon.
|
| 58 |
+
- **2025/09/22**: We release [***UniSE***](https://github.com/hyyan2k/UniSE), a foundation model for unified speech generation. The system supports target speaker extraction, universal speech enhancement.[***demo***](https://hyyan2k.github.io/UniSE/), [](https://arxiv.org/abs/2510.20441). The code is publicly available at: [QuarkAudio-UniSE](https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE), along with pretrained models and inference examples.
|
| 59 |
## key Works
|
| 60 |
### UniSE
|
| 61 |
+
[UniSE](https://github.com/alibaba/unified-audio/tree/main/QuarkAudio-UniSE): A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement[](https://arxiv.org/abs/2510.20441)
|
| 62 |
+
supported tasks: **SR**, **TSE**, **SS**
|
| 63 |
+
|