Text-to-Audio
Transformers
Safetensors
ACE-Step
feature-extraction
audio
music
text2music
custom_code
Instructions to use ACE-Step/acestep-v15-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ACE-Step/acestep-v15-base with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-audio", model="ACE-Step/acestep-v15-base", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ACE-Step/acestep-v15-base", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,89 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
license: mit
|
| 4 |
+
pipeline_tag: text-to-audio
|
| 5 |
+
tags:
|
| 6 |
+
- audio
|
| 7 |
+
- music
|
| 8 |
+
- text2music
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
<h1 align="center">ACE-Step 1.5</h1>
|
| 12 |
+
<h1 align="center">Pushing the Boundaries of Open-Source Music Generation</h1>
|
| 13 |
+
<p align="center">
|
| 14 |
+
<a href="https://ace-step.github.io/ace-step-v1.5.github.io/">Project</a> |
|
| 15 |
+
<a href="https://huggingface.co/collections/ACE-Step/ace-step-15">Hugging Face</a> |
|
| 16 |
+
<a href="https://modelscope.cn/models/ACE-Step/ACE-Step-v1-5">ModelScope</a> |
|
| 17 |
+
<a href="https://huggingface.co/spaces/ACE-Step/Ace-Step-v1.5">Space Demo</a> |
|
| 18 |
+
<a href="https://discord.gg/PeWDxrkdj7">Discord</a> |
|
| 19 |
+
<a href="">Technical Report</a>
|
| 20 |
+
</p>
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+

|
| 24 |
+
|
| 25 |
+
## Model Details
|
| 26 |
+
|
| 27 |
+
๐ We present ACE-Step v1.5, a highly efficient open-source music foundation model that brings commercial-grade generation to consumer hardware. On commonly used evaluation metrics, ACE-Step v1.5 achieves quality beyond most commercial music models while remaining extremely fastโunder 2 seconds per full song on an A100 and under 10 seconds on an RTX 3090. The model runs locally with less than 4GB of VRAM, and supports lightweight personalization: users can train a LoRA from just a few songs to capture their own style.
|
| 28 |
+
|
| 29 |
+
๐ At its core lies a novel hybrid architecture where the Language Model (LM) functions as an omni-capable planner: it transforms simple user queries into comprehensive song blueprintsโscaling from short loops to 10-minute compositionsโwhile synthesizing metadata, lyrics, and captions via Chain-of-Thought to guide the Diffusion Transformer (DiT). โก Uniquely, this alignment is achieved through intrinsic reinforcement learning relying solely on the model's internal mechanisms, thereby eliminating the biases inherent in external reward models or human preferences. ๐๏ธ
|
| 30 |
+
|
| 31 |
+
๐ฎ Beyond standard synthesis, ACE-Step v1.5 unifies precise stylistic control with versatile editing capabilitiesโsuch as cover generation, repainting, and vocal-to-BGM conversionโwhile maintaining strict adherence to prompts across 50+ languages. This paves the way for powerful tools that seamlessly integrate into the creative workflows of music artists, producers, and content creators. ๐ธ
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
- **Developed by:** [ACE-STEP]
|
| 35 |
+
- **Model type:** [Text2Music]
|
| 36 |
+
- **Language(s):** [50+ languages]
|
| 37 |
+
- **License:** [MIT]
|
| 38 |
+
|
| 39 |
+
## Evaluation
|
| 40 |
+
|
| 41 |
+

|
| 42 |
+
|
| 43 |
+
## ๐๏ธ Architecture
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+

|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
## ๐ฆ Model Zoo
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+

|
| 53 |
+
|
| 54 |
+
### DiT Models
|
| 55 |
+
|
| 56 |
+
| DiT Model | Pre-Training | SFT | RL | CFG | Step | Refer audio | Text2Music | Cover | Repaint | Extract | Lego | Complete | Quality | Diversity | Fine-Tunability | Hugging Face |
|
| 57 |
+
|-----------|:------------:|:---:|:--:|:---:|:----:|:-----------:|:----------:|:-----:|:-------:|:-------:|:----:|:--------:|:-------:|:---------:|:---------------:|--------------|
|
| 58 |
+
| `acestep-v15-base` | โ
| โ | โ | โ
| 50 | โ
| โ
| โ
| โ
| โ
| โ
| โ
| Medium | High | Easy | [Link](https://huggingface.co/ACE-Step/acestep-v15-base) |
|
| 59 |
+
| `acestep-v15-sft` | โ
| โ
| โ | โ
| 50 | โ
| โ
| โ
| โ
| โ | โ | โ | High | Medium | Easy | [Link](https://huggingface.co/ACE-Step/acestep-v15-sft) |
|
| 60 |
+
| `acestep-v15-turbo` | โ
| โ
| โ | โ | 8 | โ
| โ
| โ
| โ
| โ | โ | โ | Very High | Medium | Medium | [Link](https://huggingface.co/ACE-Step/Ace-Step1.5) |
|
| 61 |
+
| `acestep-v15-turbo-rl` | โ
| โ
| โ
| โ | 8 | โ
| โ
| โ
| โ
| โ | โ | โ | Very High | Medium | Medium | To be released |
|
| 62 |
+
|
| 63 |
+
### LM Models
|
| 64 |
+
|
| 65 |
+
| LM Model | Pretrain from | Pre-Training | SFT | RL | CoT metas | Query rewrite | Audio Understanding | Composition Capability | Copy Melody | Hugging Face |
|
| 66 |
+
|----------|---------------|:------------:|:---:|:--:|:---------:|:-------------:|:-------------------:|:----------------------:|:-----------:|--------------|
|
| 67 |
+
| `acestep-5Hz-lm-0.6B` | Qwen3-0.6B | โ
| โ
| โ
| โ
| โ
| Medium | Medium | Weak | โ
|
|
| 68 |
+
| `acestep-5Hz-lm-1.7B` | Qwen3-1.7B | โ
| โ
| โ
| โ
| โ
| Medium | Medium | Medium | โ
|
|
| 69 |
+
| `acestep-5Hz-lm-4B` | Qwen3-4B | โ
| โ
| โ
| โ
| โ
| Strong | Strong | Strong | โ
|
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
## ๐ Acknowledgements
|
| 73 |
+
|
| 74 |
+
This project is co-led by ACE Studio and StepFun.
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
## ๐ Citation
|
| 78 |
+
|
| 79 |
+
If you find this project useful for your research, please consider citing:
|
| 80 |
+
|
| 81 |
+
```BibTeX
|
| 82 |
+
@misc{gong2026acestep,
|
| 83 |
+
title={ACE-Step 1.5: Pushing the Boundaries of Open-Source Music Generation},
|
| 84 |
+
author={Junmin Gong, Yulin Song, Wenxiao Zhao, Sen Wang, Shengyuan Xu, Jing Guo},
|
| 85 |
+
howpublished={\url{https://github.com/ace-step/ACE-Step-1.5}},
|
| 86 |
+
year={2026},
|
| 87 |
+
note={GitHub repository}
|
| 88 |
+
}
|
| 89 |
+
```
|