| --- |
| license: mit |
| pipeline_tag: audio-text-to-text |
| library_name: transformers |
| tags: |
| - music |
| - audio |
| --- |
| |
| <a href="https://arxiv.org/abs/2602.00744">Tech Report</a> |
|
|
| # ACE-Step Transcriber |
|
|
| ## Description |
|
|
| ACE-Step Transcriber is the annotation model used by **ACE-Step v1.5** for training data labeling. It is a powerful multilingual audio transcription model capable of transcribing both **speech** and **singing voice** with high accuracy. |
|
|
| ### Key Features |
|
|
| - 🌍 **50+ Languages Support** - Covers major world languages and regional dialects |
| - 🎤 **Speech Transcription** - Accurately transcribes spoken content |
| - 🎵 **Singing Voice Transcription** - Specialized in lyrics transcription with musical structure annotations |
| - 🏷️ **Structure Annotation** - Automatically identifies song sections (verse, chorus, bridge, etc.) |
|
|
| ## Usage |
|
|
| The usage is the same as [Qwen2.5 Omni-7B](https://huggingface.co/Qwen/Qwen2.5-Omni-7B). |
|
|
| ### Prompt Format |
|
|
| Use the following prompt to transcribe audio: |
|
|
| ``` |
| *Task* Transcribe this audio in detail |
| <audio> |
| ``` |
|
|
| ### Output Format |
|
|
| The model outputs structured content in the following format: |
|
|
| ``` |
| # Languages |
| <language_code> |
| |
| # Lyrics |
| [Section Tag - Optional Instrument] |
| |
| <transcribed content> |
| ... |
| ``` |
|
|
| ### Example Output |
|
|
| ``` |
| # Languages |
| en |
| |
| # Lyrics |
| [Intro - Acoustic Guitar] |
| |
| [Verse 1] |
| Walking down the empty street tonight |
| Stars are shining oh so bright |
| ... |
| |
| [Chorus] |
| This is where we belong |
| Singing our favorite song |
| ... |
| ``` |
|
|
| ### Supported Section Tags |
|
|
| - `[Intro]`, `[Outro]` |
| - `[Verse 1]`, `[Verse 2]`, etc. |
| - `[Chorus]`, `[Pre-Chorus]`, `[Post-Chorus]` |
| - `[Bridge]` |
| - `[Guitar Interlude]`, `[Instrumental]` |
| - `[Spoken]` |
|
|
| ### Supported Languages (50+) |
|
|
| The model supports transcription in over 50 languages, including but not limited to: |
|
|
| | Region | Languages | |
| |--------|-----------| |
| | **East Asia** | Chinese (zh), Japanese (ja), Korean (ko) | |
| | **Southeast Asia** | Vietnamese (vi), Thai (th), Indonesian (id), Malay (ms), Filipino (tl) | |
| | **South Asia** | Hindi (hi), Bengali (bn), Tamil (ta), Urdu (ur) | |
| | **Europe** | English (en), German (de), French (fr), Spanish (es), Italian (it), Portuguese (pt), Russian (ru), Polish (pl), Dutch (nl), Greek (el), Turkish (tr) | |
| | **Middle East** | Arabic (ar), Hebrew (he), Persian (fa) | |
| | **Others** | And many more regional languages... | |
|
|
| ## Use Cases |
|
|
| - **Music Production** - Transcribe reference tracks for lyrics extraction |
| - **Dataset Creation** - Generate high-quality labeled data for music AI models |
| - **Accessibility** - Create subtitles and captions for audio content |
| - **Music Analysis** - Extract structural information from songs |
