README / README.md
tutu0604's picture
style: align organization card with GitHub README
0a01075 verified
|
Raw History Blame Contribute Delete
4.76 kB
---
title: README
emoji: 🐨
colorFrom: gray
colorTo: yellow
sdk: static
pinned: false
---
<p align="center">
<a href="https://audio-editing-challenge.github.io/">
<img src="https://audio-editing-challenge.github.io/img/audio-editing-challenge-logo-hd-4k.png" width="900" alt="MMAE — ICASSP 2027 Audio Editing Challenge">
</a>
</p>
<h1 align="center" style="font-size: 32px; font-weight: 700; line-height: 1.25; padding-bottom: 16px; border-bottom: 1px solid #d8dee4;">ICASSP 2027 Audio Editing Challenge</h1>
<p align="center" style="font-size: 18px; line-height: 1.6;">
Advancing precise, general-purpose, instruction-based audio editing
</p>
<p align="center">
<a href="https://audio-editing-challenge.github.io/"><img src="https://img.shields.io/badge/Challenge-Website-2563EB?style=flat-square" alt="Challenge website"></a>
<a href="https://github.com/Audio-Editing-Challenge/Audio-Editing-Challenge-Baseline"><img src="https://img.shields.io/badge/Baselines-Code-181717?style=flat-square&logo=github" alt="Open-source baselines"></a>
<a href="https://audio-editing-challenge.github.io/leaderboard"><img src="https://img.shields.io/badge/Leaderboard-Results-7C3AED?style=flat-square" alt="Challenge leaderboard"></a>
<a href="https://arxiv.org/abs/2606.07229"><img src="https://img.shields.io/badge/arXiv-2606.07229-B31B1B?style=flat-square&logo=arxiv" alt="MMAE paper on arXiv"></a>
</p>
## About the Challenge
The Audio Editing Challenge at ICASSP 2027 brings together researchers working on general-purpose audio editing. Built on **MMAE (A Massive Multitask Audio Editing Benchmark)**, the challenge asks systems to understand source audio, follow potentially complex instructions, and produce precise edits while preserving unrelated content and audio quality.
Tasks span **speech, music, sound, and their mixtures** — from removing applause without changing the speaker to replacing a lyric while preserving the singer and accompaniment.
## Challenge Tracks
| Track | Goal |
| --- | --- |
| **Single Model Track** | Build one end-to-end model that directly transforms input audio according to an instruction. |
| **Agent Track** | Build an autonomous system that can plan, use locally deployed models or signal-processing tools, inspect intermediate results, and refine its output. |
**Track details:** [Single Model Track](https://audio-editing-challenge.github.io/track1/) and [Agent Track](https://audio-editing-challenge.github.io/track2/).
Both tracks cover speech, music, sound, and their mixtures. The Single Model Track is restricted to the MMAE **single** complexity category; the Agent Track covers **all complexity categories**.
## Baselines
Our **[baseline repository](https://github.com/Audio-Editing-Challenge/Audio-Editing-Challenge-Baseline)** provides setup instructions, inference scripts, and submission-format outputs for both tracks:
- **Single Model:** AuK-based end-to-end editing with Prompt Enhancer disabled.
- **Agent:** An LLM router combining DSP tools, SAM-Audio source separation, and AuK generative editing.
For participation rules, test-set release details, and submission instructions, please follow the **[official website](https://audio-editing-challenge.github.io/)**.
## Evaluation
The two tracks are ranked independently using rubric-based evaluation:
- **Instruction Following Rate (IFR):** Were the requested edits correctly executed?
- **Consistency Rate (CR):** Were unrelated audio content and quality preserved?
- **Exact Match Rate (EMR):** Were all instruction-following and consistency rubrics satisfied for a sample?
Rankings are determined by **Overall EMR**, with ties broken by Overall IFR, then Overall CR. See the [evaluation protocol](https://audio-editing-challenge.github.io/#benchmark-and-evaluation-protocol) for full details.
## Key Dates
| Milestone | Date |
| --- | --- |
| Challenge test sets and submission SDK released; leaderboard opens | November 10, 2026 |
| Final submissions due; leaderboard freezes | November 25, 2026 |
| Final rankings and invited teams announced | December 8, 2026 |
| Invited two-page ICASSP papers due | January 7, 2027 |
Submission deadlines are **11:59 PM U.S. Pacific Time**. The schedule is tentative; consult the [official timeline](https://audio-editing-challenge.github.io/#timeline) for updates.
## Contact & Community
For challenge questions and invitations to the Slack workspace or WeChat group, contact [Zhikang Niu](mailto:zhikangniu@sjtu.edu.cn) or [Wenming Tu](mailto:tuwenming@sjtu.edu.cn).
The challenge prize pool is sponsored by **Tencent**. The top three teams in each track will be invited to submit a two-page ICASSP 2027 paper and present their work in the Grand Challenge session.