| license: mit | |
| tags: | |
| - audio | |
| - music-source-separation | |
| - sound-separation | |
| - demucs | |
| - htdemucs | |
| - stem-separation | |
| - inference | |
| pipeline_tag: audio-to-audio | |
| ## Music Source Separation | |
| This is the Demucs v4 models from Facebook Research. | |
| --- | |
| ## What is HTDemucs? | |
| [HTDemucs (Hybrid Transformer Demucs)](https://github.com/facebookresearch/demucs) is Meta AI's fourth-generation music source separation model, introduced in [*Hybrid Transformers for Music Source Separation* (Rouard et al., ICASSP 2023)](https://arxiv.org/abs/2211.08553). | |
| Where earlier Demucs generations processed audio purely in the time domain, HTDemucs runs **two parallel encoders simultaneously** — one operating on the raw waveform, the other on the STFT spectrogram — with a **Transformer Encoder with cross-attention** at the bottleneck connecting them. This lets the model correlate time-domain and frequency-domain features before decoding, yielding measurably better separation quality — especially on spectrally complex, temporally sparse instruments like piano and guitar. | |
| The `htdemucs_6s` variant adds dedicated guitar and piano stems on top of the standard drums/bass/other/vocals quad, making it the most capable publicly available separation model for music production use. | |
| --- | |
| From Facebook research: | |
| Demucs is based on U-Net convolutional architecture inspired by Wave-U-Net and SING, with GLUs, a BiLSTM between the encoder and decoder, specific initialization of weights and transposed convolutions in the decoder. | |
| See [facebookresearch's repository](https://github.com/facebookresearch/demucs) for more information on Demucs. |