File size: 2,006 Bytes
b7c0baf 71855d4 b7c0baf 71855d4 b7c0baf 71855d4 12ca5f8 71855d4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: apache-2.0
language:
- en
tags:
- hearing loss
- challenge
- signal processing
- source separation
- audio
- audio-to-audio
- NonCausal
---
# Cadenza Challenge: CAD2-Task2
A Causal separation model for the CAD2-Task2 system.
This model is an ensemble of the following instruments:
- Bassoon
- Cello
- Clarinet
- Flute
- Oboe
- Sax
- Viola
- Violin
Each model is based on the ConvTasNet (Kaituo XU) with multichannel support (Alexandre Defossez).
* Parameters:
* B: 256
* C: 2
* H: 512
* L: 20
* N: 256
* P: 3
* R: 3
* X: 8
* audio_channels: 2
* causal: true
* mask_nonlinear: relu
* norm_type: cLN
## Dataset
The model was trained using EnsembleSet and CadenzaWoodwind datasets.
## How to use
```
from dynamic_source_separator import DynamicSourceSeparator
model = DynamicSourceSeparator.from_pretrained(
"cadenzachallenge/Dynamic_Source_Separator_Causal"
).cpu()
```
## Description
Audio source separation model used in Sytem T002 for [Cadenza2 Task2 Challenge](https://cadenzachallenge.org/docs/cadenza2/Rebalancing%20Classical/rebalancing)
The model is a finetune of the 8 ConvTasNet models from the Task2 baseline.
The training optimised the estimated sources and the recosntructed mixture
$$
Loss = \sum_{}^{Sources}(L_1(estimated~source, ref~source)) + L_1(reconstructed~mixture, original~mixture)
$$
```Python
def dynamic_masked_loss(mixture, separated_sources, ground_truth_sources, indicator):
# Reconstruction Loss
reconstruction = sum(separated_sources.values())
reconstruction_loss = nn.L1Loss()(reconstruction, mixture)
# Separation Loss
separation_loss = 0
for instrument, active in indicator.items():
if active:
separation_loss += nn.L1Loss()(
separated_sources[instrument], ground_truth_sources[instrument]
)
return reconstruction_loss + separation_loss
```
Model and T002 recipe are shared in [Clarity toolkit](https://github.com/claritychallenge/clarity) |