Update README.md
Browse files
README.md
CHANGED
|
@@ -1,9 +1,62 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
tags:
|
| 3 |
-
-
|
| 4 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
Audio source separation model used in Sytem T002 for [Cadenza2 Task2 Challenge](https://cadenzachallenge.org/docs/cadenza2/Rebalancing%20Classical/rebalancing)
|
| 8 |
|
| 9 |
The model is a finetune of the 8 ConvTasNet models from the Task2 baseline.
|
|
@@ -12,13 +65,11 @@ The training optimised the estimated sources and the recosntructed mixture
|
|
| 12 |
$$
|
| 13 |
Loss = \sum_{}^{Sources}(L_1(estimated~source, ref~source)) + L_1(reconstructed~mixture, original~mixture)
|
| 14 |
$$
|
| 15 |
-
|
| 16 |
```Python
|
| 17 |
def dynamic_masked_loss(mixture, separated_sources, ground_truth_sources, indicator):
|
| 18 |
# Reconstruction Loss
|
| 19 |
reconstruction = sum(separated_sources.values())
|
| 20 |
reconstruction_loss = nn.L1Loss()(reconstruction, mixture)
|
| 21 |
-
|
| 22 |
# Separation Loss
|
| 23 |
separation_loss = 0
|
| 24 |
for instrument, active in indicator.items():
|
|
@@ -26,9 +77,6 @@ def dynamic_masked_loss(mixture, separated_sources, ground_truth_sources, indica
|
|
| 26 |
separation_loss += nn.L1Loss()(
|
| 27 |
separated_sources[instrument], ground_truth_sources[instrument]
|
| 28 |
)
|
| 29 |
-
|
| 30 |
return reconstruction_loss + separation_loss
|
| 31 |
```
|
| 32 |
-
|
| 33 |
-
Model and T002 recipe are shared in [Clarity toolkit](https://github.com/claritychallenge/clarity)
|
| 34 |
-
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
tags:
|
| 6 |
+
- hearing loss
|
| 7 |
+
- challenge
|
| 8 |
+
- signal processing
|
| 9 |
+
- source separation
|
| 10 |
+
- audio
|
| 11 |
+
- audio-to-audio
|
| 12 |
+
- NonCausal
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# Cadenza Challenge: CAD2-Task2
|
| 16 |
+
|
| 17 |
+
A Causal separation model for the CAD2-Task2 system.
|
| 18 |
+
|
| 19 |
+
This model is an ensemble of the following instruments:
|
| 20 |
+
- Bassoon
|
| 21 |
+
- Cello
|
| 22 |
+
- Clarinet
|
| 23 |
+
- Flute
|
| 24 |
+
- Oboe
|
| 25 |
+
- Sax
|
| 26 |
+
- Viola
|
| 27 |
+
- Violin
|
| 28 |
+
|
| 29 |
+
Each model is based on the ConvTasNet (Kaituo XU) with multichannel support (Alexandre Defossez).
|
| 30 |
+
* Parameters:
|
| 31 |
+
* B: 256
|
| 32 |
+
* C: 2
|
| 33 |
+
* H: 512
|
| 34 |
+
* L: 20
|
| 35 |
+
* N: 256
|
| 36 |
+
* P: 3
|
| 37 |
+
* R: 3
|
| 38 |
+
* X: 8
|
| 39 |
+
* audio_channels: 2
|
| 40 |
+
* causal: true
|
| 41 |
+
* mask_nonlinear: relu
|
| 42 |
+
* norm_type: cLN
|
| 43 |
+
|
| 44 |
+
## Dataset
|
| 45 |
+
The model was trained using EnsembleSet and CadenzaWoodwind datasets.
|
| 46 |
+
|
| 47 |
+
## How to use
|
| 48 |
+
|
| 49 |
+
```
|
| 50 |
+
from dynamic_source_separator import DynamicSourceSeparator
|
| 51 |
+
|
| 52 |
+
model = DynamicSourceSeparator.from_pretrained(
|
| 53 |
+
"cadenzachallenge/Dynamic_Source_Separator_Causal"
|
| 54 |
+
).cpu()
|
| 55 |
+
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
## Description
|
| 59 |
+
|
| 60 |
Audio source separation model used in Sytem T002 for [Cadenza2 Task2 Challenge](https://cadenzachallenge.org/docs/cadenza2/Rebalancing%20Classical/rebalancing)
|
| 61 |
|
| 62 |
The model is a finetune of the 8 ConvTasNet models from the Task2 baseline.
|
|
|
|
| 65 |
$$
|
| 66 |
Loss = \sum_{}^{Sources}(L_1(estimated~source, ref~source)) + L_1(reconstructed~mixture, original~mixture)
|
| 67 |
$$
|
|
|
|
| 68 |
```Python
|
| 69 |
def dynamic_masked_loss(mixture, separated_sources, ground_truth_sources, indicator):
|
| 70 |
# Reconstruction Loss
|
| 71 |
reconstruction = sum(separated_sources.values())
|
| 72 |
reconstruction_loss = nn.L1Loss()(reconstruction, mixture)
|
|
|
|
| 73 |
# Separation Loss
|
| 74 |
separation_loss = 0
|
| 75 |
for instrument, active in indicator.items():
|
|
|
|
| 77 |
separation_loss += nn.L1Loss()(
|
| 78 |
separated_sources[instrument], ground_truth_sources[instrument]
|
| 79 |
)
|
|
|
|
| 80 |
return reconstruction_loss + separation_loss
|
| 81 |
```
|
| 82 |
+
Model and T002 recipe are shared in [Clarity toolkit](https://github.com/claritychallenge/clarity)
|
|
|
|
|
|