Update Readme
Browse filesAdd more description of how the T002 model was trained
README.md
CHANGED
|
@@ -4,6 +4,31 @@ tags:
|
|
| 4 |
- model_hub_mixin
|
| 5 |
---
|
| 6 |
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
- model_hub_mixin
|
| 5 |
---
|
| 6 |
|
| 7 |
+
Audio source separation model used in Sytem T002 for [Cadenza2 Task2 Challenge](https://cadenzachallenge.org/docs/cadenza2/Rebalancing%20Classical/rebalancing)
|
| 8 |
+
|
| 9 |
+
The model is a finetune of the 8 ConvTasNet models from the Task2 baseline.
|
| 10 |
+
The training optimised the estimated sources and the recosntructed mixture
|
| 11 |
+
|
| 12 |
+
$$
|
| 13 |
+
Loss = \sum_{}^{Sources}(L_1(estimated~source, ref~source)) + L_1(reconstructed~mixture, original~mixture)
|
| 14 |
+
$$
|
| 15 |
+
|
| 16 |
+
```Python
|
| 17 |
+
def dynamic_masked_loss(mixture, separated_sources, ground_truth_sources, indicator):
|
| 18 |
+
# Reconstruction Loss
|
| 19 |
+
reconstruction = sum(separated_sources.values())
|
| 20 |
+
reconstruction_loss = nn.L1Loss()(reconstruction, mixture)
|
| 21 |
+
|
| 22 |
+
# Separation Loss
|
| 23 |
+
separation_loss = 0
|
| 24 |
+
for instrument, active in indicator.items():
|
| 25 |
+
if active:
|
| 26 |
+
separation_loss += nn.L1Loss()(
|
| 27 |
+
separated_sources[instrument], ground_truth_sources[instrument]
|
| 28 |
+
)
|
| 29 |
+
|
| 30 |
+
return reconstruction_loss + separation_loss
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
Model and T002 recipe are shared in [Clarity toolkit](https://github.com/claritychallenge/clarity)
|
| 34 |
+
|