Splitwave models (ONNX, browser-ready)
Browse files- README.md +42 -0
- bs_roformer_fp16.onnx +3 -0
- manifest.json +22 -0
- mdx_kara_2.onnx +3 -0
- mdx_voc_ft.onnx +3 -0
- mel_roformer_karaoke_fp16.onnx +3 -0
README.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: per-model
|
| 4 |
+
tags:
|
| 5 |
+
- audio-source-separation
|
| 6 |
+
- music-source-separation
|
| 7 |
+
- onnx
|
| 8 |
+
- webgpu
|
| 9 |
+
pipeline_tag: audio-to-audio
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Splitwave models
|
| 13 |
+
|
| 14 |
+
Browser-ready ONNX versions of the stem separation models used by
|
| 15 |
+
[Splitwave](https://github.com/PolysheepCodes/Splitwave), a web app that splits songs into vocals,
|
| 16 |
+
instrumental, lead and backing vocals on the visitor's own computer (WebGPU / WebAssembly).
|
| 17 |
+
|
| 18 |
+
| File | Model | Does | Precision |
|
| 19 |
+
|---|---|---|---|
|
| 20 |
+
| `bs_roformer_fp16.onnx` | BS-RoFormer (ep 317, SDR 12.98) by viperx | vocals / instrumental (Best) | fp16 |
|
| 21 |
+
| `mel_roformer_karaoke_fp16.onnx` | Mel-Band RoFormer karaoke (SDR 10.20) by aufr33 & viperx | lead / backing vocals (Best) | fp16 |
|
| 22 |
+
| `mdx_voc_ft.onnx` | UVR-MDX-NET Voc_FT by the UVR team | vocals / instrumental (Fast) | fp32, unchanged |
|
| 23 |
+
| `mdx_kara_2.onnx` | UVR_MDXNET_KARA_2 by the UVR team | lead / backing vocals (Fast) | fp32, unchanged |
|
| 24 |
+
|
| 25 |
+
`manifest.json` lists each file's size and SHA-256. Splitwave checks downloads against it.
|
| 26 |
+
|
| 27 |
+
## Conversion
|
| 28 |
+
|
| 29 |
+
The RoFormer networks were exported from the original PyTorch checkpoints with Splitwave's
|
| 30 |
+
[`convert/roformer.py`](https://github.com/PolysheepCodes/Splitwave/blob/main/convert/roformer.py). They take
|
| 31 |
+
one 8-second chunk's stereo STFT (`[1, 2050, 801, 2]`, n_fft 2048, hop 441) and return the complex mask to
|
| 32 |
+
multiply it with. The STFT and inverse STFT happen in the app. The band split was restructured for WebGPU's
|
| 33 |
+
limits, without changing the math. Each export was checked against the original model on real audio
|
| 34 |
+
before the fp16 conversion (normalisation layers and rotary position tables kept in fp32).
|
| 35 |
+
|
| 36 |
+
## Credits and licenses
|
| 37 |
+
|
| 38 |
+
The weights belong to their authors: the Ultimate Vocal Remover team (MDX-Net models), viperx
|
| 39 |
+
(BS-RoFormer), and aufr33 & viperx (Mel-Band RoFormer karaoke). They were obtained through
|
| 40 |
+
[python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)'s model repository.
|
| 41 |
+
This repo redistributes them, converted for in-browser use, under the original authors' terms. Check those
|
| 42 |
+
terms before reusing them.
|
bs_roformer_fp16.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:db70316e4cd6331fb54cabdbad9ab378dca5d35eb3206f1603e43e13ba2cfc3d
|
| 3 |
+
size 326345279
|
manifest.json
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"mdx_voc_ft": {
|
| 3 |
+
"file": "mdx_voc_ft.onnx",
|
| 4 |
+
"bytes": 66762490,
|
| 5 |
+
"sha256": "534b2070fcc7df514b13ef660dc8cbb328679c2374d04354a5c42bb14ecce111"
|
| 6 |
+
},
|
| 7 |
+
"mdx_kara_2": {
|
| 8 |
+
"file": "mdx_kara_2.onnx",
|
| 9 |
+
"bytes": 52786726,
|
| 10 |
+
"sha256": "bf32e15105a09c0f7dddd2b67346146334d6f3ecb399ed7638eba2ab07cbf5f4"
|
| 11 |
+
},
|
| 12 |
+
"bs_roformer": {
|
| 13 |
+
"file": "bs_roformer_fp16.onnx",
|
| 14 |
+
"bytes": 326345279,
|
| 15 |
+
"sha256": "db70316e4cd6331fb54cabdbad9ab378dca5d35eb3206f1603e43e13ba2cfc3d"
|
| 16 |
+
},
|
| 17 |
+
"mel_karaoke": {
|
| 18 |
+
"file": "mel_roformer_karaoke_fp16.onnx",
|
| 19 |
+
"bytes": 460112278,
|
| 20 |
+
"sha256": "3246267f38659149f8fc3a6ee33b6533c66ddf5236ced3b0a732630f5222da36"
|
| 21 |
+
}
|
| 22 |
+
}
|
mdx_kara_2.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bf32e15105a09c0f7dddd2b67346146334d6f3ecb399ed7638eba2ab07cbf5f4
|
| 3 |
+
size 52786726
|
mdx_voc_ft.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:534b2070fcc7df514b13ef660dc8cbb328679c2374d04354a5c42bb14ecce111
|
| 3 |
+
size 66762490
|
mel_roformer_karaoke_fp16.onnx
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3246267f38659149f8fc3a6ee33b6533c66ddf5236ced3b0a732630f5222da36
|
| 3 |
+
size 460112278
|