TheDailyOWO commited on
Commit
195447d
·
verified ·
1 Parent(s): 884b515

Splitwave models (ONNX, browser-ready)

Browse files
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: per-model
4
+ tags:
5
+ - audio-source-separation
6
+ - music-source-separation
7
+ - onnx
8
+ - webgpu
9
+ pipeline_tag: audio-to-audio
10
+ ---
11
+
12
+ # Splitwave models
13
+
14
+ Browser-ready ONNX versions of the stem separation models used by
15
+ [Splitwave](https://github.com/PolysheepCodes/Splitwave), a web app that splits songs into vocals,
16
+ instrumental, lead and backing vocals on the visitor's own computer (WebGPU / WebAssembly).
17
+
18
+ | File | Model | Does | Precision |
19
+ |---|---|---|---|
20
+ | `bs_roformer_fp16.onnx` | BS-RoFormer (ep 317, SDR 12.98) by viperx | vocals / instrumental (Best) | fp16 |
21
+ | `mel_roformer_karaoke_fp16.onnx` | Mel-Band RoFormer karaoke (SDR 10.20) by aufr33 & viperx | lead / backing vocals (Best) | fp16 |
22
+ | `mdx_voc_ft.onnx` | UVR-MDX-NET Voc_FT by the UVR team | vocals / instrumental (Fast) | fp32, unchanged |
23
+ | `mdx_kara_2.onnx` | UVR_MDXNET_KARA_2 by the UVR team | lead / backing vocals (Fast) | fp32, unchanged |
24
+
25
+ `manifest.json` lists each file's size and SHA-256. Splitwave checks downloads against it.
26
+
27
+ ## Conversion
28
+
29
+ The RoFormer networks were exported from the original PyTorch checkpoints with Splitwave's
30
+ [`convert/roformer.py`](https://github.com/PolysheepCodes/Splitwave/blob/main/convert/roformer.py). They take
31
+ one 8-second chunk's stereo STFT (`[1, 2050, 801, 2]`, n_fft 2048, hop 441) and return the complex mask to
32
+ multiply it with. The STFT and inverse STFT happen in the app. The band split was restructured for WebGPU's
33
+ limits, without changing the math. Each export was checked against the original model on real audio
34
+ before the fp16 conversion (normalisation layers and rotary position tables kept in fp32).
35
+
36
+ ## Credits and licenses
37
+
38
+ The weights belong to their authors: the Ultimate Vocal Remover team (MDX-Net models), viperx
39
+ (BS-RoFormer), and aufr33 & viperx (Mel-Band RoFormer karaoke). They were obtained through
40
+ [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)'s model repository.
41
+ This repo redistributes them, converted for in-browser use, under the original authors' terms. Check those
42
+ terms before reusing them.
bs_roformer_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db70316e4cd6331fb54cabdbad9ab378dca5d35eb3206f1603e43e13ba2cfc3d
3
+ size 326345279
manifest.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "mdx_voc_ft": {
3
+ "file": "mdx_voc_ft.onnx",
4
+ "bytes": 66762490,
5
+ "sha256": "534b2070fcc7df514b13ef660dc8cbb328679c2374d04354a5c42bb14ecce111"
6
+ },
7
+ "mdx_kara_2": {
8
+ "file": "mdx_kara_2.onnx",
9
+ "bytes": 52786726,
10
+ "sha256": "bf32e15105a09c0f7dddd2b67346146334d6f3ecb399ed7638eba2ab07cbf5f4"
11
+ },
12
+ "bs_roformer": {
13
+ "file": "bs_roformer_fp16.onnx",
14
+ "bytes": 326345279,
15
+ "sha256": "db70316e4cd6331fb54cabdbad9ab378dca5d35eb3206f1603e43e13ba2cfc3d"
16
+ },
17
+ "mel_karaoke": {
18
+ "file": "mel_roformer_karaoke_fp16.onnx",
19
+ "bytes": 460112278,
20
+ "sha256": "3246267f38659149f8fc3a6ee33b6533c66ddf5236ced3b0a732630f5222da36"
21
+ }
22
+ }
mdx_kara_2.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bf32e15105a09c0f7dddd2b67346146334d6f3ecb399ed7638eba2ab07cbf5f4
3
+ size 52786726
mdx_voc_ft.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:534b2070fcc7df514b13ef660dc8cbb328679c2374d04354a5c42bb14ecce111
3
+ size 66762490
mel_roformer_karaoke_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3246267f38659149f8fc3a6ee33b6533c66ddf5236ced3b0a732630f5222da36
3
+ size 460112278