README.md

---
license: mit
base_model:
- Politrees/UVR_resources
tags:
  - coreml
  - apple-silicon
  - apple-neural-engine
  - audio-source-separation
  - music-source-separation
  - mdx-net
  - uvr
  - vocals
  - instrumental
  - fp16
  - mlprogram
  - on-device
pipeline_tag: audio-to-audio
---

# UVR MDX-Net CoreML — on-device music stem separation for iOS

On-device **vocals / instrumental (music stem) separation** models — the **fp16 quantization** of the
Ultimate Vocal Remover (UVR) MDX-Net checkpoints converted to **CoreML `.mlpackage`** files (fp16,
`mlprogram`) for the Apple Neural Engine / GPU / CPU on iOS and macOS. The source ONNX models are
hosted in [Politrees/UVR_resources](https://huggingface.co/Politrees/UVR_resources).

Each `.mlpackage` is the **learned core only** — the STFT/iSTFT live in your app's DSP, so the whole
signal pipeline stays under your control and runs entirely on-device. No cloud, no network.

## Models

| Model file | Source (UVR MDX-Net) | Stem | Size | `n_fft` | `hop` | `dim_f` |
| --- | --- | --- | --- | --- | --- | --- |
| `UVR_MDXNET_9482.mlpackage` | UVR-MDXNET 9482 | **vocals** | ~15 MB | 4096 | 1024 | 2048 |
| `UVR-MDX-NET-Voc_FT.mlpackage` | UVR-MDX-NET Voc FT | **vocals** | ~32 MB | 6144 | 1024 | 3072 |
| `UVR-MDX-NET-Inst_HQ_3.mlpackage` | UVR-MDX-NET Inst HQ 3 | **instrumental** | ~32 MB | 6144 | 1024 | 3072 |

All three share the same I/O contract: **NCHW**, static `dim_t = 256`, input `[1, 4, dim_f, 256]`
complex-as-channels `[L_re, L_im, R_re, R_im]` → output of the same shape = the predicted stem's
spectrogram. Real **fp16** I/O with fp16 compute, `mlprogram`, `minimum_deployment_target = iOS16`.

> ⚠️ **Inst HQ 3 inverts the residual.** It is architecturally identical to Voc FT but predicts the
> **instrumental**, so the free residual is `vocals = mix − model(mix)` — the opposite polarity from
> 9482 / Voc FT. A caller that assumes "output = vocals" will get the stems swapped.

## Download

```bash
# huggingface_hub (Python)
pip install huggingface_hub
python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download("gyoom-sa/UVR-MDX-CoreML", local_dir="./UVR-MDX-CoreML")
PY

# or the CLI
hf download gyoom-sa/UVR-MDX-CoreML --local-dir ./UVR-MDX-CoreML
```

The `.mlpackage` folders keep their on-disk structure in this repo, so `git clone` / `hf download`
yields ready-to-compile packages. Add them to your Xcode target (or call
`MLModel.compileModel(at:)` at runtime) and they compile to `.mlmodelc`.

## Usage (iOS / Swift)

The model operates on a **spectrogram**, not raw audio. Build the STFT in your app (Apple's
Accelerate / vDSP), feed the complex planes to the model, then iSTFT the output. Per model:
**9482** `n_fft 4096, hop 1024, dim_f 2048`; **Voc FT / Inst HQ 3** `n_fft 6144, hop 1024, dim_f 3072`.
Use a periodic Hann window, `center=True` reflect padding, **unnormalized** STFT, drop the Nyquist bin
(`dim_f = n_fft/2`), and pack planes in order `[L_re, L_im, R_re, R_im]` with flat row-major index
`((plane*dim_f)+bin)*dim_t + frame`.

```swift
import CoreML

// Accelerator is a LOAD-TIME choice. .all == ANE → GPU → CPU (the baked default).
let config = MLModelConfiguration()
config.computeUnits = .all               // .cpuAndGPU | .cpuOnly also available
let model = try MLModel(contentsOf: compiledURL, configuration: config)

// Per chunk. dim_f = 2048 (9482) or 3072 (Voc FT / Inst HQ 3); dim_t = 256. Real fp16 I/O.
let input = try MLMultiArray(shape: [1, 4, dimF as NSNumber, dimT as NSNumber],
                             dataType: .float16)
let p = input.dataPointer.bindMemory(to: Float16.self, capacity: input.count)
// pack the STFT: p[((plane*dimF)+bin)*dimT + frame] = value   (plane 0..3 = L_re,L_im,R_re,R_im)

let out = try model.prediction(from: MLDictionaryFeatureProvider(dictionary: ["input": input]))
let stem = out.featureValue(for: "output")!.multiArrayValue!     // fp16 [1,4,dim_f,256] → iSTFT
```

For the stem models, `vocals = model(mix)`; for **Inst HQ 3**, `instrumental = model(mix)` and
`vocals = mix − instrumental`. The equal-power "shift trick" denoise (`0.5·model(x) − 0.5·model(−x)`)
is optional.

## Notes & limitations

- **STFT is outside the graph** — these are the learned core only; the app must reproduce the DSP
  contract above, or the stems silently degrade or swap.
- **fp16 is quality-safe for MDX** — activations stay far below the fp16 ceiling, so the ~73 dB
  output floor is well above audible noise.
- **Residual polarity** differs per model (see the Inst HQ 3 warning).
- These are **fp16-quantized, on-device artifacts** of the UVR MDX-Net models ([base model:
  Politrees/UVR_resources](https://huggingface.co/Politrees/UVR_resources)), not models loadable in
  `transformers` or via HF Inference. `pipeline_tag: audio-to-audio` is set for task-based
  discoverability only.

## Reproducibility

The exact conversion pipeline (`export_mdx_coreml.py` + pinned `requirements.txt`) and the full
technical write-up (precision, compute units, verification SNR, iOS DSP contract) live in the
[`export/`](./export/README.md) folder of this repo. `example.mp3` is provided for a quick smoke test.

## License & attribution

The model weights are derived from the Ultimate Vocal Remover (UVR) MDX-Net models by **Anjok07**
(UVR-MDXNET 9482) and **Kimberley Jensen** (UVR-MDX-NET Voc FT / Inst HQ 3), distributed under the
MIT License via https://github.com/TRvlvr/model_repo and mirrored at
[Politrees/UVR_resources](https://huggingface.co/Politrees/UVR_resources) (this repo's
`base_model`). This repository redistributes fp16-quantized, format-converted (ONNX → CoreML fp16)
copies under the same MIT terms. See [`LICENSE`](./LICENSE).