---
license: apache-2.0
base_model: alibabasglab/MossFormer2_SS_16K
pipeline_tag: audio-to-audio
library_name: coreml
tags:
  - speech-separation
  - core-ml
  - macos
---

# MossFormer2 speech separation (16 kHz), Core ML

A Core ML conversion of [MossFormer2_SS_16K](https://huggingface.co/alibabasglab/MossFormer2_SS_16K), Alibaba's speech separation model from [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio). It splits a mono 16 kHz recording of two people talking at once into one track per voice.

## Inputs and outputs

| Name | Type | Shape |
|---|---|---|
| `audio` | Float32, 16 kHz mono | `[1, 32776]` (2.05 seconds) |
| `voices` | Float32 | `[1, 2, 32776]`, one row per voice |

Longer audio goes in windows of 32,776 samples that step 24,582 samples, keeping the middle of each window, as ClearerVoice-Studio does with its 2-second windows; then each voice is scaled back to the input's loudness. Which row holds which voice can change from one window to the next.

Run it on the CPU and GPU (`MLComputeUnits.cpuAndGPU`). The Neural Engine can't compile it, and in testing on macOS 26 one run crashed inside Core ML's CPU backend, while CPU and GPU ran without problems.

## Changes from the original

- Converted to a Core ML ML program with float16 weights, for macOS 15 and later.
- A fixed input of 32,776 samples instead of any length: that's 4,096 encoder steps, 16 whole attention groups of 256, so attention needs no padding mask.
- During conversion only, two operations are written differently with the same result: padding by a negative amount becomes a crop, and `einsum` becomes the equivalent matrix products. PyTorch's output doesn't change.

On the same input, the Core ML model matches the PyTorch model at about 65 dB SI-SDR.

[convert.py](convert.py) reproduces the conversion from the original weights.

## License and credits

Apache 2.0, the license of the original weights and code; see [LICENSE](LICENSE). MossFormer2 is by Shengkui Zhao and colleagues at Alibaba: *MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation*, ICASSP 2024. All credit for the model goes to them; this repository only converts it.