moonbox-dev/FRCRN_SE_16K-MLX

FRCRN MLX

MLX builds of FRCRN, made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's files together. Only the parts listed below are converted: the rest of the pipeline comes from the original repository unless the section below says otherwise.

Files

The last column is how far each build is from the original PyTorch model on the same inputs (see the section below for what was measured).

Precision	File	Size	sha256	Checked against the original
bf16	bf16/config.json	0.1 kB	86f0962100d6	below threshold (experimental): SI-SDR 9.66 dB, DNSMOS 3.06; output vs the original's 16.9 / 2.9 dB
bf16	bf16/mlx_model.py	12.3 kB	307afe87cb76	below threshold (experimental): SI-SDR 9.66 dB, DNSMOS 3.06; output vs the original's 16.9 / 2.9 dB
bf16	bf16/model.safetensors	29.7 MB	4f4e44804287	below threshold (experimental): SI-SDR 9.66 dB, DNSMOS 3.06; output vs the original's 16.9 / 2.9 dB
fp16	fp16/config.json	0.1 kB	86f0962100d6	below threshold (experimental): SI-SDR 10.31 dB mean but 0.6-1.2 dB lower on two clips, DNSMOS 3.16; output vs the original's 25.2 / 8.2 dB
fp16	fp16/mlx_model.py	12.3 kB	307afe87cb76	below threshold (experimental): SI-SDR 10.31 dB mean but 0.6-1.2 dB lower on two clips, DNSMOS 3.16; output vs the original's 25.2 / 8.2 dB
fp16	fp16/model.safetensors	29.7 MB	8b4402088aa8	below threshold (experimental): SI-SDR 10.31 dB mean but 0.6-1.2 dB lower on two clips, DNSMOS 3.16; output vs the original's 25.2 / 8.2 dB
fp32	fp32/config.json	0.1 kB	86f0962100d6	SI-SDR 10.30 dB (original 10.30), DNSMOS 3.19 (3.19); output vs the original's 89.3 dB mean / 79.2 dB worst clip
fp32	fp32/mlx_model.py	12.3 kB	307afe87cb76	SI-SDR 10.30 dB (original 10.30), DNSMOS 3.19 (3.19); output vs the original's 89.3 dB mean / 79.2 dB worst clip
fp32	fp32/model.safetensors	56.0 MB	a8338a1874a5	SI-SDR 10.30 dB (original 10.30), DNSMOS 3.19 (3.19); output vs the original's 89.3 dB mean / 79.2 dB worst clip
8bit	8bit/config.json	0.2 kB	18aa7e9b7dd2	below threshold (experimental): SI-SDR 9.88 dB, DNSMOS 3.07; output vs the original's 18.0 / 4.1 dB
8bit	8bit/mlx_model.py	12.3 kB	307afe87cb76	below threshold (experimental): SI-SDR 9.88 dB, DNSMOS 3.07; output vs the original's 18.0 / 4.1 dB
8bit	8bit/model.safetensors	27.7 MB	88af0c96d104	below threshold (experimental): SI-SDR 9.88 dB, DNSMOS 3.07; output vs the original's 18.0 / 4.1 dB
6bit	6bit/config.json	0.2 kB	d3a700755964	below threshold (experimental): SI-SDR 10.18 dB, DNSMOS 3.04; output vs the original's 13.8 / 3.0 dB
6bit	6bit/mlx_model.py	12.3 kB	307afe87cb76	below threshold (experimental): SI-SDR 10.18 dB, DNSMOS 3.04; output vs the original's 13.8 / 3.0 dB
6bit	6bit/model.safetensors	27.2 MB	7bccd20b2f7a	below threshold (experimental): SI-SDR 10.18 dB, DNSMOS 3.04; output vs the original's 13.8 / 3.0 dB
4bit	4bit/config.json	0.2 kB	c322f3218326	below threshold (experimental): SI-SDR 4.92 dB, DNSMOS 2.25; output vs the original's 4.7 / -1.3 dB
4bit	4bit/mlx_model.py	12.3 kB	307afe87cb76	below threshold (experimental): SI-SDR 4.92 dB, DNSMOS 2.25; output vs the original's 4.7 / -1.3 dB
4bit	4bit/model.safetensors	26.6 MB	b343da5b02df	below threshold (experimental): SI-SDR 4.92 dB, DNSMOS 2.25; output vs the original's 4.7 / -1.3 dB

How to run

FRCRN SE 16K (ClearerVoice-Studio's speech enhancement for 16 kHz speech: two complex-valued UNets with FSMN memory and squeeze-excitation layers on a 640-point STFT) ported to MLX. Each folder holds model.safetensors, config.json and mlx_model.py (the port). Needs mlx, numpy, soundfile (and librosa for input that isn't 16 kHz):

import os; os.environ["MLX_ENABLE_TF32"] = "0"   # float32 matmuls (MLX uses TF32 otherwise)
import sys; sys.path.insert(0, "fp32")           # a downloaded folder
import mlx_model, soundfile as sf

model = mlx_model.load("fp32")
clean = mlx_model.enhance(model, "noisy.wav")    # float32, 16 kHz, the input's length
sf.write("clean.wav", clean, 16000)

Like ClearerVoice, enhance normalizes the input's loudness first and scales the result back, and decodes inputs over 120 s in 1 s windows. Use fp32/. The network is unusually sensitive to its weights (its BatchNorm statistics reach 3e5): every folder computes in float32, but storing the weights in 16 bits (bf16/, fp16/) or quantizing the FSMN layers (8bit/, 6bit/, 4bit/, on bf16 storage) already moves the output far enough to miss the threshold, so those are marked experimental. BatchNorm parameters and the STFT kernels are float32 in every folder.

How they were checked. 8 noisy 16 kHz clips (LibriSpeech speech with babble or pink noise, 0-10 dB SNR), enhanced by the original (ClearerVoice's clearvoice 0.1.2, PyTorch float32 on the CPU) and by every MLX build: SI-SDR against the clean speech, DNSMOS P.835, and how close each output is to the original's. Pass: SI-SDR on every clip at most 0.5 dB below the original's, mean DNSMOS at most 0.1 below.

Precision	Size	SI-SDR (dB)	DNSMOS	Output vs the original's, mean / worst clip (dB)
noisy input	-	4.98	2.04	-
original (PyTorch fp32, CPU)	-	10.30	3.19	-
fp32/	56 MB	10.30	3.19	89.3 / 79.2
fp16/ (experimental)	30 MB	10.31 (1.2 lower on one clip)	3.16	25.2 / 8.2
bf16/ (experimental)	30 MB	9.66	3.06	16.9 / 2.9
8bit/ (experimental)	28 MB	9.88	3.07	18.0 / 4.1
6bit/ (experimental)	27 MB	10.18	3.04	13.8 / 3.0
4bit/ (experimental)	27 MB	4.92	2.25	4.7 / -1.3

About 10x faster than real time on an Apple M5 (the original on the CPU: 0.6x).

How these were made

From FRCRN at revision 3766e6a64b: the original weights loaded in MLX and stored as bf16 or fp16, or quantized with MLX's affine quantization (8bit, 6bit, 4bit, group size 64; layers MLX can't quantize, such as convolutions and norms, stay bf16). Every build was checked against the original before upload; the sha256 column above is what the files must hash to.

License

Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged.

---

Original model card

The FRCRN_SE_16K model weights for 16 kHz speech enhancement in ClearerVoice-Studio repo.

This model is trained on large scale datasets inclduing open-sourced and private data.

It enhances speech audios by removing background noise.