--- base_model: alibabasglab/FRCRN_SE_16K base_model_relation: quantized license: apache-2.0 pipeline_tag: audio-to-audio library_name: mlx tags: - mlx - moonbox - mlx - frcrn - clearervoice - speech-enhancement - denoising - 16khz --- # FRCRN MLX MLX builds of [FRCRN](https://huggingface.co/alibabasglab/FRCRN_SE_16K), made by Moonbox for Apple Silicon Macs. Each precision is a folder; keep a folder's files together. Only the parts listed below are converted: the rest of the pipeline comes from the original repository unless the section below says otherwise. ## Files The last column is how far each build is from the original PyTorch model on the same inputs (see the section below for what was measured). | Precision | File | Size | sha256 | Checked against the original | |---|---|---|---|---| | bf16 | [`bf16/config.json`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/bf16/config.json) | 0.1 kB | `86f0962100d6` | below threshold (experimental): SI-SDR 9.66 dB, DNSMOS 3.06; output vs the original's 16.9 / 2.9 dB | | bf16 | [`bf16/mlx_model.py`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/bf16/mlx_model.py) | 12.3 kB | `307afe87cb76` | below threshold (experimental): SI-SDR 9.66 dB, DNSMOS 3.06; output vs the original's 16.9 / 2.9 dB | | bf16 | [`bf16/model.safetensors`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/bf16/model.safetensors) | 29.7 MB | `4f4e44804287` | below threshold (experimental): SI-SDR 9.66 dB, DNSMOS 3.06; output vs the original's 16.9 / 2.9 dB | | fp16 | [`fp16/config.json`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/fp16/config.json) | 0.1 kB | `86f0962100d6` | below threshold (experimental): SI-SDR 10.31 dB mean but 0.6-1.2 dB lower on two clips, DNSMOS 3.16; output vs the original's 25.2 / 8.2 dB | | fp16 | [`fp16/mlx_model.py`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/fp16/mlx_model.py) | 12.3 kB | `307afe87cb76` | below threshold (experimental): SI-SDR 10.31 dB mean but 0.6-1.2 dB lower on two clips, DNSMOS 3.16; output vs the original's 25.2 / 8.2 dB | | fp16 | [`fp16/model.safetensors`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/fp16/model.safetensors) | 29.7 MB | `8b4402088aa8` | below threshold (experimental): SI-SDR 10.31 dB mean but 0.6-1.2 dB lower on two clips, DNSMOS 3.16; output vs the original's 25.2 / 8.2 dB | | fp32 | [`fp32/config.json`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/fp32/config.json) | 0.1 kB | `86f0962100d6` | SI-SDR 10.30 dB (original 10.30), DNSMOS 3.19 (3.19); output vs the original's 89.3 dB mean / 79.2 dB worst clip | | fp32 | [`fp32/mlx_model.py`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/fp32/mlx_model.py) | 12.3 kB | `307afe87cb76` | SI-SDR 10.30 dB (original 10.30), DNSMOS 3.19 (3.19); output vs the original's 89.3 dB mean / 79.2 dB worst clip | | fp32 | [`fp32/model.safetensors`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/fp32/model.safetensors) | 56.0 MB | `a8338a1874a5` | SI-SDR 10.30 dB (original 10.30), DNSMOS 3.19 (3.19); output vs the original's 89.3 dB mean / 79.2 dB worst clip | | 8bit | [`8bit/config.json`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/8bit/config.json) | 0.2 kB | `18aa7e9b7dd2` | below threshold (experimental): SI-SDR 9.88 dB, DNSMOS 3.07; output vs the original's 18.0 / 4.1 dB | | 8bit | [`8bit/mlx_model.py`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/8bit/mlx_model.py) | 12.3 kB | `307afe87cb76` | below threshold (experimental): SI-SDR 9.88 dB, DNSMOS 3.07; output vs the original's 18.0 / 4.1 dB | | 8bit | [`8bit/model.safetensors`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/8bit/model.safetensors) | 27.7 MB | `88af0c96d104` | below threshold (experimental): SI-SDR 9.88 dB, DNSMOS 3.07; output vs the original's 18.0 / 4.1 dB | | 6bit | [`6bit/config.json`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/6bit/config.json) | 0.2 kB | `d3a700755964` | below threshold (experimental): SI-SDR 10.18 dB, DNSMOS 3.04; output vs the original's 13.8 / 3.0 dB | | 6bit | [`6bit/mlx_model.py`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/6bit/mlx_model.py) | 12.3 kB | `307afe87cb76` | below threshold (experimental): SI-SDR 10.18 dB, DNSMOS 3.04; output vs the original's 13.8 / 3.0 dB | | 6bit | [`6bit/model.safetensors`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/6bit/model.safetensors) | 27.2 MB | `7bccd20b2f7a` | below threshold (experimental): SI-SDR 10.18 dB, DNSMOS 3.04; output vs the original's 13.8 / 3.0 dB | | 4bit | [`4bit/config.json`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/4bit/config.json) | 0.2 kB | `c322f3218326` | below threshold (experimental): SI-SDR 4.92 dB, DNSMOS 2.25; output vs the original's 4.7 / -1.3 dB | | 4bit | [`4bit/mlx_model.py`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/4bit/mlx_model.py) | 12.3 kB | `307afe87cb76` | below threshold (experimental): SI-SDR 4.92 dB, DNSMOS 2.25; output vs the original's 4.7 / -1.3 dB | | 4bit | [`4bit/model.safetensors`](https://huggingface.co/moonbox-dev/FRCRN_SE_16K-MLX/blob/main/4bit/model.safetensors) | 26.6 MB | `b343da5b02df` | below threshold (experimental): SI-SDR 4.92 dB, DNSMOS 2.25; output vs the original's 4.7 / -1.3 dB | ## How to run FRCRN SE 16K (ClearerVoice-Studio's speech enhancement for 16 kHz speech: two complex-valued UNets with FSMN memory and squeeze-excitation layers on a 640-point STFT) ported to MLX. Each folder holds `model.safetensors`, `config.json` and `mlx_model.py` (the port). Needs `mlx`, `numpy`, `soundfile` (and `librosa` for input that isn't 16 kHz): ```python import os; os.environ["MLX_ENABLE_TF32"] = "0" # float32 matmuls (MLX uses TF32 otherwise) import sys; sys.path.insert(0, "fp32") # a downloaded folder import mlx_model, soundfile as sf model = mlx_model.load("fp32") clean = mlx_model.enhance(model, "noisy.wav") # float32, 16 kHz, the input's length sf.write("clean.wav", clean, 16000) ``` Like ClearerVoice, `enhance` normalizes the input's loudness first and scales the result back, and decodes inputs over 120 s in 1 s windows. **Use `fp32/`.** The network is unusually sensitive to its weights (its BatchNorm statistics reach 3e5): every folder computes in float32, but storing the weights in 16 bits (`bf16/`, `fp16/`) or quantizing the FSMN layers (`8bit/`, `6bit/`, `4bit/`, on bf16 storage) already moves the output far enough to miss the threshold, so those are marked experimental. BatchNorm parameters and the STFT kernels are float32 in every folder. **How they were checked.** 8 noisy 16 kHz clips (LibriSpeech speech with babble or pink noise, 0-10 dB SNR), enhanced by the original (ClearerVoice's `clearvoice` 0.1.2, PyTorch float32 on the CPU) and by every MLX build: SI-SDR against the clean speech, DNSMOS P.835, and how close each output is to the original's. Pass: SI-SDR on every clip at most 0.5 dB below the original's, mean DNSMOS at most 0.1 below. | Precision | Size | SI-SDR (dB) | DNSMOS | Output vs the original's, mean / worst clip (dB) | |---|---|---|---|---| | noisy input | - | 4.98 | 2.04 | - | | original (PyTorch fp32, CPU) | - | 10.30 | 3.19 | - | | `fp32/` | 56 MB | 10.30 | 3.19 | 89.3 / 79.2 | | `fp16/` (experimental) | 30 MB | 10.31 (1.2 lower on one clip) | 3.16 | 25.2 / 8.2 | | `bf16/` (experimental) | 30 MB | 9.66 | 3.06 | 16.9 / 2.9 | | `8bit/` (experimental) | 28 MB | 9.88 | 3.07 | 18.0 / 4.1 | | `6bit/` (experimental) | 27 MB | 10.18 | 3.04 | 13.8 / 3.0 | | `4bit/` (experimental) | 27 MB | 4.92 | 2.25 | 4.7 / -1.3 | About 10x faster than real time on an Apple M5 (the original on the CPU: 0.6x). ## How these were made From [FRCRN](https://huggingface.co/alibabasglab/FRCRN_SE_16K) at revision `3766e6a64b`: the original weights loaded in MLX and stored as bf16 or fp16, or quantized with MLX's affine quantization (`8bit`, `6bit`, `4bit`, group size 64; layers MLX can't quantize, such as convolutions and norms, stay bf16). Every build was checked against the original before upload; the `sha256` column above is what the files must hash to. ## License Apache-2.0. These files are a derivative of the original model and carry its license: it is copied in this repository, and it governs your use of these files. The original model card follows unchanged. --- # Original model card The FRCRN_SE_16K model weights for 16 kHz speech enhancement in [ClearerVoice-Studio](https://github.com/modelscope/ClearerVoice-Studio/tree/main) repo. This model is trained on large scale datasets inclduing open-sourced and private data. It enhances speech audios by removing background noise.