---
license: mit
language:
- multilingual
tags:
- speech-restoration
- dereverberation
- speech-enhancement
- voice-cloning
- coreml
- on-device
base_model: sarulab-speech/sidon-v0.1
library_name: coreml
pipeline_tag: audio-to-audio
---

# Sidon — Core ML (speech restoration / dereverberation)

On-device **speech restoration** (denoise + dereverberation + bandwidth restoration)
for Apple Silicon, exported to Core ML (runs on the Neural Engine). Turns a
noisy/reverberant clip into studio-quality **48 kHz** speech — ideal for cleaning a
**voice-cloning reference** before TTS, since it preserves speaker identity.

Two-stage pipeline:

```
16 kHz audio → [w2v-BERT log-mel front-end] → predictor (w2v-BERT 2.0, 8 layers)
            → cleansed features [1, T, 1024] → DAC decoder → 48 kHz audio
```

## Variants

| variant | predictor | vocoder | bundle size | peak RAM | RTF (ANE) |
| ------- | --------- | ------- | ----------- | -------- | --------- |
| **fp16** | FP16 | FP16 | 713 MB | 1711 MB | ~120× |
| **int8** | INT8 (k-means palettized) | FP16 | **407 MB** | **1321 MB** | ~110× |

Total 246 M params (predictor 193.6 M + DAC vocoder 52.4 M). Output sample rate 48 kHz.
int8 keeps the vocoder at FP16 (audio quality); only the predictor is palettized.

## Files

| path | description |
| ---- | ----------- |
| `fp16/Sidon-Predictor.mlpackage` | w2v-BERT 2.0 (8L) + merged LoRA → features (FP16) |
| `fp16/Sidon-Vocoder.mlpackage` | DAC decoder → 48 kHz audio (FP16) |
| `int8/Sidon-Predictor.mlpackage` | predictor, 8-bit palettized |
| `int8/Sidon-Vocoder.mlpackage` | DAC decoder (FP16) |

## Quality (no-reference MOS, 10 s clip)

DNSMOS P.835 (SIG/BAK/OVRL, higher = better) and UTMOS (naturalness, 1–5):

| audio | SIG | BAK | OVRL | UTMOS | speaker cos |
| ----- | --- | --- | ---- | ----- | ----------- |
| input (reverberant) | 3.46 | 3.40 | 2.90 | 2.99 | — |
| **fp16** | 3.53 | 4.09 | 3.28 | 3.32 | 0.797 |
| **int8** | 3.54 | 4.11 | 3.29 | 3.23 | 0.796 |

Restoration lifts OVRL 2.90 → 3.29 (driven by **BAK 3.40 → 4.11** — reverb removed).
Quantization is near-lossless on DNSMOS and speaker similarity; UTMOS shows a small
naturalness cost (fp16 −0.09, int8 −0.17). Numbers are a single clip — average over a
set for a definitive figure.

## Front-end

The graphs take `input_features [1, T, 160]` from the **w2v-BERT 2.0 SeamlessM4T
feature extractor** (16 kHz input). The sequence length is fixed (T = 499 ≈ 10 s) —
chunk longer audio in the runtime. The front-end and chunking are handled by
[speech-swift](https://github.com/soniqo/speech-swift).

## Usage

Use via the [speech-swift](https://github.com/soniqo/speech-swift) Apple SDK, e.g.:

```bash
speech enhance noisy-reference.wav -o clean.wav   # restore / dereverb on-device
```

```swift
// See speech-swift for the full API (loads the predictor + vocoder, runs the
// log-mel front-end, chunks, and writes 48 kHz audio).
```

## Source

Exported from [Sidon](https://github.com/sarulab-speech/Sidon) (sarulab-speech),
checkpoint [sidon-v0.1](https://huggingface.co/sarulab-speech/sidon-v0.1); paper
[arXiv:2509.17052](https://arxiv.org/abs/2509.17052). Base SSL encoder:
[facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0); vocoder: DAC
(descript-audio-codec). All components are MIT-licensed.

## Links

- [speech-swift](https://github.com/soniqo/speech-swift) — Apple (Swift) SDK
- [soniqo.audio](https://soniqo.audio) — website
- [blog](https://soniqo.audio/blog)