---
license: mit
tags:
  - mlx
  - audio
  - pitch-estimation
  - f0
  - voice-conversion
  - rvc
  - apple-silicon
library_name: mlx
pipeline_tag: audio-to-audio
---

# MLX-RMVPE

MLX implementation of [RMVPE](https://arxiv.org/abs/2306.15412) (Robust Model for Vocal Pitch Estimation) for Apple Silicon.

## Model Description

RMVPE extracts **fundamental frequency (F0)** from audio, essential for preserving pitch/melody in voice conversion. Unlike simpler methods (CREPE, pYIN), RMVPE is specifically designed for **polyphonic music**, making it ideal for singing voice conversion where background music may be present.

- **Architecture**: Deep U-Net with BiGRU layers
- **Parameters**: ~15.4M
- **Input**: 16kHz audio
- **Output**: F0 in Hz at 100fps (hop_length=160)
- **Pitch range**: ~32 Hz to ~1975 Hz (360 bins)

## Usage

```bash
pip install mlx-rmvpe
```

```python
import librosa
from mlx_rmvpe import RMVPE

# Load model (auto-downloads weights)
model = RMVPE.from_pretrained()

# Load audio at 16kHz
audio, sr = librosa.load("singing.wav", sr=16000, mono=True)

# Extract F0
f0 = model.infer_from_audio(audio)

print(f"F0 shape: {f0.shape} at 100fps")
print(f"Pitch range: {f0[f0 > 0].min():.1f} - {f0[f0 > 0].max():.1f} Hz")
```

## Manual Loading

```python
from huggingface_hub import hf_hub_download
from mlx_rmvpe import RMVPE

weights_path = hf_hub_download(
    repo_id="lexandstuff/mlx-rmvpe",
    filename="rmvpe.safetensors"
)

model = RMVPE()
model.load_weights(weights_path)
model.eval()
```

## Technical Details

This implementation is converted from the PyTorch weights and produces numerically similar outputs:

| Metric | Value |
|--------|-------|
| Mean F0 difference | 1.29 Hz |
| Correlation | >0.99 |

See the [GitHub repository](https://github.com/lexandstuff/mlx-rmvpe) for implementation details and the full API reference.

## Citation

```bibtex
@inproceedings{wei2023rmvpe,
  title={RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Music},
  author={Wei, Yongmao and others},
  booktitle={ISMIR},
  year={2023}
}
```

## License

MIT

## Acknowledgments

- [RMVPE](https://github.com/Dream-High/RMVPE) - Original implementation
- [RVC](https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI) - Voice conversion pipeline
- [MLX](https://github.com/ml-explore/mlx) - Apple's machine learning framework